0

I'm using treetaggerwrapper to stem Italian texts and I'm interested in the final lemmas. As you can see, while other parts of speech are transformed correctly, all numbers are changed to @card@. Is it possible to leave them unchanged?

import treetaggerwrapper
from pprint import pprint


tagger=treetaggerwrapper.TreeTagger(TAGDIR='/home/myname/treetag/',TAGLANG='it')
tags_str = tagger.tag_text("Posso riconoscere i numeri come 3,4  99.9?")
tags = treetaggerwrapper.make_tags(tags_str)
pprint(tags)

[Tag(word='Posso', pos='VER2:fin', lemma='potere'),
Tag(word='riconoscere', pos='VER:infi', lemma='riconoscere'),
Tag(word='i', pos='ART', lemma='il'),
Tag(word='numeri', pos='NOUN', lemma='numero'),
Tag(word='come', pos='WH', lemma='come'),
Tag(word='3', pos='NUM', lemma='@card@'),
Tag(word=',', pos='PUN', lemma=','),
Tag(word='4', pos='NUM', lemma='@card@'),
Tag(word='99.9', pos='NUM', lemma='@card@'),
Tag(word='?', pos='SENT', lemma='?')]

2
  • Any reason not to just run through the list and set lemma=word if lemma=='@card@'? That seems like the quickest and easiest fix.
    – Draconis
    Oct 15, 2021 at 19:36
  • Thank you @Draconis. ` for x in tags: if x.lemma=='@card@': x.lemma=x.word ` gave me the error "cannot set attribute". By reading on link it turns out Tag is a namedtuple thus immutable. So I had to modify treetaggerwrapper.py by changing the type of Tag, TagExtra and NotTag to recordtype. Oct 15, 2021 at 20:41

1 Answer 1

2

The number is preserved in the word attribute, so it's easy enough to swap it back in. But as mentioned in the comments, Tag is an immutable type. So I propose a fairly straightforward list comprehension/filter:

def fix_numbers(tag):
    if tag.lemma == '@card@': return Tag(tag.word, tag.pos, tag.word)
    return tag

fixed_list = [fix_numbers(tag) for tag in original_list]

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.