Ercong Nie, Shuzhou Yuan, Bolei Ma, Helmut Schmid, Michael Färber, Frauke Kreuter, Hinrich Schütze『Decomposed Prompting: Unveiling Multilingual Linguistic Structure Knowledge in English-Centric Large Language Models』を横目に、Mistral-7B-v0.2-hfを使って、英語Universal Dependenciesの係り受けリンクでDecomposed Promptingを試してみた。Google Colaboratory (GPU版)だと、こんな感じ。
!pip install transformers accelerate sentencepiece
ex="""
###text: And if you send me a story, that would be great to!
In Universal Dependencies of the text, ‘if’ is linked from ‘send’ with the label of mark.
###text: I'll admit I wasn't expecting much from this place, but they really did do a good job.
In Universal Dependencies of the text, ‘good’ is linked from ‘job’ with the label of amod.
###text: "I don't know." The girl shrugged once again.
In Universal Dependencies of the text, ‘girl’ is linked from ‘shrugged’ with the label of nsubj.
###text: The dancers were falling back around a Polish agriculturalist who was teaching a gangling Englishman and two young Africans an Eastern European peasant dance.
In Universal Dependencies of the text, ‘around’ is linked from ‘agriculturalist’ with the label of case.
###text: Antigua was awesome.
In Universal Dependencies of the text, ‘was’ is linked from ‘awesome’ with the label of cop.
###text: The food is fresh and taste great.
In Universal Dependencies of the text, ‘The’ is linked from ‘food’ with the label of det.
###text: Now I have wife and son.
In Universal Dependencies of the text, ‘Now’ is linked from ‘have’ with the label of advmod.
###text: However, this fruitful period was short-lived, as Greece suffered badly under the Ottoman Empire, only to recover in the 19th century as the capital of independent Greece.
In Universal Dependencies of the text, ‘suffered’ is linked from ‘lived’ with the label of advcl.
###text: I survived it without a problem.
In Universal Dependencies of the text, ‘.’ is linked from ‘survived’ with the label of punct.
###text: The food is fresh and taste great.
In Universal Dependencies of the text, ‘and’ is linked from ‘taste’ with the label of cc.
###text: you can view at dresscod.com
In Universal Dependencies of the text, ‘dresscod.com’ is linked from ‘view’ with the label of obl.
###text: "I don't know." The girl shrugged once again.
In Universal Dependencies of the text, ‘shrugged’ is linked from ‘shrugged’ with the label of root.
###text: I'll admit I wasn't expecting much from this place, but they really did do a good job.
In Universal Dependencies of the text, ‘n't’ is linked from ‘expecting’ with the label of advmod.
###text: Antigua was awesome.
In Universal Dependencies of the text, ‘Antigua’ is linked from ‘awesome’ with the label of nsubj.
###text: The dancers were falling back around a Polish agriculturalist who was teaching a gangling Englishman and two young Africans an Eastern European peasant dance.
In Universal Dependencies of the text, ‘two’ is linked from ‘Africans’ with the label of nummod.
###text: Yes, the Cyclone is almost certain to lose strength as it surges over land.
In Universal Dependencies of the text, ‘Yes’ is linked from ‘certain’ with the label of discourse.
###text: —-== Posted via Newsfeed.Com - Unlimited - Uncensored - Secure Usenet News ==—-
In Universal Dependencies of the text, ‘—-==’ is linked from ‘Posted’ with the label of punct.
"""
from transformers import AutoTokenizer,AutoModelForCausalLM,TextGenerationPipeline
tkz=AutoTokenizer.from_pretrained("alpindale/Mistral-7B-v0.2-hf")
mdl=AutoModelForCausalLM.from_pretrained("alpindale/Mistral-7B-v0.2-hf",torch_dtype="auto",device_map="auto")
tgn=TextGenerationPipeline(model=mdl,tokenizer=tkz,max_new_tokens=128)
nlp=lambda x,y=None:"\n".join(tgn(ex+f"###text: {x}\nIn Universal Dependencies of the text"+(f", ‘{y}’" if y else ","))[0]["generated_text"][len(ex):].split("\n")[0:2])
print(nlp("I have a pen.","I"))
print(nlp("I have a pen.","have"))
print(nlp("I have a pen.","a"))
print(nlp("I have a pen.","pen"))
print(nlp("I have a pen.","."))
「I have a pen.」の係り受けリンクを生成してみたところ、私(安岡孝一)の手元では以下の結果が出力された。
###text: I have a pen.
In Universal Dependencies of the text, ‘I’ is linked from ‘have’ with the label of nsubj.
###text: I have a pen.
In Universal Dependencies of the text, ‘have’ is linked from ‘pen’ with the label of aux.
###text: I have a pen.
In Universal Dependencies of the text, ‘a’ is linked from ‘pen’ with the label of det.
###text: I have a pen.
In Universal Dependencies of the text, ‘pen’ is linked from ‘have’ with the label of dobj.
###text: I have a pen.
In Universal Dependencies of the text, ‘.’ is linked from ‘pen’ with the label of punct.
うーん、残念ながら、あまり良くない。特に、objを勝手にdobjと出力している点が、かなり良くない。たとえもっと鍛えるとしても、そもそもあまりスジが良くなさそうだなぁ。