実行環境
・ 計算機： MacBookPro
・ OS: macOS Catalina
・ Python: Puython3.6.3
実行したコード
・ （GitHub） facebookresearch/ParlAI
（ 以下も参考にしました ）
・ pypi.org parlai 0.10.0
・ ParlAI – オープンに利用可能なさまざまなダイアログデータセットでAIモデルをトレーニングおよび評価するためのフレームワーク
・ 2017-10-19 「対話モデルの訓練/評価フレームワーク ParlAI がすごい」
git cloneでインストール
Terminal
ocean@AfoGuardMacBook-Pro Desktop % git clone https://github.com/facebookresearch/ParlAI.git ~/ParlAI
Cloning into '/Users/ocean/ParlAI'...
remote: Enumerating objects: 174, done.
remote: Counting objects: 100% (174/174), done.
remote: Compressing objects: 100% (123/123), done.
remote: Total 37068 (delta 105), reused 78 (delta 51), pack-reused 36894
Receiving objects: 100% (37068/37068), 64.73 MiB | 8.53 MiB/s, done.
Resolving deltas: 100% (26171/26171), done.
ocean@AfoGuardMacBook-Pro Desktop %
ocean@AfoGuardMacBook-Pro Desktop % cd ~/ParlAI; python setup.py develop
running develop
running egg_info
creating parlai.egg-info
writing parlai.egg-info/PKG-INFO
writing dependency_links to parlai.egg-info/dependency_links.txt
（ ・・・省略・・・ ）
Installed /Users/ocean/.pyenv/versions/3.9.0/lib/python3.9/site-packages/jsonlines-1.2.0-py3.9.egg
Searching for websocket-server
Reading https://pypi.org/simple/websocket-server/
Downloading https://files.pythonhosted.org/packages/74/64/e86581ee7775a2e08aca530b41e1a1e3ee6b320233b1eff301dcb86d1636/websocket_server-0.4.tar.gz#sha256=91cd4b565d1e1b00ef107abcb2840a8090868b19543f3b38e1962d5f975d0c04
Best match: websocket-server 0.4
Processing websocket_server-0.4.tar.gz
（ ・・・省略・・・ ）
Using /Users/ocean/.pyenv/versions/3.9.0/lib/python3.9/site-packages
Finished processing dependencies for parlai==0.10.0
ocean@AfoGuardMacBook-Pro ParlAI %
ディレクトリ構成
Terminal
ocean@AfoGuardMacBook-Pro ParlAI % ls
CODE_OF_CONDUCT.md MANIFEST.in autoformat.sh docs parlai pyproject.toml setup.py
CONTRIBUTING.md NEWS.md codecov.yml example_parlai_internal parlai.egg-info pytest.ini tests
LICENSE README.md conftest.py mypy.ini projects requirements.txt website
ocean@AfoGuardMacBook-Pro ParlAI %
pip install parlai
Terminal
ocean@AfoGuardMacBook-Pro ParlAI % pip install parlai
Requirement already satisfied: parlai in /Users/ocean/ParlAI (0.10.0)
Requirement already satisfied: boto3 in /Users/ocean/.pyenv/versions/3.9.0/lib/python3.9/site-packages/boto3-1.16.39-py3.9.egg (from parlai) (1.16.39)
（ ・・・省略・・・ ）
ocean@AfoGuardMacBook-Pro ParlAI %
ディレクトリ構成
Terminal
ocean@AfoGuardMacBook-Pro ParlAI % ls
CODE_OF_CONDUCT.md MANIFEST.in autoformat.sh docs parlai pyproject.toml setup.py
CONTRIBUTING.md NEWS.md codecov.yml example_parlai_internal parlai.egg-info pytest.ini tests
LICENSE README.md conftest.py mypy.ini projects requirements.txt website
ocean@AfoGuardMacBook-Pro ParlAI %
ocean@AfoGuardMacBook-Pro ParlAI % ls parlai
README.md __main__.py chat_service crowdsourcing nn tasks zoo
__init__.py agents core mturk scripts utils
ocean@AfoGuardMacBook-Pro ParlAI %
ocean@AfoGuardMacBook-Pro ParlAI % ls parlai/scripts
__init__.py data_stats.py eval_model.py multiprocessing_train.py token_stats.py
build_candidates.py detect_offensive_language.py eval_wordstat.py party.py train_model.py
build_dict.py display_data.py extract_image_feature.py profile_interactive.py vacuum.py
compare_opts.py display_model.py interactive.py profile_train.py verify_data.py
convert_data_to_parlai_format.py distributed_eval.py interactive_web.py safe_interactive.py
convo_render.py distributed_train.py multiprocessing_eval.py self_chat.py
ocean@AfoGuardMacBook-Pro ParlAI %
動かしてみる
以下は失敗
Terminal
ocean@AfoGuardMacBook-Pro ParlAI % parlai parlai/scripts/display_data -t squad
usage: parlai [-h] [--helpall] [--version] COMMAND ...
_
/")
//)
==//'=== ParlAI
/
optional arguments:
-h, --help show this help message and exit
--helpall List all commands, including advanced ones.
--version Prints version info and exit.
Commands:
display_data (dd) Display data from a task
display_model (dm) Display model predictions.
eval_model (em, eval) Evaluate a model
train_model (tm, train) Train a model
interactive (i) Interactive chat with a model on the command line
safe_interactive Like interactive, but adds a safety filter
self_chat Generate self-chats of a model
Parse Error: argument COMMAND: invalid choice: 'parlai/scripts/display_data' (choose from 'help', 'h', 'helpall', 'build_candidates', 'build_dict', 'convert_to_parlai', 'convo_render', 'data_stats', 'detect_offensive', 'display_data', 'dd', 'display_model', 'dm', 'eval_model', 'em', 'eval', 'train_model', 'tm', 'train', 'eval_wordstat', 'extract_image_feature', 'interactive', 'i', 'interactive_web', 'iweb', 'multiprocessing_eval', 'mp_eval', 'multiprocessing_train', 'mp_train', 'party', 'parrot', 'profile_interactive', 'profile_train', 'safe_interactive', 'self_chat', 'token_stats', 'vacuum', 'verify_data')
ocean@AfoGuardMacBook-Pro ParlAI %
コマンドの引数として呼び出せるのは、以下のうちいずれかである、とのこと。
- 'help'
- 'h'
- 'helpall'
- 'build_candidates'
- 'build_dict'
- 'convert_to_parlai'
- 'convo_render'
- 'data_stats'
- 'detect_offensive'
- 'display_data'
- 'dd'
- 'display_model'
- 'dm'
- 'eval_model'
- 'em'
- 'eval'
- 'train_model'
- 'tm'
- 'train'
- 'eval_wordstat'
- 'extract_image_feature'
- 'interactive'
- 'i'
- 'interactive_web'
- 'iweb'
- 'multiprocessing_eval'
- 'mp_eval'
- 'multiprocessing_train'
- 'mp_train'
- 'party'
- 'parrot'
- 'profile_interactive'
- 'profile_train'
- 'safe_interactive'
- 'self_chat'
- 'token_stats'
- 'vacuum'
- 'verify_data'
以下も失敗
Terminal
ocean@AfoGuardMacBook-Pro ParlAI % parlai parlai/scripts/display_data
usage: parlai [-h] [--helpall] [--version] COMMAND ...
_
/")
//)
==//'=== ParlAI
/
optional arguments:
-h, --help show this help message and exit
--helpall List all commands, including advanced ones.
--version Prints version info and exit.
Commands:
display_data (dd) Display data from a task
display_model (dm) Display model predictions.
eval_model (em, eval) Evaluate a model
train_model (tm, train) Train a model
interactive (i) Interactive chat with a model on the command line
safe_interactive Like interactive, but adds a safety filter
self_chat Generate self-chats of a model
Parse Error: argument COMMAND: invalid choice: 'parlai/scripts/display_data' (choose from 'help', 'h', 'helpall', 'build_candidates', 'build_dict', 'convert_to_parlai', 'convo_render', 'data_stats', 'detect_offensive', 'display_data', 'dd', 'display_model', 'dm', 'eval_model', 'em', 'eval', 'train_model', 'tm', 'train', 'eval_wordstat', 'extract_image_feature', 'interactive', 'i', 'interactive_web', 'iweb', 'multiprocessing_eval', 'mp_eval', 'multiprocessing_train', 'mp_train', 'party', 'parrot', 'profile_interactive', 'profile_train', 'safe_interactive', 'self_chat', 'token_stats', 'vacuum', 'verify_data')
ocean@AfoGuardMacBook-Pro ParlAI %
以下でスクリプトを実行できた。ただし、taskを渡し忘れた。
Terminal
ocean@AfoGuardMacBook-Pro ParlAI % parlai display_data
21:22:17 | Opt:
21:22:17 | allow_missing_init_opts: False
21:22:17 | batchsize: 1
21:22:17 | datapath: /Users/ocean/ParlAI/data
21:22:17 | datatype: train:ordered
21:22:17 | dict_class: None
21:22:17 | display_add_fields:
21:22:17 | download_path: None
21:22:17 | dynamic_batching: None
21:22:17 | hide_labels: False
21:22:17 | ignore_agent_reply: True
21:22:17 | image_cropsize: 224
21:22:17 | image_mode: raw
21:22:17 | image_size: 256
21:22:17 | init_model: None
21:22:17 | init_opt: None
21:22:17 | loglevel: info
21:22:17 | max_display_len: 1000
21:22:17 | model: None
21:22:17 | model_file: None
21:22:17 | multitask_weights: [1]
21:22:17 | num_examples: 10
21:22:17 | override: {}
21:22:17 | parlai_home: /Users/ocean/ParlAI
21:22:17 | starttime: Dec18_21-22
21:22:17 | task: None
21:22:17 | verbose: False
21:22:17 | Current ParlAI commit: 99160674564847c8ed68bc21437eab8c9301e95d
Traceback (most recent call last):
File "/Users/ocean/.pyenv/versions/3.9.0/bin/parlai", line 33, in <module>
sys.exit(load_entry_point('parlai', 'console_scripts', 'parlai')())
File "/Users/ocean/ParlAI/parlai/__main__.py", line 14, in main
superscript_main()
File "/Users/ocean/ParlAI/parlai/core/script.py", line 307, in superscript_main
return SCRIPT_REGISTRY[cmd].klass._run_from_parser_and_opt(opt, parser)
File "/Users/ocean/ParlAI/parlai/core/script.py", line 90, in _run_from_parser_and_opt
return script.run()
File "/Users/ocean/ParlAI/parlai/scripts/display_data.py", line 108, in run
return display_data(self.opt)
File "/Users/ocean/ParlAI/parlai/scripts/display_data.py", line 70, in display_data
world = create_task(opt, agent)
File "/Users/ocean/ParlAI/parlai/core/worlds.py", line 1249, in create_task
raise RuntimeError(
RuntimeError: No task specified. Please select a task with --task {task_name}.
ocean@AfoGuardMacBook-Pro ParlAI %
以下でスクリプトを実行できた。
Terminal
ocean@AfoGuardMacBook-Pro ParlAI % parlai display_data -t squad
21:22:34 | Opt:
21:22:34 | allow_missing_init_opts: False
21:22:34 | batchsize: 1
21:22:34 | datapath: /Users/ocean/ParlAI/data
21:22:34 | datatype: train:ordered
21:22:34 | dict_class: None
21:22:34 | display_add_fields:
21:22:34 | download_path: None
21:22:34 | dynamic_batching: None
21:22:34 | hide_labels: False
21:22:34 | ignore_agent_reply: True
21:22:34 | image_cropsize: 224
21:22:34 | image_mode: raw
21:22:34 | image_size: 256
21:22:34 | init_model: None
21:22:34 | init_opt: None
21:22:34 | loglevel: info
21:22:34 | max_display_len: 1000
21:22:34 | model: None
21:22:34 | model_file: None
21:22:34 | multitask_weights: [1]
21:22:34 | num_examples: 10
21:22:34 | override: "{'task': 'squad'}"
21:22:34 | parlai_home: /Users/ocean/ParlAI
21:22:34 | starttime: Dec18_21-22
21:22:34 | task: squad
21:22:34 | verbose: False
21:22:34 | Current ParlAI commit: 99160674564847c8ed68bc21437eab8c9301e95d
21:22:34 | creating task(s): squad
[building data: /Users/ocean/ParlAI/data/SQuAD]
21:22:34 | Downloading https://rajpurkar.github.io/SQuAD-explorer/dataset/train-v1.1.json to /Users/ocean/ParlAI/data/SQuAD/train-v1.1.json
Downloading train-v1.1.json: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 30.3M/30.3M [00:02<00:00, 14.4MB/s]
21:22:37 | Downloading https://rajpurkar.github.io/SQuAD-explorer/dataset/dev-v1.1.json to /Users/ocean/ParlAI/data/SQuAD/dev-v1.1.json
Downloading dev-v1.1.json: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 4.85M/4.85M [00:01<00:00, 4.60MB/s]
loading: /Users/ocean/ParlAI/data/SQuAD/train-v1.1.json
- - - NEW EPISODE: squad - - -
Architecturally, the school has a Catholic character. Atop the Main Building's gold dome is a golden statue of the Virgin Mary. Immediately in front of the Main Building and facing it, is a copper statue of Christ with arms upraised with the legend "Venite Ad Me Omnes". Next to the Main Building is the Basilica of the Sacred Heart. Immediately behind the basilica is the Grotto, a Marian place of prayer and reflection. It is a replica of the grotto at Lourdes, France where the Virgin Mary reputedly appeared to Saint Bernadette Soubirous in 1858. At the end of the main drive (and in a direct line that connects through 3 statues and the Gold Dome), is a simple, modern stone statue of Mary.
To whom did the Virgin Mary allegedly appear in 1858 in Lourdes France?
Saint Bernadette Soubirous
- - - NEW EPISODE: squad - - -
Architecturally, the school has a Catholic character. Atop the Main Building's gold dome is a golden statue of the Virgin Mary. Immediately in front of the Main Building and facing it, is a copper statue of Christ with arms upraised with the legend "Venite Ad Me Omnes". Next to the Main Building is the Basilica of the Sacred Heart. Immediately behind the basilica is the Grotto, a Marian place of prayer and reflection. It is a replica of the grotto at Lourdes, France where the Virgin Mary reputedly appeared to Saint Bernadette Soubirous in 1858. At the end of the main drive (and in a direct line that connects through 3 statues and the Gold Dome), is a simple, modern stone statue of Mary.
What is in front of the Notre Dame Main Building?
a copper statue of Christ
- - - NEW EPISODE: squad - - -
Architecturally, the school has a Catholic character. Atop the Main Building's gold dome is a golden statue of the Virgin Mary. Immediately in front of the Main Building and facing it, is a copper statue of Christ with arms upraised with the legend "Venite Ad Me Omnes". Next to the Main Building is the Basilica of the Sacred Heart. Immediately behind the basilica is the Grotto, a Marian place of prayer and reflection. It is a replica of the grotto at Lourdes, France where the Virgin Mary reputedly appeared to Saint Bernadette Soubirous in 1858. At the end of the main drive (and in a direct line that connects through 3 statues and the Gold Dome), is a simple, modern stone statue of Mary.
The Basilica of the Sacred heart at Notre Dame is beside to which structure?
the Main Building
- - - NEW EPISODE: squad - - -
Architecturally, the school has a Catholic character. Atop the Main Building's gold dome is a golden statue of the Virgin Mary. Immediately in front of the Main Building and facing it, is a copper statue of Christ with arms upraised with the legend "Venite Ad Me Omnes". Next to the Main Building is the Basilica of the Sacred Heart. Immediately behind the basilica is the Grotto, a Marian place of prayer and reflection. It is a replica of the grotto at Lourdes, France where the Virgin Mary reputedly appeared to Saint Bernadette Soubirous in 1858. At the end of the main drive (and in a direct line that connects through 3 statues and the Gold Dome), is a simple, modern stone statue of Mary.
What is the Grotto at Notre Dame?
a Marian place of prayer and reflection
- - - NEW EPISODE: squad - - -
Architecturally, the school has a Catholic character. Atop the Main Building's gold dome is a golden statue of the Virgin Mary. Immediately in front of the Main Building and facing it, is a copper statue of Christ with arms upraised with the legend "Venite Ad Me Omnes". Next to the Main Building is the Basilica of the Sacred Heart. Immediately behind the basilica is the Grotto, a Marian place of prayer and reflection. It is a replica of the grotto at Lourdes, France where the Virgin Mary reputedly appeared to Saint Bernadette Soubirous in 1858. At the end of the main drive (and in a direct line that connects through 3 statues and the Gold Dome), is a simple, modern stone statue of Mary.
What sits on top of the Main Building at Notre Dame?
a golden statue of the Virgin Mary
- - - NEW EPISODE: squad - - -
As at most other universities, Notre Dame's students run a number of news media outlets. The nine student-run outlets include three newspapers, both a radio and television station, and several magazines and journals. Begun as a one-page journal in September 1876, the Scholastic magazine is issued twice monthly and claims to be the oldest continuous collegiate publication in the United States. The other magazine, The Juggler, is released twice a year and focuses on student literature and artwork. The Dome yearbook is published annually. The newspapers have varying publication interests, with The Observer published daily and mainly reporting university and other news, and staffed by students from both Notre Dame and Saint Mary's College. Unlike Scholastic and The Dome, The Observer is an independent publication and does not have a faculty advisor or any editorial oversight from the University. In 1987, when some students believed that The Observer began to show a conservative bias, a liberal newspaper, Common Sense was published. Likewise, in 2003, when other students believed that the paper showed a liberal bias, the conservative paper Irish Rover went into production. Neither paper is published as often as The Observer; however, all three are distributed to all students. Finally, in Spring 2008 an undergraduate journal for political science research, Beyond Politics, made its debut.
When did the Scholastic Magazine of Notre dame begin publishing?
September 1876
- - - NEW EPISODE: squad - - -
As at most other universities, Notre Dame's students run a number of news media outlets. The nine student-run outlets include three newspapers, both a radio and television station, and several magazines and journals. Begun as a one-page journal in September 1876, the Scholastic magazine is issued twice monthly and claims to be the oldest continuous collegiate publication in the United States. The other magazine, The Juggler, is released twice a year and focuses on student literature and artwork. The Dome yearbook is published annually. The newspapers have varying publication interests, with The Observer published daily and mainly reporting university and other news, and staffed by students from both Notre Dame and Saint Mary's College. Unlike Scholastic and The Dome, The Observer is an independent publication and does not have a faculty advisor or any editorial oversight from the University. In 1987, when some students believed that The Observer began to show a conservative bias, a liberal newspaper, Common Sense was published. Likewise, in 2003, when other students believed that the paper showed a liberal bias, the conservative paper Irish Rover went into production. Neither paper is published as often as The Observer; however, all three are distributed to all students. Finally, in Spring 2008 an undergraduate journal for political science research, Beyond Politics, made its debut.
How often is Notre Dame's the Juggler published?
twice
- - - NEW EPISODE: squad - - -
As at most other universities, Notre Dame's students run a number of news media outlets. The nine student-run outlets include three newspapers, both a radio and television station, and several magazines and journals. Begun as a one-page journal in September 1876, the Scholastic magazine is issued twice monthly and claims to be the oldest continuous collegiate publication in the United States. The other magazine, The Juggler, is released twice a year and focuses on student literature and artwork. The Dome yearbook is published annually. The newspapers have varying publication interests, with The Observer published daily and mainly reporting university and other news, and staffed by students from both Notre Dame and Saint Mary's College. Unlike Scholastic and The Dome, The Observer is an independent publication and does not have a faculty advisor or any editorial oversight from the University. In 1987, when some students believed that The Observer began to show a conservative bias, a liberal newspaper, Common Sense was published. Likewise, in 2003, when other students believed that the paper showed a liberal bias, the conservative paper Irish Rover went into production. Neither paper is published as often as The Observer; however, all three are distributed to all students. Finally, in Spring 2008 an undergraduate journal for political science research, Beyond Politics, made its debut.
What is the daily student paper at Notre Dame called?
The Observer
- - - NEW EPISODE: squad - - -
As at most other universities, Notre Dame's students run a number of news media outlets. The nine student-run outlets include three newspapers, both a radio and television station, and several magazines and journals. Begun as a one-page journal in September 1876, the Scholastic magazine is issued twice monthly and claims to be the oldest continuous collegiate publication in the United States. The other magazine, The Juggler, is released twice a year and focuses on student literature and artwork. The Dome yearbook is published annually. The newspapers have varying publication interests, with The Observer published daily and mainly reporting university and other news, and staffed by students from both Notre Dame and Saint Mary's College. Unlike Scholastic and The Dome, The Observer is an independent publication and does not have a faculty advisor or any editorial oversight from the University. In 1987, when some students believed that The Observer began to show a conservative bias, a liberal newspaper, Common Sense was published. Likewise, in 2003, when other students believed that the paper showed a liberal bias, the conservative paper Irish Rover went into production. Neither paper is published as often as The Observer; however, all three are distributed to all students. Finally, in Spring 2008 an undergraduate journal for political science research, Beyond Politics, made its debut.
How many student news papers are found at Notre Dame?
three
- - - NEW EPISODE: squad - - -
As at most other universities, Notre Dame's students run a number of news media outlets. The nine student-run outlets include three newspapers, both a radio and television station, and several magazines and journals. Begun as a one-page journal in September 1876, the Scholastic magazine is issued twice monthly and claims to be the oldest continuous collegiate publication in the United States. The other magazine, The Juggler, is released twice a year and focuses on student literature and artwork. The Dome yearbook is published annually. The newspapers have varying publication interests, with The Observer published daily and mainly reporting university and other news, and staffed by students from both Notre Dame and Saint Mary's College. Unlike Scholastic and The Dome, The Observer is an independent publication and does not have a faculty advisor or any editorial oversight from the University. In 1987, when some students believed that The Observer began to show a conservative bias, a liberal newspaper, Common Sense was published. Likewise, in 2003, when other students believed that the paper showed a liberal bias, the conservative paper Irish Rover went into production. Neither paper is published as often as The Observer; however, all three are distributed to all students. Finally, in Spring 2008 an undergraduate journal for political science research, Beyond Politics, made its debut.
In what year did the student paper Common Sense begin publication at Notre Dame?
1987
21:22:38 | loaded 87599 episodes with a total of 87599 examples
ocean@AfoGuardMacBook-Pro ParlAI %
https://github.com/facebookresearch/ParlAI
に掲載されている以下を実行してみる。
https://github.com/facebookresearch/ParlAI
Evaluate an IR baseline model on the validation set of the Personachat task:
> parlai eval_model -m ir_baseline -t personachat -dt valid
うまく実行できた。
__IR baseline model__を__validation set__に適用したところ、正解率その他の結果一式が、次の形で出力された。
accuracy bleu-4 exs f1 hits@1 hits@10 hits@100 hits@5
.1595 .1597 7801 .2540 .1595 .6499 1 .4299
以下がその実行結果
Terminal
ocean@AfoGuardMacBook-Pro ParlAI % parlai eval_model -m ir_baseline -t personachat -dt valid
21:35:05 | Opt:
21:35:05 | aggregate_micro: False
21:35:05 | allow_missing_init_opts: False
21:35:05 | batchsize: 1
21:35:05 | bpe_add_prefix_space: None
21:35:05 | bpe_debug: False
21:35:05 | bpe_dropout: None
21:35:05 | bpe_merge: None
21:35:05 | bpe_vocab: None
21:35:05 | datapath: /Users/ocean/ParlAI/data
21:35:05 | datatype: valid
21:35:05 | dict_class: None
21:35:05 | dict_endtoken: __end__
21:35:05 | dict_file: None
21:35:05 | dict_initpath: None
21:35:05 | dict_language: english
21:35:05 | dict_loaded: False
21:35:05 | dict_lower: False
21:35:05 | dict_max_ngram_size: -1
21:35:05 | dict_maxtokens: -1
21:35:05 | dict_minfreq: 0
21:35:05 | dict_nulltoken: __null__
21:35:05 | dict_starttoken: __start__
21:35:05 | dict_textfields: text,labels
21:35:05 | dict_tokenizer: re
21:35:05 | dict_unktoken: __unk__
21:35:05 | display_examples: False
21:35:05 | download_path: None
21:35:05 | dynamic_batching: None
21:35:05 | hide_labels: False
21:35:05 | history_size: 1
21:35:05 | image_cropsize: 224
21:35:05 | image_mode: raw
21:35:05 | image_size: 256
21:35:05 | init_model: None
21:35:05 | init_opt: None
21:35:05 | label_candidates_file: None
21:35:05 | length_penalty: 0.5
21:35:05 | log_every_n_secs: 10
21:35:05 | log_keep_fields: all
21:35:05 | loglevel: info
21:35:05 | metrics: default
21:35:05 | model: ir_baseline
21:35:05 | model_file: None
21:35:05 | multitask_weights: [1]
21:35:05 | num_examples: -1
21:35:05 | override: "{'model': 'ir_baseline', 'task': 'personachat', 'datatype': 'valid'}"
21:35:05 | parlai_home: /Users/ocean/ParlAI
21:35:05 | report_filename:
21:35:05 | save_format: conversations
21:35:05 | save_world_logs: False
21:35:05 | starttime: Dec18_21-35
21:35:05 | task: personachat
21:35:05 | tensorboard_log: False
21:35:05 | tensorboard_logdir: None
21:35:05 | verbose: False
21:35:05 | Current ParlAI commit: 99160674564847c8ed68bc21437eab8c9301e95d
21:35:05 | Evaluating task personachat using datatype valid.
21:35:05 | creating task(s): personachat
[building data: /Users/ocean/ParlAI/data/Persona-Chat]
21:35:05 | Downloading http://parl.ai/downloads/personachat/personachat.tgz to /Users/ocean/ParlAI/data/Persona-Chat/personachat.tgz
Downloading personachat.tgz: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 223M/223M [00:17<00:00, 12.7MB/s]
21:35:31 | loading fbdialog data: /Users/ocean/ParlAI/data/Persona-Chat/personachat/valid_self_original.txt
21:35:36 | Finished evaluating tasks ['personachat'] using datatype valid
accuracy bleu-4 exs f1 hits@1 hits@10 hits@100 hits@5
.1595 .1597 7801 .2540 .1595 .6499 1 .4299
ocean@AfoGuardMacBook-Pro ParlAI %
次に実行するのは、これ。
Train a single layer transformer on PersonaChat (requires pytorch and torchtext). Detail: embedding size 300, 4 attention heads, 2 epochs using batchsize 64, word vectors are initialized with fasttext and the other elements of the batch are used as negative during training.
> parlai train_model -t personachat -m transformer/ranker -mf /tmp/model_tr6 --n-layers 1 --embedding-size 300 --ffn-size 600 --n-heads 4 --num-epochs 2 -veps 0.25 -bs 64 -lr 0.001 --dropout 0.1 --embedding-type fasttext_cc --candidates batch
Terminal
ocean@AfoGuardMacBook-Pro ParlAI % parlai train_model -t personachat -m transformer/ranker -mf /tmp/model_tr6 --n-layers 1 --embedding-size 300 --ffn-size 600 --n-heads 4 --num-epochs 2 -veps 0.25 -bs 64 -lr 0.001 --dropout 0.1 --embedding-type fasttext_cc --candidates batch
21:39:21 | building dictionary first...
21:39:21 | Opt:
21:39:21 | activation: relu
21:39:21 | adafactor_eps: '(1e-30, 0.001)'
21:39:21 | adam_eps: 1e-08
（ ・・・省略・・・ ）
21:39:21 | update_freq: 1
21:39:21 | use_memories: False
21:39:21 | use_reply: label
21:39:21 | validation_cutoff: 1.0
21:39:21 | validation_every_n_epochs: 0.25
21:39:21 | validation_every_n_secs: -1
21:39:21 | validation_max_exs: -1
21:39:21 | validation_metric: accuracy
21:39:21 | validation_metric_mode: None
21:39:21 | validation_patience: 10
21:39:21 | validation_share_agent: False
21:39:21 | variant: aiayn
21:39:21 | verbose: False
21:39:21 | warmup_rate: 0.0001
21:39:21 | warmup_updates: -1
21:39:21 | weight_decay: None
21:39:21 | wrap_memory_encoder: False
21:39:21 | Current ParlAI commit: 99160674564847c8ed68bc21437eab8c9301e95d
21:39:21 | creating task(s): personachat
21:39:21 | loading fbdialog data: /Users/ocean/ParlAI/data/Persona-Chat/personachat/train_self_original.txt
Building dictionary: 0%| | 0.00/65.7k [00:00<?, ?ex/s]21:39:22 | loading fbdialog data: /Users/ocean/ParlAI/data/Persona-Chat/personachat/train_self_original.txt
Building dictionary: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 65.7k/65.7k [00:03<00:00, 17.7kex/s]
21:39:25 | Saving dictionary to /tmp/model_tr6.dict
21:39:26 | dictionary built with 18745 tokens in 0.0s
21:39:26 | No model with opt yet at: /tmp/model_tr6(.opt)
21:39:26 | loading dictionary from /tmp/model_tr6.dict
21:39:26 | num words = 18745
/Users/ocean/ParlAI/data/models/fasttext_cc_vectors/crawl-300d-2M.vec.zip: 1.52GB [01:51, 13.7MB/s]
0%| | 0/1999995 [00:00<?, ?it/s]Skipping token b'1999995' with 1-dimensional vector [b'300']; likely a header
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1999995/1999995 [02:47<00:00, 11923.81it/s]
21:44:41 | Initialized embeddings for 18284 tokens (97.5%) from fasttext_cc.
21:44:41 | Total parameters: 6,834,600 (6,527,400 trainable)
21:44:42 | Opt:
21:44:42 | activation: relu
21:44:42 | adafactor_eps: '(1e-30, 0.001)'
（ ・・・省略・・・ ）
21:44:42 | validation_max_exs: -1
21:44:42 | validation_metric: accuracy
21:44:42 | validation_metric_mode: None
21:44:42 | validation_patience: 10
21:44:42 | validation_share_agent: False
21:44:42 | variant: aiayn
21:44:42 | verbose: False
21:44:42 | warmup_rate: 0.0001
21:44:42 | warmup_updates: -1
21:44:42 | weight_decay: None
21:44:42 | wrap_memory_encoder: False
21:44:42 | Current ParlAI commit: 99160674564847c8ed68bc21437eab8c9301e95d
21:44:42 | creating task(s): personachat
21:44:42 | loading fbdialog data: /Users/ocean/ParlAI/data/Persona-Chat/personachat/train_self_original.txt
21:44:43 | training...
21:44:43 | [ Executing train mode with batch labels as set of candidates. ]
21:44:54 | time:11s total_exs:960 epochs:0.01 time_left:1450s
clip ctpb ctps exps exs gnorm lr ltpb ltps mean_loss mrr rank total_train_updates tpb tps train_accuracy ups
1 8873 12513 90.12 960 4.92 .0010 782.4 1103 4.345 .07379 32.19 15 9656 13616 .01667 1.41
21:45:04 | time:21s total_exs:1792 epochs:0.03 time_left:1543s
clip ctpb ctps exps exs gnorm lr ltpb ltps mean_loss mrr rank total_train_updates tpb tps train_accuracy ups
1 9062 11620 82.06 832 1.767 .0010 772.4 990.3 4.15 .08851 29.61 28 9835 12610 .01803 1.282
21:45:15 | time:32s total_exs:2624 epochs:0.04 time_left:1570s
clip ctpb ctps exps exs gnorm lr ltpb ltps mean_loss mrr rank total_train_updates tpb tps train_accuracy ups
1 8964 10958 78.23 832 1.784 .0010 760.2 929.3 4.102 .1094 26.9 41 9724 11887 .03005 1.223
21:45:25 | time:43s total_exs:3392 epochs:0.05 time_left:1604s
clip ctpb ctps exps exs gnorm lr ltpb ltps mean_loss mrr rank total_train_updates tpb tps train_accuracy ups
1 9238 10561 73.15 768 2.906 .0010 763.8 873 4.049 .1223 24.6 53 10002 11434 .04297 1.143
（ ・・・省略・・・ ）
21:47:42 | time:179s total_exs:12544 epochs:0.19 time_left:1700s
clip ctpb ctps exps exs gnorm lr ltpb ltps mean_loss mrr rank total_train_updates tpb tps train_accuracy ups
1 9145 7998 55.96 576 4.974 .0010 764 668 3.614 .2696 16.51 196 9909 8666 .1597 .8746
21:47:53 | time:190s total_exs:13184 epochs:0.20 time_left:1705s
clip ctpb ctps exps exs gnorm lr ltpb ltps mean_loss mrr rank total_train_updates tpb tps train_accuracy ups
1 9284 8636 59.53 640 5.013 .0010 733.2 682 3.659 .2405 16.55 206 10017 9318 .1281 .9303
（ Ctrrl-Zで強制終了 ）
zsh: suspended parlai train_model -t personachat -m transformer/ranker -mf /tmp/model_tr6 1
ocean@AfoGuardMacBook-Pro ParlAI %
次にやること
次のチュートリアルサイトに倣って、Pythonで学習してみる。
・ HatenaBlog ディープラーニングブログ 2017-10-19 「対話モデルの訓練/評価フレームワーク ParlAI がすごい」
Terminal
pip install -r requirements.txt
# pytorch のインストールは http://pytorch.org/ に従う
# pyzmq, regex, spacy のインストール (requirements_ext.txt がなければ個別に)
pip install -r requirements_ext.txt
# ParlAI のインストール
sudo python setup.py develop
# ParlAI 実行
python examples/display_data.py -t babi:task1k:1
サンプルコードの最後にあるコマンドを実行すると、__display_data.py__が見つからないと怒られる。
__display_data.py__のファイルパスが正しくないからだ。
Terminal
ocean@AfoGuardMacBook-Pro ParlAI % python examples/display_data.py -t babi:task1k:1
/Users/ocean/.pyenv/versions/3.9.0/bin/python: can't open file '/Users/ocean/ParlAI/examples/display_data.py': [Errno 2] No such file or directory
ocean@AfoGuardMacBook-Pro ParlAI %
__display_data.py__は、ParlAI/parlai/scripts/の配下にある。
Terminal
ocean@AfoGuardMacBook-Pro ParlAI % ls
CODE_OF_CONDUCT.md MANIFEST.in autoformat.sh data mypy.ini projects requirements.txt website
CONTRIBUTING.md NEWS.md codecov.yml docs parlai pyproject.toml setup.py
LICENSE README.md conftest.py example_parlai_internal parlai.egg-info pytest.ini tests
ocean@AfoGuardMacBook-Pro ParlAI %
ocean@AfoGuardMacBook-Pro ParlAI % ls parlai
README.md __main__.py agents core mturk scripts utils
__init__.py __pycache__ chat_service crowdsourcing nn tasks zoo
ocean@AfoGuardMacBook-Pro ParlAI %
ocean@AfoGuardMacBook-Pro ParlAI % ls parlai/scripts
__init__.py convo_render.py distributed_train.py multiprocessing_eval.py self_chat.py
__pycache__ data_stats.py eval_model.py multiprocessing_train.py token_stats.py
build_candidates.py detect_offensive_language.py eval_wordstat.py party.py train_model.py
build_dict.py display_data.py extract_image_feature.py profile_interactive.py vacuum.py
compare_opts.py display_model.py interactive.py profile_train.py verify_data.py
convert_data_to_parlai_format.py distributed_eval.py interactive_web.py safe_interactive.py
ocean@AfoGuardMacBook-Pro ParlAI %
以下で実行成功
Terminal
ocean@AfoGuardMacBook-Pro ParlAI % python parlai/scripts/display_data.py -t babi:task1k:1
22:20:18 | Opt:
22:20:18 | allow_missing_init_opts: False
22:20:18 | batchsize: 1
22:20:18 | datapath: /Users/ocean/ParlAI/data
22:20:18 | datatype: train:ordered
22:20:18 | dict_class: None
22:20:18 | display_add_fields:
22:20:18 | download_path: None
22:20:18 | dynamic_batching: None
22:20:18 | hide_labels: False
22:20:18 | ignore_agent_reply: True
22:20:18 | image_cropsize: 224
22:20:18 | image_mode: raw
22:20:18 | image_size: 256
22:20:18 | init_model: None
22:20:18 | init_opt: None
22:20:18 | loglevel: info
22:20:18 | max_display_len: 1000
22:20:18 | model: None
22:20:18 | model_file: None
22:20:18 | multitask_weights: [1]
22:20:18 | num_examples: 10
22:20:18 | override: "{'task': 'babi:task1k:1'}"
22:20:18 | parlai_home: /Users/ocean/ParlAI
22:20:18 | starttime: Dec18_22-20
22:20:18 | task: babi:task1k:1
22:20:18 | verbose: False
22:20:18 | Current ParlAI commit: 99160674564847c8ed68bc21437eab8c9301e95d
22:20:18 | creating task(s): babi:task1k:1
[building data: /Users/ocean/ParlAI/data/bAbI]
22:20:18 | Downloading http://parl.ai/downloads/babi/babi.tar.gz to /Users/ocean/ParlAI/data/bAbI/babi.tar.gz
Downloading babi.tar.gz: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 19.2M/19.2M [00:04<00:00, 4.46MB/s]
22:20:24 | loading fbdialog data: /Users/ocean/ParlAI/data/bAbI/tasks_1-20_v1-2/en-valid-nosf/qa1_train.txt
- - - NEW EPISODE: babi:task1k:1 - - -
Mary moved to the bathroom.
John went to the hallway.
Where is Mary?
bathroom
Daniel went back to the hallway.
Sandra moved to the garden.
Where is Daniel?
hallway
John moved to the office.
Sandra journeyed to the bathroom.
Where is Daniel?
hallway
Mary moved to the hallway.
Daniel travelled to the office.
Where is Daniel?
office
John went back to the garden.
John moved to the bedroom.
Where is Sandra?
bathroom
- - - NEW EPISODE: babi:task1k:1 - - -
Sandra travelled to the office.
Sandra went to the bathroom.
Where is Sandra?
bathroom
Mary went to the bedroom.
Daniel moved to the hallway.
Where is Sandra?
bathroom
John went to the garden.
John travelled to the office.
Where is Sandra?
bathroom
Daniel journeyed to the bedroom.
Daniel travelled to the hallway.
Where is John?
office
John went to the bedroom.
John travelled to the office.
Where is Daniel?
hallway
22:20:24 | loaded 180 episodes with a total of 900 examples
ocean@AfoGuardMacBook-Pro ParlAI %
seq2seq を bAbI で訓練/評価
Terminal
ocean@AfoGuardMacBook-Pro ParlAI % mkdir -p ./parlai/agents/seq2seq/model_file
ocean@AfoGuardMacBook-Pro ParlAI % ls model_file
ls: model_file: No such file or directory
ocean@AfoGuardMacBook-Pro ParlAI %
ocean@AfoGuardMacBook-Pro ParlAI % ls ./parlai/agents/seq2seq/model_file
ocean@AfoGuardMacBook-Pro ParlAI % ls
CODE_OF_CONDUCT.md MANIFEST.in autoformat.sh data mypy.ini projects requirements.txt website
CONTRIBUTING.md NEWS.md codecov.yml docs parlai pyproject.toml setup.py
LICENSE README.md conftest.py example_parlai_internal parlai.egg-info pytest.ini tests
ocean@AfoGuardMacBook-Pro ParlAI %
ocean@AfoGuardMacBook-Pro ParlAI % ls parlai
README.md __main__.py agents core mturk scripts utils
__init__.py __pycache__ chat_service crowdsourcing nn tasks zoo
ocean@AfoGuardMacBook-Pro ParlAI %
ocean@AfoGuardMacBook-Pro ParlAI % ls parlai/agents
README.md alice bert_ranker fixed_response image_seq2seq memnn repeat_query seq2seq tfidf_retriever
__init__.py bart drqa hred ir_baseline random_candidate retriever_reader starspace transformer
__pycache__ bert_classifier examples hugging_face local_human repeat_label safe_local_human test_agents unigram
ocean@AfoGuardMacBook-Pro ParlAI %
ocean@AfoGuardMacBook-Pro ParlAI % ls parlai/agents/seq2seq
README.md __init__.py model_file modules.py seq2seq.py
ocean@AfoGuardMacBook-Pro ParlAI %
ocean@AfoGuardMacBook-Pro ParlAI % ls parlai/agents/seq2seq/model_file
ocean@AfoGuardMacBook-Pro ParlAI %
Terminal
ocean@AfoGuardMacBook-Pro ParlAI % python examples/train_model.py -m seq2seq -t babi:Task10k:1 -mf './parlai/agents/seq2seq/model_file/babi' -e 20 -lr 0.5 -bs 32 -hs 128 -ltim 2 -vtim 20
/Users/ocean/.pyenv/versions/3.9.0/bin/python: can't open file '/Users/ocean/ParlAI/examples/train_model.py': [Errno 2] No such file or directory
ocean@AfoGuardMacBook-Pro ParlAI %
ocean@AfoGuardMacBook-Pro ParlAI % python parlai/scripts/train_model.py -m seq2seq -t babi:Task10k:1 -mf './parlai/agents/seq2seq/model_file/babi' -e 20 -lr 0.5 -bs 32 -hs 128 -ltim 2 -vtim 20
usage: train_model.py [-h] [--helpall] [-o INIT_OPT] [--allow-missing-init-opts ALLOW_MISSING_INIT_OPTS] [-t TASK] [-dt DATATYPE] [-bs BATCHSIZE] [-dynb {None,batchsort,full}] [-v] [-dp DATAPATH] [-m MODEL]
[-mf MODEL_FILE] [-im INIT_MODEL] [-et EVALTASK] [-eps NUM_EPOCHS] [-ttim MAX_TRAIN_TIME] [-vtim VALIDATION_EVERY_N_SECS] [-stim SAVE_EVERY_N_SECS] [-sval SAVE_AFTER_VALID]
[-veps VALIDATION_EVERY_N_EPOCHS] [-vp VALIDATION_PATIENCE] [-vmt VALIDATION_METRIC] [-vmm {max,min}] [-mcs METRICS] [-micro AGGREGATE_MICRO] [-tblog TENSORBOARD_LOG]
[-tblogdir TENSORBOARD_LOGDIR] [--bpe-vocab BPE_VOCAB] [--bpe-merge BPE_MERGE] [--bpe-dropout BPE_DROPOUT]
Train a model
optional arguments:
-h, --help
show this help message and exit
--helpall
Show usage, including advanced arguments.
Main ParlAI Arguments:
-o, --init-opt INIT_OPT
Path to json file of options. Note: Further Command-line arguments override file-based options. (default: None)
--allow-missing-init-opts ALLOW_MISSING_INIT_OPTS
Warn instead of raising if an argument passed in with --init-opt is not in the target opt. (default: False)
-t, --task TASK
ParlAI task(s), e.g. "babi:Task1" or "babi,cbt" (default: None)
-dt, --datatype DATATYPE
choose from: train, train:ordered, valid, test. to stream data add ":stream" to any option (e.g., train:stream). by default train is random with replacement, valid is ordered, test is ordered. (default:
train)
-bs, --batchsize BATCHSIZE
batch size for minibatch training schemes (default: 1)
-dynb, --dynamic-batching {None,batchsort,full}
Use dynamic batching (default: None)
-v, --verbose
Print all messages
-dp, --datapath DATAPATH
path to datasets, defaults to {parlai_dir}/data (default: None)
ParlAI Model Arguments:
-m, --model MODEL
the model class name. can match parlai/agents/<model> for agents in that directory, or can provide a fully specified module for `from X import Y` via `-m X:Y` (e.g. `-m
parlai.agents.seq2seq.seq2seq:Seq2SeqAgent`) (default: None)
-mf, --model-file MODEL_FILE
model file name for loading and saving models (default: None)
-im, --init-model INIT_MODEL
Initialize model weights and dict from this file (default: None)
Training Loop Arguments:
-et, --evaltask EVALTASK
task to use for valid/test (defaults to the one used for training) (default: None)
-eps, --num-epochs NUM_EPOCHS
-ttim, --max-train-time MAX_TRAIN_TIME
-vtim, --validation-every-n-secs VALIDATION_EVERY_N_SECS
Validate every n seconds. Saves model to model_file (if set) whenever best val metric is found (default: -1)
-stim, --save-every-n-secs SAVE_EVERY_N_SECS
Saves the model to model_file.checkpoint after every n seconds (default -1, never). (default: -1)
-sval, --save-after-valid SAVE_AFTER_VALID
Saves the model to model_file.checkpoint after every validation (default False).
-veps, --validation-every-n-epochs VALIDATION_EVERY_N_EPOCHS
Validate every n epochs. Saves model to model_file (if set) whenever best val metric is found (default: -1)
-vp, --validation-patience VALIDATION_PATIENCE
number of iterations of validation where result does not improve before we stop training (default: 10)
-vmt, --validation-metric VALIDATION_METRIC
key into report table for selecting best validation (default: accuracy)
-vmm, --validation-metric-mode {max,min}
how to optimize validation metric (max or min) (default: None)
-mcs, --metrics METRICS
list of metrics to show/compute, e.g. all, default,or give a list split by , like ppl,f1,accuracy,hits@1,rouge,bleuthe rouge metrics will be computed as rouge-1, rouge-2 and rouge-l (default: default)
-micro, --aggregate-micro AGGREGATE_MICRO
Report micro-averaged metrics instead of macro averaged metrics. (default: False)
Tensorboard Arguments:
-tblog, --tensorboard-log TENSORBOARD_LOG
Tensorboard logging of metrics, default is False
-tblogdir, --tensorboard-logdir TENSORBOARD_LOGDIR
Tensorboard logging directory, defaults to model_file.tensorboard (default: None)
BPEHelper Arguments:
--bpe-vocab BPE_VOCAB
path to pre-trained tokenizer vocab (default: None)
--bpe-merge BPE_MERGE
path to pre-trained tokenizer merge (default: None)
--bpe-dropout BPE_DROPOUT
Use BPE dropout during training. (default: None)
Parse Error: ambiguous option: -e could match -et, -eps
ocean@AfoGuardMacBook-Pro ParlAI %
次のコメントが出た。
Parse Error: ambiguous option: -e could match -et, -eps
__-etとeps__は以下にことらしい。
-et, --evaltask EVALTASK
-eps, --num-epochs NUM_EPOCHS
http://deeplearning.hatenablog.com/entry/parlai
によると、__eは、-e {エポック数}__を指定したのだった。
そこで、__eを:eps*__に変更する。
またエラーが起きた。
Terminal
ocean@AfoGuardMacBook-Pro ParlAI % python parlai/scripts/train_model.py -m seq2seq -t babi:Task10k:1 -mf './parlai/agents/seq2seq/model_file/babi' -eps 20 -lr 0.5 -bs 32 -hs 128 -ltim 2 -vtim 20
usage: train_model.py [-h] [--helpall] [-o INIT_OPT] [--allow-missing-init-opts ALLOW_MISSING_INIT_OPTS] [-t TASK] [-dt DATATYPE] [-bs BATCHSIZE] [-dynb {None,batchsort,full}] [-v] [-dp DATAPATH] [-m MODEL]
[-mf MODEL_FILE] [-im INIT_MODEL] [-et EVALTASK] [-eps NUM_EPOCHS] [-ttim MAX_TRAIN_TIME] [-vtim VALIDATION_EVERY_N_SECS] [-stim SAVE_EVERY_N_SECS] [-sval SAVE_AFTER_VALID]
[-veps VALIDATION_EVERY_N_EPOCHS] [-vp VALIDATION_PATIENCE] [-vmt VALIDATION_METRIC] [-vmm {max,min}] [-mcs METRICS] [-micro AGGREGATE_MICRO] [-tblog TENSORBOARD_LOG]
[-tblogdir TENSORBOARD_LOGDIR] [--bpe-vocab BPE_VOCAB] [--bpe-merge BPE_MERGE] [--bpe-dropout BPE_DROPOUT]
Train a model
optional arguments:
-h, --help
show this help message and exit
--helpall
Show usage, including advanced arguments.
Main ParlAI Arguments:
-o, --init-opt INIT_OPT
Path to json file of options. Note: Further Command-line arguments override file-based options. (default: None)
--allow-missing-init-opts ALLOW_MISSING_INIT_OPTS
Warn instead of raising if an argument passed in with --init-opt is not in the target opt. (default: False)
-t, --task TASK
ParlAI task(s), e.g. "babi:Task1" or "babi,cbt" (default: None)
-dt, --datatype DATATYPE
choose from: train, train:ordered, valid, test. to stream data add ":stream" to any option (e.g., train:stream). by default train is random with replacement, valid is ordered, test is ordered. (default:
train)
-bs, --batchsize BATCHSIZE
batch size for minibatch training schemes (default: 1)
-dynb, --dynamic-batching {None,batchsort,full}
Use dynamic batching (default: None)
-v, --verbose
Print all messages
-dp, --datapath DATAPATH
path to datasets, defaults to {parlai_dir}/data (default: None)
ParlAI Model Arguments:
-m, --model MODEL
the model class name. can match parlai/agents/<model> for agents in that directory, or can provide a fully specified module for `from X import Y` via `-m X:Y` (e.g. `-m
parlai.agents.seq2seq.seq2seq:Seq2SeqAgent`) (default: None)
-mf, --model-file MODEL_FILE
model file name for loading and saving models (default: None)
-im, --init-model INIT_MODEL
Initialize model weights and dict from this file (default: None)
Training Loop Arguments:
-et, --evaltask EVALTASK
task to use for valid/test (defaults to the one used for training) (default: None)
-eps, --num-epochs NUM_EPOCHS
-ttim, --max-train-time MAX_TRAIN_TIME
-vtim, --validation-every-n-secs VALIDATION_EVERY_N_SECS
Validate every n seconds. Saves model to model_file (if set) whenever best val metric is found (default: -1)
-stim, --save-every-n-secs SAVE_EVERY_N_SECS
Saves the model to model_file.checkpoint after every n seconds (default -1, never). (default: -1)
-sval, --save-after-valid SAVE_AFTER_VALID
Saves the model to model_file.checkpoint after every validation (default False).
-veps, --validation-every-n-epochs VALIDATION_EVERY_N_EPOCHS
Validate every n epochs. Saves model to model_file (if set) whenever best val metric is found (default: -1)
-vp, --validation-patience VALIDATION_PATIENCE
number of iterations of validation where result does not improve before we stop training (default: 10)
-vmt, --validation-metric VALIDATION_METRIC
key into report table for selecting best validation (default: accuracy)
-vmm, --validation-metric-mode {max,min}
how to optimize validation metric (max or min) (default: None)
-mcs, --metrics METRICS
list of metrics to show/compute, e.g. all, default,or give a list split by , like ppl,f1,accuracy,hits@1,rouge,bleuthe rouge metrics will be computed as rouge-1, rouge-2 and rouge-l (default: default)
-micro, --aggregate-micro AGGREGATE_MICRO
Report micro-averaged metrics instead of macro averaged metrics. (default: False)
Tensorboard Arguments:
-tblog, --tensorboard-log TENSORBOARD_LOG
Tensorboard logging of metrics, default is False
-tblogdir, --tensorboard-logdir TENSORBOARD_LOGDIR
Tensorboard logging directory, defaults to model_file.tensorboard (default: None)
BPEHelper Arguments:
--bpe-vocab BPE_VOCAB
path to pre-trained tokenizer vocab (default: None)
--bpe-merge BPE_MERGE
path to pre-trained tokenizer merge (default: None)
--bpe-dropout BPE_DROPOUT
Use BPE dropout during training. (default: None)
Parse Error: argument -h/--help: ignored explicit argument 's'
ocean@AfoGuardMacBook-Pro ParlAI %
今度は、引数を与えずに実行したら、動いた。
Terminal
ocean@AfoGuardMacBook-Pro ParlAI % python parlai/scripts/train_model.py -m seq2seq -t babi:Task10k:1 -mf './parlai/agents/seq2seq/model_file/babi'
22:33:17 | building dictionary first...
22:33:17 | Opt:
22:33:17 | adafactor_eps: '(1e-30, 0.001)'
22:33:17 | adam_eps: 1e-08
22:33:17 | add_p1_after_newln: False
（ ・・・省略・・・ ）
22:33:17 | verbose: False
22:33:17 | warmup_rate: 0.0001
22:33:17 | warmup_updates: -1
22:33:17 | weight_decay: None
22:33:17 | Current ParlAI commit: 99160674564847c8ed68bc21437eab8c9301e95d
22:33:17 | creating task(s): babi:Task10k:1
22:33:17 | loading fbdialog data: /Users/ocean/ParlAI/data/bAbI/tasks_1-20_v1-2/en-valid-10k-nosf/qa1_train.txt
Building dictionary: 0%| | 0.00/9.00k [00:00<?, ?ex/s]22:33:17 | loading fbdialog data: /Users/ocean/ParlAI/data/bAbI/tasks_1-20_v1-2/en-valid-10k-nosf/qa1_train.txt
Building dictionary: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 9.00k/9.00k [00:00<00:00, 21.3kex/s]
22:33:18 | Saving dictionary to ./parlai/agents/seq2seq/model_file/babi.dict
22:33:18 | dictionary built with 26 tokens in 0.0s
22:33:18 | No model with opt yet at: ./parlai/agents/seq2seq/model_file/babi(.opt)
22:33:18 | loading dictionary from ./parlai/agents/seq2seq/model_file/babi.dict
22:33:18 | num words = 26
（ ・・・省略・・・ ）
22:33:18 | validation_max_exs: -1
22:33:18 | validation_metric: accuracy
22:33:18 | validation_metric_mode: None
22:33:18 | validation_patience: 10
22:33:18 | validation_share_agent: False
22:33:18 | verbose: False
22:33:18 | warmup_rate: 0.0001
22:33:18 | warmup_updates: -1
22:33:18 | weight_decay: None
22:33:18 | Current ParlAI commit: 99160674564847c8ed68bc21437eab8c9301e95d
22:33:18 | creating task(s): babi:Task10k:1
22:33:18 | loading fbdialog data: /Users/ocean/ParlAI/data/bAbI/tasks_1-20_v1-2/en-valid-10k-nosf/qa1_train.txt
22:33:18 | training...
22:33:28 | time:10s total_exs:260 epochs:0.03
clip ctpb ctps exps exs gnorm loss lr ltpb ltps ppl token_acc total_train_updates tpb tps ups
1 60.99 1580 25.91 260 5.732 1.307 1 2 51.81 3.694 .5673 260 62.99 1632 25.91
22:35:59 | time:161s total_exs:4166 epochs:0.46
clip ctpb ctps exps exs gnorm loss lr ltpb ltps ppl token_acc total_train_updates tpb tps ups
.8595 61.4 1485 24.19 242 5.142 1.253 1 2 48.37 3.501 .7293 4166 63.4 1534 24.19
22:36:09 | time:171s total_exs:4419 epochs:0.49
clip ctpb ctps exps exs gnorm loss lr ltpb ltps ppl token_acc total_train_updates tpb tps ups
.8458 61.14 1543 25.24 253 5.152 1.265 1 2 50.48 3.543 .7490 4419 63.14 1594 25.24
22:36:19 | time:181s total_exs:4690 epochs:0.52
clip ctpb ctps exps exs gnorm loss lr ltpb ltps ppl token_acc total_train_updates tpb tps ups
.8376 61.08 1647 26.96 271 5.386 1.275 1 2 53.91 3.577 .7509 4690 63.08 1701 26.96
22:36:29 | time:191s total_exs:4947 epochs:0.55
clip ctpb ctps exps exs gnorm loss lr ltpb ltps ppl token_acc total_train_updates tpb tps ups
.7782 61.04 1567 25.67 257 4.867 1.219 1 2 51.33 3.385 .7879 4947 63.04 1618 25.67
22:36:39 | time:201s total_exs:5214 epochs:0.58
clip ctpb ctps exps exs gnorm loss lr ltpb ltps ppl token_acc total_train_updates tpb tps ups
.7416 60.98 1628 26.69 267 5.2 1.211 1 2 53.39 3.356 .7640 5214 62.98 1681 26.7
（ Ctrl-Zで強制終了 ）
^Z
zsh: suspended python parlai/scripts/train_model.py -m seq2seq -t babi:Task10k:1 -mf
ocean@AfoGuardMacBook-Pro ParlAI %
次にやること
ローカル環境で、サンプルコードを動かすことができた。、
次は、ParlAIで、教師役と生徒役の2体のAIが、環境の中で学習を繰り返すシステム系を理解して、Google Colaboratoryさまざまなデータセットで、モデルの学習と推論を行ってみたい。
以下のサイトでは、Facebookから新しく出たBlenderbotを、Google Colaboratory環境上で、学習を試みている。
・ ColabでFacebookのSOTAなチャットボットを動かして、ついでに日本語化してみた
あと、Lightなるものもあるらしい。こちらはテキストゲームとのこと。