dockerで機械学習with anaconda(2)「ゼロから作るDeep Learning2自然言語処理編」斎藤 康毅 著
https://qiita.com/kaizen_nagoya/items/3b80dfc76933cea522c6
第７章
# python generate_better_text.py
Traceback (most recent call last):
File "generate_better_text.py", line 15, in <module>
model.load_params('../ch06/BetterRnnlm.pkl')
File "../common/base_model.py", line 39, in load_params
raise IOError('No file: ' + file_name)
OSError: No file: ../ch06/BetterRnnlm.pkl
ネットからch06のフォルダに https://www.oreilly.co.jp/pub/9784873118369/BetterRnnlm.pkl
ファイルをwget コマンドでダウンロード
# cd ../ch06
# wget https://www.oreilly.co.jp/pub/9784873118369/BetterRnnlm.pkl
--2019-02-21 05:22:37-- https://www.oreilly.co.jp/pub/9784873118369/BetterRnnlm.pkl
Resolving www.oreilly.co.jp (www.oreilly.co.jp)... 13.113.6.104, 54.65.133.130
Connecting to www.oreilly.co.jp (www.oreilly.co.jp)|13.113.6.104|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 39550902 (38M)
Saving to: ‘BetterRnnlm.pkl’
BetterRnnlm.pkl 100%[========================================================================================================================>] 37.72M 23.6MB/s in 1.6s
2019-02-21 05:22:39 (23.6 MB/s) - ‘BetterRnnlm.pkl’ saved [39550902/39550902]
# cd ../ch07
# python generate_better_text.py
you wanted to block a lot of long-term dominance by the north american singer and the human staffs.
besides mayor edward kennedy says the navy remains part of a group that has received several defendants or as much as many as its red cross or the trans world union 's oil it is the first time in legal terms in a country like the panama pop.
in russia he says that too much people do n't by this in washington there will have been executive responsibility for government and israeli capital and other governments.
a few months
--------------------------------------------------
the meaning of life is the first transfer to a succession of another known to internal allies most regulator carla hills yesterday.
the series of great japanese bank barely over the next few years and have n't changed.
mr. lee replies that banks now could be taking their own position to an outsider to determine if a move one diversity makes tough and think the solicitation of this will will be altered by the security or market.
doug contributed to this article.
a half less aggressively threatening previously divided federal benefit funding.
the slew of fixed projects often filed
# python show_addition_dataset.py
(45000, 7) (45000, 5)
(5000, 7) (5000, 5)
[ 3 0 2 0 0 11 5]
[ 6 0 11 7 5]
71+118
_189
そして
# python generate_text.py
you read.
he is trying to have to charges about eight inches in both the in tomorrow.
mr. roman will lead the introduction of lesson he is clear that mr. roman has sold some of the specialized.
he told mr. roman have no one in mr. freeman unless he would improvement at least permanent.
the chinese proposal was speed out their membership at the moment.
it happens.
he said that mr. jones succeeds i 'm told of lower business in the employer.
earlier this month which is what they can become a adequate
# python train_seq2seq.py
../common/trainer.py:7: UserWarning:
This call to matplotlib.use() has no effect because the backend has already
been chosen; matplotlib.use() must be called *before* pylab, matplotlib.pyplot,
or matplotlib.backends is imported for the first time.
The backend was *originally* set to 'Qt5Agg' by the following code:
File "train_seq2seq.py", line 5, in <module>
import matplotlib.pyplot as plt
File "/opt/conda/lib/python3.6/site-packages/matplotlib/pyplot.py", line 71, in <module>
from matplotlib.backends import pylab_setup
File "/opt/conda/lib/python3.6/site-packages/matplotlib/backends/__init__.py", line 16, in <module>
line for line in traceback.format_stack()
mpl.use('Agg')
| epoch 1 | iter 1 / 351 | time 0[s] | loss 2.56
| epoch 1 | iter 21 / 351 | time 0[s] | loss 2.53
| epoch 1 | iter 41 / 351 | time 0[s] | loss 2.17
| epoch 1 | iter 61 / 351 | time 1[s] | loss 1.96
| epoch 1 | iter 81 / 351 | time 1[s] | loss 1.92
| epoch 1 | iter 101 / 351 | time 2[s] | loss 1.87
| epoch 1 | iter 121 / 351 | time 2[s] | loss 1.85
| epoch 1 | iter 141 / 351 | time 3[s] | loss 1.83
| epoch 1 | iter 161 / 351 | time 3[s] | loss 1.79
| epoch 1 | iter 181 / 351 | time 4[s] | loss 1.77
| epoch 1 | iter 201 / 351 | time 4[s] | loss 1.77
| epoch 1 | iter 221 / 351 | time 5[s] | loss 1.76
| epoch 1 | iter 241 / 351 | time 5[s] | loss 1.76
| epoch 1 | iter 261 / 351 | time 5[s] | loss 1.76
| epoch 1 | iter 281 / 351 | time 6[s] | loss 1.75
| epoch 1 | iter 301 / 351 | time 6[s] | loss 1.74
| epoch 1 | iter 321 / 351 | time 7[s] | loss 1.75
| epoch 1 | iter 341 / 351 | time 7[s] | loss 1.74
Q 77+85
T 162
☒ 100
---
Q 975+164
T 1139
☒ 1000
---
Q 582+84
T 666
☒ 1000
---
Q 8+155
T 163
☒ 100
---
Q 367+55
T 422
☒ 1000
---
Q 600+257
T 857
☒ 1000
---
Q 761+292
T 1053
☒ 1000
---
Q 830+597
T 1427
☒ 1000
---
Q 26+838
T 864
☒ 1000
---
Q 143+93
T 236
☒ 100
---
val acc 0.180%
| epoch 2 | iter 1 / 351 | time 0[s] | loss 1.74
| epoch 2 | iter 21 / 351 | time 0[s] | loss 1.73
| epoch 2 | iter 41 / 351 | time 1[s] | loss 1.74
| epoch 2 | iter 61 / 351 | time 1[s] | loss 1.74
| epoch 2 | iter 81 / 351 | time 2[s] | loss 1.73
| epoch 2 | iter 101 / 351 | time 2[s] | loss 1.73
| epoch 2 | iter 121 / 351 | time 3[s] | loss 1.72
| epoch 2 | iter 141 / 351 | time 3[s] | loss 1.71
| epoch 2 | iter 161 / 351 | time 4[s] | loss 1.71
| epoch 2 | iter 181 / 351 | time 4[s] | loss 1.71
| epoch 2 | iter 201 / 351 | time 5[s] | loss 1.70
| epoch 2 | iter 221 / 351 | time 5[s] | loss 1.71
| epoch 2 | iter 241 / 351 | time 5[s] | loss 1.70
| epoch 2 | iter 261 / 351 | time 6[s] | loss 1.69
| epoch 2 | iter 281 / 351 | time 6[s] | loss 1.69
| epoch 2 | iter 301 / 351 | time 7[s] | loss 1.69
| epoch 2 | iter 321 / 351 | time 7[s] | loss 1.68
| epoch 2 | iter 341 / 351 | time 8[s] | loss 1.67
Q 77+85
T 162
☒ 994
---
Q 975+164
T 1139
☒ 1000
---
Q 582+84
T 666
☒ 700
---
Q 8+155
T 163
☒ 100
---
Q 367+55
T 422
☒ 400
---
Q 600+257
T 857
☒ 1000
---
Q 761+292
T 1053
☒ 1000
---
Q 830+597
T 1427
☒ 1544
---
Q 26+838
T 864
☒ 400
---
Q 143+93
T 236
☒ 400
---
val acc 0.220%
| epoch 3 | iter 1 / 351 | time 0[s] | loss 1.66
| epoch 3 | iter 21 / 351 | time 0[s] | loss 1.66
| epoch 3 | iter 41 / 351 | time 1[s] | loss 1.65
| epoch 3 | iter 61 / 351 | time 1[s] | loss 1.63
| epoch 3 | iter 81 / 351 | time 1[s] | loss 1.62
| epoch 3 | iter 101 / 351 | time 2[s] | loss 1.62
| epoch 3 | iter 121 / 351 | time 2[s] | loss 1.60
| epoch 3 | iter 141 / 351 | time 3[s] | loss 1.59
| epoch 3 | iter 161 / 351 | time 3[s] | loss 1.57
| epoch 3 | iter 181 / 351 | time 4[s] | loss 1.57
| epoch 3 | iter 201 / 351 | time 4[s] | loss 1.56
| epoch 3 | iter 221 / 351 | time 5[s] | loss 1.54
| epoch 3 | iter 241 / 351 | time 5[s] | loss 1.52
| epoch 3 | iter 261 / 351 | time 6[s] | loss 1.52
| epoch 3 | iter 281 / 351 | time 6[s] | loss 1.52
| epoch 3 | iter 301 / 351 | time 7[s] | loss 1.50
| epoch 3 | iter 321 / 351 | time 7[s] | loss 1.49
| epoch 3 | iter 341 / 351 | time 8[s] | loss 1.48
Q 77+85
T 162
☒ 108
---
Q 975+164
T 1139
☒ 1001
---
Q 582+84
T 666
☒ 648
---
Q 8+155
T 163
☒ 138
---
Q 367+55
T 422
☒ 448
---
Q 600+257
T 857
☒ 848
---
Q 761+292
T 1053
☒ 1011
---
Q 830+597
T 1427
☒ 1373
---
Q 26+838
T 864
☒ 868
---
Q 143+93
T 236
☒ 348
---
val acc 0.560%
| epoch 4 | iter 1 / 351 | time 0[s] | loss 1.47
| epoch 4 | iter 21 / 351 | time 0[s] | loss 1.46
| epoch 4 | iter 41 / 351 | time 1[s] | loss 1.44
| epoch 4 | iter 61 / 351 | time 1[s] | loss 1.43
| epoch 4 | iter 81 / 351 | time 2[s] | loss 1.42
| epoch 4 | iter 101 / 351 | time 2[s] | loss 1.41
| epoch 4 | iter 121 / 351 | time 3[s] | loss 1.40
| epoch 4 | iter 141 / 351 | time 3[s] | loss 1.40
| epoch 4 | iter 161 / 351 | time 4[s] | loss 1.38
| epoch 4 | iter 181 / 351 | time 4[s] | loss 1.38
| epoch 4 | iter 201 / 351 | time 5[s] | loss 1.37
| epoch 4 | iter 221 / 351 | time 5[s] | loss 1.35
| epoch 4 | iter 241 / 351 | time 6[s] | loss 1.33
| epoch 4 | iter 261 / 351 | time 6[s] | loss 1.33
| epoch 4 | iter 281 / 351 | time 7[s] | loss 1.33
| epoch 4 | iter 301 / 351 | time 7[s] | loss 1.32
| epoch 4 | iter 321 / 351 | time 8[s] | loss 1.31
| epoch 4 | iter 341 / 351 | time 8[s] | loss 1.30
Q 77+85
T 162
☒ 146
---
Q 975+164
T 1139
☒ 1189
---
Q 582+84
T 666
☑ 666
---
Q 8+155
T 163
☒ 162
---
Q 367+55
T 422
☒ 432
---
Q 600+257
T 857
☒ 866
---
Q 761+292
T 1053
☒ 1002
---
Q 830+597
T 1427
☒ 1406
---
Q 26+838
T 864
☒ 862
---
Q 143+93
T 236
☒ 202
---
val acc 1.060%
| epoch 5 | iter 1 / 351 | time 0[s] | loss 1.28
| epoch 5 | iter 21 / 351 | time 0[s] | loss 1.29
| epoch 5 | iter 41 / 351 | time 1[s] | loss 1.28
| epoch 5 | iter 61 / 351 | time 1[s] | loss 1.27
| epoch 5 | iter 81 / 351 | time 2[s] | loss 1.27
| epoch 5 | iter 101 / 351 | time 2[s] | loss 1.26
| epoch 5 | iter 121 / 351 | time 3[s] | loss 1.26
| epoch 5 | iter 141 / 351 | time 3[s] | loss 1.27
| epoch 5 | iter 161 / 351 | time 4[s] | loss 1.26
| epoch 5 | iter 181 / 351 | time 4[s] | loss 1.25
| epoch 5 | iter 201 / 351 | time 5[s] | loss 1.23
| epoch 5 | iter 221 / 351 | time 5[s] | loss 1.22
| epoch 5 | iter 241 / 351 | time 6[s] | loss 1.21
| epoch 5 | iter 261 / 351 | time 6[s] | loss 1.21
| epoch 5 | iter 281 / 351 | time 7[s] | loss 1.21
| epoch 5 | iter 301 / 351 | time 7[s] | loss 1.20
| epoch 5 | iter 321 / 351 | time 8[s] | loss 1.19
| epoch 5 | iter 341 / 351 | time 8[s] | loss 1.18
Q 77+85
T 162
☒ 145
---
Q 975+164
T 1139
☒ 1168
---
Q 582+84
T 666
☒ 665
---
Q 8+155
T 163
☒ 192
---
Q 367+55
T 422
☒ 431
---
Q 600+257
T 857
☒ 895
---
Q 761+292
T 1053
☒ 1015
---
Q 830+597
T 1427
☒ 1493
---
Q 26+838
T 864
☒ 891
---
Q 143+93
T 236
☒ 221
---
...
val acc 8.720%
| epoch 25 | iter 1 / 351 | time 0[s] | loss 0.79
| epoch 25 | iter 21 / 351 | time 0[s] | loss 0.80
| epoch 25 | iter 41 / 351 | time 0[s] | loss 0.79
| epoch 25 | iter 61 / 351 | time 1[s] | loss 0.78
| epoch 25 | iter 81 / 351 | time 1[s] | loss 0.79
| epoch 25 | iter 101 / 351 | time 2[s] | loss 0.77
| epoch 25 | iter 121 / 351 | time 2[s] | loss 0.76
| epoch 25 | iter 141 / 351 | time 3[s] | loss 0.78
| epoch 25 | iter 161 / 351 | time 3[s] | loss 0.81
| epoch 25 | iter 181 / 351 | time 4[s] | loss 0.80
| epoch 25 | iter 201 / 351 | time 4[s] | loss 0.78
| epoch 25 | iter 221 / 351 | time 5[s] | loss 0.77
| epoch 25 | iter 241 / 351 | time 5[s] | loss 0.80
| epoch 25 | iter 261 / 351 | time 6[s] | loss 0.85
| epoch 25 | iter 281 / 351 | time 6[s] | loss 0.77
| epoch 25 | iter 301 / 351 | time 7[s] | loss 0.76
| epoch 25 | iter 321 / 351 | time 7[s] | loss 0.77
| epoch 25 | iter 341 / 351 | time 7[s] | loss 0.77
Q 77+85
T 162
☑ 162
---
Q 975+164
T 1139
☒ 1120
---
Q 582+84
T 666
☒ 667
---
Q 8+155
T 163
☒ 162
---
Q 367+55
T 422
☒ 419
---
Q 600+257
T 857
☒ 841
---
Q 761+292
T 1053
☒ 1031
---
Q 830+597
T 1427
☒ 1414
---
Q 26+838
T 864
☒ 858
---
Q 143+93
T 236
☑ 236
---
val acc 8.500%
Traceback (most recent call last):
File "train_seq2seq.py", line 57, in <module>
plt.plot(x, acc_list, marker='o')
File "/opt/conda/lib/python3.6/site-packages/matplotlib/pyplot.py", line 3347, in plot
ax = gca()
File "/opt/conda/lib/python3.6/site-packages/matplotlib/pyplot.py", line 984, in gca
return gcf().gca(**kwargs)
File "/opt/conda/lib/python3.6/site-packages/matplotlib/pyplot.py", line 601, in gcf
return figure()
File "/opt/conda/lib/python3.6/site-packages/matplotlib/pyplot.py", line 548, in figure
**kwargs)
File "/opt/conda/lib/python3.6/site-packages/matplotlib/backend_bases.py", line 161, in new_figure_manager
return cls.new_figure_manager_given_figure(num, fig)
File "/opt/conda/lib/python3.6/site-packages/matplotlib/backend_bases.py", line 167, in new_figure_manager_given_figure
canvas = cls.FigureCanvas(figure)
File "/opt/conda/lib/python3.6/site-packages/matplotlib/backends/backend_qt5agg.py", line 24, in __init__
super(FigureCanvasQTAgg, self).__init__(figure=figure)
File "/opt/conda/lib/python3.6/site-packages/matplotlib/backends/backend_qt5.py", line 234, in __init__
_create_qApp()
File "/opt/conda/lib/python3.6/site-packages/matplotlib/backends/backend_qt5.py", line 125, in _create_qApp
raise RuntimeError('Invalid DISPLAY variable')
RuntimeError: Invalid DISPLAY variable
# cp train_seq2seq.py rev_seq2seq.py
# vi rev_seq2seq.py
rev_seq2seq.py
...
# Reverse input? =================================================
#is_reverse = False # True
is_reverse = True
if is_reverse:
x_train, x_test = x_train[:, ::-1], x_test[:, ::-1]
# ================================================================
#python rev_seq2seq.py
...
val acc 51.780%
| epoch 25 | iter 1 / 351 | time 0[s] | loss 0.29
| epoch 25 | iter 21 / 351 | time 0[s] | loss 0.29
| epoch 25 | iter 41 / 351 | time 0[s] | loss 0.28
| epoch 25 | iter 61 / 351 | time 1[s] | loss 0.26
| epoch 25 | iter 81 / 351 | time 1[s] | loss 0.26
| epoch 25 | iter 101 / 351 | time 2[s] | loss 0.27
| epoch 25 | iter 121 / 351 | time 2[s] | loss 0.29
| epoch 25 | iter 141 / 351 | time 3[s] | loss 0.28
| epoch 25 | iter 161 / 351 | time 3[s] | loss 0.28
| epoch 25 | iter 181 / 351 | time 4[s] | loss 0.28
| epoch 25 | iter 201 / 351 | time 4[s] | loss 0.27
| epoch 25 | iter 221 / 351 | time 5[s] | loss 0.29
| epoch 25 | iter 241 / 351 | time 5[s] | loss 0.27
| epoch 25 | iter 261 / 351 | time 6[s] | loss 0.28
| epoch 25 | iter 281 / 351 | time 6[s] | loss 0.28
| epoch 25 | iter 301 / 351 | time 7[s] | loss 0.27
| epoch 25 | iter 321 / 351 | time 7[s] | loss 0.28
| epoch 25 | iter 341 / 351 | time 7[s] | loss 0.28
Q 77+85
T 162
☑ 162
---
Q 975+164
T 1139
☒ 1140
---
Q 582+84
T 666
☑ 666
---
Q 8+155
T 163
☑ 163
---
Q 367+55
T 422
☑ 422
---
Q 600+257
T 857
☒ 856
---
Q 761+292
T 1053
☒ 1052
---
Q 830+597
T 1427
☒ 1426
---
Q 26+838
T 864
☑ 864
---
Q 143+93
T 236
☑ 236
---
val acc 54.100%
ファイルをコピーしてpeeky対応に変更
# cp rev_seq2seq.py rev_seq2seq.py
# vi peeky2_seq2seq.py
peeky2_seq2seq.py
...
# Reverse input? =================================================
#is_reverse = False # True
is_reverse = True
if is_reverse:
x_train, x_test = x_train[:, ::-1], x_test[:, ::-1]
# ================================================================
# ハイパーパラメータの設定
vocab_size = len(char_to_id)
wordvec_size = 16
hideen_size = 128
batch_size = 128
max_epoch = 25
max_grad = 5.0
# Normal or Peeky? ==============================================
# model = Seq2seq(vocab_size, wordvec_size, hideen_size)
model = PeekySeq2seq(vocab_size, wordvec_size, hideen_size)
# ================================================================
実行
# python peeky2_seq2seq.py
...
val acc 76.480%
| epoch 25 | iter 1 / 351 | time 0[s] | loss 0.14
| epoch 25 | iter 21 / 351 | time 0[s] | loss 0.15
| epoch 25 | iter 41 / 351 | time 1[s] | loss 0.15
| epoch 25 | iter 61 / 351 | time 1[s] | loss 0.14
| epoch 25 | iter 81 / 351 | time 2[s] | loss 0.14
| epoch 25 | iter 101 / 351 | time 2[s] | loss 0.14
| epoch 25 | iter 121 / 351 | time 3[s] | loss 0.14
| epoch 25 | iter 141 / 351 | time 4[s] | loss 0.14
| epoch 25 | iter 161 / 351 | time 4[s] | loss 0.13
| epoch 25 | iter 181 / 351 | time 5[s] | loss 0.14
| epoch 25 | iter 201 / 351 | time 5[s] | loss 0.15
| epoch 25 | iter 221 / 351 | time 6[s] | loss 0.14
| epoch 25 | iter 241 / 351 | time 6[s] | loss 0.14
| epoch 25 | iter 261 / 351 | time 7[s] | loss 0.14
| epoch 25 | iter 281 / 351 | time 8[s] | loss 0.14
| epoch 25 | iter 301 / 351 | time 8[s] | loss 0.13
| epoch 25 | iter 321 / 351 | time 9[s] | loss 0.13
| epoch 25 | iter 341 / 351 | time 9[s] | loss 0.12
Q 77+85
T 162
☑ 162
---
Q 975+164
T 1139
☑ 1139
---
Q 582+84
T 666
☑ 666
---
Q 8+155
T 163
☑ 163
---
Q 367+55
T 422
☑ 422
---
Q 600+257
T 857
☑ 857
---
Q 761+292
T 1053
☑ 1053
---
Q 830+597
T 1427
☒ 1430
---
Q 26+838
T 864
☑ 864
---
Q 143+93
T 236
☑ 236
---
val acc 81.560%
文書履歴 (document history)
ver. 0.01 初稿 20190221