0.0 前提
まずはこちらの記事を読んでください. $^1$
追記：URL間違ってました、本当にごめんなさい
0.1 この記事の所感
95%というのはまぁまぁの精度に見えますが、コメントを見る限りどうやらリーケージを起こしているようなので今回はリーケージを回避しつつ可能な限り同じ環境下で検証を行っていきたいと思います。
0.2 筆者の環境
自前PC:
OS: Windows 10
CPU:AMD Ryzen 7 4700U
Python:3.9.0
Google Colab:略
1.1 データセットの準備
まずは画像を集めますが、その過程は省略します。
今回はAIイラストを1000枚、非AIイラストを1000枚集めました。AIイラストはいずれもNovelAI産であり、今回はこの2000枚のうち1000枚をランダム抽出し、学習及びテストデータとします。
1.2 前処理
$[0.0,1.0]$に正規化、画像のサイズをOpenCVを用いて
$(256,256)$に拡大縮小$^2$。このサイズの理由は単純にGoogleColabの都合$^3$上です。
その後ラベルデータを作成、以降をTensorflowに投げます。
2.1 学習モデル
今回用いるモデルは先行研究の記事にあるAutoMLがデフォルトで用いてるResNet50V1、ではなくV2を用いました。理由は特にありません。
最適化アルゴリズムはとりあえずAdam、損失関数は交差エントロピーを用いました。
optimizer = keras.optimizers.Adam(lr=1e-4)
loss_object = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)
model.compile(loss='sparse_categorical_crossentropy',
optimizer=optimizer,
metrics=['accuracy'])
2.2 学習
バッチサイズは32,エポック数は50×2で行います。
過学習はとりあえず気にせず、Best Test Accuracyのモデルデータを保存する形にします。
3.1 学習結果
(ここ長いので注意)
一回目
Epoch 1, Loss: 0.7157273888587952, Accuracy: 71.0, Test Loss: 1426.1092529296875, Test Accuracy: 48.0
Epoch 2, Loss: 0.49907082319259644, Accuracy: 78.77777862548828, Test Loss: 217.48583984375, Test Accuracy: 50.0
Epoch 3, Loss: 0.5669065713882446, Accuracy: 73.22222137451172, Test Loss: 1.4339460134506226, Test Accuracy: 49.0
Epoch 4, Loss: 0.5255488753318787, Accuracy: 77.88888549804688, Test Loss: 0.6887185573577881, Test Accuracy: 55.0
Epoch 5, Loss: 0.46992945671081543, Accuracy: 79.0, Test Loss: 3.342252254486084, Test Accuracy: 51.0
Epoch 6, Loss: 0.46344128251075745, Accuracy: 80.0, Test Loss: 0.6161324977874756, Test Accuracy: 69.0
Epoch 7, Loss: 0.41955700516700745, Accuracy: 82.66666412353516, Test Loss: 0.7777687311172485, Test Accuracy: 63.0
Epoch 8, Loss: 0.44990402460098267, Accuracy: 80.33333587646484, Test Loss: 1.082552194595337, Test Accuracy: 70.0
Epoch 9, Loss: 0.3702699542045593, Accuracy: 84.66667175292969, Test Loss: 0.608517587184906, Test Accuracy: 63.0
Epoch 10, Loss: 0.3437049090862274, Accuracy: 87.33333587646484, Test Loss: 10.738988876342773, Test Accuracy: 50.0
Epoch 11, Loss: 0.312382310628891, Accuracy: 87.8888931274414, Test Loss: 1.4091475009918213, Test Accuracy: 51.0
Epoch 12, Loss: 0.3859584331512451, Accuracy: 85.0, Test Loss: 1.5489251613616943, Test Accuracy: 52.0
Epoch 13, Loss: 0.4053536057472229, Accuracy: 83.22222137451172, Test Loss: 1.409684658050537, Test Accuracy: 50.0
Epoch 14, Loss: 0.4536645710468292, Accuracy: 78.66667175292969, Test Loss: 1.0026456117630005, Test Accuracy: 52.999996185302734
Epoch 15, Loss: 0.3107542097568512, Accuracy: 86.1111068725586, Test Loss: 13.516963005065918, Test Accuracy: 48.0
Epoch 16, Loss: 0.3484424352645874, Accuracy: 84.77777862548828, Test Loss: 0.7037116289138794, Test Accuracy: 68.0
Epoch 17, Loss: 0.32275402545928955, Accuracy: 85.11111450195312, Test Loss: 1.6125630140304565, Test Accuracy: 56.0
Epoch 18, Loss: 0.31793472170829773, Accuracy: 87.55555725097656, Test Loss: 2.1533820629119873, Test Accuracy: 49.0
Epoch 19, Loss: 0.2902267277240753, Accuracy: 88.22222137451172, Test Loss: 1.2860796451568604, Test Accuracy: 56.0
Epoch 20, Loss: 0.21393410861492157, Accuracy: 91.11111450195312, Test Loss: 1.4367811679840088, Test Accuracy: 56.0
Epoch 21, Loss: 0.21021857857704163, Accuracy: 91.33333587646484, Test Loss: 1.758449912071228, Test Accuracy: 52.999996185302734
Epoch 22, Loss: 0.18354931473731995, Accuracy: 92.11111450195312, Test Loss: 2.014467716217041, Test Accuracy: 56.0
Epoch 23, Loss: 0.2113305926322937, Accuracy: 91.44444274902344, Test Loss: 6.90259313583374, Test Accuracy: 52.0
Epoch 24, Loss: 0.18431633710861206, Accuracy: 92.11111450195312, Test Loss: 54.218143463134766, Test Accuracy: 51.0
Epoch 25, Loss: 0.1335298866033554, Accuracy: 95.11111450195312, Test Loss: 3.9828073978424072, Test Accuracy: 51.0
Epoch 26, Loss: 0.15646666288375854, Accuracy: 95.22222137451172, Test Loss: 4.31303071975708, Test Accuracy: 64.0
Epoch 27, Loss: 0.23114003241062164, Accuracy: 90.22222137451172, Test Loss: 1.9974653720855713, Test Accuracy: 56.0
Epoch 28, Loss: 0.16292358934879303, Accuracy: 95.22222137451172, Test Loss: 1.298332929611206, Test Accuracy: 71.0
Epoch 29, Loss: 0.3076359033584595, Accuracy: 89.44444274902344, Test Loss: 125.18321228027344, Test Accuracy: 48.0
Epoch 30, Loss: 0.1999155879020691, Accuracy: 93.55555725097656, Test Loss: 38.11824417114258, Test Accuracy: 52.0
Epoch 31, Loss: 0.21001307666301727, Accuracy: 91.0, Test Loss: 22.305002212524414, Test Accuracy: 52.0
Epoch 32, Loss: 0.11487580090761185, Accuracy: 96.0, Test Loss: 0.6619657278060913, Test Accuracy: 78.0
Epoch 33, Loss: 0.09758392721414566, Accuracy: 97.0, Test Loss: 2.265392303466797, Test Accuracy: 60.000003814697266
Epoch 34, Loss: 0.09574073553085327, Accuracy: 96.55555725097656, Test Loss: 9.234281539916992, Test Accuracy: 52.0
Epoch 35, Loss: 0.049194104969501495, Accuracy: 98.44444274902344, Test Loss: 1.1101315021514893, Test Accuracy: 67.0
Epoch 36, Loss: 0.08387941122055054, Accuracy: 97.66667175292969, Test Loss: 3.426330804824829, Test Accuracy: 51.0
Epoch 37, Loss: 0.08278490602970123, Accuracy: 98.66667175292969, Test Loss: 0.9813934564590454, Test Accuracy: 74.0
Epoch 38, Loss: 0.32420089840888977, Accuracy: 88.55555725097656, Test Loss: 1.3521316051483154, Test Accuracy: 62.0
Epoch 39, Loss: 0.14531110227108002, Accuracy: 94.1111068725586, Test Loss: 7.018132209777832, Test Accuracy: 49.0
Epoch 40, Loss: 0.0942535400390625, Accuracy: 96.66666412353516, Test Loss: 1.7162564992904663, Test Accuracy: 65.0
Epoch 41, Loss: 0.07604198157787323, Accuracy: 97.33333587646484, Test Loss: 1.1039272546768188, Test Accuracy: 72.0
Epoch 42, Loss: 0.0318535678088665, Accuracy: 99.1111068725586, Test Loss: 1.7948288917541504, Test Accuracy: 63.0
Epoch 43, Loss: 0.04657680541276932, Accuracy: 98.33333587646484, Test Loss: 3.5491998195648193, Test Accuracy: 56.0
Epoch 44, Loss: 0.07854939997196198, Accuracy: 97.66667175292969, Test Loss: 2.7407844066619873, Test Accuracy: 57.0
Epoch 45, Loss: 0.1404239386320114, Accuracy: 95.44445037841797, Test Loss: 0.8437129259109497, Test Accuracy: 78.0
Epoch 46, Loss: 0.060639187693595886, Accuracy: 98.33333587646484, Test Loss: 4.828827857971191, Test Accuracy: 52.0
Epoch 47, Loss: 0.052321385592222214, Accuracy: 98.0, Test Loss: 2.869962692260742, Test Accuracy: 58.0
Epoch 48, Loss: 0.11068885028362274, Accuracy: 97.0, Test Loss: 1.3316360712051392, Test Accuracy: 73.0
Epoch 49, Loss: 0.15208984911441803, Accuracy: 93.77777862548828, Test Loss: 4.048267364501953, Test Accuracy: 63.0
Epoch 50, Loss: 0.11603520810604095, Accuracy: 95.66666412353516, Test Loss: 4.944867134094238, Test Accuracy: 62.0
二回目
Epoch 1, Loss: 0.04397699609398842, Accuracy: 98.22222137451172, Test Loss: 2.740225315093994, Test Accuracy: 58.0
Epoch 2, Loss: 0.03070162422955036, Accuracy: 99.55555725097656, Test Loss: 1.8769280910491943, Test Accuracy: 65.0
Epoch 3, Loss: 0.3112526834011078, Accuracy: 91.44444274902344, Test Loss: 36.306434631347656, Test Accuracy: 52.0
Epoch 4, Loss: 0.2797890901565552, Accuracy: 88.22222137451172, Test Loss: 1.7372690439224243, Test Accuracy: 46.0
Epoch 5, Loss: 0.08925849199295044, Accuracy: 97.44444274902344, Test Loss: 2.0602898597717285, Test Accuracy: 58.999996185302734
Epoch 6, Loss: 0.09523572027683258, Accuracy: 96.55555725097656, Test Loss: 0.9280775785446167, Test Accuracy: 76.0
Epoch 7, Loss: 0.13290512561798096, Accuracy: 96.66666412353516, Test Loss: 3.8039588928222656, Test Accuracy: 52.999996185302734
Epoch 8, Loss: 0.17789818346500397, Accuracy: 93.8888931274414, Test Loss: 1.9895827770233154, Test Accuracy: 60.000003814697266
Epoch 9, Loss: 0.11989210546016693, Accuracy: 95.88888549804688, Test Loss: 1.743853211402893, Test Accuracy: 63.0
Epoch 10, Loss: 0.08432938158512115, Accuracy: 99.1111068725586, Test Loss: 1.4307432174682617, Test Accuracy: 73.0
Epoch 11, Loss: 0.17306260764598846, Accuracy: 95.22222137451172, Test Loss: 1.7963205575942993, Test Accuracy: 58.0
Epoch 12, Loss: 0.08047390729188919, Accuracy: 97.44444274902344, Test Loss: 3.389617919921875, Test Accuracy: 52.0
Epoch 13, Loss: 0.04342600330710411, Accuracy: 99.33333587646484, Test Loss: 1.1987446546554565, Test Accuracy: 69.0
Epoch 14, Loss: 0.0732155442237854, Accuracy: 97.0, Test Loss: 2.213702917098999, Test Accuracy: 61.0
Epoch 15, Loss: 0.031067024916410446, Accuracy: 98.77777862548828, Test Loss: 0.8578950762748718, Test Accuracy: 78.0
Epoch 16, Loss: 0.07684523612260818, Accuracy: 99.33333587646484, Test Loss: 1.3886973857879639, Test Accuracy: 75.0
Epoch 17, Loss: 0.14652517437934875, Accuracy: 95.55555725097656, Test Loss: 0.9708963632583618, Test Accuracy: 77.0
Epoch 18, Loss: 0.07171519845724106, Accuracy: 98.77777862548828, Test Loss: 1.0008617639541626, Test Accuracy: 78.0
Epoch 19, Loss: 0.09542503952980042, Accuracy: 96.66666412353516, Test Loss: 1.1017062664031982, Test Accuracy: 71.0
Epoch 20, Loss: 0.02457316778600216, Accuracy: 99.77777862548828, Test Loss: 1.167754054069519, Test Accuracy: 73.0
Epoch 21, Loss: 0.011691317893564701, Accuracy: 99.8888931274414, Test Loss: 0.9585479497909546, Test Accuracy: 71.0
Epoch 22, Loss: 0.04506567493081093, Accuracy: 99.8888931274414, Test Loss: 0.7130100131034851, Test Accuracy: 81.0
Epoch 23, Loss: 0.16665779054164886, Accuracy: 93.55555725097656, Test Loss: 3.7309446334838867, Test Accuracy: 52.0
Epoch 24, Loss: 0.08500046283006668, Accuracy: 97.0, Test Loss: 3.3733842372894287, Test Accuracy: 54.000003814697266
Epoch 25, Loss: 0.03525979816913605, Accuracy: 99.66666412353516, Test Loss: 4.644826889038086, Test Accuracy: 52.0
Epoch 26, Loss: 0.05835489556193352, Accuracy: 98.11111450195312, Test Loss: 5.652350902557373, Test Accuracy: 54.000003814697266
Epoch 27, Loss: 0.034206293523311615, Accuracy: 99.44444274902344, Test Loss: 6.8744354248046875, Test Accuracy: 51.0
Epoch 28, Loss: 0.1467190980911255, Accuracy: 96.33332824707031, Test Loss: 17.84250259399414, Test Accuracy: 51.0
Epoch 29, Loss: 0.19621680676937103, Accuracy: 92.44444274902344, Test Loss: 2.6562626361846924, Test Accuracy: 54.000003814697266
Epoch 30, Loss: 0.05551134794950485, Accuracy: 98.44444274902344, Test Loss: 1.0437568426132202, Test Accuracy: 76.0
Epoch 31, Loss: 0.016516171395778656, Accuracy: 99.55555725097656, Test Loss: 0.7771217226982117, Test Accuracy: 76.0
Epoch 32, Loss: 0.019318904727697372, Accuracy: 99.8888931274414, Test Loss: 0.8954141139984131, Test Accuracy: 80.0
Epoch 33, Loss: 0.04180620610713959, Accuracy: 98.44444274902344, Test Loss: 2.114068031311035, Test Accuracy: 62.0
Epoch 34, Loss: 0.03431808203458786, Accuracy: 98.77777862548828, Test Loss: 1.140421748161316, Test Accuracy: 76.0
Epoch 35, Loss: 0.1008048951625824, Accuracy: 96.11111450195312, Test Loss: 0.893524169921875, Test Accuracy: 77.0
Epoch 36, Loss: 0.03103010356426239, Accuracy: 98.77777862548828, Test Loss: 0.98885178565979, Test Accuracy: 82.0
Epoch 37, Loss: 0.010856691747903824, Accuracy: 99.66666412353516, Test Loss: 0.9545056819915771, Test Accuracy: 80.0
Epoch 38, Loss: 0.012889016419649124, Accuracy: 99.55555725097656, Test Loss: 0.7835126519203186, Test Accuracy: 83.0
Epoch 39, Loss: 0.01777203194797039, Accuracy: 99.22222137451172, Test Loss: 0.960686445236206, Test Accuracy: 80.0
Epoch 40, Loss: 0.01426771841943264, Accuracy: 99.66666412353516, Test Loss: 0.8545937538146973, Test Accuracy: 82.0
Epoch 41, Loss: 0.03507606312632561, Accuracy: 98.44444274902344, Test Loss: 1.3454233407974243, Test Accuracy: 70.0
Epoch 42, Loss: 0.017473625019192696, Accuracy: 99.66666412353516, Test Loss: 0.9965317249298096, Test Accuracy: 81.0
Epoch 43, Loss: 0.10058202594518661, Accuracy: 96.88888549804688, Test Loss: 0.9679105281829834, Test Accuracy: 74.0
Epoch 44, Loss: 0.07124639302492142, Accuracy: 97.22222137451172, Test Loss: 1.296515703201294, Test Accuracy: 76.0
Epoch 45, Loss: 0.09724083542823792, Accuracy: 99.0, Test Loss: 0.7879502773284912, Test Accuracy: 78.0
Epoch 46, Loss: 0.12031930685043335, Accuracy: 96.44444274902344, Test Loss: 3.9726345539093018, Test Accuracy: 52.0
Epoch 47, Loss: 0.06404001265764236, Accuracy: 97.88888549804688, Test Loss: 3.5960469245910645, Test Accuracy: 52.0
Epoch 48, Loss: 0.03390665352344513, Accuracy: 99.22222137451172, Test Loss: 3.9181220531463623, Test Accuracy: 52.999996185302734
Epoch 49, Loss: 0.040301524102687836, Accuracy: 98.66667175292969, Test Loss: 3.3265697956085205, Test Accuracy: 62.0
Epoch 50, Loss: 0.019738871604204178, Accuracy: 99.55555725097656, Test Loss: 2.264172077178955, Test Accuracy: 65.0
Epoch=50+38 回目がBest Test Accuracyを出してます。
3.2 学習曲線
以下すべてepoch<=50となります
オレンジがTest Loss,青がTrain lossになります
わ、わからないッピ。
なのでepoch>5で範囲を指定すると
わ、わからないッピ
仕方ないので分離します。
Test loss
Train loss
わからないッピ
$^4$
Test lossはグラフだけだと意味わからないです。過学習してるかどうかもわかりません。
Train lossは見る限りかなり乱高下していますが、一応学習してるように見えます。ただし、Accuracyを見る限り過学習してる可能性が非常に高いです。
3.3 予測結果
学習時に用いたテストデータと同じテストデータを用いた結果
#print(tp,tn,fp,fn)
41 42 10 7
4.1 検証終えた所感
色々雑にやった箇所もあるので、もう少しまじめにやれば精度は向上するかもしれない
$\to$95%という数値はこの条件下では達成可能かもしれない。
ただどう考えてもデータセットが少なすぎる
学習における大部分はTrain Dataに対して過学習してる上で、その中でたまたまTest Dataに合致したモデルが出てきたようにしか思えない。
4.2 今後の展望
・学習曲線がカオスすぎてちゃんと学習できてるか不安すぎる
・検証なのだからResNet50V1使え
・それはそれとしてほかのモデルも試してみたい
・データセットの前処理を別ので試してみたい$^2$
・データセットの数増やしたい
・交差検証してください
・データセットの吟味
・一周しかできていないので不確かさなどを考慮するためにあと数回やりたい
5.1 引用注
$^1:$無許可です、問題あれば消します。
$^2:$例えば$(256\times 256)$を超えるサイズの画像に対しては縮小するのではなくトリミングする等
$^3:$無料範囲で利用可能なメモリー超えてしまうので苦肉の策になります。
$^4:$一応y軸の範囲を指定することでそれぽいものが見れるかもしれないが、多分結果は変わらない。