【Elixir/Phoenix】AI対戦4目並べを作る（強化学習編）では、強化学習によってAIの対戦相手のQ関数をAxonのニューラルネットワークで作成しました。
この動作をlivebookを使って、確認してみます。
Livebookの環境がある方は、livebookでみてください。
動作させるには、Q関数のパラメータを読み込ませる必要があります。
以下のファイルをダウンロードして、livebookのq_net=の行にファイル保存先の絶対パスを記載してください。
Livebookの内容まとめ
五目並べの動作説明
Gomokuモジュール
newを実行すると、なにも石の置かれていない初期状態のボードの%Gomoku{}を返します。
Gomoku.new(5, 4)
%Gomoku{
board: %{
{0, 0} => 0,
{0, 1} => 0,
{0, 2} => 0,
{0, 3} => 0,
{0, 4} => 0,
{1, 0} => 0,
{1, 1} => 0,
{1, 2} => 0,
{1, 3} => 0,
{1, 4} => 0,
{2, 0} => 0,
{2, 1} => 0,
{2, 2} => 0,
{2, 3} => 0,
{2, 4} => 0,
{3, 0} => 0,
{3, 1} => 0,
{3, 2} => 0,
{3, 3} => 0,
{3, 4} => 0,
{4, 0} => 0,
{4, 1} => 0,
{4, 2} => 0,
{4, 3} => 0,
{4, 4} => 0
},
size: 5,
turn: 1,
done: false,
draw: false,
miss: false,
k: 4
}
put!()を使って石を置く操作ができます。(1,1)に白（先手）と(2,2)に黒を置いた状態を作ってみます。
render()を使うとテキストで盤面が表示されます。
Gomoku.new(5, 4)
|> Gomoku.put!({1, 1})
|> Gomoku.put!({2, 2})
|> Gomoku.render()
|> IO.puts()
abcde
1: -----
2: -○---
3: --●--
4: -----
5: -----
:ok
先手が(1,4)に置いた時点で、勝った状態です。
勝つと、done=trueとなります。
これ以上石を置けなくなります。
gomoku =
Gomoku.new(5, 4)
|> Gomoku.put!({1, 1})
|> Gomoku.put!({3, 1})
|> Gomoku.put!({1, 2})
|> Gomoku.put!({3, 2})
|> Gomoku.put!({1, 3})
|> Gomoku.put!({3, 3})
|> Gomoku.put!({1, 4})
|> tap(fn x -> Gomoku.render(x) |> IO.puts() end)
IO.inspect(gomoku.done, label: "done=")
IO.inspect(gomoku.turn, label: "turn=")
abcde
1: -----
2: -○○○○
3: -----
4: -●●●-
5: -----
done=: true
turn=: 1
1
AI対戦相手の動作確認
gomoku-qnet-5x5-4-200k.axonは、Axonで作成したニューラルネットワークを強化学習を行い、が学習が完了した時点の状態をファイルに保存したものです。
これを読み込みます。
q_netは、盤面の状態を入力に与えると、置く場所ごとの評価値を返すものです。
# 将来的にlivebookのFILESでアップロードしたファイルを参照できるようになれば使えるかも？
# q_net =
# Kino.FS.file_path("gomoku-qnet-5x5-4-200k.axon")
# |> QNet.load_model_state()
# gomoku-qnet-5x5-4-200k.axonの保存場所を絶対パスで指定する
q_net = QNet.load_model_state("/home/masa/livebook/gomoku_livemd/gomoku-qnet-5x5-4-200k.axon")
%QNet{
step_state: %{
model_state: %{
"conv_0" => %{
"bias" => #Nx.Tensor<
f32[1024]
EXLA.Backend<host:0, 0.768359907.414842904.60919>
[-6.865009781904519e-4, 2.6975880609825253e-4, -4.27446560934186e-4, 2.8637019568122923e-4, 2.4668051628395915e-4, -2.71380296908319e-4, 0.001203116844408214, -3.8025336107239127e-4, 4.3718135566450655e-4, -2.3740943288430572e-4, 8.888402953743935e-4, 2.596177509985864e-4, 6.559949251823127e-4, 1.3511056022252887e-4, 4.2229934479109943e-4, -4.8138288548216224e-5, -4.796475695911795e-4, 7.190658943727612e-4, 1.5164012438617647e-4, -6.91958237439394e-4, -3.6078502307645977e-4, 4.10037231631577e-4, 8.169953362084925e-4, 4.595319114741869e-5, 3.500354432617314e-5, 3.06337809888646e-4, -5.300036718836054e-5, 1.5694304602220654e-4, -1.8838050891645253e-4, 7.786192581988871e-4, -5.060079274699092e-4, 3.490550152491778e-4, 1.335590350208804e-4, -6.98409101460129e-4, 1.718695421004668e-4, 1.6559856885578483e-4, 2.165897749364376e-4, 5.708548269467428e-5, -1.666508978814818e-5, -0.0010185283608734608, -6.320475949905813e-5, -2.2417341824620962e-4, 3.461632295511663e-4, 6.992384442128241e-4, 5.098163383081555e-4, 4.032696015201509e-4, ...]
>,
"kernel" => #Nx.Tensor<
f32[1024][2][3][3]
EXLA.Backend<host:0, 0.768359907.414842896.60485>
[
[
[
[-6.453375099226832e-4, 5.136739346198738e-4, -1.6104470705613494e-4],
[-8.737275493331254e-4, -8.622565655969083e-4, 0.0011200233129784465],
[-5.521141574718058e-4, -2.3767590755596757e-4, -2.533512088120915e-5]
],
[
[-7.190259639173746e-4, 9.655762551119551e-5, -2.1242990624159575e-4],
[1.6510917339473963e-4, -7.705824100412428e-4, 6.572505226358771e-4],
[-3.1077436869964004e-4, -3.3675116719678044e-4, -1.8666754476726055e-4]
]
],
[
[
[-4.334681434556842e-4, -1.4176234253682196e-4, -3.442674351390451e-4],
[-2.3966465960256755e-4, 4.2674082214944065e-4, -2.0280502212699503e-4],
[-3.3631979022175074e-4, 2.634760458022356e-4, -2.3665478511247784e-4]
],
[
[-2.987074258271605e-4, -3.439004358369857e-4, 2.1668054614565335e-5],
[-1.4281825860962272e-4, 2.5652762269601226e-4, -2.6817002799361944e-4],
[-2.86101276287809e-5, 6.803639116697013e-4, 1.2451678048819304e-4]
]
],
[
[
[-3.9591972017660737e-4, -2.7315382612869143e-4, 4.1988671000581235e-5],
[-5.948112229816616e-4, -4.118077631574124e-4, 5.055955843999982e-4],
[7.528160349465907e-4, 2.3815837266738527e-5, 1.0651882621459663e-4]
],
...
],
...
]
>
},
"conv_1" => %{
"bias" => #Nx.Tensor<
f32[1024]
EXLA.Backend<host:0, 0.768359907.414842904.60921>
[8.683644700795412e-4, 0.001804275787435472, 1.0674787336029112e-4, 6.011570221744478e-4, -8.705105283297598e-4, 1.6934289305936545e-4, 8.451482863165438e-4, -0.0023760783951729536, -3.5029411083087325e-4, -7.284199236892164e-4, 0.0011869489680975676, 5.100449197925627e-5, -2.7308124117553234e-4, 3.460573498159647e-4, -7.781445165164769e-4, 0.035786911845207214, 9.787371382117271e-4, 0.0010845151264220476, -2.9858005291316658e-5, -0.01174168474972248, 9.264206164516509e-5, -7.528800051659346e-4, 2.595106780063361e-4, -8.286238153232262e-5, 0.003002566983923316, -4.64680721051991e-4, 8.265592041425407e-4, 9.130724938586354e-4, 9.03524924069643e-4, -5.513489595614374e-4, 4.2970041977241635e-4, -5.077148671261966e-4, 0.020045168697834015, 8.76010803040117e-4, -0.0013982179807499051, 2.8561436920426786e-4, 0.0011817713966593146, 0.0020339267794042826, 3.9535772521048784e-4, 6.836307002231479e-4, 0.0016378742875531316, -8.9845439651981e-4, -4.4627235183725134e-5, 9.538605809211731e-4, 4.3758025276474655e-4, ...]
>,
"kernel" => #Nx.Tensor<
f32[1024][1024][2][2]
EXLA.Backend<host:0, 0.768359907.414842896.60486>
[
[
[
[-0.008320383727550507, -0.0038595516234636307],
[0.020770065486431122, 0.007943713106215]
],
[
[0.003305597696453333, -0.0066918181255459785],
[-0.007257736753672361, 0.00754633778706193]
],
[
[-0.001249559223651886, -0.0037352496292442083],
[0.013622534461319447, -0.008761556819081306]
],
[
[0.010119768790900707, -0.013311078771948814],
[0.0037256511859595776, -0.0034604857210069895]
],
[
[-8.684721979079768e-5, -0.0014562765136361122],
[-4.8771718866191804e-4, 0.002147988649085164]
],
[
[0.0077948179095983505, 0.009032581932842731],
[-0.004804715979844332, -0.01107831671833992]
],
[
[-0.0028885575011372566, 0.0054298993200063705],
[-0.010713104158639908, 0.010669575072824955]
],
[
[-0.002680492354556918, -6.865154136903584e-4],
[-0.003934227861464024, 0.016834920272231102]
],
[
[0.01549330074340105, 0.010792897082865238],
[-0.0028085713274776936, 0.0011950198095291853]
],
[
[-0.0027078629937022924, 0.01374782808125019],
[0.0046119303442537785, -2.4119541922118515e-4]
],
[
[-0.005815272685140371, -0.004150833003222942],
[0.004684442654252052, 0.015126642771065235]
],
...
],
...
]
>
},
"conv_2" => %{
"bias" => #Nx.Tensor<
f32[1024]
EXLA.Backend<host:0, 0.768359907.414842904.60923>
[0.20426452159881592, -0.08040864765644073, -0.026650741696357727, -0.09082607924938202, -0.6581783294677734, 0.4342668950557709, -0.046322330832481384, -0.04729534313082695, 0.23745515942573547, -0.01742425374686718, 0.3131365180015564, -0.24050799012184143, -0.29049864411354065, 0.07995155453681946, -0.020527416840195656, 0.34410396218299866, -0.2783336639404297, -0.5348480939865112, -0.004993646405637264, -0.03742041811347008, 0.11218204349279404, 0.8373739123344421, 9.056683629751205e-4, -0.190827876329422, 0.07124796509742737, -0.14236895740032196, -0.48845019936561584, 0.03738477826118469, -0.08130612969398499, -0.5372570157051086, -0.05493773892521858, 0.07042037695646286, -0.16134409606456757, -0.07172698527574539, -0.01851494424045086, 0.0030176907312124968, -0.026425471529364586, -0.17772817611694336, -0.11034564673900604, -0.20054976642131805, 0.061634283512830734, -0.28646138310432434, -0.17268642783164978, 0.06671646237373352, ...]
>,
"kernel" => #Nx.Tensor<
f32[1024][1024][2][2]
EXLA.Backend<host:0, 0.768359907.414842896.60487>
[
[
[
[-0.026130622252821922, 0.026978498324751854],
[-2.7143224724568427e-4, -0.012676007114350796]
],
[
[-0.010834332555532455, 0.021714529022574425],
[-0.003949263133108616, -0.03677662834525108]
],
[
[-0.03372590243816376, 0.003968650009483099],
[-0.004475691355764866, -0.0057575334794819355]
],
[
[0.03218585625290871, 0.03719136491417885],
[0.017435654997825623, 0.02838548831641674]
],
[
[-4.891125136055052e-4, -0.030359407886862755],
[-0.027995917946100235, -0.0140770198777318]
],
[
[0.024753862991929054, 0.016965674236416817],
[-0.0019283657893538475, 0.007043934892863035]
],
[
[0.002658985322341323, 0.020549768581986427],
[-0.0049821119755506516, 0.007399563677608967]
],
[
[-0.0020071903709322214, -0.0029949359595775604],
[0.0413161925971508, -0.005580292083323002]
],
[
[-0.018524164333939552, -0.05770149454474449],
[0.020264161750674248, 4.464320663828403e-4]
],
[
[0.010185417719185352, -0.01833392307162285],
[-0.008595358580350876, 0.004408789798617363]
],
[
[-0.01904064230620861, -0.006703596096485853],
[-0.014089282602071762, ...]
],
...
],
...
]
>
},
"dense_0" => %{
"bias" => #Nx.Tensor<
f32[1024]
EXLA.Backend<host:0, 0.768359907.414842904.60925>
[-0.031154267489910126, 0.033644743263721466, -0.17443379759788513, 0.08570174127817154, -0.2143303006887436, -0.21706104278564453, -0.2039143592119217, -0.14886517822742462, -0.2127504199743271, -0.19501951336860657, -0.23268118500709534, -0.04660746827721596, -0.23270364105701447, -0.20262539386749268, -0.24665553867816925, -0.15771719813346863, -0.3104729652404785, -0.018197864294052124, -0.1944357007741928, -0.28142961859703064, -0.22209195792675018, -0.20982830226421356, -0.20106644928455353, -0.2023465782403946, -0.14004366099834442, -0.13079792261123657, -0.1504126787185669, -0.16527439653873444, -0.07844796776771545, -0.18298931419849396, -0.25238537788391113, -0.10398094356060028, -0.16711805760860443, -0.09529539942741394, -0.29737651348114014, -0.15718886256217957, -0.04248633608222008, -0.22238785028457642, -0.118897944688797, -0.15248319506645203, -0.15689602494239807, -0.19035615026950836, -0.1718175858259201, ...]
>,
"kernel" => #Nx.Tensor<
f32[1024][1024]
EXLA.Backend<host:0, 0.768359907.414842896.60488>
[
[-0.15172797441482544, 0.2134985327720642, 0.11192941665649414, 0.1782056838274002, -0.0012918836437165737, 0.007915112189948559, -0.0249418206512928, 0.14754195511341095, -0.024335261434316635, 0.10817378759384155, -0.06433845311403275, -0.06597162038087845, -0.01614392176270485, 0.014331812039017677, 0.09983636438846588, -0.20430460572242737, 0.16488419473171234, 0.022479679435491562, 0.09201326221227646, 0.10491468757390976, -0.08444728702306747, 0.057163454592227936, -0.07442694157361984, 0.1604597270488739, -0.061095476150512695, 0.08477046340703964, 0.057111967355012894, -0.04622781276702881, 0.1856028437614441, 0.041447486728429794, 0.04755445569753647, 0.18408679962158203, 0.0021392160560935736, 0.1329835057258606, 0.07194358110427856, -0.06379897892475128, -0.08607301115989685, 0.1423013061285019, 0.13879309594631195, -0.12589620053768158, 0.06998002529144287, 0.13039007782936096, ...],
...
]
>
},
"dense_1" => %{
"bias" => #Nx.Tensor<
f32[1024]
EXLA.Backend<host:0, 0.768359907.414842904.60927>
[0.05895552411675453, -0.22049130499362946, -0.15484623610973358, -0.09844455868005753, -0.056260593235492706, -0.20017829537391663, -0.1286424994468689, -0.05683509260416031, -0.14222633838653564, -0.00700366823002696, -0.0616883747279644, -0.09968362748622894, -0.11000186204910278, 0.2407834678888321, 0.07629863917827606, -0.034234512597322464, -0.16366323828697205, 0.008337277919054031, -0.3592047691345215, -0.5924045443534851, -0.15730048716068268, 0.031205177307128906, -0.07605075091123581, -0.046294860541820526, -0.14076851308345795, -0.05861268565058708, -0.22387176752090454, 0.20337767899036407, -0.07836879789829254, -0.15547533333301544, -0.18547193706035614, 0.32658371329307556, 0.027189010754227638, -0.03638291731476784, -0.05731037259101868, -0.05941126123070717, 0.09660078585147858, 0.031022846698760986, -0.21842698752880096, -0.15241166949272156, -0.008083070628345013, 0.1301325559616089, ...]
>,
"kernel" => #Nx.Tensor<
f32[1024][1024]
EXLA.Backend<host:0, 0.768359907.414842896.60489>
[
[0.08165369182825089, -0.07726620882749557, -0.051898159086704254, -0.21661116182804108, 0.022586891427636147, -0.17559120059013367, -0.2204238325357437, -0.07328879833221436, -0.12264105677604675, -0.3183484375476837, -0.031077124178409576, -0.19565421342849731, 0.04069676622748375, -0.019510313868522644, -0.14163461327552795, -0.046110063791275024, -0.1152062863111496, -0.1128043383359909, 0.13186460733413696, 0.0822858214378357, -0.16803793609142303, -0.25634583830833435, -0.21455004811286926, -0.02853085659444332, -0.2851652204990387, 0.0034511771518737078, -0.07769278436899185, 0.036540478467941284, 0.13904576003551483, -0.03702455013990402, -0.16332198679447174, -0.12440665811300278, -0.17709209024906158, 0.02778264880180359, -0.027504419907927513, 0.09225784242153168, -0.16767722368240356, 0.01247845683246851, -0.10521605610847473, -0.14159485697746277, 7.330697262659669e-4, ...],
...
]
>
},
"dense_2" => %{
"bias" => #Nx.Tensor<
f32[1024]
EXLA.Backend<host:0, 0.768359907.414842904.60929>
[-0.11586960405111313, 0.07172057032585144, -0.41916728019714355, -0.1737828403711319, -0.2966299057006836, 0.5471561551094055, 0.1351793110370636, -0.6721233129501343, -0.19660839438438416, -0.3779280483722687, -0.20500215888023376, 0.4976629614830017, 0.8647434711456299, -0.24950282275676727, -0.14410144090652466, -0.022000763565301895, -0.0011676738504320383, 0.5147851705551147, -0.0184311605989933, 0.9383683800697327, 0.32579725980758667, 0.16584241390228271, 0.08866454660892487, -0.3169492483139038, 0.08261159807443619, 1.0248641967773438, -0.14976750314235687, 0.6954894661903381, 0.3771381378173828, 1.0223124027252197, -0.39192309975624084, -0.1724528968334198, -0.4606069028377533, -0.015331311151385307, 0.0993984267115593, -1.547669529914856, -0.5242733955383301, 0.35959622263908386, 0.1548265516757965, 0.4733356833457947, 0.20164036750793457, ...]
>,
"kernel" => #Nx.Tensor<
f32[1024][1024]
EXLA.Backend<host:0, 0.768359907.414842896.60490>
[
[-0.3601296544075012, -0.0630219578742981, 0.007619611453264952, -0.17283344268798828, -0.1710580289363861, -0.31675273180007935, -0.3087081015110016, -0.07884972542524338, -0.09599712491035461, -0.22349072992801666, -0.23065781593322754, 0.16348150372505188, -0.004494990222156048, -0.0653984323143959, -0.020797332748770714, -0.12741439044475555, 0.046380236744880676, -0.05380033329129219, -0.2662489116191864, -0.24073417484760284, -0.15865828096866608, -0.13790974020957947, -0.07373072952032089, -0.06897489726543427, -0.021952209994196892, -0.03712752088904381, -0.21351321041584015, -0.023745227605104446, 0.012808443047106266, 0.0370711088180542, 0.05359698459506035, -0.06689450889825821, -0.20794905722141266, -0.1579226553440094, -0.09690264612436295, -0.06358955055475235, 0.08135075867176056, -0.07869145274162292, 0.02226913720369339, 0.07279863953590393, ...],
...
]
>
},
"dense_3" => %{
"bias" => #Nx.Tensor<
f32[25]
EXLA.Backend<host:0, 0.768359907.414842904.60931>
[-0.2585510015487671, -0.17763929069042206, -0.22898036241531372, -0.17345108091831207, -0.20653462409973145, -0.20326517522335052, -0.25719544291496277, -0.24604849517345428, -0.2605043649673462, -0.1739831119775772, -0.24936942756175995, -0.21570858359336853, -0.38963302969932556, -0.23091642558574677, -0.2232494056224823, -0.14953315258026123, -0.2943863272666931, -0.4156269133090973, -0.3454320728778839, -0.20263618230819702, -0.22590823471546173, -0.1830047070980072, -0.21756970882415771, -0.20767024159431458, -0.24025936424732208]
>,
"kernel" => #Nx.Tensor<
f32[1024][25]
EXLA.Backend<host:0, 0.768359907.414842896.60491>
[
[0.05158635973930359, -0.09923555701971054, -0.03200432285666466, 0.07705419510602951, -3.0659453477710485e-4, 0.02819657512009144, 0.04957503452897072, -2.3773159773554653e-4, 0.020957347005605698, -0.05532548576593399, -0.02296423353254795, -0.021394122391939163, 0.04691838100552559, 0.06185632571578026, 0.07876424491405487, 0.007513448130339384, -0.041531141847372055, 0.04776705056428909, -0.02578279748558998, -0.020000247284770012, -0.028481563553214073, 0.06275158375501633, 0.03990684077143669, -0.04783887788653374, -0.06134241074323654],
[0.10875598341226578, -0.03528052195906639, -0.03108753263950348, 0.004315380938351154, -0.004030246753245592, -0.013291563838720322, -0.025896728038787842, -0.05855654552578926, -0.029562778770923615, 0.0039621456526219845, -0.041805289685726166, -0.01271834596991539, -0.05051732435822487, -0.023917708545923233, ...],
...
]
>
}
}
},
model: #Axon<
inputs: %{"x" => {nil, 2, 5, 5}}
outputs: "dense_3"
nodes: 15
>,
init_fn: nil,
step_fn: nil
}
いくつかの盤面でのq_netの出力値をみてみます
abc
1: -----
2: -○○○-
3: -----
4: -●●●-
5: -----
次は、黒の手番です。
q_netの出力をみてみると、置くと5個並ぶことになる、(1,4)と(1,0)の値が高くなっています。
Heatmapでも値が高く（白い色）で表示されています。
# 背景を黒、文字の色を白にする
IO.puts("\e[40m\e[37m")
gomoku =
Gomoku.new(5, 4)
|> Gomoku.put!({1, 1})
|> Gomoku.put!({3, 1})
|> Gomoku.put!({1, 2})
|> Gomoku.put!({3, 2})
|> Gomoku.put!({1, 3})
|> Gomoku.put!({3, 3})
|> tap(fn x -> Gomoku.render(x) |> IO.puts() end)
IO.puts("\e[0m")
Nx.tensor([Gomoku.to_array(gomoku)])
|> IO.inspect(label: "in=")
|> QNet.forward(q_net)
|> Nx.reshape({5, 5})
|> IO.inspect(label: "out=")
|> Nx.to_heatmap()
abcde
1: -----
2: -○○○-
3: -----
4: -●●●-
5: -----
in=: #Nx.Tensor<
s64[1][2][5][5]
EXLA.Backend<host:0, 0.768359907.414842904.60933>
[
[
[
[0, 0, 0, 0, 0],
[0, 1, 1, 1, 0],
[0, 0, 0, 0, 0],
[0, 0, 0, 0, 0],
[0, 0, 0, 0, 0]
],
[
[0, 0, 0, 0, 0],
[0, 0, 0, 0, 0],
[0, 0, 0, 0, 0],
[0, 1, 1, 1, 0],
[0, 0, 0, 0, 0]
]
]
]
>
out=: #Nx.Tensor<
f32[5][5]
EXLA.Backend<host:0, 0.768359907.414842896.60514>
[
[-0.6659831404685974, -0.5541943311691284, 0.16300129890441895, -0.48581135272979736, -0.46773701906204224],
[0.8283513784408569, -0.6478000283241272, -1.2770978212356567, -1.2387440204620361, 0.9820483922958374],
[0.06604059040546417, 0.4480787217617035, 0.0710911750793457, -0.549408495426178, 0.07041525840759277],
[-0.0648372694849968, -0.6948455572128296, -0.527321457862854, -0.757087767124176, -0.2815285623073578],
[-0.4432772099971771, -0.3601277768611908, -1.005516529083252, -0.4045620560646057, -0.25490087270736694]
]
>
次は黒の手番です。
白が２つ並んでいるので、これを止めないと負けます。
abc
1: -----
2: -○○--
3: --●--
4: -----
5: -----
q_netの出力をみてみると、(1,3)が最も高くなっていて正しい手になっています。
# 背景を黒、文字の色を白にする
IO.puts("\e[40m\e[37m")
gomoku =
Gomoku.new(5, 4)
|> Gomoku.put!({1, 1})
|> Gomoku.put!({2, 2})
|> Gomoku.put!({1, 2})
|> tap(fn x -> Gomoku.render(x) |> IO.puts() end)
IO.puts("\e[0m")
Nx.tensor([Gomoku.to_array(gomoku)])
|> IO.inspect(label: "in=")
|> QNet.forward(q_net)
|> Nx.reshape({5, 5})
|> IO.inspect(label: "out=")
|> Nx.to_heatmap()
abcde
1: -----
2: -○○--
3: --●--
4: -----
5: -----
in=: #Nx.Tensor<
s64[1][2][5][5]
EXLA.Backend<host:0, 0.768359907.414842904.60943>
[
[
[
[0, 0, 0, 0, 0],
[0, 0, 0, 0, 0],
[0, 0, 1, 0, 0],
[0, 0, 0, 0, 0],
[0, 0, 0, 0, 0]
],
[
[0, 0, 0, 0, 0],
[0, 1, 1, 0, 0],
[0, 0, 0, 0, 0],
[0, 0, 0, 0, 0],
[0, 0, 0, 0, 0]
]
]
]
>
out=: #Nx.Tensor<
f32[5][5]
EXLA.Backend<host:0, 0.768359907.414842896.60537>
[
[-1.1858656406402588, -0.7687045931816101, -0.9617680311203003, -1.4758198261260986, -1.1471666097640991],
[-0.8074132204055786, -1.0422759056091309, -1.0469446182250977, -0.06851829588413239, -0.8615824580192566],
[-0.9570947885513306, -0.888636589050293, -0.646882951259613, -0.8544129729270935, -1.0251939296722412],
[-1.3585505485534668, -0.7337592840194702, -0.9061002135276794, -1.2567375898361206, -1.032645344734192],
[-0.9246397018432617, -1.069082498550415, -1.004856824874878, -0.616108238697052, -0.8607216477394104]
]
>
q_netの出力の最大値を選ぶ関数Dqn.get_action()を使うことで、次の手を取得する事ができます。
gomoku =
Gomoku.new(5, 4)
|> Gomoku.put!({1, 1})
|> Gomoku.put!({2, 2})
|> Gomoku.put!({1, 2})
|> tap(fn x -> Gomoku.render(x) |> IO.puts() end)
pos = Dqn.get_action(gomoku, q_net)
IO.inspect(pos, label: "next action")
gomoku
|> Gomoku.put!(pos)
|> tap(fn x -> Gomoku.render(x) |> IO.puts() end)
abcde
1: -----
2: -○○--
3: --●--
4: -----
5: -----
next action: {1, 3}
abcde
1: -----
2: -○○●-
3: --●--
4: -----
5: -----
%Gomoku{
board: %{
{0, 0} => 0,
{0, 1} => 0,
{0, 2} => 0,
{0, 3} => 0,
{0, 4} => 0,
{1, 0} => 0,
{1, 1} => 1,
{1, 2} => 1,
{1, 3} => 2,
{1, 4} => 0,
{2, 0} => 0,
{2, 1} => 0,
{2, 2} => 2,
{2, 3} => 0,
{2, 4} => 0,
{3, 0} => 0,
{3, 1} => 0,
{3, 2} => 0,
{3, 3} => 0,
{3, 4} => 0,
{4, 0} => 0,
{4, 1} => 0,
{4, 2} => 0,
{4, 3} => 0,
{4, 4} => 0
},
size: 5,
turn: 1,
done: false,
draw: false,
miss: false,
k: 4
}
IO.inspect(pos, label: "next move:")
next move:: {1, 3}
{1, 3}
次は、AIの対戦相手のGUIへの組み込みを行います。
【Elixir/Phoenix】AI対戦4目並べを作る（完成編）
関連記事