Go to Qiita Advent Calendar 2024 Top
LoginSignup
1
2

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?

More than 5 years have passed since last update.

@7of9

TensorFlow > tf.train.shuffle_batch() + eval()の効果の確認 > sine curve学習失敗の理由 > .eval()時に異なるインデックスのデータを取っていた

Last updated at Posted at 2016-11-20
動作環境
GeForce GTX 1070 (8GB)
ASRock Z170M Pro4S [Intel Z170chipset]
Ubuntu 14.04 LTS desktop amd64
TensorFlow v0.11
cuDNN v5.1 for Linux
CUDA v8.0
Python 2.7.6
IPython 5.1.0 -- An enhanced Interactive Python.

v0.1 http://qiita.com/7of9/items/b364d897b95476a30754

http://qiita.com/7of9/items/ce72fea33fc0a2dea479
においてsine curveの学習が進まない。

なぜなのか。

tf.train.shuffle_batch() + eval()の効果を確認することにした。

失敗編

シーケンスデータの作成

shuffleした時の効果が分かるように、入力するcsvファイルを作ることにした。　

  • 1列目: 1から100
  • 2列目: 501から600

これで、１列目と２列目の対応が分かる。

python prep_seq_data.py > input2.csv
を実行する。

Jupyterで確認。

%matplotlib inline

import numpy as np
import matplotlib.pyplot as plt

data1 = np.loadtxt('input2.csv', delimiter=',')

input1 = data1[:,0]
output1 = data1[:,1]

fig = plt.figure()
ax1 = fig.add_subplot(1,1,1)

#ax1.plot(input1, output1, color='black', linestyle='solid', label='original')
ax1.scatter(input1, output1)

ax1.set_title('loss')
ax1.set_xlabel('step')
ax1.set_ylabel('loss')
ax1.grid(True)
ax1.legend()

fig.show()

qiita.png

tf.train.shuffle_batch() + eval()の効果

test_batch.py
#!/usr/bin/env python
# -*- coding: utf-8 -*-

import sys
import tensorflow as tf
import tensorflow.contrib.slim as slim

filename_queue = tf.train.string_input_producer(["input2.csv"])
reader = tf.TextLineReader()
key, value = reader.read(filename_queue)
input1, output = tf.decode_csv(value, record_defaults=[[0.], [0.]])
inputs = tf.pack([input1])
output = tf.pack([output])

batch_size=4 # [4]
inputs_batch, output_batch = tf.train.shuffle_batch([inputs, output], batch_size, capacity=40, min_after_dequeue=batch_size)

input_ph = tf.placeholder("float", [None,1])
output_ph = tf.placeholder("float",[None,1])

## NN のグラフ生成
hiddens = slim.stack(input_ph, slim.fully_connected, [1,7,7,7], 
  activation_fn=tf.nn.sigmoid, scope="hidden")
prediction = slim.fully_connected(hiddens, 1, activation_fn=tf.nn.sigmoid, scope="output")
loss = tf.contrib.losses.mean_squared_error(prediction, output_ph)

#train_op = slim.learning.create_train_op(loss, tf.train.AdamOptimizer(0.01))
train_op = slim.learning.create_train_op(loss, tf.train.AdamOptimizer(0.001))

#def feed_dict(inputs, output):
#    return {input_ph: inputs.eval(), output_ph: output.eval()}

init_op = tf.initialize_all_variables()

with tf.Session() as sess:
  coord = tf.train.Coordinator()
  threads = tf.train.start_queue_runners(sess=sess, coord=coord)

  try:
    sess.run(init_op)
    for i in range(3): #[30000]
    	print(inputs_batch.eval(), output_batch.eval())

#      _, t_loss = sess.run([train_op, loss], feed_dict={input_ph:inputs_batch.eval(), output_ph:output_batch.eval()})
#      if (i+1) % 100 == 0:
#        print("%d,%f" % (i+1, t_loss))
#        print("%d,%f,#step, loss" % (i+1, t_loss))
  finally:
    coord.request_stop()

  coord.join(threads)
結果
$ python test_batch.py 
I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcublas.so locally
I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcudnn.so locally
I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcufft.so locally
I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcuda.so.1 locally
I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcurand.so locally
WARNING:tensorflow:sum_of_squares (from tensorflow.contrib.losses.python.losses.loss_ops) is deprecated and will be removed after 2016-10-01.
Instructions for updating:
Use mean_squared_error.
I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:925] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
I tensorflow/core/common_runtime/gpu/gpu_device.cc:951] Found device 0 with properties: 
name: GeForce GTX 1070
major: 6 minor: 1 memoryClockRate (GHz) 1.7715
pciBusID 0000:01:00.0
Total memory: 7.91GiB
Free memory: 7.24GiB
I tensorflow/core/common_runtime/gpu/gpu_device.cc:972] DMA: 0 
I tensorflow/core/common_runtime/gpu/gpu_device.cc:982] 0:   Y 
I tensorflow/core/common_runtime/gpu/gpu_device.cc:1041] Creating TensorFlow device (/gpu:0) -> (device: 0, name: GeForce GTX 1070, pci bus id: 0000:01:00.0)
(array([[ 27.],
       [ 13.],
       [ 21.],
       [ 40.]], dtype=float32), array([[ 541.],
       [ 531.],
       [ 520.],
       [ 506.]], dtype=float32))
(array([[ 14.],
       [ 44.],
       [ 34.],
       [  4.]], dtype=float32), array([[ 537.],
       [ 519.],
       [ 515.],
       [ 535.]], dtype=float32))
(array([[ 38.],
       [ 11.],
       [ 10.],
       [ 23.]], dtype=float32), array([[ 542.],
       [ 503.],
       [ 546.],
       [ 530.]], dtype=float32))

一列目 {27, 13, 21, 40}に対して、二列目{541, 531, 520, 506}になっている。

つまり、inputs_batch.eval()をした時のインデックスとoutput_batch().eval()した時のインデックスが一致していない。

これではsine curveが学習できるわけがない。

.eval()の使い方を学習しないといけない。

evalの代わり

参考 http://stackoverflow.com/questions/34010987/does-tensorflow-rerun-for-each-eval-call

.eval()を使わず、sess.run()を使う。

#       print(inputs_batch.eval(), output_batch.eval())
      inp, out = sess.run([inputs_batch, output_batch])
      print (inp, out)
test_batch2.py
#!/usr/bin/env python
# -*- coding: utf-8 -*-

import sys
import tensorflow as tf
import tensorflow.contrib.slim as slim

filename_queue = tf.train.string_input_producer(["input2.csv"])
reader = tf.TextLineReader()
key, value = reader.read(filename_queue)
input1, output = tf.decode_csv(value, record_defaults=[[0.], [0.]])
inputs = tf.pack([input1])
output = tf.pack([output])

batch_size=4 # [4]
inputs_batch, output_batch = tf.train.shuffle_batch([inputs, output], batch_size, capacity=40, min_after_dequeue=batch_size)

input_ph = tf.placeholder("float", [None,1])
output_ph = tf.placeholder("float",[None,1])

## NN のグラフ生成
hiddens = slim.stack(input_ph, slim.fully_connected, [1,7,7,7], 
  activation_fn=tf.nn.sigmoid, scope="hidden")
prediction = slim.fully_connected(hiddens, 1, activation_fn=tf.nn.sigmoid, scope="output")
loss = tf.contrib.losses.mean_squared_error(prediction, output_ph)

#train_op = slim.learning.create_train_op(loss, tf.train.AdamOptimizer(0.01))
train_op = slim.learning.create_train_op(loss, tf.train.AdamOptimizer(0.001))

#def feed_dict(inputs, output):
#    return {input_ph: inputs.eval(), output_ph: output.eval()}

init_op = tf.initialize_all_variables()

with tf.Session() as sess:
  coord = tf.train.Coordinator()
  threads = tf.train.start_queue_runners(sess=sess, coord=coord)

  try:
    sess.run(init_op)
    for i in range(3): #[30000]
#    	print(inputs_batch.eval(), output_batch.eval())
  
      inp, out = sess.run([inputs_batch, output_batch])
      print (inp, out)

#      _, t_loss = sess.run([train_op, loss], feed_dict={input_ph:inputs_batch.eval(), output_ph:output_batch.eval()})
#      if (i+1) % 100 == 0:
#        print("%d,%f" % (i+1, t_loss))
#        print("%d,%f,#step, loss" % (i+1, t_loss))
  finally:
    coord.request_stop()

  coord.join(threads)
結果　
$ python test_batch2.py 
I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcublas.so locally
I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcudnn.so locally
I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcufft.so locally
I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcuda.so.1 locally
I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcurand.so locally
WARNING:tensorflow:sum_of_squares (from tensorflow.contrib.losses.python.losses.loss_ops) is deprecated and will be removed after 2016-10-01.
Instructions for updating:
Use mean_squared_error.
I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:925] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
I tensorflow/core/common_runtime/gpu/gpu_device.cc:951] Found device 0 with properties: 
name: GeForce GTX 1070
major: 6 minor: 1 memoryClockRate (GHz) 1.7715
pciBusID 0000:01:00.0
Total memory: 7.91GiB
Free memory: 7.18GiB
I tensorflow/core/common_runtime/gpu/gpu_device.cc:972] DMA: 0 
I tensorflow/core/common_runtime/gpu/gpu_device.cc:982] 0:   Y 
I tensorflow/core/common_runtime/gpu/gpu_device.cc:1041] Creating TensorFlow device (/gpu:0) -> (device: 0, name: GeForce GTX 1070, pci bus id: 0000:01:00.0)
(array([[ 36.],
       [ 29.],
       [ 11.],
       [ 37.]], dtype=float32), array([[ 536.],
       [ 529.],
       [ 511.],
       [ 537.]], dtype=float32))
(array([[ 43.],
       [ 31.],
       [ 24.],
       [ 34.]], dtype=float32), array([[ 543.],
       [ 531.],
       [ 524.],
       [ 534.]], dtype=float32))
(array([[  1.],
       [ 21.],
       [ 41.],
       [ 45.]], dtype=float32), array([[ 501.],
       [ 521.],
       [ 541.],
       [ 545.]], dtype=float32))

一列目 {36, 29, 11, 37}に対して、二列目{536, 529, 511, 537}になっている。

成功。

1
2
0

Register as a new user and use Qiita more conveniently

  1. You get articles that match your needs
  2. You can efficiently read back useful information
  3. You can use dark theme
What you can do with signing up
Sign upLogin
1
2

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?