0
0

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?

More than 3 years have passed since last update.

Tensorflow-gpu で Resource exhausted error が起きた意外な理由

Posted at

#概要
意外な理由で Resource exhausted error が出てどハマりしたので解決策を残しておきます.

#状況

  • tensorflow-gpuを使って深層学習のモデルをいくつか試していた.
  • ずっと問題なく動いていた model_A が,ある日突然 Resource exhausted error を吐いて動かなくなった
  • コードは動作時と全く同じ(gitHubで確認済み)
  • conda の仮想環境も全く同じ

#原因

model_Bを動かすために,$HOME/.keras/keras.json の設定を書き換えてしまっていた.

keras.json (for model_A)
{
    "floatx": "float32",
    "epsilon": 1e-07,
    "backend": "tensorflow",
    "image_data_format": "channels_last" 
}
keras.json (for model_B)
{
    "floatx": "float32",
    "epsilon": 1e-07,
    "backend": "tensorflow",
    "image_data_format": "channels_first"
}

#コメント

model_B で model_A の設定を使うと,単純に配列の型が違うというエラーが出るので簡単に気付けるのですが,逆だとメモリ不足のエラーが出るようで,原因の特定に苦労しました...
恐らくサンプル数の部分を1サンプル内のある次元の大きさとみなしてしまうことが原因かと.

0
0
0

Register as a new user and use Qiita more conveniently

  1. You get articles that match your needs
  2. You can efficiently read back useful information
  3. You can use dark theme
What you can do with signing up
0
0

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?