転ばぬ先の杖-機械学習エラー備忘録(matplotlibエラー、scikit-learnエラー、kerasエラー)

はじめに

本記事は、機械学習を行った際に遭遇したエラーについての記録である。

環境：Docker（ubuntu:16.04）

matplotlibエラー

tkinter.TclError: no display name and no $DISPLAY environment variable

Traceback (most recent call last):
  File "bmi-plot.py", line 12, in <module>
    fig = plt.figure()
  File "/usr/local/lib/python3.5/dist-packages/matplotlib/pyplot.py", line 533, in figure
    **kwargs)
  File "/usr/local/lib/python3.5/dist-packages/matplotlib/backend_bases.py", line 161, in new_figure_manager
    return cls.new_figure_manager_given_figure(num, fig)
  File "/usr/local/lib/python3.5/dist-packages/matplotlib/backends/_backend_tk.py", line 1046, in new_figure_manager_given_figure
    window = Tk.Tk(className="matplotlib")
  File "/usr/lib/python3.5/tkinter/__init__.py", line 1871, in __init__
    self.tk = _tkinter.create(screenName, baseName, className, interactive, wantobjects, useTk, sync, use)
_tkinter.TclError: no display name and no $DISPLAY environment variable

• 解説
  DockerなどGUIのない環境で実行（描画：プロット）した場合
• 原因
  matplotlibに起因している
• 対処法
  以下の環境変数を設定する

root@50a6ff8ed584:/tmp# export MPLBACKEND="agg"

scikit-learnエラー

ImportError: No module named '_tkinter'

/usr/local/lib/python3.5/dist-packages/sklearn/cross_validation.py:41: DeprecationWarning: This module was deprecated in version 0.18 in favor of the model_selection module into which all the refactored classes and functions are moved. Also note that the interface of the new CV iterators are different from that of this module. This module will be removed in 0.20.
  "This module will be removed in 0.20.", DeprecationWarning)
Traceback (most recent call last):
  File "/usr/lib/python3.5/tkinter/__init__.py", line 36, in <module>
    import _tkinter
ImportError: No module named '_tkinter'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "bmi-test.py", line 8, in <module>
    import matplotlib.pyplot as plt
  File "/usr/local/lib/python3.5/dist-packages/matplotlib/pyplot.py", line 115, in <module>
    _backend_mod, new_figure_manager, draw_if_interactive, _show = pylab_setup()
  File "/usr/local/lib/python3.5/dist-packages/matplotlib/backends/__init__.py", line 62, in pylab_setup
    [backend_name], 0)
  File "/usr/local/lib/python3.5/dist-packages/matplotlib/backends/backend_tkagg.py", line 4, in <module>
    from . import tkagg  # Paint image to Tk photo blitter extension.
  File "/usr/local/lib/python3.5/dist-packages/matplotlib/backends/tkagg.py", line 5, in <module>
    from six.moves import tkinter as Tk
  File "/usr/local/lib/python3.5/dist-packages/six.py", line 92, in __get__
    result = self._resolve()
  File "/usr/local/lib/python3.5/dist-packages/six.py", line 115, in _resolve
    return _import_module(self.mod)
  File "/usr/local/lib/python3.5/dist-packages/six.py", line 82, in _import_module
    __import__(name)
  File "/usr/lib/python3.5/tkinter/__init__.py", line 38, in <module>
    raise ImportError(str(msg) + ', please install the python3-tk package')
ImportError: No module named '_tkinter', please install the python3-tk package

• 解説
  sklearnでモジュールエラー発生
• 原因
  モジュールの依存関係
• 対処法
  atp-getでpython3-tk packageをインストールする

root@50a6ff8ed584:/tmp# sudo apt-get install python3-tk

KeyError

/usr/local/lib/python3.5/dist-packages/sklearn/cross_validation.py:41: DeprecationWarning: This module was deprecated in version 0.18 in favor of the model_selection module into which all the refactored classes and functions are moved. Also note that the interface of the new CV iterators are different from that of this module. This module will be removed in 0.20.
  "This module will be removed in 0.20.", DeprecationWarning)
Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/pandas/core/indexes/base.py", line 3078, in get_loc
    return self._engine.get_loc(key)
  File "pandas/_libs/index.pyx", line 140, in pandas._libs.index.IndexEngine.get_loc
  File "pandas/_libs/index.pyx", line 162, in pandas._libs.index.IndexEngine.get_loc
  File "pandas/_libs/hashtable_class_helper.pxi", line 1492, in pandas._libs.hashtable.PyObjectHashTable.get_item
  File "pandas/_libs/hashtable_class_helper.pxi", line 1500, in pandas._libs.hashtable.PyObjectHashTable.get_item
KeyError: 'label'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "bmi-test.py", line 13, in <module>
    label = tbl["label"]
  File "/usr/local/lib/python3.5/dist-packages/pandas/core/frame.py", line 2688, in __getitem__
    return self._getitem_column(key)
  File "/usr/local/lib/python3.5/dist-packages/pandas/core/frame.py", line 2695, in _getitem_column
    return self._get_item_cache(key)
  File "/usr/local/lib/python3.5/dist-packages/pandas/core/generic.py", line 2489, in _get_item_cache
    values = self._data.get(item)
  File "/usr/local/lib/python3.5/dist-packages/pandas/core/internals.py", line 4115, in get
    loc = self.items.get_loc(item)
  File "/usr/local/lib/python3.5/dist-packages/pandas/core/indexes/base.py", line 3080, in get_loc
    return self._engine.get_loc(self._maybe_cast_indexer(key))
  File "pandas/_libs/index.pyx", line 140, in pandas._libs.index.IndexEngine.get_loc
  File "pandas/_libs/index.pyx", line 162, in pandas._libs.index.IndexEngine.get_loc
  File "pandas/_libs/hashtable_class_helper.pxi", line 1492, in pandas._libs.hashtable.PyObjectHashTable.get_item
  File "pandas/_libs/hashtable_class_helper.pxi", line 1500, in pandas._libs.hashtable.PyObjectHashTable.get_item
KeyError: 'label'

• 解説
  sklearnで例外エラー（KeyError）が発生
• 原因
  正規化時に利用するCSVファイルのカラムにスペースが入っていたため
• 対処法
  CSVファイルを生成するプログラムの修正

NGパターン)
# プログラムのステートメント
fp.write("height, weight, label\r\n")

生成されたCSVファイルの1行目
height, weight, label

OKパターン)
# プログラムのステートメント
fp.write("height,weight,label\r\n")

生成されたCSVファイルの1行目
height,weight,label

ModuleNotFoundError: No module named 'sklearn.cross_validation'

Using TensorFlow backend.
Traceback (most recent call last):
  File "mlp3-classify.org.py", line 8, in <module>
    from sklearn.cross_validation import train_test_split
ModuleNotFoundError: No module named 'sklearn.cross_validation'

• 解説
  sklearn実行時にモジュールエラー発生
• 原因
  最新のsklearnでは「sklearn.cross_validation」はなくなっているため
• 対処法
  sklearn.model_selectionより、「cross_val_score」をインポートする

from sklearn.model_selection import cross_val_score, train_test_split

kerasエラー

ModuleNotFoundError: No module named 'tensorflow'

Using TensorFlow backend.
Traceback (most recent call last):
  File "mlp3-classify.py", line 4, in <module>
    from keras.models import Sequential
  File "/usr/local/lib/python3.6/dist-packages/keras/__init__.py", line 3, in <module>
    from . import utils
  File "/usr/local/lib/python3.6/dist-packages/keras/utils/__init__.py", line 6, in <module>
    from . import conv_utils
  File "/usr/local/lib/python3.6/dist-packages/keras/utils/conv_utils.py", line 9, in <module>
    from .. import backend as K
  File "/usr/local/lib/python3.6/dist-packages/keras/backend/__init__.py", line 89, in <module>
    from .tensorflow_backend import *
  File "/usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py", line 5, in <module>
    import tensorflow as tf
ModuleNotFoundError: No module named 'tensorflow'

• 解説
  keras実行時にモジュールエラー発生
• 原因
  kerasはバックエンドでtensorflowを読み込んでいるので、tensorflowがインストールされていないため
• 対処法
  tensorflowをインストールする

root@dcb94689450e:/tmp/stady# pip3 install tensorflow 

ModuleNotFoundError: No module named 'keras_applications'

Using TensorFlow backend.
Traceback (most recent call last):
  File "mlp3-classify.org.py", line 4, in <module>
    from keras.models import Sequential
  File "/usr/local/lib/python3.6/dist-packages/keras/__init__.py", line 3, in <module>
    from . import activations
  File "/usr/local/lib/python3.6/dist-packages/keras/activations.py", line 4, in <module>
    from . import backend as K
  File "/usr/local/lib/python3.6/dist-packages/keras/backend/__init__.py", line 83, in <module>
    from .tensorflow_backend import *
  File "/usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py", line 1, in <module>
    import tensorflow as tf
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/__init__.py", line 24, in <module>
    from tensorflow.python import pywrap_tensorflow  # pylint: disable=unused-import
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/__init__.py", line 88, in <module>
    from tensorflow.python import keras
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/__init__.py", line 25, in <module>
    from tensorflow.python.keras import applications
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/applications/__init__.py", line 22, in <module>
    import keras_applications
ModuleNotFoundError: No module named 'keras_applications'

• 解説
  keras実行時にモジュールエラー発生（補足：kerasは、pip3 install keras==2.0.5でインストール）
• 原因
  モジュールの依存関係
• 対処法
  keras_applicationsとkeras_preprocessingをインストールする

root@dcb94689450e:/tmp/stady# pip3 install keras_applications==1.0.4 --no-deps
root@dcb94689450e:/tmp/stady# pip3 install keras_preprocessing==1.0.2 --no-deps

おわりに

scikit-learnエラー（KeyError）のような、灯台下暗しパターンがあんがい分かりにくかったりする。