参考

概要

tensorflowを使用した画像分類を試してみます。

今回使用するのはtensorflow-macosです。
tensorflow-metalもインストールしてみましたが、うまく動作しなかったため後日再チャレンジします。

目次

環境
 仮想環境の用意
 pipのアップデート
 tensorflow-macosのインストール
 画像分類器
 おまけ(tensorflow-metal)

環境

項目	名称
モデル	Mac mini(M1,2020)
CPU	Apple M1
OS	macOS Monterey

Pythonについて

Pythonはユニバーサルなものかarm64のものである必要があります。
以下のようにfileコマンドで確認することができます。

今回はarm64のPythonを使用します。

CPUのアーキテクチャがソフトウェアに与える影響がイメージできない...

2つのアーキテクチャを備えたユニバーサルバイナリ

標準でインストールされていたPython

% /usr/bin/python3 -V
Python 3.8.2

% file /usr/bin/python3                    
/usr/bin/python3: Mach-O universal binary with 2 architectures: [x86_64:Mach-O 64-bit executable x86_64
- Mach-O 64-bit executable x86_64] [arm64e:Mach-O 64-bit executable arm64e
- Mach-O 64-bit executable arm64e]
/usr/bin/python3 (for architecture x86_64):     Mach-O 64-bit executable x86_64
/usr/bin/python3 (for architecture arm64e):     Mach-O 64-bit executable arm64e

arm64のバイナリ

arm64のbrewでインストールしたPython

% /opt/homebrew/bin/python3 -V
Python 3.9.7

% file /opt/homebrew/bin/python3
/opt/homebrew/bin/python3: Mach-O 64-bit executable arm64

x86_64のバイナリ
この場合、tensorflow-macosは使用できません。

anyenv,pyenv経由でインストールしたPython

% ~/.anyenv/envs/pyenv/versions/3.7.9/bin/python3 -V
Python 3.7.9

% file ~/.anyenv/envs/pyenv/versions/3.7.9/bin/python3
~/.anyenv/envs/pyenv/versions/3.7.9/bin/python3: Mach-O 64-bit executable x86_64

仮想環境の用意

今回は仮想環境にvenvを使用します。

tensorflow-metal部分は仮想環境の名前です。
自由に決めることができます。

仮想環境の作成

% /opt/homebrew/bin/python3 -m venv tensorflow-metal

仮想環境を有効にすると仮想環境名がプロンプトの左に表示されます。

仮想環境の有効化

% source ./tensorflow-metal/bin/activate
(tensorflow-metal) %

仮想環境を無効にするとプロンプトの左の仮想環境名が非表示になります。

仮想環境の無効化

(tensorflow-metal) % deactivate
%

pipのアップデート

WARNINGが出力されるので、念の為pipをアップデートします。

一覧

(tensorflow-metal)% pip list
Package    Version
---------- -------
pip        21.2.4
setuptools 57.4.0
WARNING: You are using pip version 21.2.4; however, version 22.1.1 is available.
You should consider upgrading via the '~/workspace/qiita/image_classfication/venv/bin/python3.9 -m pip install --upgrade pip' command.

アップデート

 (tensorflow-metal)% pip install -U pip
Requirement already satisfied: pip in ./venv/lib/python3.9/site-packages (21.2.4)
Collecting pip
  Using cached pip-22.1.1-py3-none-any.whl (2.1 MB)
Installing collected packages: pip
  Attempting uninstall: pip
    Found existing installation: pip 21.2.4
    Uninstalling pip-21.2.4:
      Successfully uninstalled pip-21.2.4
Successfully installed pip-22.1.1

tensorflow-macosのインストール

Getting Started with tensorflow-metal PluggableDevice

インストール

(tensorflow-metal)% pip install tensorflow-macos

(...省略...)

Successfully installed absl-py-1.0.0 astunparse-1.6.3 cachetools-5.2.0 certifi-2022.5.18.1 charset-normalizer-2.0.12 flatbuffers-1.12 gast-0.4.0 google-auth-2.6.6 google-auth-oauthlib-0.4.6 google-pasta-0.2.0 grpcio-1.46.3 h5py-3.7.0 idna-3.3 importlib-metadata-4.11.4 keras-2.9.0 keras-preprocessing-1.1.2 libclang-14.0.1 markdown-3.3.7 numpy-1.22.4 oauthlib-3.2.0 opt-einsum-3.3.0 packaging-21.3 protobuf-4.21.1 pyasn1-0.4.8 pyasn1-modules-0.2.8 pyparsing-3.0.9 requests-2.27.1 requests-oauthlib-1.3.1 rsa-4.8 six-1.16.0 tensorboard-2.9.0 tensorboard-data-server-0.6.1 tensorboard-plugin-wit-1.8.1 tensorflow-estimator-2.9.0 tensorflow-macos-2.9.0 termcolor-1.1.0 typing-extensions-4.2.0 urllib3-1.26.9 werkzeug-2.1.2 wheel-0.37.1 wrapt-1.14.1 zipp-3.8.0

tensorflowをimportしてみて動作するか確認してみます。

動作確認：（失敗）

(tensorflow-metal)% python -c 'import tensorflow as tf; print(tf.__version__)'

(...省略...)

If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.
If you cannot immediately regenerate your protos, some other possible workarounds are:
 1. Downgrade the protobuf package to 3.20.x or lower.
 2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).

More information: https://developers.google.com/protocol-buffers/docs/news/2022-05-06#python-updates

protobufのバージョンを3.20.xもしくはそれ以下にダウングレードする必要があるようです。

protobufのダウングレード

(tensorflow-metal)% pip install -U protobuf~=3.20.0
Installing collected packages: protobuf
  Attempting uninstall: protobuf
    Found existing installation: protobuf 4.21.1
    Uninstalling protobuf-4.21.1:
      Successfully uninstalled protobuf-4.21.1
Successfully installed protobuf-3.20.1

もう一度動作確認してみます。

動作確認（成功）

(tensorflow-metal)% python -c 'import tensorflow as tf; print(tf.__version__)'
2.9.0

今度は正常にバージョンが出力されました。

画像分類器

Making your own TensorFlow model for image classification

インストール

% pip install "tensorflow-hub[make_image_classifier]"

モデルの学習

画像分類モデルを作成してみます。
自前の学習データがある場合は--image_dirパラメータを使用して学習データが格納してあるディレクトリを指定してください。

指定しない場合は--image_dirを省略してください。デモ用に用意された学習データを利用して、花の写真をデイジー、タンポポ、バラ、ヒマワリ、またはチューリップとして分類するモデルが作成されます。

学習済みモデルの作成

(tensorflow-metal)%  make_image_classifier \
  --tfhub_module https://tfhub.dev/google/tf2-preview/mobilenet_v2/feature_vector/4 \
  --image_size 224 \
  --saved_model_dir my_dir/new_model \
  --labels_output_file class_labels.txt \
  --tflite_output_file new_mobile_model.tflite \
  --summaries_dir my_log_dir

出力

I0531 19:57:31.066186 4335076736 resolver.py:105] Using /var/folders/_6/1pgtbjws1_z0l934qt03z5sw0000gn/T/tfhub_modules to cache modules.
Using module https://tfhub.dev/google/tf2-preview/mobilenet_v2/feature_vector/4 with image size (224, 224)
Found 3670 files belonging to 5 classes.
Using 2936 files for training.
Found 3670 files belonging to 5 classes.
Using 734 files for validation.
Found 5 classes: daisy, dandelion, roses, sunflowers, tulips
Model: "sequential_1"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 keras_layer (KerasLayer)    (None, 1280)              2257984   
                                                                 
 dropout (Dropout)           (None, 1280)              0         
                                                                 
 dense (Dense)               (None, 5)                 6405      
                                                                 
=================================================================
Total params: 2,264,389
Trainable params: 6,405
Non-trainable params: 2,257,984
_________________________________________________________________
None
~/workspace/qiita/image_classfication/venv/lib/python3.9/site-packages/keras/optimizers/optimizer_v2/gradient_descent.py:108: UserWarning: The `lr` argument is deprecated, use `learning_rate` instead.
  super(SGD, self).__init__(name, **kwargs)
2022-05-31 19:57:33.357699: W tensorflow/core/platform/profile_utils/cpu_utils.cc:128] Failed to get CPU frequency: 0 Hz
Epoch 1/5
91/91 [==============================] - 29s 282ms/step - loss: 0.9438 - accuracy: 0.7414 - val_loss: 0.7305 - val_accuracy: 0.8594
Epoch 2/5
91/91 [==============================] - 23s 258ms/step - loss: 0.7186 - accuracy: 0.8609 - val_loss: 0.7642 - val_accuracy: 0.8395
Epoch 3/5
91/91 [==============================] - 23s 258ms/step - loss: 0.6677 - accuracy: 0.8988 - val_loss: 0.6882 - val_accuracy: 0.8835
Epoch 4/5
91/91 [==============================] - 26s 282ms/step - loss: 0.6389 - accuracy: 0.9039 - val_loss: 0.6776 - val_accuracy: 0.8892
Epoch 5/5
91/91 [==============================] - 26s 286ms/step - loss: 0.6253 - accuracy: 0.9194 - val_loss: 0.6292 - val_accuracy: 0.9148
Done with training.
Labels written to class_labels.txt
INFO:tensorflow:Assets written to: my_dir/new_model/assets
I0531 19:59:45.298975 4335076736 builder_impl.py:779] Assets written to: my_dir/new_model/assets
SavedModel model exported to my_dir/new_model
2022-05-31 19:59:52.812172: W tensorflow/compiler/mlir/lite/python/tf_tfl_flatbuffer_helpers.cc:362] Ignored output_format.
2022-05-31 19:59:52.812196: W tensorflow/compiler/mlir/lite/python/tf_tfl_flatbuffer_helpers.cc:365] Ignored drop_control_dependency.
2022-05-31 19:59:52.813008: I tensorflow/cc/saved_model/reader.cc:43] Reading SavedModel from: my_dir/new_model
2022-05-31 19:59:52.821533: I tensorflow/cc/saved_model/reader.cc:81] Reading meta graph with tags { serve }
2022-05-31 19:59:52.821554: I tensorflow/cc/saved_model/reader.cc:122] Reading SavedModel debug info (if present) from: my_dir/new_model
2022-05-31 19:59:52.843147: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:354] MLIR V1 optimization pass is not enabled
2022-05-31 19:59:52.853360: I tensorflow/cc/saved_model/loader.cc:228] Restoring SavedModel bundle.
2022-05-31 19:59:53.085311: I tensorflow/cc/saved_model/loader.cc:212] Running initialization op on SavedModel bundle at path: my_dir/new_model
2022-05-31 19:59:53.175132: I tensorflow/cc/saved_model/loader.cc:301] SavedModel load for tags { serve }; Status: success: OK. Took 362134 microseconds.
2022-05-31 19:59:53.306882: I tensorflow/compiler/mlir/tensorflow/utils/dump_mlir_util.cc:263] disabling MLIR crash reproducer, set env var `MLIR_CRASH_REPRODUCER_DIRECTORY` to enable.
TFLite model exported to new_mobile_model.tflite

学習が完了するとclass_labels.txt,new_mobile_model.tflite,my_dir,my_log_dirが出力されます。

tree

(tensorflow-metal)% tree -L 1
.
├── class_labels.txt
├── himawari.jpg
├── label_image.py
├── my_dir
├── my_log_dir
├── new_mobile_model.tflite
└── tensorflow-metal

推論

学習したモデルで推論してみます。画像は適当なお花の画像をご用意ください。
今回はひまわりの画像（himawari.jpg）を使用します。

推論を行うためのPythonスクリプトは以下のコマンドでダウンロードできます。

推論スクリプトダウンロード

(tensorflow-metal)% curl -LO https://raw.githubusercontent.com/tensorflow/tensorflow/master/tensorflow/lite/examples/python/label_image.py

推論の実行

(tensorflow-metal)% python label_image.py \
  --input_mean 0 --input_std 255 \
  --model_file new_mobile_model.tflite --label_file class_labels.txt \
  --image ./himawari.jpg                                                                                                            
0.987842: sunflowers
0.005505: daisy
0.004386: roses
0.001776: dandelion
0.000491: tulips
time: 32.995ms

ひまわりが98%という結果になりました。

おまけ

tensorflow-metalを使用して、GPUを使った画像分類も試してみようと思いましたが、エラーが出たので、一旦ここで終了します。

(tensorflow-metal)% pip install tensorflow-metal
Collecting tensorflow-metal
  Downloading tensorflow_metal-0.5.0-cp39-cp39-macosx_11_0_arm64.whl (1.4 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.4/1.4 MB 18.5 MB/s eta 0:00:00
Requirement already satisfied: wheel~=0.35 in ./venv/lib/python3.9/site-packages (from tensorflow-metal) (0.37.1)
Collecting six~=1.15.0
  Using cached six-1.15.0-py2.py3-none-any.whl (10 kB)
Installing collected packages: six, tensorflow-metal
  Attempting uninstall: six
    Found existing installation: six 1.16.0
    Uninstalling six-1.16.0:
      Successfully uninstalled six-1.16.0
Successfully installed six-1.15.0 tensorflow-metal-0.5.0

(tensorflow-metal)% make_image_classifier \
  --tfhub_module https://tfhub.dev/google/tf2-preview/mobilenet_v2/feature_vector/4 \
  --image_size 224 \
  --saved_model_dir my_dir/new_model \
  --labels_output_file class_labels.txt \
  --tflite_output_file new_mobile_model.tflite \
  --summaries_dir my_log_dir
I0531 20:37:24.875428 4333831552 resolver.py:105] Using /var/folders/_6/1pgtbjws1_z0l934qt03z5sw0000gn/T/tfhub_modules to cache modules.
Metal device set to: Apple M1

systemMemory: 16.00 GB
maxCacheSize: 5.33 GB

2022-05-31 20:37:25.495737: I tensorflow/core/common_runtime/pluggable_device/pluggable_device_factory.cc:305] Could not identify NUMA node of platform GPU ID 0, defaulting to 0. Your kernel may not have been built with NUMA support.
2022-05-31 20:37:25.495872: I tensorflow/core/common_runtime/pluggable_device/pluggable_device_factory.cc:271] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 0 MB memory) -> physical PluggableDevice (device: 0, name: METAL, pci bus id: <undefined>)
Using module https://tfhub.dev/google/tf2-preview/mobilenet_v2/feature_vector/4 with image size (224, 224)
Found 3670 files belonging to 5 classes.
Using 2936 files for training.
Found 3670 files belonging to 5 classes.
Using 734 files for validation.
Found 5 classes: daisy, dandelion, roses, sunflowers, tulips
Model: "sequential_1"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 keras_layer (KerasLayer)    (None, 1280)              2257984   
                                                                 
 dropout (Dropout)           (None, 1280)              0         
                                                                 
 dense (Dense)               (None, 5)                 6405      
                                                                 
=================================================================
Total params: 2,264,389
Trainable params: 6,405
Non-trainable params: 2,257,984
_________________________________________________________________
None
~/workspace/qiita/image_classfication/venv/lib/python3.9/site-packages/keras/optimizers/optimizer_v2/gradient_descent.py:108: UserWarning: The `lr` argument is deprecated, use `learning_rate` instead.
  super(SGD, self).__init__(name, **kwargs)
2022-05-31 20:37:27.160845: W tensorflow/core/platform/profile_utils/cpu_utils.cc:128] Failed to get CPU frequency: 0 Hz
Epoch 1/5
2022-05-31 20:37:28.775242: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:113] Plugin optimizer for device_type GPU is enabled.
91/91 [==============================] - ETA: 0s - loss: 0.9849 - accuracy: 0.74002022-05-31 20:38:09.551583: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:113] Plugin optimizer for device_type GPU is enabled.

(...省略...)

tensorflow.python.framework.errors_impl.InvalidArgumentError: Multiple Default OpKernel registrations match NodeDef '{{node ZerosLike}}': 'op: "ZerosLike" device_type: "DEFAULT" constraint { name: "T" allowed_values { list { type: DT_INT32 } } } host_memory_arg: "y"' and 'op: "ZerosLike" device_type: "DEFAULT" constraint { name: "T" allowed_values { list { type: DT_INT32 } } } host_memory_arg: "y"' [Op:ZerosLike]

During handling of the above exception, another exception occurred:

(...省略...)

AttributeError: 'range_iterator' object has no attribute 'throw'