これは何？
2022-04-17時点:
MacBook Pro (M1 Pro), MontereyでOpenVINO ARM plugin が動くよ、しかも MYRIAD X (Neural Compute Stick 2)にも負けてないよ、というお話。（やっと動いたし、意外に速かったので嬉しくなって書きたくなりました）
ベンチマーク
benchmark_appのFP32での実行結果です。M1ではFP16は現状では未対応のようです。
$ build-OpenVINO-2022.1/openvino/bin/arm64/Release/benchmark_app -d CPU -m FP32/yolo-v3-tf.xml
[Step 1/11] Parsing and validating input arguments
[ INFO ] Parsing input parameters
[Step 2/11] Loading Inference Engine
[ INFO ] OpenVINO: OpenVINO Runtime version ......... 2022.1.0
[ INFO ] Build ........... custom_HEAD_cdb9bec7210f8c24fde3e416c7ada820faaaa23e
[ INFO ]
[ INFO ] Device info:
[ INFO ] CPU
[ INFO ] openvino_arm_cpu_plugin version ......... 2022.1.0
[ INFO ] Build ........... custom_HEAD_cdb9bec7210f8c24fde3e416c7ada820faaaa23e
[ INFO ]
[ INFO ]
[Step 3/11] Setting device configuration
[ WARNING ] -nstreams default value is determined automatically for CPU device. Although the automatic selection usually provides a reasonable performance, but it still may be non-optimal for some cases, for more information look at README.
[Step 4/11] Reading network files
[ INFO ] Loading network files
[ INFO ] Read network took 84.57 ms
[Step 5/11] Resizing network to match image sizes and given batch
[ WARNING ] input_1: layout is not set explicitly, so it is defaulted to NCHW. It is STRONGLY recommended to set layout manually to avoid further issues.
[Step 6/11] Configuring input of the model
[ INFO ] Network batch size: 1
Network inputs:
input_1 (node: input_1) : u8 / [N,C,H,W]
Network outputs:
conv2d_58/Conv2D/YoloRegion (node: conv2d_58/Conv2D/YoloRegion) : f32 / [...]
conv2d_66/Conv2D/YoloRegion (node: conv2d_66/Conv2D/YoloRegion) : f32 / [...]
conv2d_74/Conv2D/YoloRegion (node: conv2d_74/Conv2D/YoloRegion) : f32 / [...]
[Step 7/11] Loading the model to the device
[ INFO ] Load network took 15.82 ms
[Step 8/11] Setting optimal runtime parameters
[ INFO ] Device: CPU
[ INFO ] { NETWORK_NAME , yolo-v3-tf }
[ INFO ] { OPTIMAL_NUMBER_OF_INFER_REQUESTS , 5 }
[ INFO ] { NUM_STREAMS , 5 }
[ INFO ] { INFERENCE_NUM_THREADS , 0 }
[Step 9/11] Creating infer requests and preparing input blobs with data
[ WARNING ] No input files were given: all inputs will be filled with random values!
[ INFO ] Test Config 0
[ INFO ] input_1 ([N,C,H,W], u8, {1, 3, 416, 416}, static): random (image is expected)
[Step 10/11] Measuring performance (Start inference asynchronously, 5 inference requests, limits: 60000 ms duration)
[ INFO ] BENCHMARK IS IN INFERENCE ONLY MODE.
[ INFO ] Input blobs will be filled once before performance measurements.
[ INFO ] First inference took 767.14 ms
[Step 11/11] Dumping statistics report
[ INFO ] Count: 340 iterations
[ INFO ] Duration: 60389.01 ms
[ INFO ] Latency:
[ INFO ] Median: 783.28 ms
[ INFO ] Average: 885.71 ms
[ INFO ] Min: 306.94 ms
[ INFO ] Max: 4426.69 ms
[ INFO ] Throughput: 5.63 FPS
ビルド
- macOS Monterey 12.3.1
- Xcode 13.3
- Homebrew
- NCS2を動かす場合は libusb が必要。
- Miniforge
以下の手順でビルドします。
OpenCV
git clone https://github.com/kose/build-OpenVINO.git
cd build-OpenVINO
git checkout 2022.1
git submodule update --init --recursive --depth 1
cd opencv
mkdir build
cd build
../../macOS/opencv.sh
sudo make install
cd /opt/intel
sudo ln -s openvino_2022.1 openvino_2022
OpenVINO
以下のページに従いパッチを当てます。
cd openvino_contrib/modules/arm_plugin/thirdparty/ComputeLibrary && \
wget "https://review.mlplatform.org/changes/ml%2FComputeLibrary~6706/revisions/4/patch?zip" -O patch.zip && \
unzip patch.zip && \
git apply 48f2615.diff
cmake/dependencies.cmakeがarm64に対応していないので、とりあえずfaital-errorを回避します
diff --git a/cmake/dependencies.cmake b/cmake/dependencies.cmake
index e2f941fc..c931867b 100644
--- a/cmake/dependencies.cmake
+++ b/cmake/dependencies.cmake
@@ -154,7 +154,7 @@ if(THREADING STREQUAL "TBB" OR THREADING STREQUAL "TBB_AUTO")
ENVIRONMENT "TBBROOT"
SHA256 "ad9cf52e657660058aa6c6844914bc0fc66241fec89a392d8b79a7ff69c3c7f6")
else()
- message(FATAL_ERROR "TBB is not available on current platform")
+ message("M1 env: TBB is not available on current platform")
endif()
update_deps_cache(TBBROOT "${TBB}" "Path to TBB root folder")
export INTEL_OPENVINO_DIR=/opt/intel/openvino_2022
export OpenCV_DIR=$INTEL_OPENVINO_DIR/extras/opencv/lib/cmake/opencv4
export PYTHONPATH=$INTEL_OPENVINO_DIR/extras/opencv/lib/python3.9/site-packages
cd openvino
mkdir build
cd build
../../macOS/openvino_with-arm-plugin.sh
sudo make install
re-build OpenCV
Inference Engine対応OpenCVをビルドします。
export INTEL_OPENVINO_DIR=/opt/intel/openvino_2022
source $INTEL_OPENVINO_DIR/setupvars.sh
re-build OpenCV
でも残念ながら PyPi openvino-dev がねぇ。
は残念ながら M1 Mac (ARM)では失敗します。私は仕方がないのでIntel Mac miniでモデルのダンロード・コンバートをしています。
dockerで以下のようにx86_64環境を作っても失敗します。（Docker x86_64をqemuではなくRosseta2で動かせばなんとかなりそうだけど面倒なので試してないです）
docker buildx build --platform linux/amd64 ....