Raspberry Pi4: OpenVINO ARM plugin

Introduction

2021-03-26時点の内容。 〜OpenVINO 2021.3で簡単にビルドできるようになった〜

openvino_contrib / arm_plugin をビルドした。
Neural Compute Stick 2 (MYRIAD) 同等の速度を期待したんだけど、ARMの速度は全くの期待はずれだった。
Apple Silicon M1 を使ってるんで、ARM の実力を盲信してた。のだなあ。

$ uname -a
Linux raspi4b 5.8.0-1017-raspi #20-Ubuntu SMP PREEMPT Mon Mar 1 14:19:20 UTC 2021 aarch64 aarch64 aarch64 GNU/Linux

$ ./benchmark_app -m yolo-v3-tiny-tf.xml -d MYRIAD
[Step 1/11] Parsing and validating input arguments
[ INFO ] Parsing input parameters
[Step 2/11] Loading Inference Engine
[ INFO ] InferenceEngine:
        API version ............ 2.1
        Build .................. custom_HEAD_18e83a217702c650280c6abfc43f3285a3aadb61
        Description ....... API
[ INFO ] Device info:
        MYRIAD
        myriadPlugin version ......... 2.1
        Build ........... custom_HEAD_18e83a217702c650280c6abfc43f3285a3aadb61

[Step 3/11] Setting device configuration
[ WARNING ] -nstreams default value is determined automatically for MYRIAD device. Although the automatic selection usually provides a reasonable performance, but it still may be non-optimal for some cases, for more information look at README.
[Step 4/11] Reading network files
[ INFO ] Loading network files
[ INFO ] Read network took 424.53 ms
[Step 5/11] Resizing network to match image sizes and given batch
[ INFO ] Network batch size: 1
[Step 6/11] Configuring input of the model
[Step 7/11] Loading the model to the device
[ INFO ] Load network took 2229.08 ms
[Step 8/11] Setting optimal runtime parameters
[Step 9/11] Creating infer requests and filling input blobs with images
[ INFO ] Network input 'image_input' precision U8, dimensions (NCHW): 1 3 416 416
[ WARNING ] No input files were given: all inputs will be filled with random values!
[ INFO ] Infer Request 0 filling
[ INFO ] Fill input 'image_input' with random values (image is expected)
[ INFO ] Infer Request 1 filling
[ INFO ] Fill input 'image_input' with random values (image is expected)
[ INFO ] Infer Request 2 filling
[ INFO ] Fill input 'image_input' with random values (image is expected)
[ INFO ] Infer Request 3 filling
[ INFO ] Fill input 'image_input' with random values (image is expected)
[Step 10/11] Measuring performance (Start inference asynchronously, 4 inference requests, limits: 60000 ms duration)
[ INFO ] First inference took 55.09 ms

[Step 11/11] Dumping statistics report
Count:      2792 iterations
Duration:   60134.43 ms
Latency:    85.97 ms
Throughput: 46.43 FPS

$ ./benchmark_app -m yolo-v3-tiny-tf.xml -d ARM
[Step 1/11] Parsing and validating input arguments
[ INFO ] Parsing input parameters
[Step 2/11] Loading Inference Engine
[ INFO ] InferenceEngine: 
        API version ............ 2.1
        Build .................. custom_HEAD_18e83a217702c650280c6abfc43f3285a3aadb61
        Description ....... API
[ INFO ] Device info: 
        ARM
        armPlugin version ......... 2.1
        Build ........... custom_HEAD_18e83a217702c650280c6abfc43f3285a3aadb61

[Step 3/11] Setting device configuration
[Step 4/11] Reading network files
[ INFO ] Loading network files
[ INFO ] Read network took 58.37 ms
[Step 5/11] Resizing network to match image sizes and given batch
[ INFO ] Network batch size: 1
[Step 6/11] Configuring input of the model
[Step 7/11] Loading the model to the device
[ INFO ] Load network took 143.40 ms
[Step 8/11] Setting optimal runtime parameters
[ WARNING ] Default duration 120 seconds for unknown device 'ARM' is used
[Step 9/11] Creating infer requests and filling input blobs with images
[ INFO ] Network input 'image_input' precision U8, dimensions (NCHW): 1 3 416 416 
[ WARNING ] No input files were given: all inputs will be filled with random values!
[ INFO ] Infer Request 0 filling
[ INFO ] Fill input 'image_input' with random values (image is expected)
[Step 10/11] Measuring performance (Start inference asynchronously, 1 inference requests, limits: 120000 ms duration)
[ INFO ] First inference took 1708.97 ms

[Step 11/11] Dumping statistics report
Count:      401 iterations
Duration:   120373.05 ms
Latency:    299.37 ms
Throughput: 3.33 FPS

Build 2021.3

ソースツリーは、-b 2021.3 使います。

• https://github.com/openvinotoolkit/openvino.git
• https://github.com/openvinotoolkit/openvino_contrib.git

cmake \
    -DCMAKE_INSTALL_PREFIX=/opt/intel/openvino \
    -DCMAKE_BUILD_TYPE=Release \
    -DENABLE_SSE42=OFF \
    -DTHREADING=SEQ \
    -DENABLE_GNA=OFF \
    -DENABLE_PYTHON=ON \
    -DPYTHON_EXECUTABLE=/usr/bin/python3.8 \
    -DPYTHON_LIBRARY=/usr/lib/aarch64-linux-gnu/libpython3.8.so \
    -DPYTHON_INCLUDE_DIR=/usr/include/python3.8 \
    -DIE_EXTRA_MODULES=../../openvino_contrib/modules \
    -DBUILD_java_api=OFF ..

でビルドするだけ。出来上がったlibarmPlugin.soを所定の場所に置きます。

openvino-2021.3/bin/aarch64/Release/lib/libarmPlugin.so /opt/intel/openvino/deployment_tools/inference_engine/lib/aarch64/

Note

Raspberry Pi4 でセルフコンパイルするのはお薦めしません。OpenCVも合わせてビルドするのなら多分８時間かかっても終わるかどうか。ですから。

私は「Raspberry Pi4の開発をApple Silicon (M1) MacBookでやろう。（aarch64編）」環境でビルドしたので、2時間弱だったかな。