openvino_contrib: ARM plugin とは何？

openvino_contribはOpenVINOをARM上で動かすプロジェクトです。まだ対応しているモデルは少ないですが動くことは動きます。スッゲェ遅いけど。

Open Model Zooの以下のデモ、モデルが動きました。

Crossroad Camera C++ Demo
- person-vehicle-bike-detection-crossroad-0078
- person-attributes-recognition-crossroad-0230
- person-reidentification-retail-0277
Human Pose Estimation C++ Demo
- human-pose-estimation-0001

ベンチマーク

human-pose-estimation-0001をARMとMYRIADで動かしたものはこちら。
MYRIAD Xは、ARMより13倍も速いですけどね。

$ ./benchmark_app -m human-pose-estimation-0001.xml
[Step 1/11] Parsing and validating input arguments
[ INFO ] Parsing input parameters
[Step 2/11] Loading Inference Engine
[ INFO ] OpenVINO: OpenVINO Runtime version ......... 2022.1.0
[ INFO ] Build ........... custom_HEAD_cdb9bec7210f8c24fde3e416c7ada820faaaa23e
[ INFO ] 
[ INFO ] Device info: 
[ INFO ] CPU
[ INFO ] openvino_arm_cpu_plugin version ......... 2022.1.0
[ INFO ] Build ........... custom_HEAD_cdb9bec7210f8c24fde3e416c7ada820faaaa23e
[ INFO ] 
[ INFO ] 
[Step 3/11] Setting device configuration
[ WARNING ] -nstreams default value is determined automatically for CPU device. Although the automatic selection usually provides a reasonable performance, but it still may be non-optimal for some cases, for more information look at README.
[Step 4/11] Reading network files
[ INFO ] Loading network files
[ INFO ] Read network took 79.65 ms
[Step 5/11] Resizing network to match image sizes and given batch
[Step 6/11] Configuring input of the model
[ INFO ] Network batch size: 1
Network inputs:
    data (node: data) : u8 / [N,C,H,W]
Network outputs:
    Mconv7_stage2_L1 (node: Mconv7_stage2_L1) : f32 / [...]
    Mconv7_stage2_L2 (node: Mconv7_stage2_L2) : f32 / [...]
[Step 7/11] Loading the model to the device
[ INFO ] Load network took 160.04 ms
[Step 8/11] Setting optimal runtime parameters
[ INFO ] Device: CPU
[ INFO ]   { NETWORK_NAME , human-pose-estimation-0001 }
[ INFO ]   { OPTIMAL_NUMBER_OF_INFER_REQUESTS , 4 }
[ INFO ]   { NUM_STREAMS , 4 }
[ INFO ]   { INFERENCE_NUM_THREADS , 0 }
[Step 9/11] Creating infer requests and preparing input blobs with data
[ WARNING ] No input files were given: all inputs will be filled with random values!
[ INFO ] Test Config 0
[ INFO ] data  ([N,C,H,W], u8, {1, 3, 256, 456}, static):	random (image is expected)
[Step 10/11] Measuring performance (Start inference asynchronously, 4 inference requests, limits: 60000 ms duration)
[ INFO ] BENCHMARK IS IN INFERENCE ONLY MODE.
[ INFO ] Input blobs will be filled once before performance measurements.
[ INFO ] First inference took 1439.38 ms

[Step 11/11] Dumping statistics report
[ INFO ] Count:      48 iterations
[ INFO ] Duration:   64681.41 ms
[ INFO ] Latency: 
[ INFO ] 	Median:     5345.90 ms
[ INFO ] 	Average:    5367.20 ms
[ INFO ] 	Min:        4455.30 ms
[ INFO ] 	Max:        6981.99 ms
[ INFO ] Throughput: 0.74 FPS

$ ./benchmark_app -m human-pose-estimation-0001.xml -d MYRIAD
[Step 1/11] Parsing and validating input arguments
[ INFO ] Parsing input parameters
[Step 2/11] Loading Inference Engine
[ INFO ] OpenVINO: OpenVINO Runtime version ......... 2022.1.0
[ INFO ] Build ........... custom_HEAD_cdb9bec7210f8c24fde3e416c7ada820faaaa23e
[ INFO ] 
[ INFO ] Device info: 
[ INFO ] MYRIAD
[ INFO ] openvino_intel_myriad_plugin version ......... 2022.1.0
[ INFO ] Build ........... custom_HEAD_cdb9bec7210f8c24fde3e416c7ada820faaaa23e
[ INFO ] 
[ INFO ] 
[Step 3/11] Setting device configuration
[ WARNING ] Performance hint was not explicitly specified in command line. Device(MYRIAD) performance hint will be set to THROUGHPUT.
[Step 4/11] Reading network files
[ INFO ] Loading network files
[ INFO ] Read network took 80.21 ms
[Step 5/11] Resizing network to match image sizes and given batch
[Step 6/11] Configuring input of the model
[ INFO ] Network batch size: 1
Network inputs:
    data (node: data) : u8 / [N,C,H,W]
Network outputs:
    Mconv7_stage2_L1 (node: Mconv7_stage2_L1) : f32 / [...]
    Mconv7_stage2_L2 (node: Mconv7_stage2_L2) : f32 / [...]
[Step 7/11] Loading the model to the device
[ INFO ] Load network took 3651.06 ms
[Step 8/11] Setting optimal runtime parameters
[ INFO ] Device: MYRIAD
[ INFO ]   { NETWORK_NAME , human-pose-estimation-0001 }
[ INFO ]   { OPTIMAL_NUMBER_OF_INFER_REQUESTS , 4 }
[ INFO ]   { DEVICE_THERMAL , 47.128 }
[Step 9/11] Creating infer requests and preparing input blobs with data
[ WARNING ] No input files were given: all inputs will be filled with random values!
[ INFO ] Test Config 0
[ INFO ] data  ([N,C,H,W], u8, {1, 3, 256, 456}, static):	random (image is expected)
[Step 10/11] Measuring performance (Start inference asynchronously, 4 inference requests, limits: 60000 ms duration)
[ INFO ] BENCHMARK IS IN INFERENCE ONLY MODE.
[ INFO ] Input blobs will be filled once before performance measurements.
[ INFO ] First inference took 199.66 ms

[Step 11/11] Dumping statistics report
[ INFO ] Count:      608 iterations
[ INFO ] Duration:   60608.78 ms
[ INFO ] Latency: 
[ INFO ] 	Median:     398.52 ms
[ INFO ] 	Average:    398.04 ms
[ INFO ] 	Min:        215.69 ms
[ INFO ] 	Max:        415.68 ms
[ INFO ] Throughput: 10.03 FPS

ビルド

Raspberry Pi4, Ubuntu21.10, OpenVINO2022.1, NCS2環境構築のページに従ってください。

スクリプトでcmakeを実行して、make installすれば出来上がります。

Ubuntu/openvino_with-arm-plugin.sh