openvino_contrib: ARM plugin とは何？
openvino_contribはOpenVINOをARM上で動かすプロジェクトです。まだ対応しているモデルは少ないですが動くことは動きます。スッゲェ遅いけど。
Open Model Zooの以下のデモ、モデルが動きました。
- Crossroad Camera C++ Demo
- person-vehicle-bike-detection-crossroad-0078
- person-attributes-recognition-crossroad-0230
- person-reidentification-retail-0277
- Human Pose Estimation C++ Demo
- human-pose-estimation-0001
ベンチマーク
human-pose-estimation-0001をARMとMYRIADで動かしたものはこちら。
MYRIAD Xは、ARMより13倍も速いですけどね。
$ ./benchmark_app -m human-pose-estimation-0001.xml
[Step 1/11] Parsing and validating input arguments
[ INFO ] Parsing input parameters
[Step 2/11] Loading Inference Engine
[ INFO ] OpenVINO: OpenVINO Runtime version ......... 2022.1.0
[ INFO ] Build ........... custom_HEAD_cdb9bec7210f8c24fde3e416c7ada820faaaa23e
[ INFO ]
[ INFO ] Device info:
[ INFO ] CPU
[ INFO ] openvino_arm_cpu_plugin version ......... 2022.1.0
[ INFO ] Build ........... custom_HEAD_cdb9bec7210f8c24fde3e416c7ada820faaaa23e
[ INFO ]
[ INFO ]
[Step 3/11] Setting device configuration
[ WARNING ] -nstreams default value is determined automatically for CPU device. Although the automatic selection usually provides a reasonable performance, but it still may be non-optimal for some cases, for more information look at README.
[Step 4/11] Reading network files
[ INFO ] Loading network files
[ INFO ] Read network took 79.65 ms
[Step 5/11] Resizing network to match image sizes and given batch
[Step 6/11] Configuring input of the model
[ INFO ] Network batch size: 1
Network inputs:
data (node: data) : u8 / [N,C,H,W]
Network outputs:
Mconv7_stage2_L1 (node: Mconv7_stage2_L1) : f32 / [...]
Mconv7_stage2_L2 (node: Mconv7_stage2_L2) : f32 / [...]
[Step 7/11] Loading the model to the device
[ INFO ] Load network took 160.04 ms
[Step 8/11] Setting optimal runtime parameters
[ INFO ] Device: CPU
[ INFO ] { NETWORK_NAME , human-pose-estimation-0001 }
[ INFO ] { OPTIMAL_NUMBER_OF_INFER_REQUESTS , 4 }
[ INFO ] { NUM_STREAMS , 4 }
[ INFO ] { INFERENCE_NUM_THREADS , 0 }
[Step 9/11] Creating infer requests and preparing input blobs with data
[ WARNING ] No input files were given: all inputs will be filled with random values!
[ INFO ] Test Config 0
[ INFO ] data ([N,C,H,W], u8, {1, 3, 256, 456}, static): random (image is expected)
[Step 10/11] Measuring performance (Start inference asynchronously, 4 inference requests, limits: 60000 ms duration)
[ INFO ] BENCHMARK IS IN INFERENCE ONLY MODE.
[ INFO ] Input blobs will be filled once before performance measurements.
[ INFO ] First inference took 1439.38 ms
[Step 11/11] Dumping statistics report
[ INFO ] Count: 48 iterations
[ INFO ] Duration: 64681.41 ms
[ INFO ] Latency:
[ INFO ] Median: 5345.90 ms
[ INFO ] Average: 5367.20 ms
[ INFO ] Min: 4455.30 ms
[ INFO ] Max: 6981.99 ms
[ INFO ] Throughput: 0.74 FPS
$ ./benchmark_app -m human-pose-estimation-0001.xml -d MYRIAD
[Step 1/11] Parsing and validating input arguments
[ INFO ] Parsing input parameters
[Step 2/11] Loading Inference Engine
[ INFO ] OpenVINO: OpenVINO Runtime version ......... 2022.1.0
[ INFO ] Build ........... custom_HEAD_cdb9bec7210f8c24fde3e416c7ada820faaaa23e
[ INFO ]
[ INFO ] Device info:
[ INFO ] MYRIAD
[ INFO ] openvino_intel_myriad_plugin version ......... 2022.1.0
[ INFO ] Build ........... custom_HEAD_cdb9bec7210f8c24fde3e416c7ada820faaaa23e
[ INFO ]
[ INFO ]
[Step 3/11] Setting device configuration
[ WARNING ] Performance hint was not explicitly specified in command line. Device(MYRIAD) performance hint will be set to THROUGHPUT.
[Step 4/11] Reading network files
[ INFO ] Loading network files
[ INFO ] Read network took 80.21 ms
[Step 5/11] Resizing network to match image sizes and given batch
[Step 6/11] Configuring input of the model
[ INFO ] Network batch size: 1
Network inputs:
data (node: data) : u8 / [N,C,H,W]
Network outputs:
Mconv7_stage2_L1 (node: Mconv7_stage2_L1) : f32 / [...]
Mconv7_stage2_L2 (node: Mconv7_stage2_L2) : f32 / [...]
[Step 7/11] Loading the model to the device
[ INFO ] Load network took 3651.06 ms
[Step 8/11] Setting optimal runtime parameters
[ INFO ] Device: MYRIAD
[ INFO ] { NETWORK_NAME , human-pose-estimation-0001 }
[ INFO ] { OPTIMAL_NUMBER_OF_INFER_REQUESTS , 4 }
[ INFO ] { DEVICE_THERMAL , 47.128 }
[Step 9/11] Creating infer requests and preparing input blobs with data
[ WARNING ] No input files were given: all inputs will be filled with random values!
[ INFO ] Test Config 0
[ INFO ] data ([N,C,H,W], u8, {1, 3, 256, 456}, static): random (image is expected)
[Step 10/11] Measuring performance (Start inference asynchronously, 4 inference requests, limits: 60000 ms duration)
[ INFO ] BENCHMARK IS IN INFERENCE ONLY MODE.
[ INFO ] Input blobs will be filled once before performance measurements.
[ INFO ] First inference took 199.66 ms
[Step 11/11] Dumping statistics report
[ INFO ] Count: 608 iterations
[ INFO ] Duration: 60608.78 ms
[ INFO ] Latency:
[ INFO ] Median: 398.52 ms
[ INFO ] Average: 398.04 ms
[ INFO ] Min: 215.69 ms
[ INFO ] Max: 415.68 ms
[ INFO ] Throughput: 10.03 FPS
ビルド
Raspberry Pi4, Ubuntu21.10, OpenVINO2022.1, NCS2環境構築のページに従ってください。
スクリプトでcmakeを実行して、make installすれば出来上がります。
Ubuntu/openvino_with-arm-plugin.sh