Go to Qiita Advent Calendar 2024 Top
LoginSignup
7
0

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?

More than 1 year has passed since last update.

OpenCVAdvent Calendar 2021

Day 7
@hon_no_mushi

「アイエエエエ! ニンジャ!? ニンジャナンデ!?」(OpenCV module開発のおともに)

Last updated at Posted at 2021-12-06

■ 「アイエエエエ! ニンジャ!? ニンジャナンデ!?」(OpenCV module開発のおともに)

大規模になりつつあるOpenCVのコンパイル。「遅い」「待たされる」といった印象があるかもしれない。

特に、上手くアプリケーションソフトが動かない場合には、
OpenCVのライブラリ側にもデバッグログや試作コードを仕込んだりもしたくなるはず。

「何度も何度も、OpenCV Moduleをコンパイルしたいが、そのために待たされるのは嫌！」

こんな時に手助けになるツールとして、Ninja を紹介したい。

■ What is Ninja ?

Ninja is a small build system with a focus on speed. It differs from other build systems in two major respects: it is designed to have its input files generated by a higher-level build system, and it is designed to run builds as fast as possible.

（和訳）Ninja とは スピードに焦点を当てた、small build systemである。他のbuild systemと比べて、２つの方向性が異なっている。これは、higher-level build systemによって生成された入力ファイルのために設計されており、そして、できる限り高速にbuildすることを目的に設計されている。

（makeコマンドを使って並列化オプションを適切に設定できていると、それと比較すると）
動かしてみた感じ、初回のコンパイルから爆速になる！わけではない。
地味に、２回目、３回目のコンパイル時に、ちょっとだけ早くなる、そんな縁の下の力持ち的なツール。

■テスト環境

◯ハードウェア

  • intel 4770K 3.2GHz 4C8T 1
  • 8GB Memory

◯ソフトウェア

Ubuntu-21.10 OpenCV-4.5.4 OpenCV_CONTRIB-4.5.4 GCC-11.2.0

■ OpenCVのbuildにNinjaを使うには？

いつものbuildだとこんな感じだろうか？

いつもの平凡なbuild方法
$ cmake -S opencv-4.5.4 -B build_make  -DOPENCV_EXTRA_MODULES_PATH=opencv_contrib-4.5.4/modules/
$ cd build_make
$ make -j 8

Ninjaのある生活には、（１）cmakeのオプションに-GNinjaを追加する。
そして、（２）makeの代わりにninjaを実行する、たったこれだけでよい！

Ninjaのある生活
$ apt-get install ninja-build % これは初回だけ
$ $ cmake -S opencv-4.5.4 -B build_ninja -DOPENCV_EXTRA_MODULES_PATH=opencv_contrib-4.5.4/modules/ -GNinja
$ cd build_ninja
$ ninja -j 8

使い方そのものはmakeとそこまで違わない。これならば、明日、いや今日からNinjaれますね！

■テスト方法

下記条件で何度か試して、real時間を比較した。

  • (0) ninja-buildとccacheをあらかじめインストールしておく
  • (1) makeコマンドとninjaコマンド、それぞれのコンパイル環境を作る。
  • (2) 部分更新のテストに、 freetypeモジュールのソースコードをtouchし、リコンパイル対象にする。
  • (2) ccacheのキャッシュをクリアしたり、残したままビルドをして、部分更新に要する時間を比較する。

■実行結果

    1. ccacheを有効にした状態で、1/6程にまで実行時間の短縮を確認できました。(1.2sec -> 0.20 sec)
    1. ccacheなしmakeに比べて、ccacheありninjaは実行時間1/10にまで短縮ができました(2.1sec -> 0.20 sec)
make -j8 make -j8 (ccache) ninja -j8 ninja -j8 (ccache)
全build - 1397.793 - 1353.996
部分更新-1 2.077 1.211 1.326 0.198
部分更新-2 2.051 1.213 1.320 0.195
部分更新-3 2.082 1.206 1.320 0.195

ただし、全体をbuildする時間は多少短くなっている程度。爆速にはならない。

「あれ？コンパイル対象になるモジュールを絞れば、速くなるのでは…」という真理に到達されると、非常に悲しいです。

■まとめ

  • **「Ninja」**は、OpenCVのモジュールをデバッグ・開発する人もちょっとだけ幸せにしてくれます！！
  • ccacheは有効にしておきましょう。

■ Appendix

cmake結果（クリックすると開きます）

◯cmake結果

-- General configuration for OpenCV 4.5.4 =====================================
--   Version control:               unknown
--
--   Extra modules:
--     Location (extra):            /home/kmtr/work/opencv_contrib-4.5.4/modules
--     Version control (extra):     unknown
--
--   Platform:
--     Timestamp:                   2021-12-01T20:37:02Z
--     Host:                        Linux 5.13.0-22-generic x86_64
--     CMake:                       3.18.4
--     CMake generator:             Ninja
--     CMake build tool:            /usr/bin/ninja
--     Configuration:               Release
--
--   CPU/HW features:
--     Baseline:                    SSE SSE2 SSE3
--       requested:                 SSE3
--     Dispatched code generation:  SSE4_1 SSE4_2 FP16 AVX AVX2 AVX512_SKX
--       requested:                 SSE4_1 SSE4_2 AVX FP16 AVX2 AVX512_SKX
--       SSE4_1 (17 files):         + SSSE3 SSE4_1
--       SSE4_2 (2 files):          + SSSE3 SSE4_1 POPCNT SSE4_2
--       FP16 (1 files):            + SSSE3 SSE4_1 POPCNT SSE4_2 FP16 AVX
--       AVX (5 files):             + SSSE3 SSE4_1 POPCNT SSE4_2 AVX
--       AVX2 (32 files):           + SSSE3 SSE4_1 POPCNT SSE4_2 FP16 FMA3 AVX AVX2
--       AVX512_SKX (8 files):      + SSSE3 SSE4_1 POPCNT SSE4_2 FP16 FMA3 AVX AVX2 AVX_512F AVX512_COMMON AVX512_SKX
--
--   C/C++:
--     Built as dynamic libs?:      YES
--     C++ standard:                11
--     C++ Compiler:                /usr/bin/c++  (ver 11.2.0)
--     C++ flags (Release):         -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Wsuggest-override -Wno-delete-non-virtual-dtor -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -Wno-long-long -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections  -msse -msse2 -msse3 -fvisibility=hidden -fvisibility-inlines-hidden -O3 -DNDEBUG  -DNDEBUG
--     C++ flags (Debug):           -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Wsuggest-override -Wno-delete-non-virtual-dtor -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -Wno-long-long -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections  -msse -msse2 -msse3 -fvisibility=hidden -fvisibility-inlines-hidden -g  -O0 -DDEBUG -D_DEBUG
--     C Compiler:                  /usr/bin/cc
--     C flags (Release):           -fsigned-char -W -Wall -Werror=return-type -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wuninitialized -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -Wno-long-long -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections  -msse -msse2 -msse3 -fvisibility=hidden -O3 -DNDEBUG  -DNDEBUG
--     C flags (Debug):             -fsigned-char -W -Wall -Werror=return-type -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wuninitialized -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -Wno-long-long -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections  -msse -msse2 -msse3 -fvisibility=hidden -g  -O0 -DDEBUG -D_DEBUG
--     Linker flags (Release):      -Wl,--exclude-libs,libippicv.a -Wl,--exclude-libs,libippiw.a   -Wl,--gc-sections -Wl,--as-needed
--     Linker flags (Debug):        -Wl,--exclude-libs,libippicv.a -Wl,--exclude-libs,libippiw.a   -Wl,--gc-sections -Wl,--as-needed
--     ccache:                      YES
--     Precompiled headers:         NO
--     Extra dependencies:          dl m pthread rt
--     3rdparty dependencies:
--
--   OpenCV modules:
--     To be built:                 alphamat aruco barcode bgsegm bioinspired calib3d ccalib 
                                    core datasets dnn dnn_objdetect dnn_superres dpm face features2d 
                                    flann freetype fuzzy gapi hdf hfs highgui img_hash imgcodecs 
                                    imgproc intensity_transform line_descriptor mcc ml objdetect optflow 
                                    phase_unwrapping photo plot quality rapid reg rgbd saliency shape 
                                    stereo stitching structured_light superres surface_matching text 
                                    tracking ts video videoio videostab viz wechat_qrcode xfeatures2d 
                                    ximgproc xobjdetect xphoto
--     Disabled:                    world
--     Disabled by dependency:      -
--     Unavailable:                 cudaarithm cudabgsegm cudacodec cudafeatures2d cudafilters 
                                    cudaimgproc cudalegacy cudaobjdetect cudaoptflow cudastereo 
                                    cudawarping cudev cvv java julia matlab ovis python2 python3 sfm
--     Applications:                tests perf_tests apps
--     Applications:                tests perf_tests apps
--     Documentation:               NO
--     Non-free algorithms:         NO
--
--   GUI:                           NONE
--     GTK+:                        NO
--     VTK support:                 YES (ver 9.1.20211127)
--
--   Media I/O:
--     ZLib:                        /usr/lib/x86_64-linux-gnu/libz.so (ver 1.2.11)
--     JPEG:                        /usr/lib/x86_64-linux-gnu/libjpeg.so (ver 80)
--     WEBP:                        /usr/lib/x86_64-linux-gnu/libwebp.so (ver encoder: 0x020e)
--     PNG:                         /usr/lib/x86_64-linux-gnu/libpng.so (ver 1.6.37)
--     TIFF:                        /usr/lib/x86_64-linux-gnu/libtiff.so (ver 42 / 4.3.0)
--     JPEG 2000:                   OpenJPEG (ver 2.3.1)
--     OpenEXR:                     build (ver 2.3.0)
--     HDR:                         YES
--     SUNRASTER:                   YES
--     PXM:                         YES
--     PFM:                         YES
--
--   Video I/O:
--     DC1394:                      NO
--     FFMPEG:                      YES
--       avcodec:                   YES (58.134.100)
--       avformat:                  YES (58.76.100)
--       avutil:                    YES (56.70.100)
--       swscale:                   YES (5.9.100)
--       avresample:                NO
--     GStreamer:                   NO
--     v4l/v4l2:                    YES (linux/videodev2.h)
--
--   Parallel framework:            pthreads
--
--   Trace:                         YES (with Intel ITT)
--
--   Other third-party libraries:
--     Intel IPP:                   2020.0.0 Gold [2020.0.0]
--            at:                   /home/kmtr/work/build_ninja/3rdparty/ippicv/ippicv_lnx/icv
--     Intel IPP IW:                sources (2020.0.0)
--               at:                /home/kmtr/work/build_ninja/3rdparty/ippicv/ippicv_lnx/iw
--     VA:                          NO
--     Lapack:                      NO
--     Eigen:                       YES (ver 3.3.9)
--     Custom HAL:                  NO
--     Protobuf:                    build (3.5.1)
--
--   OpenCL:                        YES (no extra features)
--     Include path:                /home/kmtr/work/opencv-4.5.4/3rdparty/include/opencl/1.2
--     Link libraries:              Dynamic load
--
--   Python (for build):            /usr/bin/python3
--
--   Java:
--     ant:                         NO
--     JNI:                         /usr/lib/jvm/default-java/include /usr/lib/jvm/default-java/include/linux /usr/lib/jvm/default-java/include
--     Java wrappers:               NO
--     Java tests:                  NO
--
--   Install to:                    /usr/local
-- -----------------------------------------------------------------
--
-- Configuring done
-- Generating done
-- Build files have been written to: /home/kmtr/work/build_ninja
実行結果詳細（クリックすると開きます）

◯実行結果詳細

kmtr@kmtr-virtual-machine:~/work/build_make$ ccache -C
Clearing... 100.0% [===================================================================================================]
kmtr@kmtr-virtual-machine:~/work/build_make$ time make -j 8 > /dev/null

real    23m17.793s
user    167m18.139s
sys     13m39.182s

kmtr@kmtr-virtual-machine:~/work/build_make$ ccache -C && touch ../opencv_contrib-4.5.4/modules/freetype/src/freetype.cpp && time make -j 8 > /dev/null
Clearing... 100.0% [===================================================================================================]

real    0m2.077s
user    0m5.312s
sys     0m2.138s
kmtr@kmtr-virtual-machine:~/work/build_make$ ccache -C && touch ../opencv_contrib-4.5.4/modules/freetype/src/freetype.cpp && time make -j 8 > /dev/null
Clearing... 100.0% [===================================================================================================]

real    0m2.051s
user    0m5.292s
sys     0m2.183s
kmtr@kmtr-virtual-machine:~/work/build_make$ ccache -C && touch ../opencv_contrib-4.5.4/modules/freetype/src/freetype.cpp && time make -j 8 > /dev/null
Clearing... 100.0% [===================================================================================================]

real    0m2.082s
user    0m5.260s
sys     0m2.255s
kmtr@kmtr-virtual-machine:~/work/build_make$ touch ../opencv_contrib-4.5.4/modules/freetype/src/freetype.cpp && time make -j 8 > /dev/null

real    0m1.211s
user    0m3.935s
sys     0m2.036s
kmtr@kmtr-virtual-machine:~/work/build_make$ touch ../opencv_contrib-4.5.4/modules/freetype/src/freetype.cpp && time make -j 8 > /dev/null

real    0m1.213s
user    0m4.092s
sys     0m1.883s
kmtr@kmtr-virtual-machine:~/work/build_make$ touch ../opencv_contrib-4.5.4/modules/freetype/src/freetype.cpp && time make -j 8 > /dev/null

real    0m1.206s
user    0m3.993s
sys     0m2.008s

kmtr@kmtr-virtual-machine:~/work/build_make$ ccache -C
Clearing... 100.0% [===================================================================================================]
kmtr@kmtr-virtual-machine:~/work/build_make$ time make -j 8 > /dev/null

real    0m1.209s
user    0m3.895s
sys     0m1.796s
kmtr@kmtr-virtual-machine:~/work/build_make$ ccache -C
Clearing... 100.0% [===================================================================================================]
kmtr@kmtr-virtual-machine:~/work/build_make$ time make -j 8 > /dev/null

real    0m1.183s
user    0m3.774s
sys     0m1.911s
kmtr@kmtr-virtual-machine:~/work/build_make$ ccache -C
Clearing... 100.0% [===================================================================================================]
kmtr@kmtr-virtual-machine:~/work/build_make$ time make -j 8 > /dev/null

real    0m1.193s
user    0m3.962s
sys     0m1.770s

kmtr@kmtr-virtual-machine:~/work/build_ninja$ time ninja -j8 > /dev/null

real    22m33.996s
user    165m51.780s
sys     12m25.374s

kmtr@kmtr-virtual-machine:~/work/build_ninja$ touch ../opencv_contrib-4.5.4/modules/freetype/src/freetype.cpp && time ninja -j
8 > /dev/null

real    0m0.198s
user    0m0.149s
sys     0m0.049s
kmtr@kmtr-virtual-machine:~/work/build_ninja$ touch ../opencv_contrib-4.5.4/modules/freetype/src/freetype.cpp && time ninja -j 8 > /dev/null

real    0m0.195s
user    0m0.128s
sys     0m0.068s
kmtr@kmtr-virtual-machine:~/work/build_ninja$ touch ../opencv_contrib-4.5.4/modules/freetype/src/freetype.cpp && time ninja -j 8 > /dev/null

real    0m0.198s
user    0m0.124s
sys     0m0.074s
kmtr@kmtr-virtual-machine:~/work/build_ninja$ ccache -C && touch ../opencv_contrib-4.5.4/modules/freetype/src/freetype.cpp && time ninja -j 8 > /dev/null
Clearing... 100.0% [===================================================================================================]

real    0m1.326s
user    0m1.146s
sys     0m0.179s
kmtr@kmtr-virtual-machine:~/work/build_ninja$ ccache -C && touch ../opencv_contrib-4.5.4/modules/freetype/src/freetype.cpp && time ninja -j 8 > /dev/null
Clearing... 100.0% [===================================================================================================]

real    0m1.320s
user    0m1.154s
sys     0m0.165s
kmtr@kmtr-virtual-machine:~/work/build_ninja$ ccache -C && touch ../opencv_contrib-4.5.4/modules/freetype/src/freetype.cpp && time ninja -j 8 > /dev/null
Clearing... 100.0% [===================================================================================================]

real    0m1.320s
user    0m1.135s
sys     0m0.184s

  1. 今年の冬、ボーナスが万が一間違って出たら買い替えるんだ・・・ ↩

7
0
0

Register as a new user and use Qiita more conveniently

  1. You get articles that match your needs
  2. You can efficiently read back useful information
  3. You can use dark theme
What you can do with signing up
Sign upLogin
7
0

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?