12
Help us understand the problem. What are the problem?

More than 1 year has passed since last update.

posted at

Raspberry Pi 3にPyTorch v1.3.0を入れる

はじめに

ラズパイでPyTorch動かしたい、そう考える方々はそれなりにいらっしゃると思う。
私もそうだ。遅いとわかっててラズパイでPyTorchを動かすのは男のロマンだ。
aptやpipで簡単にインストールできればいいのに、世の中そう甘くはなかったので、苦労しながら入れた話を書く。

ハード Raspberry Pi 3 Model B
OS Raspbian (buster)
PyTorch v1.3.0a0
torchvision v0.5.0a0

ググれ、話はそれからだ

まずはググった結果から

ググった結果1:野良ビルドを試す

どこかの親切な人がビルドしたwhlファイルがあるのでこれを試す。
pytorch-arm-builds
うまく動かなかったので、これはダメだと判断。

ググった結果2:頑張ってソースからビルドする

ありがたいことにそれっぽい記事があった。
RaspberryPi3BモデルにPytorch v1.1.0をインストールする
この手順に従ってビルドを試したが、ちょいちょい手順を変更したので記録として残す。

ビルド&インストール手順

準備

ビルドにふざけた勢いでメモリを使用しやがるので、先人の教えに従いswap領域を2Gに拡張する。
参考:https://qiita.com/nyas/items/f4d0675061ee8cdcc3e7

/etc/dphys-swapfile
# set size to absolute value, leaving empty (default) then uses computed value
#   you most likely don't want this, unless you have an special disk situation
# CONF_SWAPSIZE=100
CONF_SWAPSIZE=2048

下記で反映。

root@k8s03:~# /etc/init.d/dphys-swapfile restart

PyTorchのソースを拾ってビルドする

まずはgit clone

root@k8s03:~# git clone --recursive https://github.com/pytorch/pytorch

ビルドに必要なパッケージを入れる

root@k8s03:~# apt-get install libopenblas-dev cython  libatlas-base-dev m4 libblas-dev python3-dev cmake python3-yaml

パラメータをいろいろ指定

root@k8s03:~# export NO_CUDA=1
root@k8s03:~# export NO_DISTRIBUTED=1
root@k8s03:~# export NO_MKLDNN=1 
root@k8s03:~# export BUILD_TEST=0
root@k8s03:~# export MAX_JOBS=2

ビルド(途中で失敗する)

root@k8s03:~# cd pytorch
root@k8s03:~/pytorch# python3 setup.py build
・・・
[ 45%] Built target sleef
/usr/bin/ld: ../../../lib/libprotobuf.a(arena.cc.o): in function `google::protobuf::internal::ArenaImpl::Init()':
arena.cc:(.text+0x24): undefined reference to `__atomic_fetch_add_8'
collect2: error: ld returned 1 exit status
make[2]: *** [third_party/protobuf/cmake/CMakeFiles/protoc.dir/build.make:87: bin/protoc] Error 1
make[1]: *** [CMakeFiles/Makefile2:241: third_party/protobuf/cmake/CMakeFiles/protoc.dir/all] Error 2
make: *** [Makefile:141: all] Error 2
Traceback (most recent call last):
  File "setup.py", line 756, in <module>
    build_deps()
  File "setup.py", line 321, in build_deps
    cmake=cmake)
  File "/root/pytorch/tools/build_pytorch_libs.py", line 63, in build_caffe2
    cmake.build(my_env)
  File "/root/pytorch/tools/setup_helpers/cmake.py", line 331, in build
    self.run(build_args, my_env)
  File "/root/pytorch/tools/setup_helpers/cmake.py", line 142, in run
    check_call(command, cwd=self.build_dir, env=env)
  File "/usr/lib/python3.7/subprocess.py", line 347, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['cmake', '--build', '.', '--target', 'install', '--config', 'Release', '--', '-j', '4']' returned non-zero exit status 2.

ググった結果、パッチ当てが必要とのこと
https://github.com/pytorch/pytorch/issues/22564

root@k8s03:~/pytorch# git submodule update --remote third_party/protobuf
remote: Enumerating objects: 4, done.
remote: Counting objects: 100% (4/4), done.
remote: Compressing objects: 100% (3/3), done.
remote: Total 4 (delta 1), reused 1 (delta 1), pack-reused 0
Unpacking objects: 100% (4/4), done.
From https://github.com/protocolbuffers/protobuf
   584643971..4702ba904  master     -> origin/master
Submodule path 'third_party/protobuf': checked out '4702ba904318ef5b29a197454c7356bae82d4940'
root@k8s03:~/pytorch# 

気を取り直してもう一度・・・

root@k8s03:~/pytorch# python3 setup.py build
・・・
死ぬほど時間がかかった挙句、OSが固まって死ぬ

作戦変更

CPUが4コアあるからビルドも4つ並列で行ってて、そのせいでCPUもメモリもバカ食いでハングしてしまう。並列実行数減らしたいな、と思ってmakeしてるところ探したら、たまたまpytorch/scripts/build_raspbian.shというスクリプト発見。ありがてえ。
make -j 2make -j 1に変更。(並列数2でもメモリ不足でハングした)

pytorch/scripts/build_raspbian.sh
# Note: while Raspberry pi has 4 cores, running too many builds in parallel may
# cause out of memory errors so we will simply run -j 2 only.
#make -j 2 || exit 1
make -j 1 || exit 1

そしてbuild_raspbian.shを実行。

root@k8s03:~/pytorch/scripts# ./build_raspbian.sh 

5時間ぐらいビルド時間がかかったが、無事ビルド完了。

パッケージにする

python3 setup.py installで入れてる方が多いが、他のラズパイでもすぐに使えるようにパッケージにする。

root@k8s03:~/pytorch# python3 setup.py bdist_wheel
・・・

20分ぐらいかかるが、無事whlパッケージが出来上がる。

root@k8s03:~/pytorch/dist# ls
torch-1.3.0a0+e2515a4-cp37-cp37m-linux_armv7l.whl

インストール

root@k8s03:~/pytorch/dist# pip3 install torch-1.3.0a0+e2515a4-cp37-cp37m-linux_armv7l.whl 
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Processing ./torch-1.3.0a0+e2515a4-cp37-cp37m-linux_armv7l.whl
Installing collected packages: torch
Successfully installed torch-1.3.0a0+e2515a4

確認

root@k8s03:~/pytorch/dist# python3
Python 3.7.3 (default, Apr  3 2019, 05:39:12) 
[GCC 8.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch
>>> torch.__version__    
'1.3.0a0+e2515a4'
>>> 

torchvisionも入れる

root@k8s03:~# git clone https://github.com/pytorch/vision.git
root@k8s03:~# cd vision
root@k8s03:~/vision# python3 setup.py bdist_wheel

ビルドに時間かかるがPyTorchほどではない。
インストールもすんなりと行く。

root@k8s03:~/vision# cd dist/
root@k8s03:~/vision/dist# pip3 install torchvision-0.5.0a0+3394c0f-cp37-cp37m-linux_armv7l.whl 
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Processing ./torchvision-0.5.0a0+3394c0f-cp37-cp37m-linux_armv7l.whl
Requirement already satisfied: numpy in /usr/lib/python3/dist-packages (from torchvision==0.5.0a0+3394c0f) (1.16.2)
Requirement already satisfied: six in /usr/lib/python3/dist-packages (from torchvision==0.5.0a0+3394c0f) (1.12.0)
Requirement already satisfied: torch in /usr/local/lib/python3.7/dist-packages (from torchvision==0.5.0a0+3394c0f) (1.3.0a0+e2515a4)
Requirement already satisfied: pillow>=4.1.1 in /usr/lib/python3/dist-packages (from torchvision==0.5.0a0+3394c0f) (5.4.1)
Installing collected packages: torchvision
Successfully installed torchvision-0.5.0a0+3394c0f
root@k8s03:~/vision/dist# 

動作確認

野良ビルドの人のパッケージは使えなかったが、動作確認用スクリプトは使わせてもらった。
ちゃんと動いて満足満足。

root@k8s03:~/pytorch-arm-builds# python3 inference.py tiger.jpg 
/usr/local/lib/python3.7/dist-packages/torchvision/io/_video_opt.py:17: UserWarning: video reader based on ffmpeg c++ ops not available
  warnings.warn("video reader based on ffmpeg c++ ops not available")
[('tiger, Panthera tigris', 91.03487396240234),
 ('tiger cat', 8.865656852722168),
 ('tabby, tabby cat', 0.01799888163805008),
 ('lynx, catamount', 0.0097279641777277),
 ('cougar, puma, catamount, mountain lion, painter, panther, Felis concolor',
  0.004849130753427744)]
8.845822811126709
[('tiger, Panthera tigris', 91.03487396240234),
 ('tiger cat', 8.865656852722168),
 ('tabby, tabby cat', 0.01799888163805008),
 ('lynx, catamount', 0.0097279641777277),
 ('cougar, puma, catamount, mountain lion, painter, panther, Felis concolor',
  0.004849130753427744)]
8.334195137023926
[('tiger, Panthera tigris', 91.03487396240234),
 ('tiger cat', 8.865656852722168),
 ('tabby, tabby cat', 0.01799888163805008),
 ('lynx, catamount', 0.0097279641777277),
 ('cougar, puma, catamount, mountain lion, painter, panther, Felis concolor',
  0.004849130753427744)]
3.9416961669921875
[('tiger, Panthera tigris', 91.03487396240234),
 ('tiger cat', 8.865656852722168),
 ('tabby, tabby cat', 0.01799888163805008),
 ('lynx, catamount', 0.0097279641777277),
 ('cougar, puma, catamount, mountain lion, painter, panther, Felis concolor',
  0.004849130753427744)]
2.6082096099853516
[('tiger, Panthera tigris', 91.03487396240234),
 ('tiger cat', 8.865656852722168),
 ('tabby, tabby cat', 0.01799888163805008),
 ('lynx, catamount', 0.0097279641777277),
 ('cougar, puma, catamount, mountain lion, painter, panther, Felis concolor',
  0.004849130753427744)]
9.647208452224731
[('tiger, Panthera tigris', 91.03487396240234),
 ('tiger cat', 8.865656852722168),
 ('tabby, tabby cat', 0.01799888163805008),
 ('lynx, catamount', 0.0097279641777277),
 ('cougar, puma, catamount, mountain lion, painter, panther, Felis concolor',
  0.004849130753427744)]
2.0369913578033447
[('tiger, Panthera tigris', 91.03487396240234),
 ('tiger cat', 8.865656852722168),
 ('tabby, tabby cat', 0.01799888163805008),
 ('lynx, catamount', 0.0097279641777277),
 ('cougar, puma, catamount, mountain lion, painter, panther, Felis concolor',
  0.004849130753427744)]
2.741065263748169
[('tiger, Panthera tigris', 91.03487396240234),
 ('tiger cat', 8.865656852722168),
 ('tabby, tabby cat', 0.01799888163805008),
 ('lynx, catamount', 0.0097279641777277),
 ('cougar, puma, catamount, mountain lion, painter, panther, Felis concolor',
  0.004849130753427744)]
5.699397802352905
[('tiger, Panthera tigris', 91.03487396240234),
 ('tiger cat', 8.865656852722168),
 ('tabby, tabby cat', 0.01799888163805008),
 ('lynx, catamount', 0.0097279641777277),
 ('cougar, puma, catamount, mountain lion, painter, panther, Felis concolor',
  0.004849130753427744)]
4.371294736862183
[('tiger, Panthera tigris', 91.03487396240234),
 ('tiger cat', 8.865656852722168),
 ('tabby, tabby cat', 0.01799888163805008),
 ('lynx, catamount', 0.0097279641777277),
 ('cougar, puma, catamount, mountain lion, painter, panther, Felis concolor',
  0.004849130753427744)]
5.568331956863403
root@k8s03:~/pytorch-arm-builds# 
Why not register and get more from Qiita?
  1. We will deliver articles that match you
    By following users and tags, you can catch up information on technical fields that you are interested in as a whole
  2. you can read useful information later efficiently
    By "stocking" the articles you like, you can search right away
Sign upLogin
12
Help us understand the problem. What are the problem?