TensorFlowの内部構造を知りたいと思ったのでまずはbuildしてみたというお話。基本的にここに書いてある通りに進めれば良いのだけど、protobufのところだけ例外があったのでメモしておく。
環境
- Ubuntu 16.04 on VMware Workstation
- 2 vCPUs
- 8 GB Memory (4 GBだと途中でbuildがとまった..)
- 以下、Python3前提
準備
TensorFlowのbuild toolにBazelを使っているのでインストールしておく。
- https://www.bazel.io/versions/master/docs/install.html#install-on-ubuntu
$ sudo apt install openjdk-8-jdk
$ echo "deb [arch=amd64] http://storage.googleapis.com/bazel-apt stable jdk1.8" | \
sudo tee /etc/apt/sources.list.d/bazel.list
$ curl https://storage.googleapis.com/bazel-apt/doc/apt-key.pub.gpg | \
sudo apt-key add -
$ sudo apt install bazel
$ sudo apt install swig
numpyはpip3でinstall済みと仮定してある。python3-devとかpython3-wheelもapt install済み。
TensorFlowのBuild
git pull済みとする。
$ git log -1
commit 3d35376a66cde4f3e614c746d3c8708d15caa1b5
Merge: aaeb50c 23c1156
Author: Vincent Vanhoucke <vanhoucke@google.com>
Date: Sun Oct 2 15:11:18 2016 -0700
Merge pull request #4716 from ultraklon/patch-1
small typo fix
$ ./configure
~/tensorflow ~/tensorflow
Please specify the location of python. [Default is /usr/bin/python]: /usr/bin/python3
Do you wish to build TensorFlow with Google Cloud Platform support? [y/N] N
No Google Cloud Platform support will be enabled for TensorFlow
Do you wish to build TensorFlow with Hadoop File System support? [y/N] N
No Hadoop File System support will be enabled for TensorFlow
Found possible Python library paths:
/usr/local/lib/python2.7/dist-packages
/usr/lib/python2.7/dist-packages
Please input the desired Python library path to use. Default is [/usr/local/lib/python2.7/dist-packages]
/usr/local/lib/python3.5/dist-packages
Do you wish to build TensorFlow with GPU support? [y/N] N
No GPU support will be enabled for TensorFlow
Configuration finished
Extracting Bazel installation...
.
INFO: Starting clean (this may take a while). Consider using --expunge_async if the clean takes more than several minutes.
.
INFO: All external dependencies fetched successfully.
$ bazel build -c opt //tensorflow/tools/pip_package:build_pip_package
Core i5-4570で大体30分程度かかった。次にdeployだが、pipのパッケージを作るのではなく、ここの記述に従って、今後あれこれしやすいようにする。
$ mkdir _python_build
$ cd _python_build
$ ln -s ../bazel-bin/tensorflow/tools/pip_package/build_pip_package.runfiles/org_tensorflow/* .
$ ln -s ../tensorflow/tools/pip_package/* .
$ sudo python3 setup.py develop
[sudo] password for ohara:
running develop
running egg_info
creating tensorflow.egg-info
writing dependency_links to tensorflow.egg-info/dependency_links.txt
writing entry points to tensorflow.egg-info/entry_points.txt
writing tensorflow.egg-info/PKG-INFO
writing top-level names to tensorflow.egg-info/top_level.txt
writing requirements to tensorflow.egg-info/requires.txt
writing manifest file 'tensorflow.egg-info/SOURCES.txt'
reading manifest file 'tensorflow.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'tensorflow.egg-info/SOURCES.txt'
running build_ext
Creating /usr/local/lib/python3.5/dist-packages/tensorflow.egg-link (link to .)
Adding tensorflow 0.10.0 to easy-install.pth file
Installing tensorboard script to /usr/local/bin
Installed /home/ohara/tensorflow/_python_build
Processing dependencies for tensorflow==0.10.0
Searching for protobuf==3.1.0
Reading https://pypi.python.org/simple/protobuf/
No local packages or working download links found for protobuf==3.1.0
error: Could not find suitable distribution for Requirement.parse('protobuf==3.1.0')
あ、はい、、ってなったのでprotobufをbuildする。
$ sudo apt-get install autoconf automake libtool curl make g++ unzip
$ git clone https://github.com/google/protobuf.git
$ cd protobuf
$ git checkout -B my-v3.1.0 v3.1.0
$ ./autogen.sh
$ CXXFLAGS="-fPIC -g -O2" ./configure
$ make
$ sudo pip3 uninstall protobuf
$ sudo pip3 install dist/protobuf-3.1.0-cp35-cp35m-linux_x86_64.whl
もういちどdeployしてみる。
$ sudo python3 setup.py develop
[sudo] password for ohara:
running develop
running egg_info
writing dependency_links to tensorflow.egg-info/dependency_links.txt
writing top-level names to tensorflow.egg-info/top_level.txt
writing entry points to tensorflow.egg-info/entry_points.txt
writing requirements to tensorflow.egg-info/requires.txt
writing tensorflow.egg-info/PKG-INFO
reading manifest file 'tensorflow.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'tensorflow.egg-info/SOURCES.txt'
running build_ext
Creating /usr/local/lib/python3.5/dist-packages/tensorflow.egg-link (link to .)
tensorflow 0.10.0 is already the active version in easy-install.pth
Installing tensorboard script to /usr/local/bin
Installed /home/ohara/tensorflow/_python_build
Processing dependencies for tensorflow==0.10.0
Searching for wheel==0.29.0
Best match: wheel 0.29.0
Adding wheel 0.29.0 to easy-install.pth file
Installing wheel script to /usr/local/bin
Using /usr/lib/python3/dist-packages
Searching for protobuf==3.1.0
Best match: protobuf 3.1.0
Adding protobuf 3.1.0 to easy-install.pth file
Using /usr/local/lib/python3.5/dist-packages
Searching for six==1.10.0
Best match: six 1.10.0
Adding six 1.10.0 to easy-install.pth file
Using /usr/lib/python3/dist-packages
Searching for numpy==1.11.1
Best match: numpy 1.11.1
Adding numpy 1.11.1 to easy-install.pth file
Using /usr/local/lib/python3.5/dist-packages
Searching for setuptools==28.2.0
Best match: setuptools 28.2.0
Adding setuptools 28.2.0 to easy-install.pth file
Installing easy_install-3.5 script to /usr/local/bin
Installing easy_install script to /usr/local/bin
Using /usr/local/lib/python3.5/dist-packages
Finished processing dependencies for tensorflow==0.10.0
今度は大丈夫そう。/usr/local/lib/python3.5/dist-packages/tensorflow.egg-linkから/home/ohara/tensorflow/_python_buildへのリンクがはられている。
$ pip3 list|grep tensorflow
tensorflow (0.10.0, /home/ohara/tensorflow/_python_build)
次はTensorFlowがLinear Algebra Libraryとして採用しているEigenを調べたい。TensorFlowはSMP CPUに対応しているのだけどそのScalability向上についてはEigenに強く依存している。