Cuckoo Sandboxをセットアップしようとしてpip
でmaecのバージョン4.0.1.0
をインストールしようとしたらエラーが出たのでその調査。
試行錯誤した結果なので、他に正しい方法が絶対あると思うので誰か教えてくださいませ。
(※追記しました。)
環境
- OS
- Ubuntu 14.04
- maec
- 4.0.1.0
エラー内容
$ sudo pip install maec==4.0.1.0
Downloading/unpacking maec==4.0.1.0
Downloading maec-4.0.1.0.tar.gz (114kB): 114kB downloaded
Running setup.py (path:/tmp/pip_build_root/maec/setup.py) egg_info for package maec
Traceback (most recent call last):
File "<string>", line 17, in <module>
File "/tmp/pip_build_root/maec/setup.py", line 1, in <module>
import maec
File "maec/__init__.py", line 6, in <module>
import bindings.maec_bundle as bundle_binding
File "maec/bindings/maec_bundle.py", line 12, in <module>
from cybox.bindings import cybox_core
ImportError: No module named cybox.bindings
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "<string>", line 17, in <module>
File "/tmp/pip_build_root/maec/setup.py", line 1, in <module>
import maec
File "maec/__init__.py", line 6, in <module>
import bindings.maec_bundle as bundle_binding
File "maec/bindings/maec_bundle.py", line 12, in <module>
from cybox.bindings import cybox_core
ImportError: No module named cybox.bindings
----------------------------------------
Cleaning up...
Command python setup.py egg_info failed with error code 1 in /tmp/pip_build_root/maec
Storing debug log for failure in /home/vagrant/.pip/pip.log
こんな感じのエラーが出ます。
cyboxのインストール
エラー内容を見るとcyboxがないと怒られているようなので、cyboxを入れます。
$ sudo pip install cybox
(中略)
creating build/temp.linux-x86_64-2.7
creating build/temp.linux-x86_64-2.7/src
creating build/temp.linux-x86_64-2.7/src/lxml
x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/include/libxml2 -I/tmp/pip_build_root/lxml/src/lxml/includes -I/usr/include/python2.7 -c src/lxml/lxml.etree.c -o build/temp.linux-x86_64-2.7/src/lxml/lxml.etree.o -w
x86_64-linux-gnu-gcc: internal compiler error: Killed (program cc1)
Please submit a full bug report,
with preprocessed source if appropriate.
See <file:///usr/share/doc/gcc-4.8/README.Bugs> for instructions.
error: command 'x86_64-linux-gnu-gcc' failed with exit status 4
----------------------------------------
Cleaning up...
Command /usr/bin/python -c "import setuptools, tokenize;__file__='/tmp/pip_build_root/lxml/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-unrw2N-record/install-record.txt --single-version-externally-managed --compile failed with error code 1 in /tmp/pip_build_root/lxml
Storing debug log for failure in /home/vagrant/.pip/pip.log
エラーが出ます。次から次へとエラーが出て心が折れそうになりますが、公式のドキュメントを見てみます。
http://cybox.readthedocs.org/en/latest/installation.html
いくつか依存しているライブラリがあるようです。
それを入れれば解決しそうです。
$ sudo apt-get install libxml2-dev libxslt1-dev zlib1g-dev
そして再びインストール。
$ sudo pip install cybox
(中略)
creating build/temp.linux-x86_64-2.7
creating build/temp.linux-x86_64-2.7/src
creating build/temp.linux-x86_64-2.7/src/lxml
x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/include/libxml2 -I/tmp/pip_build_root/lxml/src/lxml/includes -I/usr/include/python2.7 -c src/lxml/lxml.etree.c -o build/temp.linux-x86_64-2.7/src/lxml/lxml.etree.o -w
x86_64-linux-gnu-gcc: internal compiler error: Killed (program cc1)
Please submit a full bug report,
with preprocessed source if appropriate.
See <file:///usr/share/doc/gcc-4.8/README.Bugs> for instructions.
error: command 'x86_64-linux-gnu-gcc' failed with exit status 4
----------------------------------------
Cleaning up...
Command /usr/bin/python -c "import setuptools, tokenize;__file__='/tmp/pip_build_root/lxml/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-unrw2N-record/install-record.txt --single-version-externally-managed --compile failed with error code 1 in /tmp/pip_build_root/lxml
Storing debug log for failure in /home/vagrant/.pip/pip.log
エラー変わってないじゃん!
どうやらlxml
がうまく行ってなさそう、とあたりを付けて今度はlxml
のインストールを試してみます。
lxmlのインストール
pip
でインストールできそうなのでやってみます。
$ sudo pip install lxml
(中略)
creating build/temp.linux-x86_64-2.7
creating build/temp.linux-x86_64-2.7/src
creating build/temp.linux-x86_64-2.7/src/lxml
x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/include/libxml2 -I/tmp/pip_build_root/lxml/src/lxml/includes -I/usr/include/python2.7 -c src/lxml/lxml.etree.c -o build/temp.linux-x86_64-2.7/src/lxml/lxml.etree.o -w
x86_64-linux-gnu-gcc: internal compiler error: Killed (program cc1)
Please submit a full bug report,
with preprocessed source if appropriate.
See <file:///usr/share/doc/gcc-4.8/README.Bugs> for instructions.
error: command 'x86_64-linux-gnu-gcc' failed with exit status 4
----------------------------------------
Cleaning up...
Command /usr/bin/python -c "import setuptools, tokenize;__file__='/tmp/pip_build_root/lxml/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-7X8_bC-record/install-record.txt --single-version-externally-managed --compile failed with error code 1 in /tmp/pip_build_root/lxml
Storing debug log for failure in /home/vagrant/.pip/pip.log
なんか同じっぽいエラーが出るので、やはりlxmlが原因だったぽいです。
一応公式ドキュメント(http://lxml.de/installation.html) を見ていると、ビルドを早めるためにCの最適化をオフにするやり方があったので、念のため試してみると。。
vagrant@vagrant-ubuntu-trusty-64:~$ sudo CFLAGS="-O0" pip install lxml
Downloading/unpacking lxml
Downloading lxml-3.4.0.tar.gz (3.5MB): 3.5MB downloaded
Running setup.py (path:/tmp/pip_build_root/lxml/setup.py) egg_info for package lxml
/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'bugtrack_url'
warnings.warn(msg)
Building lxml version 3.4.0.
Building without Cython.
Using build configuration of libxslt 1.1.28
warning: no previously-included files found matching '*.py'
Installing collected packages: lxml
Running setup.py install for lxml
/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'bugtrack_url'
warnings.warn(msg)
Building lxml version 3.4.0.
Building without Cython.
Using build configuration of libxslt 1.1.28
building 'lxml.etree' extension
x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -O0 -fPIC -I/usr/include/libxml2 -I/tmp/pip_build_root/lxml/src/lxml/includes -I/usr/include/python2.7 -c src/lxml/lxml.etree.c -o build/temp.linux-x86_64-2.7/src/lxml/lxml.etree.o -w
x86_64-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -D_FORTIFY_SOURCE=2 -g -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -O0 build/temp.linux-x86_64-2.7/src/lxml/lxml.etree.o -lxslt -lexslt -lxml2 -lz -lm -o build/lib.linux-x86_64-2.7/lxml/etree.so
building 'lxml.objectify' extension
x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -O0 -fPIC -I/usr/include/libxml2 -I/tmp/pip_build_root/lxml/src/lxml/includes -I/usr/include/python2.7 -c src/lxml/lxml.objectify.c -o build/temp.linux-x86_64-2.7/src/lxml/lxml.objectify.o -w
x86_64-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -D_FORTIFY_SOURCE=2 -g -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -O0 build/temp.linux-x86_64-2.7/src/lxml/lxml.objectify.o -lxslt -lexslt -lxml2 -lz -lm -o build/lib.linux-x86_64-2.7/lxml/objectify.so
Successfully installed lxml
Cleaning up...
いけたっぽい。。
最適化するところでコケていたのだろうか。
詳細を調べたいのですが、疲れたので誰かお願いします。
cyboxのインストール(再チャレンジ)
$ sudo pip install cybox
Requirement already satisfied (use --upgrade to upgrade): cybox in /usr/local/lib/python2.7/dist-packages
Requirement already satisfied (use --upgrade to upgrade): lxml>=2.3 in /usr/local/lib/python2.7/dist-packages (from cybox)
Downloading/unpacking python-dateutil (from cybox)
Downloading python-dateutil-2.2.tar.gz (259kB): 259kB downloaded
Running setup.py (path:/tmp/pip_build_root/python-dateutil/setup.py) egg_info for package python-dateutil
Requirement already satisfied (use --upgrade to upgrade): six in /usr/lib/python2.7/dist-packages (from python-dateutil->cybox)
Installing collected packages: python-dateutil
Running setup.py install for python-dateutil
Successfully installed python-dateutil
Cleaning up...
やっといけた。
maecのインストール(再チャレンジ)
これでいけるはず!
$ sudo pip install maec==4.0.1.0
Downloading/unpacking maec==4.0.1.0
Downloading maec-4.0.1.0.tar.gz (114kB): 114kB downloaded
Running setup.py (path:/tmp/pip_build_root/maec/setup.py) egg_info for package maec
Traceback (most recent call last):
File "<string>", line 17, in <module>
File "/tmp/pip_build_root/maec/setup.py", line 1, in <module>
import maec
File "maec/__init__.py", line 6, in <module>
import bindings.maec_bundle as bundle_binding
File "maec/bindings/maec_bundle.py", line 5424, in <module>
'Relationships': cybox_core.RelationshipsType,
AttributeError: 'module' object has no attribute 'RelationshipsType'
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "<string>", line 17, in <module>
File "/tmp/pip_build_root/maec/setup.py", line 1, in <module>
import maec
File "maec/__init__.py", line 6, in <module>
import bindings.maec_bundle as bundle_binding
File "maec/bindings/maec_bundle.py", line 5424, in <module>
'Relationships': cybox_core.RelationshipsType,
AttributeError: 'module' object has no attribute 'RelationshipsType'
----------------------------------------
Cleaning up...
Command python setup.py egg_info failed with error code 1 in /tmp/pip_build_root/maec
Storing debug log for failure in /home/vagrant/.pip/pip.log
違うエラー。。もう無理だ。。
自分を励まし何とか調べてみたところ、どうやらcybox
のバージョン違いのようです。
v2.0.1.4を落としてきたらいけるとのこと。
cyboxのインストール(再々チャレンジ)
$ wget https://github.com/CybOXProject/python-cybox/archive/v2.0.1.4.tar.gz
$ tar zxvf v2.0.1.4.tar.gz
$ cd python-cybox-2.0.1.4/
$ sudo python setup.py install
(中略)
Installed /usr/local/lib/python2.7/dist-packages/cybox-2.0.1.4-py2.7.egg
Processing dependencies for cybox==2.0.1.4
Searching for python-dateutil==2.2
Best match: python-dateutil 2.2
Processing python_dateutil-2.2-py2.7.egg
python-dateutil 2.2 is already the active version in easy-install.pth
Using /usr/local/lib/python2.7/dist-packages/python_dateutil-2.2-py2.7.egg
Searching for lxml==3.4.0
Best match: lxml 3.4.0
Processing lxml-3.4.0-py2.7-linux-x86_64.egg
lxml 3.4.0 is already the active version in easy-install.pth
Using /usr/local/lib/python2.7/dist-packages/lxml-3.4.0-py2.7-linux-x86_64.egg
Searching for six==1.5.2
Best match: six 1.5.2
six 1.5.2 is already the active version in easy-install.pth
Using /usr/lib/python2.7/dist-packages
Finished processing dependencies for cybox==2.0.1.4
maecのインストール(再々チャレンジ)
$ sudo pip install maec==4.0.1.0
Downloading/unpacking maec==4.0.1.0
Downloading maec-4.0.1.0.tar.gz (114kB): 114kB downloaded
Running setup.py (path:/tmp/pip_build_root/maec/setup.py) egg_info for package maec
Requirement already satisfied (use --upgrade to upgrade): lxml>=2.3 in /usr/local/lib/python2.7/dist-packages/lxml-3.4.0-py2.7-linux-x86_64.egg (from maec==4.0.1.0)
Requirement already satisfied (use --upgrade to upgrade): cybox>=2.0.1.0,<2.0.2.0 in /usr/local/lib/python2.7/dist-packages/cybox-2.0.1.4-py2.7.egg (from maec==4.0.1.0)
Requirement already satisfied (use --upgrade to upgrade): python-dateutil in /usr/local/lib/python2.7/dist-packages/python_dateutil-2.2-py2.7.egg (from cybox>=2.0.1.0,<2.0.2.0->maec==4.0.1.0)
Requirement already satisfied (use --upgrade to upgrade): six in /usr/lib/python2.7/dist-packages (from python-dateutil->cybox>=2.0.1.0,<2.0.2.0->maec==4.0.1.0)
Installing collected packages: maec
Running setup.py install for maec
Successfully installed maec
Cleaning up...
おぉ。。ついにインストールできた。。
ということでようやく出来ました!
まとめ
試行錯誤の過程を長々と書きましたが、結局まとめると以下の順番です。
- lxmlを入れる
- cybox(v2.0.1.4)を入れる
- maec(4.0.1.0)を入れる
#lxmlが依存するライブラリを入れる
$ sudo apt-get install libxml2-dev libxslt1-dev zlib1g-dev
#lxmlを入れる
$ sudo CFLAGS="-O0" pip install lxml
#cyboxを入れる
$ wget https://github.com/CybOXProject/python-cybox/archive/v2.0.1.4.tar.gz
$ tar zxvf v2.0.1.4.tar.gz
$ cd python-cybox-2.0.1.4/
$ sudo python setup.py install
#maecを入れる
$ sudo pip install maec==4.0.1.0
追記
lxmlはapt-getで入れられるようです。
$ sudo apt-get install python-lxml
これを用いて、上のコマンドを修正すると、
#python-lxmlだけでいいかも
$ sudo apt-get install libxml2-dev libxslt1-dev zlib1g-dev python-lxml
#cyboxを入れる
$ wget https://github.com/CybOXProject/python-cybox/archive/v2.0.1.4.tar.gz
$ tar zxvf v2.0.1.4.tar.gz
$ cd python-cybox-2.0.1.4/
$ sudo python setup.py install
#maecを入れる
$ sudo pip install maec==4.0.1.0
大分簡略化されました。