Go to Qiita Advent Calendar 2024 Top
LoginSignup
8
12

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?

More than 5 years have passed since last update.

@richi40

データサイエンティストのためのDockerfileを作ります（とりあえず完成）

Last updated at Posted at 2017-01-16

#とりあえず完成しました。

Nginxは全然触ったことがないので、触る機会があれば追加します。

このDockerfileでchainer GPU、Stan、server Juman++、monary mongo等がpythonから直ぐに使えます。

大変でした。色々やりこのDockerfileに落ち着きました。これでOSが立ち上がらなくなっても、直ぐに分析環境を復元できます。

image を docker hubにputしようと思いましたが、imageがデカすぎるのでやめました。

注意点としてはdockerへのディスク割当を多くしなければならないことです。

以下のサイトを参考にすればできると思います。
http://stackoverflow.com/questions/33784295/setting-dns-for-docker-daemon-on-os-with-systemd

/var/lib/dockerの削除は盲点になりやすいので、注意してください。

Dockerfile
# --------------------
# hostos is centos7.3
# --------------------

FROM nvidia/cuda:8.0-cudnn5-devel-centos7

# -----
# user
# -----
#RUN useradd -m plenty
#RUN echo "plenty ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers
WORKDIR /home/plenty
#USER plenty

# -------------------
# packages on centos
# -------------------
RUN yum -y install epel-release && yum clean all
RUN yum -y update && yum clean all
RUN yum -y install wget sudo vim python-pip && yum clean all
RUN pip install --upgrade pip

# ------
# pyenv
# ------
RUN yum -y install gcc gcc-c++ \
                   zlib-devel bzip2 bzip2-devel readline readline-devel \
                   sqlite sqlite-devel openssl openssl-devel git && yum clean all
RUN git clone git://github.com/yyuu/pyenv.git .pyenv
ENV HOME /home/plenty
ENV PYENV_ROOT $HOME/.pyenv
ENV PATH $PYENV_ROOT/shims:$PYENV_ROOT/bin:$PATH

# ---------------
# juman++ server
# ---------------
RUN rpm -ivh http://packages.groonga.org/centos/groonga-release-1.1.0-1.noarch.rpm
RUN yum -y install mecab mecab-devel mecab-ipadic && yum clean all

RUN wget -O juman7.0.1.tar.bz2 "http://nlp.ist.i.kyoto-u.ac.jp/DLcounter/lime.cgi?down=http://nlp.ist.i.kyoto-u.ac.jp/nl-resource/juman/juman-7.01.tar.bz2&name=juman-7.01.tar.bz2"
RUN bzip2 -dc juman7.0.1.tar.bz2  | tar xvf -
RUN cd juman-7.01 && ./configure && make && make install

RUN yum -y install python-devel && yum clean all
RUN wget https://sourceforge.net/projects/boost/files/boost/1.62.0/boost_1_62_0.tar.gz
RUN tar xvzf boost_1_62_0.tar.gz
RUN cd boost_1_62_0 && sh bootstrap.sh && ./b2 install -j2

RUN yum -y install ruby 
RUN wget http://lotus.kuee.kyoto-u.ac.jp/nl-resource/jumanpp/jumanpp-1.01.tar.xz
RUN tar xJvf jumanpp-1.01.tar.xz
RUN cd jumanpp-1.01/ && ./configure && make && make install
RUN sed -i -e "s/\('command'=>'jumanpp',\)/\1 'host'=>'localhost',/g" ./jumanpp-1.01/script/server.rb
RUN echo 'ruby jumanpp-1.01/script/server.rb --cmd "jumanpp -B 5" &' >> .bashrc

ENV LIBRARY_PATH /usr/lib64:$LIBRARY_PATH 
RUN yum -y install make
RUN wget http://www.phontron.com/kytea/download/kytea-0.4.7.tar.gz
RUN tar -xvf kytea-0.4.7.tar.gz
RUN cd kytea-0.4.7 && ./configure && make && make install
RUN pip install kytea

# --------
# mongodb
# --------
RUN echo -e "[mongodb-org-3.4]\nname=MongoDB Repository\nbaseurl=https://repo.mongodb.org/yum/redhat/\$releasever/mongodb-org/3.4/x86_64/\ngpgcheck=1\nenabled=1\ngpgkey=https://www.mongodb.org/static/pgp/server-3.4.asc" > /etc/yum.repos.d/mongodb-org-3.4.repo
RUN yum -y install mongodb-org
RUN mkdir -p data/db
RUN echo '/usr/bin/mongod --dbpath /home/plenty/data/db &' >> .bashrc

# ---------
# anaconda
# ---------
RUN yum -y install libX11-devel libXext-devel libXdmcp-devel
RUN pyenv install anaconda3-4.4.0
RUN pyenv global anaconda3-4.4.0
RUN pyenv rehash
RUN pip install --upgrade pip
RUN sed -i -e "s/backend      : Qt5Agg/#backend      : Qt5Agg/g" .pyenv/versions/anaconda3-4.4.0/lib/python3.6/site-packages/matplotlib/mpl-data/matplotlibrc
RUN sed -i -e "s/#backend.qt4 : PyQt4/backend.qt4 : PyQt4/g" .pyenv/versions/anaconda3-4.4.0/lib/python3.6/site-packages/matplotlib/mpl-data/matplotlibrc
RUN sed -i -e "s/backend      : Qt5Agg/#backend      : Qt5Agg/g" .pyenv/versions/anaconda3-4.4.0/pkgs/matplotlib-2.0.2-np112py36_0/lib/python3.6/site-packages/matplotlib/mpl-data/matplotlibrc
RUN sed -i -e "s/#backend.qt4 : PyQt4/backend.qt4 : PyQt4/g" .pyenv/versions/anaconda3-4.4.0/pkgs/matplotlib-2.0.2-np112py36_0/lib/python3.6/site-packages/matplotlib/mpl-data/matplotlibrc

# ------------------
# modules on python
# ------------------
RUN yum -y install libxml2-devel libffi-devel python-devel libxslt-devel
RUN pip install scrapy
RUN pip install chainer
RUN pip install seaborn
RUN pip install -U scikit-learn
RUN pip install -U pandas
RUN pip install pystan
RUN pip install pymongo
RUN pip install nimfa
RUN pip install keras
RUN pip install tensorflow-gpu
RUN pip install gensim
RUN pip install python-crfsuite
RUN pip install sklearn-crfsuite
RUN pip install fbprophet

RUN wget https://github.com/mongodb/mongo-c-driver/releases/download/1.5.3/mongo-c-driver-1.5.3.tar.gz
RUN tar xzf mongo-c-driver-1.5.3.tar.gz
RUN cd mongo-c-driver-1.5.3 && ./configure && make && make install
RUN yum -y install cyrus-sasl-devel
ENV LD_LIBRARY_PATH /usr/local/lib:$LD_LIBRARY_PATH
RUN pip install monary

# --------------
# gym
# --------------
RUN yum -y groupinstall "GNOME Desktop"
RUN yum -y install cmake
RUN pip install gym
RUN pip install keras-rl

# ------
# mxnet
# ------
ENV LD_LIBRARY_PATH /usr/local/cuda/lib64/:$LD_LIBRARY_PATH
RUN pip install mxnet-cu80==0.11.0

# -----------------
# setting japanese
# -----------------
RUN yum -y reinstall glibc-common && yum clean all
RUN localedef -v -c -i ja_JP -f UTF-8 ja_JP.UTF-8; echo "";
ENV LANG ja_JP.UTF-8
RUN rm -f /etc/localtime
RUN ln -fs /usr/share/zoneinfo/Asia/Tokyo /etc/localtime

RUN pip install JapaneseTokenizer

# ------------------
# entry point
# ------------------
RUN echo 'export HOME=/share' >> .bashrc
RUN echo 'cd $HOME' >> .bashrc
ENTRYPOINT ["/bin/bash"]
create_container.sh
#!/bin/sh

sudo docker rm $(sudo docker ps -a | awk '{print $1}' | grep -v CONTAINER)

sudo nvidia-docker run --privileged=true --rm -e DISPLAY=$DISPLAY -v /tmp/.X11-unix:/tmp/.X11-unix -it --name plenty -v ~/work/plenty:/share:rw plenty
init_docker.sh
#!/bin/sh
sudo docker rm $(sudo docker ps -a | awk '{print $1}' | grep -v CONTAINER)
sudo docker rmi -f $(sudo docker images | awk '{print $3}' | grep -v IMAGE)
run_docker.sh
#!/bin/sh
sudo docker rm $(sudo docker ps -a | awk '{print $1}' | grep -v CONTAINER)
sudo nvidia-docker build -t plenty .
sudo nvidia-docker run --privileged=true --rm -e DISPLAY=$DISPLAY -v /tmp/.X11-unix:/tmp/.X11-unix -it --name plenty -v ~/work/plenty:/share:rw plenty
start_container.sh
#!/bin/sh
sudo docker start plenty
sudo docker attach plenty
8
12
0

Register as a new user and use Qiita more conveniently

  1. You get articles that match your needs
  2. You can efficiently read back useful information
  3. You can use dark theme
What you can do with signing up
Sign upLogin
8
12

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?