Ubuntu 18.04にGeForceを積んで遊んでみる（2：Docker/nvidia-dockerで遊ぶ）

前回からの続き。

Dockerで遊ぶ

基本手順はこちらを参照：
Get Docker CE for Ubuntu | Docker Documentation

1 sudo apt-get install apt-transport-https ca-certificates curl gnupg-agent software-properties-common
2 curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
3 sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
4 sudo apt-get install docker-ce docker-ce-cli containerd.io
とやれば、

$ apt-cache madison docker-ce
 docker-ce | 5:18.09.1~3-0~ubuntu-bionic | https://download.docker.com/linux/ubuntu bionic/stable amd64 Packages
 docker-ce | 5:18.09.0~3-0~ubuntu-bionic | https://download.docker.com/linux/ubuntu bionic/stable amd64 Packages
 docker-ce | 18.06.1~ce~3-0~ubuntu | https://download.docker.com/linux/ubuntu bionic/stable amd64 Packages
 docker-ce | 18.06.0~ce~3-0~ubuntu | https://download.docker.com/linux/ubuntu bionic/stable amd64 Packages
 docker-ce | 18.03.1~ce~3-0~ubuntu | https://download.docker.com/linux/ubuntu bionic/stable amd64 Packages

こんな感じで使えるdockerが見えるようになる。
あとは必要に応じてインストールしたりsudo apt-get install docker-ce=5:18.09.1~3-0~ubuntu-bionic docker-ce-cli=5:18.09.1~3-0~ubuntu-bionic containerd.io、実行したりすれば良い。

$ sudo docker run hello-world
Unable to find image 'hello-world:latest' locally
latest: Pulling from library/hello-world
1b930d010525: Pull complete
Digest: sha256:2557e3c07ed1e38f26e389462d03ed943586f744621577a99efb77324b0fe535
Status: Downloaded newer image for hello-world:latest

Hello from Docker!
This message shows that your installation appears to be working correctly.

To generate this message, Docker took the following steps:
 1. The Docker client contacted the Docker daemon.
 2. The Docker daemon pulled the "hello-world" image from the Docker Hub.
    (amd64)
 3. The Docker daemon created a new container from that image which runs the
    executable that produces the output you are currently reading.
 4. The Docker daemon streamed that output to the Docker client, which sent it
    to your terminal.

To try something more ambitious, you can run an Ubuntu container with:
 $ docker run -it ubuntu bash

Share images, automate workflows, and more with a free Docker ID:
 https://hub.docker.com/

For more examples and ideas, visit:
 https://docs.docker.com/get-started/

nvidia-dockerで遊ぶ

本題。
GitHub - NVIDIA/nvidia-docker: Build and run Docker containers leveraging NVIDIA GPUs に従ってみる。

1 curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | sudo apt-key add -
2 distribution=$(. /etc/os-release;echo $ID$VERSION_ID)
3 curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | sudo tee /etc/apt/sources.list.d/nvidia-docker.list
4 sudo apt-get update
5 sudo apt-get install -y nvidia-docker2
6 sudo pkill -SIGHUP dockerd

これでdocker run --runtime=nvidia --rm nvidia/cuda:9.0-base nvidia-smiを叩けば良い……と思いきや、permission denied。

$ docker run --runtime=nvidia --rm nvidia/cuda:9.0-base nvidia-smi
docker: Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Post http://%2Fvar%2Frun%2Fdocker.sock/v1.39/containers/create: dial unix /var/run/docker.sock: connect: permission denied.
See 'docker run --help'.
$ sudo docker run --runtime=nvidia --rm nvidia/cuda:9.0-base nvidia-smi
Unable to find image 'nvidia/cuda:9.0-base' locally
9.0-base: Pulling from nvidia/cuda
7b722c1070cd: Pull complete
5fbf74db61f1: Pull complete
ed41cb72e5c9: Pull complete
7ea47a67709e: Pull complete
35400734fa04: Pull complete
195acf8a5739: Pull complete
127028f911f6: Pull complete
Digest: sha256:157d05b8a9f3a26dce71c9e824d3fab769d77326f471d0143a236c37d278450d
Status: Downloaded newer image for nvidia/cuda:9.0-base
docker: Error response from daemon: OCI runtime create failed: container_linux.go:344: starting container process caused "process_linux.go:424: container init caused \"process_linux.go:407: running prestart hook 1 caused \\\"error running hook: exit status 1, stdout: , stderr: exec command: [/usr/bin/nvidia-container-cli --load-kmods configure --ldconfig=@/sbin/ldconfig.real --device=all --compute --utility --require=cuda>=9.0 --pid=10886 /var/lib/docker/overlay2/86df41b0c70553a9530d4ed7e0ff5774eb653239a80d49657e388ff6077eab33/merged]\\\\nnvidia-container-cli: initialization error: cuda error: no cuda-capable device is detected\\\\n\\\"\"": unknown.

おや？sudoでもダメ？と調べたら、/dev/nv*系のパーミッションとの兼ね合いらしい。
Problems with "no cuda-capable device is detected" after Ubuntu upgrade · Issue #37 · NVIDIA/libnvidia-container · GitHub
/etc/nvidia-container-runtime/config.tomlにuser = "root:vglusers"を書き足したらsudoで実行可能になった。

$ sudo docker run --runtime=nvidia --rm nvidia/cuda:9.0-base nvidia-smi
Tue Feb  5 06:27:05 2019
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 410.48                 Driver Version: 410.48                    |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  GeForce RTX 2080    Off  | 00000000:1C:00.0  On |                  N/A |
| 24%   29C    P8     6W / 215W |    203MiB /  7949MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID   Type   Process name                             Usage      |
|=============================================================================|
+-----------------------------------------------------------------------------+

とりあえずDockerを動かせる環境を作っただけ。
機械学習のサンプルを動かすとか、中身を改造するとか、できたらいいな？（という資料は腐るほどあるハズ）