Qiita Teams that are logged in
You are not logged in to any team

Log in to Qiita Team
OrganizationAdvent CalendarQiitadon (β)
Qiita JobsQiita ZineQiita Blog
Help us understand the problem. What is going on with this article?

onnxruntime-gpu のセットアップ

More than 1 year has passed since last update.

ONNX runtime gpu のセットアップ


空の環境でpythonから onnxruntime-gpu を利用して推論することを目指します。


OS Ubuntu 18.04
GPU GeForce GTX 2080Ti


sudo apt-get update
sudo apt-get upgrade

sudo apt-get install -y vim csh flex gfortran libgfortran3 g++ \
                 cmake xorg-dev patch zlib1g-dev libbz2-dev \
                 libboost-all-dev openssh-server libcairo2 \
                 libcairo2-dev libeigen3-dev lsb-core \
                 lsb-base net-tools network-manager \
                 git-core git-gui git-doc xclip gdebi-core libffi-dev \
                 make build-essential libssl-dev zlib1g-dev libbz2-dev \
                 libreadline-dev libsqlite3-dev wget curl llvm \
                 libncurses5-dev libncursesw5-dev \
                 xz-utils tk-dev libffi-dev liblzma-dev python-openssl

Nvidia-driver, CUDA のインストール

▼TensorFlowのGPU supportページは更新が早く精度が高めです。

~/.bashrc などに以下を記載

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda/extras/CUPTI/lib64
export PATH="/usr/local/cuda/bin:$PATH"
export LD_LIBRARY_PATH="/usr/local/cuda/lib64:$LD_LIBRARY_PATH"
# Add NVIDIA package repositories
wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/cuda-repo-ubuntu1804_10.1.243-1_amd64.deb
sudo dpkg -i cuda-repo-ubuntu1804_10.1.243-1_amd64.deb
sudo apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/7fa2af80.pub
sudo apt-get update
wget http://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1804/x86_64/nvidia-machine-learning-repo-ubuntu1804_1.0.0-1_amd64.deb
sudo apt install ./nvidia-machine-learning-repo-ubuntu1804_1.0.0-1_amd64.deb
sudo apt-get update

# Install NVIDIA driver
sudo apt-get install --no-install-recommends nvidia-driver-430
# Reboot. Check that GPUs are visible using the command: nvidia-smi

# Install development and runtime libraries (~4GB)
sudo apt-get install --no-install-recommends \
    cuda-10-1 \
    libcudnn7=  \

# Install TensorRT. Requires that libcudnn7 is installed above.
sudo apt-get install -y --no-install-recommends libnvinfer6=6.0.1-1+cuda10.1 \
    libnvinfer-dev=6.0.1-1+cuda10.1 \


git clone https://github.com/yyuu/pyenv.git ~/.pyenv


$ vim ~/.bashrc
export PYENV_ROOT="$HOME/.pyenv"
export PATH="$PYENV_ROOT/bin:$PATH"
eval "$(pyenv init -)"
$ source ~/.bashrc


pyenv install 3.6.8
pyenv install 3.7.2
pyenv global 3.6.8
pyenv rehash
python -V # 3.6.8


pip install onnxruntime-gpu

を実行して、プログラム中でonnxruntimeをimport すると以下のエラー

    self._handle = _dlopen(self._name, mode)
OSError: libcublas.so.10.0: cannot open shared object file: No such file or directory

onnxruntime-gpu の Requirements は以下の通り

GPU builds require CUDA runtime libraries being installed on the system:

  • Version: CUDA 10.0 and cuDNN 7.6 on Linux
  • cuDNN 7.3 on Windows

    -- Older ONNX Runtime releases used CUDA 9.1 and cuDNN 7.1 - please -refer to prior release notes for more details.


確かに10.0 のライブラリはありません

 ls -la /usr/local
drwxr-xr-x 12 root root 4096  2月 21 13:52 .
drwxr-xr-x 11 root root 4096  8月  7  2019 ..
drwxr-xr-x  2 root root 4096  2月 21 13:52 bin
lrwxrwxrwx  1 root root    9  2月 21 13:52 cuda -> cuda-10.1
drwxr-xr-x 15 root root 4096  2月 21 13:52 cuda-10.1
drwxr-xr-x  3 root root 4096  2月 21 13:51 cuda-10.2
drwxr-xr-x  2 root root 4096  8月  7  2019 etc
drwxr-xr-x  2 root root 4096  8月  7  2019 games
drwxr-xr-x  2 root root 4096  8月  7  2019 include
drwxr-xr-x  4 root root 4096  2月 21 13:43 lib
lrwxrwxrwx  1 root root    9  2月 21 13:35 man -> share/man
drwxr-xr-x  2 root root 4096  8月  7  2019 sbin
drwxr-xr-x  6 root root 4096  8月  7  2019 share
drwxr-xr-x  2 root root 4096  8月  7  2019 src
ls /usr/local/cuda/lib64
libOpenCL.so                  libnppicom.so.10
libOpenCL.so.1                libnppicom.so.
libOpenCL.so.1.1              libnppicom_static.a
libaccinj64.so                libnppidei.so
libaccinj64.so.10.1           libnppidei.so.10
libaccinj64.so.10.1.243       libnppidei.so.
libcudadevrt.a                libnppidei_static.a
libcudart.so                  libnppif.so
libcudart.so.10.1             libnppif.so.10
libcudart.so.10.1.243         libnppif.so.
libcudart_static.a            libnppif_static.a
libcufft.so                   libnppig.so
libcufft.so.10                libnppig.so.10
libcufft.so.        libnppig.so.
libcufft_static.a             libnppig_static.a
libcufft_static_nocallback.a  libnppim.so
libcufftw.so                  libnppim.so.10
libcufftw.so.10               libnppim.so.
libcufftw.so.       libnppim_static.a
libcufftw_static.a            libnppist.so
libcuinj64.so                 libnppist.so.10
libcuinj64.so.10.1            libnppist.so.
libcuinj64.so.10.1.243        libnppist_static.a
libculibos.a                  libnppisu.so
libcurand.so                  libnppisu.so.10
libcurand.so.10               libnppisu.so.
libcurand.so.       libnppisu_static.a
libcurand_static.a            libnppitc.so
libcusolver.so                libnppitc.so.10
libcusolver.so.10             libnppitc.so.
libcusolver.so.     libnppitc_static.a
libcusolverMg.so              libnpps.so
libcusolverMg.so.10           libnpps.so.10
libcusolverMg.so.   libnpps.so.
libcusolver_static.a          libnpps_static.a
libcusparse.so                libnvToolsExt.so
libcusparse.so.10             libnvToolsExt.so.1
libcusparse.so.     libnvToolsExt.so.1.0.0
libcusparse_static.a          libnvgraph.so
liblapack_static.a            libnvgraph.so.10
libmetis_static.a             libnvgraph.so.10.1.243
libnppc.so                    libnvgraph_static.a
libnppc.so.10                 libnvjpeg.so
libnppc.so.         libnvjpeg.so.10
libnppc_static.a              libnvjpeg.so.
libnppial.so                  libnvjpeg_static.a
libnppial.so.10               libnvrtc-builtins.so
libnppial.so.       libnvrtc-builtins.so.10.1
libnppial_static.a            libnvrtc-builtins.so.10.1.243
libnppicc.so                  libnvrtc.so
libnppicc.so.10               libnvrtc.so.10.1
libnppicc.so.       libnvrtc.so.10.1.243
libnppicc_static.a            stubs

▼NVIDIA 公式から CUDA 10.0 のライブラリをダウンロードしインストール

▼CUDA 10.0 をインストール

sudo apt-get install --no-install-recommends \
    cuda-10-0 \
    libcudnn7=  \


import onnxruntime
model_path = "path/to/onnxfile"
session = onnxruntime.InferenceSession(model_path)


['CUDAExecutionProvider', 'CPUExecutionProvider']
Help us understand the problem. What is going on with this article?
Why not register and get more from Qiita?
  1. We will deliver articles that match you
    By following users and tags, you can catch up information on technical fields that you are interested in as a whole
  2. you can read useful information later efficiently
    By "stocking" the articles you like, you can search right away
クリエイティブ系のAIツールをワンストップで使えるサービス開発をしています。 https://cre8tiveai.com/ 絶賛採用中ですので、興味ある方はFacebookやTwitterでご連絡ください。 Deep Learning の Researcher と MLOps 等をやっています。 画像処理、映像、Computer Visionが多めです


No comments
Sign up for free and join this conversation.
Sign Up
If you already have a Qiita account Login
Help us understand the problem. What is going on with this article?