Install NVIDIA's driver and CUDA Toolkit 10.2

Install GPU Driver

If use TensorFlow 2, software requirements (RTX Series)

nvidia-driver-440
CUDA Toolkit 10.1
TensorRT 6
cuDNN 7

If do not use TensorFlow 2, software requirements (RTX Series)

nvidia-driver-440
CUDA Toolkit 10.2
TensorRT 7
cuDNN 7

ubuntu-drivers devices
user@host:~$ ubuntu-drivers devices
== /sys/devices/pci0000:00/0000:00:01.0/0000:01:00.0 ==
modalias : pci:v000010DEd00001E84sv000010DEsd0000139Fbc03sc00i00
vendor   : NVIDIA Corporation
driver   : nvidia-driver-430 - distro non-free
driver   : nvidia-driver-435 - distro non-free
driver   : nvidia-driver-440 - third-party free recommended
driver   : xserver-xorg-video-nouveau - distro free builtin

sudo ubuntu-drivers autoinstall

Installed recommended driver.

UEFI Secure Boot

Your system has UEFI Secure Boot enabled.

あなたのシステムでは UEFI Secure Boot が有効なので、サードパーティ製のドライバを利用するために、いくつかの設定が必要です。

新しいMOK（Machine-Owner Key）を作成して、システムのファームウェアに登録せねばなりません。

この操作をするために、今、パスワードを決めてください。システムのリブート時に操作をするときに必要になります。

Enter password and wait for install process.
[IMPORTANT!] If install process has completed, reboot your machine.

Reboot and enroll MOK

1.Enter an any password.
2.Reboot
3.Select Enroll MOK
4.Enter the password.

user@host:~$ lspci | grep -i nvidia
01:00.0 VGA compatible controller: NVIDIA Corporation Device 1e84 (rev a1)
01:00.1 Audio device: NVIDIA Corporation Device 10f8 (rev a1)
01:00.2 USB controller: NVIDIA Corporation Device 1ad8 (rev a1)
01:00.3 Serial bus controller [0c80]: NVIDIA Corporation Device 1ad9 (rev a1)

user@host:~$ gcc --version
gcc (Ubuntu 7.4.0-1ubuntu1~18.04.1) 7.4.0
Copyright (C) 2017 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

user@host:~$ sudo apt-get install linux-headers-$(uname -r)

Cuda toolkit install(v10.2)

wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/cuda-ubuntu1804.pin
sudo mv cuda-ubuntu1804.pin /etc/apt/preferences.d/cuda-repository-pin-600wget http://developer.download.nvidia.com/compute/cuda/10.2/Prod/local_installers/cuda-repo-ubuntu1804-10-2-local-10.2.89-440.33.01_1.0-1_amd64.deb
sudo dpkg -i cuda-repo-ubuntu1804-10-2-local-10.2.89-440.33.01_1.0-1_amd64.deb
sudo apt-key add /var/cuda-repo-10-2-local-10.2.89-440.33.01/7fa2af80.pub
sudo apt-get updatesudo apt-get -y install cuda

Cuda toolkit install(v10.1)

wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/cuda-ubuntu1804.pin
sudo mv cuda-ubuntu1804.pin /etc/apt/preferences.d/cuda-repository-pin-600wget http://developer.download.nvidia.com/compute/cuda/10.1/Prod/local_installers/cuda-repo-ubuntu1804-10-1-local-10.1.243-418.87.00_1.0-1_amd64.deb
sudo dpkg -i cuda-repo-ubuntu1804-10-1-local-10.1.243-418.87.00_1.0-1_amd64.deb
sudo apt-key add /var/cuda-repo-10-1-local-10.1.243-418.87.00/7fa2af80.pub
sudo apt-get updatesudo apt-get -y install cuda

If Cuda toolkit installed (v10.1), the driver version was down grade to v418(RTX2070 Super do not work.)
So, using RTX series reinstall the GPU driver.
When using the apt installation, a v10.2 toolkit ".deb" file is required as a requirement.

Install repository only.

wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/cuda-ubuntu1804.pin
sudo mv cuda-ubuntu1804.pin /etc/apt/preferences.d/cuda-repository-pin-600wget http://developer.download.nvidia.com/compute/cuda/10.2/Prod/local_installers/cuda-repo-ubuntu1804-10-2-local-10.2.89-440.33.01_1.0-1_amd64.deb
sudo dpkg -i cuda-repo-ubuntu1804-10-2-local-10.2.89-440.33.01_1.0-1_amd64.deb
sudo apt-key add /var/cuda-repo-10-2-local-10.2.89-440.33.01/7fa2af80.pub

Then Install the GPU driver.

sudo apt install nvidia-driver-440 (or latest)

Post Install

kernel version hold

nano /etc/apt/apt.conf.d/50unattended-upgrades
Locate the blacklist section, and edit to include the packages - even a regex is supported:

// List of packages to not update (regexp are supported)
Unattended-Upgrade::Package-Blacklist {
        "linux-generic";
        "linux-image-generic";
        "linux-headers-generic";
        "nvidia-driver-440";
        "nvidia-driver"; 
        "cuda";
        "cuda-10-1";
//      "vim";
//      "libc6";
//      "libc6-dev";
//      "libc6-i686";
};

or

sudo apt-mark hold linux-generic linux-image-generic linux-headers-generic

Check toolkit version.

nvcc --version

Add some cuda toolkit's environment variable

export PATH="/usr/local/cuda/bin:$PATH"
export LD_LIBRARY_PATH="/usr/local/cuda/lib64:$LD_LIBRARY_PATH"

check nvidia-persistenced

sudo systemctl status nvidia-persistenced
sudo systemctl enable nvidia-persistenced
and reboot

Build cuda samle

V10.2

cuda-install-samples-10.2.sh <target_path>
cd <target_path>/NVIDIA_CUDA-10.1_Samples
make

or V10.1

cuda-install-samples-10.1.sh <target_path>
cd <target_path>/NVIDIA_CUDA-10.2_Samples
make

Install cuDNN

sudo dpkg -i libcudnn7_7.6.5.32-1+cuda10.2_amd64.deb
sudo dpkg -i libcudnn7-dev_7.6.5.32-1+cuda10.2_amd64.deb
sudo dpkg -i libcudnn7-doc_7.6.5.32-1+cuda10.2_amd64.deb

Test sample.

$cp -r /usr/src/cudnn_samples_v7/ $HOME
Go to the writable path.
$ cd  $HOME/cudnn_samples_v7/mnistCUDNN
Compile the mnistCUDNN sample.
$make clean && make
Run the mnistCUDNN sample.
$ ./mnistCUDNN
If cuDNN is properly installed and running on your Linux system, you will see a message similar to the following:
Test passed!

Install TensorRT

https://docs.nvidia.com/deeplearning/sdk/tensorrt-install-guide/index.html#downloading
https://docs.nvidia.com/deeplearning/sdk/tensorrt-archived/index.html#trt_6
https://docs.nvidia.com/deeplearning/sdk/tensorrt-archived/tensorrt-601/tensorrt-install-guide/index.html

Add local repository. TesnsorRT v7(Without Tensor flow2)

sudo dpkg -i nv-tensorrt-repo-ubuntu1804-cuda10.2-trt7.0.0.11-ga-20191216_1-1_amd64.deb
sudo apt-key add /var/nv-tensorrt-repo-cuda10.2-trt7.0.0.11-ga-20191216/7fa2af80.pub

Add local repository. TensorRT v6(With using Tensor flow2)

sudo dpkg -i nv-tensorrt-repo-ubuntu1804-cuda10.1-trt6.0.1.5-ga-20190913_1-1_amd64.deb
sudo apt-key add /var/nv-tensorrt-repo-cuda10.1-trt6.0.1.5-ga-20190913/7fa2af80.pub

Install

sudo apt-get install python3-libnvinfer-dev
#The following additional packages will be installed:
#python3-libnvinfer
#If you plan to use TensorRT with TensorFlow:
sudo apt-get install uff-converter-tf

Install Tensorflow

Software requirements

The following NVIDIA® software must be installed on your system:

NVIDIA® GPU drivers —CUDA 10.1 requires 418.x or higher.
CUDA® Toolkit —TensorFlow supports CUDA 10.1 (TensorFlow >= 2.1.0)
CUPTI ships with the CUDA Toolkit.
cuDNN SDK (>= 7.6)
(Optional) TensorRT 6.0 to improve latency and throughput for inference on some models.

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda/extras/CUPTI/lib64

Install by pip

virtualenv --system-site-packages -p python3 ./venv
source ./venv/bin/activate  # sh, bash, ksh, or zsh
pip install --upgrade tensorflow

Check

python -c "import tensorflow as tf;print(tf.reduce_sum(tf.random.normal([1000, 1000])))"

Disable CUDA toolkit v10.1 repository

If using CUDA v10.1 toolkit, v10.2 repository is not needed, so v10.2 repository to be disabled.

mv /etc/apt/sources.list.d/cuda-10-2-local-10.2.89-440.33.01.list /etc/apt/sources.list.d/disable/cuda-10-2-local-10.2.89-440.33.01.list.disable

Install JRE8

Profiler tools needs Java8, e.g, NVIDIA Nsight Systems

Install jre8

sudo apt-get install openjdk-8-jre

nvvp dosen't start with GUI,so launch from CUI.

nvvp -vm /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java {profile_file}(option)

Link.

https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html
https://docs.nvidia.com/cuda/cuda-samples/index.html#getting-started-with-cuda-samples
https://developer.nvidia.com/rdp/cudnn-download
https://qiita.com/junkoda/items/0c58e0c996b56b773d50
https://krhb.hatenablog.com/entry/2019/05/23/232719#CUDA-Toolkit%E3%81%AE%E3%82%A4%E3%83%B3%E3%82%B9%E3%83%88%E3%83%BC%E3%83%AB