とりあえずローカルでtenosorflowからgpu認識されるまで

最初これを参考にした。基本的には追っていけばいいが、
https://towardsdatascience.com/installing-tensorflow-gpu-in-ubuntu-20-04-4ee3ca4cb75d

自分の環境ではcudnn.h, libcudnn*は別の場所にあったのでこの部分は変更した。

sudo cp /usr/include/cudnn.h /usr/lib/cuda/include
sudo cp /usr/lib/x86_64-linux-gnu/libcudnn* /usr/lib/cuda/lib64/

pip install tensorflow==2.2.0

In [1]: import tensorflow as tf

In [2]: from tensorflow.python.client import device_lib

In [3]: device_lib.list_local_devices()
2022-11-06 14:49:53.616845: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2022-11-06 14:49:53.638508: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3699850000 Hz
2022-11-06 14:49:53.638904: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7fd23c000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2022-11-06 14:49:53.638922: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version
2022-11-06 14:49:53.640664: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1
2022-11-06 14:49:53.710467: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-11-06 14:49:53.710639: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x55b2352d08a0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices:
2022-11-06 14:49:53.710655: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): NVIDIA GeForce RTX 3080, Compute Capability 8.6
2022-11-06 14:49:53.710771: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-11-06 14:49:53.710851: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: 
pciBusID: 0000:01:00.0 name: NVIDIA GeForce RTX 3080 computeCapability: 8.6
coreClock: 1.71GHz coreCount: 68 deviceMemorySize: 9.78GiB deviceMemoryBandwidth: 707.88GiB/s
2022-11-06 14:49:53.711016: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1
2022-11-06 14:49:53.712088: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10
2022-11-06 14:49:53.712754: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10
2022-11-06 14:49:53.712893: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10
2022-11-06 14:49:53.714593: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10
2022-11-06 14:49:53.715217: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10
2022-11-06 14:49:53.717576: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
2022-11-06 14:49:53.717650: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-11-06 14:49:53.717768: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-11-06 14:49:53.717831: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0
2022-11-06 14:49:53.717853: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1
2022-11-06 14:49:53.718006: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix:
2022-11-06 14:49:53.718016: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108]      0 
2022-11-06 14:49:53.718021: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0:   N 
2022-11-06 14:49:53.718082: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-11-06 14:49:53.718173: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-11-06 14:49:53.718259: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/device:GPU:0 with 8866 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 3080, pci bus id: 0000:01:00.0, compute capability: 8.6)
Out[4]: 
[name: "/device:CPU:0"
 device_type: "CPU"
 memory_limit: 268435456
 locality {
 }
 incarnation: 16727521364971791492,
 name: "/device:XLA_CPU:0"
 device_type: "XLA_CPU"
 memory_limit: 17179869184
 locality {
 }
 incarnation: 3091467780931466636
 physical_device_desc: "device: XLA_CPU device",
 name: "/device:XLA_GPU:0"
 device_type: "XLA_GPU"
 memory_limit: 17179869184
 locality {
 }
 incarnation: 4940470567429118899
 physical_device_desc: "device: XLA_GPU device",
 name: "/device:GPU:0"
 device_type: "GPU"
 memory_limit: 9297436352
 locality {
   bus_id: 1
   links {
   }
 }
 incarnation: 3026933686697730393
 physical_device_desc: "device: 0, name: NVIDIA GeForce RTX 3080, pci bus id: 0000:01:00.0, compute capability: 8.6"]

tensorflow==2.4.0の場合

Could not load dynamic library 'libcudnn.so.8'; dlerror: libcudnn.so.8: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/lib/cuda/include:/usr/lib/cuda/lib64:/usr/local/cuda/lib64:/usr/local/cuda/lib64:

cudnnのバージョンは7.5.6だったので仕方ない
cudnnのバージョンをいちいち揃えるのは面倒なので
nvidia-docker2

nvidia-docker2

公式のinstallation guide
https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html

nvidia docker image

python cuda image

cudnn, cudaとtensorflowのバージョン対応表
https://www.tensorflow.org/install/source?hl=ja#gpu_support_2