More than 3 years have passed since last update.

GPUのメモリ解放コマンド

Posted at 2021-10-22

結論

下記を実行すれば良い。

$ kill -9 $(lsof -t /dev/nvidia*)

実行結果

JupyterでGPUとか動かしてると、よくメモリがパンパンになって、なんかよくわからんプロセスが残ってGPU使えなくなるけど、たいていはこれで良いと思う。備忘録。

$ nvidia-smi
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 455.23.04    Driver Version: 455.23.04    CUDA Version: 11.1     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  GeForce RTX 208...  Off  | 00000000:21:00.0 Off |                  N/A |
|  0%   43C    P8    23W / 300W |  10232MiB / 11016MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|    0   N/A  N/A     16103      C   ...vs/py37_pt_gpu/bin/python    10229MiB |
+-----------------------------------------------------------------------------+

$ kill -9 $(lsof -t /dev/nvidia*)

$ nvidia-smi
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 455.23.04    Driver Version: 455.23.04    CUDA Version: 11.1     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  GeForce RTX 208...  Off  | 00000000:21:00.0 Off |                  N/A |
|  0%   44C    P0    33W / 300W |      0MiB / 11016MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+

You get articles that match your needs
You can efficiently read back useful information
You can use dark theme

What you can do with signing up