LoginSignup
0
0

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?

@yusuke_s_yusuke(Yusuke Sakai)in立教大学

RAMエラーは突然に？Google Colabでクラッシュを避けるコツ

Last updated at Posted at 2025-04-19

はじめに

RAMエラーは、突然にやってくる。
……とはいえ、たいていはこちら側に原因があるんですけどね。


Google Colab を使っていると、こんなメッセージに出会ったことはありませんか？

セッションがクラッシュしました。自動的に再起動します。
使用可能な RAM をすべて使用した後で、セッションがクラッシュしました。


この記事では、その原因と、クラッシュを避けるためのメモリチェックのコツを紹介します。

Colab無料版でRAMクラッシュを再現したサンプルコードはこちらから閲覧できます。

環境と前提

  • Google Colabの無料版を使用
  • 無料版のRAMは約12GB程度

原因

このエラーはシンプルに、使用可能なRAMを使い切ったときに発生します。
特に、大きな配列を扱う場合や、機械学習・画像処理系のタスクでは起こりやすいです。

エラー時の出力例

RAMを使いすぎたときに表示されるエラーの一例です。
ログは長いので折り畳み表示します。

エラーログ詳細
Timestamp Level Message
Apr 19, 2025, 11:37:53 PM INFO Authentication of /metrics is OFF, since other authentication is disabled.
Apr 19, 2025, 11:37:53 PM INFO Authentication of /metrics is OFF, since other authentication is disabled.
Apr 19, 2025, 11:38:00 PM INFO Jupyter Notebook 6.5.5 is running at:
Apr 19, 2025, 11:38:00 PM INFO Jupyter Notebook 6.5.7 is running at:
Apr 19, 2025, 11:39:31 PM INFO Kernel started: 2edf6342-8553-4fd8-995b-c37e76db6788, name: python3
Apr 19, 2025, 11:47:46 PM INFO KernelRestarter: restarting kernel (1/5), keep random ports
Apr 19, 2025, 11:45:55 PM INFO KernelRestarter: restarting kernel (1/5), keep random ports
Apr 19, 2025, 11:44:07 PM INFO KernelRestarter: restarting kernel (1/5), keep random ports
Apr 19, 2025, 11:43:13 PM INFO KernelRestarter: restarting kernel (1/5), keep random ports
Apr 19, 2025, 11:37:54 PM INFO Loading IPython parallel extension
Apr 19, 2025, 11:37:53 PM INFO Loading IPython parallel extension
Apr 19, 2025, 11:38:00 PM INFO Serving notebooks from local directory: /
Apr 19, 2025, 11:38:00 PM INFO Serving notebooks from local directory: /
Apr 19, 2025, 11:38:00 PM INFO Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
Apr 19, 2025, 11:38:00 PM INFO Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
Apr 19, 2025, 11:37:53 PM INFO Writing notebook server cookie secret to /root/.local/share/jupyter/runtime/notebook_cookie_secret
Apr 19, 2025, 11:37:53 PM INFO Writing notebook server cookie secret to /root/.local/share/jupyter/runtime/notebook_cookie_secret
Apr 19, 2025, 11:37:53 PM INFO google.colab serverextension initialized.
Apr 19, 2025, 11:37:53 PM INFO google.colab serverextension initialized.
Apr 19, 2025, 11:38:00 PM INFO http://172\.28\.0\.12:9000/
Apr 19, 2025, 11:38:00 PM INFO http://172\.28\.0\.2:9000/
Apr 19, 2025, 11:37:53 PM WARNING /etc/jupyter/jupyter_notebook_config.json
Apr 19, 2025, 11:37:53 PM WARNING /etc/jupyter/jupyter_notebook_config.json
Apr 19, 2025, 11:37:53 PM WARNING /root/.jupyter/jupyter_notebook_config.json
Apr 19, 2025, 11:37:53 PM WARNING /root/.jupyter/jupyter_notebook_config.json
Apr 19, 2025, 11:37:53 PM WARNING /root/.local/etc/jupyter/jupyter_notebook_config.json
Apr 19, 2025, 11:37:53 PM WARNING /root/.local/etc/jupyter/jupyter_notebook_config.json
Apr 19, 2025, 11:37:53 PM WARNING /usr/etc/jupyter/jupyter_notebook_config.json
Apr 19, 2025, 11:37:53 PM WARNING /usr/etc/jupyter/jupyter_notebook_config.json
Apr 19, 2025, 11:37:53 PM WARNING /usr/local/etc/jupyter/jupyter_notebook_config.d/ipyparallel.json
Apr 19, 2025, 11:37:53 PM WARNING /usr/local/etc/jupyter/jupyter_notebook_config.d/ipyparallel.json
Apr 19, 2025, 11:37:53 PM WARNING /usr/local/etc/jupyter/jupyter_notebook_config.d/panel-client-jupyter.json
Apr 19, 2025, 11:37:53 PM WARNING /usr/local/etc/jupyter/jupyter_notebook_config.d/panel-client-jupyter.json
Apr 19, 2025, 11:37:53 PM WARNING /usr/local/etc/jupyter/jupyter_notebook_config.json
Apr 19, 2025, 11:37:53 PM WARNING /usr/local/etc/jupyter/jupyter_notebook_config.json
Apr 19, 2025, 11:47:46 PM WARNING 0.00s - Debugger warning: It seems that frozen modules are being used, which may
Apr 19, 2025, 11:45:56 PM WARNING 0.00s - Debugger warning: It seems that frozen modules are being used, which may
Apr 19, 2025, 11:44:08 PM WARNING 0.00s - Debugger warning: It seems that frozen modules are being used, which may
Apr 19, 2025, 11:43:14 PM WARNING 0.00s - Debugger warning: It seems that frozen modules are being used, which may
Apr 19, 2025, 11:39:31 PM WARNING 0.00s - Debugger warning: It seems that frozen modules are being used, which may
Apr 19, 2025, 11:39:30 PM WARNING 0.00s - Debugger warning: It seems that frozen modules are being used, which may
Apr 19, 2025, 11:47:46 PM WARNING 0.00s - Note: Debugging will proceed. Set PYDEVD_DISABLE_FILE_VALIDATION=1 to disable this validation.
Apr 19, 2025, 11:45:56 PM WARNING 0.00s - Note: Debugging will proceed. Set PYDEVD_DISABLE_FILE_VALIDATION=1 to disable this validation.
Apr 19, 2025, 11:44:08 PM WARNING 0.00s - Note: Debugging will proceed. Set PYDEVD_DISABLE_FILE_VALIDATION=1 to disable this validation.
Apr 19, 2025, 11:43:14 PM WARNING 0.00s - Note: Debugging will proceed. Set PYDEVD_DISABLE_FILE_VALIDATION=1 to disable this validation.
Apr 19, 2025, 11:39:31 PM WARNING 0.00s - Note: Debugging will proceed. Set PYDEVD_DISABLE_FILE_VALIDATION=1 to disable this validation.
Apr 19, 2025, 11:39:30 PM WARNING 0.00s - Note: Debugging will proceed. Set PYDEVD_DISABLE_FILE_VALIDATION=1 to disable this validation.
Apr 19, 2025, 11:47:46 PM WARNING 0.00s - make the debugger miss breakpoints. Please pass -Xfrozen_modules=off
Apr 19, 2025, 11:45:56 PM WARNING 0.00s - make the debugger miss breakpoints. Please pass -Xfrozen_modules=off
Apr 19, 2025, 11:44:08 PM WARNING 0.00s - make the debugger miss breakpoints. Please pass -Xfrozen_modules=off
Apr 19, 2025, 11:43:14 PM WARNING 0.00s - make the debugger miss breakpoints. Please pass -Xfrozen_modules=off
Apr 19, 2025, 11:39:31 PM WARNING 0.00s - make the debugger miss breakpoints. Please pass -Xfrozen_modules=off
Apr 19, 2025, 11:39:30 PM WARNING 0.00s - make the debugger miss breakpoints. Please pass -Xfrozen_modules=off
Apr 19, 2025, 11:47:46 PM WARNING 0.00s - to python to disable frozen modules.
Apr 19, 2025, 11:45:56 PM WARNING 0.00s - to python to disable frozen modules.
Apr 19, 2025, 11:44:08 PM WARNING 0.00s - to python to disable frozen modules.
Apr 19, 2025, 11:43:14 PM WARNING 0.00s - to python to disable frozen modules.
Apr 19, 2025, 11:39:31 PM WARNING 0.00s - to python to disable frozen modules.
Apr 19, 2025, 11:39:30 PM WARNING 0.00s - to python to disable frozen modules.
Apr 19, 2025, 11:47:46 PM WARNING WARNING:root:kernel 2edf6342-8553-4fd8-995b-c37e76db6788 restarted
Apr 19, 2025, 11:45:55 PM WARNING WARNING:root:kernel 2edf6342-8553-4fd8-995b-c37e76db6788 restarted
Apr 19, 2025, 11:44:07 PM WARNING WARNING:root:kernel 2edf6342-8553-4fd8-995b-c37e76db6788 restarted
Apr 19, 2025, 11:43:13 PM WARNING WARNING:root:kernel 2edf6342-8553-4fd8-995b-c37e76db6788 restarted
Apr 19, 2025, 11:37:52 PM WARNING [D 14:37:52.298 NotebookApp] Searching ['/root/.jupyter', '/root/.local/etc/jupyter', '/usr/etc/jupyter', '/usr/local/etc/jupyter', '/etc/jupyter'] for config files
Apr 19, 2025, 11:37:52 PM WARNING [D 14:37:52.308 NotebookApp] Looking for jupyter_config in /etc/jupyter
Apr 19, 2025, 11:37:52 PM WARNING [D 14:37:52.312 NotebookApp] Looking for jupyter_config in /usr/local/etc/jupyter
Apr 19, 2025, 11:37:52 PM WARNING [D 14:37:52.316 NotebookApp] Looking for jupyter_config in /usr/etc/jupyter
Apr 19, 2025, 11:37:52 PM WARNING [D 14:37:52.317 NotebookApp] Looking for jupyter_config in /root/.local/etc/jupyter
Apr 19, 2025, 11:37:52 PM WARNING [D 14:37:52.318 NotebookApp] Looking for jupyter_config in /root/.jupyter
Apr 19, 2025, 11:37:52 PM WARNING [D 14:37:52.319 NotebookApp] Looking for jupyter_notebook_config in /etc/jupyter
Apr 19, 2025, 11:37:52 PM WARNING [D 14:37:52.323 NotebookApp] Loaded config file: /etc/jupyter/jupyter_notebook_config.py
Apr 19, 2025, 11:37:52 PM WARNING [D 14:37:52.324 NotebookApp] Looking for jupyter_notebook_config in /usr/local/etc/jupyter
Apr 19, 2025, 11:37:52 PM WARNING [D 14:37:52.326 NotebookApp] Loaded config file: /usr/local/etc/jupyter/jupyter_notebook_config.json
Apr 19, 2025, 11:37:52 PM WARNING [D 14:37:52.327 NotebookApp] Looking for jupyter_notebook_config in /root/.jupyter
Apr 19, 2025, 11:37:52 PM WARNING [D 14:37:52.327 NotebookApp] Looking for jupyter_notebook_config in /root/.local/etc/jupyter
Apr 19, 2025, 11:37:52 PM WARNING [D 14:37:52.327 NotebookApp] Looking for jupyter_notebook_config in /usr/etc/jupyter
Apr 19, 2025, 11:37:52 PM WARNING [D 14:37:52.332 NotebookApp] Loaded config file: /root/.jupyter/jupyter_notebook_config.py
Apr 19, 2025, 11:37:52 PM WARNING [D 14:37:52.990 NotebookApp] Searching ['/root/.jupyter', '/root/.local/etc/jupyter', '/usr/etc/jupyter', '/usr/local/etc/jupyter', '/etc/jupyter'] for config files
Apr 19, 2025, 11:37:53 PM WARNING [D 14:37:52.992 NotebookApp] Looking for jupyter_config in /etc/jupyter
Apr 19, 2025, 11:37:53 PM WARNING [D 14:37:52.992 NotebookApp] Looking for jupyter_config in /usr/local/etc/jupyter
Apr 19, 2025, 11:37:53 PM WARNING [D 14:37:52.997 NotebookApp] Looking for jupyter_config in /usr/etc/jupyter
Apr 19, 2025, 11:37:53 PM WARNING [D 14:37:52.998 NotebookApp] Looking for jupyter_config in /root/.jupyter
Apr 19, 2025, 11:37:53 PM WARNING [D 14:37:52.998 NotebookApp] Looking for jupyter_config in /root/.local/etc/jupyter
Apr 19, 2025, 11:37:53 PM WARNING [D 14:37:52.999 NotebookApp] Looking for jupyter_notebook_config in /etc/jupyter
Apr 19, 2025, 11:37:53 PM WARNING [D 14:37:53.002 NotebookApp] Loaded config file: /etc/jupyter/jupyter_notebook_config.py
Apr 19, 2025, 11:37:53 PM WARNING [D 14:37:53.003 NotebookApp] Looking for jupyter_notebook_config in /usr/local/etc/jupyter
Apr 19, 2025, 11:37:53 PM WARNING [D 14:37:53.006 NotebookApp] Loaded config file: /usr/local/etc/jupyter/jupyter_notebook_config.json
Apr 19, 2025, 11:37:53 PM WARNING [D 14:37:53.006 NotebookApp] Looking for jupyter_notebook_config in /root/.jupyter
Apr 19, 2025, 11:37:53 PM WARNING [D 14:37:53.006 NotebookApp] Looking for jupyter_notebook_config in /root/.local/etc/jupyter
Apr 19, 2025, 11:37:53 PM WARNING [D 14:37:53.006 NotebookApp] Looking for jupyter_notebook_config in /usr/etc/jupyter
Apr 19, 2025, 11:37:53 PM WARNING [D 14:37:53.013 NotebookApp] Loaded config file: /root/.jupyter/jupyter_notebook_config.py

RAMの使用状況を事前にチェックする

対策として、まずはColabのRAMについて知っておくために、メモリ情報を確認します。

RAMの使用状況の確認
import os
import psutil

def print_memory_status():
    virtual_mem = psutil.virtual_memory()
    print(f"Total RAM     : {virtual_mem.total / 1024**3:.2f} GB")
    print(f"Available RAM : {virtual_mem.available / 1024**3:.2f} GB")
    print(f"Used RAM      : {(virtual_mem.total - virtual_mem.available) / 1024**3:.2f} GB")
    print(f"Memory usage %: {virtual_mem.percent} %")

    process = psutil.Process(os.getpid())
    print(f"Current process memory usage: {process.memory_info().rss / 1024**3:.2f} GB")

print_memory_status()
出力例
Total RAM     : 12.67 GB  
Available RAM : 11.60 GB  
Used RAM      : 1.07 GB  
Memory usage %: 8.4 %  
Current process memory usage: 0.11 GB

配列を作る前にメモリを見積もる

Colabでは、np.ones()やnp.zeros()で巨大な配列を作ると一発アウトです。
事前にメモリを計算しておくことでこれから作ろうとしているはいれつ

メモリの事前確認
import numpy as np
import psutil

# 使用可能なメモリ（GB）
available_gb = psutil.virtual_memory().available / 1024**3
print(f"Available RAM: {available_gb:.2f} GB")

# 作成予定の配列のサイズ（例：42,000 × 42,000）
shape = (42000, 42000)
bytes_needed = np.prod(shape) * np.dtype('float64').itemsize
gb_needed = bytes_needed / 1024**3
print(f"Estimated memory needed: {gb_needed:.2f} GB")
出力例
Available RAM: 11.60 GB  
Estimated memory needed: 13.14 GB

この時点でセーフかアウトか判断できます。
このまま進めば、Colabが静かに落ちていきます。

実際に実行してみると……

⚠️ クラッシュするので自己責任で実行してください

実際にメモリを使用
arr = np.ones(shape, dtype='float64')
print(f"Array size: {arr.shape}")
print(f"Total size (GB): {arr.nbytes / 1024**3:.2f}")

Colabは限界を迎えました。
……そっと、エラーを残して。

補足：TPU環境で“メモリが爆増”することがある？

Colabでは、TPUランタイムを選択すると、使用可能メモリが300GBを超えて表示されることもあります（以下は筆者の実例です）。

選択方法は「編集 → ノートブックの設定 → ハードウェア アクセラレータ → TPUに選択」

v2-8 TPUの出力例
Total RAM     : 334.56 GB
Available RAM : 330.40 GB
Used RAM      : 4.16 GB
Memory usage %: 1.2 %
Current process memory usage: 0.10 GB

筆者自身、TPUについて詳しいわけではありませんが、
この環境では一時的に大きなメモリを使えるようで、13GB超の NumPy 配列も作成できました。

ただし、TPUは NumPy 処理に最適化されておらず、動作が不安定になったり、
Colab全体に不要な負荷をかける可能性もあるため、利用は慎重にすべきだと感じています。

まとめ

Google Colabでは、RAMを使いすぎると、セッションがクラッシュしてしまいます。


「この配列って、今キャパオーバーじゃないかな？」
そんな何気ない気遣いが、Colabにとっては案外大事だったりするんです。

0
0
0

Register as a new user and use Qiita more conveniently

  1. You get articles that match your needs
  2. You can efficiently read back useful information
  3. You can use dark theme
What you can do with signing up
Sign upLogin
0
0

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?