pipの厳格な依存関係チェックにハマってしまった時の解決方法備忘録メモ

事の起こり

Tensor Flowのアップグレードを実行した時に以下のエラーが発生しました。

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
numba 0.55.1 requires numpy<1.22,>=1.18, but you have numpy 1.22.2 which is incompatible.

詳細な実行ログ

$ pip install -U tensorflow
Requirement already satisfied: tensorflow in /opt/conda/lib/python3.9/site-packages (2.6.2)
Collecting tensorflow
  Downloading tensorflow-2.8.0-cp39-cp39-manylinux2010_x86_64.whl (497.6 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 497.6/497.6 MB 864.7 kB/s eta 0:00:00
Requirement already satisfied: keras-preprocessing>=1.1.1 in /opt/conda/lib/python3.9/site-packages (from tensorflow) (1.1.2)
WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ProtocolError('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer'))': /simple/tensorflow-io-gcs-filesystem/
Collecting tensorflow-io-gcs-filesystem>=0.23.1
  Downloading tensorflow_io_gcs_filesystem-0.24.0-cp39-cp39-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (2.1 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.1/2.1 MB 1.0 MB/s eta 0:00:00
Requirement already satisfied: grpcio<2.0,>=1.24.3 in /opt/conda/lib/python3.9/site-packages (from tensorflow) (1.41.1)
Requirement already satisfied: setuptools in /opt/conda/lib/python3.9/site-packages (from tensorflow) (59.8.0)
Requirement already satisfied: opt-einsum>=2.3.2 in /opt/conda/lib/python3.9/site-packages (from tensorflow) (3.3.0)
Requirement already satisfied: absl-py>=0.4.0 in /opt/conda/lib/python3.9/site-packages (from tensorflow) (0.15.0)
Requirement already satisfied: typing-extensions>=3.6.6 in /opt/conda/lib/python3.9/site-packages (from tensorflow) (3.7.4.3)
Collecting keras<2.9,>=2.8.0rc0
  Downloading keras-2.8.0-py2.py3-none-any.whl (1.4 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.4/1.4 MB 1.0 MB/s eta 0:00:00
Collecting libclang>=9.0.1
  Downloading libclang-13.0.0-py2.py3-none-manylinux1_x86_64.whl (14.5 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 14.5/14.5 MB 1.2 MB/s eta 0:00:00
Requirement already satisfied: termcolor>=1.1.0 in /opt/conda/lib/python3.9/site-packages (from tensorflow) (1.1.0)
Collecting tensorboard<2.9,>=2.8
  Downloading tensorboard-2.8.0-py3-none-any.whl (5.8 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 5.8/5.8 MB 912.8 kB/s eta 0:00:00
Requirement already satisfied: six>=1.12.0 in /opt/conda/lib/python3.9/site-packages (from tensorflow) (1.15.0)
Collecting numpy>=1.20
  Downloading numpy-1.22.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (16.8 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 16.8/16.8 MB 2.8 MB/s eta 0:00:00
Requirement already satisfied: h5py>=2.9.0 in /opt/conda/lib/python3.9/site-packages (from tensorflow) (3.1.0)
Requirement already satisfied: astunparse>=1.6.0 in /opt/conda/lib/python3.9/site-packages (from tensorflow) (1.6.3)
Collecting tf-estimator-nightly==2.8.0.dev2021122109
  Downloading tf_estimator_nightly-2.8.0.dev2021122109-py2.py3-none-any.whl (462 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 462.5/462.5 KB 3.5 MB/s eta 0:00:00
Requirement already satisfied: gast>=0.2.1 in /opt/conda/lib/python3.9/site-packages (from tensorflow) (0.4.0)
Requirement already satisfied: flatbuffers>=1.12 in /opt/conda/lib/python3.9/site-packages (from tensorflow) (1.12)
Requirement already satisfied: wrapt>=1.11.0 in /opt/conda/lib/python3.9/site-packages (from tensorflow) (1.12.1)
Requirement already satisfied: google-pasta>=0.1.1 in /opt/conda/lib/python3.9/site-packages (from tensorflow) (0.2.0)
Requirement already satisfied: protobuf>=3.9.2 in /opt/conda/lib/python3.9/site-packages (from tensorflow) (3.18.1)
Requirement already satisfied: wheel<1.0,>=0.23.0 in /opt/conda/lib/python3.9/site-packages (from astunparse>=1.6.0->tensorflow) (0.37.1)
Requirement already satisfied: requests<3,>=2.21.0 in /opt/conda/lib/python3.9/site-packages (from tensorboard<2.9,>=2.8->tensorflow) (2.27.1)
Requirement already satisfied: google-auth-oauthlib<0.5,>=0.4.1 in /opt/conda/lib/python3.9/site-packages (from tensorboard<2.9,>=2.8->tensorflow) (0.4.6)
Requirement already satisfied: werkzeug>=0.11.15 in /opt/conda/lib/python3.9/site-packages (from tensorboard<2.9,>=2.8->tensorflow) (2.0.3)
Requirement already satisfied: tensorboard-plugin-wit>=1.6.0 in /opt/conda/lib/python3.9/site-packages (from tensorboard<2.9,>=2.8->tensorflow) (1.8.1)
Requirement already satisfied: google-auth<3,>=1.6.3 in /opt/conda/lib/python3.9/site-packages (from tensorboard<2.9,>=2.8->tensorflow) (1.35.0)
Requirement already satisfied: tensorboard-data-server<0.7.0,>=0.6.0 in /opt/conda/lib/python3.9/site-packages (from tensorboard<2.9,>=2.8->tensorflow) (0.6.0)
Requirement already satisfied: markdown>=2.6.8 in /opt/conda/lib/python3.9/site-packages (from tensorboard<2.9,>=2.8->tensorflow) (3.3.6)
Requirement already satisfied: pyasn1-modules>=0.2.1 in /opt/conda/lib/python3.9/site-packages (from google-auth<3,>=1.6.3->tensorboard<2.9,>=2.8->tensorflow) (0.2.7)
Requirement already satisfied: cachetools<5.0,>=2.0.0 in /opt/conda/lib/python3.9/site-packages (from google-auth<3,>=1.6.3->tensorboard<2.9,>=2.8->tensorflow) (4.2.4)
Requirement already satisfied: rsa<5,>=3.1.4 in /opt/conda/lib/python3.9/site-packages (from google-auth<3,>=1.6.3->tensorboard<2.9,>=2.8->tensorflow) (4.8)
Requirement already satisfied: requests-oauthlib>=0.7.0 in /opt/conda/lib/python3.9/site-packages (from google-auth-oauthlib<0.5,>=0.4.1->tensorboard<2.9,>=2.8->tensorflow) (1.3.1)
Requirement already satisfied: importlib-metadata>=4.4 in /opt/conda/lib/python3.9/site-packages (from markdown>=2.6.8->tensorboard<2.9,>=2.8->tensorflow) (4.11.1)
Requirement already satisfied: idna<4,>=2.5 in /opt/conda/lib/python3.9/site-packages (from requests<3,>=2.21.0->tensorboard<2.9,>=2.8->tensorflow) (3.3)
Requirement already satisfied: charset-normalizer~=2.0.0 in /opt/conda/lib/python3.9/site-packages (from requests<3,>=2.21.0->tensorboard<2.9,>=2.8->tensorflow) (2.0.12)
Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/lib/python3.9/site-packages (from requests<3,>=2.21.0->tensorboard<2.9,>=2.8->tensorflow) (2021.10.8)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /opt/conda/lib/python3.9/site-packages (from requests<3,>=2.21.0->tensorboard<2.9,>=2.8->tensorflow) (1.26.8)
Requirement already satisfied: zipp>=0.5 in /opt/conda/lib/python3.9/site-packages (from importlib-metadata>=4.4->markdown>=2.6.8->tensorboard<2.9,>=2.8->tensorflow) (3.7.0)
Requirement already satisfied: pyasn1<0.5.0,>=0.4.6 in /opt/conda/lib/python3.9/site-packages (from pyasn1-modules>=0.2.1->google-auth<3,>=1.6.3->tensorboard<2.9,>=2.8->tensorflow) (0.4.8)
Requirement already satisfied: oauthlib>=3.0.0 in /opt/conda/lib/python3.9/site-packages (from requests-oauthlib>=0.7.0->google-auth-oauthlib<0.5,>=0.4.1->tensorboard<2.9,>=2.8->tensorflow) (3.2.0)
Installing collected packages: tf-estimator-nightly, libclang, keras, tensorflow-io-gcs-filesystem, numpy, tensorboard, tensorflow
  Attempting uninstall: keras
    Found existing installation: keras 2.6.0
    Uninstalling keras-2.6.0:
      Successfully uninstalled keras-2.6.0
  Attempting uninstall: numpy
    Found existing installation: numpy 1.19.5
    Uninstalling numpy-1.19.5:
      Successfully uninstalled numpy-1.19.5
  Attempting uninstall: tensorboard
    Found existing installation: tensorboard 2.6.0
    Uninstalling tensorboard-2.6.0:
      Successfully uninstalled tensorboard-2.6.0
  Attempting uninstall: tensorflow
    Found existing installation: tensorflow 2.6.2
    Uninstalling tensorflow-2.6.2:
      Successfully uninstalled tensorflow-2.6.2
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
numba 0.55.1 requires numpy<1.22,>=1.18, but you have numpy 1.22.2 which is incompatible.
Successfully installed keras-2.8.0 libclang-13.0.0 numpy-1.22.2 tensorboard-2.8.0 tensorflow-2.8.0 tensorflow-io-gcs-filesystem-0.24.0 tf-estimator-nightly-2.8.0.dev2021122109

以下の記事に記載されている様に、依存関係チェックが厳しくなっている様です。以前のpipでは今回の様なエラーは発生しませんでした。

対処方法

依存関係チェックコマンドのpip checkを実行しても同様に依存関係の問題が出力されます。これはNumbaの最新版である0.55.1ではNumPyのバージョンが0.18以上かつ1.22未満である必要が有るという条件に外れていることが原因の様です。今回依存関係に問題が発生したのは、Tnesor Flowをアップグレードした時に、NumPyが最新版にアップグレードされたことによって依存関係を満たせなくなったことが原因の様です。

$ pip check
numba 0.55.1 has requirement numpy<1.22,>=1.18, but you have numpy 1.22.2.

因みにNumPyのバージョンを調べてみると、最新版の1.22.2がインストールされています。

$ pip list | grep numpy
numpy                         1.22.2

対処方法として、Numbaのアップグレードを実行します。-Uオプションか--upgradeオプションを付けて、pip installを実行します。これで、依存関係を修復されます。

2022/5/27追記: -Uオプションは依存関係のコンフリクトが大量かつ複雑な場合の最後の手段と考えた方が良い場合も有ります．全て最新版になるため，思わぬバグに遭遇する可能性も有ります．コンフリクト数が少ない場合は手動で最新版の1つ手前の安定リリース版をインストールする方が安全に修正出来る場合も有ります．(特にTensorFlow,PyTorchなどをインストールしている場合)

$ pip install -U numba
Requirement already satisfied: numba in /opt/conda/lib/python3.9/site-packages (0.55.1)
Requirement already satisfied: setuptools in /opt/conda/lib/python3.9/site-packages (from numba) (59.8.0)
Requirement already satisfied: llvmlite<0.39,>=0.38.0rc1 in /opt/conda/lib/python3.9/site-packages (from numba) (0.38.0)
Collecting numpy<1.22,>=1.18
  Downloading numpy-1.21.5-cp39-cp39-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (15.7 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 15.7/15.7 MB 7.3 MB/s eta 0:00:00
Installing collected packages: numpy
  Attempting uninstall: numpy
    Found existing installation: numpy 1.22.2
    Uninstalling numpy-1.22.2:
      Successfully uninstalled numpy-1.22.2
Successfully installed numpy-1.21.5

再度依存関係チェック

再度依存関係をチェックしました。無事依存関係問題が解決されました。

$ pip check
No broken requirements found.