Go to Qiita Advent Calendar 2024 Top
LoginSignup
0
0

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?

@yuji38kwmtin有限会社来栖川電算

Ubuntu 22.04: `python3-pip`でインストールしたpipを`python3.12 -m pip`で実行すると、`No module named 'distutils'`というエラーが発生する

Last updated at Posted at 2024-07-18

環境

  • Ubuntu 22.04

はじめに

以下のDockerfileは、VSCode Dev Containers用のDockerfileです。Python 3.11用の開発環境を構築しています。

Dockerfile.python311
FROM ubuntu:22.04

# Install prerequisites
RUN set -x \
    && export DEBIAN_FRONTEND=noninteractive \
    && apt-get update \
    && apt-get install -y --no-install-recommends \
        gnupg2 \
        software-properties-common \
        language-pack-ja \
        tzdata \
        curl \
        lsb-release \
    && apt-get -y clean \
    && rm -rf /var/lib/apt/lists/*

# Set locale & timezone
RUN update-locale LANG=ja_JP.UTF-8 LANGUAGE=ja_JP:ja \
    && ln -sf /usr/share/zoneinfo/Asia/Tokyo /etc/localtime \
    && echo "Asia/Tokyo" > /etc/timezone

ENV LANG=ja_JP.UTF-8
ENV LC_ALL=ja_JP.UTF-8
ENV LC_CTYPE=ja_JP.UTF-8

# Install packages
RUN set -x \
    # Python3.11をインストールするために、deadsnakes PPAを追加
    && add-apt-repository ppa:deadsnakes/ppa \
    && apt-get update \
    && apt-get install -y --no-install-recommends \
        python3.11-dev \
        python3-pip \
        # common tools
        bash-completion \
        build-essential \
        git \
        iputils-ping \
        jq \
        less \
        net-tools \
        openssh-client \
        sudo \
        tar \
        time \
        unzip \
        vim \
        wget \
        xz-utils \
        zip \        
    && apt-get -y clean \
    && rm -rf /var/lib/apt/lists/*

# Python / pip
RUN ln -s $(which python3.11) /usr/bin/python

# poetry
RUN set -x \
    && python -m pip install -U \
        pip \
    && python -m pip install \
        poetry

# Add user / Grant sudo privileges
RUN useradd -m -s /bin/bash -u 5000 -U vscode \
    && echo "vscode ALL=(ALL) NOPASSWD:ALL" >> /etc/sudoers.d/ALL

USER vscode

RUN poetry config virtualenvs.in-project true

$ docker buildx build --file Dockerfile.python311 .

 => [internal] load build definition from Dockerfile.python311                                                                                                                         0.0s
 => => transferring dockerfile: 1.72kB                                                                                                                                                 0.0s
 => [internal] load metadata for docker.io/library/ubuntu:22.04                                                                                                                        1.6s
 => [internal] load .dockerignore                                                                                                                                                      0.0s
 => => transferring context: 2B                                                                                                                                                        0.0s
 => [1/8] FROM docker.io/library/ubuntu:22.04@sha256:340d9b015b194dc6e2a13938944e0d016e57b9679963fdeb9ce021daac430221                                                                  0.0s
 => CACHED [2/8] RUN set -x     && export DEBIAN_FRONTEND=noninteractive     && apt-get update     && apt-get install -y --no-install-recommends         gnupg2         software-prop  0.0s
 => CACHED [3/8] RUN update-locale LANG=ja_JP.UTF-8 LANGUAGE=ja_JP:ja     && ln -sf /usr/share/zoneinfo/Asia/Tokyo /etc/localtime     && echo "Asia/Tokyo" > /etc/timezone             0.0s
 => [4/8] RUN set -x     && add-apt-repository ppa:deadsnakes/ppa     && apt-get update     && apt-get install -y --no-install-recommends         python3.11-dev         python3-p  1551.7s
 => [5/8] RUN ln -s $(which python3.11) /usr/bin/python                                                                                                                                0.3s
 => [6/8] RUN set -x     && python -m pip install -U         pip     && python -m pip install         poetry                                                                         141.3s
 => [7/8] RUN useradd -m -s /bin/bash -u 5000 -U vscode     && echo "vscode ALL=(ALL) NOPASSWD:ALL" >> /etc/sudoers.d/ALL                                                              0.3s
 => [8/8] RUN poetry config virtualenvs.in-project true                                                                                                                                0.7s
 => exporting to image                                                                                                                                                                 1.0s
 => => exporting layers                                                                                                                                                                1.0s
 => => writing image sha256:dd91ef557975345566381dbf282135bdb9ebdb0a6c86d5dcd4de5815354513d2

何が起きたのか

Pythonのバージョンを3.11から3.12に変更するため、以下のようなDockerfileに変更しました。

Dockerfile.python312-ng
# ...
RUN set -x \
    # Python3.12をインストールするために、deadsnakes PPAを追加
    && add-apt-repository ppa:deadsnakes/ppa \
    && apt-get update \
    && apt-get install -y --no-install-recommends \
        python3.12-dev \
        python3-pip \

# ...
RUN ln -s $(which python3.12) /usr/bin/python

$ diff Dockerfile.python311 Dockerfile.python312-ng
28c28
<     # Python3.11をインストールするために、deadsnakes PPAを追加
---
>     # Python3.12をインストールするために、deadsnakes PPAを追加
32c32
<         python3.11-dev \
---
>         python3.12-dev \
55c55
< RUN ln -s $(which python3.11) /usr/bin/python
---
> RUN ln -s $(which python3.12) /usr/bin/python

しかし、docker buildx buildは失敗しました。python -m pip install -U pipで失敗していました。

$ docker buildx build --file Dockerfile.python312-ng .

=> [internal] load build definition from Dockerfile.python312-ng                                                                                                                      0.0s
=> => transferring dockerfile: 1.73kB                                                                                                                                                 0.0s
=> [internal] load metadata for docker.io/library/ubuntu:22.04                                                                                                                        1.5s
=> [internal] load .dockerignore                                                                                                                                                      0.0s
=> => transferring context: 2B                                                                                                                                                        0.0s
=> [1/8] FROM docker.io/library/ubuntu:22.04@sha256:340d9b015b194dc6e2a13938944e0d016e57b9679963fdeb9ce021daac430221                                                                  0.0s
=> CACHED [2/8] RUN set -x     && export DEBIAN_FRONTEND=noninteractive     && apt-get update     && apt-get install -y --no-install-recommends         gnupg2         software-prop  0.0s
=> CACHED [3/8] RUN update-locale LANG=ja_JP.UTF-8 LANGUAGE=ja_JP:ja     && ln -sf /usr/share/zoneinfo/Asia/Tokyo /etc/localtime     && echo "Asia/Tokyo" > /etc/timezone             0.0s
=> [4/8] RUN set -x     && add-apt-repository ppa:deadsnakes/ppa     && apt-get update     && apt-get install -y --no-install-recommends         python3.12-dev         python3-p  1695.2s
=> [5/8] RUN ln -s $(which python3.12) /usr/bin/python                                                                                                                                0.3s
=> ERROR [6/8] RUN set -x     && python -m pip install -U         pip     && python -m pip install         poetry                                                                     0.8s
------
> [6/8] RUN set -x     && python -m pip install -U         pip     && python -m pip install         poetry:
0.332 + python -m pip install -U pip
0.710 Traceback (most recent call last):
0.710   File "<frozen runpy>", line 198, in _run_module_as_main
0.710   File "<frozen runpy>", line 88, in _run_code
0.710   File "/usr/lib/python3/dist-packages/pip/__main__.py", line 29, in <module>
0.710     from pip._internal.cli.main import main as _main
0.710   File "/usr/lib/python3/dist-packages/pip/_internal/cli/main.py", line 9, in <module>
0.710     from pip._internal.cli.autocompletion import autocomplete
0.710   File "/usr/lib/python3/dist-packages/pip/_internal/cli/autocompletion.py", line 10, in <module>
0.710     from pip._internal.cli.main_parser import create_main_parser
0.710   File "/usr/lib/python3/dist-packages/pip/_internal/cli/main_parser.py", line 8, in <module>
0.710     from pip._internal.cli import cmdoptions
0.710   File "/usr/lib/python3/dist-packages/pip/_internal/cli/cmdoptions.py", line 23, in <module>
0.710     from pip._internal.cli.parser import ConfigOptionParser
0.710   File "/usr/lib/python3/dist-packages/pip/_internal/cli/parser.py", line 12, in <module>
0.710     from pip._internal.configuration import Configuration, ConfigurationError
0.710   File "/usr/lib/python3/dist-packages/pip/_internal/configuration.py", line 26, in <module>
0.710     from pip._internal.utils.logging import getLogger
0.710   File "/usr/lib/python3/dist-packages/pip/_internal/utils/logging.py", line 27, in <module>
0.710     from pip._internal.utils.misc import ensure_dir
0.710   File "/usr/lib/python3/dist-packages/pip/_internal/utils/misc.py", line 39, in <module>
0.710     from pip._internal.locations import get_major_minor_version
0.710   File "/usr/lib/python3/dist-packages/pip/_internal/locations/__init__.py", line 14, in <module>
0.710     from . import _distutils, _sysconfig
0.710   File "/usr/lib/python3/dist-packages/pip/_internal/locations/_distutils.py", line 9, in <module>
0.710     from distutils.cmd import Command as DistutilsCommand
0.710 ModuleNotFoundError: No module named 'distutils'
------
Dockerfile.python312-ng:58
--------------------
 57 |     # poetry
 58 | >>> RUN set -x \
 59 | >>>     && python -m pip install -U \
 60 | >>>         pip \
 61 | >>>     && python -m pip install \
 62 | >>>         poetry
 63 |
--------------------
ERROR: failed to solve: process "/bin/sh -c set -x     && python -m pip install -U         pip     && python -m pip install         poetry" did not complete successfully: exit code: 1

エラーの原因

Python3.12ではdistutilsは削除されました。

このDockerfileでは、pipコマンドをpython3-pipパッケージでインストールしています。
apt showコマンドでpython3-pipを調べると、python3-distutilsに依存していました。
したがって、python -m pipコマンドを実行したときに、「ModuleNotFoundError: No module named 'distutils'」というエラーが発生したようです。

vscode@c3aecfddeafb:/$ sudo apt show python3-pip
Package: python3-pip
Version: 22.0.2+dfsg-1ubuntu0.4
Status: install ok installed
Priority: optional
Section: python
Source: python-pip
Maintainer: Ubuntu Developers <ubuntu-devel-discuss@lists.ubuntu.com>
Original-Maintainer: Debian Python Team <team+python@tracker.debian.org>
Installed-Size: 7,043 kB
Provides: pip
Depends: ca-certificates, python3-distutils, python3-setuptools, python3-wheel, python3:any
Recommends: build-essential, python3-dev (>= 3.2)
Breaks: python-pip
Replaces: python-pip
Homepage: https://pip.pypa.io/en/stable/
Download-Size: 不明
APT-Manual-Installed: yes
APT-Sources: /var/lib/dpkg/status
Description: Python package installer
 pip is the Python package installer.  It integrates with virtualenv, doesn't
 do partial installs, can save package state for replaying, can install from
 non-egg sources, and can install from version control repositories.
 .
 This is the Python 3 version of the package.

なお、python -m pipでなくpipコマンドならば動作します。/usr/bin/pipはPython3.10環境で実行されているためです。

vscode@c3aecfddeafb:/$ which pip
/usr/bin/pip
vscode@c3aecfddeafb:/$ pip --version
pip 22.0.2 from /usr/lib/python3/dist-packages/pip (python 3.10)
/usr/bin/pip
#!/usr/bin/python3
# -*- coding: utf-8 -*-
import re
import sys
from pip._internal.cli.main import main
if __name__ == "__main__":
    sys.argv[0] = re.sub(r"(-script\.pyw|\.exe)?$", "", sys.argv[0])
    sys.exit(main())

vscode@c3aecfddeafb:/$ /usr/bin/python3 --version
Python 3.10.12

python3.10は、software-properties-commonパッケージをインストールしたとときに、一緒にインストールされました。software-properties-commonはpython3に依存しています。そして、python3パッケージのバージョンは3.10です。1

$ apt show software-properties-common
Package: software-properties-common
Version: 0.99.22.9
Priority: optional
Section: admin
Source: software-properties
Origin: Ubuntu
Maintainer: Michael Vogt <michael.vogt@ubuntu.com>
Bugs: https://bugs.launchpad.net/ubuntu/+filebug
Installed-Size: 225 kB
Depends: ca-certificates, gir1.2-glib-2.0, gir1.2-packagekitglib-1.0 (>= 1.1.0-2), packagekit, python-apt-common (>= 0.9), python3, python3-dbus, python3-gi, python3-software-properties (= 0.99.22.9), python3:any
Breaks: python-software-properties (<< 0.85), python3-software-properties (<< 0.85)
Replaces: python-software-properties (<< 0.85), python3-software-properties (<< 0.85)
Task: ubuntu-desktop-minimal, ubuntu-desktop, cloud-image, ubuntu-wsl, server, ubuntu-server-raspi, kubuntu-desktop, xubuntu-core, xubuntu-desktop, lubuntu-desktop, ubuntustudio-desktop-core, ubuntustudio-desktop, ubuntukylin-desktop, ubuntu-mate-core, ubuntu-mate-desktop, ubuntu-budgie-desktop, ubuntu-budgie-desktop-raspi
Download-Size: 14.1 kB
APT-Manual-Installed: yes
APT-Sources: http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages
Description: manage the repositories that you install software from (common)

$ apt show python3
Package: python3
Version: 3.10.6-1~22.04
Priority: important
Section: python
Source: python3-defaults
Origin: Ubuntu
Maintainer: Ubuntu Developers <ubuntu-devel-discuss@lists.ubuntu.com>
Original-Maintainer: Matthias Klose <doko@debian.org>
Bugs: https://bugs.launchpad.net/ubuntu/+filebug
Installed-Size: 92.2 kB
Provides: python3-profiler
Pre-Depends: python3-minimal (= 3.10.6-1~22.04)
Depends: python3.10 (>= 3.10.6-1~), libpython3-stdlib (= 3.10.6-1~22.04)
Suggests: python3-doc (>= 3.10.6-1~22.04), python3-tk (>= 3.10.6-1~), python3-venv (>= 3.10.6-1~22.04)
Replaces: python3-minimal (<< 3.1.2-2)
Homepage: https://www.python.org/
Task: minimal, server-minimal
Download-Size: 22.8 kB
APT-Manual-Installed: no
APT-Sources: http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages
Description: interactive high-level object-oriented language (default python3 version)

解決策

https://bootstrap.pypa.io/get-pip.py からget-pip.pyをダウンロードして実行すれば、Python3.12でも動作するpipがインストールされます。

vscode@c3aecfddeafb:/$ curl -sSL https://bootstrap.pypa.io/get-pip.py | python -
Defaulting to user installation because normal site-packages is not writeable
Collecting pip
  Downloading pip-24.1.2-py3-none-any.whl.metadata (3.6 kB)
Downloading pip-24.1.2-py3-none-any.whl (1.8 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.8/1.8 MB 235.3 kB/s eta 0:00:00
Installing collected packages: pip
  WARNING: The scripts pip, pip3 and pip3.12 are installed in '/home/vscode/.local/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
Successfully installed pip-24.1.2

vscode@c3aecfddeafb:/$ python -m pip --version
pip 24.1.2 from /home/vscode/.local/lib/python3.12/site-packages/pip (python 3.12)

vscode@c3aecfddeafb:/$ ls -la  ~/.local/bin/pip*
-rwxr-xr-x 1 vscode vscode 220  7月 18 16:18 /home/vscode/.local/bin/pip
-rwxr-xr-x 1 vscode vscode 220  7月 18 16:18 /home/vscode/.local/bin/pip3
-rwxr-xr-x 1 vscode vscode 220  7月 18 16:18 /home/vscode/.local/bin/pip3.12

以下は、Python3.12用のDockerfileの全体になります。

Dockerfile.python312
FROM ubuntu:22.04

# Install prerequisites
RUN set -x \
    && export DEBIAN_FRONTEND=noninteractive \
    && apt-get update \
    && apt-get install -y --no-install-recommends \
        gnupg2 \
        software-properties-common \
        language-pack-ja \
        tzdata \
        curl \
        lsb-release \
    && apt-get -y clean \
    && rm -rf /var/lib/apt/lists/*

# Set locale & timezone
RUN update-locale LANG=ja_JP.UTF-8 LANGUAGE=ja_JP:ja \
    && ln -sf /usr/share/zoneinfo/Asia/Tokyo /etc/localtime \
    && echo "Asia/Tokyo" > /etc/timezone

ENV LANG=ja_JP.UTF-8
ENV LC_ALL=ja_JP.UTF-8
ENV LC_CTYPE=ja_JP.UTF-8

# Install packages
RUN set -x \
    # Python3.12をインストールするために、deadsnakes PPAを追加
    && add-apt-repository ppa:deadsnakes/ppa \
    && apt-get update \
    && apt-get install -y --no-install-recommends \
        python3.12-dev \
        # common tools
        bash-completion \
        build-essential \
        git \
        iputils-ping \
        jq \
        less \
        net-tools \
        openssh-client \
        sudo \
        tar \
        time \
        unzip \
        vim \
        wget \
        xz-utils \
        zip \        
    && apt-get -y clean \
    && rm -rf /var/lib/apt/lists/*

# Python / pip
RUN ln -s $(which python3.12) /usr/bin/python
RUN curl -sSL https://bootstrap.pypa.io/get-pip.py | python -

# poetry
RUN set -x \
    && python -m pip install \
        poetry

# Add user / Grant sudo privileges
RUN useradd -m -s /bin/bash -u 5000 -U vscode \
    && echo "vscode ALL=(ALL) NOPASSWD:ALL" >> /etc/sudoers.d/ALL

USER vscode

RUN poetry config virtualenvs.in-project true

$ docker buildx build --file Dockerfile.python312 .

 => [internal] load build definition from Dockerfile.python312                                                                                                                         0.0s
 => => transferring dockerfile: 1.72kB                                                                                                                                                 0.0s
 => [internal] load metadata for docker.io/library/ubuntu:22.04                                                                                                                        1.8s
 => [internal] load .dockerignore                                                                                                                                                      0.0s
 => => transferring context: 2B                                                                                                                                                        0.0s
 => [1/9] FROM docker.io/library/ubuntu:22.04@sha256:340d9b015b194dc6e2a13938944e0d016e57b9679963fdeb9ce021daac430221                                                                  0.0s
 => CACHED [2/9] RUN set -x     && export DEBIAN_FRONTEND=noninteractive     && apt-get update     && apt-get install -y --no-install-recommends         gnupg2         software-prop  0.0s
 => CACHED [3/9] RUN update-locale LANG=ja_JP.UTF-8 LANGUAGE=ja_JP:ja     && ln -sf /usr/share/zoneinfo/Asia/Tokyo /etc/localtime     && echo "Asia/Tokyo" > /etc/timezone             0.0s
 => [4/9] RUN set -x     && add-apt-repository ppa:deadsnakes/ppa     && apt-get update     && apt-get install -y --no-install-recommends         python3.12-dev         bash-comp  1777.7s
 => [5/9] RUN ln -s $(which python3.12) /usr/bin/python                                                                                                                                0.2s
 => [6/9] RUN curl -sSL https://bootstrap.pypa.io/get-pip.py | python -                                                                                                               58.6s
 => [7/9] RUN set -x     && python -m pip install         poetry                                                                                                                      92.6s
 => [8/9] RUN useradd -m -s /bin/bash -u 5000 -U vscode     && echo "vscode ALL=(ALL) NOPASSWD:ALL" >> /etc/sudoers.d/ALL                                                              0.4s
 => [9/9] RUN poetry config virtualenvs.in-project true                                                                                                                                0.7s
 => exporting to image                                                                                                                                                                 0.9s
 => => exporting layers                                                                                                                                                                0.9s
 => => writing image sha256:f942eb288ef5102cdc4e8b2f3daabad1f74962e673e5d4e809f995d8aec7b874

まとめ

  • Ubuntu 22.04のpython3-pipパッケージはpython3-distutilsに依存している
  • Python 3.12からdistutilsは廃止された
  • python3-pipでインストールしたpipをpython3.12 -m pipで実行すると、No module named 'distutils'というエラーが発生する

補足

関連するパッケージの情報を掲載します。

vscode@c3aecfddeafb:/$ sudo apt show python3.12-dev
Package: python3.12-dev
Version: 3.12.4-1+jammy1
Status: install ok installed
Priority: optional
Section: python
Source: python3.12
Maintainer: Matthias Klose <doko@debian.org>
Installed-Size: 514 kB
Depends: python3.12 (= 3.12.4-1+jammy1), libpython3.12-dev (= 3.12.4-1+jammy1), libpython3.12 (= 3.12.4-1+jammy1), libexpat1-dev
Recommends: libc6-dev | libc-dev
Download-Size: 不明
APT-Manual-Installed: yes
APT-Sources: /var/lib/dpkg/status
Description: Header files and a static library for Python (v3.12)
 Header files, a static library and development tools for building
 Python (v3.12) modules, extending the Python interpreter or embedding
 Python (v3.12) in applications.
 .
 Maintainers of Python packages should read README.maintainers.


vscode@c3aecfddeafb:/$ apt show python3-distutils
Package: python3-distutils
Version: 3.10.8-1~22.04
Status: install ok installed
Priority: optional
Section: python
Source: python3-stdlib-extensions
Maintainer: Ubuntu Developers <ubuntu-devel-discuss@lists.ubuntu.com>
Original-Maintainer: Matthias Klose <doko@debian.org>
Installed-Size: 775 kB
Provides: python3.10-distutils, python3.11-distutils
Depends: python3:any (>= 3.10.5-0~), python3:any (<< 3.12), python3-lib2to3 (= 3.10.8-1~22.04)
Breaks: libpython3.10-stdlib (<< 3.10.0~b1), libpython3.6-stdlib (<< 3.6.5~rc1-3), libpython3.7-stdlib (<< 3.7.0~b2-2), libpython3.8-stdlib (<< 3.8.0~b2-5)
Replaces: libpython3.6-stdlib (<< 3.6.4~rc1-2), libpython3.7-stdlib (<< 3.7.0~a3-2)
Download-Size: 不明
APT-Manual-Installed: no
APT-Sources: /var/lib/dpkg/status
Description: distutils package for Python 3.x
 Distutils package for Python 3.x.  This package contains the distutils module
 from the Python standard library.

  1. apt show python3のバージョンは3.10.6で、/usr/bin/python3.10 --versionのバージョンは3.10.12でパッチバージョンが異なっていました。deadsnakes PPAを追加したため、3.10.6より新しいバージョンがインストールされたのかもしれません。原因はよく分かっていません。 ↩

0
0
1

Register as a new user and use Qiita more conveniently

  1. You get articles that match your needs
  2. You can efficiently read back useful information
  3. You can use dark theme
What you can do with signing up
Sign upLogin
0
0

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?