環境

Ubuntu 22.04

はじめに

以下のDockerfileは、VSCode Dev Containers用のDockerfileです。Python 3.11用の開発環境を構築しています。

Dockerfile.python311

FROM ubuntu:22.04

# Install prerequisites
RUN set -x \
    && export DEBIAN_FRONTEND=noninteractive \
    && apt-get update \
    && apt-get install -y --no-install-recommends \
        gnupg2 \
        software-properties-common \
        language-pack-ja \
        tzdata \
        curl \
        lsb-release \
    && apt-get -y clean \
    && rm -rf /var/lib/apt/lists/*

# Set locale & timezone
RUN update-locale LANG=ja_JP.UTF-8 LANGUAGE=ja_JP:ja \
    && ln -sf /usr/share/zoneinfo/Asia/Tokyo /etc/localtime \
    && echo "Asia/Tokyo" > /etc/timezone

ENV LANG=ja_JP.UTF-8
ENV LC_ALL=ja_JP.UTF-8
ENV LC_CTYPE=ja_JP.UTF-8

# Install packages
RUN set -x \
    # Python3.11をインストールするために、deadsnakes PPAを追加
    && add-apt-repository ppa:deadsnakes/ppa \
    && apt-get update \
    && apt-get install -y --no-install-recommends \
        python3.11-dev \
        python3-pip \
        # common tools
        bash-completion \
        build-essential \
        git \
        iputils-ping \
        jq \
        less \
        net-tools \
        openssh-client \
        sudo \
        tar \
        time \
        unzip \
        vim \
        wget \
        xz-utils \
        zip \        
    && apt-get -y clean \
    && rm -rf /var/lib/apt/lists/*

# Python / pip
RUN ln -s $(which python3.11) /usr/bin/python

# poetry
RUN set -x \
    && python -m pip install -U \
        pip \
    && python -m pip install \
        poetry

# Add user / Grant sudo privileges
RUN useradd -m -s /bin/bash -u 5000 -U vscode \
    && echo "vscode ALL=(ALL) NOPASSWD:ALL" >> /etc/sudoers.d/ALL

USER vscode

RUN poetry config virtualenvs.in-project true

$ docker buildx build --file Dockerfile.python311 .

 => [internal] load build definition from Dockerfile.python311                                                                                                                         0.0s
 => => transferring dockerfile: 1.72kB                                                                                                                                                 0.0s
 => [internal] load metadata for docker.io/library/ubuntu:22.04                                                                                                                        1.6s
 => [internal] load .dockerignore                                                                                                                                                      0.0s
 => => transferring context: 2B                                                                                                                                                        0.0s
 => [1/8] FROM docker.io/library/ubuntu:22.04@sha256:340d9b015b194dc6e2a13938944e0d016e57b9679963fdeb9ce021daac430221                                                                  0.0s
 => CACHED [2/8] RUN set -x     && export DEBIAN_FRONTEND=noninteractive     && apt-get update     && apt-get install -y --no-install-recommends         gnupg2         software-prop  0.0s
 => CACHED [3/8] RUN update-locale LANG=ja_JP.UTF-8 LANGUAGE=ja_JP:ja     && ln -sf /usr/share/zoneinfo/Asia/Tokyo /etc/localtime     && echo "Asia/Tokyo" > /etc/timezone             0.0s
 => [4/8] RUN set -x     && add-apt-repository ppa:deadsnakes/ppa     && apt-get update     && apt-get install -y --no-install-recommends         python3.11-dev         python3-p  1551.7s
 => [5/8] RUN ln -s $(which python3.11) /usr/bin/python                                                                                                                                0.3s
 => [6/8] RUN set -x     && python -m pip install -U         pip     && python -m pip install         poetry                                                                         141.3s
 => [7/8] RUN useradd -m -s /bin/bash -u 5000 -U vscode     && echo "vscode ALL=(ALL) NOPASSWD:ALL" >> /etc/sudoers.d/ALL                                                              0.3s
 => [8/8] RUN poetry config virtualenvs.in-project true                                                                                                                                0.7s
 => exporting to image                                                                                                                                                                 1.0s
 => => exporting layers                                                                                                                                                                1.0s
 => => writing image sha256:dd91ef557975345566381dbf282135bdb9ebdb0a6c86d5dcd4de5815354513d2

何が起きたのか

Pythonのバージョンを3.11から3.12に変更するため、以下のようなDockerfileに変更しました。

Dockerfile.python312-ng

# ...
RUN set -x \
    # Python3.12をインストールするために、deadsnakes PPAを追加
    && add-apt-repository ppa:deadsnakes/ppa \
    && apt-get update \
    && apt-get install -y --no-install-recommends \
        python3.12-dev \
        python3-pip \

# ...
RUN ln -s $(which python3.12) /usr/bin/python

$ diff Dockerfile.python311 Dockerfile.python312-ng
28c28
<     # Python3.11をインストールするために、deadsnakes PPAを追加
---
>     # Python3.12をインストールするために、deadsnakes PPAを追加
32c32
<         python3.11-dev \
---
>         python3.12-dev \
55c55
< RUN ln -s $(which python3.11) /usr/bin/python
---
> RUN ln -s $(which python3.12) /usr/bin/python

しかし、docker buildx buildは失敗しました。python -m pip install -U pipで失敗していました。

$ docker buildx build --file Dockerfile.python312-ng .

=> [internal] load build definition from Dockerfile.python312-ng                                                                                                                      0.0s
=> => transferring dockerfile: 1.73kB                                                                                                                                                 0.0s
=> [internal] load metadata for docker.io/library/ubuntu:22.04                                                                                                                        1.5s
=> [internal] load .dockerignore                                                                                                                                                      0.0s
=> => transferring context: 2B                                                                                                                                                        0.0s
=> [1/8] FROM docker.io/library/ubuntu:22.04@sha256:340d9b015b194dc6e2a13938944e0d016e57b9679963fdeb9ce021daac430221                                                                  0.0s
=> CACHED [2/8] RUN set -x     && export DEBIAN_FRONTEND=noninteractive     && apt-get update     && apt-get install -y --no-install-recommends         gnupg2         software-prop  0.0s
=> CACHED [3/8] RUN update-locale LANG=ja_JP.UTF-8 LANGUAGE=ja_JP:ja     && ln -sf /usr/share/zoneinfo/Asia/Tokyo /etc/localtime     && echo "Asia/Tokyo" > /etc/timezone             0.0s
=> [4/8] RUN set -x     && add-apt-repository ppa:deadsnakes/ppa     && apt-get update     && apt-get install -y --no-install-recommends         python3.12-dev         python3-p  1695.2s
=> [5/8] RUN ln -s $(which python3.12) /usr/bin/python                                                                                                                                0.3s
=> ERROR [6/8] RUN set -x     && python -m pip install -U         pip     && python -m pip install         poetry                                                                     0.8s
------
> [6/8] RUN set -x     && python -m pip install -U         pip     && python -m pip install         poetry:
0.332 + python -m pip install -U pip
0.710 Traceback (most recent call last):
0.710   File "<frozen runpy>", line 198, in _run_module_as_main
0.710   File "<frozen runpy>", line 88, in _run_code
0.710   File "/usr/lib/python3/dist-packages/pip/__main__.py", line 29, in <module>
0.710     from pip._internal.cli.main import main as _main
0.710   File "/usr/lib/python3/dist-packages/pip/_internal/cli/main.py", line 9, in <module>
0.710     from pip._internal.cli.autocompletion import autocomplete
0.710   File "/usr/lib/python3/dist-packages/pip/_internal/cli/autocompletion.py", line 10, in <module>
0.710     from pip._internal.cli.main_parser import create_main_parser
0.710   File "/usr/lib/python3/dist-packages/pip/_internal/cli/main_parser.py", line 8, in <module>
0.710     from pip._internal.cli import cmdoptions
0.710   File "/usr/lib/python3/dist-packages/pip/_internal/cli/cmdoptions.py", line 23, in <module>
0.710     from pip._internal.cli.parser import ConfigOptionParser
0.710   File "/usr/lib/python3/dist-packages/pip/_internal/cli/parser.py", line 12, in <module>
0.710     from pip._internal.configuration import Configuration, ConfigurationError
0.710   File "/usr/lib/python3/dist-packages/pip/_internal/configuration.py", line 26, in <module>
0.710     from pip._internal.utils.logging import getLogger
0.710   File "/usr/lib/python3/dist-packages/pip/_internal/utils/logging.py", line 27, in <module>
0.710     from pip._internal.utils.misc import ensure_dir
0.710   File "/usr/lib/python3/dist-packages/pip/_internal/utils/misc.py", line 39, in <module>
0.710     from pip._internal.locations import get_major_minor_version
0.710   File "/usr/lib/python3/dist-packages/pip/_internal/locations/__init__.py", line 14, in <module>
0.710     from . import _distutils, _sysconfig
0.710   File "/usr/lib/python3/dist-packages/pip/_internal/locations/_distutils.py", line 9, in <module>
0.710     from distutils.cmd import Command as DistutilsCommand
0.710 ModuleNotFoundError: No module named 'distutils'
------
Dockerfile.python312-ng:58
--------------------
 57 |     # poetry
 58 | >>> RUN set -x \
 59 | >>>     && python -m pip install -U \
 60 | >>>         pip \
 61 | >>>     && python -m pip install \
 62 | >>>         poetry
 63 |
--------------------
ERROR: failed to solve: process "/bin/sh -c set -x     && python -m pip install -U         pip     && python -m pip install         poetry" did not complete successfully: exit code: 1

エラーの原因

Python3.12ではdistutilsは削除されました。

このDockerfileでは、pipコマンドをpython3-pipパッケージでインストールしています。
apt showコマンドでpython3-pipを調べると、python3-distutilsに依存していました。
したがって、python -m pipコマンドを実行したときに、「ModuleNotFoundError: No module named 'distutils'」というエラーが発生したようです。

vscode@c3aecfddeafb:/$ sudo apt show python3-pip
Package: python3-pip
Version: 22.0.2+dfsg-1ubuntu0.4
Status: install ok installed
Priority: optional
Section: python
Source: python-pip
Maintainer: Ubuntu Developers <ubuntu-devel-discuss@lists.ubuntu.com>
Original-Maintainer: Debian Python Team <team+python@tracker.debian.org>
Installed-Size: 7,043 kB
Provides: pip
Depends: ca-certificates, python3-distutils, python3-setuptools, python3-wheel, python3:any
Recommends: build-essential, python3-dev (>= 3.2)
Breaks: python-pip
Replaces: python-pip
Homepage: https://pip.pypa.io/en/stable/
Download-Size: 不明
APT-Manual-Installed: yes
APT-Sources: /var/lib/dpkg/status
Description: Python package installer
 pip is the Python package installer.  It integrates with virtualenv, doesn't
 do partial installs, can save package state for replaying, can install from
 non-egg sources, and can install from version control repositories.
 .
 This is the Python 3 version of the package.

なお、python -m pipでなくpipコマンドならば動作します。/usr/bin/pipはPython3.10環境で実行されているためです。

vscode@c3aecfddeafb:/$ which pip
/usr/bin/pip
vscode@c3aecfddeafb:/$ pip --version
pip 22.0.2 from /usr/lib/python3/dist-packages/pip (python 3.10)

/usr/bin/pip

#!/usr/bin/python3
# -*- coding: utf-8 -*-
import re
import sys
from pip._internal.cli.main import main
if __name__ == "__main__":
    sys.argv[0] = re.sub(r"(-script\.pyw|\.exe)?$", "", sys.argv[0])
    sys.exit(main())

vscode@c3aecfddeafb:/$ /usr/bin/python3 --version
Python 3.10.12

python3.10は、software-properties-commonパッケージをインストールしたとときに、一緒にインストールされました。software-properties-commonはpython3に依存しています。そして、python3パッケージのバージョンは3.10です。¹

$ apt show software-properties-common
Package: software-properties-common
Version: 0.99.22.9
Priority: optional
Section: admin
Source: software-properties
Origin: Ubuntu
Maintainer: Michael Vogt <michael.vogt@ubuntu.com>
Bugs: https://bugs.launchpad.net/ubuntu/+filebug
Installed-Size: 225 kB
Depends: ca-certificates, gir1.2-glib-2.0, gir1.2-packagekitglib-1.0 (>= 1.1.0-2), packagekit, python-apt-common (>= 0.9), python3, python3-dbus, python3-gi, python3-software-properties (= 0.99.22.9), python3:any
Breaks: python-software-properties (<< 0.85), python3-software-properties (<< 0.85)
Replaces: python-software-properties (<< 0.85), python3-software-properties (<< 0.85)
Task: ubuntu-desktop-minimal, ubuntu-desktop, cloud-image, ubuntu-wsl, server, ubuntu-server-raspi, kubuntu-desktop, xubuntu-core, xubuntu-desktop, lubuntu-desktop, ubuntustudio-desktop-core, ubuntustudio-desktop, ubuntukylin-desktop, ubuntu-mate-core, ubuntu-mate-desktop, ubuntu-budgie-desktop, ubuntu-budgie-desktop-raspi
Download-Size: 14.1 kB
APT-Manual-Installed: yes
APT-Sources: http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages
Description: manage the repositories that you install software from (common)

$ apt show python3
Package: python3
Version: 3.10.6-1~22.04
Priority: important
Section: python
Source: python3-defaults
Origin: Ubuntu
Maintainer: Ubuntu Developers <ubuntu-devel-discuss@lists.ubuntu.com>
Original-Maintainer: Matthias Klose <doko@debian.org>
Bugs: https://bugs.launchpad.net/ubuntu/+filebug
Installed-Size: 92.2 kB
Provides: python3-profiler
Pre-Depends: python3-minimal (= 3.10.6-1~22.04)
Depends: python3.10 (>= 3.10.6-1~), libpython3-stdlib (= 3.10.6-1~22.04)
Suggests: python3-doc (>= 3.10.6-1~22.04), python3-tk (>= 3.10.6-1~), python3-venv (>= 3.10.6-1~22.04)
Replaces: python3-minimal (<< 3.1.2-2)
Homepage: https://www.python.org/
Task: minimal, server-minimal
Download-Size: 22.8 kB
APT-Manual-Installed: no
APT-Sources: http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages
Description: interactive high-level object-oriented language (default python3 version)

解決策

https://bootstrap.pypa.io/get-pip.py からget-pip.pyをダウンロードして実行すれば、Python3.12でも動作するpipがインストールされます。

vscode@c3aecfddeafb:/$ curl -sSL https://bootstrap.pypa.io/get-pip.py | python -
Defaulting to user installation because normal site-packages is not writeable
Collecting pip
  Downloading pip-24.1.2-py3-none-any.whl.metadata (3.6 kB)
Downloading pip-24.1.2-py3-none-any.whl (1.8 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.8/1.8 MB 235.3 kB/s eta 0:00:00
Installing collected packages: pip
  WARNING: The scripts pip, pip3 and pip3.12 are installed in '/home/vscode/.local/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
Successfully installed pip-24.1.2

vscode@c3aecfddeafb:/$ python -m pip --version
pip 24.1.2 from /home/vscode/.local/lib/python3.12/site-packages/pip (python 3.12)

vscode@c3aecfddeafb:/$ ls -la  ~/.local/bin/pip*
-rwxr-xr-x 1 vscode vscode 220  7月 18 16:18 /home/vscode/.local/bin/pip
-rwxr-xr-x 1 vscode vscode 220  7月 18 16:18 /home/vscode/.local/bin/pip3
-rwxr-xr-x 1 vscode vscode 220  7月 18 16:18 /home/vscode/.local/bin/pip3.12

以下は、Python3.12用のDockerfileの全体になります。

Dockerfile.python312

FROM ubuntu:22.04

# Install prerequisites
RUN set -x \
    && export DEBIAN_FRONTEND=noninteractive \
    && apt-get update \
    && apt-get install -y --no-install-recommends \
        gnupg2 \
        software-properties-common \
        language-pack-ja \
        tzdata \
        curl \
        lsb-release \
    && apt-get -y clean \
    && rm -rf /var/lib/apt/lists/*

# Set locale & timezone
RUN update-locale LANG=ja_JP.UTF-8 LANGUAGE=ja_JP:ja \
    && ln -sf /usr/share/zoneinfo/Asia/Tokyo /etc/localtime \
    && echo "Asia/Tokyo" > /etc/timezone

ENV LANG=ja_JP.UTF-8
ENV LC_ALL=ja_JP.UTF-8
ENV LC_CTYPE=ja_JP.UTF-8

# Install packages
RUN set -x \
    # Python3.12をインストールするために、deadsnakes PPAを追加
    && add-apt-repository ppa:deadsnakes/ppa \
    && apt-get update \
    && apt-get install -y --no-install-recommends \
        python3.12-dev \
        # common tools
        bash-completion \
        build-essential \
        git \
        iputils-ping \
        jq \
        less \
        net-tools \
        openssh-client \
        sudo \
        tar \
        time \
        unzip \
        vim \
        wget \
        xz-utils \
        zip \        
    && apt-get -y clean \
    && rm -rf /var/lib/apt/lists/*

# Python / pip
RUN ln -s $(which python3.12) /usr/bin/python
RUN curl -sSL https://bootstrap.pypa.io/get-pip.py | python -

# poetry
RUN set -x \
    && python -m pip install \
        poetry

# Add user / Grant sudo privileges
RUN useradd -m -s /bin/bash -u 5000 -U vscode \
    && echo "vscode ALL=(ALL) NOPASSWD:ALL" >> /etc/sudoers.d/ALL

USER vscode

RUN poetry config virtualenvs.in-project true

$ docker buildx build --file Dockerfile.python312 .

 => [internal] load build definition from Dockerfile.python312                                                                                                                         0.0s
 => => transferring dockerfile: 1.72kB                                                                                                                                                 0.0s
 => [internal] load metadata for docker.io/library/ubuntu:22.04                                                                                                                        1.8s
 => [internal] load .dockerignore                                                                                                                                                      0.0s
 => => transferring context: 2B                                                                                                                                                        0.0s
 => [1/9] FROM docker.io/library/ubuntu:22.04@sha256:340d9b015b194dc6e2a13938944e0d016e57b9679963fdeb9ce021daac430221                                                                  0.0s
 => CACHED [2/9] RUN set -x     && export DEBIAN_FRONTEND=noninteractive     && apt-get update     && apt-get install -y --no-install-recommends         gnupg2         software-prop  0.0s
 => CACHED [3/9] RUN update-locale LANG=ja_JP.UTF-8 LANGUAGE=ja_JP:ja     && ln -sf /usr/share/zoneinfo/Asia/Tokyo /etc/localtime     && echo "Asia/Tokyo" > /etc/timezone             0.0s
 => [4/9] RUN set -x     && add-apt-repository ppa:deadsnakes/ppa     && apt-get update     && apt-get install -y --no-install-recommends         python3.12-dev         bash-comp  1777.7s
 => [5/9] RUN ln -s $(which python3.12) /usr/bin/python                                                                                                                                0.2s
 => [6/9] RUN curl -sSL https://bootstrap.pypa.io/get-pip.py | python -                                                                                                               58.6s
 => [7/9] RUN set -x     && python -m pip install         poetry                                                                                                                      92.6s
 => [8/9] RUN useradd -m -s /bin/bash -u 5000 -U vscode     && echo "vscode ALL=(ALL) NOPASSWD:ALL" >> /etc/sudoers.d/ALL                                                              0.4s
 => [9/9] RUN poetry config virtualenvs.in-project true                                                                                                                                0.7s
 => exporting to image                                                                                                                                                                 0.9s
 => => exporting layers                                                                                                                                                                0.9s
 => => writing image sha256:f942eb288ef5102cdc4e8b2f3daabad1f74962e673e5d4e809f995d8aec7b874

まとめ

Ubuntu 22.04のpython3-pipパッケージはpython3-distutilsに依存している
Python 3.12からdistutilsは廃止された
python3-pipでインストールしたpipをpython3.12 -m pipで実行すると、No module named 'distutils'というエラーが発生する

補足

関連するパッケージの情報を掲載します。

vscode@c3aecfddeafb:/$ sudo apt show python3.12-dev
Package: python3.12-dev
Version: 3.12.4-1+jammy1
Status: install ok installed
Priority: optional
Section: python
Source: python3.12
Maintainer: Matthias Klose <doko@debian.org>
Installed-Size: 514 kB
Depends: python3.12 (= 3.12.4-1+jammy1), libpython3.12-dev (= 3.12.4-1+jammy1), libpython3.12 (= 3.12.4-1+jammy1), libexpat1-dev
Recommends: libc6-dev | libc-dev
Download-Size: 不明
APT-Manual-Installed: yes
APT-Sources: /var/lib/dpkg/status
Description: Header files and a static library for Python (v3.12)
 Header files, a static library and development tools for building
 Python (v3.12) modules, extending the Python interpreter or embedding
 Python (v3.12) in applications.
 .
 Maintainers of Python packages should read README.maintainers.


vscode@c3aecfddeafb:/$ apt show python3-distutils
Package: python3-distutils
Version: 3.10.8-1~22.04
Status: install ok installed
Priority: optional
Section: python
Source: python3-stdlib-extensions
Maintainer: Ubuntu Developers <ubuntu-devel-discuss@lists.ubuntu.com>
Original-Maintainer: Matthias Klose <doko@debian.org>
Installed-Size: 775 kB
Provides: python3.10-distutils, python3.11-distutils
Depends: python3:any (>= 3.10.5-0~), python3:any (<< 3.12), python3-lib2to3 (= 3.10.8-1~22.04)
Breaks: libpython3.10-stdlib (<< 3.10.0~b1), libpython3.6-stdlib (<< 3.6.5~rc1-3), libpython3.7-stdlib (<< 3.7.0~b2-2), libpython3.8-stdlib (<< 3.8.0~b2-5)
Replaces: libpython3.6-stdlib (<< 3.6.4~rc1-2), libpython3.7-stdlib (<< 3.7.0~a3-2)
Download-Size: 不明
APT-Manual-Installed: no
APT-Sources: /var/lib/dpkg/status
Description: distutils package for Python 3.x
 Distutils package for Python 3.x.  This package contains the distutils module
 from the Python standard library.

apt show python3のバージョンは3.10.6で、/usr/bin/python3.10 --versionのバージョンは3.10.12でパッチバージョンが異なっていました。deadsnakes PPAを追加したため、3.10.6より新しいバージョンがインストールされたのかもしれません。原因はよく分かっていません。 ↩