はじめに

久しぶりに純粋なPython環境にPoetryを入れたのですが、以前入っていた構成でエラーがでるようになっていました。
2020年7月にはインストールできたので最近変更があったのかと思います。

記事があまりなかったためまとめたいと思います。

環境

以下のDockerfileを利用します。

FROM python:3.7-alpine

USER root
WORKDIR /app

RUN apk update
RUN apk --no-cache add\
		musl\
		musl-dev\
		libffi-dev\
		gcc\
		g++\
		make\
		gfortran\
		openblas-dev\
		python3-dev\
		zeromq-dev

RUN pip install --upgrade pip

RUN pip install\
		jupyterlab==3.1.4\
		poetry==1.1.7

COPY ./config/jupyter_notebook_config.py /root/.jupyter/jupyter_notebook_config.py

問題

Python環境にPoetryをインストールしようとしたところ、以下のようなエラーがでました。

# 8 64.12   warning: no previously-included files matching '*' found under directory '.zuul.playbooks'
# 8 64.12   adding license file 'LICENSE'
# 8 64.12   adding license file 'LICENSE.APACHE'
# 8 64.12   adding license file 'LICENSE.BSD'
# 8 64.12   adding license file 'LICENSE.PSF'
# 8 64.12   writing manifest file 'src/cryptography.egg-info/SOURCES.txt'    
# 8 64.12   copying src/cryptography/py.typed -> build/lib.linux-x86_64-3.7/cryptography
# 8 64.12   running build_ext
# 8 64.12   generating cffi module 'build/temp.linux-x86_64-3.7/_padding.c'  
# 8 64.12   creating build/temp.linux-x86_64-3.7
# 8 64.12   generating cffi module 'build/temp.linux-x86_64-3.7/_openssl.c'  
# 8 64.12   running build_rust
# 8 64.12
# 8 64.12       =============================DEBUG ASSISTANCE=============================
# 8 64.12       If you are seeing a compilation error please try the following steps to
# 8 64.12       successfully install cryptography:
# 8 64.12       1) Upgrade to the latest pip and try again. This will fix errors for most
# 8 64.12          users. See: https://pip.pypa.io/en/stable/installing/#upgrading-pip
# 8 64.12       2) Read https://cryptography.io/en/latest/installation.html for specific
# 8 64.12          instructions for your platform.
# 8 64.12       3) Check our frequently asked questions for more information:
# 8 64.12          https://cryptography.io/en/latest/faq.html
# 8 64.12       4) Ensure you have a recent Rust toolchain installed:        
# 8 64.12          https://cryptography.io/en/latest/installation.html#rust  
# 8 64.12       5) If you are experiencing issues with Rust for *this release only* you may
# 8 64.12          set the environment variable `CRYPTOGRAPHY_DONT_BUILD_RUST=1`.
# 8 64.12       =============================DEBUG ASSISTANCE=============================
# 8 64.12
# 8 64.12   error: can't find Rust compiler
# 8 64.12
# 8 64.12   If you are using an outdated pip version, it is possible a prebuilt wheel is available for this package but pip is not able to install from it. Installing from the wheel would avoid the need for a Rust compiler.      
# 8 64.12
# 8 64.12   To update pip, run:
# 8 64.12
# 8 64.12       pip install --upgrade pip
# 8 64.12
# 8 64.12   and then retry package installation.
# 8 64.12
# 8 64.12   If you did intend to build this package from source, try installing a Rust compiler from your system package manager and ensure it is on the PATH during installation. Alternatively, rustup (available at https://rustup.rs) is the recommended way to download and update the Rust compiler toolchain.
# 8 64.12
# 8 64.12   This package requires Rust >=1.41.0.
# 8 64.12   ----------------------------------------
# 8 64.12   ERROR: Failed building wheel for cryptography
# 8 64.12   Building wheel for tornado (setup.py): started
# 8 64.52   Building wheel for tornado (setup.py): finished with status 'done'
# 8 64.52   Created wheel for tornado: filename=tornado-6.1-cp37-cp37m-linux_x86_64.whl size=416726 sha256=4dfb0cbde3aa1c45981851c9871ecb724fdd1c335a3aeaa46736c1eb1d156899
# 8 64.52   Stored in directory: /root/.cache/pip/wheels/02/62/2c/f52c662d8ae374c4abda0c13ce432fb58eb1c75281a27b406c
# 8 64.53   Building wheel for argon2-cffi (PEP 517): started
# 8 66.01   Building wheel for argon2-cffi (PEP 517): finished with status 'done'
# 8 66.01   Created wheel for argon2-cffi: filename=argon2_cffi-20.1.0-cp37-abi3-linux_x86_64.whl size=39542 sha256=b47426e306d0ea2eb7625ea8cf02ede2055cd0ef534273d44f4301c51767dca1
# 8 66.01   Stored in directory: /root/.cache/pip/wheels/57/12/fd/e064dbe3ee0236aa6c8c6c318a18821413886309a24e5f8fe7
# 8 66.01   Building wheel for pandocfilters (setup.py): started
# 8 66.21   Building wheel for pandocfilters (setup.py): finished with status 'done'
# 8 66.21   Created wheel for pandocfilters: filename=pandocfilters-1.4.3-py3-none-any.whl size=8006 sha256=baa7bd528fec4a5ddbeb16d8e79614527ae1435bbdebaaeafe75d6e2446ceaea
# 8 66.21   Stored in directory: /root/.cache/pip/wheels/42/81/34/545dc2fbf0e9137811e901108d37fc04650e81d48f97078000
# 8 66.21 Successfully built MarkupSafe msgpack pyrsistent pyzmq tornado argon2-cffi pandocfilters
# 8 66.21 Failed to build cryptography
# 8 66.21 ERROR: Could not build wheels for cryptography which use PEP 517 and cannot be installed directly
------
executor failed running [/bin/sh -c pip install         jupyterlab==3.1.4  poetry==1.1.7]: exit code: 1
ERROR: Service 'python3' failed to build : Build failed
PS C:\Users\jinwa\Desktop\workspace\docker-kadai>

2020年7月ごろにインストールしたときには問題なくインストールできていました。

エラーのこの行でCryptographyがないといわれているので別途インストールします。

pip install cryptography

FROM python:3.7-alpine

USER root
WORKDIR /app

RUN apk update
RUN apk --no-cache add\
		musl\
		musl-dev\
		libffi-dev\
		gcc\
		g++\
		make\
		gfortran\
		openblas-dev\
		python3-dev\
		zeromq-dev

RUN pip install --upgrade pip

RUN pip install\
		jupyterlab==3.1.4\
		cryptography\
		poetry==1.1.7

COPY ./config/jupyter_notebook_config.py /root/.jupyter/jupyter_notebook_config.py

しかし、次はCryptographyのインストールで以下のエラーがでます。

# 10 40.47   writing manifest file 'src/cryptography.egg-info/SOURCES.txt'   
# 10 40.47   copying src/cryptography/py.typed -> build/lib.linux-x86_64-3.7/cryptography
# 10 40.47   running build_ext
# 10 40.47   generating cffi module 'build/temp.linux-x86_64-3.7/_padding.c' 
# 10 40.47   creating build/temp.linux-x86_64-3.7
# 10 40.47   generating cffi module 'build/temp.linux-x86_64-3.7/_openssl.c' 
# 10 40.47   running build_rust
# 10 40.47
# 10 40.47       =============================DEBUG ASSISTANCE=============================
# 10 40.47       If you are seeing a compilation error please try the following steps to
# 10 40.47       successfully install cryptography:
# 10 40.47       1) Upgrade to the latest pip and try again. This will fix errors for most
# 10 40.47          users. See: https://pip.pypa.io/en/stable/installing/#upgrading-pip
# 10 40.47       2) Read https://cryptography.io/en/latest/installation.html for specific
# 10 40.47          instructions for your platform.
# 10 40.47       3) Check our frequently asked questions for more information:
# 10 40.47          https://cryptography.io/en/latest/faq.html
# 10 40.47       4) Ensure you have a recent Rust toolchain installed:       
# 10 40.47          https://cryptography.io/en/latest/installation.html#rust 
# 10 40.47       5) If you are experiencing issues with Rust for *this release only* you may
# 10 40.47          set the environment variable `CRYPTOGRAPHY_DONT_BUILD_RUST=1`.
# 10 40.47       =============================DEBUG ASSISTANCE=============================
# 10 40.47
# 10 40.47   error: can't find Rust compiler
# 10 40.47
# 10 40.47   If you are using an outdated pip version, it is possible a prebuilt wheel is available for this package but pip is not able to install from it. Installing from the wheel would avoid the need for a Rust compiler.     
# 10 40.47
# 10 40.47   To update pip, run:
# 10 40.47
# 10 40.47       pip install --upgrade pip
# 10 40.47
# 10 40.47   and then retry package installation.
# 10 40.47
# 10 40.47   If you did intend to build this package from source, try installing a Rust compiler from your system package manager and ensure it is on the PATH during installation. Alternatively, rustup (available at https://rustup.rs) is the recommended way to download and update the Rust compiler toolchain.
# 10 40.47
# 10 40.47   This package requires Rust >=1.41.0.
# 10 40.47   ----------------------------------------
# 10 40.47   ERROR: Failed building wheel for cryptography
# 10 40.47   Building wheel for MarkupSafe (setup.py): started
# 10 40.89   Building wheel for MarkupSafe (setup.py): finished with status 'done'
# 10 40.89   Created wheel for MarkupSafe: filename=MarkupSafe-2.0.1-cp37-cp37m-linux_x86_64.whl size=14614 sha256=b97ae5e0e1a407416bd63fcc0d2573dcd015e6d91d47bcb6420bf7ae82bb46f8
# 10 40.89   Stored in directory: /root/.cache/pip/wheels/1a/18/04/e3b5bd888f000c2716bccc94a565239f9defc47ef93d9e7bea
# 10 40.90   Building wheel for msgpack (setup.py): started
# 10 44.77   Building wheel for msgpack (setup.py): finished with status 'done'
# 10 44.78   Created wheel for msgpack: filename=msgpack-1.0.2-cp37-cp37m-linux_x86_64.whl size=75254 sha256=3a5fecd1f4058df1bbf2947f503c6e55843a31629df0ee1a97d0284a965b3d4d
# 10 44.78   Stored in directory: /root/.cache/pip/wheels/ea/83/11/4a476c036e5a0df485a31ab1cf05d2678a6d31621c2c1d1b23
# 10 44.78   Building wheel for pyrsistent (PEP 517): started
# 10 45.73   Building wheel for pyrsistent (PEP 517): finished with status 'done'
# 10 45.73   Created wheel for pyrsistent: filename=pyrsistent-0.18.0-cp37-cp37m-linux_x86_64.whl size=72168 sha256=736fc8cf64b5f57a7d48696feb1fd3c12b83f1e9c28508b9d7f91b2c2cb40872
# 10 45.73   Stored in directory: /root/.cache/pip/wheels/54/35/6e/9827c774ec40ae001ae2a2ef063268add65762c4014951e97c
# 10 45.74   Building wheel for pyzmq (PEP 517): started
# 10 62.29   Building wheel for pyzmq (PEP 517): finished with status 'done' 
# 10 62.29   Created wheel for pyzmq: filename=pyzmq-22.2.1-cp37-cp37m-linux_x86_64.whl size=494634 sha256=426c4bbcde51377b41fb3efeb582350a1f56983adffc916a5b1eda2994a44ca8
# 10 62.29   Stored in directory: /root/.cache/pip/wheels/3a/27/44/03391fba33f48d9af48569d79dcc99f8e9935cf175c8a4d588
# 10 62.30   Building wheel for tornado (setup.py): started
# 10 62.68   Building wheel for tornado (setup.py): finished with status 'done'
# 10 62.68   Created wheel for tornado: filename=tornado-6.1-cp37-cp37m-linux_x86_64.whl size=416726 sha256=8562d180bdb55f611f16770448a196bc683938cfa8509315e5717c055d8864a1
# 10 62.68   Stored in directory: /root/.cache/pip/wheels/02/62/2c/f52c662d8ae374c4abda0c13ce432fb58eb1c75281a27b406c
# 10 62.69   Building wheel for argon2-cffi (PEP 517): started
# 10 64.08   Building wheel for argon2-cffi (PEP 517): finished with status 'done'
# 10 64.08   Created wheel for argon2-cffi: filename=argon2_cffi-20.1.0-cp37-abi3-linux_x86_64.whl size=39542 sha256=2aa48ebbaf40f8778605b0f7c28c8c3097f4a838be98baacaadb9a15b68343e1
# 10 64.08   Stored in directory: /root/.cache/pip/wheels/57/12/fd/e064dbe3ee0236aa6c8c6c318a18821413886309a24e5f8fe7
# 10 64.08   Building wheel for pandocfilters (setup.py): started
# 10 64.28   Building wheel for pandocfilters (setup.py): finished with status 'done'
# 10 64.28   Created wheel for pandocfilters: filename=pandocfilters-1.4.3-py3-none-any.whl size=8006 sha256=626527f324c6808ad496f529c8277a1426ca64f07678519dc55d535a4ddc7396
# 10 64.28   Stored in directory: /root/.cache/pip/wheels/42/81/34/545dc2fbf0e9137811e901108d37fc04650e81d48f97078000
# 10 64.28 Successfully built MarkupSafe msgpack pyrsistent pyzmq tornado argon2-cffi pandocfilters
# 10 64.28 Failed to build cryptography
# 10 64.28 ERROR: Could not build wheels for cryptography which use PEP 517 and cannot be installed directly
------
executor failed running [/bin/sh -c pip install         jupyterlab==3.1.4  cryptography             poetry==1.1.7]: exit code: 1
ERROR: Service 'python3' failed to build : Build failed

おそらくCryptographyをインストールができれば、Poetryもインストールできそうです。

解決方法

色々、試してみたところ以下の2点を行うことでできるようになりました。

openssl-devをインストールする

以下の行をDockerfileに追加します。

apk add openssl-dev

rustのコンパイル設定を回避する

[Python cryptography がインストールできなくなったら rustup で rust をインストールする

をみたところ解決方法がわかりました。

現在(2021年8月)のCryptographyのバージョン3.4.7はRustを使ってライブラリができているため、Rustのコンパイラが必要になります。
Rustの環境が必要なのですがPoetryをインストールするために使うので、以前(3.4.1以下)のバージョンをRustなしで使う方法で回避します。

How to get rid of cryptography build error?を参考に設定しました。

まずDockerfileに以下の行を追加します。

ENV CRYPTOGRAPHY_DONT_BUILD_RUST=1

そのあと、Cryptographyのバージョンを3.4.1に固定します。

pip install cryptography==3.4.1

すべての修正を行ったDockerfileが以下となります。

FROM python:3.7-alpine

USER root
WORKDIR /app

RUN apk update
RUN apk --no-cache add\
		musl\
		musl-dev\
		libffi-dev\
		gcc\
		g++\
		make\
		gfortran\
		openblas-dev\
		python3-dev\
		zeromq-dev\
		openssl-dev

RUN pip install --upgrade pip

ENV CRYPTOGRAPHY_DONT_BUILD_RUST=1

RUN pip install\
		jupyterlab==3.1.4\
		cryptography==3.4.1\
		poetry==1.1.7

COPY ./config/jupyter_notebook_config.py /root/.jupyter/jupyter_notebook_config.py

無事インストールができました。
openssl-devをインストールし忘れるので注意してください。

おわりに

前使っていたコンテナもやはり時間とともに利用できなくなってしまします。
つねに調べないと維持していくのは難しいです。

参考