LoginSignup
0
1

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?

More than 3 years have passed since last update.

@DS27

【メモ】ライブラリーのバージョン管理について

Posted at
  • 製造業出身のデータサイエンティストがお送りする記事
  • 今回はメモとして、ライブラリーのバージョン管理方法を残しておきます。

はじめに

自分で作成したスクリプトを関係者へ渡す際に、一緒に動作環境も提供するかと思います。その際、最近はdockerを使っているのですが、ライブラリーのバージョンを管理する方法をメモとして残しておきます。

pip freezeで一覧を見る

今回、dockerで構築した環境は下記です。

dockerfiel
FROM ubuntu:18.04

ENV PYTHON_VERSION 3.8.6
ENV PYTHON_ROOT /tmp/Python/python-$PYTHON_VERSION
ENV PATH $PYTHON_ROOT/bin:$PATH
ENV PYENV_ROOT /tmp/.pyenv

WORKDIR /usr/src/app
ENV DEBIAN_FRONTEND noninteractive

RUN apt-get update \
    && apt-get install -y locales \
    && locale-gen ja_JP.UTF-8 \
    && echo "export LANG=ja_JP.UTF-8" >> ~/.bashrc \
    && apt install -y --no-install-recommends \
    build-essential \
    ca-certificates \
    libssl-dev \
    zlib1g-dev \
    libbz2-dev \
    libreadline-dev \
    libsqlite3-dev \
    libncurses5-dev \
    libncursesw5-dev \
    libffi-dev \
    liblzma-dev \
    vim \
    ssh \
    wget \
    xz-utils \
    tk-dev \
    git \
    && apt clean \
    && rm -rf /var/lib/apt/lists/*

# # Python環境構築
RUN git clone https://github.com/pyenv/pyenv.git $PYENV_ROOT
RUN $PYENV_ROOT/plugins/python-build/install.sh
RUN /usr/local/bin/python-build -v $PYTHON_VERSION $PYTHON_ROOT
RUN rm -rf $PYENV_ROOT

RUN pip install --upgrade pip

COPY requirements.txt ./
RUN pip install -r requirements.txt

WORKDIR home/work
docker-compose.yml
version: "3"
services:
  jupyter:
    build: .
    volumes:
      - ./:/work
    working_dir: /work
    ports:
      - 8888:8888
    command: jupyter lab --ip=0.0.0.0 --allow-root --no-browser --NotebookApp.token=''
requirements.txt
numpy==1.20.1
pandas==1.2.4
jupyterlab==3.0.12
matplotlib==3.3.4
seaborn==0.11.1
scikit-learn==0.24.1
plotly==4.14.3

実際にdockerで構築した環境でpip freezeでライブラリー一覧を確認してみた結果が下記です。

anyio==3.0.1
argon2-cffi==20.1.0
async-generator==1.10
attrs==21.2.0
Babel==2.9.1
backcall==0.2.0
bleach==3.3.0
certifi==2020.12.5
cffi==1.14.5
chardet==4.0.0
cycler==0.10.0
decorator==5.0.8
defusedxml==0.7.1
entrypoints==0.3
idna==2.10
ipykernel==5.5.5
ipython==7.23.1
ipython-genutils==0.2.0
jedi==0.18.0
Jinja2==3.0.0
joblib==1.0.1
json5==0.9.5
jsonschema==3.2.0
jupyter-client==6.1.12
jupyter-core==4.7.1
jupyter-packaging==0.7.12
jupyter-server==1.7.0
jupyterlab==3.0.12
jupyterlab-pygments==0.1.2
jupyterlab-server==2.5.1
kiwisolver==1.3.1
MarkupSafe==2.0.0
matplotlib==3.3.4
matplotlib-inline==0.1.2
mistune==0.8.4
nbclassic==0.2.8
nbclient==0.5.3
nbconvert==6.0.7
nbformat==5.1.3
nest-asyncio==1.5.1
notebook==6.3.0
numpy==1.20.1
packaging==20.9
pandas==1.2.4
pandocfilters==1.4.3
parso==0.8.2
pexpect==4.8.0
pickleshare==0.7.5
Pillow==8.2.0
plotly==4.14.3
prometheus-client==0.10.1
prompt-toolkit==3.0.18
ptyprocess==0.7.0
pycparser==2.20
Pygments==2.9.0
pyparsing==2.4.7
pyrsistent==0.17.3
python-dateutil==2.8.1
pytz==2021.1
pyzmq==22.0.3
requests==2.25.1
retrying==1.3.3
scikit-learn==0.24.1
scipy==1.6.3
seaborn==0.11.1
Send2Trash==1.5.0
six==1.16.0
sniffio==1.2.0
terminado==0.9.5
testpath==0.4.4
threadpoolctl==2.1.0
tornado==6.1
traitlets==5.0.5
urllib3==1.26.4
wcwidth==0.2.5
webencodings==0.5.1
websocket-client==0.59.0

複数のパッケージの依存関係で様々なライブラリーが一緒にインストールされていることが分かります。
普段はこれをrequirements.txtに書きます。

pip-chillを使ってみた

今回はpip-chillというライブラリーを使ってみました。
まずは、インストールします。

$ pip install pip-chill

使い方はpip freezeと似ておりますので、以下のコマンドで実行してみます。

$ pip-chill

# jupyterlab==3.0.12
# pip-chill==1.0.1
# plotly==4.14.3
# scikit-learn==0.24.1
# seaborn==0.11.1

自分でインストールしたライブラリーがわかります。
requirements.txtで書いたものと少し異なりますが、細かい部分は今回は省略します。

次にパッケージ名だけを出力させます。

$ pip-chill --no-version

# jupyterlab
# pip-chill
# plotly
# scikit-learn
# seaborn

最後に、ライブラリーの依存関係を確認することもできます。

$ pip-chill -v

jupyterlab==3.0.12
pip-chill==1.0.1
plotly==4.14.3
scikit-learn==0.24.1
seaborn==0.11.1
# anyio==3.0.1 # Installed as dependency for jupyter-server
# argon2-cffi==20.1.0 # Installed as dependency for jupyter-server, notebook
# async-generator==1.10 # Installed as dependency for nbclient
# attrs==21.2.0 # Installed as dependency for jsonschema
# babel==2.9.1 # Installed as dependency for jupyterlab-server
# backcall==0.2.0 # Installed as dependency for ipython
# bleach==3.3.0 # Installed as dependency for nbconvert
# certifi==2020.12.5 # Installed as dependency for requests
# cffi==1.14.5 # Installed as dependency for argon2-cffi
# chardet==4.0.0 # Installed as dependency for requests
# cycler==0.10.0 # Installed as dependency for matplotlib
# decorator==5.0.8 # Installed as dependency for ipython
# defusedxml==0.7.1 # Installed as dependency for nbconvert
# entrypoints==0.3 # Installed as dependency for nbconvert
# idna==2.10 # Installed as dependency for anyio, requests
# ipykernel==5.5.5 # Installed as dependency for notebook
# ipython==7.23.1 # Installed as dependency for jupyterlab, ipykernel
# ipython-genutils==0.2.0 # Installed as dependency for jupyter-server, traitlets, nbformat, notebook
# jedi==0.18.0 # Installed as dependency for ipython
# jinja2==3.0.0 # Installed as dependency for jupyter-server, jupyterlab-server, notebook, nbconvert, jupyterlab
# joblib==1.0.1 # Installed as dependency for scikit-learn
# json5==0.9.5 # Installed as dependency for jupyterlab-server
# jsonschema==3.2.0 # Installed as dependency for jupyterlab-server, nbformat
# jupyter-client==6.1.12 # Installed as dependency for jupyter-server, nbclient, ipykernel, notebook
# jupyter-core==4.7.1 # Installed as dependency for jupyter-server, jupyter-client, notebook, nbformat, nbconvert, jupyterlab
# jupyter-packaging==0.7.12 # Installed as dependency for jupyterlab
# jupyter-server==1.7.0 # Installed as dependency for nbclassic, jupyterlab, jupyterlab-server
# jupyterlab-pygments==0.1.2 # Installed as dependency for nbconvert
# jupyterlab-server==2.5.1 # Installed as dependency for jupyterlab
# kiwisolver==1.3.1 # Installed as dependency for matplotlib
# markupsafe==2.0.0 # Installed as dependency for jinja2
# matplotlib==3.3.4 # Installed as dependency for seaborn
# matplotlib-inline==0.1.2 # Installed as dependency for ipython
# mistune==0.8.4 # Installed as dependency for nbconvert
# nbclassic==0.2.8 # Installed as dependency for jupyterlab
# nbclient==0.5.3 # Installed as dependency for nbconvert
# nbconvert==6.0.7 # Installed as dependency for jupyter-server, notebook
# nbformat==5.1.3 # Installed as dependency for jupyter-server, nbconvert, nbclient, notebook
# nest-asyncio==1.5.1 # Installed as dependency for nbclient
# notebook==6.3.0 # Installed as dependency for nbclassic
# numpy==1.20.1 # Installed as dependency for matplotlib, scipy, scikit-learn, seaborn, pandas
# packaging==20.9 # Installed as dependency for jupyterlab, bleach, jupyterlab-server, jupyter-packaging
# pandas==1.2.4 # Installed as dependency for seaborn
# pandocfilters==1.4.3 # Installed as dependency for nbconvert
# parso==0.8.2 # Installed as dependency for jedi
# pexpect==4.8.0 # Installed as dependency for ipython
# pickleshare==0.7.5 # Installed as dependency for ipython
# pillow==8.2.0 # Installed as dependency for matplotlib
# prometheus-client==0.10.1 # Installed as dependency for jupyter-server, notebook
# prompt-toolkit==3.0.18 # Installed as dependency for ipython
# ptyprocess==0.7.0 # Installed as dependency for terminado, pexpect
# pycparser==2.20 # Installed as dependency for cffi
# pygments==2.9.0 # Installed as dependency for ipython, nbconvert, jupyterlab-pygments
# pyparsing==2.4.7 # Installed as dependency for packaging, matplotlib
# pyrsistent==0.17.3 # Installed as dependency for jsonschema
# python-dateutil==2.8.1 # Installed as dependency for jupyter-client, matplotlib, pandas
# pytz==2021.1 # Installed as dependency for pandas, babel
# pyzmq==22.0.3 # Installed as dependency for jupyter-server, jupyter-client, notebook
# requests==2.25.1 # Installed as dependency for jupyterlab-server
# retrying==1.3.3 # Installed as dependency for plotly
# scipy==1.6.3 # Installed as dependency for scikit-learn, seaborn
# send2trash==1.5.0 # Installed as dependency for jupyter-server, notebook
# six==1.16.0 # Installed as dependency for websocket-client, argon2-cffi, cycler, plotly, retrying, python-dateutil, bleach, jsonschema
# sniffio==1.2.0 # Installed as dependency for anyio
# terminado==0.9.5 # Installed as dependency for jupyter-server, notebook
# testpath==0.4.4 # Installed as dependency for nbconvert
# threadpoolctl==2.1.0 # Installed as dependency for scikit-learn
# tornado==6.1 # Installed as dependency for jupyter-server, jupyter-client, notebook, terminado, jupyterlab, ipykernel
# traitlets==5.0.5 # Installed as dependency for jupyter-server, jupyter-client, notebook, ipython, nbclient, nbformat, nbconvert, jupyter-core, matplotlib-inline, ipykernel
# urllib3==1.26.4 # Installed as dependency for requests
# wcwidth==0.2.5 # Installed as dependency for prompt-toolkit
# webencodings==0.5.1 # Installed as dependency for bleach
# websocket-client==0.59.0 # Installed as dependency for jupyter-server

これでライブラリー同士の依存関係がわかります。
依存関係があるライブラリーに関してはコメントアウトされて出力されます。

さいごに

最後まで読んで頂き、ありがとうございました。
簡単ですが、ライブラリーのバージョンを管理する方法をメモとして残しておきました。

訂正要望がありましたら、ご連絡頂けますと幸いです。

0
1
0

Register as a new user and use Qiita more conveniently

  1. You get articles that match your needs
  2. You can efficiently read back useful information
  3. You can use dark theme
What you can do with signing up
Sign upLogin
0
1

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?