GitLab.comの「パッケージレジストリ」を使って無料でPyPIミラーのレジストリをサクっと使えたのでメモ。

トークンを発行して権限制御しつつ、.lockを使ったライブラリバージョンの固定もいい感じにできた。

動作環境

GitLab.com: 13.11-pre (※2021/4/7時点)
poetry: 1.1.5
twine: 3.4.1
pipenv: 2020.11.15

ライブラリの準備

サンプル用に簡単なライブラリを用意してみた。

ただ足し算するだけ

>>> from awesum import awesum
>>> awesum(1, 2)
3
>>> awesum(1, 2, 3)
6
>>> awesum(1, -1, 2, -2)
0

パッケージレジストリ準備

デプロイトークン準備

事前にトークンを発行しておく必要がある。

メニューバー[設定] > [リポジトリ]
[Deploy tokens]でパッケージレジストリへの権限をつける
- read_package_registry
- write_package_registry

※[名前]に設定した値も後ほど使用するので控えておく

認証用の設定ファイル作成

~/.pypircにトークンの情報を含める。

参考: PyPI packages in the Package Registry | GitLab

~/.pypirc

# 参考URLママ
[distutils]
index-servers =
    gitlab

[gitlab]
repository = https://gitlab.example.com/api/v4/projects/<project_id>/packages/pypi
username = <deploy token username>
password = <deploy token>

上記のトークンと組み合わせると下記のようになる

~/.pypirc

[distutils]
index-servers =
    gitlab

[gitlab]
repository = https://gitlab.com/api/v4/projects/25700635/packages/pypi
username = skokado
password = <deploy token>

※<project_id>はリポジトリのトップ画面で確認できる

パッケージング、アップロード

poetryとtwineを使う

$ python3 -m pip install poetry twine

$ # パッケージング
$ git clone git@gitlab.com:skokado/awesum.git
$ poetry build --format wheel
Building awesum (0.1.0)
  - Building wheel
  - Built awesum-0.1.0-py3-none-any.whl

$ ls -l dist/
-rw-r--r-- 1 skokado skokado 1.5K Apr  7 22:44 awesum-0.1.0-py3-none-any.whl

$ # アップロード
$ twine upload --repository gitlab dist/*

無事にアップロードできると[パッケージレジストリ]が以下のようになる

パッケージレジストリの使用

※pipenvを使用する例で紹介。

アクセストークンの準備

レジストリにアクセスするためのパーソナルアクセストークンを発行しておく

※上記で作成した[Deploy tokens]とは別

右上メニュー[Preference](設定)
左側メニュー[アクセストークン]よりトークンを作成
- [スコープ]はread_apiのみでOK

プロジェクト作成

$ pipenv --three

自動作成されるPipfileにPyPIミラーの情報を追加する

※環境変数$GITLAB_TOKENを利用できるようにする

Pipfile

[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

+ [[source]]
+ url = "https://__token__:$GITLAB_TOKEN@gitlab.com/api/v4/projects/25700635/packages/pypi/simple"
+ verify_ssl = true
+ name = "gitlab"

[packages]

[dev-packages]

[requires]
python_version = "3.8"

インストール

GitLab内[パッケージレジストリ]のページでは下記のコマンドが紹介されている

※コマンドがpipの部分をpipenvに置き換える

$ export GITLAB_TOKEN=xxx
$ pipenv install awesum --extra-index-url https://__token__:$GITLAB_TOKEN@gitlab.com/api/v4/projects/25700635/packages/pypi/simple
Installing awesum...
Adding awesum to Pipfile's [packages]...
✔ Installation Succeeded 
Pipfile.lock not found, creating...
Locking [dev-packages] dependencies...
Locking [packages] dependencies...
Building requirements...
Resolving dependencies...
✔ Success! 
Updated Pipfile.lock (0f9f1e)!
Installing dependencies from Pipfile.lock (0f9f1e)...
  🐍   ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 0/0 — 00:00:00
To activate this project's virtualenv, run pipenv shell.
Alternatively, run a command inside the virtualenv with pipenv run.

ここで、自動作成されたPipfile.lockにおいてindexがpypiとなってしまっているので修正する。

Pipfile.lock

{
    "_meta": {
        "hash": {
            "sha256": "2ca0c992446d43a908934fa7b5de037410087de92fbbf2ad346b5f152d43ff89"
        },
        "pipfile-spec": 6,
        "requires": {
            "python_version": "3.8"
        },
        "sources": [
            {
                "name": "pypi",
                "url": "https://pypi.org/simple",
                "verify_ssl": true
            },
            {
                "name": "gitlab",
                "url": "https://__token__:$GITLAB_TOKEN@gitlab.com/api/v4/projects/25700635/packages/pypi/simple",
                "verify_ssl": true
            }
        ]
    },
    "default": {
        "awesum": {
            "hashes": [
                "sha256:cba516281d2d0d296b118edab45de09185810a6a28f7b3ce7caba6e8df67e413"
            ],
-             "index": "pypi",
+             "index": "gitlab",
            "version": "==0.1.0"
        }
    },
    "develop": {}
}

ここまで実施しておけば、次回以降は下記のように環境再現が可能となる。

$ # 仮想環境ごと事前に削除しておく
$ ls -l
-rw-rw-r--  1 skokado skokado  413 Apr  7 23:04 Pipfile
-rw-r--r--  1 skokado skokado  983 Apr  7 23:04 Pipfile.lock

$ pipenv sync -v
Using python: 3.8
Path to python: /home/skokado/.pyenv/versions/3.8.8/bin/python3.8
Creating a virtualenv for this project...
Pipfile: /home/skokado/workspace/hello-awesum/Pipfile
Using /home/skokado/.pyenv/versions/3.8.8/bin/python3.8 (3.8.8) to create virtualenv...
⠸ Creating virtual environment...created virtual environment CPython3.8.8.final.0-64 in 219ms
  creator CPython3Posix(dest=/home/skokado/.local/share/virtualenvs/hello-awesum-A6Q6PeYw, clear=False, no_vcs_ignore=False, global=False)
  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/home/skokado/.local/share/virtualenv)
    added seed packages: pip==21.0.1, setuptools==54.0.0, wheel==0.36.2
  activators BashActivator,CShellActivator,FishActivator,PowerShellActivator,PythonActivator,XonshActivator

✔ Successfully created virtual environment! 
Virtualenv location: /home/skokado/.local/share/virtualenvs/hello-awesum-A6Q6PeYw
Installing dependencies from Pipfile.lock (0f9f1e)...
Writing supplied requirement line to temporary file: 'awesum==0.1.0 --hash=sha256:cba516281d2d0d296b118edab45de09185810a6a28f7b3ce7caba6e8df67e413'
Installing 'awesum'
$ ['/home/skokado/.local/share/virtualenvs/hello-awesum-A6Q6PeYw/bin/pip', 'install', '--verbose', '--upgrade', '--require-hashes', '--no-deps', '--exists-action=i', '-r', '/tmp/pipenv-__k3ethv-requirements/pipenv-yq4kpesd-requirement.txt', '-i', 'https://pypi.org/simple', '--extra-index-url', 'https://__token__:R7P3gwHCr27xGF_g5r8r@gitlab.com/api/v4/projects/25700635/packages/pypi/simple', '--extra-index-url', 'https://__token__:R7P3gwHCr27xGF_g5r8r@gitlab.com/api/v4/projects/25700635/packages/pypi/simple']
Using source directory: '/home/skokado/.local/share/virtualenvs/hello-awesum-A6Q6PeYw/src'
Using pip 21.0.1 from /home/skokado/.local/share/virtualenvs/hello-awesum-A6Q6PeYw/lib/python3.8/site-packages/pip (python 3.8)
Non-user install by explicit request
Created temporary directory: /tmp/pip-ephem-wheel-cache-t8ranvj1
Created temporary directory: /tmp/pip-req-tracker-h3ie50r6
Initialized build tracking at /tmp/pip-req-tracker-h3ie50r6
Created build tracker: /tmp/pip-req-tracker-h3ie50r6
Entered build tracker: /tmp/pip-req-tracker-h3ie50r6
Created temporary directory: /tmp/pip-install-7zltel63
Looking in indexes: https://pypi.org/simple, https://__token__:****@gitlab.com/api/v4/projects/25700635/packages/pypi/simple, https://__token__:****@gitlab.com/api/v4/projects/25700635/packages/pypi/simple
2 location(s) to search for versions of awesum:
* https://pypi.org/simple/awesum/
* https://__token__:****@gitlab.com/api/v4/projects/25700635/packages/pypi/simple/awesum/
Fetching project page and analyzing links: https://pypi.org/simple/awesum/
Getting page https://pypi.org/simple/awesum/
Found index url https://pypi.org/simple
Looking up "https://pypi.org/simple/awesum/" in the cache
Request header has "max_age" as 0, cache bypassed
Starting new HTTPS connection (1): pypi.org:443
https://pypi.org:443 "GET /simple/awesum/ HTTP/1.1" 404 13
Status code 404 not in (200, 203, 300, 301)
Could not fetch URL https://pypi.org/simple/awesum/: 404 Client Error: Not Found for url: https://pypi.org/simple/awesum/ - skipping
Fetching project page and analyzing links: https://__token__:****@gitlab.com/api/v4/projects/25700635/packages/pypi/simple/awesum/
Getting page https://__token__:****@gitlab.com/api/v4/projects/25700635/packages/pypi/simple/awesum/
Found credentials in url for gitlab.com
Looking up "https://gitlab.com/api/v4/projects/25700635/packages/pypi/simple/awesum/" in the cache
Request header has "max_age" as 0, cache bypassed
Starting new HTTPS connection (1): gitlab.com:443
https://gitlab.com:443 "GET /api/v4/projects/25700635/packages/pypi/simple/awesum/ HTTP/1.1" 304 0
  Found link https://gitlab.com/api/v4/projects/25700635/packages/pypi/files/cba516281d2d0d296b118edab45de09185810a6a28f7b3ce7caba6e8df67e413/awesum-0.1.0-py3-none-any.whl#sha256=cba516281d2d0d296b118edab45de09185810a6a28f7b3ce7caba6e8df67e413 (from https://gitlab.com/api/v4/projects/25700635/packages/pypi/simple/awesum/) (requires-python:>=3.6,<4.0), version: 0.1.0
Checked 1 links for project 'awesum' against 1 hashes (1 matches, 0 no digest): discarding no candidates
Collecting awesum==0.1.0
  Created temporary directory: /tmp/pip-unpack-3ai1rpb3
  Looking up "https://gitlab.com/api/v4/projects/25700635/packages/pypi/files/cba516281d2d0d296b118edab45de09185810a6a28f7b3ce7caba6e8df67e413/awesum-0.1.0-py3-none-any.whl" in the cache
  Current age based on date: 162
  Freshness lifetime from max-age: 0
  https://gitlab.com:443 "GET /api/v4/projects/25700635/packages/pypi/files/cba516281d2d0d296b118edab45de09185810a6a28f7b3ce7caba6e8df67e413/awesum-0.1.0-py3-none-any.whl HTTP/1.1" 304 0
  Using cached https://gitlab.com/api/v4/projects/25700635/packages/pypi/files/cba516281d2d0d296b118edab45de09185810a6a28f7b3ce7caba6e8df67e413/awesum-0.1.0-py3-none-any.whl (1.4 kB)
  Added awesum==0.1.0 from https://gitlab.com/api/v4/projects/25700635/packages/pypi/files/cba516281d2d0d296b118edab45de09185810a6a28f7b3ce7caba6e8df67e413/awesum-0.1.0-py3-none-any.whl#sha256=cba516281d2d0d296b118edab45de09185810a6a28f7b3ce7caba6e8df67e413 (from -r /tmp/pipenv-__k3ethv-requirements/pipenv-yq4kpesd-requirement.txt (line 1)) to build tracker '/tmp/pip-req-tracker-h3ie50r6'
  Removed awesum==0.1.0 from https://gitlab.com/api/v4/projects/25700635/packages/pypi/files/cba516281d2d0d296b118edab45de09185810a6a28f7b3ce7caba6e8df67e413/awesum-0.1.0-py3-none-any.whl#sha256=cba516281d2d0d296b118edab45de09185810a6a28f7b3ce7caba6e8df67e413 (from -r /tmp/pipenv-__k3ethv-requirements/pipenv-yq4kpesd-requirement.txt (line 1)) from build tracker '/tmp/pip-req-tracker-h3ie50r6'
Created temporary directory: /tmp/pip-unpack-zlqrqbuy
Installing collected packages: awesum

Successfully installed awesum-0.1.0
Removed build tracker: '/tmp/pip-req-tracker-h3ie50r6'
  🐍   ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 1/1 — 00:00:01
To activate this project's virtualenv, run pipenv shell.
Alternatively, run a command inside the virtualenv with pipenv run.
All dependencies are now up-to-date!

GitLab.comでサクっと無料のPyPIミラーを使う