Harbor on Alibaba Cloud ECS (CentOS7) を構築してみた

はじめに

オープンソースのコンテナレジストリ「Harbor」をアリババクラウド上で構築してみたので、手順などをご紹介します。

なお、本手順は Harbor on Kubernetes ではなく ECS 上に構築する手順になります。

公式の手順は以下を参照ください。
https://github.com/goharbor/harbor/blob/master/docs/installation_guide.md

前提

・Alibaba Cloud Elastic Compute Service（ECS） にて CentOS7 の購入や接続手順は割愛します。
・本番利用ではなく、SSL化やドメインの設定をしないテスト目的での構築手順となります。
・環境はシングル構成で以下の様な感じです。 NASを使うのも良いかもしれません。

Docker Engine をインストールする

sudo yum install -y yum-utils device-mapper-persistent-data lvm2
sudo yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
sudo yum install -y docker-ce docker-ce-cli containerd.io
sudo systemctl start docker
sudo systemctl enable docker

Docker Compose をインストールする

sudo curl -L "https://github.com/docker/compose/releases/download/1.24.1/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
sudo chmod +x /usr/local/bin/docker-compose
docker-compose --version

Harbor のインストール

Download the Installer

Go to the Harbor releases page.
https://github.com/goharbor/harbor/releases

Harbor のリリースページからインストールモジュールをダウンロードします。

wget https://github.com/goharbor/harbor/releases/download/v1.9.2/harbor-online-installer-v1.9.2.tgz

取得したモジュールを展開します。

tar xvf harbor-online-installer-v1.9.2.tgz 
 harbor/prepare
 harbor/LICENSE
 harbor/install.sh
 harbor/harbor.yml

Configure Harbor

harbor.ymlが構成ファイルになります。

cat harbor/harbor.yml 

初期の定義は以下の様になっています。

# Configuration file of Harbor

# The IP address or hostname to access admin UI and registry service.
# DO NOT use localhost or 127.0.0.1, because Harbor needs to be accessed by external clients.
hostname: reg.mydomain.com

# http related config
http:
  # port for http, default is 80. If https enabled, this port will redirect to https port
  port: 80

# https related config
# https:
#   # https port for harbor, default is 443
#   port: 443
#   # The path of cert and key files for nginx
#   certificate: /your/certificate/path
#   private_key: /your/private/key/path

# Uncomment external_url if you want to enable external proxy
# And when it enabled the hostname will no longer used
# external_url: https://reg.mydomain.com:8433

# The initial password of Harbor admin
# It only works in first time to install harbor
# Remember Change the admin password from UI after launching Harbor.
harbor_admin_password: Harbor12345

# Harbor DB configuration
database:
  # The password for the root user of Harbor DB. Change this before any production use.
  password: root123
  # The maximum number of connections in the idle connection pool. If it <=0, no idle connections are retained.
  max_idle_conns: 50
  # The maximum number of open connections to the database. If it <= 0, then there is no limit on the number of open connections.
  # Note: the default number of connections is 100 for postgres.
  max_open_conns: 100

# The default data volume
data_volume: /data

# Harbor Storage settings by default is using /data dir on local filesystem
# Uncomment storage_service setting If you want to using external storage
# storage_service:
#   # ca_bundle is the path to the custom root ca certificate, which will be injected into the truststore
#   # of registry's and chart repository's containers.  This is usually needed when the user hosts a internal storage with self signed certificate.
#   ca_bundle:

#   # storage backend, default is filesystem, options include filesystem, azure, gcs, s3, swift and oss
#   # for more info about this configuration please refer https://docs.docker.com/registry/configuration/
#   filesystem:
#     maxthreads: 100
#   # set disable to true when you want to disable registry redirect
#   redirect:
#     disabled: false

# Clair configuration
clair:
  # The interval of clair updaters, the unit is hour, set to 0 to disable the updaters.
  updaters_interval: 12

jobservice:
  # Maximum number of job workers in job service
  max_job_workers: 10

notification:
  # Maximum retry count for webhook job
  webhook_job_max_retry: 10

chart:
  # Change the value of absolute_url to enabled can enable absolute url in chart
  absolute_url: disabled

# Log configurations
log:
  # options are debug, info, warning, error, fatal
  level: info
  # configs for logs in local storage
  local:
    # Log files are rotated log_rotate_count times before being removed. If count is 0, old versions are removed rather than rotated.
    rotate_count: 50
    # Log files are rotated only if they grow bigger than log_rotate_size bytes. If size is followed by k, the size is assumed to be in kilobytes.
    # If the M is used, the size is in megabytes, and if G is used, the size is in gigabytes. So size 100, size 100k, size 100M and size 100G
    # are all valid.
    rotate_size: 200M
    # The directory on your host that store log
    location: /var/log/harbor

  # Uncomment following lines to enable external syslog endpoint.
  # external_endpoint:
  #   # protocol used to transmit log to external endpoint, options is tcp or udp
  #   protocol: tcp
  #   # The host of external endpoint
  #   host: localhost
  #   # Port of external endpoint
  #   port: 5140

# This attribute is for migrator to detect the version of the .cfg file, DO NOT MODIFY!
_version: 1.9.0

# Uncomment external_database if using external database.
# external_database:
#   harbor:
#     host: harbor_db_host
#     port: harbor_db_port
#     db_name: harbor_db_name
#     username: harbor_db_username
#     password: harbor_db_password
#     ssl_mode: disable
#     max_idle_conns: 2
#     max_open_conns: 0
#   clair:
#     host: clair_db_host
#     port: clair_db_port
#     db_name: clair_db_name
#     username: clair_db_username
#     password: clair_db_password
#     ssl_mode: disable
#   notary_signer:
#     host: notary_signer_db_host
#     port: notary_signer_db_port
#     db_name: notary_signer_db_name
#     username: notary_signer_db_username
#     password: notary_signer_db_password
#     ssl_mode: disable
#   notary_server:
#     host: notary_server_db_host
#     port: notary_server_db_port
#     db_name: notary_server_db_name
#     username: notary_server_db_username
#     password: notary_server_db_password
#     ssl_mode: disable

# Uncomment external_redis if using external Redis server
# external_redis:
#   host: redis
#   port: 6379
#   password:
#   # db_index 0 is for core, it's unchangeable
#   registry_db_index: 1
#   jobservice_db_index: 2
#   chartmuseum_db_index: 3

# Uncomment uaa for trusting the certificate of uaa instance that is hosted via self-signed cert.
# uaa:
#   ca_file: /path/to/ca

# Global proxy
# Config http proxy for components, e.g. http://my.proxy.com:3128
# Components doesn't need to connect to each others via http proxy.
# Remove component from `components` array if want disable proxy
# for it. If you want use proxy for replication, MUST enable proxy
# for core and jobservice, and set `http_proxy` and `https_proxy`.
# Add domain to the `no_proxy` field, when you want disable proxy
# for some special registry.
proxy:
  http_proxy:
  https_proxy:
  no_proxy: 127.0.0.1,localhost,.local,.internal,log,db,redis,nginx,core,portal,postgresql,jobservice,registry,registryctl,clair
  components:
    - core
    - jobservice
    - clair

とりあえずhostnameだけ変えます。
今回は簡単にIPアドレスでのHTTPアクセスを行うためココだけの変更。
ちゃんと使う場合はHTTPS化やドメイン名の付与をすること。

hostname 47.74.63.216

データボリューム用の領域（ディレクトリ）を作成します。
可用性を考えてAlibaba Cloud NASをマウントするとかもありだと思います。

sudo mkdir /data

Installating and starting Harbor

notary, clair, chartmuseum を含まない場合は以下のコマンドでインストール。

sudo harbor/install.sh

notary, clair, chartmuseum を含む場合は以下のコマンドでインストール。
ちなみに notary を使うにはHTTPS化が必要です。

sudo harbor/install.sh --with-notary --with-clair --with-chartmuseum

notary, clair, chartmuseum を含まない場合は以下の様なログが表示されます。

[Step 0]: checking installation environment ...

Note: docker version: 19.03.4

Note: docker-compose version: 1.24.1


[Step 1]: preparing environment ...
prepare base dir is set to /root/harbor
Generated configuration file: /config/log/logrotate.conf
Generated configuration file: /config/log/rsyslog_docker.conf
Generated configuration file: /config/nginx/nginx.conf
Generated configuration file: /config/core/env
Generated configuration file: /config/core/app.conf
Generated configuration file: /config/registry/config.yml
Generated configuration file: /config/registryctl/env
Generated configuration file: /config/db/env
Generated configuration file: /config/jobservice/env
Generated configuration file: /config/jobservice/config.yml
Generated and saved secret to file: /secret/keys/secretkey
Generated certificate, key file: /secret/core/private_key.pem, cert file: /secret/registry/root.crt
Generated configuration file: /compose_location/docker-compose.yml
Clean up the input dir



[Step 2]: starting Harbor ...
Creating network "harbor_harbor" with the default driver
Pulling log (goharbor/harbor-log:v1.9.2)...
v1.9.2: Pulling from goharbor/harbor-log
b950b5dd94ab: Already exists
b6a7312503ba: Pull complete
760803b9b21b: Pull complete
9c3fc2887d1d: Pull complete
faea1cb0344f: Pull complete
d5399dcf8730: Pull complete
a02b766dea1d: Pull complete
e43fcf0795ba: Pull complete
Digest: sha256:10c8c48c9e3726e1881897678f9d5dacbad5a39be3b5eb0d64125499d50fc970
Status: Downloaded newer image for goharbor/harbor-log:v1.9.2
Pulling registry (goharbor/registry-photon:v2.7.1-patch-2819-2553-v1.9.2)...
v2.7.1-patch-2819-2553-v1.9.2: Pulling from goharbor/registry-photon
b950b5dd94ab: Already exists
23964a389c95: Pull complete
5dfec0eb3cae: Pull complete
82799791afc5: Pull complete
9e7766341161: Pull complete
6d0b96e9cb3a: Pull complete
Digest: sha256:90d02b4b665555b8188dd3b9bca65c6d9a948199931e70119c3043b2ff70c5fb
Status: Downloaded newer image for goharbor/registry-photon:v2.7.1-patch-2819-2553-v1.9.2
Pulling registryctl (goharbor/harbor-registryctl:v1.9.2)...
v1.9.2: Pulling from goharbor/harbor-registryctl
b950b5dd94ab: Already exists
566b9f7336c0: Pull complete
e842e809901d: Pull complete
a270e8eee3b6: Pull complete
ce812a2873dd: Pull complete
939135ef2476: Pull complete
f0be85117918: Pull complete
Digest: sha256:0124fde3752883caa57279da5e117650d5eed58625b91c2af78159c022c74683
Status: Downloaded newer image for goharbor/harbor-registryctl:v1.9.2
Pulling postgresql (goharbor/harbor-db:v1.9.2)...
v1.9.2: Pulling from goharbor/harbor-db
b950b5dd94ab: Already exists
c7e639172b16: Pull complete
127f22b5670c: Pull complete
cf89cb6c5479: Pull complete
80f918f3a69a: Pull complete
2afd0dc2405f: Pull complete
2003c3603415: Pull complete
c5fd78d0ce54: Pull complete
2e71ced1c9f0: Pull complete
Digest: sha256:1ec1781aafc408002aef1bfc69b02ce85b2866b760fc52b3eeb3c857846b6e79
Status: Downloaded newer image for goharbor/harbor-db:v1.9.2
Pulling portal (goharbor/harbor-portal:v1.9.2)...
v1.9.2: Pulling from goharbor/harbor-portal
b950b5dd94ab: Already exists
ce141502b953: Pull complete
ce05d705a764: Pull complete
ae33dd8259a6: Pull complete
3cb6296b1dfe: Pull complete
6877ccdb46c1: Pull complete
cb57b0d38086: Pull complete
Digest: sha256:556887904bd72709fbef90ee83185c284b572116e85282823e56355f57802c37
Status: Downloaded newer image for goharbor/harbor-portal:v1.9.2
Pulling redis (goharbor/redis-photon:v1.9.2)...
v1.9.2: Pulling from goharbor/redis-photon
b950b5dd94ab: Already exists
e0ddba13389b: Pull complete
49cf9705895c: Pull complete
43b65b6111b8: Pull complete
7cdaf28af490: Pull complete
Digest: sha256:2de70968cd2e84bfd3dbc7f0496f86c69a43451ef2f117ae319c3e1985a37bce
Status: Downloaded newer image for goharbor/redis-photon:v1.9.2
Pulling core (goharbor/harbor-core:v1.9.2)...
v1.9.2: Pulling from goharbor/harbor-core
b950b5dd94ab: Already exists
4a85783cb579: Pull complete
a1a8eeb23a7f: Pull complete
2df127e1c3c2: Pull complete
892aafd3189c: Pull complete
8b0051daebb2: Pull complete
Digest: sha256:d6a7d0d1f0ebb5e7e040b28b6434bc7dd429b7814c6b1205d47cc2ae13228b07
Status: Downloaded newer image for goharbor/harbor-core:v1.9.2
Pulling jobservice (goharbor/harbor-jobservice:v1.9.2)...
v1.9.2: Pulling from goharbor/harbor-jobservice
b950b5dd94ab: Already exists
0ecb5506a657: Pull complete
e2a2439c5edc: Pull complete
Digest: sha256:c78ebeb1935d374db2bdd493c9a33a2a962f29bee075e57ea2b4825f74e63b22
Status: Downloaded newer image for goharbor/harbor-jobservice:v1.9.2
Pulling proxy (goharbor/nginx-photon:v1.9.2)...
v1.9.2: Pulling from goharbor/nginx-photon
b950b5dd94ab: Already exists
3a57ffa3b6e9: Pull complete
Digest: sha256:15638ca254a3f8b82ecc0d51e3d86ec09b0e78dff2d75ea4bbd0c45cd7645f52
Status: Downloaded newer image for goharbor/nginx-photon:v1.9.2
Creating harbor-log ... done
Creating redis         ... done
Creating harbor-db     ... done
Creating registryctl   ... done
Creating harbor-portal ... done
Creating registry      ... done
Creating harbor-core   ... done
Creating harbor-jobservice ... done
Creating nginx             ... done

✔ ----Harbor has been installed and started successfully.----

Now you should be able to visit the admin portal at http://47.74.63.216. 
For more details, please visit https://github.com/goharbor/harbor .

以下の様なメッセージが表示されていればインストールは成功です。

✔ ----Harbor has been installed and started successfully.----

Harbor へのアクセス

http://47.74.63.216 にアクセスすると以下の様なHarborのログイン画面が表示されます。

デフォルトのユーザー名とパスワードを使用してログインできます。

ユーザー名： admin
パスワード： Harbor12345

ユーザー作成

admin 以外のアクセス用のユーザーを作成します。
Harbor へのアクセスや Docker login に使用します。

必須項目を設定し「OK」を選択します。

ユーザーが追加されます。

プロジェクトの作成

プロジェクトはリポジトリの上位グループです。Harborでは全てのリポジトリはプロジェクトに属す必要があります。
また、RBACはプロジェクトに対して適用されます。

Harborにはプロジェクトのタイプとして以下の2種類があります。

• パブリック：すべてのユーザーはパブリックプロジェクトに対する読み取り権限を持っています。
• プライベート：プライベートプロジェクトには、適切な権限を持つユーザーのみがアクセスできます。

Access Level の部分がプロジェクトのタイプを決める設定になります。
これは後からでも変えられる項目になります。

プロジェクトへのユーザーのアタッチ

作成したプロジェクトへユーザーを参加させます。

Name にユーザー名を入力します。
Role でユーザーに対するアクセス権限を設定します。

Docker Login

作成したユーザーでレジストリにアクセスをしますが、
今回はHTTPSのポートを解放していないので、このままだとエラーになります。

docker login 47.74.63.216
Username: harbor-user
Password: 

Error response from daemon: Get https://47.74.63.216/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)

HTTPでアクセスするにはクライアント側で「安全でないレジストリ」を設定する必要があります。
詳細は以下をご参照ください。
https://docs.docker.com/registry/insecure/

Macの場合

もう一度docker loginすると無事にアクセスができました。

docker login 47.74.63.216
Username: harbor-user
Password: 

Login Succeeded

Docker Image 作成

Docker Image を Push するため、テスト用にイメージを作成してみます。
※下記はサンプルなのでなんでも良いです。

mkdir test
echo 'Hello! Harbor' > test/index.html
vi test/Dockerfile

Dockerfile を編集します。

FROM ubuntu:18.04
RUN apt update \
 && apt install -y --no-install-recommends nginx \
 && apt -y clean \
 && rm -rf /var/lib/apt/lists/*
ADD index.html /usr/share/nginx/html/

ビルドしてみます。

docker build -t test:0.1 test/

タグをつけます。

docker tag test:0.1 47.74.63.216/example/test:0.1

Harbor へ Push してみます。

docker push 47.74.63.216/example/test:0.1

The push refers to repository [47.74.63.216/example/test]
2cf37d6733a5: Pushed 
92e88dc1e7ec: Pushed 
e0b3afb09dc3: Pushed 
6c01b5a53aac: Pushed 
2c6ac8e5063e: Pushed 
cc967c529ced: Pushed 
0.1: digest: sha256:86a3bbef01a4e367a10543ecc2a1ebba313c5c508cb59d20186995d22a41908d size: 1571

無事に Harbor へ Push できました。

おわりに

なかなか情報のない Harbor ですが、結構簡単に使い始めることはできるので使ってみてください。
notary, clair, chartmuseum を含んだ構成や Alibaba のコンテナレジストリと連携する方法などもご紹介して行きたいと思います。