更新： GitLab v11.0よりGitLab ChartがBeta版になりましたので別の記事でインストール手順を更新しました： https://qiita.com/jb-vasseur/items/22518a32d28244d906bd
GitLab v10.6のリリースでCloud Native GitLab Helm ChartがついにAlpha版で公開されて、色々と検証できるようになりましたのでインストールの手順を簡単にご紹介したいと思います。
始める前に
Cloud Native GitLab Helm Chartとは
https://gitlab.com/charts/gitlab/blob/master/README.md
背景、どんな問題を解決したかったのか
GitLabプロダクトの進化により「all-in-one」のようなomnibusコンテナでは構成管理やスケーラビリティが困難になってきました。
メリット
- 各サービスのスケーリングがより容易になる
- 小さくて最適化された複数コンテナイメージ
- １サービス内に個別アップグレードやカナリアリリースが可能になる
制限（GitLab v10.7.xの時点）
現時点でGitLab EEP以上のみに対応しており、EESやCore版への対応はObject StorageのCore対応により今後される予定です。
Cloud Native GitLab Helm Chartはでalpha版で本番環境で利用できるレベルではありません。あくまで検証用に使ってください。
事前準備（Mac OS）
> kubectl version
Client Version: version.Info{Major:"1", Minor:"10", GitVersion:"v1.10.2", GitCommit:"81753b10df112992bf51bbc2c2f85208aad78335", GitTreeState:"clean", BuildDate:"2018-05-12T04:12:12Z", GoVersion:"go1.9.6", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"8", GitVersion:"v1.8.1", GitCommit:"f38e43b221d08850172a9a4ea785a86a3ffa3b3a", GitTreeState:"clean", BuildDate:"2017-10-11T23:16:41Z", GoVersion:"go1.8.3", Compiler:"gc", Platform:"linux/amd64"}
> helm version
Client: &version.Version{SemVer:"v2.9.0", GitCommit:"f6025bb9ee7daf9fee0026541c90a6f557a3e0bc", GitTreeState:"clean"}
Server: &version.Version{SemVer:"v2.8.0", GitCommit:"14af25f1de6832228539259b821949d20069a222", GitTreeState:"clean"}
> az --version
azure-cli (2.0.26)
acr (2.0.20)
acs (2.0.25)
advisor (0.1.1)
appservice (0.1.25)
backup (1.0.6)
batch (3.1.9)
batchai (0.1.5)
billing (0.1.7)
cdn (0.0.12)
cloud (2.0.12)
cognitiveservices (0.1.10)
command-modules-nspkg (2.0.1)
configure (2.0.13)
consumption (0.2.1)
container (0.1.16)
core (2.0.26)
cosmosdb (0.1.17)
dla (0.0.18)
dls (0.0.19)
eventgrid (0.1.9)
extension (0.0.8)
feedback (2.0.8)
find (0.2.8)
interactive (0.3.15)
iot (0.1.16)
keyvault (2.0.17)
lab (0.0.16)
monitor (0.1.1)
network (2.0.22)
nspkg (3.0.1)
profile (2.0.18)
rdbms (0.0.11)
redis (0.2.11)
reservations (0.1.1)
resource (2.0.22)
role (2.0.17)
servicefabric (0.0.9)
sql (2.0.20)
storage (2.0.24)
vm (2.0.25)
Python location '/usr/local/opt/python/bin/python3.6'
Extensions directory '/Users/jb/.azure/cliextensions'
Python (Darwin) 3.6.4 (default, Mar 28 2018, 12:43:57)
[GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.39.2)]
Legal docs and information: aka.ms/AzureCliLegal
AzureクラウドでKubernetesクラスタを用意する
今回はAzure ウェブコンソールで行います。
Kubernetesサービス(AKS)を作成
- サービスメニューより
Kubernetes servicesを選択し、Createを選択 - クラスタ名に情報を記入（
gitlab-kube-native） - DNS prefixに記入（
cluster） - Create a new Resource groupを選択し、新規グループ名を記入（
gitlab-kube-native-group） - Locationを選択（
East US） - Review and Createボタンを選択
構築時間は15分程度かかります。
CLIでクラスタに接続する
これからはターミナルのコマンドラインで進めていきます。
- クラスタへの接続を行う
> az aks get-credentials --resource-group gitlab-kube-native-group --name gitlab-kube-native
Merged "gitlab-kube-native" as current context in /Users/jb/.kube/config
- クラスタ接続後の稼働状況を確認する
> kubectl get nodes
NAME STATUS ROLES AGE VERSION
aks-agentpool-11243670-0 Ready agent 10m v1.9.6
aks-agentpool-11243670-1 Ready agent 10m v1.9.6
aks-agentpool-11243670-2 Ready agent 10m v1.9.6
aks-agentpool-11243670-3 Ready agent 10m v1.9.6
> kubectl get deployments --all-namespaces
NAMESPACE NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
kube-system heapster 1 1 1 1 34m
kube-system kube-dns-v20 2 2 2 2 34m
kube-system kubernetes-dashboard 1 1 1 1 34m
kube-system tunnelfront 1 1 1 1 34m
GitLabをデプロイする
Cloud Native GitLab Helm Chartのレポジトリをダウンロードする
> git clone git@gitlab.com:charts/gitlab.git
Cloning into 'gitlab'...
remote: Counting objects: 6467, done.
remote: Compressing objects: 100% (2554/2554), done.
remote: Total 6467 (delta 4322), reused 5624 (delta 3703)
Receiving objects: 100% (6467/6467), 1.07 MiB | 269.00 KiB/s, done.
Resolving deltas: 100% (4322/4322), done.
> cd gitlab
> ls
CHANGELOG.md Dockerfile bin ci requirements.yaml values.yaml
CONTRIBUTING.md LICENSE.md changelogs dependencies scripts
Chart.yaml README.md charts doc templates
RBAC admin-roleを追加する
Azure Kubernetes serviceではcluster-adminのroleは作られていないので手動で作成します。
cluster-admin-role.yamlの新規ファイルを作成し：
cluster-admin-role.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
creationTimestamp: null
name: cluster-admin
annotations:
rbac.authorization.kubernetes.io/autoupdate: "true"
rules:
- apiGroups:
- '*'
resources:
- '*'
verbs:
- '*'
- nonResourceURLs:
- '*'
verbs:
- '*'
そしてクラスタ内に作成します。
> kubectl --namespace kube-system apply -f cluster-admin-role.yaml
clusterrole.rbac.authorization.k8s.io "cluster-admin" created
RBAC configを作成する
> curl -L -w '%{http_code}' -o rbac-config.yaml -s "https://gitlab.com/charts/gitlab/raw/master/doc/helm/examples/rbac-config.yaml"
> kubectl create -f rbac-config.yaml
serviceaccount "tiller" created
clusterrolebinding.rbac.authorization.k8s.io "tiller" created
Helmの依存パッケージをインストールする
> helm dependencies update
Hang tight while we grab the latest from your chart repositories...
...Unable to get an update from the "local" chart repository (http://127.0.0.1:8879/charts):
Get http://127.0.0.1:8879/charts/index.yaml: dial tcp 127.0.0.1:8879: connect: connection refused
...Successfully got an update from the "gitlab" chart repository
...Successfully got an update from the "stable" chart repository
Update Complete. ⎈Happy Helming!⎈
Saving 3 charts
Downloading cert-manager from repo https://kubernetes-charts.storage.googleapis.com/
Downloading prometheus from repo https://kubernetes-charts.storage.googleapis.com/
Deleting outdated charts
HelmにTillerコンポーネントをインストールする
> helm init --wait --service-account tiller
$HELM_HOME has been configured at /Users/jb/.helm.
Tiller (the Helm server-side component) has been installed into your Kubernetes Cluster.
Please note: by default, Tiller is deployed with an insecure 'allow unauthenticated users' policy.
For more information on securing your installation see: https://docs.helm.sh/using_helm/#securing-your-helm-installation
Happy Helming!
Chartをデプロイする
以下のページに詳細な設定が説明されています。今回はシンプルな構成で行きますので、postgresやredisをクラスタ内のサービスをクラスタ内に構築します（デフォルト）。
https://gitlab.com/charts/gitlab/blob/master/doc/installation/deployment.md
> helm upgrade --install gitlab . \
--timeout 600 \
--set global.hosts.domain=yourdomain.io \
--set gitlab.migrations.initialRootPassword="xxx" \
--set certmanager-issuer.email=xxx@mail.com
Release "gitlab" does not exist. Installing it now.
NAME: gitlab
LAST DEPLOYED: Wed May 16 09:13:06 2018
NAMESPACE: default
STATUS: DEPLOYED
RESOURCES:
==> v1/Job
NAME DESIRED SUCCESSFUL AGE
gitlab-issuer.1 1 0 3s
gitlab-migrations.1 1 0 3s
gitlab-minio-create-buckets.1 1 0 3s
==> v1/Secret
NAME TYPE DATA AGE
gitlab-postgresql Opaque 0 4s
==> v1/ConfigMap
NAME DATA AGE
gitlab-certmanager-issuer-certmanager 2 4s
gitlab-gitaly 3 4s
gitlab-gitlab-runner 3 4s
gitlab-gitlab-shell 2 4s
gitlab-migrations 4 4s
gitlab-sidekiq-all-in-1 1 4s
gitlab-sidekiq 5 4s
gitlab-unicorn 7 4s
gitlab-minio-config-cm 3 4s
gitlab-nginx-tcp 1 4s
gitlab-nginx 7 4s
gitlab-prometheus-server 3 4s
gitlab-redis 2 4s
gitlab-registry 2 4s
==> v1/PersistentVolumeClaim
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
gitlab-minio Pending default 4s
gitlab-postgresql Pending default 4s
gitlab-prometheus-server Pending default 4s
gitlab-redis Pending default 4s
==> v1beta1/RoleBinding
NAME AGE
gitlab-gitlab-runner 3s
gitlab-nginx 3s
==> v1beta2/StatefulSet
NAME DESIRED CURRENT AGE
gitlab-gitaly 1 1 3s
==> v1beta1/CustomResourceDefinition
NAME AGE
certificates.certmanager.k8s.io 4s
clusterissuers.certmanager.k8s.io 4s
issuers.certmanager.k8s.io 4s
==> v1beta1/ClusterRole
certmanager-gitlab 4s
gitlab-nginx 4s
gitlab-prometheus-kube-state-metrics 4s
gitlab-prometheus-server 4s
==> v1beta1/ClusterRoleBinding
NAME AGE
gitlab-certmanager-issuer-admin 4s
certmanager-gitlab 4s
gitlab-nginx 4s
gitlab-prometheus-alertmanager 3s
gitlab-prometheus-kube-state-metrics 3s
gitlab-prometheus-node-exporter 3s
gitlab-prometheus-server 3s
==> v1beta1/DaemonSet
NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE
gitlab-nginx 4 4 0 4 0 <none> 3s
==> v1beta1/Ingress
NAME HOSTS ADDRESS PORTS AGE
gitlab-unicorn gitlab.yourdomain.io 80, 443 3s
gitlab-minio minio.yourdomain.io 80, 443 3s
gitlab-registry registry.yourdomain.io 80, 443 3s
==> v1/Pod(related)
NAME READY STATUS RESTARTS AGE
gitlab-nginx-5t29l 0/1 ContainerCreating 0 3s
gitlab-nginx-c7zkn 0/1 ContainerCreating 0 3s
gitlab-nginx-fkbgb 0/1 ContainerCreating 0 3s
gitlab-nginx-q77wc 0/1 ContainerCreating 0 3s
certmanager-gitlab-56f5486fdb-xzk94 0/2 ContainerCreating 0 3s
gitlab-gitlab-runner-5b64646b46-mrqh8 0/1 Init:0/1 0 3s
gitlab-gitlab-shell-7c9df464c6-bxbkt 0/1 Init:0/1 0 3s
gitlab-sidekiq-all-in-1-6584797cc-pjsdn 0/1 Init:0/2 0 3s
gitlab-unicorn-5c6d69f5c5-fp25l 0/1 Init:0/2 0 3s
gitlab-minio-5bd95c8786-nvlmp 0/1 Pending 0 3s
gitlab-minio-create-buckets.1-rgqpz 0/1 ContainerCreating 0 3s
gitlab-nginx-default-backend-566d88d447-rxfz6 0/1 ContainerCreating 0 3s
gitlab-postgresql-5b8ff4b678-c29mw 0/2 Pending 0 3s
gitlab-prometheus-server-8cf4fdd8-bj4vd 0/2 Pending 0 3s
gitlab-redis-798d568cf8-z5zvl 0/2 Pending 0 3s
gitlab-registry-866f58cd65-wqj7f 0/1 Init:0/1 0 2s
gitlab-gitaly-0 0/1 Pending 0 3s
gitlab-issuer.1-hdtpg 0/1 ContainerCreating 0 3s
gitlab-migrations.1-tlfr7 0/1 Init:0/1 0 3s
==> v1/ServiceAccount
NAME SECRETS AGE
gitlab-certmanager-issuer-admin 1 4s
certmanager-gitlab 1 4s
gitlab-gitlab-runner 1 4s
gitlab-nginx 1 4s
gitlab-prometheus-alertmanager 1 4s
gitlab-prometheus-kube-state-metrics 1 4s
gitlab-prometheus-node-exporter 1 4s
gitlab-prometheus-server 1 4s
==> v1beta1/Role
NAME AGE
gitlab-gitlab-runner 3s
gitlab-nginx 3s
==> v1/Service
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
gitlab-gitaly ClusterIP None <none> 8075/TCP,9236/TCP 3s
gitlab-gitlab-shell ClusterIP 10.0.197.156 <none> 22/TCP 3s
gitlab-unicorn ClusterIP 10.0.6.10 <none> 8080/TCP,8181/TCP 3s
gitlab-minio-svc ClusterIP 10.0.79.49 <none> 9000/TCP 3s
gitlab-nginx-default-backend ClusterIP 10.0.220.0 <none> 80/TCP 3s
gitlab-nginx LoadBalancer 10.0.244.49 <pending> 80:30227/TCP,443:31016/TCP,22:32043/TCP 3s
gitlab-postgresql ClusterIP 10.0.237.92 <none> 5432/TCP 3s
gitlab-prometheus-server ClusterIP 10.0.35.130 <none> 80/TCP 3s
gitlab-redis ClusterIP 10.0.6.80 <none> 6379/TCP,9121/TCP 3s
gitlab-registry ClusterIP 10.0.181.165 <none> 5000/TCP 3s
==> v1beta1/Deployment
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
certmanager-gitlab 1 1 1 0 3s
gitlab-gitlab-runner 1 1 1 0 3s
gitlab-gitlab-shell 1 1 1 0 3s
gitlab-sidekiq-all-in-1 1 1 1 0 3s
gitlab-unicorn 1 1 1 0 3s
gitlab-minio 1 1 1 0 3s
gitlab-nginx-default-backend 1 1 1 0 3s
gitlab-postgresql 1 1 1 0 3s
gitlab-prometheus-server 1 1 1 0 3s
gitlab-redis 1 1 1 0 3s
gitlab-registry 1 1 1 0 3s
そして、各サービスが立ち上がっていることを確認するためのコマンドは以下です。
> helm status gitlab
LAST DEPLOYED: Wed May 16 09:13:06 2018
NAMESPACE: default
STATUS: DEPLOYED
RESOURCES:
==> v1/Service
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
gitlab-gitaly ClusterIP None <none> 8075/TCP,9236/TCP 35m
gitlab-gitlab-shell ClusterIP 10.0.197.156 <none> 22/TCP 35m
gitlab-unicorn ClusterIP 10.0.6.10 <none> 8080/TCP,8181/TCP 35m
gitlab-minio-svc ClusterIP 10.0.79.49 <none> 9000/TCP 35m
gitlab-nginx-default-backend ClusterIP 10.0.220.0 <none> 80/TCP 35m
gitlab-nginx LoadBalancer 10.0.244.49 168.62.167.215 80:30227/TCP,443:31016/TCP,22:32043/TCP 35m
gitlab-postgresql ClusterIP 10.0.237.92 <none> 5432/TCP 35m
gitlab-prometheus-server ClusterIP 10.0.35.130 <none> 80/TCP 35m
gitlab-redis ClusterIP 10.0.6.80 <none> 6379/TCP,9121/TCP 35m
gitlab-registry ClusterIP 10.0.181.165 <none> 5000/TCP 35m
==> v1beta1/Deployment
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
certmanager-gitlab 1 1 1 1 35m
gitlab-gitlab-runner 1 1 1 1 35m
gitlab-gitlab-shell 1 1 1 1 35m
gitlab-sidekiq-all-in-1 1 1 1 1 35m
gitlab-unicorn 1 1 1 1 35m
gitlab-minio 1 1 1 1 35m
gitlab-nginx-default-backend 1 1 1 1 35m
gitlab-postgresql 1 1 1 1 35m
gitlab-prometheus-server 1 1 1 1 35m
gitlab-redis 1 1 1 1 35m
gitlab-registry 1 1 1 1 35m
==> v1beta2/StatefulSet
NAME DESIRED CURRENT AGE
gitlab-gitaly 1 1 35m
==> v1/Job
NAME DESIRED SUCCESSFUL AGE
gitlab-issuer.1 1 1 35m
gitlab-migrations.1 1 1 35m
gitlab-minio-create-buckets.1 1 1 35m
==> v1/PersistentVolumeClaim
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
gitlab-minio Bound pvc-ef05d77b-589d-11e8-b29c-ae5881fe4758 10Gi RWO default 35m
gitlab-postgresql Bound pvc-ef06736f-589d-11e8-b29c-ae5881fe4758 8Gi RWO default 35m
gitlab-prometheus-server Bound pvc-ef06ffbe-589d-11e8-b29c-ae5881fe4758 8Gi RWO default 35m
gitlab-redis Bound pvc-ef07d798-589d-11e8-b29c-ae5881fe4758 5Gi RWO default 35m
==> v1beta1/ClusterRole
NAME AGE
certmanager-gitlab 35m
gitlab-nginx 35m
gitlab-prometheus-kube-state-metrics 35m
gitlab-prometheus-server 35m
==> v1/Pod(related)
NAME READY STATUS RESTARTS AGE
gitlab-nginx-5t29l 1/1 Running 2 35m
gitlab-nginx-c7zkn 1/1 Running 2 35m
gitlab-nginx-fkbgb 1/1 Running 2 35m
gitlab-nginx-q77wc 1/1 Running 2 35m
certmanager-gitlab-56f5486fdb-xzk94 2/2 Running 0 35m
gitlab-gitlab-runner-5b64646b46-mrqh8 1/1 Running 11 35m
gitlab-gitlab-shell-7c9df464c6-bxbkt 1/1 Running 0 35m
gitlab-sidekiq-all-in-1-6584797cc-pjsdn 1/1 Running 0 35m
gitlab-unicorn-5c6d69f5c5-fp25l 1/1 Running 0 35m
gitlab-minio-5bd95c8786-nvlmp 1/1 Running 0 35m
gitlab-minio-create-buckets.1-rgqpz 0/1 Completed 1 35m
gitlab-nginx-default-backend-566d88d447-rxfz6 1/1 Running 0 35m
gitlab-postgresql-5b8ff4b678-c29mw 2/2 Running 0 35m
gitlab-prometheus-server-8cf4fdd8-bj4vd 2/2 Running 0 35m
gitlab-redis-798d568cf8-z5zvl 2/2 Running 0 35m
gitlab-registry-866f58cd65-wqj7f 1/1 Running 0 35m
gitlab-gitaly-0 1/1 Running 0 35m
gitlab-issuer.1-hdtpg 0/1 Completed 0 35m
gitlab-migrations.1-tlfr7 0/1 Completed 0 35m
==> v1/ConfigMap
NAME DATA AGE
gitlab-certmanager-issuer-certmanager 2 35m
gitlab-gitaly 3 35m
gitlab-gitlab-runner 3 35m
gitlab-gitlab-shell 2 35m
gitlab-migrations 4 35m
gitlab-sidekiq-all-in-1 1 35m
gitlab-sidekiq 5 35m
gitlab-unicorn 7 35m
gitlab-minio-config-cm 3 35m
gitlab-nginx-tcp 1 35m
gitlab-nginx 7 35m
gitlab-prometheus-server 3 35m
gitlab-redis 2 35m
gitlab-registry 2 35m
==> v1beta1/Ingress
NAME HOSTS ADDRESS PORTS AGE
gitlab-unicorn gitlab.yourdomain.io 168.62.167.215 80, 443 35m
gitlab-minio minio.yourdomain.io 168.62.167.215 80, 443 35m
gitlab-registry registry.yourdomain.io 168.62.167.215 80, 443 35m
==> v1beta1/RoleBinding
NAME AGE
gitlab-gitlab-runner 35m
gitlab-nginx 35m
==> v1beta1/DaemonSet
NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE
gitlab-nginx 4 4 4 4 4 <none> 35m
==> v1beta1/CustomResourceDefinition
NAME AGE
certificates.certmanager.k8s.io 35m
clusterissuers.certmanager.k8s.io 35m
issuers.certmanager.k8s.io 35m
==> v1beta1/ClusterRoleBinding
NAME AGE
gitlab-certmanager-issuer-admin 35m
certmanager-gitlab 35m
gitlab-nginx 35m
gitlab-prometheus-alertmanager 35m
gitlab-prometheus-kube-state-metrics 35m
gitlab-prometheus-node-exporter 35m
gitlab-prometheus-server 35m
==> v1beta1/Role
NAME AGE
gitlab-gitlab-runner 35m
gitlab-nginx 35m
==> v1/Secret
NAME TYPE DATA AGE
gitlab-postgresql Opaque 0 35m
==> v1/ServiceAccount
NAME SECRETS AGE
gitlab-certmanager-issuer-admin 1 35m
certmanager-gitlab 1 35m
gitlab-gitlab-runner 1 35m
gitlab-nginx 1 35m
gitlab-prometheus-alertmanager 1 35m
gitlab-prometheus-kube-state-metrics 1 35m
gitlab-prometheus-node-exporter 1 35m
gitlab-prometheus-server 1 35m
公開IPアドレスを自分のドメインのDNSレコードに登録することを忘れずに！！
しばらくして、全てのPodがRunningとなり、GitLabへログインできるようになります。
ここまでにたどり着いたときは大分感動しました。RBAC周りで多少苦戦しましたが、現在推奨されているGitLab Omnibus Chartより更に簡単にデプロイできるようになった印象です（例えば、HTTPS化は勝手にやってくれます）。
今後は以下をもう少し探ってみたいと思います。
- Azure Consoleなしで上記の作業を自動化
- GitLabの細かい設定と連携サービスの有効化
- Backup/Restore/Migrate
- Version Upgrade