事象
CICDについての理解を深めたく、久しぶりにインフラCI 実践ガイドを読み進めようと思いましたところ、なぜか環境が動きませんでした。
具体的には以前は問題なく動いておりましたCIパイプラインが、途中でエラー終了するようになってしまわれました。
多分何も悪いことはしていないはずなのに・・・！
エラーはKetchup-vagrant-ansibleプロジェクトのパイプラインで、Unit_Packageのところで異常終了しておりました。
生ログ
Running with gitlab-runner 10.8.0 (079aad9e)
on gitlab-runner dd1cfd4c
Using Docker executor with image docker:latest ...
Pulling docker image docker:latest ...
Using docker image sha256:37123e40ec5295ac68158ace2d5c302a8a5d33f6e40b9a3397be063f941d3705 for docker:latest ...
Running on runner-dd1cfd4c-project-2-concurrent-0 via gitlab-runner...
Fetching changes...
Removing lint-rules/
HEAD is now at 2e7a19e modify gitlab-ci
Checking out 2e7a19ea as master...
Skipping Git submodules setup
$ docker login -u gitlab-ci-token -p ${CI_BUILD_TOKEN} ${CI_REGISTRY}
WARNING! Using --password via the CLI is insecure. Use --password-stdin.
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credential-stores
Login Succeeded
$ docker build . -t ${CONTAINER_IMAGE_PATH}
#0 building with "default" instance using docker driver
#1 [internal] load .dockerignore
#1 transferring context: 2B done
#1 DONE 0.1s
#2 [internal] load build definition from Dockerfile
#2 transferring dockerfile: 1.12kB done
#2 DONE 0.1s
#3 [internal] load metadata for docker.io/library/centos:7
#3 DONE 0.0s
#4 [1/3] FROM docker.io/library/centos:7
#4 DONE 0.0s
#5 [internal] load build context
#5 transferring context: 640.62kB 0.0s done
#5 DONE 0.0s
#4 [1/3] FROM docker.io/library/centos:7
#4 CACHED
#6 [2/3] COPY ./ ./
#6 DONE 0.1s
#7 [3/3] RUN (cd /lib/systemd/system/sysinit.target.wants/; for i in *; do [ $i == systemd-tmpfiles-setup.service ] || rm -f $i; done); rm -f /lib/systemd/system/multi-user.target.wants/*; rm -f /etc/systemd/system/*.wants/*; rm -f /lib/systemd/system/local-fs.target.wants/*; rm -f /lib/systemd/system/sockets.target.wants/*udev*; rm -f /lib/systemd/system/sockets.target.wants/*initctl*; rm -f /lib/systemd/system/basic.target.wants/*; rm -f /lib/systemd/system/anaconda.target.wants/*; yum install -y epel-release && yum install -y git && yum install -y https://github.com/infra-ci-book/support/raw/master/obsoleted/ansible-2.4.2.0-2.el7.noarch.rpm && yum install -y https://github.com/infra-ci-book/support/raw/master/obsoleted/ansible-lint-3.4.21-1.el7.centos.noarch.rpm && yum clean all
#7 0.754 Loaded plugins: fastestmirror, ovl
#7 1.212 Determining fastest mirrors
#7 1.467 Could not retrieve mirrorlist http://mirrorlist.centos.org/?release=7&arch=x86_64&repo=os&infra=container error was
#7 1.467 14: curl#6 - "Could not resolve host: mirrorlist.centos.org; Unknown error"
#7 1.472
#7 1.472
#7 1.472 One of the configured repositories failed (Unknown),
#7 1.472 and yum doesn't have enough cached data to continue. At this point the only
#7 1.472 safe thing yum can do is fail. There are a few ways to work "fix" this:
#7 1.472
#7 1.472 1. Contact the upstream for the repository and get them to fix the problem.
#7 1.472
#7 1.472 2. Reconfigure the baseurl/etc. for the repository, to point to a working
#7 1.472 upstream. This is most often useful if you are using a newer
#7 1.472 distribution release than is supported by the repository (and the
#7 1.472 packages for the previous distribution release still work).
#7 1.472
#7 1.472 3. Run the command with the repository temporarily disabled
#7 1.472 yum --disablerepo=<repoid> ...
#7 1.472
#7 1.472 4. Disable the repository permanently, so yum won't use it by default. Yum
#7 1.472 will then just ignore the repository until you permanently enable it
#7 1.472 again or use --enablerepo for temporary usage:
#7 1.472
#7 1.472 yum-config-manager --disable <repoid>
#7 1.472 or
#7 1.472 subscription-manager repos --disable=<repoid>
#7 1.472
#7 1.472 5. Configure the failing repository to be skipped, if it is unavailable.
#7 1.472 Note that yum will try to contact the repo. when it runs most commands,
#7 1.472 so will have to try and fail each time (and thus. yum will be be much
#7 1.472 slower). If it is a very temporary problem though, this is often a nice
#7 1.472 compromise:
#7 1.472
#7 1.472 yum-config-manager --save --setopt=<repoid>.skip_if_unavailable=true
#7 1.472
#7 1.472 Cannot find a valid baseurl for repo: base/7/x86_64
#7 ERROR: process "/bin/sh -c (cd /lib/systemd/system/sysinit.target.wants/; for i in *; do [ $i == systemd-tmpfiles-setup.service ] || rm -f $i; done); rm -f /lib/systemd/system/multi-user.target.wants/*; rm -f /etc/systemd/system/*.wants/*; rm -f /lib/systemd/system/local-fs.target.wants/*; rm -f /lib/systemd/system/sockets.target.wants/*udev*; rm -f /lib/systemd/system/sockets.target.wants/*initctl*; rm -f /lib/systemd/system/basic.target.wants/*; rm -f /lib/systemd/system/anaconda.target.wants/*; yum install -y epel-release && yum install -y git && yum install -y ${ANSIBLE_RPM:?} && yum install -y ${ANSIBLE_LINT_RPM:?} && yum clean all" did not complete successfully: exit code: 1
------
> [3/3] RUN (cd /lib/systemd/system/sysinit.target.wants/; for i in *; do [ $i == systemd-tmpfiles-setup.service ] || rm -f $i; done); rm -f /lib/systemd/system/multi-user.target.wants/*; rm -f /etc/systemd/system/*.wants/*; rm -f /lib/systemd/system/local-fs.target.wants/*; rm -f /lib/systemd/system/sockets.target.wants/*udev*; rm -f /lib/systemd/system/sockets.target.wants/*initctl*; rm -f /lib/systemd/system/basic.target.wants/*; rm -f /lib/systemd/system/anaconda.target.wants/*; yum install -y epel-release && yum install -y git && yum install -y https://github.com/infra-ci-book/support/raw/master/obsoleted/ansible-2.4.2.0-2.el7.noarch.rpm && yum install -y https://github.com/infra-ci-book/support/raw/master/obsoleted/ansible-lint-3.4.21-1.el7.centos.noarch.rpm && yum clean all:
1.472
1.472 5. Configure the failing repository to be skipped, if it is unavailable.
1.472 Note that yum will try to contact the repo. when it runs most commands,
1.472 so will have to try and fail each time (and thus. yum will be be much
1.472 slower). If it is a very temporary problem though, this is often a nice
1.472 compromise:
1.472
1.472 yum-config-manager --save --setopt=<repoid>.skip_if_unavailable=true
1.472
1.472 Cannot find a valid baseurl for repo: base/7/x86_64
------
Dockerfile:11
--------------------
10 |
11 | >>> RUN (cd /lib/systemd/system/sysinit.target.wants/; for i in *; do [ $i == \
12 | >>> systemd-tmpfiles-setup.service ] || rm -f $i; done); \
13 | >>> rm -f /lib/systemd/system/multi-user.target.wants/*;\
14 | >>> rm -f /etc/systemd/system/*.wants/*;\
15 | >>> rm -f /lib/systemd/system/local-fs.target.wants/*; \
16 | >>> rm -f /lib/systemd/system/sockets.target.wants/*udev*; \
17 | >>> rm -f /lib/systemd/system/sockets.target.wants/*initctl*; \
18 | >>> rm -f /lib/systemd/system/basic.target.wants/*;\
19 | >>> rm -f /lib/systemd/system/anaconda.target.wants/*;\
20 | >>> yum install -y epel-release && \
21 | >>> yum install -y git && \
22 | >>> yum install -y ${ANSIBLE_RPM:?} && \
23 | >>> yum install -y ${ANSIBLE_LINT_RPM:?} && \
24 | >>> yum clean all
25 |
--------------------
ERROR: failed to solve: process "/bin/sh -c (cd /lib/systemd/system/sysinit.target.wants/; for i in *; do [ $i == systemd-tmpfiles-setup.service ] || rm -f $i; done); rm -f /lib/systemd/system/multi-user.target.wants/*; rm -f /etc/systemd/system/*.wants/*; rm -f /lib/systemd/system/local-fs.target.wants/*; rm -f /lib/systemd/system/sockets.target.wants/*udev*; rm -f /lib/systemd/system/sockets.target.wants/*initctl*; rm -f /lib/systemd/system/basic.target.wants/*; rm -f /lib/systemd/system/anaconda.target.wants/*; yum install -y epel-release && yum install -y git && yum install -y ${ANSIBLE_RPM:?} && yum install -y ${ANSIBLE_LINT_RPM:?} && yum clean all" did not complete successfully: exit code: 1
ERROR: Job failed: exit code 1
こちらのログで気になったのは以下の文章です。
Could not retrieve mirrorlist http://mirrorlist.centos.org/?release=7&arch=x86_64&repo=os&infra=container error was
これはCent OSのミラーサイトが接続できなくて失敗していることを示唆します。
この文言で検索してみますと、Cent OSのサービス終了に伴いこちらのレポジトリが停止してしまわれたとのことです。
これによりCent OSでyum installなどのコマンドが使えず、結果として私のGitLab Runnerも環境構築ができずに失敗してしまったように見えます。
対策は以下のサイト様に記載されております通り、レポジトリ設定を修正すれば良いそうです。
なのですが、これにはRunnerが環境構築に利用しているDocker Imageの中身を修正する必要がありそうです。何とかRunnerが利用しているDocker Imageの場所は突き止めたのですが、RunnerのサーバーでDocker execコマンドを実行してもうまく動かず、止まってしまっております。
また何か新しいことが分かりましたら更新してまいります。
それでは皆様おやすみなさいませ。