Dockerのマルチステージビルドについて。

これまでコンテナ環境で動かすアプリケーション自体のビルド(warファイル作成)はホストOSとかのコンテナ外で行っていた。
というのも、やろうと思えばカスタムイメージのビルド内でアプリケーションのビルド(Dockerfileでwarをコピーする代わりに、内部でビルドする)も当然可能だが、アプリケーションの実行環境(ランタイム環境)にビルドだけに必要な(Mavenなどの)パッケージを入れるのはあまり良い構成ではない。

そこで、「アプリケーションのビルド(warファイル作成)だけを行うイメージ」と「作成されたwarファイルをtomcatイメージに取り込んだイメージ(既存処理)」の二つを順番に行うマルチステージビルドというDockerの機能を使うことで、ビルド環境とランタイム環境を分離しつつ、ぜんぶコンテナ内で完結する構成とすることができる。

todo: 登場人物の図を描こう…

アプリケーションをMaven Buildするイメージ作成の準備

まずはDockerfileを作るための手順を確認する。

今までUbuntuでやっていたのでここでもUbuntuで(なぜかDebianだとうまくいかなかった)

コンテナ起動

root@chaource:~# docker run -it ubuntu bash
root@d8b1e0c1361f:/# 
root@d8b1e0c1361f:/# ls
bin  boot  dev  etc  home  lib  lib64  media  mnt  opt  proc  root  run  sbin  srv  sys  tmp  usr  var
root@d8b1e0c1361f:/#

`Maven`と`Git`のインストール

root@d8b1e0c1361f:/# cat /etc/apt/sources.list   
# See http://help.ubuntu.com/community/UpgradeNotes for how to upgrade to
# newer versions of the distribution.
deb http://archive.ubuntu.com/ubuntu/ bionic main restricted
# deb-src http://archive.ubuntu.com/ubuntu/ bionic main restricted

## Major bug fix updates produced after the final release of the
## distribution.
deb http://archive.ubuntu.com/ubuntu/ bionic-updates main restricted
# deb-src http://archive.ubuntu.com/ubuntu/ bionic-updates main restricted

## N.B. software from this repository is ENTIRELY UNSUPPORTED by the Ubuntu
## team. Also, please note that software in universe WILL NOT receive any
## review or updates from the Ubuntu security team.
deb http://archive.ubuntu.com/ubuntu/ bionic universe
# deb-src http://archive.ubuntu.com/ubuntu/ bionic universe
deb http://archive.ubuntu.com/ubuntu/ bionic-updates universe
# deb-src http://archive.ubuntu.com/ubuntu/ bionic-updates universe

## N.B. software from this repository is ENTIRELY UNSUPPORTED by the Ubuntu
## team, and may not be under a free licence. Please satisfy yourself as to
## your rights to use the software. Also, please note that software in
## multiverse WILL NOT receive any review or updates from the Ubuntu
## security team.
deb http://archive.ubuntu.com/ubuntu/ bionic multiverse
# deb-src http://archive.ubuntu.com/ubuntu/ bionic multiverse
deb http://archive.ubuntu.com/ubuntu/ bionic-updates multiverse
# deb-src http://archive.ubuntu.com/ubuntu/ bionic-updates multiverse

## N.B. software from this repository may not have been tested as
## extensively as that contained in the main release, although it includes
## newer versions of some applications which may provide useful features.
## Also, please note that software in backports WILL NOT receive any review
## or updates from the Ubuntu security team.
deb http://archive.ubuntu.com/ubuntu/ bionic-backports main restricted universe multiverse
# deb-src http://archive.ubuntu.com/ubuntu/ bionic-backports main restricted universe multiverse

## Uncomment the following two lines to add software from Canonical's
## 'partner' repository.
## This software is not part of Ubuntu, but is offered by Canonical and the
## respective vendors as a service to Ubuntu users.
# deb http://archive.canonical.com/ubuntu bionic partner
# deb-src http://archive.canonical.com/ubuntu bionic partner

deb http://security.ubuntu.com/ubuntu/ bionic-security main restricted
# deb-src http://security.ubuntu.com/ubuntu/ bionic-security main restricted
deb http://security.ubuntu.com/ubuntu/ bionic-security universe
# deb-src http://security.ubuntu.com/ubuntu/ bionic-security universe
deb http://security.ubuntu.com/ubuntu/ bionic-security multiverse
# deb-src http://security.ubuntu.com/ubuntu/ bionic-security multiverse
root@d8b1e0c1361f:/#

universeは入ってるけど有効になってない模様

root@d8b1e0c1361f:/# grep universe /etc/apt/sources.list
## team. Also, please note that software in universe WILL NOT receive any
deb http://archive.ubuntu.com/ubuntu/ bionic universe
# deb-src http://archive.ubuntu.com/ubuntu/ bionic universe
deb http://archive.ubuntu.com/ubuntu/ bionic-updates universe
# deb-src http://archive.ubuntu.com/ubuntu/ bionic-updates universe
deb http://archive.ubuntu.com/ubuntu/ bionic-backports main restricted universe multiverse
# deb-src http://archive.ubuntu.com/ubuntu/ bionic-backports main restricted universe multiverse
deb http://security.ubuntu.com/ubuntu/ bionic-security universe
# deb-src http://security.ubuntu.com/ubuntu/ bionic-security universe
root@d8b1e0c1361f:/# apt install git maven
Reading package lists... Done
Building dependency tree       
Reading state information... Done
E: Unable to locate package git
E: Unable to locate package maven
root@d8b1e0c1361f:/#

root@d8b1e0c1361f:/# apt update
Get:1 http://archive.ubuntu.com/ubuntu bionic InRelease [242 kB]
Get:2 http://security.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB]
Get:3 http://security.ubuntu.com/ubuntu bionic-security/main amd64 Packages [339 kB]
Get:4 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB]
Get:5 http://archive.ubuntu.com/ubuntu bionic-backports InRelease [74.6 kB]
Get:6 http://archive.ubuntu.com/ubuntu bionic/universe amd64 Packages [11.3 MB]
:
:
Fetched 15.5 MB in 1min 2s (248 kB/s)                                                                                                                                                                                            
Reading package lists... Done
Building dependency tree       
Reading state information... Done
9 packages can be upgraded. Run 'apt list --upgradable' to see them.
root@d8b1e0c1361f:/#

root@d8b1e0c1361f:/# apt install git maven
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following additional packages will be installed:
:
:

root@d8b1e0c1361f:/# which git
/usr/bin/git
root@d8b1e0c1361f:/# which mvn
/usr/bin/mvn
root@d8b1e0c1361f:/#

git cloneする

root@d8b1e0c1361f:/# cd
root@d8b1e0c1361f:~# ls
root@d8b1e0c1361f:~# mkdir src
root@d8b1e0c1361f:~# cd src/
root@d8b1e0c1361f:~/src# git clone https://github.com/zaki-lknr/javaee-memoapp2.git -b v1.0.0
Cloning into 'javaee-memoapp2'...
remote: Enumerating objects: 384, done.
remote: Counting objects: 100% (384/384), done.
remote: Compressing objects: 100% (158/158), done.
remote: Total 384 (delta 144), reused 355 (delta 115), pack-reused 0
Receiving objects: 100% (384/384), 39.46 KiB | 1.64 MiB/s, done.
Resolving deltas: 100% (144/144), done.
Note: checking out '3656f7114c526027c96c0a585b6a5152ce43947c'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

  git checkout -b <new-branch-name>

root@d8b1e0c1361f:~/src#

Mavenビルド

root@d8b1e0c1361f:~/src# cd javaee-memoapp2/
root@d8b1e0c1361f:~/src/javaee-memoapp2# 
root@d8b1e0c1361f:~/src/javaee-memoapp2# 
root@d8b1e0c1361f:~/src/javaee-memoapp2# 
root@d8b1e0c1361f:~/src/javaee-memoapp2# mvn package
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.google.inject.internal.cglib.core.$ReflectUtils$1 (file:/usr/share/maven/lib/guice.jar) to method java.lang.ClassLoader.defineClass(java.lang.String,byte[],int,int,java.security.ProtectionDomain)
WARNING: Please consider reporting this to the maintainers of com.google.inject.internal.cglib.core.$ReflectUtils$1
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
[INFO] Scanning for projects...
[INFO] 
[INFO] ------------------------------------------------------------------------
[INFO] Building memoapp2 Maven Webapp 0.0.1-SNAPSHOT
[INFO] ------------------------------------------------------------------------
Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-resources-plugin/2.6/maven-resources-plugin-2.6.pom
:
:
Downloaded from central: https://repo.maven.apache.org/maven2/xpp3/xpp3_min/1.1.4c/xpp3_min-1.1.4c.jar (25 kB at 46 kB/s)
Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-filtering/1.0-beta-2/maven-filtering-1.0-beta-2.jar (33 kB at 46 kB/s)
Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.0/plexus-utils-3.0.jar (226 kB at 309 kB/s)
[INFO] Packaging webapp
[INFO] Assembling webapp [memoapp2] in [/root/src/javaee-memoapp2/target/memoapp2]
[INFO] Processing war project
[INFO] Copying webapp resources [/root/src/javaee-memoapp2/src/main/webapp]
[INFO] Webapp assembled in [127 msecs]
[INFO] Building war: /root/src/javaee-memoapp2/target/memoapp2.war
[INFO] WEB-INF/web.xml already added, skipping
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 01:07 min
[INFO] Finished at: 2019-02-11T01:45:09Z
[INFO] Final Memory: 19M/46M
[INFO] ------------------------------------------------------------------------
root@d8b1e0c1361f:~/src/javaee-memoapp2# 
root@d8b1e0c1361f:~/src/javaee-memoapp2# 
root@d8b1e0c1361f:~/src/javaee-memoapp2# ll target/memoapp2.war 
-rw-r--r-- 1 root root 3905409 Feb 11 01:45 target/memoapp2.war
root@d8b1e0c1361f:~/src/javaee-memoapp2# ll /root/src/javaee-memoapp2/target/memoapp2.war 
-rw-r--r-- 1 root root 3905409 Feb 11 01:45 /root/src/javaee-memoapp2/target/memoapp2.war
root@d8b1e0c1361f:~/src/javaee-memoapp2#

無事に完了。

まとめ

必要な手順は以下の通り

apt update
apt install git maven
git clone
mvn package

Dockerfileの作成

ubuntuベースの場合

というわけでマルチステージビルド本番。
やることは、前段で確認した「アプリケーションのビルドそのもの(warファイル作成)の実行」と「作成されたwarファイルをtomcatイメージに取り込み(既存の処理)」の2点

※ before

Dockerfile

FROM ubuntu AS memoapp-build

RUN apt update && apt install -y git maven
WORKDIR /usr/local/src
RUN git clone https://github.com/zaki-lknr/javaee-memoapp2.git \
    && cd javaee-memoapp2 \
    && mvn package


FROM tomcat:8.5

COPY --from=memoapp-build /usr/local/src/javaee-memoapp2/target/memoapp2.war /usr/local/tomcat/webapps/

一つ目のFROM ubuntu AS memoapp-buildは、memoapp-buildという名前でアプリケーションをビルドしwarファイルを作成する処理までを行うイメージを作成する。
二つ目のFROM tomcat:8.5で、memoapp-buildで作成したwarファイルを(COPYの--fromオプションで指定して)取り出し、サービス実行用のイメージに取り込んでいる。

root@chaource:~# docker images
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
debian              latest              d508d16c64cd        4 days ago          101MB
memoapp             latest              2cd39d9db4c2        5 days ago          466MB
tomcat              8.5                 7ee26c09afb3        2 weeks ago         462MB
mysql               5.7                 141eda20897f        2 weeks ago         372MB
ubuntu              latest              20bb25d32758        2 weeks ago         87.5MB

memoapp:latestがすでにあるので

root@chaource:~# docker build -t memoapp-msb .
Sending build context to Docker daemon  33.24MB
Step 1/5 : FROM ubuntu AS mvn-build
 ---> 20bb25d32758
Step 2/5 : RUN apt update && apt install -y git maven
 ---> Running in ec5267d45093

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

Get:1 http://security.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB]
Get:2 http://security.ubuntu.com/ubuntu bionic-security/multiverse amd64 Packages [3451 B]
Get:3 http://archive.ubuntu.com/ubuntu bionic InRelease [242 kB]
Get:4 http://security.ubuntu.com/ubuntu bionic-security/universe amd64 Packages [147 kB]
Get:5 http://security.ubuntu.com/ubuntu bionic-security/main amd64 Packages [339 kB]
Get:6 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB]
Get:7 http://archive.ubuntu.com/ubuntu bionic-backports InRelease [74.6 kB]
Get:8 http://archive.ubuntu.com/ubuntu bionic/main amd64 Packages [1344 kB]
Get:9 http://archive.ubuntu.com/ubuntu bionic/restricted amd64 Packages [13.5 kB]
Get:10 http://archive.ubuntu.com/ubuntu bionic/universe amd64 Packages [11.3 MB]
:
:
[INFO] Packaging webapp
[INFO] Assembling webapp [memoapp2] in [/usr/local/src/javaee-memoapp2/target/memoapp2]
[INFO] Processing war project
[INFO] Copying webapp resources [/usr/local/src/javaee-memoapp2/src/main/webapp]
[INFO] Webapp assembled in [151 msecs]
[INFO] Building war: /usr/local/src/javaee-memoapp2/target/memoapp2.war
[INFO] WEB-INF/web.xml already added, skipping
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 01:14 min
[INFO] Finished at: 2019-02-11T08:30:19Z
[INFO] Final Memory: 19M/46M
[INFO] ------------------------------------------------------------------------
Removing intermediate container 2e53cba3f4b3
 ---> 853035d3ed5d
Step 5/6 : FROM tomcat:8.5
 ---> 7ee26c09afb3
Step 6/6 : COPY --from=memoapp-build /usr/local/src/javaee-memoapp2/target/memoapp2.war /usr/local/tomcat/webapps/
 ---> 4fd6539d4164
Successfully built 4fd6539d4164
Successfully tagged memoapp-msb:latest

できました。

ただ結構時間かかる。

mavenベースの場合

ubuntuコンテナ内でgit & mavenをapt update && apt installするのに結構な時間かかっているので、MavenのOfficialイメージを使ってみよう。
これならmavenは当然最初から入っているし、gitも内蔵しているので、aptを使う必要がなく、大幅に(私の環境では5,6分)時間短縮できる。

できたDockerfileがこちら

Dockerfile

FROM maven:3.6 AS memoapp-build

WORKDIR /usr/local/src
RUN git clone https://github.com/zaki-lknr/javaee-memoapp2.git \
    && cd javaee-memoapp2 \
    && mvn package


FROM tomcat:8.5

COPY --from=memoapp-build /usr/local/src/javaee-memoapp2/target/memoapp2.war /usr/local/tomcat/webapps/

Docker Composeと連携

連携といっても、参照するDockerfileをマルチステージビルドしている内容にするだけ。

APイメージのビルドも含めてYAMLで定義では、

あらかじめwarファイルを作成
↑のwarファイルを使用するカスタムイメージをビルド
↑のカスタムイメージとMySQLイメージをコンテナ実行

という手順になっていたが、本記事のマルチステージビルドのDockerfileを使うことで

warファイルを作成するコンテナ実行
↑のwarファイルを使用するカスタムイメージをビルド
↑のカスタムイメージとMySQLイメージをコンテナ実行

という手順になり、ホストOS上にビルド環境を用意しなくても実行環境を用意できるようになった。

Mavenみたいな便利なビルドツールがあると、コンテナ化のメリットもあまり感じられないかもしれないけど、ビルドするのに依存する複数のライブラリを手動で入れる必要があったり、特定のプラットフォームでのビルドだと動作に不安があったり、人によってコンパイラのバージョンが異なっていたり、そもそもJavaみたいなクロスプラットフォームなでなく、「ソースを書くのはWindowsだけどビルドはLinux」みたいな場合でも、常に同一の環境でビルドできる、というメリットがある。

ここまでやっておいてなんだけど、OpenShiftで使えるS2Iってこのあたりの機能を含んでいる感じだなー