EC2
docker

初心者インフラエンジニアがEC2 & Docker でApache 動かすまで

More than 3 years have passed since last update.

背景

ローカル環境

・Linux(Ubuntu)
・ec2はAPIアクセス可能
・ec2無料枠

やってみたこと

・docker-machine & swarm でec2上にDocker使えるようにする
・LB経由で複数のコンテナがバランシングされること

Docker-Machineでホスト作成

docker-machineインストール

# curl -L https://github.com/docker/machine/releases/download/v0.4.0/docker-machine_linux-amd64 > /usr/local/bin/docker-machine
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   400    0   400    0     0    580      0 --:--:-- --:--:-- --:--:--   580
100 12.0M  100 12.0M    0     0   356k      0  0:00:34  0:00:34 --:--:--  620k
# chmod +x /usr/local/bin/docker-machine
# docker-machine -v
docker-machine version 0.4.0 (9d0dc7a)
# 

EC2上にホスト作成

# sudo docker run swarm create
0f3ec72d0be788ec0be3834cbce7ddea
#
# docker-machine create                                               \
>   --driver amazonec2                                                \
>   --amazonec2-access-key xxxxx                                      \
>   --amazonec2-secret-key xxxxx                                      \
>   --amazonec2-vpc-id xxxxx                                          \
>   --amazonec2-region ap-northeast-1                                 \
>   --swarm --swarm-master                                            \
>   --swarm-discovery token://f3ec72d0be788ec0be3834cbce7ddea         \
>   swarm-master
Launching instance...
To see how to connect Docker to this machine, run: docker-machine env swarm-master
#
# docker-machine create                                              \
>  --driver amazonec2                                                \
>  --amazonec2-access-key  xxxxxxxxxxxxxxxxxxxx                      \
>  --amazonec2-secret-key xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx   \
>  --amazonec2-vpc-id xxxxxxxxxxxx                                   \
>  --amazonec2-region ap-northeast-1                                 \
>  --swarm                                                           \
>  --swarm-discovery token://f3ec72d0be788ec0be3834cbce7ddea         \
>  swarm-node-00
Launching instance...
To see how to connect Docker to this machine, run: docker-machine env swarm-node-00
#
# docker-machine create                                              \
>  --driver amazonec2                                                \
>  --amazonec2-access-key  xxxxxxxxxxxxxxxxxxxx                      \
>  --amazonec2-secret-key xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx   \
>  --amazonec2-vpc-id xxxxxxxxxxxx                                   \
>  --amazonec2-region ap-northeast-1                                 \
>  --swarm                                                           \
>  --swarm-discovery token://f3ec72d0be788ec0be3834cbce7ddea         \
>  swarm-node-01
Launching instance...
To see how to connect Docker to this machine, run: docker-machine env swarm-node-01
#
# eval "$(docker-machine env --swarm swarm-master)"
# docker-machine ls
NAME            ACTIVE   DRIVER      STATE     URL                        SWARM
swarm-master             amazonec2   Running   tcp://xx.xx.xx.xx:2376    swarm-master (master)
swarm-node-00            amazonec2   Running   tcp://xx.xx.xx.xx:2376   swarm-master
swarm-node-01            amazonec2   Running   tcp://xx.xx.xx.xx:2376    swarm-master
# docker ps
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS                              NAMES
# 

image作成

# docker run -it centos /bin/bash
[root@8985e66c927c /]# cat /etc/redhat-release 
CentOS Linux release 7.1.1503 (Core) 
[root@8985e66c927c /]# 
[root@8985e66c927c /]# yum install httpd php
・
・
[root@8985e66c927c /]# mkdir -p /var/www/html
[root@8985e66c927c /]# cd /var/www/html/
[root@8985e66c927c html]# vi index.php
<?php
echo gethostname();
echo "\n";
?>
[root@8985e66c927c html]# chown -R apache /var/www/
[root@8985e66c927c html]# mkdir /var/log/httpd
[root@8985e66c927c html]# chown -R apache /var/log/httpd/
[root@8985e66c927c html]# httpd
[root@8985e66c927c html]# exit
# docker commit 8985e66c927c centos:php_hostname
# docker run -d -p 80:80 centos:php_hostname apachectl -D FOREGROUND
# docker run -d -p 81:80 centos:php_hostname apachectl -D FOREGROUND

haproxyでどのホストにアクセス来てもコンテナに流す

・3台ともに設定

# docker-machine ssh swarm-master
ubuntu@swarm-master:~$ sudo su -
root@swarm-master:~# 
root@swarm-master:~# apt-get install haproxy
root@swarm-master:~# vi /etc/haproxy/haproxy.cfg
frontend  main *:8080
    default_backend             static

backend static
    balance     roundrobin
    server      web1 172.31.24.88:80 check
    server      web2 172.31.30.72:80 check
    server      web3 172.31.20.155:80 check
    server      web4 172.31.24.88:81 check
    server      web5 172.31.30.72:81 check
    server      web6 172.31.20.155:81 check
root@swarm-master:~# haproxy -f /etc/haproxy/haproxy.cfg
root@swarm-master:~# logout
ubuntu@swarm-master:~$ logout
# curl xx.xx.xx.xx:8080
66e3d4b36a16
# curl xx.xx.xx.xx:8080
f61a903cabd9
# 

おまけ docker-machineであそぶ

help

help
Commands:
  active        Print which machine is active
  config        Print the connection config for machine
  create        Create a machine
  env           Display the commands to set up the environment for the Docker client
  inspect       Inspect information about a machine
  ip            Get the IP address of a machine
  kill          Kill a machine
  ls            List machines
  regenerate-certs  Regenerate TLS Certificates for a machine
  restart       Restart a machine
  rm            Remove a machine
  ssh           Log into or run a command on a machine with SSH.
  scp           Copy files between machines
  start         Start a machine
  status        Get the status of a machine
  stop          Stop a machine
  upgrade       Upgrade a machine to the latest version of Docker
  url           Get the URL of a machine
  help, h       Shows a list of commands or help for one command

実行結果

# docker-machine ls
NAME     ACTIVE   DRIVER      STATE     URL                      SWARM
master            amazonec2   Running   tcp://xx.xx.xx.xx:2376   
# docker-machine active
master
# 
# docker-machine status master
Running
# 
# docker-machine url master
tcp://xx.xx.xx.xx:2376
# 
# docker-machine env master
export DOCKER_TLS_VERIFY="1"
export DOCKER_HOST="tcp://xx.xx.xx.xx:2376"
export DOCKER_CERT_PATH="/home/xxxx/.docker/machine/machines/master"
export DOCKER_MACHINE_NAME="master"
# Run this command to configure your shell: 
# eval "$(docker-machine env master)"
# eval "$(docker-machine env master)"
# docker-machine config master
--tlsverify --tlscacert="/home/xxxx/.docker/machine/machines/master/ca.pem" --tlscert="/home/xxxx/.docker/machine/machines/master/cert.pem" --tlskey="/home/xxxx/.docker/machine/machines/master/key.pem" -H=tcp://xx.xx.xx.xx:2376
# docker-machine ip master
xx.xx.xx.xx
# 
# docker-machine inspect master |jq .
{
  "StorePath": "/home/xxxx/.docker/machine/machines/master",
  "HostOptions": {
    "AuthOptions": {
      "ClientCertPath": "/home/xxxx/.docker/machine/certs/cert.pem",
      "PrivateKeyPath": "/home/xxxx/.docker/machine/certs/ca-key.pem",
      "StorePath": "",
      "CaCertPath": "/home/xxxx/.docker/machine/certs/ca.pem",
      "CaCertRemotePath": "",
      "ServerCertPath": "/home/xxxx/.docker/machine/machines/master/server.pem",
      "ServerKeyPath": "/home/xxxx/.docker/machine/machines/master/server-key.pem",
      "ClientKeyPath": "/home/xxxx/.docker/machine/certs/key.pem",
      "ServerCertRemotePath": "",
      "ServerKeyRemotePath": ""
    },
    "SwarmOptions": {
      "ArbitraryFlags": [],
      "TlsVerify": false,
      "TlsKey": "",
      "TlsCert": "",
      "TlsCaCert": "",
      "Overcommit": 0,
      "IsSwarm": false,
      "Address": "",
      "Discovery": "",
      "Master": false,
      "Host": "tcp://0.0.0.0:3376",
      "Image": "swarm:latest",
      "Strategy": "spread",
      "Heartbeat": 0
    },
    "EngineOptions": {
      "InstallURL": "https://get.docker.com",
      "RegistryMirror": [],
      "TlsVerify": true,
      "TlsKey": "",
      "TlsCert": "",
      "TlsCaCert": "",
      "SelinuxEnabled": false,
      "StorageDriver": "",
      "ArbitraryFlags": [],
      "Dns": null,
      "GraphDir": "",
      "Env": [],
      "Ipv6": false,
      "InsecureRegistry": [],
      "Labels": [],
      "LogLevel": ""
    },
    "Disk": 0,
    "Memory": 0,
    "Driver": ""
  },
  "DriverName": "amazonec2",
  "Driver": {
    "Monitoring": false,
    "PrivateIPOnly": false,
    "SpotPrice": "0.50",
    "RequestSpotInstance": false,
    "Zone": "a",
    "SubnetId": "xxxx",
    "VpcId": "xxxx",
    "IamInstanceProfile": "",
    "RootSize": 16,
    "ReservationId": "",
    "SecurityGroupName": "docker-machine",
    "SecurityGroupId": "xxxx",
    "PrivateIPAddress": "172.31.24.153",
    "InstanceType": "t2.micro",
    "InstanceId": "xxxx",
    "KeyName": "master",
    "SwarmHost": "tcp://0.0.0.0:3376",
    "SwarmMaster": false,
    "PrivateKeyPath": "/home/xxxx/.docker/machine/certs/ca-key.pem",
    "CaCertPath": "/home/xxxx/.docker/machine/certs/ca.pem",
    "MachineName": "master",
    "SSHPort": 22,
    "SSHUser": "ubuntu",
    "IPAddress": "xx.xx.xx.xx",
    "SwarmDiscovery": "",
    "Id": "xxxx",
    "AccessKey": "xxxx",
    "SecretKey": "xxxx",
    "SessionToken": "",
    "Region": "ap-northeast-1",
    "AMI": "ami-f4b06cf4",
    "SSHKeyID": 0
  },
  "ConfigVersion": 1
}
# 
# docker-machine restart master
Restarted machines may have new IP addresses. You may need to re-run the `docker-machine env` command.
# 
# docker-machine ssh master
Welcome to Ubuntu 14.04.2 LTS (GNU/Linux 3.13.0-62-generic x86_64)

 * Documentation:  https://help.ubuntu.com/

  System information as of Mon Aug 31 06:55:28 UTC 2015

  System load:  0.0               Processes:              99
  Usage of /:   9.1% of 15.61GB   Users logged in:        0
  Memory usage: 6%                IP address for eth0:    172.31.24.153
  Swap usage:   0%                IP address for docker0: 172.17.42.1

  Graph this data and manage this system at:
    https://landscape.canonical.com/

  Get cloud support with Ubuntu Advantage Cloud Guest:
    http://www.ubuntu.com/business/services/cloud


Last login: Mon Aug 31 06:55:28 2015 from p1806252-ipngn16201marunouchi.tokyo.ocn.ne.jp
ubuntu@master:~$ ps -ef |grep docker
root      2106     1  0 06:46 ?        00:00:00 /usr/bin/docker daemon -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --storage-driver aufs --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=amazonec2
ubuntu    2287  2270  0 06:55 pts/1    00:00:00 grep --color=auto docker
ubuntu@master:~$ exit
# docker-machine ssh master 'hostname ; id ; echo "リモートからのssh経由でコマンド実行もできru"'
master
uid=1000(ubuntu) gid=1000(ubuntu) groups=1000(ubuntu),4(adm),20(dialout),24(cdrom),25(floppy),27(sudo),29(audio),30(dip),44(video),46(plugdev),102(netdev)
リモートからのssh経由でコマンド実行もできru
# 
# docker-machine ssh master echo '"こぴーてすと" > /tmp/test'
# docker-machine scp master:/tmp/test master:/var/tmp/
# docker-machine ssh master 'cat /var/tmp/test'
こぴーてすと
# 
# docker-machine kill master
# docker-machine ls
NAME     ACTIVE   DRIVER      STATE     URL   SWARM
master            amazonec2   Stopped         
# docker-machine rm master
Successfully removed master
# ec2-describe-instances 
RESERVATION xxxx    xxxx    
INSTANCE    xxxx    xxxx            terminated  master  0       t2.micro    2015-08-31T06:43:07+0000    ap-northeast-1a             monitoring-disabled                 ebs             hvm xen     default false   
TAG instance    xxxx    Name    master
# ec2kill x-xxxx
INSTANCE    x-xxxx  terminated  terminated
# 

Upgradeだけ試せていない。。。エラーになる

# docker-machine upgrade master
SSH cmd error!
command: DEBIAN_FRONTEND=noninteractive sudo -E apt-get upgrade -y  lxc-docker
err    : exit status 100
output : Reading package lists...
Building dependency tree...
Reading state information...
Package lxc-docker is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source

E: Package 'lxc-docker' has no installation candidate

#