EC2インスタンスにconsulを導入してクラスタを作成する手順

AWS上にEC２インスタンスを２台立てて、それぞれをconsulのServer, Clientとしてクラスタを構築するまでの手順です。
EC2インスタンスに決まったタグをつけて自動でConsulクラスタにjoinできるような設定にしました。

インストール

$ wget https://releases.hashicorp.com/consul/1.0.3/consul_1.0.3_linux_amd64.zip
$ unzip consul_1.0.3_linux_amd64.zip
$ ./consul -v
Consul v1.0.3
Protocol 2 spoken by default, understands 2 to 3 (agent will automatically use protocol >2 when speaking to compatible agents)

バージョン確認できればOK

Installing Consul - Consul by HashiCorp

必要なディレクトリの作成とパーミッションの設定

$ sudo mv ./consul /usr/local/bin/ # anywhere you can detect by the PATH variable
$ sudo mkdir /etc/consul.d
$ sudo chown ec2-user:ec2-user /etc/consul.d
$ sudo mkdir /opt/consul
$ sudo chown ec2-user:ec2-user /opt/consul

• /etc/consul.d
  設定ファイルを置くディレクトリ
• /opt/consul
  consulが状態を管理したりのためにデータを置くディレクトリ

↑どちらもオプションなどで指定できるので他のディレクトリでも可

設定ファイルを書く

サーバー側

config-server.json

{
  "datacenter": "aws",
  "data_dir": "/opt/consul",
  "server": true,
  "retry_join": [
    "provider=aws tag_key=Name tag_value=consul"
  ]
}

クライアント側

config-client

{
  "datacenter": "aws",
  "data_dir": "/opt/consul",
  "retry_join": [
    "provider=aws tag_key=Name tag_value=consul"
  ]
}

Configuration - Consul by HashiCorp#amazon-ec2

EC2インスタンスにタグを設定する

設定ファイルでtag_key=Name tag_value=consulと指定したので、Nameタグにconsulという値を指定しました。

IAMロールでec2:DescribeInstancesを許可

どちらのインスタンスにも付与する

iam-role

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": "ec2:DescribeInstances",
            "Resource": "*"
        }
    ]
}

IAMロールじゃなくても

• 直接consulの起動オプションにaccess_key_idとsecret_access_keyを渡す
• 環境変数AWS_ACCESS_KEY_IDとAWS_SECRET_ACCESS_KEYを設定する
• shared credential file(~/.aws/credentials等)を準備する

でも可能

Security GroupでConsulが使うポート番号を許可

8300と8301を開けました。
下記のリンク等を参考にして追加で必要なポートがあれば許可する必要があります。

https://www.consul.io/docs/agent/options.html#ports-used
https://qiita.com/ryo0301/items/a65709c4d99408f1df0a

consul agentを起動

サーバー側

$ consul agent --config-dir=/etc/consul.d --bootstrap-expect 1
BootstrapExpect is set to 1; this is the same as Bootstrap mode.
bootstrap = true: do not enable unless necessary
==> Starting Consul agent...
==> Consul agent running!
           Version: 'v1.0.3'
           Node ID: '973f32e6-875b-542d-bf18-83e37da1ae62'
         Node name: 'ip-10-0-3-27'
        Datacenter: 'aws' (Segment: '<all>')
            Server: true (Bootstrap: true)
       Client Addr: [127.0.0.1] (HTTP: 8500, HTTPS: -1, DNS: 8600)
      Cluster Addr: 10.0.3.27 (LAN: 8301, WAN: 8302)
           Encrypt: Gossip: false, TLS-Outgoing: false, TLS-Incoming: false

==> Log data will now stream in as it occurs:

    2018/02/11 07:13:42 [INFO] raft: Initial configuration (index=1): [{Suffrage:Voter ID:973f32e6-875b-542d-bf18-83e37da1ae62 Address:10.0.3.27:8300}]
    2018/02/11 07:13:42 [INFO] serf: EventMemberJoin: ip-10-0-3-27.aws 10.0.3.27
    2018/02/11 07:13:42 [INFO] serf: EventMemberJoin: ip-10-0-3-27 10.0.3.27
    2018/02/11 07:13:42 [INFO] agent: Started DNS server 127.0.0.1:8600 (udp)
    2018/02/11 07:13:42 [INFO] raft: Node at 10.0.3.27:8300 [Follower] entering Follower state (Leader: "")
    2018/02/11 07:13:42 [WARN] serf: Failed to re-join any previously known node
    2018/02/11 07:13:42 [INFO] serf: Attempting re-join to previously known node: ip-10-0-3-19: 10.0.3.19:8301
    2018/02/11 07:13:42 [INFO] consul: Adding LAN server ip-10-0-3-27 (Addr: tcp/10.0.3.27:8300) (DC: aws)
    2018/02/11 07:13:42 [INFO] consul: Handled member-join event for server "ip-10-0-3-27.aws" in area "wan"
    2018/02/11 07:13:42 [INFO] agent: Started DNS server 127.0.0.1:8600 (tcp)
    2018/02/11 07:13:42 [INFO] agent: Started HTTP server on 127.0.0.1:8500 (tcp)
    2018/02/11 07:13:42 [INFO] agent: started state syncer
    2018/02/11 07:13:42 [INFO] agent: Retry join LAN is supported for: aliyun aws azure digitalocean gce os scaleway softlayer
    2018/02/11 07:13:42 [INFO] agent: Joining LAN cluster...
    2018/02/11 07:13:42 [INFO] discover-aws: Address type  is not supported. Valid values are {private_v4,public_v4,public_v6}. Falling back to 'private_v4'
    2018/02/11 07:13:42 [INFO] discover-aws: Region not provided. Looking up region in metadata...
    2018/02/11 07:13:42 [WARN] serf: Failed to re-join any previously known node
    2018/02/11 07:13:42 [INFO] discover-aws: Region is ap-northeast-1
    2018/02/11 07:13:42 [INFO] discover-aws: Filter instances with Name=consul
    2018/02/11 07:13:42 [INFO] discover-aws: Instance i-05cbabfc1ec12b005 has private ip 10.0.3.27
    2018/02/11 07:13:42 [INFO] discover-aws: Instance i-01503c36e371ea41a has private ip 10.0.3.19
    2018/02/11 07:13:42 [INFO] agent: Discovered LAN servers: 10.0.3.27 10.0.3.19
    2018/02/11 07:13:42 [INFO] agent: (LAN) joining: [10.0.3.27 10.0.3.19]
    2018/02/11 07:13:42 [INFO] agent: (LAN) joined: 1 Err: <nil>
    2018/02/11 07:13:42 [INFO] agent: Join LAN completed. Synced with 1 initial agents
    2018/02/11 07:13:49 [WARN] raft: Heartbeat timeout from "" reached, starting election
    2018/02/11 07:13:49 [INFO] raft: Node at 10.0.3.27:8300 [Candidate] entering Candidate state in term 3
    2018/02/11 07:13:49 [INFO] raft: Election won. Tally: 1
    2018/02/11 07:13:49 [INFO] raft: Node at 10.0.3.27:8300 [Leader] entering Leader state
    2018/02/11 07:13:49 [INFO] consul: cluster leadership acquired
    2018/02/11 07:13:49 [INFO] consul: New leader elected: ip-10-0-3-27
    2018/02/11 07:13:49 [INFO] consul: member 'ip-10-0-3-19' reaped, deregistering
    2018/02/11 07:13:49 [INFO] agent: Synced node info

クライアント側

$ consul agent --config-dir=/etc/consul.d
==> Starting Consul agent...
==> Consul agent running!
           Version: 'v1.0.3'
           Node ID: '473abde2-1a30-f946-a1b0-e73cdb67a1ba'
         Node name: 'ip-10-0-3-19'
        Datacenter: 'aws' (Segment: '')
            Server: false (Bootstrap: false)
       Client Addr: [127.0.0.1] (HTTP: 8500, HTTPS: -1, DNS: 8600)
      Cluster Addr: 10.0.3.19 (LAN: 8301, WAN: 8302)
           Encrypt: Gossip: false, TLS-Outgoing: false, TLS-Incoming: false

==> Log data will now stream in as it occurs:

    2018/02/11 07:14:15 [INFO] serf: EventMemberJoin: ip-10-0-3-19 10.0.3.19
    2018/02/11 07:14:15 [INFO] agent: Started DNS server 127.0.0.1:8600 (udp)
    2018/02/11 07:14:15 [INFO] agent: Started DNS server 127.0.0.1:8600 (tcp)
    2018/02/11 07:14:15 [INFO] agent: Started HTTP server on 127.0.0.1:8500 (tcp)
    2018/02/11 07:14:15 [INFO] agent: started state syncer
    2018/02/11 07:14:15 [INFO] agent: Retry join LAN is supported for: aliyun aws azure digitalocean gce os scaleway softlayer
    2018/02/11 07:14:15 [INFO] agent: Joining LAN cluster...
    2018/02/11 07:14:15 [INFO] discover-aws: Address type  is not supported. Valid values are {private_v4,public_v4,public_v6}. Falling back to 'private_v4'
    2018/02/11 07:14:15 [INFO] discover-aws: Region not provided. Looking up region in metadata...
    2018/02/11 07:14:15 [WARN] manager: No servers available
    2018/02/11 07:14:15 [ERR] agent: failed to sync remote state: No known Consul servers
    2018/02/11 07:14:15 [INFO] discover-aws: Region is ap-northeast-1
    2018/02/11 07:14:15 [INFO] discover-aws: Filter instances with Name=consul
    2018/02/11 07:14:15 [INFO] discover-aws: Instance i-05cbabfc1ec12b005 has private ip 10.0.3.27
    2018/02/11 07:14:15 [INFO] discover-aws: Instance i-01503c36e371ea41a has private ip 10.0.3.19
    2018/02/11 07:14:15 [INFO] agent: Discovered LAN servers: 10.0.3.27 10.0.3.19
    2018/02/11 07:14:15 [INFO] agent: (LAN) joining: [10.0.3.27 10.0.3.19]
    2018/02/11 07:14:15 [INFO] serf: EventMemberJoin: ip-10-0-3-27 10.0.3.27
    2018/02/11 07:14:15 [INFO] consul: adding server ip-10-0-3-27 (Addr: tcp/10.0.3.27:8300) (DC: aws)
    2018/02/11 07:14:15 [INFO] agent: (LAN) joined: 2 Err: <nil>
    2018/02/11 07:14:15 [INFO] agent: Join LAN completed. Synced with 2 initial agents
    2018/02/11 07:14:17 [INFO] agent: Synced node info

確認

どちらかのインスタンスで

$ consul members
Node          Address         Status  Type    Build  Protocol  DC   Segment
ip-10-0-3-27  10.0.3.27:8301  alive   server  1.0.3  2         aws  <all>
ip-10-0-3-19  10.0.3.19:8301  alive   client  1.0.3  2         aws  <default>

以上