1
0

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?

Day 15

IBM Cloud Advent Calendar 2025

@shiyasu(Shinobu Yasuda)inIBM

IBM Cloud: ZoneまたぎのHA構成環境をkeepalivedを使って自動化してみた

1
Last updated at Posted at 2025-12-25

1. はじめに

似たような構成をここでもやったことがあるが、要はkeepalivedで自動化しただけ。

image.png
image.png

2. 各サーバーでIP Spoofingを有効にする

image.png
image.png

3. Apacheの設定

Apacheインストール
[root@syasuda-server1 ~]# dnf install -y httpd
[root@syasuda-server1 ~]# echo "This is $(hostname)" >  /var/www/html/index.html
[root@syasuda-server1 ~]# systemctl enable httpd
[root@syasuda-server1 ~]# systemctl start httpd

4. keepalivedの設定

共通
[root@syasuda-server1 ~]# dnf install -y keepalived
[root@syasuda-server1 ~]# mv /etc/keepalived/keepalived.conf /etc/keepalived/keepalived.conf_old
1台目
[root@syasuda-server1 ~]# cat << 'EOF' > /etc/keepalived/keepalived.conf
global_defs {
    router_id KEEPALIVED_REPLICA
}

vrrp_instance VI_1 {
    state BACKUP
    interface eth0
    virtual_router_id 51
    priority 200
    advert_int 1
    nopreempt

    unicast_src_ip 10.50.0.20
    unicast_peer {
        10.50.4.20
    }

    virtual_ipaddress {
        192.168.200.10
        192.168.200.11
        192.168.200.12
    }

    notify "/usr/bin/bash /work/ibmcloud-scripts/changeroute-to-1.sh"
    
}
EOF
2台目
[root@syasuda-server2 ~]# cat << 'EOF' > /etc/keepalived/keepalived.conf
global_defs {
    router_id KEEPALIVED_REPLICA
}

vrrp_instance VI_1 {
    state BACKUP
    interface eth0
    virtual_router_id 51
    priority 100
    advert_int 1
    nopreempt
    
    unicast_src_ip 10.50.4.20
    unicast_peer {
        10.50.0.20
    }

    virtual_ipaddress {
        192.168.200.10
        192.168.200.11
        192.168.200.12
    }

    notify "/usr/bin/bash /work/ibmcloud-scripts/changeroute-to-2.sh"
    
}
EOF
共通
[root@syasuda-server1 ~]# systemctl enable keepalived
[root@syasuda-server1 ~]# systemctl start keepalived

5. Custom Routeの設定

image.png

  • Ingress routeとしてTransit GatewayやDirect LinkにAdvertiseするように構成してある
  • jp-tok-1正常時：jp-tok-1がpriority 2, jp-tok-2がpriority 3 => jp-tok-1が優先

と構成しておき、jp-tok-2側のkeepalivedがMASTERになった時はjp-tok-2のpriorityを1に変更することで経路を切り替えるスクリプトを実行する。

6. IBM Cloud CLIおよびスクリプトのセットアップ

共通
[root@syasuda-server1 ~]# curl -fsSL https://clis.cloud.ibm.com/install/linux | sh
[root@syasuda-server1 ~]# ibmcloud plugin install is
[root@syasuda-server1 ~]# mkdir -p /work/ibmcloud-scripts
[root@syasuda-server1 ~]# echo <API Key情報 > /work/ibmcloud-scripts/ibmcloud_apikey
1台目: このスクリプトが呼ばれた時にはjp-tok-2の優先度を3にする
[root@syasuda-server1 ~]# cat << 'EOF' > /work/ibmcloud-scripts/changeroute-to-1.sh
export LANG=C
export IBMCLOUD_HOME=/root
export LOGNAME=changeroute.log
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
{
  echo "##############################################"
  date +"%Y-%m-%d %H:%M:%S %Z (%z)"
  whoami
  env
  echo "##############################################"
  ibmcloud login -a cloud.ibm.com --apikey @${SCRIPT_DIR}/ibmcloud_apikey -r jp-tok
  ibmcloud is vpc-routing-table-route-update vpc-50-tok main-ingress-vpc-50-tok vip1-jp-tok-2 --priority 1
  ibmcloud is vpc-routing-table-route-update vpc-50-tok main-ingress-vpc-50-tok vip2-jp-tok-2 --priority 1
  ibmcloud is vpc-routing-table-route-update vpc-50-tok main-ingress-vpc-50-tok vip3-jp-tok-2 --priority 1
} >> ${SCRIPT_DIR}/${LOGNAME} 2>&1
EOF
2台目：このスクリプトが呼ばれた時にはjp-tok-2の優先度を1にする
[root@syasuda-server2 ~]# cat << 'EOF' > /work/ibmcloud-scripts/changeroute-to-2.sh
export LANG=C
export IBMCLOUD_HOME=/root
export LOGNAME=changeroute.log
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
{
  echo "##############################################"
  date +"%Y-%m-%d %H:%M:%S %Z (%z)"
  env
  echo "##############################################"
  ibmcloud login -a cloud.ibm.com --apikey @${SCRIPT_DIR}/ibmcloud_apikey -r jp-tok
  ibmcloud is vpc-routing-table-route-update vpc-50-tok main-ingress-vpc-50-tok vip1-jp-tok-2 --priority 3
  ibmcloud is vpc-routing-table-route-update vpc-50-tok main-ingress-vpc-50-tok vip2-jp-tok-2 --priority 3
  ibmcloud is vpc-routing-table-route-update vpc-50-tok main-ingress-vpc-50-tok vip3-jp-tok-2 --priority 3
} >> ${SCRIPT_DIR}/${LOGNAME} 2>&1
EOF

昔、ここでも書いたように、ibmcloud CLIはホームディレクトリ配下に.bluemixという構成ファイルを作成し、そこでセッション情報やPlugin情報を管理します。notifyで実行されるスクリプトが実行される際には、どこに構成ファイルがあるかがわからないため、ibmcloud isをインストールしているにも関わらずが見つからないというエラーになる可能性があります。それを避けるために、ここではIBMCLOUD_HOME環境変数を明示的に指定しています。

7. 動作確認

7-1. tcpdumpでUnicast方式のVRRPが実行されていることを確認

[root@syasuda-server1 ~]# tcpdump -i any proto 112 -nn
tcpdump: data link type LINUX_SLL2
dropped privs to tcpdump
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on any, link-type LINUX_SLL2 (Linux cooked v2), snapshot length 262144 bytes
09:53:15.298027 eth0  Out IP 10.50.0.20 > 10.50.4.20: VRRPv2, Advertisement, vrid 51, prio 200, authtype none, intvl 1s, length 28
09:53:16.298185 eth0  Out IP 10.50.0.20 > 10.50.4.20: VRRPv2, Advertisement, vrid 51, prio 200, authtype none, intvl 1s, length 28
09:53:17.298433 eth0  Out IP 10.50.0.20 > 10.50.4.20: VRRPv2, Advertisement, vrid 51, prio 200, authtype none, intvl 1s, length 28
09:53:18.298606 eth0  Out IP 10.50.0.20 > 10.50.4.20: VRRPv2, Advertisement, vrid 51, prio 200, authtype none, intvl 1s, length 28
09:53:19.298778 eth0  Out IP 10.50.0.20 > 10.50.4.20: VRRPv2, Advertisement, vrid 51, prio 200, authtype none, intvl 1s, length 28

[root@syasuda-server2 ~]# tcpdump -i any proto 112 -nn
tcpdump: data link type LINUX_SLL2
dropped privs to tcpdump
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on any, link-type LINUX_SLL2 (Linux cooked v2), snapshot length 262144 bytes
09:53:22.297512 eth0  In  IP 10.50.0.20 > 10.50.4.20: VRRPv2, Advertisement, (ttl 254), vrid 51, prio 200, authtype none, intvl 1s, length 28
09:53:23.297820 eth0  In  IP 10.50.0.20 > 10.50.4.20: VRRPv2, Advertisement, (ttl 254), vrid 51, prio 200, authtype none, intvl 1s, length 28
09:53:24.297843 eth0  In  IP 10.50.0.20 > 10.50.4.20: VRRPv2, Advertisement, (ttl 254), vrid 51, prio 200, authtype none, intvl 1s, length 28
09:53:25.298142 eth0  In  IP 10.50.0.20 > 10.50.4.20: VRRPv2, Advertisement, (ttl 254), vrid 51, prio 200, authtype none, intvl 1s, length 28
09:53:26.298221 eth0  In  IP 10.50.0.20 > 10.50.4.20: VRRPv2, Advertisement, (ttl 254), vrid 51, prio 200, authtype none, intvl 1s, length 28

7-2. 最初はjp-tok-1側にVIPが割り振られている。

[root@syasuda-server1 ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 02:00:5f:6d:95:c9 brd ff:ff:ff:ff:ff:ff
    altname enp0s3
    altname ens3
    inet 10.50.0.20/24 brd 10.50.0.255 scope global dynamic noprefixroute eth0
       valid_lft 328sec preferred_lft 328sec
    inet 192.168.200.10/32 scope global eth0
       valid_lft forever preferred_lft forever
    inet 192.168.200.11/32 scope global eth0
       valid_lft forever preferred_lft forever
    inet 192.168.200.12/32 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::5fff:fe6d:95c9/64 scope link noprefixroute
       valid_lft forever preferred_lft forever


[root@syasuda-server2 ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 02:00:fd:6a:8d:9d brd ff:ff:ff:ff:ff:ff
    altname enp0s3
    altname ens3
    inet 10.50.4.20/24 brd 10.50.4.255 scope global dynamic noprefixroute eth0
       valid_lft 323sec preferred_lft 323sec
    inet6 fe80::fdff:fe6a:8d9d/64 scope link noprefixroute
       valid_lft forever preferred_lft forever
HTTPアクセスでの確認(jp-tok-1側でのapacheで応答）
syasuda@MacBook-Pro ~ % curl 192.168.200.10
This is syasuda-server1

syasuda@MacBook-Pro ~ % curl 192.168.200.11
This is syasuda-server1

syasuda@MacBook-Pro ~ % curl 192.168.200.12
This is syasuda-server1

image.png

7-3. jp-tok-1側のサーバーを再起動する

VIPはjp-tok-2側に移動した
[root@syasuda-server1 ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 02:00:5f:6d:95:c9 brd ff:ff:ff:ff:ff:ff
    altname enp0s3
    altname ens3
    inet 10.50.0.20/24 brd 10.50.0.255 scope global dynamic noprefixroute eth0
       valid_lft 349sec preferred_lft 349sec
    inet6 fe80::5fff:fe6d:95c9/64 scope link noprefixroute
       valid_lft forever preferred_lft forever


[root@syasuda-server2 ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 02:00:fd:6a:8d:9d brd ff:ff:ff:ff:ff:ff
    altname enp0s3
    altname ens3
    inet 10.50.4.20/24 brd 10.50.4.255 scope global dynamic noprefixroute eth0
       valid_lft 276sec preferred_lft 276sec
    inet 192.168.200.10/32 scope global eth0
       valid_lft forever preferred_lft forever
    inet 192.168.200.11/32 scope global eth0
       valid_lft forever preferred_lft forever
    inet 192.168.200.12/32 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::fdff:fe6a:8d9d/64 scope link noprefixroute
       valid_lft forever preferred_lft forever
HTTPアクセスでの確認(jp-tok-2側でのapacheで応答）
syasuda@MacBook-Pro ~ % curl 192.168.200.10
This is syasuda-server2

syasuda@MacBook-Pro ~ % curl 192.168.200.11
This is syasuda-server2

syasuda@MacBook-Pro ~ % curl 192.168.200.12
This is syasuda-server2

image.png

7-4. jp-tok-2側のサーバーを再起動する

VIPがjp-tok-1側に移動した
[root@syasuda-server1 ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 02:00:5f:6d:95:c9 brd ff:ff:ff:ff:ff:ff
    altname enp0s3
    altname ens3
    inet 10.50.0.20/24 brd 10.50.0.255 scope global dynamic noprefixroute eth0
       valid_lft 244sec preferred_lft 244sec
    inet 192.168.200.10/32 scope global eth0
       valid_lft forever preferred_lft forever
    inet 192.168.200.11/32 scope global eth0
       valid_lft forever preferred_lft forever
    inet 192.168.200.12/32 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::5fff:fe6d:95c9/64 scope link noprefixroute
       valid_lft forever preferred_lft forever

[root@syasuda-server2 ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 02:00:fd:6a:8d:9d brd ff:ff:ff:ff:ff:ff
    altname enp0s3
    altname ens3
    inet 10.50.4.20/24 brd 10.50.4.255 scope global dynamic noprefixroute eth0
       valid_lft 358sec preferred_lft 358sec
    inet6 fe80::fdff:fe6a:8d9d/64 scope link noprefixroute
       valid_lft forever preferred_lft forever
HTTPアクセスでの確認(jp-tok-1側でのapacheで応答）
syasuda@MacBook-Pro ~ % curl 192.168.200.10
This is syasuda-server1

syasuda@MacBook-Pro ~ % curl 192.168.200.11
This is syasuda-server1

syasuda@MacBook-Pro ~ % curl 192.168.200.12
This is syasuda-server1

image.png

1
0
0

Register as a new user and use Qiita more conveniently

  1. You get articles that match your needs
  2. You can efficiently read back useful information
  3. You can use dark theme
What you can do with signing up
Sign upLogin
1
0

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?