IBMCloud vSRX(HA)障害時のダウンタイム

目的

IBMCloudのGatewayApplianceであるvSRX(HA)を利用し、障害時ダウンタイムを確認する
今回は、インタフェースの障害およびノード障害を試す

構成

• ローカル端末：macOS Catalina 10.15.6 1台
• プラットフォーム:IBMCloud
  • GatewayAppliance:Juniper vSRX Junos18.4R1-S1.3
  • 接続先サーバ:CentOS 7.7.1908(Virtual Server for Classic) 1台

状態確認

Node0->Primary
Node1->Secondary

srx00-vsrx-vSRX-Node0> show chassis cluster status
Redundancy group: 1 , Failover count: 3
node0  100      primary              yes     no       None
node1  1        secondary            yes     no       None

デフォルトWeightの確認

• デフォルトではRedundancy Group(RG)のWeightは255になっている
• デフォルトでは各インタフェースのWeightは130になっている

各インタフェースが2本ダウンすると、RG切り替りの閾値である250を超過するため、RGが切り替わる

srx00-vsrx-vSRX-Node0> show chassis cluster information
node0:
--------------------------------------------------------------------------
Redundancy Group Information:
    Redundancy Group 1 , Current State: primary, Weight: 255

node1:
--------------------------------------------------------------------------
Redundancy Group Information:
    Redundancy Group 1 , Current State: secondary, Weight: 255

{primary:node0}

srx00-vsrx-vSRX-Node0>show configuration | display set | no-more | grep weight
set chassis cluster redundancy-group 1 interface-monitor ge-0/0/3 weight 130
set chassis cluster redundancy-group 1 interface-monitor ge-0/0/4 weight 130
set chassis cluster redundancy-group 1 interface-monitor ge-7/0/3 weight 130
set chassis cluster redundancy-group 1 interface-monitor ge-7/0/4 weight 130
set chassis cluster redundancy-group 1 interface-monitor ge-0/0/1 weight 130
set chassis cluster redundancy-group 1 interface-monitor ge-0/0/2 weight 130
set chassis cluster redundancy-group 1 interface-monitor ge-0/0/5 weight 130
set chassis cluster redundancy-group 1 interface-monitor ge-0/0/6 weight 130
set chassis cluster redundancy-group 1 interface-monitor ge-0/0/7 weight 130
set chassis cluster redundancy-group 1 interface-monitor ge-0/0/8 weight 130
set chassis cluster redundancy-group 1 interface-monitor ge-7/0/1 weight 130
set chassis cluster redundancy-group 1 interface-monitor ge-7/0/2 weight 130
set chassis cluster redundancy-group 1 interface-monitor ge-7/0/5 weight 130
set chassis cluster redundancy-group 1 interface-monitor ge-7/0/6 weight 130
set chassis cluster redundancy-group 1 interface-monitor ge-7/0/7 weight 130
set chassis cluster redundancy-group 1 interface-monitor ge-7/0/8 weight 130

インタフェース障害テスト

シャットダウン

reth0のインタフェースを無効にし、RG1をNode0からNode1に移した時のダウンタイムを見る。

srx00-vsrx-vSRX-Node0> show interfaces terse
Interface               Admin Link Proto    Local                 Remote
ge-0/0/1.0              up    down aenet    --> reth0.0
ge-0/0/2.0              up    down aenet    --> reth0.0

srx00-vsrx-vSRX-Node0# set interfaces ge-0/0/1 disable
srx00-vsrx-vSRX-Node0# set interfaces ge-0/0/2 disable

srx00-vsrx-vSRX-Node0> show chassis cluster status
Redundancy group: 1 , Failover count: 6
node0  0        secondary            yes     no       None
node1  1        primary              yes     no       None

通信確認

HTTPのコンテンツGETに1秒かかったが、PINGへの影響もなく切替時のダウンタイムはほぼなし

while true; do curl -w "gettime:%{time_total} " http://10.193.88.35; done
gettime:0.023291 server 2
gettime:0.033151 server 2
gettime:0.040722 server 2
gettime:0.032113 server 2

gettime:1.035203 server 2

gettime:0.036267 server 2
gettime:0.034896 server 2
gettime:0.033021 server 2
gettime:0.025397 server 2

% ping 10.193.88.35
64 bytes from 10.193.88.xx: icmp_seq=8 ttl=54 time=12.057 ms
64 bytes from 10.193.88.xx: icmp_seq=9 ttl=54 time=8.829 ms
64 bytes from 10.193.88.xx: icmp_seq=10 ttl=54 time=8.164 ms
64 bytes from 10.193.88.xx: icmp_seq=11 ttl=54 time=8.065 ms
64 bytes from 10.193.88.xx: icmp_seq=12 ttl=54 time=8.912 ms
64 bytes from 10.193.88.xx: icmp_seq=13 ttl=54 time=7.506 ms
64 bytes from 10.193.88.xx: icmp_seq=14 ttl=54 time=15.076 ms
64 bytes from 10.193.88.xx: icmp_seq=15 ttl=54 time=9.245 ms
64 bytes from 10.193.88.xx: icmp_seq=16 ttl=54 time=9.330 ms

Node障害テスト

Node0をシャットダウンし、Node1に切り替えた時のダウンタイムを見る

srx00-vsrx-vSRX-Node0> show chassis cluster status
Redundancy group: 1 , Failover count: 1
node0  100      primary              yes     no       None
node1  1        secondary            yes     no       None

admin@kou-srx00-vsrx-vSRX-Node0> request system reboot
Reboot the system ? [yes,no] (no) yes


*** FINAL System shutdown message from admin@kou-srx00-vsrx-vSRX-Node0 ***

System going down IMMEDIATELY


Shutdown NOW!

通信確認

Pingは2発落ち、HTTPではコンテンツのGETに2秒程度なので優秀と思われる。

% while true; do curl -w "gettime:%{time_total} " http://10.193.88.35; done
gettime:0.137449 server 2
gettime:0.149001 server 2
gettime:0.325414 server 2
gettime:0.027884 server 2

gettime:2.051283 server 2

gettime:0.037877 server 2
gettime:0.032860 server 2
gettime:0.034432 server 2
gettime:0.018913 server 2

% ping 10.193.88.35
64 bytes from 10.193.88.xx: icmp_seq=481 ttl=54 time=12.412 ms
64 bytes from 10.193.88.xx: icmp_seq=482 ttl=54 time=8.621 ms
64 bytes from 10.193.88.xx: icmp_seq=483 ttl=54 time=48.676 ms
64 bytes from 10.193.88.xx: icmp_seq=484 ttl=54 time=83.593 ms

Request timeout for icmp_seq 485
Request timeout for icmp_seq 486

64 bytes from 10.193.88.xx: icmp_seq=487 ttl=54 time=16.597 ms
64 bytes from 10.193.88.xx: icmp_seq=488 ttl=54 time=10.303 ms
64 bytes from 10.193.88.xx: icmp_seq=489 ttl=54 time=8.398 ms
64 bytes from 10.193.88.xx: icmp_seq=490 ttl=54 time=7.999 ms