#■目的
Interop Tokyo 2019にいってきました。
NECさんのUNIVERGE QX-W2120がShowNetの構成要素に組まれており、安心の冗長構成がされていました。
ということで、UNIVERGE IXルーターとOCIをBGPでIPsec接続して冗長化した２つのIPSec TunnelをAS-Path PrependでTunnel１を優先するよう経路制御を行い、可用性検証してみます。
BGPの設定と確認方法はIXプロフェッショナル @hongmnh さんに協力いただきました。
#■ X2015 構成
#■手順
主な手順は、こちらを参考としてStatic Routesの設定部分をBGP設定に変更して構成します。
Oracle Cloud:NEC UNIVERGE IXルーターでOCIへIPsec VPN接続してみてみた
##●Oracle Cloud IPSec作成
OCI Webコンソールから、[Network] > [IPSec Connection]から[Create IPsec COnnect]をクリックし、以下の様に設定
・IPSec情報
・IPSec Tunnel01情報
・IPSec Tunnel02情報
設定完了したら、[Create IPSec COnnect]をクリックし作成
作成すると、「Oracle VPN IP Address」に各TunnnelにPublic IPが付与されるので、このIPをルーター設定時に使用します。
##●BGP設定
!
ip prefix-list as-path-in 10 permit any
ip prefix-list as-path-out 10 permit any
!
route-map bgp permit 65000
!
!AS-Path制御設定
route-map rmap-in permit 10
match ip address prefix-list as-path-in
set as-path prepend 31898 31898
!
route-map rmap-out permit 10
match ip address prefix-list as-path-out
set as-path prepend 65000 65000
!
! Keepaliveメッセージを10秒、holdtimeを30秒として設定
router bgp 65000
neighbor 10.254.200.202 remote-as 31898
neighbor 10.254.200.202 timers 10 30
neighbor 10.254.200.206 remote-as 31898
neighbor 10.254.200.206 timers 10 30
address-family ipv4 unicast
neighbor 10.254.200.206 route-map rmap-in in
neighbor 10.254.200.206 route-map rmap-out out
network 192.168.100.0/24
!
!Tunnel interface にBGP用IPを設定
interface Tunnel0.0
tunnel mode ipsec
ip address 10.254.200.201/30
ip tcp adjust-mss auto
ipsec policy tunnel ipsec-map1 out
no shutdown
!
interface Tunnel1.0
tunnel mode ipsec
ip address 10.254.200.205/30
ip tcp adjust-mss auto
ipsec policy tunnel ipsec-map2 out
no shutdown
##●config確認
Router(config)# show config
Using 3714 out of 524288 bytes
! NEC Portable Internetwork Core Operating System Software
! IX Series IX2105 (magellan-sec) Software, Version 9.6.12A, MAINTENANCE RELEASE SOFTWARE
! Compiled Oct 05-Thu-2017 19:27:09 JST #2
! Last updated Jun 14-Fri-2019 00:58:40 JST
!
!% You must restart the router for this configuration to take effect.
!
timezone +09 00
!
username user password hash 123ABAD44 administrator
!
!
ip ufs-cache enable
ip multipath per-flow
ip route default GigaEthernet0.1
ip prefix-list as-path-in 10 permit any
ip prefix-list as-path-out 10 permit any
ip access-list sec-list permit ip src any dest any
ip access-list web-http-acl permit ip src any dest 192.168.100.240/32
ip access-list web_console permit ip src any dest 192.168.100.240/32
!
!
!
ike proposal ike-prop encryption aes-256 hash sha2-256 group 1536-bit
!
ike policy ike-pol1 peer 100.100.100.101 key IPSecIX200Password ike-prop
!
ike policy ike-pol2 peer 100.100.100.102 key IPSecIX200Password ike-prop
!
ipsec autokey-proposal ipsec-prop esp-aes-256 esp-sha lifetime time 3600
!
ipsec autokey-map ipsec-map1 sec-list peer 100.100.100.101 ipsec-prop pfs 1536-bit
!
ipsec autokey-map ipsec-map2 sec-list peer 100.100.100.102 ipsec-prop pfs 1536-bit
!
!
!
!
!
!
!
proxy-dns ip enable
proxy-dns interface GigaEthernet0.1 priority 254
!
telnet-server ip enable
!
ssh-server ip enable
!
http-server username user
http-server ip access-list web-http-acl
http-server ip enable
!
!
!
watch-group watch_tunnel0 10
event 20 ip unreach-host 100.100.100.101 Tunnel0.0 source GigaEthernet1.0
action 20 ipsec clear-sa Tunnel0.0
!
network-monitor watch_tunnel0 enable
!
watch-group watch_tunnel1 10
event 20 ip unreach-host 100.100.100.102 Tunnel1.0 source GigaEthernet1.0
action 20 ipsec clear-sa Tunnel1.0
!
network-monitor watch_tunnel1 enable
!
!
route-map bgp permit 65000
!
route-map rmap-in permit 10
match ip address prefix-list as-path-in
set as-path prepend 31898 31898
!
route-map rmap-out permit 10
match ip address prefix-list as-path-out
set as-path prepend 65000 65000
!
ppp profile web-ppp-gigaethernet0.1
authentication myname user@ntt.ne.jp
authentication password user@ntt.ne.jp a12cpe
!
ip dhcp profile web-dhcp-gigaethernet1.0
dns-server 192.168.100.240
!
router bgp 65000
neighbor 10.254.200.202 remote-as 31898
neighbor 10.254.200.202 timers 10 30
neighbor 10.254.200.206 remote-as 31898
neighbor 10.254.200.206 timers 10 30
address-family ipv4 unicast
neighbor 10.254.200.206 route-map rmap-in in
neighbor 10.254.200.206 route-map rmap-out out
network 192.168.100.0/24
!
device GigaEthernet0
!
device GigaEthernet1
!
interface GigaEthernet0.0
no ip address
shutdown
!
interface GigaEthernet1.0
description LAN1
ip address 192.168.100.240/24
ip dhcp binding web_gigaethernet1.0
no shutdown
!
interface GigaEthernet0.1
description WAN1
encapsulation pppoe
auto-connect
ppp binding web-ppp-gigaethernet0.1
ip address ipcp
ip tcp adjust-mss auto
ip napt enable
ip napt hairpinning
no shutdown
!
interface Loopback0.0
no ip address
!
interface Null0.0
no ip address
!
interface Tunnel0.0
tunnel mode ipsec
ip address 10.254.200.201/30
ip tcp adjust-mss auto
ipsec policy tunnel ipsec-map1 out
no shutdown
!
interface Tunnel1.0
tunnel mode ipsec
ip address 10.254.200.205/30
ip tcp adjust-mss auto
ipsec policy tunnel ipsec-map2 out
no shutdown
!
web-console system information
o lan1 GigaEthernet1.0
o wizard easy-inet-only-pppoe
o wan1 GigaEthernet0.1
#■BGP確認
##●BGP確立確認
すべてのStatusがESTABLISHEDであることを確認
Router(config)# show ip bgp summary
BGP router ID 192.168.100.240, local AS number 65000
1 BGP AS-PATH entries
Neighbor V AS MsgRcvd MsgSent Up/DownTime State
10.254.200.202 4 31898 31 23 0:00:36 ESTABLISHED
10.254.200.206 4 31898 29 23 0:00:35 ESTABLISHED
Total number of neighbors 2
##●ルート情報伝搬確認
Router(config)# show ip route
IP Routing Table - 8 entries, 3 hidden, 2037 frees
Entries: 4 Connected, 1 Static, 0 RIP, 0 OSPF, 3 BGP
Codes: C - Connected, S - Static, R - RIP, O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2, B - BGP
* - Candidate default, s - Summary
Timers: Age
S* 0.0.0.0/0 [1/1] is directly connected, GigaEthernet0.1
10.0.0.0/8 is subnetted, 3 subnets
B 10.0.0.0/24 [20/0] via 10.254.200.202, Tunnel0.0
10.254.0.0/16 is subnetted, 2 subnets
C 10.254.200.200/30 [0/1] is directly connected, Tunnel0.0
C 10.254.200.204/30 [0/1] is directly connected, Tunnel1.0
C 192.168.100.0/24 [0/1] is directly connected, GigaEthernet1.0
200.200.200.0/24 is subnetted, 1 subnets
C 200.200.200.201/32 [0/1] is directly connected, GigaEthernet0.1
・BGP AS Path設定確認
バックアップ・パスに：AS番号が３つ付与されていることを確認
・プライマリ・パス(10.254.200.202) ：31898 i
・バップアツプ・パス(10.254.200.206)：31898 31898 31898 i
Router(config)# show ip bgp
BGP table version is 25, local router ID is 192.168.100.240
Local AS number 65000
Status codes: s - suppressed, * - valid, h - history
> - best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Path
* 10.0.0.0/24 10.254.200.206 31898 31898 31898 i
*> 10.254.200.202 31898 i
*> 192.168.100.0/24 0.0.0.0 1 i
Total number of prefixes 7
Router(config)# show ip bgp paths
Address RefCount Next Hop MED LocPrf Path
0x0302 12 10.254.200.202 0 100 31898 i
0x0313 3 10.254.200.206 0 100 31898 31898 31898 i
0x0314 2 0.0.0.0 1* 100 i
0x0311 1 10.254.200.201 1* 100 i
0x0301 1 10.254.200.205 1* 100 65000 65000 i
##●プライマリ・パス/バップアツプ・パス確認
・バップアツプ・パス
Router(config)# show ip bgp neighbors 10.254.200.202 advertised-routes
BGP table version is 25, local router ID is 192.168.100.240
Local AS number 65000
Status codes: s - suppressed, * - valid, h - history
> - best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Path
*> 192.168.100.0/24 10.254.200.201 1 i
Total number of prefixes 1
Router(config)# show ip bgp neighbors 10.254.200.202 received-routes
BGP table version is 25, local router ID is 192.168.100.240
Local AS number 65000
Status codes: s - suppressed, * - valid, h - history
> - best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Path
*> 10.0.0.0/24 10.254.200.202 31898 i
Total number of prefixes 1
・プライマリ・パス
Router(config)# show ip bgp neighbors 10.254.200.206 received-routes
BGP table version is 25, local router ID is 192.168.100.240
Local AS number 65000
Status codes: s - suppressed, * - valid, h - history
> - best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Path
* 10.0.0.0/24 10.254.200.206 31898 31898 31898 i
Total number of prefixes 1
Router(config)# show ip bgp neighbors 10.254.200.206 advertised-routes
BGP table version is 25, local router ID is 192.168.100.240
Local AS number 65000
Status codes: s - suppressed, * - valid, h - history
> - best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Path
*> 192.168.100.0/24 10.254.200.205 1 65000 65000 i
Total number of prefixes 1
#■Oracle Cloud IPSec/BGPステータス確認
ルータと接続が確立されると「IPSec Status」と「BGP Status」がUPします
#■プライマリ・パス障害検証
プライマリ・パス Interfaceを停止すると非疎通になるが、holdtime30秒以内に疎通が復旧することを確認し、
切り戻しで停止したプライマリ・パス Tunnnel0.0 Interfaceを起動し疎通に問題が無いことを確認します。
##●事前確認
・ping監視実行
On-Premises側からOCI側のInstanseへpingを行い、障害試験実施によりどのタイミングで疎通できなくなるか確認
onp-inst01:~ user$ ping 10.0.0.2
PING 10.0.0.2 (10.0.0.2): 56 data bytes
64 bytes from 10.0.0.2: icmp_seq=0 ttl=62 time=6.210 ms
64 bytes from 10.0.0.2: icmp_seq=1 ttl=62 time=6.067 ms
64 bytes from 10.0.0.2: icmp_seq=2 ttl=62 time=14.000 ms
・BGP確立確認
すべてのStatusがESTABLISHEDであることを確認
Router(config)# show ip bgp summary
BGP router ID 192.168.100.240, local AS number 65000
4 BGP AS-PATH entries
Neighbor V AS MsgRcvd MsgSent Up/DownTime State
10.254.200.202 4 31898 18268 16511 1d21h48m28s ESTABLISHED
10.254.200.206 4 31898 18284 16520 1d21h50m27s ESTABLISHED
Total number of neighbors 2
・BGPパス確認
すべてのPathがあることを確認
Router(config)# show ip bgp
BGP table version is 25, local router ID is 192.168.100.240
Local AS number 65000
Status codes: s - suppressed, * - valid, h - history
> - best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Path
* 10.0.0.0/24 10.254.200.206 31898 31898 31898 i
*> 10.254.200.202 31898 i
*> 192.168.100.0/24 0.0.0.0 1 i
Total number of prefixes 5
##●プライマリ・パス Interface停止実行
Router(config)# interface Tunnel0.0
Router(config-Tunnel1.0)# shutdown
Router(config)# show interfaces Tunnel0.0
Interface Tunnel0.0 is administratively down
・・・
・Ping:応答確認
プライマリ・パス Interface停止により疎通しなくなることを確認
onp-inst01:~ user$ ping 10.0.0.2
・・・
64 bytes from 10.0.0.2: icmp_seq=0 ttl=62 time=6.210 ms
64 bytes from 10.0.0.2: icmp_seq=1 ttl=62 time=6.067 ms
64 bytes from 10.0.0.2: icmp_seq=2 ttl=62 time=14.000 ms
64 bytes from 10.0.0.2: icmp_seq=3 ttl=62 time=4.979 ms
Request timeout for icmp_seq 4
Request timeout for icmp_seq 5
Request timeout for icmp_seq 6
Request timeout for icmp_seq 7
Request timeout for icmp_seq 8
・プライマリ・パス BGP Status IDLE確認
Router(config)# show ip bgp summary
BGP router ID 192.168.100.240, local AS number 65000
3 BGP AS-PATH entries
Neighbor V AS MsgRcvd MsgSent Up/DownTime State
10.254.200.202 4 31898 721 641 0:02:28 ESTABLISHED
10.254.200.206 4 31898 415 378 0:00:24 IDLE
・BGP Path確認
バックアップ・パスのみがあることを確認
Router(config)# show ip bgp
BGP table version is 94, local router ID is 192.168.100.240
Local AS number 65000
Status codes: s - suppressed, * - valid, h - history
> - best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Path
*> 10.0.0.0/24 10.254.200.202 31898 31898 31898 i
10.254.200.0/24 0.0.0.0 0 i
*> 192.168.100.0/24 0.0.0.0 1 i
Total number of prefixes 3
Router(config)#
・Ping 30秒で復旧確認
holdtime30秒以内に疎通が復旧することを確認
onp-inst01:~ user$ ping 10.0.0.2
PING 10.0.0.2 (10.0.0.2): 56 data bytes
64 bytes from 10.0.0.2: icmp_seq=0 ttl=62 time=6.210 ms
64 bytes from 10.0.0.2: icmp_seq=1 ttl=62 time=6.067 ms
64 bytes from 10.0.0.2: icmp_seq=2 ttl=62 time=14.000 ms
64 bytes from 10.0.0.2: icmp_seq=3 ttl=62 time=4.979 ms
Request timeout for icmp_seq 4
Request timeout for icmp_seq 5
Request timeout for icmp_seq 6
Request timeout for icmp_seq 7
Request timeout for icmp_seq 8
Request timeout for icmp_seq 9
Request timeout for icmp_seq 10
Request timeout for icmp_seq 11
Request timeout for icmp_seq 12
Request timeout for icmp_seq 13
Request timeout for icmp_seq 14
Request timeout for icmp_seq 15
Request timeout for icmp_seq 16
Request timeout for icmp_seq 17
Request timeout for icmp_seq 18
Request timeout for icmp_seq 19
Request timeout for icmp_seq 20
Request timeout for icmp_seq 21
Request timeout for icmp_seq 22
Request timeout for icmp_seq 23
Request timeout for icmp_seq 24
Request timeout for icmp_seq 25
Request timeout for icmp_seq 26
Request timeout for icmp_seq 27
Request timeout for icmp_seq 28
Request timeout for icmp_seq 29
64 bytes from 10.0.0.2: icmp_seq=30 ttl=62 time=5.853 ms
64 bytes from 10.0.0.2: icmp_seq=31 ttl=62 time=5.505 ms
64 bytes from 10.0.0.2: icmp_seq=32 ttl=62 time=5.388 ms
64 bytes from 10.0.0.2: icmp_seq=33 ttl=62 time=38.859 ms
64 bytes from 10.0.0.2: icmp_seq=34 ttl=62 time=29.210 ms
64 bytes from 10.0.0.2: icmp_seq=35 ttl=62 time=8.350 ms
#■切り戻し
##●プライマリ・パス Interface起動
Router(config)# interface Tunnel0.0
Router(config-Tunnel0.0)# no shutdown
・プライマリ・パス Interface起動確認
Router(config-Tunnel0.0)# show interfaces stats
Interface Tunnel0.0 is up
・・・
・BGP Status ESTABLESHED確認
プライマリ・パスがIDLEからESTABLISHEDに復旧したことを確認
Router(config)# show ip bgp summary
BGP router ID 192.168.100.240, local AS number 65000
4 BGP AS-PATH entries
Neighbor V AS MsgRcvd MsgSent Up/DownTime State
10.254.200.202 4 31898 806 719 0:15:20 ESTABLISHED
10.254.200.206 4 31898 424 386 0:00:57 ESTABLISHED
Total number of neighbors 2
・BGP Path確認
プライマリ・パスが復旧していることを確認
Router(config)# show ip bgp
BGP table version is 133, local router ID is 192.168.100.240
Local AS number 65000
Status codes: s - suppressed, * - valid, h - history
> - best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Path
*> 10.0.0.0/24 10.254.200.206 31898 i
* 10.254.200.202 31898 31898 31898 i
10.254.200.0/24 0.0.0.0 0 i
*> 192.168.100.0/24 0.0.0.0 1 i
Total number of prefixes 6
・ping 異常無し確認
切り戻しによる非疎通が無いことを確認
・・・
Request timeout for icmp_seq 27
Request timeout for icmp_seq 28
Request timeout for icmp_seq 29
64 bytes from 10.0.0.2: icmp_seq=30 ttl=62 time=5.853 ms
64 bytes from 10.0.0.2: icmp_seq=31 ttl=62 time=5.505 ms
64 bytes from 10.0.0.2: icmp_seq=32 ttl=62 time=5.388 ms
64 bytes from 10.0.0.2: icmp_seq=33 ttl=62 time=38.859 ms
64 bytes from 10.0.0.2: icmp_seq=34 ttl=62 time=29.210 ms
64 bytes from 10.0.0.2: icmp_seq=35 ttl=62 time=8.350 ms
・・・
64 bytes from 10.0.0.2: icmp_seq=300 ttl=62 time=11.249 ms
64 bytes from 10.0.0.2: icmp_seq=301 ttl=62 time=43.254 ms
64 bytes from 10.0.0.2: icmp_seq=302 ttl=62 time=12.917 ms
64 bytes from 10.0.0.2: icmp_seq=303 ttl=62 time=12.908 ms
64 bytes from 10.0.0.2: icmp_seq=304 ttl=62 time=25.170 ms
64 bytes from 10.0.0.2: icmp_seq=305 ttl=62 time=49.422 ms
64 bytes from 10.0.0.2: icmp_seq=306 ttl=62 time=13.343 ms
64 bytes from 10.0.0.2: icmp_seq=307 ttl=62 time=38.864 ms
64 bytes from 10.0.0.2: icmp_seq=308 ttl=62 time=25.237 ms
#■BGP確立連続稼働確認
1日以上連続稼働させてBGPの設定に問題がないことを確認
Router(config)# show ip bgp summary
BGP router ID 192.168.100.240, local AS number 65000
4 BGP AS-PATH entries
Neighbor V AS MsgRcvd MsgSent Up/DownTime State
10.254.200.202 4 31898 18268 16511 1d21h48m28s ESTABLISHED
10.254.200.206 4 31898 18284 16520 1d21h50m27s ESTABLISHED
##■参考
・マニュアル：https://jpn.nec.com/univerge/ix/Manual/index.html#crm