Edited at

UNIVERGE IXルーターとOracle CloudをIPSec接続してAS-Path Prependで経路制御してみてみた


■目的

Interop Tokyo 2019にいってきました。

NECさんのUNIVERGE QX-W2120がShowNetの構成要素に組まれており、安心の冗長構成がされていました。

ということで、UNIVERGE IXルーターとOCIをBGPでIPsec接続して冗長化した2つのIPSec TunnelをAS-Path PrependでTunnel1を優先するよう経路制御を行い、可用性検証してみます。

BGPの設定と確認方法はIXプロフェッショナル @hongmnh さんに協力いただきました。


■構成


■手順

主な手順は、こちらを参考としてStatic Routesの設定部分をBGP設定に変更して構成します。

Oracle Cloud:NEC UNIVERGE IXルーターでOCIへIPsec VPN接続してみてみた


●BGP設定

!

ip prefix-list as-path-in 10 permit any
ip prefix-list as-path-out 10 permit any
!
route-map bgp permit 65000
!
!AS-Path制御設定
route-map rmap-in permit 10
match ip address prefix-list as-path-in
set as-path prepend 31898 31898
!
route-map rmap-out permit 10
match ip address prefix-list as-path-out
set as-path prepend 65000 65000
!
! Keepaliveメッセージを10秒、holdtimeを30秒として設定
router bgp 65000
neighbor 10.254.200.202 remote-as 31898
neighbor 10.254.200.202 timers 10 30
neighbor 10.254.200.206 remote-as 31898
neighbor 10.254.200.206 timers 10 30
address-family ipv4 unicast
neighbor 10.254.200.206 route-map rmap-in in
neighbor 10.254.200.206 route-map rmap-out out
network 192.168.100.0/24
!
!Tunnel interface にBGP用IPを設定
interface Tunnel0.0
tunnel mode ipsec
ip address 10.254.200.201/30
ip tcp adjust-mss auto
ipsec policy tunnel ipsec-map1 out
no shutdown
!
interface Tunnel1.0
tunnel mode ipsec
ip address 10.254.200.205/30
ip tcp adjust-mss auto
ipsec policy tunnel ipsec-map2 out
no shutdown


●config確認

Router(config)# show config

Using 3714 out of 524288 bytes

! NEC Portable Internetwork Core Operating System Software
! IX Series IX2105 (magellan-sec) Software, Version 9.6.12A, MAINTENANCE RELEASE SOFTWARE
! Compiled Oct 05-Thu-2017 19:27:09 JST #2
! Last updated Jun 14-Fri-2019 00:58:40 JST
!
!% You must restart the router for this configuration to take effect.
!
timezone +09 00
!
username user password hash 123ABAD44 administrator
!
!
ip ufs-cache enable
ip multipath per-flow
ip route default GigaEthernet0.1
ip prefix-list as-path-in 10 permit any
ip prefix-list as-path-out 10 permit any
ip access-list sec-list permit ip src any dest any
ip access-list web-http-acl permit ip src any dest 192.168.100.240/32
ip access-list web_console permit ip src any dest 192.168.100.240/32
!
!
!
ike proposal ike-prop encryption aes-256 hash sha2-256 group 1536-bit
!
ike policy ike-pol1 peer 100.100.100.101 key IPSecIX200Password ike-prop
!
ike policy ike-pol2 peer 100.100.100.102 key IPSecIX200Password ike-prop
!
ipsec autokey-proposal ipsec-prop esp-aes-256 esp-sha lifetime time 3600
!
ipsec autokey-map ipsec-map1 sec-list peer 100.100.100.101 ipsec-prop pfs 1536-bit
!
ipsec autokey-map ipsec-map2 sec-list peer 100.100.100.102 ipsec-prop pfs 1536-bit
!
!
!
!
!
!
!
proxy-dns ip enable
proxy-dns interface GigaEthernet0.1 priority 254
!
telnet-server ip enable
!
ssh-server ip enable
!
http-server username user
http-server ip access-list web-http-acl
http-server ip enable
!
!
!
watch-group watch_tunnel0 10
event 20 ip unreach-host 100.100.100.101 Tunnel0.0 source GigaEthernet1.0
action 20 ipsec clear-sa Tunnel0.0
!
network-monitor watch_tunnel0 enable
!
watch-group watch_tunnel1 10
event 20 ip unreach-host 100.100.100.102 Tunnel1.0 source GigaEthernet1.0
action 20 ipsec clear-sa Tunnel1.0
!
network-monitor watch_tunnel1 enable
!
!
route-map bgp permit 65000
!
route-map rmap-in permit 10
match ip address prefix-list as-path-in
set as-path prepend 31898 31898
!
route-map rmap-out permit 10
match ip address prefix-list as-path-out
set as-path prepend 65000 65000
!
ppp profile web-ppp-gigaethernet0.1
authentication myname user@ntt.ne.jp
authentication password user@ntt.ne.jp a12cpe
!
ip dhcp profile web-dhcp-gigaethernet1.0
dns-server 192.168.100.240
!
router bgp 65000
neighbor 10.254.200.202 remote-as 31898
neighbor 10.254.200.202 timers 10 30
neighbor 10.254.200.206 remote-as 31898
neighbor 10.254.200.206 timers 10 30
address-family ipv4 unicast
neighbor 10.254.200.206 route-map rmap-in in
neighbor 10.254.200.206 route-map rmap-out out
network 192.168.100.0/24
!
device GigaEthernet0
!
device GigaEthernet1
!
interface GigaEthernet0.0
no ip address
shutdown
!
interface GigaEthernet1.0
description LAN1
ip address 192.168.100.240/24
ip dhcp binding web_gigaethernet1.0
no shutdown
!
interface GigaEthernet0.1
description WAN1
encapsulation pppoe
auto-connect
ppp binding web-ppp-gigaethernet0.1
ip address ipcp
ip tcp adjust-mss auto
ip napt enable
ip napt hairpinning
no shutdown
!
interface Loopback0.0
no ip address
!
interface Null0.0
no ip address
!
interface Tunnel0.0
tunnel mode ipsec
ip address 10.254.200.201/30
ip tcp adjust-mss auto
ipsec policy tunnel ipsec-map1 out
no shutdown
!
interface Tunnel1.0
tunnel mode ipsec
ip address 10.254.200.205/30
ip tcp adjust-mss auto
ipsec policy tunnel ipsec-map2 out
no shutdown
!
web-console system information
o lan1 GigaEthernet1.0
o wizard easy-inet-only-pppoe
o wan1 GigaEthernet0.1


■BGP確認


●BGP確立確認

すべてのStatusがESTABLISHEDであることを確認

Router(config)# show ip bgp summary

BGP router ID 192.168.100.240, local AS number 65000
1 BGP AS-PATH entries

Neighbor V AS MsgRcvd MsgSent Up/DownTime State
10.254.200.202 4 31898 31 23 0:00:36 ESTABLISHED
10.254.200.206 4 31898 29 23 0:00:35 ESTABLISHED

Total number of neighbors 2


●ルート情報伝搬確認

Router(config)# show ip route

IP Routing Table - 8 entries, 3 hidden, 2037 frees
Entries: 4 Connected, 1 Static, 0 RIP, 0 OSPF, 3 BGP
Codes: C - Connected, S - Static, R - RIP, O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2, B - BGP
* - Candidate default, s - Summary
Timers: Age
S* 0.0.0.0/0 [1/1] is directly connected, GigaEthernet0.1
10.0.0.0/8 is subnetted, 3 subnets
B 10.0.0.0/24 [20/0] via 10.254.200.202, Tunnel0.0
10.254.0.0/16 is subnetted, 2 subnets
C 10.254.200.200/30 [0/1] is directly connected, Tunnel0.0
C 10.254.200.204/30 [0/1] is directly connected, Tunnel1.0
C 192.168.100.0/24 [0/1] is directly connected, GigaEthernet1.0
200.200.200.0/24 is subnetted, 1 subnets
C 200.200.200.201/32 [0/1] is directly connected, GigaEthernet0.1

・BGP AS Path設定確認

バックアップ・パスに:AS番号が3つ付与されていることを確認

・プライマリ・パス(10.254.200.202) :31898 i

・バップアツプ・パス(10.254.200.206):31898 31898 31898 i

Router(config)# show ip bgp

BGP table version is 25, local router ID is 192.168.100.240
Local AS number 65000
Status codes: s - suppressed, * - valid, h - history
> - best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete

Network Next Hop Metric LocPrf Path
* 10.0.0.0/24 10.254.200.206 31898 31898 31898 i
*> 10.254.200.202 31898 i
*> 192.168.100.0/24 0.0.0.0 1 i

Total number of prefixes 7

Router(config)# show ip bgp paths
Address RefCount Next Hop MED LocPrf Path
0x0302 12 10.254.200.202 0 100 31898 i
0x0313 3 10.254.200.206 0 100 31898 31898 31898 i
0x0314 2 0.0.0.0 1* 100 i
0x0311 1 10.254.200.201 1* 100 i
0x0301 1 10.254.200.205 1* 100 65000 65000 i


●プライマリ・パス/バップアツプ・パス確認

・バップアツプ・パス

Router(config)# show ip bgp neighbors 10.254.200.202 advertised-routes

BGP table version is 25, local router ID is 192.168.100.240
Local AS number 65000
Status codes: s - suppressed, * - valid, h - history
> - best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete

Network Next Hop Metric LocPrf Path
*> 192.168.100.0/24 10.254.200.201 1 i

Total number of prefixes 1

Router(config)# show ip bgp neighbors 10.254.200.202 received-routes
BGP table version is 25, local router ID is 192.168.100.240
Local AS number 65000
Status codes: s - suppressed, * - valid, h - history
> - best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete

Network Next Hop Metric LocPrf Path
*> 10.0.0.0/24 10.254.200.202 31898 i

Total number of prefixes 1

・プライマリ・パス

Router(config)# show ip bgp neighbors 10.254.200.206 received-routes

BGP table version is 25, local router ID is 192.168.100.240
Local AS number 65000
Status codes: s - suppressed, * - valid, h - history
> - best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete

Network Next Hop Metric LocPrf Path
* 10.0.0.0/24 10.254.200.206 31898 31898 31898 i

Total number of prefixes 1

Router(config)# show ip bgp neighbors 10.254.200.206 advertised-routes
BGP table version is 25, local router ID is 192.168.100.240
Local AS number 65000
Status codes: s - suppressed, * - valid, h - history
> - best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete

Network Next Hop Metric LocPrf Path
*> 192.168.100.0/24 10.254.200.205 1 65000 65000 i

Total number of prefixes 1


■プライマリ・パス障害検証

プライマリ・パス Interfaceを停止すると非疎通になるが、holdtime30秒以内に疎通が復旧することを確認し、

切り戻しで停止したプライマリ・パス Tunnnel0.0 Interfaceを起動し疎通に問題が無いことを確認します。


●事前確認

・ping監視実行

On-Premises側からOCI側のInstanseへpingを行い、障害試験実施によりどのタイミングで疎通できなくなるか確認

onp-inst01:~ user$ ping 10.0.0.2

PING 10.0.0.2 (10.0.0.2): 56 data bytes
64 bytes from 10.0.0.2: icmp_seq=0 ttl=62 time=6.210 ms
64 bytes from 10.0.0.2: icmp_seq=1 ttl=62 time=6.067 ms
64 bytes from 10.0.0.2: icmp_seq=2 ttl=62 time=14.000 ms

・BGP確立確認

すべてのStatusがESTABLISHEDであることを確認

Router(config)# show ip bgp summary

BGP router ID 192.168.100.240, local AS number 65000
4 BGP AS-PATH entries

Neighbor V AS MsgRcvd MsgSent Up/DownTime State
10.254.200.202 4 31898 18268 16511 1d21h48m28s ESTABLISHED
10.254.200.206 4 31898 18284 16520 1d21h50m27s ESTABLISHED

Total number of neighbors 2

・BGPパス確認

すべてのPathがあることを確認

Router(config)# show ip bgp

BGP table version is 25, local router ID is 192.168.100.240
Local AS number 65000
Status codes: s - suppressed, * - valid, h - history
> - best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete

Network Next Hop Metric LocPrf Path
* 10.0.0.0/24 10.254.200.206 31898 31898 31898 i
*> 10.254.200.202 31898 i
*> 192.168.100.0/24 0.0.0.0 1 i

Total number of prefixes 5


●プライマリ・パス Interface停止実行

Router(config)# interface Tunnel0.0

Router(config-Tunnel1.0)# shutdown
Router(config)# show interfaces Tunnel0.0
Interface Tunnel0.0 is administratively down
・・・

・Ping:応答確認

プライマリ・パス Interface停止により疎通しなくなることを確認

onp-inst01:~ user$ ping 10.0.0.2

・・・
64 bytes from 10.0.0.2: icmp_seq=0 ttl=62 time=6.210 ms
64 bytes from 10.0.0.2: icmp_seq=1 ttl=62 time=6.067 ms
64 bytes from 10.0.0.2: icmp_seq=2 ttl=62 time=14.000 ms
64 bytes from 10.0.0.2: icmp_seq=3 ttl=62 time=4.979 ms
Request timeout for icmp_seq 4
Request timeout for icmp_seq 5
Request timeout for icmp_seq 6
Request timeout for icmp_seq 7
Request timeout for icmp_seq 8

・プライマリ・パス BGP Status IDLE確認

Router(config)# show ip bgp summary

BGP router ID 192.168.100.240, local AS number 65000
3 BGP AS-PATH entries

Neighbor V AS MsgRcvd MsgSent Up/DownTime State
10.254.200.202 4 31898 721 641 0:02:28 ESTABLISHED
10.254.200.206 4 31898 415 378 0:00:24 IDLE

・BGP Path確認

バックアップ・パスのみがあることを確認

Router(config)# show ip bgp

BGP table version is 94, local router ID is 192.168.100.240
Local AS number 65000
Status codes: s - suppressed, * - valid, h - history
> - best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete

Network Next Hop Metric LocPrf Path
*> 10.0.0.0/24 10.254.200.202 31898 31898 31898 i
10.254.200.0/24 0.0.0.0 0 i
*> 192.168.100.0/24 0.0.0.0 1 i

Total number of prefixes 3
Router(config)#

・Ping 30秒で復旧確認

holdtime30秒以内に疎通が復旧することを確認

onp-inst01:~ user$ ping 10.0.0.2

PING 10.0.0.2 (10.0.0.2): 56 data bytes
64 bytes from 10.0.0.2: icmp_seq=0 ttl=62 time=6.210 ms
64 bytes from 10.0.0.2: icmp_seq=1 ttl=62 time=6.067 ms
64 bytes from 10.0.0.2: icmp_seq=2 ttl=62 time=14.000 ms
64 bytes from 10.0.0.2: icmp_seq=3 ttl=62 time=4.979 ms
Request timeout for icmp_seq 4
Request timeout for icmp_seq 5
Request timeout for icmp_seq 6
Request timeout for icmp_seq 7
Request timeout for icmp_seq 8
Request timeout for icmp_seq 9
Request timeout for icmp_seq 10
Request timeout for icmp_seq 11
Request timeout for icmp_seq 12
Request timeout for icmp_seq 13
Request timeout for icmp_seq 14
Request timeout for icmp_seq 15
Request timeout for icmp_seq 16
Request timeout for icmp_seq 17
Request timeout for icmp_seq 18
Request timeout for icmp_seq 19
Request timeout for icmp_seq 20
Request timeout for icmp_seq 21
Request timeout for icmp_seq 22
Request timeout for icmp_seq 23
Request timeout for icmp_seq 24
Request timeout for icmp_seq 25
Request timeout for icmp_seq 26
Request timeout for icmp_seq 27
Request timeout for icmp_seq 28
Request timeout for icmp_seq 29
64 bytes from 10.0.0.2: icmp_seq=30 ttl=62 time=5.853 ms
64 bytes from 10.0.0.2: icmp_seq=31 ttl=62 time=5.505 ms
64 bytes from 10.0.0.2: icmp_seq=32 ttl=62 time=5.388 ms
64 bytes from 10.0.0.2: icmp_seq=33 ttl=62 time=38.859 ms
64 bytes from 10.0.0.2: icmp_seq=34 ttl=62 time=29.210 ms
64 bytes from 10.0.0.2: icmp_seq=35 ttl=62 time=8.350 ms


■切り戻し


●プライマリ・パス Interface起動

Router(config)# interface Tunnel0.0

Router(config-Tunnel0.0)# no shutdown

・プライマリ・パス Interface起動確認

Router(config-Tunnel0.0)# show interfaces stats

Interface Tunnel0.0 is up
・・・

・BGP Status ESTABLESHED確認

プライマリ・パスがIDLEからESTABLISHEDに復旧したことを確認

Router(config)# show ip bgp summary

BGP router ID 192.168.100.240, local AS number 65000
4 BGP AS-PATH entries

Neighbor V AS MsgRcvd MsgSent Up/DownTime State
10.254.200.202 4 31898 806 719 0:15:20 ESTABLISHED
10.254.200.206 4 31898 424 386 0:00:57 ESTABLISHED

Total number of neighbors 2

・BGP Path確認

プライマリ・パスが復旧していることを確認

Router(config)# show ip bgp

BGP table version is 133, local router ID is 192.168.100.240
Local AS number 65000
Status codes: s - suppressed, * - valid, h - history
> - best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete

Network Next Hop Metric LocPrf Path
*> 10.0.0.0/24 10.254.200.206 31898 i
* 10.254.200.202 31898 31898 31898 i
10.254.200.0/24 0.0.0.0 0 i
*> 192.168.100.0/24 0.0.0.0 1 i

Total number of prefixes 6

・ping 異常無し確認

切り戻しによる非疎通が無いことを確認

    ・・・

Request timeout for icmp_seq 27
Request timeout for icmp_seq 28
Request timeout for icmp_seq 29
64 bytes from 10.0.0.2: icmp_seq=30 ttl=62 time=5.853 ms
64 bytes from 10.0.0.2: icmp_seq=31 ttl=62 time=5.505 ms
64 bytes from 10.0.0.2: icmp_seq=32 ttl=62 time=5.388 ms
64 bytes from 10.0.0.2: icmp_seq=33 ttl=62 time=38.859 ms
64 bytes from 10.0.0.2: icmp_seq=34 ttl=62 time=29.210 ms
64 bytes from 10.0.0.2: icmp_seq=35 ttl=62 time=8.350 ms
・・・
64 bytes from 10.0.0.2: icmp_seq=300 ttl=62 time=11.249 ms
64 bytes from 10.0.0.2: icmp_seq=301 ttl=62 time=43.254 ms
64 bytes from 10.0.0.2: icmp_seq=302 ttl=62 time=12.917 ms
64 bytes from 10.0.0.2: icmp_seq=303 ttl=62 time=12.908 ms
64 bytes from 10.0.0.2: icmp_seq=304 ttl=62 time=25.170 ms
64 bytes from 10.0.0.2: icmp_seq=305 ttl=62 time=49.422 ms
64 bytes from 10.0.0.2: icmp_seq=306 ttl=62 time=13.343 ms
64 bytes from 10.0.0.2: icmp_seq=307 ttl=62 time=38.864 ms
64 bytes from 10.0.0.2: icmp_seq=308 ttl=62 time=25.237 ms


■BGP確立連続稼働確認

1日以上連続稼働させてBGPの設定に問題がないことを確認

Router(config)# show ip bgp summary

BGP router ID 192.168.100.240, local AS number 65000
4 BGP AS-PATH entries

Neighbor V AS MsgRcvd MsgSent Up/DownTime State
10.254.200.202 4 31898 18268 16511 1d21h48m28s ESTABLISHED
10.254.200.206 4 31898 18284 16520 1d21h50m27s ESTABLISHED


■参考

・マニュアル:https://jpn.nec.com/univerge/ix/Manual/index.html#crm