LoginSignup
3
0

More than 3 years have passed since last update.

@shirok(Shiro Kobayashi)in日本オラクル株式会社

UNIVERGE IXルーターとOracle CloudをIPSec接続してAS-Path Prependで経路制御してみてみた

Last updated at Posted at 2019-06-16

■目的

Interop Tokyo 2019にいってきました。
NECさんのUNIVERGE QX-W2120がShowNetの構成要素に組まれており、安心の冗長構成がされていました。
IMG_5862.JPG

ということで、UNIVERGE IXルーターとOCIをBGPでIPsec接続して冗長化した２つのIPSec TunnelをAS-Path PrependでTunnel１を優先するよう経路制御を行い、可用性検証してみます。
BGPの設定と確認方法はIXプロフェッショナル @hongmnh さんに協力いただきました。

■ X2015 構成

IX冗長構成.png

■手順

主な手順は、こちらを参考としてStatic Routesの設定部分をBGP設定に変更して構成します。
Oracle Cloud:NEC UNIVERGE IXルーターでOCIへIPsec VPN接続してみてみた

●Oracle Cloud IPSec作成

OCI Webコンソールから、[Network] > [IPSec Connection]から[Create IPsec COnnect]をクリックし、以下の様に設定

・IPSec情報
01_IPSec01.png

・IPSec Tunnel01情報
01_IPSec02.png

・IPSec Tunnel02情報
01_IPSec03.png

設定完了したら、[Create IPSec COnnect]をクリックし作成

01_IPSec04.png

作成すると、「Oracle VPN IP Address」に各TunnnelにPublic IPが付与されるので、このIPをルーター設定時に使用します。

●BGP設定

!
ip prefix-list as-path-in 10 permit any
ip prefix-list as-path-out 10 permit any
!
route-map bgp permit 65000
!
!AS-Path制御設定
route-map rmap-in permit 10
  match ip address prefix-list as-path-in
  set as-path prepend 31898 31898 
!
route-map rmap-out permit 10
  match ip address prefix-list as-path-out
  set as-path prepend 65000 65000 
!
! Keepaliveメッセージを10秒、holdtimeを30秒として設定
router bgp 65000
  neighbor 10.254.200.202 remote-as 31898
  neighbor 10.254.200.202 timers 10 30
  neighbor 10.254.200.206 remote-as 31898
  neighbor 10.254.200.206 timers 10 30
  address-family ipv4 unicast
    neighbor 10.254.200.206 route-map rmap-in in
    neighbor 10.254.200.206 route-map rmap-out out
    network 192.168.100.0/24
!
!Tunnel interface にBGP用IPを設定
interface Tunnel0.0
  tunnel mode ipsec
  ip address 10.254.200.201/30
  ip tcp adjust-mss auto
  ipsec policy tunnel ipsec-map1 out
  no shutdown
!
interface Tunnel1.0
  tunnel mode ipsec
  ip address 10.254.200.205/30
  ip tcp adjust-mss auto
  ipsec policy tunnel ipsec-map2 out
  no shutdown

●config確認

Router(config)# show config
    Using 3714 out of 524288 bytes

    ! NEC Portable Internetwork Core Operating System Software
    ! IX Series IX2105 (magellan-sec) Software, Version 9.6.12A, MAINTENANCE RELEASE SOFTWARE
    ! Compiled Oct 05-Thu-2017 19:27:09 JST #2
    ! Last updated Jun 14-Fri-2019 00:58:40 JST
    !
    !% You must restart the router for this configuration to take effect.
    !
    timezone +09 00
    !
    username user password hash 123ABAD44 administrator
    !
    !
    ip ufs-cache enable
    ip multipath per-flow
    ip route default GigaEthernet0.1
    ip prefix-list as-path-in 10 permit any
    ip prefix-list as-path-out 10 permit any
    ip access-list sec-list permit ip src any dest any
    ip access-list web-http-acl permit ip src any dest 192.168.100.240/32
    ip access-list web_console permit ip src any dest 192.168.100.240/32
    !
    !
    !
    ike proposal ike-prop encryption aes-256 hash sha2-256 group 1536-bit
    !
    ike policy ike-pol1 peer 100.100.100.101 key IPSecIX200Password ike-prop
    !
    ike policy ike-pol2 peer 100.100.100.102 key IPSecIX200Password ike-prop
    !
    ipsec autokey-proposal ipsec-prop esp-aes-256 esp-sha lifetime time 3600
    !
    ipsec autokey-map ipsec-map1 sec-list peer 100.100.100.101 ipsec-prop pfs 1536-bit
    !
    ipsec autokey-map ipsec-map2 sec-list peer 100.100.100.102 ipsec-prop pfs 1536-bit
    !
    !
    !
    !
    !
    !
    !
    proxy-dns ip enable
    proxy-dns interface GigaEthernet0.1 priority 254
    !
    telnet-server ip enable
    !
    ssh-server ip enable
    !
    http-server username user
    http-server ip access-list web-http-acl
    http-server ip enable
    !
    !
    !
    watch-group watch_tunnel0 10
    event 20 ip unreach-host 100.100.100.101 Tunnel0.0 source GigaEthernet1.0
    action 20 ipsec clear-sa Tunnel0.0
    !
    network-monitor watch_tunnel0 enable
    !
    watch-group watch_tunnel1 10
    event 20 ip unreach-host 100.100.100.102 Tunnel1.0 source GigaEthernet1.0
    action 20 ipsec clear-sa Tunnel1.0
    !
    network-monitor watch_tunnel1 enable
    !
    !
    route-map bgp permit 65000
    !
    route-map rmap-in permit 10
    match ip address prefix-list as-path-in
    set as-path prepend 31898 31898
    !
    route-map rmap-out permit 10
    match ip address prefix-list as-path-out
    set as-path prepend 65000 65000
    !
    ppp profile web-ppp-gigaethernet0.1
    authentication myname user@ntt.ne.jp
    authentication password user@ntt.ne.jp a12cpe
    !
    ip dhcp profile web-dhcp-gigaethernet1.0
    dns-server 192.168.100.240
    !
    router bgp 65000
    neighbor 10.254.200.202 remote-as 31898
    neighbor 10.254.200.202 timers 10 30
    neighbor 10.254.200.206 remote-as 31898
    neighbor 10.254.200.206 timers 10 30
    address-family ipv4 unicast
        neighbor 10.254.200.206 route-map rmap-in in
        neighbor 10.254.200.206 route-map rmap-out out
        network 192.168.100.0/24
    !
    device GigaEthernet0
    !
    device GigaEthernet1
    !
    interface GigaEthernet0.0
    no ip address
    shutdown
    !
    interface GigaEthernet1.0
    description LAN1
    ip address 192.168.100.240/24
    ip dhcp binding web_gigaethernet1.0
    no shutdown
    !
    interface GigaEthernet0.1
    description WAN1
    encapsulation pppoe
    auto-connect
    ppp binding web-ppp-gigaethernet0.1
    ip address ipcp
    ip tcp adjust-mss auto
    ip napt enable
    ip napt hairpinning
    no shutdown
    !
    interface Loopback0.0
    no ip address
    !
    interface Null0.0
    no ip address
    !
    interface Tunnel0.0
    tunnel mode ipsec
    ip address 10.254.200.201/30
    ip tcp adjust-mss auto
    ipsec policy tunnel ipsec-map1 out
    no shutdown
    !
    interface Tunnel1.0
    tunnel mode ipsec
    ip address 10.254.200.205/30
    ip tcp adjust-mss auto
    ipsec policy tunnel ipsec-map2 out
    no shutdown
    !
    web-console system information
    o lan1 GigaEthernet1.0
    o wizard easy-inet-only-pppoe
    o wan1 GigaEthernet0.1

■BGP確認

●BGP確立確認

すべてのStatusがESTABLISHEDであることを確認

Router(config)# show ip bgp summary
    BGP router ID 192.168.100.240, local AS number 65000
    1 BGP AS-PATH entries

    Neighbor         V    AS    MsgRcvd MsgSent Up/DownTime   State
    10.254.200.202 4    31898 31      23      0:00:36       ESTABLISHED
    10.254.200.206 4    31898 29      23      0:00:35       ESTABLISHED

    Total number of neighbors 2

●ルート情報伝搬確認

Router(config)# show ip route
    IP Routing Table - 8 entries, 3 hidden, 2037 frees
    Entries: 4 Connected, 1 Static, 0 RIP, 0 OSPF, 3 BGP
    Codes: C - Connected, S - Static, R - RIP, O - OSPF, IA - OSPF inter area
        N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
        E1 - OSPF external type 1, E2 - OSPF external type 2, B - BGP
        * - Candidate default, s - Summary
    Timers: Age
    S*   0.0.0.0/0 [1/1] is directly connected, GigaEthernet0.1
        10.0.0.0/8 is subnetted, 3 subnets
    B      10.0.0.0/24 [20/0] via 10.254.200.202, Tunnel0.0
        10.254.0.0/16 is subnetted, 2 subnets
    C      10.254.200.200/30 [0/1] is directly connected, Tunnel0.0
    C      10.254.200.204/30 [0/1] is directly connected, Tunnel1.0
    C    192.168.100.0/24 [0/1] is directly connected, GigaEthernet1.0
        200.200.200.0/24 is subnetted, 1 subnets
    C      200.200.200.201/32 [0/1] is directly connected, GigaEthernet0.1

・BGP AS Path設定確認
バックアップ・パスに：AS番号が３つ付与されていることを確認
・プライマリ・パス(10.254.200.202)　：31898 i
・バップアツプ・パス(10.254.200.206)：31898 31898 31898 i

Router(config)# show ip bgp
    BGP table version is 25, local router ID is 192.168.100.240
    Local AS number 65000
    Status codes: s - suppressed, * - valid, h - history
                > - best, i - internal
    Origin codes: i - IGP, e - EGP, ? - incomplete

    Network            Next Hop            Metric     LocPrf  Path
    *  10.0.0.0/24        10.254.200.206                        31898 31898 31898 i
    *>                    10.254.200.202                        31898 i
    *> 192.168.100.0/24   0.0.0.0                  1             i

    Total number of prefixes 7

Router(config)# show ip bgp paths
    Address    RefCount Next Hop               MED      LocPrf  Path
    0x0302       12 10.254.200.202          0         100  31898 i
    0x0313        3 10.254.200.206          0         100  31898 31898 31898 i
    0x0314        2 0.0.0.0                  1*        100  i
    0x0311        1 10.254.200.201          1*        100  i
    0x0301        1 10.254.200.205          1*        100  65000 65000 i

●プライマリ・パス/バップアツプ・パス確認

・バップアツプ・パス

Router(config)# show ip bgp neighbors 10.254.200.202 advertised-routes
    BGP table version is 25, local router ID is 192.168.100.240
    Local AS number 65000
    Status codes: s - suppressed, * - valid, h - history
                > - best, i - internal
    Origin codes: i - IGP, e - EGP, ? - incomplete

    Network            Next Hop            Metric     LocPrf  Path
    *> 192.168.100.0/24   10.254.200.201          1             i

    Total number of prefixes 1

Router(config)# show ip bgp neighbors 10.254.200.202 received-routes
    BGP table version is 25, local router ID is 192.168.100.240
    Local AS number 65000
    Status codes: s - suppressed, * - valid, h - history
                > - best, i - internal
    Origin codes: i - IGP, e - EGP, ? - incomplete

    Network            Next Hop            Metric     LocPrf  Path
    *> 10.0.0.0/24        10.254.200.202                        31898 i

    Total number of prefixes 1

・プライマリ・パス

Router(config)# show ip bgp neighbors 10.254.200.206 received-routes
    BGP table version is 25, local router ID is 192.168.100.240
    Local AS number 65000
    Status codes: s - suppressed, * - valid, h - history
                > - best, i - internal
    Origin codes: i - IGP, e - EGP, ? - incomplete

    Network            Next Hop            Metric     LocPrf  Path
    *  10.0.0.0/24        10.254.200.206                        31898 31898 31898 i

    Total number of prefixes 1

Router(config)# show ip bgp neighbors 10.254.200.206 advertised-routes
    BGP table version is 25, local router ID is 192.168.100.240
    Local AS number 65000
    Status codes: s - suppressed, * - valid, h - history
                > - best, i - internal
    Origin codes: i - IGP, e - EGP, ? - incomplete

    Network            Next Hop            Metric     LocPrf  Path
    *> 192.168.100.0/24   10.254.200.205          1             65000 65000 i

    Total number of prefixes 1

■Oracle Cloud IPSec/BGPステータス確認

ルータと接続が確立されると「IPSec Status」と「BGP Status」がUPします
01_IPSec05.png

■プライマリ・パス障害検証

プライマリ・パス Interfaceを停止すると非疎通になるが、holdtime30秒以内に疎通が復旧することを確認し、
切り戻しで停止したプライマリ・パス Tunnnel0.0 Interfaceを起動し疎通に問題が無いことを確認します。

●事前確認

・ping監視実行
On-Premises側からOCI側のInstanseへpingを行い、障害試験実施によりどのタイミングで疎通できなくなるか確認

onp-inst01:~ user$ ping 10.0.0.2
    PING 10.0.0.2 (10.0.0.2): 56 data bytes
    64 bytes from 10.0.0.2: icmp_seq=0 ttl=62 time=6.210 ms
    64 bytes from 10.0.0.2: icmp_seq=1 ttl=62 time=6.067 ms
    64 bytes from 10.0.0.2: icmp_seq=2 ttl=62 time=14.000 ms

・BGP確立確認
すべてのStatusがESTABLISHEDであることを確認

Router(config)# show ip bgp summary
    BGP router ID 192.168.100.240, local AS number 65000
    4 BGP AS-PATH entries

    Neighbor         V    AS    MsgRcvd MsgSent Up/DownTime   State
    10.254.200.202 4    31898 18268   16511   1d21h48m28s   ESTABLISHED
    10.254.200.206 4    31898 18284   16520   1d21h50m27s   ESTABLISHED

    Total number of neighbors 2

・BGPパス確認
すべてのPathがあることを確認

Router(config)# show ip bgp
    BGP table version is 25, local router ID is 192.168.100.240
    Local AS number 65000
    Status codes: s - suppressed, * - valid, h - history
                > - best, i - internal
    Origin codes: i - IGP, e - EGP, ? - incomplete

    Network            Next Hop            Metric     LocPrf  Path
    *  10.0.0.0/24        10.254.200.206                        31898 31898 31898 i
    *>                    10.254.200.202                        31898 i
    *> 192.168.100.0/24   0.0.0.0                  1             i

    Total number of prefixes 5

●プライマリ・パス Interface停止実行

Router(config)# interface Tunnel0.0
Router(config-Tunnel1.0)# shutdown
Router(config)# show interfaces Tunnel0.0
    Interface Tunnel0.0 is administratively down
    ・・・

・Ping:応答確認
プライマリ・パス Interface停止により疎通しなくなることを確認

onp-inst01:~ user$ ping 10.0.0.2
    ・・・
    64 bytes from 10.0.0.2: icmp_seq=0 ttl=62 time=6.210 ms
    64 bytes from 10.0.0.2: icmp_seq=1 ttl=62 time=6.067 ms
    64 bytes from 10.0.0.2: icmp_seq=2 ttl=62 time=14.000 ms
    64 bytes from 10.0.0.2: icmp_seq=3 ttl=62 time=4.979 ms
    Request timeout for icmp_seq 4
    Request timeout for icmp_seq 5
    Request timeout for icmp_seq 6
    Request timeout for icmp_seq 7
    Request timeout for icmp_seq 8

・プライマリ・パス BGP Status IDLE確認

Router(config)# show ip bgp summary
    BGP router ID 192.168.100.240, local AS number 65000
    3 BGP AS-PATH entries

    Neighbor         V    AS    MsgRcvd MsgSent Up/DownTime   State
    10.254.200.202 4    31898 721     641     0:02:28       ESTABLISHED
    10.254.200.206 4    31898 415     378     0:00:24       IDLE

・BGP Path確認
バックアップ・パスのみがあることを確認

Router(config)# show ip bgp
    BGP table version is 94, local router ID is 192.168.100.240
    Local AS number 65000
    Status codes: s - suppressed, * - valid, h - history
                > - best, i - internal
    Origin codes: i - IGP, e - EGP, ? - incomplete

    Network            Next Hop            Metric     LocPrf  Path
    *> 10.0.0.0/24        10.254.200.202                        31898 31898 31898 i
    10.254.200.0/24   0.0.0.0                  0             i
    *> 192.168.100.0/24   0.0.0.0                  1             i

    Total number of prefixes 3
    Router(config)#

・Ping　30秒で復旧確認
holdtime30秒以内に疎通が復旧することを確認

onp-inst01:~ user$ ping 10.0.0.2
    PING 10.0.0.2 (10.0.0.2): 56 data bytes
    64 bytes from 10.0.0.2: icmp_seq=0 ttl=62 time=6.210 ms
    64 bytes from 10.0.0.2: icmp_seq=1 ttl=62 time=6.067 ms
    64 bytes from 10.0.0.2: icmp_seq=2 ttl=62 time=14.000 ms
    64 bytes from 10.0.0.2: icmp_seq=3 ttl=62 time=4.979 ms
    Request timeout for icmp_seq 4
    Request timeout for icmp_seq 5
    Request timeout for icmp_seq 6
    Request timeout for icmp_seq 7
    Request timeout for icmp_seq 8
    Request timeout for icmp_seq 9
    Request timeout for icmp_seq 10
    Request timeout for icmp_seq 11
    Request timeout for icmp_seq 12
    Request timeout for icmp_seq 13
    Request timeout for icmp_seq 14
    Request timeout for icmp_seq 15
    Request timeout for icmp_seq 16
    Request timeout for icmp_seq 17
    Request timeout for icmp_seq 18
    Request timeout for icmp_seq 19
    Request timeout for icmp_seq 20
    Request timeout for icmp_seq 21
    Request timeout for icmp_seq 22
    Request timeout for icmp_seq 23
    Request timeout for icmp_seq 24
    Request timeout for icmp_seq 25
    Request timeout for icmp_seq 26
    Request timeout for icmp_seq 27
    Request timeout for icmp_seq 28
    Request timeout for icmp_seq 29
    64 bytes from 10.0.0.2: icmp_seq=30 ttl=62 time=5.853 ms
    64 bytes from 10.0.0.2: icmp_seq=31 ttl=62 time=5.505 ms
    64 bytes from 10.0.0.2: icmp_seq=32 ttl=62 time=5.388 ms
    64 bytes from 10.0.0.2: icmp_seq=33 ttl=62 time=38.859 ms
    64 bytes from 10.0.0.2: icmp_seq=34 ttl=62 time=29.210 ms
    64 bytes from 10.0.0.2: icmp_seq=35 ttl=62 time=8.350 ms

■切り戻し

●プライマリ・パス Interface起動

Router(config)# interface Tunnel0.0
Router(config-Tunnel0.0)# no shutdown

・プライマリ・パス Interface起動確認

Router(config-Tunnel0.0)# show interfaces stats
    Interface Tunnel0.0 is up
    ・・・

・BGP Status ESTABLESHED確認
プライマリ・パスがIDLEからESTABLISHEDに復旧したことを確認

Router(config)# show ip bgp summary
    BGP router ID 192.168.100.240, local AS number 65000
    4 BGP AS-PATH entries

    Neighbor         V    AS    MsgRcvd MsgSent Up/DownTime   State
    10.254.200.202 4    31898 806     719     0:15:20       ESTABLISHED
    10.254.200.206 4    31898 424     386     0:00:57       ESTABLISHED

    Total number of neighbors 2

・BGP Path確認
プライマリ・パスが復旧していることを確認

Router(config)# show ip bgp
    BGP table version is 133, local router ID is 192.168.100.240
    Local AS number 65000
    Status codes: s - suppressed, * - valid, h - history
                > - best, i - internal
    Origin codes: i - IGP, e - EGP, ? - incomplete

    Network            Next Hop            Metric     LocPrf  Path
    *> 10.0.0.0/24        10.254.200.206                        31898 i
    *                     10.254.200.202                        31898 31898 31898 i
    10.254.200.0/24   0.0.0.0                  0             i
    *> 192.168.100.0/24   0.0.0.0                  1             i

    Total number of prefixes 6

・ping 異常無し確認
切り戻しによる非疎通が無いことを確認

    ・・・
    Request timeout for icmp_seq 27
    Request timeout for icmp_seq 28
    Request timeout for icmp_seq 29
    64 bytes from 10.0.0.2: icmp_seq=30 ttl=62 time=5.853 ms
    64 bytes from 10.0.0.2: icmp_seq=31 ttl=62 time=5.505 ms
    64 bytes from 10.0.0.2: icmp_seq=32 ttl=62 time=5.388 ms
    64 bytes from 10.0.0.2: icmp_seq=33 ttl=62 time=38.859 ms
    64 bytes from 10.0.0.2: icmp_seq=34 ttl=62 time=29.210 ms
    64 bytes from 10.0.0.2: icmp_seq=35 ttl=62 time=8.350 ms
    ・・・
    64 bytes from 10.0.0.2: icmp_seq=300 ttl=62 time=11.249 ms
    64 bytes from 10.0.0.2: icmp_seq=301 ttl=62 time=43.254 ms
    64 bytes from 10.0.0.2: icmp_seq=302 ttl=62 time=12.917 ms
    64 bytes from 10.0.0.2: icmp_seq=303 ttl=62 time=12.908 ms
    64 bytes from 10.0.0.2: icmp_seq=304 ttl=62 time=25.170 ms
    64 bytes from 10.0.0.2: icmp_seq=305 ttl=62 time=49.422 ms
    64 bytes from 10.0.0.2: icmp_seq=306 ttl=62 time=13.343 ms
    64 bytes from 10.0.0.2: icmp_seq=307 ttl=62 time=38.864 ms
    64 bytes from 10.0.0.2: icmp_seq=308 ttl=62 time=25.237 ms

■BGP確立連続稼働確認

1日以上連続稼働させてBGPの設定に問題がないことを確認

Router(config)# show ip bgp summary
    BGP router ID 192.168.100.240, local AS number 65000
    4 BGP AS-PATH entries

    Neighbor         V    AS    MsgRcvd MsgSent Up/DownTime   State
    10.254.200.202 4    31898 18268   16511   1d21h48m28s   ESTABLISHED
    10.254.200.206 4    31898 18284   16520   1d21h50m27s   ESTABLISHED

■参考

・マニュアル：https://jpn.nec.com/univerge/ix/Manual/index.html#crm

3
0
0

Register as a new user and use Qiita more conveniently

  1. You get articles that match your needs
  2. You can efficiently read back useful information
  3. You can use dark theme
What you can do with signing up
Sign upLogin
3
0