やりたいこと
- データセンター間をIPsec VPNで接続
- 各データセンターでは、デフォルトで割り当てられるPrimary IPサブネットに加えて、独自のIPアドレスのサブネットを作成(BYOIPサブネット)
- BYOIPサブネットのルーティング情報を、データセンター間でBGPで交換する
# IPアドレスはサンプルです。
途中経過
vyatta#1側で、vyatta#2から192.168.2.0/24のルートを受けっとってはいるが、ルーティングには出てこない状況。
(vyatta#1)$ show ip route
Codes: K - kernel, C - connected, S - static, R - RIP, B - BGP
O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area
> - selected route, * - FIB route, p - stale info
Gateway of last resort is 1.1.1.14 to network 0.0.0.0
S *> 0.0.0.0/0 [1/0] via 1.1.1.14, bond1
S *> 10.0.0.0/8 [1/0] via 10.140.24.1, bond0
C * 10.140.24.0/26 is directly connected, bond0v1
C *> 10.140.24.0/26 is directly connected, bond0
C *> 127.0.0.0/8 is directly connected, lo
C * 1.1.1.0/28 is directly connected, bond1v1
C *> 1.1.1.0/28 is directly connected, bond1
C *> 192.168.1.0/24 is directly connected, bond0
C *> 192.168.10.0/30 is directly connected, vti0
(vyatta#1)$ show ip bgp neighbors 192.168.10.2 received-routes
BGP table version is 534, local router ID is 192.168.10.1
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Weight Path
*> 192.168.2.0 192.168.10.2 1 0 65002 i
Total number of prefixes 1
同じ症状の事例
- networking - Simple Vyatta configuration / BGP routes not injecting into routing table - Server Fault
- BGP route is not installed (IPSec + BGP) - Brocade Community Forums - 58130
Vyatta config
追加部分のconfigのみ記載
Vyatta #1
# IPsec configuration
## IKE/ESP policy
set vpn ipsec esp-group ESP1 lifetime 3600
set vpn ipsec esp-group ESP1 proposal 1 encryption aes256
set vpn ipsec esp-group ESP1 proposal 1 hash sha1
set vpn ipsec ike-group IKE1 lifetime 28800
set vpn ipsec ike-group IKE1 proposal 1 dh-group 5
set vpn ipsec ike-group IKE1 proposal 1 encryption aes256
set vpn ipsec ike-group IKE1 proposal 1 hash sha1
set vpn ipsec ike-group IKE1 dead-peer-detection action restart
set vpn ipsec ike-group IKE1 dead-peer-detection interval 15
set vpn ipsec ike-group IKE1 dead-peer-detection timeout 30
set vpn ipsec ipsec-interfaces interface bond1
set vpn ipsec nat-traversal enable
## IPsec tunnel configuration
set vpn ipsec site-to-site peer 2.2.2.2 authentication pre-shared-secret !secret!
set vpn ipsec site-to-site peer 2.2.2.2 ike-group IKE1
set vpn ipsec site-to-site peer 2.2.2.2 local-address 1.1.1.1
set vpn ipsec site-to-site peer 2.2.2.2 vti bind vti0
set vpn ipsec site-to-site peer 2.2.2.2 vti esp-group ESP1
## Virtual tunnel interface configuration
set interfaces vti vti0 address 192.168.10.1/30
set interfaces vti vti0 mtu 1436
# BYOIPサブネット用のGWアドレス
set interface bonding bond0 address 192.168.1.1/24
# BGP configuration
## BGP peering configuration
set protocols bgp 65001 parameters router-id 192.168.10.1
set protocols bgp 65001 neighbor 192.168.10.2 remote-as 65002
set protocols bgp 65001 neighbor 192.168.10.2 soft-reconfiguration 'inbound'
set protocols bgp 65001 neighbor 192.168.10.2 timers holdtime '30'
set protocols bgp 65001 neighbor 192.168.10.2 timers keepalive '30'
## Originating network configuration
set protocols bgp 65001 network 192.168.1.0/24
Vyatta #2
# IPsec configuration
## IKE/ESP policy
set vpn ipsec esp-group ESP1 lifetime 3600
set vpn ipsec esp-group ESP1 proposal 1 encryption aes256
set vpn ipsec esp-group ESP1 proposal 1 hash sha1
set vpn ipsec ike-group IKE1 lifetime 28800
set vpn ipsec ike-group IKE1 proposal 1 dh-group 5
set vpn ipsec ike-group IKE1 proposal 1 encryption aes256
set vpn ipsec ike-group IKE1 proposal 1 hash sha1
set vpn ipsec ike-group IKE1 dead-peer-detection action restart
set vpn ipsec ike-group IKE1 dead-peer-detection interval 15
set vpn ipsec ike-group IKE1 dead-peer-detection timeout 30
set vpn ipsec ipsec-interfaces interface eth1
set vpn ipsec nat-traversal enable
## IPsec tunnel configuration
set vpn ipsec site-to-site peer 1.1.1.1 authentication pre-shared-secret !secret!
set vpn ipsec site-to-site peer 1.1.1.1 ike-group IKE1
set vpn ipsec site-to-site peer 1.1.1.1 local-address 2.2.2.2
set vpn ipsec site-to-site peer 1.1.1.1 vti bind vti0
set vpn ipsec site-to-site peer 1.1.1.1 vti esp-group ESP1
## Virtual tunnel interface configuration
set interfaces vti vti0 address 192.168.10.2/30
set interfaces vti vti0 mtu 1436
# BYOIPサブネット用のGWアドレス
set interface ethernet eth0 address 192.168.2.1/24
# BGP configuration
## BGP peering configuration
set protocols bgp 65002 parameters router-id 192.168.10.2
set protocols bgp 65002 neighbor 192.168.10.1 remote-as 65001
set protocols bgp 65002 neighbor 192.168.10.1 soft-reconfiguration 'inbound'
set protocols bgp 65002 neighbor 192.168.10.1 timers holdtime '30'
set protocols bgp 65002 neighbor 192.168.10.1 timers keepalive '30'
## Originating network configuration
set protocols bgp 65002 network 192.168.2.0/24
確認
ルーティング
show ip route
(vyatta#1)$ show ip route
Codes: K - kernel, C - connected, S - static, R - RIP, B - BGP
O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area
> - selected route, * - FIB route, p - stale info
Gateway of last resort is 1.1.1.14 to network 0.0.0.0
S *> 0.0.0.0/0 [1/0] via 1.1.1.14, bond1
S *> 10.0.0.0/8 [1/0] via 10.140.24.1, bond0
C * 10.140.24.0/26 is directly connected, bond0v1
C *> 10.140.24.0/26 is directly connected, bond0
C *> 127.0.0.0/8 is directly connected, lo
C * 1.1.1.0/28 is directly connected, bond1v1
C *> 1.1.1.0/28 is directly connected, bond1
C *> 192.168.1.0/24 is directly connected, bond0
C *> 192.168.10.0/30 is directly connected, vti0
(vyatta#2)$ show ip route
Codes: K - kernel route, C - connected, S - static, R - RIP, O - OSPF,
I - ISIS, B - BGP, > - selected route, * - FIB route
S>* 0.0.0.0/0 [1/0] via 2.2.2.6, eth1
S>* 10.0.0.0/8 [1/0] via 10.64.234.65, eth0
C>* 10.64.234.64/26 is directly connected, eth0
C>* 2.2.2.0/29 is directly connected, eth1
C>* 127.0.0.0/8 is directly connected, lo
B>* 192.168.1.0/24 [20/0] via 192.168.10.1, vti0, 00:09:00
C>* 192.168.2.0/24 is directly connected, eth0
C>* 192.168.10.0/30 is directly connected, vti0
Neighbor情報
show ip bgp neighbors
(vyatta#1)$ show ip bgp neighbors
BGP neighbor is 192.168.10.2, remote AS 65002, local AS 65001, external link
BGP version 4, remote router ID 192.168.10.2
BGP state = Established, up for 03:23:54
Last read 03:23:54, hold time is 30, keepalive interval is 10 seconds
Configured hold time is 30, keepalive interval is 30 seconds
Neighbor capabilities:
Route refresh: advertised and received (old and new)
Address family IPv4 Unicast: advertised and received
Received 3257 messages, 2 notifications, 0 in queue
Sent 3371 messages, 71 notifications, 0 in queue
Route refresh request: received 0, sent 0
Minimum time between advertisement runs is 30 seconds
For address family: IPv4 Unicast
BGP table version 532, neighbor version 532
Index 1, Offset 0, Mask 0x2
Inbound soft reconfiguration allowed
Community attribute sent to this neighbor (both)
0 accepted prefixes
1 announced prefixes
Connections established 74; dropped 73
Local host: 192.168.10.1, Local port: 179
Foreign host: 192.168.10.2, Foreign port: 58991
Nexthop: 192.168.10.1
Nexthop global: ::
Nexthop local: ::
BGP connection: non shared network
Last Reset: 03:23:59, due to BGP Notification received
Notification Error Message: (Cease/Other Configuration Change.)
(vyatta#2)$ show ip bgp neighbors
BGP neighbor is 192.168.10.1, remote AS 65001, local AS 65002, external link
BGP version 4, remote router ID 192.168.10.1
BGP state = Established, up for 03:24:17
Last read 14:40:31, hold time is 30, keepalive interval is 10 seconds
Configured hold time is 30, keepalive interval is 10 seconds
Neighbor capabilities:
4 Byte AS: advertised
Route refresh: advertised and received(old & new)
Address family IPv4 Unicast: advertised and received
Message statistics:
Inq depth is 0
Outq depth is 0
Sent Rcvd
Opens: 74 73
Notifications: 5 68
Updates: 142 77
Keepalives: 3251 3080
Route Refresh: 0 0
Capability: 0 0
Total: 3472 3298
Minimum time between advertisement runs is 30 seconds
For address family: IPv4 Unicast
Inbound soft reconfiguration allowed
Community attribute sent to this neighbor(both)
1 accepted prefixes
Connections established 74; dropped 73
Last reset 03:24:28, due to Update source change
Local host: 192.168.10.2, Local port: 58991
Foreign host: 192.168.10.1, Foreign port: 179
Nexthop: 192.168.10.2
Nexthop global: ::
Nexthop local: ::
BGP connection: non shared network
Read thread: on Write thread: off
Advertiseしたルート情報
show ip bgp neighbors <neighbor's IP> advertised-routes
(vyatta#1)$ show ip bgp neighbors 192.168.10.2 advertised-routes
BGP table version is 536, local router ID is 192.168.10.1
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Weight Path
*> 192.168.1.0 192.168.10.1 100 32768 i
Total number of prefixes 1
(vyatta#2)$ show ip bgp neighbors 192.168.10.1 advertised-routes
BGP table version is 0, local router ID is 192.168.10.2
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
r RIB-failure, S Stale, R Removed
Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Weight Path
*> 192.168.2.0 192.168.10.2 1 32768 i
Total number of prefixes 1
受信したルート情報
show ip bgp neighbors <neighbor's IP> received-routes
(vyatta#1)$ show ip bgp neighbors 192.168.10.2 received-routes
BGP table version is 534, local router ID is 192.168.10.1
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Weight Path
*> 192.168.2.0 192.168.10.2 1 0 65002 i
Total number of prefixes 1
(vyatta#2)$ show ip bgp neighbors 192.168.10.1 received-routes
BGP table version is 0, local router ID is 192.168.10.2
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
r RIB-failure, S Stale, R Removed
Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Weight Path
*> 192.168.1.0 192.168.10.1 0 65001 i
Total number of prefixes 1