はじめに
VyOS で ローカルで eBGP で拠点間接続したいなぁ
とりあえず、ESXi上でVyOSを３つたててやってみよ。。。
からのVyOSを使ったeBGP接続のメモ
環境
VyOSによるルータは3つ作成。
ESXi上での、VyOSでのルータ作成は以下の記事を参照。
【2023年8月版】VyOSインストールiso作成から、ESXi7/8へのインストール
【2023年8月版】VyOSを使ったESXi内ルーティング
- vyos03
- スタブAS、natゲートウェイ
- eth0, eth1, eth2
- AS65003
- vyos04
- トランジットAS
- eth0, eth1, eth2
- AS65004
- AS65003を経由して、インターネットアクセス
- vyos05
- スタブAS
- eth0, eth1
- AS65005
- AS65004を経由して、AS65003を経由してインターネットアクセス
各ルータ間にvSwitchを作成して、接続が混ざらないように構築を行った
設定
vyos03
conf
set system host-name vyos03 # ホスト名設定
set interfaces ethernet eth0 address 10.100.39.100/24 # eth0 にIPアドレス設定
set interfaces ethernet eth1 address 10.201.50.254/24 # eth1 にIPアドレス設定
set interfaces ethernet eth2 address 172.16.100.254/24 # eth2 にIPアドレス設定
set protocols static route 0.0.0.0/0 next-hop 10.100.39.254 # デフォルトゲートウェイ設定
set system name-server 8.8.8.8 # DNS設定
set system ipv6 disable # IPv6無効化
set system time-zone Asia/Tokyo # タイムゾーン設定
set service ssh # SSH有効化
set protocols static route 0.0.0.0/0 blackhole #
set protocols static route 172.16.100.0/24 blackhole distance 254 #
set interfaces loopback lo address 10.255.255.3/32 # ループバックインターフェースを設定
set nat source rule 1 outbound-interface eth0 # NATの外側インターフェイスの設定
set nat source rule 1 translation address masquerade # NAPTの設定
set protocols bgp 65003 parameters router-id 10.255.255.3 # BGPのルータID設定
set protocols bgp 65003 address-family ipv4-unicast redistribute connected # 接続されているインターフェイスをBGPでアドバタイズ
set protocols bgp 65003 address-family ipv4-unicast redistribute static # スタティックで設定されているルーティングをアドバタイズ
#set protocols bgp 65003 address-family ipv4-unicast network 10.100.39.0/24
#set protocols bgp 65003 address-family ipv4-unicast network 10.201.50.0/24
#set protocols bgp 65003 address-family ipv4-unicast network 172.16.100.0/24
set protocols bgp 65003 neighbor 10.201.50.253 remote-as 65004 # BGPの相手先ASを設定
#set protocols bgp 65003 neighbor 10.201.50.253 update-source lo
commit
save
vyos04
conf
set system host-name vyos04
set interfaces ethernet eth0 address 10.201.50.253/24
set interfaces ethernet eth1 address 10.202.60.254/24
set interfaces ethernet eth2 address 172.16.110.254/24
set system name-server 8.8.8.8
set system ipv6 disable
set system time-zone Asia/Tokyo
set service ssh
set protocols static route 172.16.110.0/24 blackhole distance 254
set interfaces loopback lo address 10.255.255.4/32
set protocols bgp 65004 parameters router-id 10.255.255.4
set protocols bgp 65004 address-family ipv4-unicast redistribute connected
set protocols bgp 65004 address-family ipv4-unicast redistribute static
#set protocols bgp 65004 address-family ipv4-unicast network 10.201.50.0/24
#set protocols bgp 65004 address-family ipv4-unicast network 10.202.60.0/24
#set protocols bgp 65004 address-family ipv4-unicast network 172.16.110.0/24
set protocols bgp 65004 neighbor 10.201.50.254 remote-as 65003
#set protocols bgp 65004 neighbor 10.201.50.254 update-source lo
#set protocols bgp 65004 neighbor 10.201.50.254 address-family ipv4-unicast route-reflector-client
#set protocols bgp 65004 neighbor 10.201.50.254 address-family ipv4-unicast nexthop-self
set protocols bgp 65004 neighbor 10.202.60.253 remote-as 65005
#set protocols bgp 65004 neighbor 10.202.60.253 update-source lo
#set protocols bgp 65004 neighbor 10.202.60.253 address-family ipv4-unicast route-reflector-client
#set protocols bgp 65004 neighbor 10.202.60.253 address-family ipv4-unicast nexthop-self
commit
save
vyos05
conf
set system host-name vyos05
set interfaces ethernet eth0 address 10.202.60.253/24
set interfaces ethernet eth1 address 172.16.120.254/24
set system name-server 8.8.8.8
set system ipv6 disable
set system time-zone Asia/Tokyo
set service ssh
set protocols static route 172.16.120.0/24 blackhole distance 254
set interfaces loopback lo address 10.255.255.5/32
set protocols bgp 65005 parameters router-id 10.255.255.5
set protocols bgp 65005 address-family ipv4-unicast redistribute connected
set protocols bgp 65005 address-family ipv4-unicast redistribute static
#set protocols bgp 65005 address-family ipv4-unicast network 10.202.60.0/24
#set protocols bgp 65005 address-family ipv4-unicast network 172.16.120.0/24
set protocols bgp 65005 neighbor 10.202.60.254 remote-as 65004
#set protocols bgp 65005 neighbor 10.202.60.254 update-source lo
commit
save
結果
vyos03
$ show ip bgp
BGP table version is 16, local router ID is 10.255.255.3, vrf id 0
Default local pref 100, local AS 65003
Status codes: s suppressed, d damped, h history, * valid, > best, = multipath,
i internal, r RIB-failure, S Stale, R Removed
Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self
Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Weight Path
*> 0.0.0.0/0 10.100.39.254 0 32768 ?
*> 10.100.39.0/24 0.0.0.0 0 32768 ?
* 10.201.50.0/24 10.201.50.253 0 0 65004 ?
*> 0.0.0.0 0 32768 ?
*> 10.202.60.0/24 10.201.50.253 0 0 65004 ?
*> 10.255.255.3/32 0.0.0.0 0 32768 ?
*> 10.255.255.4/32 10.201.50.253 0 0 65004 ?
*> 10.255.255.5/32 10.201.50.253 0 65004 65005 ?
*> 172.16.100.0/24 0.0.0.0 0 32768 ?
*> 172.16.110.0/24 10.201.50.253 0 0 65004 ?
*> 172.16.120.0/24 10.201.50.253 0 65004 65005 ?
Displayed 10 routes and 11 total paths
$
$ ip route
default nhid 10 via 10.100.39.254 dev eth0 proto static metric 20
10.100.39.0/24 dev eth0 proto kernel scope link src 10.100.39.100
10.201.50.0/24 dev eth1 proto kernel scope link src 10.201.50.254
10.202.60.0/24 nhid 30 via 10.201.50.253 dev eth1 proto bgp metric 20
10.255.255.4 nhid 30 via 10.201.50.253 dev eth1 proto bgp metric 20
10.255.255.5 nhid 30 via 10.201.50.253 dev eth1 proto bgp metric 20
172.16.100.0/24 dev eth2 proto kernel scope link src 172.16.100.254
172.16.110.0/24 nhid 30 via 10.201.50.253 dev eth1 proto bgp metric 20
172.16.120.0/24 nhid 30 via 10.201.50.253 dev eth1 proto bgp metric 20
$
$ /usr/bin/ping -nc 3 172.16.110.254
PING 172.16.110.254 (172.16.110.254) 56(84) bytes of data.
64 bytes from 172.16.110.254: icmp_seq=1 ttl=64 time=0.163 ms
64 bytes from 172.16.110.254: icmp_seq=2 ttl=64 time=0.157 ms
64 bytes from 172.16.110.254: icmp_seq=3 ttl=64 time=0.181 ms
--- 172.16.110.254 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 78ms
rtt min/avg/max/mdev = 0.157/0.167/0.181/0.010 ms
$
$ /usr/bin/traceroute 172.16.110.254
traceroute to 172.16.110.254 (172.16.110.254), 30 hops max, 60 byte packets
1 172.16.110.254 (172.16.110.254) 0.234 ms 0.195 ms 0.166 ms
$
$ /usr/bin/ping -nc 3 172.16.120.254
PING 172.16.120.254 (172.16.120.254) 56(84) bytes of data.
64 bytes from 172.16.120.254: icmp_seq=1 ttl=63 time=0.253 ms
64 bytes from 172.16.120.254: icmp_seq=2 ttl=63 time=0.252 ms
64 bytes from 172.16.120.254: icmp_seq=3 ttl=63 time=0.268 ms
--- 172.16.120.254 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 85ms
rtt min/avg/max/mdev = 0.252/0.257/0.268/0.019 ms$
$
$ /usr/bin/traceroute 172.16.120.254
traceroute to 172.16.120.254 (172.16.120.254), 30 hops max, 60 byte packets
1 10.201.50.253 (10.201.50.253) 0.203 ms 0.158 ms 0.114 ms
2 172.16.120.254 (172.16.120.254) 0.294 ms 0.283 ms 0.258 ms
$
$ /usr/bin/ping -nc 3 www.google.com
PING www.google.com (172.217.25.164) 56(84) bytes of data.
64 bytes from 172.217.25.164: icmp_seq=1 ttl=114 time=3.23 ms
64 bytes from 172.217.25.164: icmp_seq=2 ttl=114 time=3.27 ms
64 bytes from 172.217.25.164: icmp_seq=3 ttl=114 time=3.48 ms
$
$ /usr/bin/traceroute -n www.google.com
traceroute to www.google.com (172.217.25.164), 30 hops max, 60 byte packets
1 10.100.39.254 0.077 ms 0.045 ms 0.055 ms
2 * * *
:
:
11 108.170.243.106 4.446 ms 72.14.238.23 3.242 ms 172.217.25.164 3.255 ms
vyos04
$ show ip bgp
BGP table version is 24, local router ID is 10.255.255.4, vrf id 0
Default local pref 100, local AS 65004
Status codes: s suppressed, d damped, h history, * valid, > best, = multipath,
i internal, r RIB-failure, S Stale, R Removed
Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self
Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Weight Path
*> 0.0.0.0/0 10.201.50.254 0 0 65003 ?
*> 10.100.39.0/24 10.201.50.254 0 0 65003 ?
*> 10.201.50.0/24 0.0.0.0 0 32768 ?
* 10.201.50.254 0 0 65003 ?
* 10.202.60.0/24 10.202.60.253 0 0 65005 ?
*> 0.0.0.0 0 32768 ?
*> 10.255.255.3/32 10.201.50.254 0 0 65003 ?
*> 10.255.255.4/32 0.0.0.0 0 32768 ?
*> 10.255.255.5/32 10.202.60.253 0 0 65005 ?
*> 172.16.100.0/24 10.201.50.254 0 0 65003 ?
*> 172.16.110.0/24 0.0.0.0 0 32768 ?
*> 172.16.120.0/24 10.202.60.253 0 0 65005 ?
Displayed 10 routes and 12 total paths
$
$ ip route
default nhid 41 via 10.201.50.254 dev eth0 proto bgp metric 20
10.100.39.0/24 nhid 41 via 10.201.50.254 dev eth0 proto bgp metric 20
10.201.50.0/24 dev eth0 proto kernel scope link src 10.201.50.253
10.202.60.0/24 dev eth1 proto kernel scope link src 10.202.60.254
10.255.255.3 nhid 41 via 10.201.50.254 dev eth0 proto bgp metric 20
10.255.255.5 nhid 45 via 10.202.60.253 dev eth1 proto bgp metric 20
172.16.100.0/24 nhid 41 via 10.201.50.254 dev eth0 proto bgp metric 20
172.16.110.0/24 dev eth2 proto kernel scope link src 172.16.110.254
172.16.120.0/24 nhid 45 via 10.202.60.253 dev eth1 proto bgp metric 20
$
$ /usr/bin/ping -nc 3 172.16.100.254
PING 172.16.100.254 (172.16.100.254) 56(84) bytes of data.
64 bytes from 172.16.100.254: icmp_seq=1 ttl=64 time=0.117 ms
64 bytes from 172.16.100.254: icmp_seq=2 ttl=64 time=0.224 ms
64 bytes from 172.16.100.254: icmp_seq=3 ttl=64 time=0.193 ms
--- 172.16.100.254 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 32ms
rtt min/avg/max/mdev = 0.117/0.178/0.224/0.044 ms
$
$ /usr/bin/traceroute 172.16.100.254
traceroute to 172.16.100.254 (172.16.100.254), 30 hops max, 60 byte packets
1 172.16.100.254 (172.16.100.254) 0.063 ms 0.041 ms 0.039 ms
$
$ /usr/bin/ping -nc 3 172.16.120.254
PING 172.16.120.254 (172.16.120.254) 56(84) bytes of data.
64 bytes from 172.16.120.254: icmp_seq=1 ttl=64 time=0.138 ms
64 bytes from 172.16.120.254: icmp_seq=2 ttl=64 time=0.161 ms
64 bytes from 172.16.120.254: icmp_seq=3 ttl=64 time=0.252 ms
--- 172.16.120.254 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 74ms
rtt min/avg/max/mdev = 0.138/0.183/0.252/0.051 ms
$
$ /usr/bin/traceroute 172.16.120.254
traceroute to 172.16.120.254 (172.16.120.254), 30 hops max, 60 byte packets
1 172.16.120.254 (172.16.120.254) 0.153 ms 0.144 ms 0.125 ms
$
$ /usr/bin/ping -nc 3 www.google.com
PING www.google.com (172.217.25.164) 56(84) bytes of data.
64 bytes from 172.217.25.164: icmp_seq=1 ttl=113 time=3.08 ms
64 bytes from 172.217.25.164: icmp_seq=2 ttl=113 time=4.06 ms
64 bytes from 172.217.25.164: icmp_seq=3 ttl=113 time=3.55 ms
--- www.google.com ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 5ms
rtt min/avg/max/mdev = 3.075/3.563/4.063/0.406 ms
$
$ /usr/bin/traceroute -n www.google.com
traceroute to www.google.com (172.217.25.164), 30 hops max, 60 byte packets
1 10.201.50.254 0.046 ms 0.042 ms 0.055 ms
2 10.100.39.254 0.105 ms 0.137 ms 0.128 ms
:
:
11 108.170.243.97 4.390 ms 4.463 ms 172.217.25.164 3.350 ms
vyos05
$ show ip bgp
BGP table version is 25, local router ID is 10.255.255.5, vrf id 0
Default local pref 100, local AS 65005
Status codes: s suppressed, d damped, h history, * valid, > best, = multipath,
i internal, r RIB-failure, S Stale, R Removed
Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self
Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Weight Path
*> 0.0.0.0/0 10.202.60.254 0 65004 65003 ?
*> 10.100.39.0/24 10.202.60.254 0 65004 65003 ?
*> 10.201.50.0/24 10.202.60.254 0 0 65004 ?
*> 10.202.60.0/24 0.0.0.0 0 32768 ?
* 10.202.60.254 0 0 65004 ?
*> 10.255.255.3/32 10.202.60.254 0 65004 65003 ?
*> 10.255.255.4/32 10.202.60.254 0 0 65004 ?
*> 10.255.255.5/32 0.0.0.0 0 32768 ?
*> 172.16.100.0/24 10.202.60.254 0 65004 65003 ?
*> 172.16.110.0/24 10.202.60.254 0 0 65004 ?
*> 172.16.120.0/24 0.0.0.0 0 32768 ?
Displayed 10 routes and 11 total paths
$
$ ip route
default nhid 22 via 10.202.60.254 dev eth0 proto bgp metric 20
10.100.39.0/24 nhid 22 via 10.202.60.254 dev eth0 proto bgp metric 20
10.201.50.0/24 nhid 22 via 10.202.60.254 dev eth0 proto bgp metric 20
10.202.60.0/24 dev eth0 proto kernel scope link src 10.202.60.253
10.255.255.3 nhid 22 via 10.202.60.254 dev eth0 proto bgp metric 20
10.255.255.4 nhid 22 via 10.202.60.254 dev eth0 proto bgp metric 20
172.16.100.0/24 nhid 22 via 10.202.60.254 dev eth0 proto bgp metric 20
172.16.110.0/24 nhid 22 via 10.202.60.254 dev eth0 proto bgp metric 20
172.16.120.0/24 dev eth1 proto kernel scope link src 172.16.120.254
$
$ /usr/bin/ping -nc 3 172.16.100.254
PING 172.16.100.254 (172.16.100.254) 56(84) bytes of data.
64 bytes from 172.16.100.254: icmp_seq=1 ttl=63 time=0.270 ms
64 bytes from 172.16.100.254: icmp_seq=2 ttl=63 time=0.282 ms
64 bytes from 172.16.100.254: icmp_seq=3 ttl=63 time=0.368 ms
--- 172.16.100.254 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 52ms
rtt min/avg/max/mdev = 0.270/0.306/0.368/0.048 ms
$
$ /usr/bin/traceroute 172.16.100.254
traceroute to 172.16.100.254 (172.16.100.254), 30 hops max, 60 byte packets
1 10.202.60.254 (10.202.60.254) 0.122 ms 0.119 ms 0.110 ms
2 172.16.100.254 (172.16.100.254) 0.312 ms 0.331 ms 0.314 ms
$
$ /usr/bin/ping -nc 3 172.16.110.254
PING 172.16.110.254 (172.16.110.254) 56(84) bytes of data.
64 bytes from 172.16.110.254: icmp_seq=1 ttl=64 time=0.088 ms
64 bytes from 172.16.110.254: icmp_seq=2 ttl=64 time=0.216 ms
64 bytes from 172.16.110.254: icmp_seq=3 ttl=64 time=0.202 ms
--- 172.16.110.254 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 69ms
rtt min/avg/max/mdev = 0.088/0.168/0.216/0.059 ms
$
$ /usr/bin/traceroute 172.16.110.254
traceroute to 172.16.110.254 (172.16.110.254), 30 hops max, 60 byte packets
1 172.16.110.254 (172.16.110.254) 0.054 ms 0.043 ms 0.027 ms
$
$ /usr/bin/ping -nc 3 www.google.com
PING www.google.com (172.217.25.164) 56(84) bytes of data.
64 bytes from 172.217.25.164: icmp_seq=1 ttl=112 time=3.43 ms
64 bytes from 172.217.25.164: icmp_seq=2 ttl=112 time=3.78 ms
64 bytes from 172.217.25.164: icmp_seq=3 ttl=112 time=3.72 ms
--- www.google.com ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 6ms
rtt min/avg/max/mdev = 3.432/3.644/3.778/0.167 ms
$
$ /usr/bin/traceroute -n www.google.com
traceroute to www.google.com (172.217.25.164), 30 hops max, 60 byte packets
1 10.202.60.254 0.101 ms 0.066 ms 0.046 ms
2 10.201.50.254 0.115 ms 0.109 ms 0.099 ms
3 10.100.39.254 0.184 ms 0.174 ms 0.188 ms
:
:
13 108.170.243.106 5.063 ms 172.217.25.164 4.339 ms 108.170.243.139 3.449 ms
うまくいかないときは
- だいたいの場合が設定の打ち間違いとかなので、
show confとかで設定を出力して確認する - show ip bgp summary で 接続されているか確認する、
never connectedは繋がってない。 - show ip bgp で経路が valid になっているか確認する (valid = *、例：
*> 10.255.255.3/32) - ip route でも経路が存在するか確認する
- tcpdump -n icmp で ping がどこまで飛んでいるのか、戻っているのかを確認する
やりなおしたいときは
delete protocols bgp で設定削除すればやり直せる
感想
最初は、iBGPでやろうとしたけど、フルメッシュが面倒だったのと、
フルメッシュをやめたら、route-refrect や nexthop-self を使っても AS間のルーティングがうまくいかなかったのでやめました。。。
さいごに
かんたんでしたね