やりたいこと
vxlan/evpnはコントローラー無しでoverlayネットワークを実装できる利点
EVPN asymmetric vs symmetric model比較の前に、vxlan/evpnの基本動作、トラフィックフロー確認など
基礎知識も踏まえてroute type 2/3の確認
multi-homing機能を前提としたroute type 1/4はスコープ外とする
なので、MLAGを用いたVXLAN Active-active Modeを使用
route type-5は次会で、、、
環境
- GNS3 2.1.15
- GNS3 VM (6vCPUs、6G RAM)
- Cumulus VX 3.7.3
- Ubuntu Docker Guest
ToRはOVSで代用可能(lacpとvlan喋ればなんでもいい)
Ubuntu Docker GuestもvPCで代用可能(IP設定してping送れば)
- underlay layer3
各ベンダーの例はほとんどがBGP unnumberedで設定しているが、運用上わかりやすく /30アドレスで設定
環境に合わせてfast-external-failoverとBFDを適用してもいいかも
各loopbackアドレスを広報するだけ、
underlay設定
初期ログインは**cumulus/CumulusLinux!**でログイン
rootアカウントのパスワード変更
cumulus全台
cumulus@cumulus:~$ sudo passwd root
[sudo] password for cumulus:
Enter new UNIX password:
Retype new UNIX password:
passwd: password updated successfully
cumulus@cumulus:~$
rootアカウントでログイン後underlayから設定していく
BB01
#hostname/各IFの設定
net add hostname BB01
net add interface swp1 alias to-spine01-swp1
net add interface swp1 mtu 9000
net add interface swp1 ip address 10.10.10.1/30
net add interface swp2 alias to-spine02-swp1
net add interface swp2 mtu 9000
net add interface swp2 ip address 10.10.10.5/30
net add interface swp3 alias to-spine03-swp1
net add interface swp3 mtu 9000
net add interface swp3 ip address 10.10.10.9/30
net add interface swp4 alias to-spine04-swp1
net add interface swp4 mtu 9000
net add interface swp4 ip address 10.10.10.13/30
net add loopback lo ip address 172.16.222.1/32
#underlay-bgp設定
net add bgp autonomous-system 65000
net add bgp router-id 172.16.222.1
net add bgp redistribute connected
net add bgp bestpath as-path multipath-relax
net add bgp neighbor PEER_SPINE peer-group
net add bgp neighbor 10.10.10.2 remote-as 65001
net add bgp neighbor 10.10.10.2 description spine01
net add bgp neighbor 10.10.10.2 peer-group PEER_SPINE
net add bgp neighbor 10.10.10.6 remote-as 65001
net add bgp neighbor 10.10.10.6 description spine02
net add bgp neighbor 10.10.10.6 peer-group PEER_SPINE
net add bgp neighbor 10.10.10.10 remote-as 65002
net add bgp neighbor 10.10.10.10 description spine03
net add bgp neighbor 10.10.10.10 peer-group PEER_SPINE
net add bgp neighbor 10.10.10.14 remote-as 65002
net add bgp neighbor 10.10.10.14 description spine04
net add bgp neighbor 10.10.10.14 peer-group PEER_SPINE
BB02
#hostname/各IFの設定
net add hostname BB02
net add interface swp1 alias to-spine01-swp2
net add interface swp1 mtu 9000
net add interface swp1 ip address 10.10.10.17/30
net add interface swp2 alias to-spine02-swp2
net add interface swp2 mtu 9000
net add interface swp2 ip address 10.10.10.21/30
net add interface swp3 alias to-spine03-swp2
net add interface swp3 mtu 9000
net add interface swp3 ip address 10.10.10.25/30
net add interface swp4 alias to-spine04-swp2
net add interface swp4 mtu 9000
net add interface swp4 ip address 10.10.10.29/30
net add loopback lo ip address 172.16.222.2/32
#underlay-bgp設定
net add bgp autonomous-system 65000
net add bgp router-id 172.16.222.2
net add bgp redistribute connected
net add bgp bestpath as-path multipath-relax
net add bgp neighbor PEER_SPINE peer-group
net add bgp neighbor 10.10.10.18 remote-as 65001
net add bgp neighbor 10.10.10.18 description spine01
net add bgp neighbor 10.10.10.18 peer-group PEER_SPINE
net add bgp neighbor 10.10.10.22 remote-as 65001
net add bgp neighbor 10.10.10.22 description spine02
net add bgp neighbor 10.10.10.22 peer-group PEER_SPINE
net add bgp neighbor 10.10.10.26 remote-as 65002
net add bgp neighbor 10.10.10.26 description spine03
net add bgp neighbor 10.10.10.26 peer-group PEER_SPINE
net add bgp neighbor 10.10.10.30 remote-as 65002
net add bgp neighbor 10.10.10.30 description spine04
net add bgp neighbor 10.10.10.30 peer-group PEER_SPINE
spine01
#hostname/各IFの設定
net add hostname spine01
net add interface swp1 alias to-BB01-swp1
net add interface swp1 mtu 9000
net add interface swp1 ip address 10.10.10.2/30
net add interface swp2 alias to-BB02-swp1
net add interface swp2 mtu 9000
net add interface swp2 ip address 10.10.10.18/30
net add loopback lo ip address 172.16.1.2/32
#underlay-bgp設定
net add bgp autonomous-system 65001
net add bgp router-id 172.16.1.2
net add bgp redistribute connected
net add bgp bestpath as-path multipath-relax
net add bgp neighbor PEER_BB peer-group
net add bgp neighbor 10.10.10.1 remote-as 65000
net add bgp neighbor 10.10.10.1 description BB01
net add bgp neighbor 10.10.10.1 peer-group PEER_BB
net add bgp neighbor 10.10.10.17 remote-as 65000
net add bgp neighbor 10.10.10.17 description BB02
net add bgp neighbor 10.10.10.17 peer-group PEER_BB
spine02
#hostname/各IFの設定
net add hostname spine02
net add interface swp1 alias to-BB01-swp2
net add interface swp1 mtu 9000
net add interface swp1 ip address 10.10.10.6/30
net add interface swp2 alias to-BB02-swp2
net add interface swp2 mtu 9000
net add interface swp2 ip address 10.10.10.22/30
net add loopback lo ip address 172.16.1.3/32
#underlay-bgp設定
net add bgp autonomous-system 65001
net add bgp router-id 172.16.1.3
net add bgp redistribute connected
net add bgp bestpath as-path multipath-relax
net add bgp neighbor PEER_BB peer-group
net add bgp neighbor 10.10.10.5 remote-as 65000
net add bgp neighbor 10.10.10.5 description BB01
net add bgp neighbor 10.10.10.5 peer-group PEER_BB
net add bgp neighbor 10.10.10.21 remote-as 65000
net add bgp neighbor 10.10.10.21 description BB02
net add bgp neighbor 10.10.10.21 peer-group PEER_BB
spine03
#hostname/各IFの設定
net add hostname spine03
net add interface swp1 alias to-BB01-swp3
net add interface swp1 mtu 9000
net add interface swp1 ip address 10.10.10.10/30
net add interface swp2 alias to-BB02-swp3
net add interface swp2 mtu 9000
net add interface swp2 ip address 10.10.10.26/30
net add loopback lo ip address 172.16.1.5/32
#underlay-bgp設定
net add bgp autonomous-system 65002
net add bgp router-id 172.16.1.5
net add bgp redistribute connected
net add bgp bestpath as-path multipath-relax
net add bgp neighbor PEER_BB peer-group
net add bgp neighbor 10.10.10.9 remote-as 65000
net add bgp neighbor 10.10.10.9 description BB01
net add bgp neighbor 10.10.10.9 peer-group PEER_BB
net add bgp neighbor 10.10.10.25 remote-as 65000
net add bgp neighbor 10.10.10.25 description BB02
net add bgp neighbor 10.10.10.25 peer-group PEER_BB
spine04
#hostname/各IFの設定
net add hostname spine04
net add interface swp1 alias to-BB01-swp4
net add interface swp1 mtu 9000
net add interface swp1 ip address 10.10.10.14/30
net add interface swp2 alias to-BB02-swp4
net add interface swp2 mtu 9000
net add interface swp2 ip address 10.10.10.30/30
net add loopback lo ip address 172.16.1.6/32
#underlay-bgp設定
net add bgp autonomous-system 65002
net add bgp router-id 172.16.1.6
net add bgp redistribute connected
net add bgp bestpath as-path multipath-relax
net add bgp neighbor PEER_BB peer-group
net add bgp neighbor 10.10.10.13 remote-as 65000
net add bgp neighbor 10.10.10.13 description BB01
net add bgp neighbor 10.10.10.13 peer-group PEER_BB
net add bgp neighbor 10.10.10.29 remote-as 65000
net add bgp neighbor 10.10.10.29 description BB02
net add bgp neighbor 10.10.10.29 peer-group PEER_BB
以下のコマンドで確認後、commit
net pending
net commit
続いて、spineのMLAG設定
peer IPアドレスなど手動で設定できるが、今回は自動作成で
spine01
net add clag peer sys-mac 44:38:39:FF:00:00 interface swp3-4 primary
net add vlan 100,200
net add clag port bond bond-to-ToR01 interface swp5 clag-id 1
net add bond bond-to-ToR01 bridge trunk vlans 100,200
net add bond bond-to-ToR01 mtu 9000
#仮想VTEPのIPアドレス設定
net add loopback lo clag vxlan-anycast-ip 172.16.1.1
spine02
net add clag peer sys-mac 44:38:39:FF:00:00 interface swp3-4 secondary
net add vlan 100,200
net add clag port bond bond-to-ToR01 interface swp5 clag-id 1
net add bond bond-to-ToR01 bridge trunk vlans 100,200
net add bond bond-to-ToR01 mtu 9000
#仮想VTEPのIPアドレス設定
net add loopback lo clag vxlan-anycast-ip 172.16.1.1
spine03
net add clag peer sys-mac 44:38:39:FF:00:01 interface swp3-4 primary
net add vlan 100,200
net add clag port bond bond-to-ToR02 interface swp5 clag-id 1
net add bond bond-to-ToR02 bridge trunk vlans 100,200
net add bond bond-to-ToR02 mtu 9000
#仮想VTEPのIPアドレス設定
net add loopback lo clag vxlan-anycast-ip 172.16.1.4
spine04
net add clag peer sys-mac 44:38:39:FF:00:01 interface swp3-4 secondary
net add vlan 100,200
net add clag port bond bond-to-ToR02 interface swp5 clag-id 1
net add bond bond-to-ToR02 bridge trunk vlans 100,200
net add bond bond-to-ToR02 mtu 9000
#仮想VTEPのIPアドレス設定
net add loopback lo clag vxlan-anycast-ip 172.16.1.4
以下のコマンドで確認後、commit
net pending
net commit
続いて、ToRのlacp/vlan設定
OVSでも代用可能なので、使いやすい方で設定すればいいかなーと
ToR01
net add hostname ToR01
net add bond bond-to-SPINE bond slaves swp1-2
net add bond bond-to-SPINE bridge trunk vlans 100,200
net add bond bond-to-SPINE mtu 9000
net add interface swp3 bridge access 100
net add interface swp4 bridge access 200
ToR02
net add hostname ToR02
net add bond bond-to-SPINE bond slaves swp1-2
net add bond bond-to-SPINE bridge trunk vlans 100,200
net add bond bond-to-SPINE mtu 9000
net add interface swp3 bridge access 100
net add interface swp4 bridge access 200
各機器にてルーティングテーブル、IF、mlag状態確認
※spineたちのmlag用IPアドレスが見えちゃうが、気になるならip prefix-listで広報するアドレスを制御
BB01
root@BB01:~# net show route
show ip route
=============
Codes: K - kernel route, C - connected, S - static, R - RIP,
O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
F - PBR,
> - selected route, * - FIB route
C>* 10.10.10.0/30 is directly connected, swp1, 00:04:14
C>* 10.10.10.4/30 is directly connected, swp2, 00:04:14
C>* 10.10.10.8/30 is directly connected, swp3, 00:04:14
C>* 10.10.10.12/30 is directly connected, swp4, 00:04:14
B>* 10.10.10.16/30 [20/0] via 10.10.10.2, swp1, 00:04:03
B>* 10.10.10.20/30 [20/0] via 10.10.10.6, swp2, 00:03:58
B>* 10.10.10.24/30 [20/0] via 10.10.10.10, swp3, 00:04:05
B>* 10.10.10.28/30 [20/0] via 10.10.10.14, swp4, 00:04:03
B>* 169.254.1.0/30 [20/0] via 10.10.10.10, swp3, 00:03:58
* via 10.10.10.2, swp1, 00:03:58
* via 10.10.10.6, swp2, 00:03:58
* via 10.10.10.14, swp4, 00:03:58
B>* 172.16.1.1/32 [20/0] via 10.10.10.2, swp1, 00:02:23
B>* 172.16.1.2/32 [20/0] via 10.10.10.2, swp1, 00:04:03
B>* 172.16.1.3/32 [20/0] via 10.10.10.6, swp2, 00:03:58
B>* 172.16.1.4/32 [20/0] via 10.10.10.10, swp3, 00:02:23
B>* 172.16.1.5/32 [20/0] via 10.10.10.10, swp3, 00:04:05
B>* 172.16.1.6/32 [20/0] via 10.10.10.14, swp4, 00:04:03
C>* 172.16.222.1/32 is directly connected, lo, 00:04:14
show ipv6 route
===============
Codes: K - kernel route, C - connected, S - static, R - RIPng,
O - OSPFv3, I - IS-IS, B - BGP, N - NHRP, T - Table,
v - VNC, V - VNC-Direct, A - Babel, D - SHARP, F - PBR,
> - selected route, * - FIB route
C * fe80::/64 is directly connected, swp4, 00:04:14
C * fe80::/64 is directly connected, swp3, 00:04:14
C * fe80::/64 is directly connected, swp2, 00:04:14
C * fe80::/64 is directly connected, swp1, 00:04:14
C>* fe80::/64 is directly connected, eth0, 00:04:14
root@BB01:~# net show interface all mac
State Name Spd MAC MTU Mode LLDP Summary
----- ---- --- ----------------- ----- ------------- -------------- -------------------
UP lo N/A 00:00:00:00:00:00 65536 Loopback IP: 127.0.0.1/8
lo IP: 172.16.222.1/32
lo IP: ::1/128
UP eth0 1G 0c:1e:ed:fe:04:00 1500 Mgmt
UP swp1 1G 0c:1e:ed:fe:04:01 9000 Interface/L3 spine01 (swp1) IP: 10.10.10.1/30
UP swp2 1G 0c:1e:ed:fe:04:02 9000 Interface/L3 spine02 (swp1) IP: 10.10.10.5/30
UP swp3 1G 0c:1e:ed:fe:04:03 9000 Interface/L3 spine03 (swp1) IP: 10.10.10.9/30
UP swp4 1G 0c:1e:ed:fe:04:04 9000 Interface/L3 spine04 (swp1) IP: 10.10.10.13/30
ADMDN swp5 N/A 0c:1e:ed:fe:04:05 1500 NotConfigured
ADMDN swp6 N/A 0c:1e:ed:fe:04:06 1500 NotConfigured
root@BB01:~#
BB02
root@BB02:~# net show route
show ip route
=============
Codes: K - kernel route, C - connected, S - static, R - RIP,
O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
F - PBR,
> - selected route, * - FIB route
B>* 10.10.10.0/30 [20/0] via 10.10.10.18, swp1, 00:04:33
B>* 10.10.10.4/30 [20/0] via 10.10.10.22, swp2, 00:04:27
B>* 10.10.10.8/30 [20/0] via 10.10.10.26, swp3, 00:04:37
B>* 10.10.10.12/30 [20/0] via 10.10.10.30, swp4, 00:04:33
C>* 10.10.10.16/30 is directly connected, swp1, 00:04:46
C>* 10.10.10.20/30 is directly connected, swp2, 00:04:46
C>* 10.10.10.24/30 is directly connected, swp3, 00:04:46
C>* 10.10.10.28/30 is directly connected, swp4, 00:04:46
B>* 169.254.1.0/30 [20/0] via 10.10.10.26, swp3, 00:04:27
* via 10.10.10.18, swp1, 00:04:27
* via 10.10.10.22, swp2, 00:04:27
* via 10.10.10.30, swp4, 00:04:27
B>* 172.16.1.1/32 [20/0] via 10.10.10.18, swp1, 00:02:52
B>* 172.16.1.2/32 [20/0] via 10.10.10.18, swp1, 00:04:33
B>* 172.16.1.3/32 [20/0] via 10.10.10.22, swp2, 00:04:27
B>* 172.16.1.4/32 [20/0] via 10.10.10.26, swp3, 00:02:52
B>* 172.16.1.5/32 [20/0] via 10.10.10.26, swp3, 00:04:37
B>* 172.16.1.6/32 [20/0] via 10.10.10.30, swp4, 00:04:33
C>* 172.16.222.2/32 is directly connected, lo, 00:04:46
show ipv6 route
===============
Codes: K - kernel route, C - connected, S - static, R - RIPng,
O - OSPFv3, I - IS-IS, B - BGP, N - NHRP, T - Table,
v - VNC, V - VNC-Direct, A - Babel, D - SHARP, F - PBR,
> - selected route, * - FIB route
C * fe80::/64 is directly connected, swp4, 00:04:45
C * fe80::/64 is directly connected, swp1, 00:04:45
C * fe80::/64 is directly connected, swp2, 00:04:46
C * fe80::/64 is directly connected, swp3, 00:04:46
C>* fe80::/64 is directly connected, eth0, 00:04:46
root@BB02:~# net show interface all mac
State Name Spd MAC MTU Mode LLDP Summary
----- ---- --- ----------------- ----- ------------- -------------- -------------------
UP lo N/A 00:00:00:00:00:00 65536 Loopback IP: 127.0.0.1/8
lo IP: 172.16.222.2/32
lo IP: ::1/128
UP eth0 1G 0c:1e:ed:33:95:00 1500 Mgmt
UP swp1 1G 0c:1e:ed:33:95:01 9000 Interface/L3 spine01 (swp2) IP: 10.10.10.17/30
UP swp2 1G 0c:1e:ed:33:95:02 9000 Interface/L3 spine02 (swp2) IP: 10.10.10.21/30
UP swp3 1G 0c:1e:ed:33:95:03 9000 Interface/L3 spine03 (swp2) IP: 10.10.10.25/30
UP swp4 1G 0c:1e:ed:33:95:04 9000 Interface/L3 spine04 (swp2) IP: 10.10.10.29/30
ADMDN swp5 N/A 0c:1e:ed:33:95:05 1500 NotConfigured
ADMDN swp6 N/A 0c:1e:ed:33:95:06 1500 NotConfigured
root@BB02:~#
spine01
root@spine01:~# net show route
show ip route
=============
Codes: K - kernel route, C - connected, S - static, R - RIP,
O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
F - PBR,
> - selected route, * - FIB route
C>* 10.10.10.0/30 is directly connected, swp1, 00:07:16
B>* 10.10.10.4/30 [20/0] via 10.10.10.1, swp1, 00:07:04
B>* 10.10.10.8/30 [20/0] via 10.10.10.1, swp1, 00:07:04
B>* 10.10.10.12/30 [20/0] via 10.10.10.1, swp1, 00:07:04
C>* 10.10.10.16/30 is directly connected, swp2, 00:07:16
B>* 10.10.10.20/30 [20/0] via 10.10.10.17, swp2, 00:07:05
B>* 10.10.10.24/30 [20/0] via 10.10.10.17, swp2, 00:07:05
B>* 10.10.10.28/30 [20/0] via 10.10.10.17, swp2, 00:07:05
C>* 169.254.1.0/30 is directly connected, peerlink.4094, 00:07:07
C>* 172.16.1.1/32 is directly connected, lo, 00:05:25
C>* 172.16.1.2/32 is directly connected, lo, 00:07:16
B>* 172.16.1.4/32 [20/0] via 10.10.10.1, swp1, 00:05:25
* via 10.10.10.17, swp2, 00:05:25
B>* 172.16.1.5/32 [20/0] via 10.10.10.17, swp2, 00:07:04
* via 10.10.10.1, swp1, 00:07:04
B>* 172.16.1.6/32 [20/0] via 10.10.10.17, swp2, 00:07:04
* via 10.10.10.1, swp1, 00:07:04
B>* 172.16.222.1/32 [20/0] via 10.10.10.1, swp1, 00:07:04
B>* 172.16.222.2/32 [20/0] via 10.10.10.17, swp2, 00:07:05
show ipv6 route
===============
Codes: K - kernel route, C - connected, S - static, R - RIPng,
O - OSPFv3, I - IS-IS, B - BGP, N - NHRP, T - Table,
v - VNC, V - VNC-Direct, A - Babel, D - SHARP, F - PBR,
> - selected route, * - FIB route
C * fe80::/64 is directly connected, bridge, 00:06:49
C * fe80::/64 is directly connected, peerlink.4094, 00:07:04
C * fe80::/64 is directly connected, swp2, 00:07:16
C * fe80::/64 is directly connected, swp1, 00:07:16
C>* fe80::/64 is directly connected, eth0, 00:07:16
root@spine01:~# net show interface all mac
State Name Spd MAC MTU Mode LLDP Summary
----- ------------- --- ----------------- ----- ------------- -------------- -------------------------
UP lo N/A 00:00:00:00:00:00 65536 Loopback IP: 127.0.0.1/8
lo IP: 172.16.1.2/32
lo IP: 172.16.1.1/32
lo IP: ::1/128
UP eth0 1G 0c:1e:ed:91:74:00 1500 Mgmt
UP swp1 1G 0c:1e:ed:91:74:01 9000 Interface/L3 BB01 (swp1) IP: 10.10.10.2/30
UP swp2 1G 0c:1e:ed:91:74:02 9000 Interface/L3 BB02 (swp1) IP: 10.10.10.18/30
UP swp3 1G 0c:1e:ed:91:74:03 1500 BondMember spine02 (swp3) Master: peerlink(UP)
UP swp4 1G 0c:1e:ed:91:74:03 1500 BondMember spine02 (swp4) Master: peerlink(UP)
UP swp5 1G 0c:1e:ed:91:74:05 9000 BondMember ToR01 (swp1) Master: bond-to-ToR01(UP)
ADMDN swp6 N/A 0c:1e:ed:91:74:06 1500 NotConfigured
UP bond-to-ToR01 1G 0c:1e:ed:91:74:05 9000 802.3ad Master: bridge(UP)
bond-to-ToR01 Bond Members: swp5(UP)
UP bridge N/A 0c:1e:ed:91:74:03 1500 Bridge/L2
UP peerlink 2G 0c:1e:ed:91:74:03 1500 802.3ad Master: bridge(UP)
peerlink Bond Members: swp3(UP)
peerlink Bond Members: swp4(UP)
UP peerlink.4094 2G 0c:1e:ed:91:74:03 1500 SubInt/L3 IP: 169.254.1.1/30
root@spine01:~# net show clag
The peer is alive
Our Priority, ID, and Role: 1000 0c:1e:ed:91:74:03 primary
Peer Priority, ID, and Role: 2000 0c:1e:ed:b9:90:03 secondary
Peer Interface and IP: peerlink.4094 169.254.1.2
VxLAN Anycast IP: 172.16.1.1
Backup IP: (inactive)
System MAC: 44:38:39:ff:00:00
CLAG Interfaces
Our Interface Peer Interface CLAG Id Conflicts Proto-Down Reason
---------------- ---------------- ------- -------------------- -----------------
bond-to-ToR01 bond-to-ToR01 1 - -
root@spine01:~#
spine02
root@spine02:~# net show route
show ip route
=============
Codes: K - kernel route, C - connected, S - static, R - RIP,
O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
F - PBR,
> - selected route, * - FIB route
B>* 10.10.10.0/30 [20/0] via 10.10.10.5, swp1, 00:07:43
C>* 10.10.10.4/30 is directly connected, swp1, 00:07:54
B>* 10.10.10.8/30 [20/0] via 10.10.10.5, swp1, 00:07:43
B>* 10.10.10.12/30 [20/0] via 10.10.10.5, swp1, 00:07:43
B>* 10.10.10.16/30 [20/0] via 10.10.10.21, swp2, 00:07:43
C>* 10.10.10.20/30 is directly connected, swp2, 00:07:54
B>* 10.10.10.24/30 [20/0] via 10.10.10.21, swp2, 00:07:43
B>* 10.10.10.28/30 [20/0] via 10.10.10.21, swp2, 00:07:43
C>* 169.254.1.0/30 is directly connected, peerlink.4094, 00:07:44
C>* 172.16.1.3/32 is directly connected, lo, 00:07:54
B>* 172.16.1.4/32 [20/0] via 10.10.10.5, swp1, 00:06:08
* via 10.10.10.21, swp2, 00:06:08
B>* 172.16.1.5/32 [20/0] via 10.10.10.5, swp1, 00:07:43
* via 10.10.10.21, swp2, 00:07:43
B>* 172.16.1.6/32 [20/0] via 10.10.10.5, swp1, 00:07:43
* via 10.10.10.21, swp2, 00:07:43
B>* 172.16.222.1/32 [20/0] via 10.10.10.5, swp1, 00:07:43
B>* 172.16.222.2/32 [20/0] via 10.10.10.21, swp2, 00:07:43
show ipv6 route
===============
Codes: K - kernel route, C - connected, S - static, R - RIPng,
O - OSPFv3, I - IS-IS, B - BGP, N - NHRP, T - Table,
v - VNC, V - VNC-Direct, A - Babel, D - SHARP, F - PBR,
> - selected route, * - FIB route
C * fe80::/64 is directly connected, bridge, 00:07:32
C * fe80::/64 is directly connected, peerlink.4094, 00:07:42
C * fe80::/64 is directly connected, swp2, 00:07:54
C * fe80::/64 is directly connected, swp1, 00:07:54
C>* fe80::/64 is directly connected, eth0, 00:07:54
root@spine02:~# net show interface all mac
State Name Spd MAC MTU Mode LLDP Summary
----- ------------- --- ----------------- ----- ------------- -------------- -------------------------
UP lo N/A 00:00:00:00:00:00 65536 Loopback IP: 127.0.0.1/8
lo IP: 172.16.1.3/32
lo IP: ::1/128
UP eth0 1G 0c:1e:ed:b9:90:00 1500 Mgmt
UP swp1 1G 0c:1e:ed:b9:90:01 9000 Interface/L3 BB01 (swp2) IP: 10.10.10.6/30
UP swp2 1G 0c:1e:ed:b9:90:02 9000 Interface/L3 BB02 (swp2) IP: 10.10.10.22/30
UP swp3 1G 0c:1e:ed:b9:90:03 1500 BondMember spine01 (swp3) Master: peerlink(UP)
UP swp4 1G 0c:1e:ed:b9:90:03 1500 BondMember spine01 (swp4) Master: peerlink(UP)
UP swp5 1G 0c:1e:ed:b9:90:05 9000 BondMember ToR01 (swp2) Master: bond-to-ToR01(UP)
ADMDN swp6 N/A 0c:1e:ed:b9:90:06 1500 NotConfigured
UP bond-to-ToR01 1G 0c:1e:ed:b9:90:05 9000 802.3ad Master: bridge(UP)
bond-to-ToR01 Bond Members: swp5(UP)
UP bridge N/A 0c:1e:ed:b9:90:03 1500 Bridge/L2
UP peerlink 2G 0c:1e:ed:b9:90:03 1500 802.3ad Master: bridge(UP)
peerlink Bond Members: swp3(UP)
peerlink Bond Members: swp4(UP)
UP peerlink.4094 2G 0c:1e:ed:b9:90:03 1500 SubInt/L3 IP: 169.254.1.2/30
root@spine02:~# net show clag
The peer is alive
Peer Priority, ID, and Role: 1000 0c:1e:ed:91:74:03 primary
Our Priority, ID, and Role: 2000 0c:1e:ed:b9:90:03 secondary
Peer Interface and IP: peerlink.4094 169.254.1.1
VxLAN Anycast IP: 172.16.1.1
Backup IP: (inactive)
System MAC: 44:38:39:ff:00:00
CLAG Interfaces
Our Interface Peer Interface CLAG Id Conflicts Proto-Down Reason
---------------- ---------------- ------- -------------------- -----------------
bond-to-ToR01 bond-to-ToR01 1 - -
root@spine02:~#
spine03
root@spine03:~# net show route
show ip route
=============
Codes: K - kernel route, C - connected, S - static, R - RIP,
O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
F - PBR,
> - selected route, * - FIB route
B>* 10.10.10.0/30 [20/0] via 10.10.10.9, swp1, 00:08:09
B>* 10.10.10.4/30 [20/0] via 10.10.10.9, swp1, 00:08:09
C>* 10.10.10.8/30 is directly connected, swp1, 00:08:21
B>* 10.10.10.12/30 [20/0] via 10.10.10.9, swp1, 00:08:09
B>* 10.10.10.16/30 [20/0] via 10.10.10.25, swp2, 00:08:11
B>* 10.10.10.20/30 [20/0] via 10.10.10.25, swp2, 00:08:11
C>* 10.10.10.24/30 is directly connected, swp2, 00:08:21
B>* 10.10.10.28/30 [20/0] via 10.10.10.25, swp2, 00:08:11
C>* 169.254.1.0/30 is directly connected, peerlink.4094, 00:08:17
B>* 172.16.1.1/32 [20/0] via 10.10.10.9, swp1, 00:06:26
* via 10.10.10.25, swp2, 00:06:26
B>* 172.16.1.2/32 [20/0] via 10.10.10.25, swp2, 00:08:06
* via 10.10.10.9, swp1, 00:08:06
B>* 172.16.1.3/32 [20/0] via 10.10.10.9, swp1, 00:08:01
* via 10.10.10.25, swp2, 00:08:01
C>* 172.16.1.4/32 is directly connected, lo, 00:06:26
C>* 172.16.1.5/32 is directly connected, lo, 00:08:21
B>* 172.16.222.1/32 [20/0] via 10.10.10.9, swp1, 00:08:09
B>* 172.16.222.2/32 [20/0] via 10.10.10.25, swp2, 00:08:11
show ipv6 route
===============
Codes: K - kernel route, C - connected, S - static, R - RIPng,
O - OSPFv3, I - IS-IS, B - BGP, N - NHRP, T - Table,
v - VNC, V - VNC-Direct, A - Babel, D - SHARP, F - PBR,
> - selected route, * - FIB route
C * fe80::/64 is directly connected, bridge, 00:07:53
C * fe80::/64 is directly connected, peerlink.4094, 00:08:15
C * fe80::/64 is directly connected, swp2, 00:08:21
C * fe80::/64 is directly connected, swp1, 00:08:21
C>* fe80::/64 is directly connected, eth0, 00:08:21
root@spine03:~# net show interface all mac
State Name Spd MAC MTU Mode LLDP Summary
----- ------------- --- ----------------- ----- ------------- -------------- -------------------------
UP lo N/A 00:00:00:00:00:00 65536 Loopback IP: 127.0.0.1/8
lo IP: 172.16.1.5/32
lo IP: 172.16.1.4/32
lo IP: ::1/128
UP eth0 1G 0c:1e:ed:49:da:00 1500 Mgmt
UP swp1 1G 0c:1e:ed:49:da:01 9000 Interface/L3 BB01 (swp3) IP: 10.10.10.10/30
UP swp2 1G 0c:1e:ed:49:da:02 9000 Interface/L3 BB02 (swp3) IP: 10.10.10.26/30
UP swp3 1G 0c:1e:ed:49:da:03 1500 BondMember spine04 (swp3) Master: peerlink(UP)
UP swp4 1G 0c:1e:ed:49:da:03 1500 BondMember spine04 (swp4) Master: peerlink(UP)
UP swp5 1G 0c:1e:ed:49:da:05 9000 BondMember ToR02 (swp1) Master: bond-to-ToR02(UP)
ADMDN swp6 N/A 0c:1e:ed:49:da:06 1500 NotConfigured
UP bond-to-ToR02 1G 0c:1e:ed:49:da:05 9000 802.3ad Master: bridge(UP)
bond-to-ToR02 Bond Members: swp5(UP)
UP bridge N/A 0c:1e:ed:49:da:03 1500 Bridge/L2
UP peerlink 2G 0c:1e:ed:49:da:03 1500 802.3ad Master: bridge(UP)
peerlink Bond Members: swp3(UP)
peerlink Bond Members: swp4(UP)
UP peerlink.4094 2G 0c:1e:ed:49:da:03 1500 SubInt/L3 IP: 169.254.1.1/30
root@spine03:~# net show clag
The peer is alive
Our Priority, ID, and Role: 1000 0c:1e:ed:49:da:03 primary
Peer Priority, ID, and Role: 2000 0c:1e:ed:5a:03:03 secondary
Peer Interface and IP: peerlink.4094 169.254.1.2
VxLAN Anycast IP: 172.16.1.4
Backup IP: (inactive)
System MAC: 44:38:39:ff:00:01
CLAG Interfaces
Our Interface Peer Interface CLAG Id Conflicts Proto-Down Reason
---------------- ---------------- ------- -------------------- -----------------
bond-to-ToR02 bond-to-ToR02 1 - -
root@spine03:~#
spine04
root@spine04:~# net show route
net show clagshow ip route
=============
Codes: K - kernel route, C - connected, S - static, R - RIP,
O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
F - PBR,
> - selected route, * - FIB route
B>* 10.10.10.0/30 [20/0] via 10.10.10.13, swp1, 00:08:19
B>* 10.10.10.4/30 [20/0] via 10.10.10.13, swp1, 00:08:19
B>* 10.10.10.8/30 [20/0] via 10.10.10.13, swp1, 00:08:19
C>* 10.10.10.12/30 is directly connected, swp1, 00:08:31
B>* 10.10.10.16/30 [20/0] via 10.10.10.29, swp2, 00:08:20
B>* 10.10.10.20/30 [20/0] via 10.10.10.29, swp2, 00:08:20
B>* 10.10.10.24/30 [20/0] via 10.10.10.29, swp2, 00:08:20
C>* 10.10.10.28/30 is directly connected, swp2, 00:08:31
C>* 169.254.1.0/30 is directly connected, peerlink.4094, 00:08:21
B>* 172.16.1.1/32 [20/0] via 10.10.10.13, swp1, 00:06:39
* via 10.10.10.29, swp2, 00:06:39
B>* 172.16.1.2/32 [20/0] via 10.10.10.29, swp2, 00:08:19
* via 10.10.10.13, swp1, 00:08:19
B>* 172.16.1.3/32 [20/0] via 10.10.10.13, swp1, 00:08:14
* via 10.10.10.29, swp2, 00:08:14
C>* 172.16.1.6/32 is directly connected, lo, 00:08:31
B>* 172.16.222.1/32 [20/0] via 10.10.10.13, swp1, 00:08:19
B>* 172.16.222.2/32 [20/0] via 10.10.10.29, swp2, 00:08:20
show ipv6 route
===============
Codes: K - kernel route, C - connected, S - static, R - RIPng,
O - OSPFv3, I - IS-IS, B - BGP, N - NHRP, T - Table,
v - VNC, V - VNC-Direct, A - Babel, D - SHARP, F - PBR,
> - selected route, * - FIB route
C * fe80::/64 is directly connected, bridge, 00:08:03
C * fe80::/64 is directly connected, peerlink.4094, 00:08:20
C * fe80::/64 is directly connected, swp2, 00:08:31
C * fe80::/64 is directly connected, swp1, 00:08:31
C>* fe80::/64 is directly connected, eth0, 00:08:31
root@spine04:~# net show interface all mac
State Name Spd MAC MTU Mode LLDP Summary
----- ------------- --- ----------------- ----- ------------- -------------- -------------------------
UP lo N/A 00:00:00:00:00:00 65536 Loopback IP: 127.0.0.1/8
lo IP: 172.16.1.6/32
lo IP: ::1/128
UP eth0 1G 0c:1e:ed:5a:03:00 1500 Mgmt
UP swp1 1G 0c:1e:ed:5a:03:01 9000 Interface/L3 BB01 (swp4) IP: 10.10.10.14/30
UP swp2 1G 0c:1e:ed:5a:03:02 9000 Interface/L3 BB02 (swp4) IP: 10.10.10.30/30
UP swp3 1G 0c:1e:ed:5a:03:03 1500 BondMember spine03 (swp3) Master: peerlink(UP)
UP swp4 1G 0c:1e:ed:5a:03:03 1500 BondMember spine03 (swp4) Master: peerlink(UP)
UP swp5 1G 0c:1e:ed:5a:03:05 9000 BondMember ToR02 (swp2) Master: bond-to-ToR02(UP)
ADMDN swp6 N/A 0c:1e:ed:5a:03:06 1500 NotConfigured
UP bond-to-ToR02 1G 0c:1e:ed:5a:03:05 9000 802.3ad Master: bridge(UP)
bond-to-ToR02 Bond Members: swp5(UP)
UP bridge N/A 0c:1e:ed:5a:03:03 1500 Bridge/L2
UP peerlink 2G 0c:1e:ed:5a:03:03 1500 802.3ad Master: bridge(UP)
peerlink Bond Members: swp3(UP)
peerlink Bond Members: swp4(UP)
UP peerlink.4094 2G 0c:1e:ed:5a:03:03 1500 SubInt/L3 IP: 169.254.1.2/30
root@spine04:~# net show clag
The peer is alive
Peer Priority, ID, and Role: 1000 0c:1e:ed:49:da:03 primary
Our Priority, ID, and Role: 2000 0c:1e:ed:5a:03:03 secondary
Peer Interface and IP: peerlink.4094 169.254.1.1
VxLAN Anycast IP: 172.16.1.4
Backup IP: (inactive)
System MAC: 44:38:39:ff:00:01
CLAG Interfaces
Our Interface Peer Interface CLAG Id Conflicts Proto-Down Reason
---------------- ---------------- ------- -------------------- -----------------
bond-to-ToR02 bond-to-ToR02 1 - -
root@spine04:~#
ToR01
root@ToR01:~# net show interface all mac
State Name Spd MAC MTU Mode LLDP Summary
----- ------------- --- ----------------- ----- ------------- -------------- -------------------------
UP lo N/A 00:00:00:00:00:00 65536 Loopback IP: 127.0.0.1/8
lo IP: ::1/128
UP eth0 1G 0c:1e:ed:9f:32:00 1500 Mgmt
UP swp1 1G 0c:1e:ed:9f:32:02 9000 BondMember spine01 (swp5) Master: bond-to-SPINE(UP)
UP swp2 1G 0c:1e:ed:9f:32:02 9000 BondMember spine02 (swp5) Master: bond-to-SPINE(UP)
UP swp3 1G 0c:1e:ed:9f:32:03 1500 Access/L2 Master: bridge(UP)
UP swp4 1G 0c:1e:ed:9f:32:04 1500 Access/L2 Master: bridge(UP)
ADMDN swp5 N/A 0c:1e:ed:9f:32:05 1500 NotConfigured
ADMDN swp6 N/A 0c:1e:ed:9f:32:06 1500 NotConfigured
UP bond-to-SPINE 2G 0c:1e:ed:9f:32:02 9000 802.3ad Master: bridge(UP)
bond-to-SPINE Bond Members: swp1(UP)
bond-to-SPINE Bond Members: swp2(UP)
UP bridge N/A 0c:1e:ed:9f:32:02 1500 Bridge/L2
root@ToR01:~#
ToR02
root@ToR02:~# net show interface all mac
State Name Spd MAC MTU Mode LLDP Summary
----- ------------- --- ----------------- ----- ------------- -------------- -------------------------
UP lo N/A 00:00:00:00:00:00 65536 Loopback IP: 127.0.0.1/8
lo IP: ::1/128
UP eth0 1G 0c:1e:ed:2c:26:00 1500 Mgmt
UP swp1 1G 0c:1e:ed:2c:26:02 9000 BondMember spine03 (swp5) Master: bond-to-SPINE(UP)
UP swp2 1G 0c:1e:ed:2c:26:02 9000 BondMember spine04 (swp5) Master: bond-to-SPINE(UP)
UP swp3 1G 0c:1e:ed:2c:26:03 1500 Access/L2 Master: bridge(UP)
UP swp4 1G 0c:1e:ed:2c:26:04 1500 Access/L2 Master: bridge(UP)
ADMDN swp5 N/A 0c:1e:ed:2c:26:05 1500 NotConfigured
ADMDN swp6 N/A 0c:1e:ed:2c:26:06 1500 NotConfigured
UP bond-to-SPINE 2G 0c:1e:ed:2c:26:02 9000 802.3ad Master: bridge(UP)
bond-to-SPINE Bond Members: swp1(UP)
bond-to-SPINE Bond Members: swp2(UP)
UP bridge N/A 0c:1e:ed:2c:26:02 1500 Bridge/L2
root@ToR02:~#
overlay設定
BB01/02
#vxlanに無関係なBB01/02も転送のためMP-BGP EVPNを有効化
net add bgp l2vpn evpn neighbor PEER_SPINE activate
spine01だけ注釈を付ける
spine01
net add vxlan vxlan10100 vxlan id 10100
net add vxlan vxlan10100 bridge access 100
net add vxlan vxlan10200 vxlan id 10200
net add vxlan vxlan10200 bridge access 200
#Disable Data Plane MAC Learning over VXLAN Tunnels
net add vxlan vxlan10100,vxlan10200 bridge learning off
#The VNIs are configured identically on both peers. However, vxlan-local-tunnelip must be sourced from unique loopback stanza IP address of the switch.
net add vxlan vxlan10100,vxlan10200 vxlan local-tunnelip 172.16.1.2
#MTU設定
net add vxlan vxlan10100 mtu 1550
net add vxlan vxlan10200 mtu 1550
#evpn有効化
net add bgp l2vpn evpn neighbor PEER_BB activate
net add bgp l2vpn evpn advertise-all-vni
spine02
net add vxlan vxlan10100 vxlan id 10100
net add vxlan vxlan10100 bridge access 100
net add vxlan vxlan10200 vxlan id 10200
net add vxlan vxlan10200 bridge access 200
net add vxlan vxlan10100,vxlan10200 bridge learning off
net add vxlan vxlan10100,vxlan10200 vxlan local-tunnelip 172.16.1.3
net add vxlan vxlan10100 mtu 1550
net add vxlan vxlan10200 mtu 1550
net add bgp l2vpn evpn neighbor PEER_BB activate
net add bgp l2vpn evpn advertise-all-vni
spine03
net add vxlan vxlan10100 vxlan id 10100
net add vxlan vxlan10100 bridge access 100
net add vxlan vxlan10200 vxlan id 10200
net add vxlan vxlan10200 bridge access 200
net add vxlan vxlan10100,vxlan10200 bridge learning off
net add vxlan vxlan10100,vxlan10200 vxlan local-tunnelip 172.16.1.5
net add vxlan vxlan10100 mtu 1550
net add vxlan vxlan10200 mtu 1550
net add bgp l2vpn evpn neighbor PEER_BB activate
net add bgp l2vpn evpn advertise-all-vni
spine04
net add vxlan vxlan10100 vxlan id 10100
net add vxlan vxlan10100 bridge access 100
net add vxlan vxlan10200 vxlan id 10200
net add vxlan vxlan10200 bridge access 200
net add vxlan vxlan10100,vxlan10200 bridge learning off
net add vxlan vxlan10100,vxlan10200 vxlan local-tunnelip 172.16.1.6
net add vxlan vxlan10100 mtu 1550
net add vxlan vxlan10200 mtu 1550
net add bgp l2vpn evpn neighbor PEER_BB activate
net add bgp l2vpn evpn advertise-all-vni
確認後、commit
net pending
net commit
nodeにIPアドレス設定
UbuntuDockerGuest-1~4
#UbuntuDockerGuest-1
ifconfig eth0 192.168.1.1 netmask 255.255.255.0
#UbuntuDockerGuest-2
ifconfig eth0 192.168.100.1 netmask 255.255.255.0
#UbuntuDockerGuest-3
ifconfig eth0 192.168.1.2 netmask 255.255.255.0
#UbuntuDockerGuest-4
ifconfig eth0 192.168.100.2 netmask 255.255.255.0
動作確認
- ping確認
UbuntuDockerGuest-1
root@UbuntuDockerGuest-1:~# ping 192.168.1.2
PING 192.168.1.2 (192.168.1.2) 56(84) bytes of data.
64 bytes from 192.168.1.2: icmp_seq=1 ttl=64 time=5.60 ms
64 bytes from 192.168.1.2: icmp_seq=2 ttl=64 time=3.53 ms
64 bytes from 192.168.1.2: icmp_seq=3 ttl=64 time=3.41 ms
64 bytes from 192.168.1.2: icmp_seq=4 ttl=64 time=3.51 ms
64 bytes from 192.168.1.2: icmp_seq=5 ttl=64 time=3.42 ms
^C
--- 192.168.1.2 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4008ms
rtt min/avg/max/mdev = 3.412/3.899/5.606/0.857 ms
root@UbuntuDockerGuest-1:~#
UbuntuDockerGuest-2
root@UbuntuDockerGuest-2:~# ping 192.168.100.2
PING 192.168.100.2 (192.168.100.2) 56(84) bytes of data.
64 bytes from 192.168.100.2: icmp_seq=1 ttl=64 time=4.20 ms
64 bytes from 192.168.100.2: icmp_seq=2 ttl=64 time=3.50 ms
64 bytes from 192.168.100.2: icmp_seq=3 ttl=64 time=3.48 ms
64 bytes from 192.168.100.2: icmp_seq=4 ttl=64 time=2.81 ms
64 bytes from 192.168.100.2: icmp_seq=5 ttl=64 time=3.35 ms
^C
--- 192.168.100.2 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4011ms
rtt min/avg/max/mdev = 2.815/3.473/4.200/0.442 ms
root@UbuntuDockerGuest-2:~#
- 各スイッチのテーブル確認
remote-VTEPのアドレスがroute type-2/3で確認できる
spine01
root@spine01:~# net show evpn
L2 VNIs: 2
L3 VNIs: 0
Advertise gateway mac-ip: No
Duplicate address detection: Enable
Detection max-moves 5, time 180
root@spine01:~# net show evpn mac vni all
VNI 10200 #MACs (local and remote) 2
MAC Type Intf/Remote VTEP VLAN
12:ed:20:2e:b3:1a remote 172.16.1.4
7a:a1:d2:fa:86:9b local bond-to-ToR01 200
VNI 10100 #MACs (local and remote) 2
MAC Type Intf/Remote VTEP VLAN
a2:af:dc:0a:cd:6c local bond-to-ToR01 100
4e:4f:c1:03:32:f3 remote 172.16.1.4
root@spine01:~# net show evpn vni detail
VNI: 10200
Type: L2
Tenant VRF: default
VxLAN interface: vxlan10200
VxLAN ifIndex: 9
Local VTEP IP: 172.16.1.1
Remote VTEPs for this VNI:
172.16.1.4
Number of MACs (local and remote) known for this VNI: 2
Number of ARPs (IPv4 and IPv6, local and remote) known for this VNI: 0
Advertise-gw-macip: No
VNI: 10100
Type: L2
Tenant VRF: default
VxLAN interface: vxlan10100
VxLAN ifIndex: 10
Local VTEP IP: 172.16.1.1
Remote VTEPs for this VNI:
172.16.1.4
Number of MACs (local and remote) known for this VNI: 2
Number of ARPs (IPv4 and IPv6, local and remote) known for this VNI: 0
Advertise-gw-macip: No
root@spine01:~# net show bgp
show bgp ipv4 unicast
=====================
BGP table version is 16, local router ID is 172.16.1.2
Status codes: s suppressed, d damped, h history, * valid, > best, = multipath,
i internal, r RIB-failure, S Stale, R Removed
Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Weight Path
* 10.10.10.0/30 10.10.10.1 0 0 65000 ?
*> 0.0.0.0 0 32768 ?
*> 10.10.10.4/30 10.10.10.1 0 0 65000 ?
*> 10.10.10.8/30 10.10.10.1 0 0 65000 ?
* 10.10.10.17 0 65000 65002 ?
*> 10.10.10.12/30 10.10.10.1 0 0 65000 ?
* 10.10.10.17 0 65000 65002 ?
* 10.10.10.16/30 10.10.10.17 0 0 65000 ?
*> 0.0.0.0 0 32768 ?
*> 10.10.10.20/30 10.10.10.17 0 0 65000 ?
* 10.10.10.24/30 10.10.10.1 0 65000 65002 ?
*> 10.10.10.17 0 0 65000 ?
* 10.10.10.28/30 10.10.10.1 0 65000 65002 ?
*> 10.10.10.17 0 0 65000 ?
* 169.254.1.0/30 10.10.10.1 0 65000 65002 ?
* 10.10.10.17 0 65000 65002 ?
*> 0.0.0.0 0 32768 ?
*> 172.16.1.1/32 0.0.0.0 0 32768 ?
*> 172.16.1.2/32 0.0.0.0 0 32768 ?
*> 172.16.1.4/32 10.10.10.1 0 65000 65002 ?
*= 10.10.10.17 0 65000 65002 ?
*> 172.16.1.5/32 10.10.10.1 0 65000 65002 ?
*= 10.10.10.17 0 65000 65002 ?
*> 172.16.1.6/32 10.10.10.1 0 65000 65002 ?
*= 10.10.10.17 0 65000 65002 ?
*> 172.16.222.1/32 10.10.10.1 0 0 65000 ?
*> 172.16.222.2/32 10.10.10.17 0 0 65000 ?
Displayed 16 routes and 27 total paths
show bgp ipv6 unicast
=====================
No BGP prefixes displayed, 0 exist
root@spine01:~# net show bgp evpn route vni all
VNI: 10200
BGP table version is 4, local router ID is 172.16.1.2
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete
EVPN type-2 prefix: [2]:[ESI]:[EthTag]:[MAClen]:[MAC]:[IPlen]:[IP]
EVPN type-3 prefix: [3]:[EthTag]:[IPlen]:[OrigIP]
EVPN type-5 prefix: [5]:[ESI]:[EthTag]:[IPlen]:[IP]
Network Next Hop Metric LocPrf Weight Path
* [2]:[0]:[0]:[48]:[12:ed:20:2e:b3:1a]
172.16.1.4 0 65000 65002 i
* [2]:[0]:[0]:[48]:[12:ed:20:2e:b3:1a]
172.16.1.4 0 65000 65002 i
* [2]:[0]:[0]:[48]:[12:ed:20:2e:b3:1a]
172.16.1.4 0 65000 65002 i
*> [2]:[0]:[0]:[48]:[12:ed:20:2e:b3:1a]
172.16.1.4 0 65000 65002 i
*> [2]:[0]:[0]:[48]:[7a:a1:d2:fa:86:9b]
172.16.1.1 32768 i
*> [3]:[0]:[32]:[172.16.1.1]
172.16.1.1 32768 i
* [3]:[0]:[32]:[172.16.1.4]
172.16.1.4 0 65000 65002 i
* [3]:[0]:[32]:[172.16.1.4]
172.16.1.4 0 65000 65002 i
* [3]:[0]:[32]:[172.16.1.4]
172.16.1.4 0 65000 65002 i
*> [3]:[0]:[32]:[172.16.1.4]
172.16.1.4 0 65000 65002 i
Displayed 4 prefixes (10 paths)
VNI: 10100
BGP table version is 4, local router ID is 172.16.1.2
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete
EVPN type-2 prefix: [2]:[ESI]:[EthTag]:[MAClen]:[MAC]:[IPlen]:[IP]
EVPN type-3 prefix: [3]:[EthTag]:[IPlen]:[OrigIP]
EVPN type-5 prefix: [5]:[ESI]:[EthTag]:[IPlen]:[IP]
Network Next Hop Metric LocPrf Weight Path
* [2]:[0]:[0]:[48]:[4e:4f:c1:03:32:f3]
172.16.1.4 0 65000 65002 i
* [2]:[0]:[0]:[48]:[4e:4f:c1:03:32:f3]
172.16.1.4 0 65000 65002 i
* [2]:[0]:[0]:[48]:[4e:4f:c1:03:32:f3]
172.16.1.4 0 65000 65002 i
*> [2]:[0]:[0]:[48]:[4e:4f:c1:03:32:f3]
172.16.1.4 0 65000 65002 i
*> [2]:[0]:[0]:[48]:[a2:af:dc:0a:cd:6c]
172.16.1.1 32768 i
*> [3]:[0]:[32]:[172.16.1.1]
172.16.1.1 32768 i
* [3]:[0]:[32]:[172.16.1.4]
172.16.1.4 0 65000 65002 i
* [3]:[0]:[32]:[172.16.1.4]
172.16.1.4 0 65000 65002 i
* [3]:[0]:[32]:[172.16.1.4]
172.16.1.4 0 65000 65002 i
*> [3]:[0]:[32]:[172.16.1.4]
172.16.1.4 0 65000 65002 i
Displayed 4 prefixes (10 paths)
root@spine01:~# net show bgp evpn summary
BGP router identifier 172.16.1.2, local AS number 65001 vrf-id 0
BGP table version 0
RIB entries 15, using 2280 bytes of memory
Peers 2, using 39 KiB of memory
Peer groups 1, using 64 bytes of memory
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd
BB01(10.10.10.1) 4 65000 387 383 0 0 0 00:18:01 8
BB02(10.10.10.17) 4 65000 387 383 0 0 0 00:18:01 8
Total number of neighbors 2
root@spine01:~# net show bgp evpn vni
Advertise Gateway Macip: Disabled
Advertise All VNI flag: Enabled
BUM flooding: Head-end replication
Number of L2 VNIs: 2
Number of L3 VNIs: 0
Flags: * - Kernel
VNI Type RD Import RT Export RT Tenant VRF
* 10200 L2 172.16.1.2:2 65001:10200 65001:10200 default
* 10100 L2 172.16.1.2:3 65001:10100 65001:10100 default
root@spine01:~# net show route
show ip route
=============
Codes: K - kernel route, C - connected, S - static, R - RIP,
O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
F - PBR,
> - selected route, * - FIB route
C>* 10.10.10.0/30 is directly connected, swp1, 00:18:12
B>* 10.10.10.4/30 [20/0] via 10.10.10.1, swp1, 00:18:08
B>* 10.10.10.8/30 [20/0] via 10.10.10.1, swp1, 00:18:08
B>* 10.10.10.12/30 [20/0] via 10.10.10.1, swp1, 00:18:08
C>* 10.10.10.16/30 is directly connected, swp2, 00:18:12
B>* 10.10.10.20/30 [20/0] via 10.10.10.17, swp2, 00:18:08
B>* 10.10.10.24/30 [20/0] via 10.10.10.17, swp2, 00:18:08
B>* 10.10.10.28/30 [20/0] via 10.10.10.17, swp2, 00:18:08
C>* 169.254.1.0/30 is directly connected, peerlink.4094, 00:18:10
C>* 172.16.1.1/32 is directly connected, lo, 00:18:00
C>* 172.16.1.2/32 is directly connected, lo, 00:18:12
B>* 172.16.1.4/32 [20/0] via 10.10.10.1, swp1, 00:18:06
* via 10.10.10.17, swp2, 00:18:06
B>* 172.16.1.5/32 [20/0] via 10.10.10.1, swp1, 00:18:08
* via 10.10.10.17, swp2, 00:18:08
B>* 172.16.1.6/32 [20/0] via 10.10.10.1, swp1, 00:18:08
* via 10.10.10.17, swp2, 00:18:08
B>* 172.16.222.1/32 [20/0] via 10.10.10.1, swp1, 00:18:08
B>* 172.16.222.2/32 [20/0] via 10.10.10.17, swp2, 00:18:08
show ipv6 route
===============
Codes: K - kernel route, C - connected, S - static, R - RIPng,
O - OSPFv3, I - IS-IS, B - BGP, N - NHRP, T - Table,
v - VNC, V - VNC-Direct, A - Babel, D - SHARP, F - PBR,
> - selected route, * - FIB route
C * fe80::/64 is directly connected, bridge, 00:18:06
C * fe80::/64 is directly connected, peerlink.4094, 00:18:09
C * fe80::/64 is directly connected, swp1, 00:18:11
C * fe80::/64 is directly connected, swp2, 00:18:11
C>* fe80::/64 is directly connected, eth0, 00:18:12
root@spine01:~# net show bridge macs
VLAN Master Interface MAC TunnelDest State Flags LastSeen
-------- ------ ------------- ----------------- ---------- --------- ------------- --------
1 bridge bond-to-ToR01 0c:1e:ed:9f:32:02 00:00:09
1 bridge bond-to-ToR01 0e:1e:ed:9f:32:01 00:00:03
1 bridge bond-to-ToR01 0e:1e:ed:9f:32:02 00:00:09
100 bridge bond-to-ToR01 a2:af:dc:0a:cd:6c 00:11:11
100 bridge vxlan10100 4e:4f:c1:03:32:f3 offload 00:14:23
200 bridge bond-to-ToR01 7a:a1:d2:fa:86:9b 00:00:09
200 bridge vxlan10200 12:ed:20:2e:b3:1a offload 00:14:14
untagged vxlan10100 00:00:00:00:00:00 172.16.1.4 permanent self 00:18:15
untagged vxlan10100 4e:4f:c1:03:32:f3 172.16.1.4 self, offload 00:14:23
untagged vxlan10200 00:00:00:00:00:00 172.16.1.4 permanent self 00:18:15
untagged vxlan10200 12:ed:20:2e:b3:1a 172.16.1.4 self, offload 00:14:14
untagged bridge bond-to-ToR01 0c:1e:ed:91:74:05 permanent 00:18:20
untagged bridge peerlink 0c:1e:ed:91:74:03 permanent 00:18:20
untagged bridge vxlan10100 76:ce:61:90:e8:67 permanent 00:18:20
untagged bridge vxlan10200 06:3d:ed:af:51:d5 permanent 00:18:20
root@spine01:~#
spine02
root@spine02:~# net show evpn
L2 VNIs: 2
L3 VNIs: 0
Advertise gateway mac-ip: No
Duplicate address detection: Enable
Detection max-moves 5, time 180
root@spine02:~# net show evpn mac vni all
VNI 10200 #MACs (local and remote) 2
MAC Type Intf/Remote VTEP VLAN
12:ed:20:2e:b3:1a remote 172.16.1.4
7a:a1:d2:fa:86:9b local bond-to-ToR01 200
VNI 10100 #MACs (local and remote) 2
MAC Type Intf/Remote VTEP VLAN
a2:af:dc:0a:cd:6c local bond-to-ToR01 100
4e:4f:c1:03:32:f3 remote 172.16.1.4
root@spine02:~# net show evpn vni detail
VNI: 10200
Type: L2
Tenant VRF: default
VxLAN interface: vxlan10200
VxLAN ifIndex: 9
Local VTEP IP: 172.16.1.1
Remote VTEPs for this VNI:
172.16.1.4
Number of MACs (local and remote) known for this VNI: 2
Number of ARPs (IPv4 and IPv6, local and remote) known for this VNI: 0
Advertise-gw-macip: No
VNI: 10100
Type: L2
Tenant VRF: default
VxLAN interface: vxlan10100
VxLAN ifIndex: 10
Local VTEP IP: 172.16.1.1
Remote VTEPs for this VNI:
172.16.1.4
Number of MACs (local and remote) known for this VNI: 2
Number of ARPs (IPv4 and IPv6, local and remote) known for this VNI: 0
Advertise-gw-macip: No
root@spine02:~# net show bgp
show bgp ipv4 unicast
=====================
BGP table version is 20, local router ID is 172.16.1.3
Status codes: s suppressed, d damped, h history, * valid, > best, = multipath,
i internal, r RIB-failure, S Stale, R Removed
Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Weight Path
*> 10.10.10.0/30 10.10.10.5 0 0 65000 ?
* 10.10.10.4/30 10.10.10.5 0 0 65000 ?
*> 0.0.0.0 0 32768 ?
* 10.10.10.8/30 10.10.10.21 0 65000 65002 ?
*> 10.10.10.5 0 0 65000 ?
* 10.10.10.12/30 10.10.10.21 0 65000 65002 ?
*> 10.10.10.5 0 0 65000 ?
*> 10.10.10.16/30 10.10.10.21 0 0 65000 ?
* 10.10.10.20/30 10.10.10.21 0 0 65000 ?
*> 0.0.0.0 0 32768 ?
* 10.10.10.24/30 10.10.10.5 0 65000 65002 ?
*> 10.10.10.21 0 0 65000 ?
*> 10.10.10.28/30 10.10.10.21 0 0 65000 ?
* 10.10.10.5 0 65000 65002 ?
*> 169.254.1.0/30 0.0.0.0 0 32768 ?
* 10.10.10.21 0 65000 65002 ?
* 10.10.10.5 0 65000 65002 ?
*> 172.16.1.1/32 0.0.0.0 0 32768 ?
*> 172.16.1.3/32 0.0.0.0 0 32768 ?
*> 172.16.1.4/32 10.10.10.5 0 65000 65002 ?
*= 10.10.10.21 0 65000 65002 ?
*> 172.16.1.5/32 10.10.10.5 0 65000 65002 ?
*= 10.10.10.21 0 65000 65002 ?
*= 172.16.1.6/32 10.10.10.21 0 65000 65002 ?
*> 10.10.10.5 0 65000 65002 ?
*> 172.16.222.1/32 10.10.10.5 0 0 65000 ?
*> 172.16.222.2/32 10.10.10.21 0 0 65000 ?
Displayed 16 routes and 27 total paths
show bgp ipv6 unicast
=====================
No BGP prefixes displayed, 0 exist
root@spine02:~# net show bgp evpn route vni all
VNI: 10200
BGP table version is 4, local router ID is 172.16.1.3
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete
EVPN type-2 prefix: [2]:[ESI]:[EthTag]:[MAClen]:[MAC]:[IPlen]:[IP]
EVPN type-3 prefix: [3]:[EthTag]:[IPlen]:[OrigIP]
EVPN type-5 prefix: [5]:[ESI]:[EthTag]:[IPlen]:[IP]
Network Next Hop Metric LocPrf Weight Path
* [2]:[0]:[0]:[48]:[12:ed:20:2e:b3:1a]
172.16.1.4 0 65000 65002 i
* [2]:[0]:[0]:[48]:[12:ed:20:2e:b3:1a]
172.16.1.4 0 65000 65002 i
* [2]:[0]:[0]:[48]:[12:ed:20:2e:b3:1a]
172.16.1.4 0 65000 65002 i
*> [2]:[0]:[0]:[48]:[12:ed:20:2e:b3:1a]
172.16.1.4 0 65000 65002 i
*> [2]:[0]:[0]:[48]:[7a:a1:d2:fa:86:9b]
172.16.1.1 32768 i
*> [3]:[0]:[32]:[172.16.1.1]
172.16.1.1 32768 i
* [3]:[0]:[32]:[172.16.1.4]
172.16.1.4 0 65000 65002 i
* [3]:[0]:[32]:[172.16.1.4]
172.16.1.4 0 65000 65002 i
* [3]:[0]:[32]:[172.16.1.4]
172.16.1.4 0 65000 65002 i
*> [3]:[0]:[32]:[172.16.1.4]
172.16.1.4 0 65000 65002 i
Displayed 4 prefixes (10 paths)
VNI: 10100
BGP table version is 4, local router ID is 172.16.1.3
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete
EVPN type-2 prefix: [2]:[ESI]:[EthTag]:[MAClen]:[MAC]:[IPlen]:[IP]
EVPN type-3 prefix: [3]:[EthTag]:[IPlen]:[OrigIP]
EVPN type-5 prefix: [5]:[ESI]:[EthTag]:[IPlen]:[IP]
Network Next Hop Metric LocPrf Weight Path
* [2]:[0]:[0]:[48]:[4e:4f:c1:03:32:f3]
172.16.1.4 0 65000 65002 i
* [2]:[0]:[0]:[48]:[4e:4f:c1:03:32:f3]
172.16.1.4 0 65000 65002 i
* [2]:[0]:[0]:[48]:[4e:4f:c1:03:32:f3]
172.16.1.4 0 65000 65002 i
*> [2]:[0]:[0]:[48]:[4e:4f:c1:03:32:f3]
172.16.1.4 0 65000 65002 i
*> [2]:[0]:[0]:[48]:[a2:af:dc:0a:cd:6c]
172.16.1.1 32768 i
*> [3]:[0]:[32]:[172.16.1.1]
172.16.1.1 32768 i
* [3]:[0]:[32]:[172.16.1.4]
172.16.1.4 0 65000 65002 i
* [3]:[0]:[32]:[172.16.1.4]
172.16.1.4 0 65000 65002 i
* [3]:[0]:[32]:[172.16.1.4]
172.16.1.4 0 65000 65002 i
*> [3]:[0]:[32]:[172.16.1.4]
172.16.1.4 0 65000 65002 i
Displayed 4 prefixes (10 paths)
root@spine02:~# net show bgp evpn summary
BGP router identifier 172.16.1.3, local AS number 65001 vrf-id 0
BGP table version 0
RIB entries 15, using 2280 bytes of memory
Peers 2, using 39 KiB of memory
Peer groups 1, using 64 bytes of memory
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd
BB01(10.10.10.5) 4 65000 393 387 0 0 0 00:18:05 8
BB02(10.10.10.21) 4 65000 390 385 0 0 0 00:18:03 8
Total number of neighbors 2
root@spine02:~# net show bgp evpn vni
Advertise Gateway Macip: Disabled
Advertise All VNI flag: Enabled
BUM flooding: Head-end replication
Number of L2 VNIs: 2
Number of L3 VNIs: 0
Flags: * - Kernel
VNI Type RD Import RT Export RT Tenant VRF
* 10200 L2 172.16.1.3:2 65001:10200 65001:10200 default
* 10100 L2 172.16.1.3:3 65001:10100 65001:10100 default
root@spine02:~# net show route
show ip route
=============
Codes: K - kernel route, C - connected, S - static, R - RIP,
O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
F - PBR,
> - selected route, * - FIB route
B>* 10.10.10.0/30 [20/0] via 10.10.10.5, swp1, 00:18:13
C>* 10.10.10.4/30 is directly connected, swp1, 00:18:19
B>* 10.10.10.8/30 [20/0] via 10.10.10.5, swp1, 00:18:13
B>* 10.10.10.12/30 [20/0] via 10.10.10.5, swp1, 00:18:13
B>* 10.10.10.16/30 [20/0] via 10.10.10.21, swp2, 00:18:11
C>* 10.10.10.20/30 is directly connected, swp2, 00:18:19
B>* 10.10.10.24/30 [20/0] via 10.10.10.21, swp2, 00:18:11
B>* 10.10.10.28/30 [20/0] via 10.10.10.21, swp2, 00:18:11
C>* 169.254.1.0/30 is directly connected, peerlink.4094, 00:18:11
C>* 172.16.1.1/32 is directly connected, lo, 00:18:07
C>* 172.16.1.3/32 is directly connected, lo, 00:18:19
B>* 172.16.1.4/32 [20/0] via 10.10.10.5, swp1, 00:18:07
* via 10.10.10.21, swp2, 00:18:07
B>* 172.16.1.5/32 [20/0] via 10.10.10.5, swp1, 00:18:11
* via 10.10.10.21, swp2, 00:18:11
B>* 172.16.1.6/32 [20/0] via 10.10.10.5, swp1, 00:18:11
* via 10.10.10.21, swp2, 00:18:11
B>* 172.16.222.1/32 [20/0] via 10.10.10.5, swp1, 00:18:13
B>* 172.16.222.2/32 [20/0] via 10.10.10.21, swp2, 00:18:11
show ipv6 route
===============
Codes: K - kernel route, C - connected, S - static, R - RIPng,
O - OSPFv3, I - IS-IS, B - BGP, N - NHRP, T - Table,
v - VNC, V - VNC-Direct, A - Babel, D - SHARP, F - PBR,
> - selected route, * - FIB route
C * fe80::/64 is directly connected, bridge, 00:18:09
C * fe80::/64 is directly connected, peerlink.4094, 00:18:09
C * fe80::/64 is directly connected, swp1, 00:18:18
C * fe80::/64 is directly connected, swp2, 00:18:19
C>* fe80::/64 is directly connected, eth0, 00:18:19
root@spine02:~# net show bridge macs
VLAN Master Interface MAC TunnelDest State Flags LastSeen
-------- ------ ------------- ----------------- ---------- --------- ------------- --------
1 bridge bond-to-ToR01 0c:1e:ed:9f:32:02 00:00:01
1 bridge bond-to-ToR01 0e:1e:ed:9f:32:01 00:00:16
1 bridge bond-to-ToR01 0e:1e:ed:9f:32:02 00:00:10
100 bridge bond-to-ToR01 a2:af:dc:0a:cd:6c 00:00:16
100 bridge vxlan10100 4e:4f:c1:03:32:f3 offload 00:14:23
200 bridge bond-to-ToR01 7a:a1:d2:fa:86:9b 00:14:14
200 bridge vxlan10200 12:ed:20:2e:b3:1a offload 00:14:14
untagged vxlan10100 00:00:00:00:00:00 172.16.1.4 permanent self 00:18:15
untagged vxlan10100 4e:4f:c1:03:32:f3 172.16.1.4 self, offload 00:14:23
untagged vxlan10200 00:00:00:00:00:00 172.16.1.4 permanent self 00:18:15
untagged vxlan10200 12:ed:20:2e:b3:1a 172.16.1.4 self, offload 00:14:14
untagged bridge bond-to-ToR01 0c:1e:ed:b9:90:05 permanent 00:18:27
untagged bridge peerlink 0c:1e:ed:b9:90:03 permanent 00:18:27
untagged bridge vxlan10100 f6:01:80:3a:ab:8d permanent 00:18:27
untagged bridge vxlan10200 4a:c7:eb:94:82:ad permanent 00:18:27
root@spine02:~#
spine03
root@spine03:~# net show evpn
L2 VNIs: 2
L3 VNIs: 0
Advertise gateway mac-ip: No
Duplicate address detection: Enable
Detection max-moves 5, time 180
root@spine03:~# net show evpn mac vni all
VNI 10200 #MACs (local and remote) 2
MAC Type Intf/Remote VTEP VLAN
12:ed:20:2e:b3:1a local bond-to-ToR02 200
7a:a1:d2:fa:86:9b remote 172.16.1.1
VNI 10100 #MACs (local and remote) 2
MAC Type Intf/Remote VTEP VLAN
a2:af:dc:0a:cd:6c remote 172.16.1.1
4e:4f:c1:03:32:f3 local bond-to-ToR02 100
root@spine03:~# net show evpn vni detail
VNI: 10200
Type: L2
Tenant VRF: default
VxLAN interface: vxlan10200
VxLAN ifIndex: 9
Local VTEP IP: 172.16.1.4
Remote VTEPs for this VNI:
172.16.1.1
Number of MACs (local and remote) known for this VNI: 2
Number of ARPs (IPv4 and IPv6, local and remote) known for this VNI: 0
Advertise-gw-macip: No
VNI: 10100
Type: L2
Tenant VRF: default
VxLAN interface: vxlan10100
VxLAN ifIndex: 11
Local VTEP IP: 172.16.1.4
Remote VTEPs for this VNI:
172.16.1.1
Number of MACs (local and remote) known for this VNI: 2
Number of ARPs (IPv4 and IPv6, local and remote) known for this VNI: 0
Advertise-gw-macip: No
root@spine03:~# net show bgp
show bgp ipv4 unicast
=====================
BGP table version is 18, local router ID is 172.16.1.5
Status codes: s suppressed, d damped, h history, * valid, > best, = multipath,
i internal, r RIB-failure, S Stale, R Removed
Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Weight Path
* 10.10.10.0/30 10.10.10.25 0 65000 65001 ?
*> 10.10.10.9 0 0 65000 ?
*> 10.10.10.4/30 10.10.10.9 0 0 65000 ?
* 10.10.10.25 0 65000 65001 ?
* 10.10.10.8/30 10.10.10.9 0 0 65000 ?
*> 0.0.0.0 0 32768 ?
*> 10.10.10.12/30 10.10.10.9 0 0 65000 ?
* 10.10.10.16/30 10.10.10.9 0 65000 65001 ?
*> 10.10.10.25 0 0 65000 ?
* 10.10.10.20/30 10.10.10.9 0 65000 65001 ?
*> 10.10.10.25 0 0 65000 ?
* 10.10.10.24/30 10.10.10.25 0 0 65000 ?
*> 0.0.0.0 0 32768 ?
*> 10.10.10.28/30 10.10.10.25 0 0 65000 ?
*> 169.254.1.0/30 0.0.0.0 0 32768 ?
*= 172.16.1.1/32 10.10.10.25 0 65000 65001 ?
*> 10.10.10.9 0 65000 65001 ?
*= 172.16.1.2/32 10.10.10.25 0 65000 65001 ?
*> 10.10.10.9 0 65000 65001 ?
*= 172.16.1.3/32 10.10.10.9 0 65000 65001 ?
*> 10.10.10.25 0 65000 65001 ?
*> 172.16.1.4/32 0.0.0.0 0 32768 ?
*> 172.16.1.5/32 0.0.0.0 0 32768 ?
*> 172.16.222.1/32 10.10.10.9 0 0 65000 ?
*> 172.16.222.2/32 10.10.10.25 0 0 65000 ?
Displayed 16 routes and 25 total paths
show bgp ipv6 unicast
=====================
No BGP prefixes displayed, 0 exist
root@spine03:~# net show bgp evpn route vni all
VNI: 10200
BGP table version is 4, local router ID is 172.16.1.5
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete
EVPN type-2 prefix: [2]:[ESI]:[EthTag]:[MAClen]:[MAC]:[IPlen]:[IP]
EVPN type-3 prefix: [3]:[EthTag]:[IPlen]:[OrigIP]
EVPN type-5 prefix: [5]:[ESI]:[EthTag]:[IPlen]:[IP]
Network Next Hop Metric LocPrf Weight Path
*> [2]:[0]:[0]:[48]:[12:ed:20:2e:b3:1a]
172.16.1.4 32768 i
* [2]:[0]:[0]:[48]:[7a:a1:d2:fa:86:9b]
172.16.1.1 0 65000 65001 i
* [2]:[0]:[0]:[48]:[7a:a1:d2:fa:86:9b]
172.16.1.1 0 65000 65001 i
* [2]:[0]:[0]:[48]:[7a:a1:d2:fa:86:9b]
172.16.1.1 0 65000 65001 i
*> [2]:[0]:[0]:[48]:[7a:a1:d2:fa:86:9b]
172.16.1.1 0 65000 65001 i
* [3]:[0]:[32]:[172.16.1.1]
172.16.1.1 0 65000 65001 i
* [3]:[0]:[32]:[172.16.1.1]
172.16.1.1 0 65000 65001 i
* [3]:[0]:[32]:[172.16.1.1]
172.16.1.1 0 65000 65001 i
*> [3]:[0]:[32]:[172.16.1.1]
172.16.1.1 0 65000 65001 i
*> [3]:[0]:[32]:[172.16.1.4]
172.16.1.4 32768 i
Displayed 4 prefixes (10 paths)
VNI: 10100
BGP table version is 4, local router ID is 172.16.1.5
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete
EVPN type-2 prefix: [2]:[ESI]:[EthTag]:[MAClen]:[MAC]:[IPlen]:[IP]
EVPN type-3 prefix: [3]:[EthTag]:[IPlen]:[OrigIP]
EVPN type-5 prefix: [5]:[ESI]:[EthTag]:[IPlen]:[IP]
Network Next Hop Metric LocPrf Weight Path
*> [2]:[0]:[0]:[48]:[4e:4f:c1:03:32:f3]
172.16.1.4 32768 i
* [2]:[0]:[0]:[48]:[a2:af:dc:0a:cd:6c]
172.16.1.1 0 65000 65001 i
* [2]:[0]:[0]:[48]:[a2:af:dc:0a:cd:6c]
172.16.1.1 0 65000 65001 i
* [2]:[0]:[0]:[48]:[a2:af:dc:0a:cd:6c]
172.16.1.1 0 65000 65001 i
*> [2]:[0]:[0]:[48]:[a2:af:dc:0a:cd:6c]
172.16.1.1 0 65000 65001 i
* [3]:[0]:[32]:[172.16.1.1]
172.16.1.1 0 65000 65001 i
* [3]:[0]:[32]:[172.16.1.1]
172.16.1.1 0 65000 65001 i
* [3]:[0]:[32]:[172.16.1.1]
172.16.1.1 0 65000 65001 i
*> [3]:[0]:[32]:[172.16.1.1]
172.16.1.1 0 65000 65001 i
*> [3]:[0]:[32]:[172.16.1.4]
172.16.1.4 32768 i
Displayed 4 prefixes (10 paths)
root@spine03:~# net show bgp evpn summary
BGP router identifier 172.16.1.5, local AS number 65002 vrf-id 0
BGP table version 0
RIB entries 15, using 2280 bytes of memory
Peers 2, using 39 KiB of memory
Peer groups 1, using 64 bytes of memory
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd
BB01(10.10.10.9) 4 65000 390 385 0 0 0 00:18:03 8
BB02(10.10.10.25) 4 65000 390 385 0 0 0 00:18:03 8
Total number of neighbors 2
root@spine03:~# net show bgp evpn vni
Advertise Gateway Macip: Disabled
Advertise All VNI flag: Enabled
BUM flooding: Head-end replication
Number of L2 VNIs: 2
Number of L3 VNIs: 0
Flags: * - Kernel
VNI Type RD Import RT Export RT Tenant VRF
* 10200 L2 172.16.1.5:2 65002:10200 65002:10200 default
* 10100 L2 172.16.1.5:3 65002:10100 65002:10100 default
root@spine03:~# net show route
show ip route
=============
Codes: K - kernel route, C - connected, S - static, R - RIP,
O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
F - PBR,
> - selected route, * - FIB route
B>* 10.10.10.0/30 [20/0] via 10.10.10.9, swp1, 00:18:10
B>* 10.10.10.4/30 [20/0] via 10.10.10.9, swp1, 00:18:10
C>* 10.10.10.8/30 is directly connected, swp1, 00:18:14
B>* 10.10.10.12/30 [20/0] via 10.10.10.9, swp1, 00:18:10
B>* 10.10.10.16/30 [20/0] via 10.10.10.25, swp2, 00:18:11
B>* 10.10.10.20/30 [20/0] via 10.10.10.25, swp2, 00:18:11
C>* 10.10.10.24/30 is directly connected, swp2, 00:18:14
B>* 10.10.10.28/30 [20/0] via 10.10.10.25, swp2, 00:18:11
C>* 169.254.1.0/30 is directly connected, peerlink.4094, 00:18:12
B>* 172.16.1.1/32 [20/0] via 10.10.10.9, swp1, 00:18:07
* via 10.10.10.25, swp2, 00:18:07
B>* 172.16.1.2/32 [20/0] via 10.10.10.9, swp1, 00:18:08
* via 10.10.10.25, swp2, 00:18:08
B>* 172.16.1.3/32 [20/0] via 10.10.10.25, swp2, 00:18:10
* via 10.10.10.9, swp1, 00:18:10
C>* 172.16.1.4/32 is directly connected, lo, 00:18:02
C>* 172.16.1.5/32 is directly connected, lo, 00:18:14
B>* 172.16.222.1/32 [20/0] via 10.10.10.9, swp1, 00:18:10
B>* 172.16.222.2/32 [20/0] via 10.10.10.25, swp2, 00:18:11
show ipv6 route
===============
Codes: K - kernel route, C - connected, S - static, R - RIPng,
O - OSPFv3, I - IS-IS, B - BGP, N - NHRP, T - Table,
v - VNC, V - VNC-Direct, A - Babel, D - SHARP, F - PBR,
> - selected route, * - FIB route
C * fe80::/64 is directly connected, bridge, 00:18:10
C * fe80::/64 is directly connected, peerlink.4094, 00:18:10
C * fe80::/64 is directly connected, swp1, 00:18:13
C * fe80::/64 is directly connected, swp2, 00:18:13
C>* fe80::/64 is directly connected, eth0, 00:18:14
root@spine03:~# net show bridge macs
VLAN Master Interface MAC TunnelDest State Flags LastSeen
-------- ------ ------------- ----------------- ---------- --------- ------------- --------
1 bridge bond-to-ToR02 0c:1e:ed:2c:26:02 00:00:11
1 bridge bond-to-ToR02 0e:1e:ed:2c:26:01 00:00:05
1 bridge bond-to-ToR02 0e:1e:ed:2c:26:02 00:00:11
100 bridge bond-to-ToR02 4e:4f:c1:03:32:f3 00:11:11
100 bridge vxlan10100 a2:af:dc:0a:cd:6c offload 00:14:23
200 bridge bond-to-ToR02 12:ed:20:2e:b3:1a 00:10:08
200 bridge vxlan10200 7a:a1:d2:fa:86:9b offload 00:14:14
untagged vxlan10100 00:00:00:00:00:00 172.16.1.1 permanent self 00:18:16
untagged vxlan10100 a2:af:dc:0a:cd:6c 172.16.1.1 self, offload 00:14:23
untagged vxlan10200 00:00:00:00:00:00 172.16.1.1 permanent self 00:18:16
untagged vxlan10200 7a:a1:d2:fa:86:9b 172.16.1.1 self, offload 00:14:14
untagged bridge bond-to-ToR02 0c:1e:ed:49:da:05 permanent 00:18:21
untagged bridge peerlink 0c:1e:ed:49:da:03 permanent 00:18:21
untagged bridge vxlan10100 1e:92:4b:a3:78:2e permanent 00:18:21
untagged bridge vxlan10200 d2:ba:54:1d:df:35 permanent 00:18:21
root@spine03:~#
spine04
oot@spine04:~# net show evpn
L2 VNIs: 2
L3 VNIs: 0
Advertise gateway mac-ip: No
Duplicate address detection: Enable
Detection max-moves 5, time 180
root@spine04:~# net show evpn mac vni all
VNI 10200 #MACs (local and remote) 2
MAC Type Intf/Remote VTEP VLAN
12:ed:20:2e:b3:1a local bond-to-ToR02 200
7a:a1:d2:fa:86:9b remote 172.16.1.1
VNI 10100 #MACs (local and remote) 2
MAC Type Intf/Remote VTEP VLAN
a2:af:dc:0a:cd:6c remote 172.16.1.1
4e:4f:c1:03:32:f3 local bond-to-ToR02 100
root@spine04:~# net show evpn vni detail
VNI: 10200
Type: L2
Tenant VRF: default
VxLAN interface: vxlan10200
VxLAN ifIndex: 9
Local VTEP IP: 172.16.1.4
Remote VTEPs for this VNI:
172.16.1.1
Number of MACs (local and remote) known for this VNI: 2
Number of ARPs (IPv4 and IPv6, local and remote) known for this VNI: 0
Advertise-gw-macip: No
VNI: 10100
Type: L2
Tenant VRF: default
VxLAN interface: vxlan10100
VxLAN ifIndex: 11
Local VTEP IP: 172.16.1.4
Remote VTEPs for this VNI:
172.16.1.1
Number of MACs (local and remote) known for this VNI: 2
Number of ARPs (IPv4 and IPv6, local and remote) known for this VNI: 0
Advertise-gw-macip: No
root@spine04:~# net show bgp
show bgp ipv4 unicast
=====================
BGP table version is 18, local router ID is 172.16.1.6
Status codes: s suppressed, d damped, h history, * valid, > best, = multipath,
i internal, r RIB-failure, S Stale, R Removed
Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Weight Path
* 10.10.10.0/30 10.10.10.29 0 65000 65001 ?
*> 10.10.10.13 0 0 65000 ?
* 10.10.10.4/30 10.10.10.29 0 65000 65001 ?
*> 10.10.10.13 0 0 65000 ?
*> 10.10.10.8/30 10.10.10.13 0 0 65000 ?
* 10.10.10.12/30 10.10.10.13 0 0 65000 ?
*> 0.0.0.0 0 32768 ?
* 10.10.10.16/30 10.10.10.13 0 65000 65001 ?
*> 10.10.10.29 0 0 65000 ?
*> 10.10.10.20/30 10.10.10.29 0 0 65000 ?
* 10.10.10.13 0 65000 65001 ?
*> 10.10.10.24/30 10.10.10.29 0 0 65000 ?
* 10.10.10.28/30 10.10.10.29 0 0 65000 ?
*> 0.0.0.0 0 32768 ?
*> 169.254.1.0/30 0.0.0.0 0 32768 ?
*= 172.16.1.1/32 10.10.10.29 0 65000 65001 ?
*> 10.10.10.13 0 65000 65001 ?
*= 172.16.1.2/32 10.10.10.29 0 65000 65001 ?
*> 10.10.10.13 0 65000 65001 ?
*= 172.16.1.3/32 10.10.10.29 0 65000 65001 ?
*> 10.10.10.13 0 65000 65001 ?
*> 172.16.1.4/32 0.0.0.0 0 32768 ?
*> 172.16.1.6/32 0.0.0.0 0 32768 ?
*> 172.16.222.1/32 10.10.10.13 0 0 65000 ?
*> 172.16.222.2/32 10.10.10.29 0 0 65000 ?
Displayed 16 routes and 25 total paths
show bgp ipv6 unicast
=====================
No BGP prefixes displayed, 0 exist
root@spine04:~# net show bgp evpn route vni all
VNI: 10200
BGP table version is 4, local router ID is 172.16.1.6
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete
EVPN type-2 prefix: [2]:[ESI]:[EthTag]:[MAClen]:[MAC]:[IPlen]:[IP]
EVPN type-3 prefix: [3]:[EthTag]:[IPlen]:[OrigIP]
EVPN type-5 prefix: [5]:[ESI]:[EthTag]:[IPlen]:[IP]
Network Next Hop Metric LocPrf Weight Path
*> [2]:[0]:[0]:[48]:[12:ed:20:2e:b3:1a]
172.16.1.4 32768 i
* [2]:[0]:[0]:[48]:[7a:a1:d2:fa:86:9b]
172.16.1.1 0 65000 65001 i
* [2]:[0]:[0]:[48]:[7a:a1:d2:fa:86:9b]
172.16.1.1 0 65000 65001 i
* [2]:[0]:[0]:[48]:[7a:a1:d2:fa:86:9b]
172.16.1.1 0 65000 65001 i
*> [2]:[0]:[0]:[48]:[7a:a1:d2:fa:86:9b]
172.16.1.1 0 65000 65001 i
* [3]:[0]:[32]:[172.16.1.1]
172.16.1.1 0 65000 65001 i
* [3]:[0]:[32]:[172.16.1.1]
172.16.1.1 0 65000 65001 i
* [3]:[0]:[32]:[172.16.1.1]
172.16.1.1 0 65000 65001 i
*> [3]:[0]:[32]:[172.16.1.1]
172.16.1.1 0 65000 65001 i
*> [3]:[0]:[32]:[172.16.1.4]
172.16.1.4 32768 i
Displayed 4 prefixes (10 paths)
VNI: 10100
BGP table version is 4, local router ID is 172.16.1.6
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete
EVPN type-2 prefix: [2]:[ESI]:[EthTag]:[MAClen]:[MAC]:[IPlen]:[IP]
EVPN type-3 prefix: [3]:[EthTag]:[IPlen]:[OrigIP]
EVPN type-5 prefix: [5]:[ESI]:[EthTag]:[IPlen]:[IP]
Network Next Hop Metric LocPrf Weight Path
*> [2]:[0]:[0]:[48]:[4e:4f:c1:03:32:f3]
172.16.1.4 32768 i
* [2]:[0]:[0]:[48]:[a2:af:dc:0a:cd:6c]
172.16.1.1 0 65000 65001 i
* [2]:[0]:[0]:[48]:[a2:af:dc:0a:cd:6c]
172.16.1.1 0 65000 65001 i
* [2]:[0]:[0]:[48]:[a2:af:dc:0a:cd:6c]
172.16.1.1 0 65000 65001 i
*> [2]:[0]:[0]:[48]:[a2:af:dc:0a:cd:6c]
172.16.1.1 0 65000 65001 i
* [3]:[0]:[32]:[172.16.1.1]
172.16.1.1 0 65000 65001 i
* [3]:[0]:[32]:[172.16.1.1]
172.16.1.1 0 65000 65001 i
* [3]:[0]:[32]:[172.16.1.1]
172.16.1.1 0 65000 65001 i
*> [3]:[0]:[32]:[172.16.1.1]
172.16.1.1 0 65000 65001 i
*> [3]:[0]:[32]:[172.16.1.4]
172.16.1.4 32768 i
Displayed 4 prefixes (10 paths)
root@spine04:~# net show bgp evpn summary
BGP router identifier 172.16.1.6, local AS number 65002 vrf-id 0
BGP table version 0
RIB entries 15, using 2280 bytes of memory
Peers 2, using 39 KiB of memory
Peer groups 1, using 64 bytes of memory
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd
BB01(10.10.10.13) 4 65000 393 388 0 0 0 00:18:05 8
BB02(10.10.10.29) 4 65000 390 387 0 0 0 00:18:03 8
Total number of neighbors 2
root@spine04:~# net show bgp evpn vni
Advertise Gateway Macip: Disabled
Advertise All VNI flag: Enabled
BUM flooding: Head-end replication
Number of L2 VNIs: 2
Number of L3 VNIs: 0
Flags: * - Kernel
VNI Type RD Import RT Export RT Tenant VRF
* 10200 L2 172.16.1.6:2 65002:10200 65002:10200 default
* 10100 L2 172.16.1.6:3 65002:10100 65002:10100 default
root@spine04:~# net show route
show ip route
=============
Codes: K - kernel route, C - connected, S - static, R - RIP,
O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
F - PBR,
> - selected route, * - FIB route
B>* 10.10.10.0/30 [20/0] via 10.10.10.13, swp1, 00:18:12
B>* 10.10.10.4/30 [20/0] via 10.10.10.13, swp1, 00:18:12
B>* 10.10.10.8/30 [20/0] via 10.10.10.13, swp1, 00:18:12
C>* 10.10.10.12/30 is directly connected, swp1, 00:18:19
B>* 10.10.10.16/30 [20/0] via 10.10.10.29, swp2, 00:18:11
B>* 10.10.10.20/30 [20/0] via 10.10.10.29, swp2, 00:18:11
B>* 10.10.10.24/30 [20/0] via 10.10.10.29, swp2, 00:18:11
C>* 10.10.10.28/30 is directly connected, swp2, 00:18:19
C>* 169.254.1.0/30 is directly connected, peerlink.4094, 00:18:12
B>* 172.16.1.1/32 [20/0] via 10.10.10.13, swp1, 00:18:06
* via 10.10.10.29, swp2, 00:18:06
B>* 172.16.1.2/32 [20/0] via 10.10.10.13, swp1, 00:18:08
* via 10.10.10.29, swp2, 00:18:08
B>* 172.16.1.3/32 [20/0] via 10.10.10.13, swp1, 00:18:11
* via 10.10.10.29, swp2, 00:18:11
C>* 172.16.1.4/32 is directly connected, lo, 00:18:06
C>* 172.16.1.6/32 is directly connected, lo, 00:18:19
B>* 172.16.222.1/32 [20/0] via 10.10.10.13, swp1, 00:18:12
B>* 172.16.222.2/32 [20/0] via 10.10.10.29, swp2, 00:18:11
show ipv6 route
===============
Codes: K - kernel route, C - connected, S - static, R - RIPng,
O - OSPFv3, I - IS-IS, B - BGP, N - NHRP, T - Table,
v - VNC, V - VNC-Direct, A - Babel, D - SHARP, F - PBR,
> - selected route, * - FIB route
C * fe80::/64 is directly connected, bridge, 00:18:07
C * fe80::/64 is directly connected, peerlink.4094, 00:18:10
C * fe80::/64 is directly connected, swp1, 00:18:18
C * fe80::/64 is directly connected, swp2, 00:18:18
C>* fe80::/64 is directly connected, eth0, 00:18:19
root@spine04:~# net show bridge macs
VLAN Master Interface MAC TunnelDest State Flags LastSeen
-------- ------ ------------- ----------------- ---------- --------- ------------- --------
1 bridge bond-to-ToR02 0c:1e:ed:2c:26:02 <1 sec
1 bridge bond-to-ToR02 0e:1e:ed:2c:26:01 00:00:15
1 bridge bond-to-ToR02 0e:1e:ed:2c:26:02 00:00:10
100 bridge bond-to-ToR02 4e:4f:c1:03:32:f3 00:00:15
100 bridge vxlan10100 a2:af:dc:0a:cd:6c offload 00:14:23
200 bridge bond-to-ToR02 12:ed:20:2e:b3:1a 00:00:15
200 bridge vxlan10200 7a:a1:d2:fa:86:9b offload 00:14:14
untagged vxlan10100 00:00:00:00:00:00 172.16.1.1 permanent self 00:18:16
untagged vxlan10100 a2:af:dc:0a:cd:6c 172.16.1.1 self, offload 00:14:23
untagged vxlan10200 00:00:00:00:00:00 172.16.1.1 permanent self 00:18:16
untagged vxlan10200 7a:a1:d2:fa:86:9b 172.16.1.1 self, offload 00:14:14
untagged bridge bond-to-ToR02 0c:1e:ed:5a:03:05 permanent 00:18:27
untagged bridge peerlink 0c:1e:ed:5a:03:03 permanent 00:18:27
untagged bridge vxlan10100 76:42:cb:ab:0a:84 permanent 00:18:27
untagged bridge vxlan10200 2e:70:48:a6:59:a6 permanent 00:18:27
root@spine04:~#
VNIを持たないBBたちもMP-BGP EVPNへ参加し、各VTEPへのルーティングテーブルが確認できることがわかります
BB01
root@BB01:~# net show bgp evpn route
BGP table version is 2, local router ID is 172.16.222.1
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete
EVPN type-2 prefix: [2]:[ESI]:[EthTag]:[MAClen]:[MAC]:[IPlen]:[IP]
EVPN type-3 prefix: [3]:[EthTag]:[IPlen]:[OrigIP]
EVPN type-5 prefix: [5]:[ESI]:[EthTag]:[IPlen]:[IP]
Network Next Hop Metric LocPrf Weight Path
Route Distinguisher: 172.16.1.2:2
*> [2]:[0]:[0]:[48]:[7a:a1:d2:fa:86:9b]
172.16.1.1 0 65001 i
*> [3]:[0]:[32]:[172.16.1.1]
172.16.1.1 0 65001 i
Route Distinguisher: 172.16.1.2:3
*> [2]:[0]:[0]:[48]:[a2:af:dc:0a:cd:6c]
172.16.1.1 0 65001 i
*> [3]:[0]:[32]:[172.16.1.1]
172.16.1.1 0 65001 i
Route Distinguisher: 172.16.1.3:2
*> [2]:[0]:[0]:[48]:[7a:a1:d2:fa:86:9b]
172.16.1.1 0 65001 i
*> [3]:[0]:[32]:[172.16.1.1]
172.16.1.1 0 65001 i
Route Distinguisher: 172.16.1.3:3
*> [2]:[0]:[0]:[48]:[a2:af:dc:0a:cd:6c]
172.16.1.1 0 65001 i
*> [3]:[0]:[32]:[172.16.1.1]
172.16.1.1 0 65001 i
Route Distinguisher: 172.16.1.5:2
*> [2]:[0]:[0]:[48]:[12:ed:20:2e:b3:1a]
172.16.1.4 0 65002 i
*> [3]:[0]:[32]:[172.16.1.4]
172.16.1.4 0 65002 i
Route Distinguisher: 172.16.1.5:3
*> [2]:[0]:[0]:[48]:[4e:4f:c1:03:32:f3]
172.16.1.4 0 65002 i
*> [3]:[0]:[32]:[172.16.1.4]
172.16.1.4 0 65002 i
Route Distinguisher: 172.16.1.6:2
*> [2]:[0]:[0]:[48]:[12:ed:20:2e:b3:1a]
172.16.1.4 0 65002 i
*> [3]:[0]:[32]:[172.16.1.4]
172.16.1.4 0 65002 i
Route Distinguisher: 172.16.1.6:3
*> [2]:[0]:[0]:[48]:[4e:4f:c1:03:32:f3]
172.16.1.4 0 65002 i
*> [3]:[0]:[32]:[172.16.1.4]
172.16.1.4 0 65002 i
Displayed 16 prefixes (16 paths)
root@BB01:~#
BB02
root@BB02:~# net show bgp evpn route
BGP table version is 2, local router ID is 172.16.222.2
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete
EVPN type-2 prefix: [2]:[ESI]:[EthTag]:[MAClen]:[MAC]:[IPlen]:[IP]
EVPN type-3 prefix: [3]:[EthTag]:[IPlen]:[OrigIP]
EVPN type-5 prefix: [5]:[ESI]:[EthTag]:[IPlen]:[IP]
Network Next Hop Metric LocPrf Weight Path
Route Distinguisher: 172.16.1.2:2
*> [2]:[0]:[0]:[48]:[7a:a1:d2:fa:86:9b]
172.16.1.1 0 65001 i
*> [3]:[0]:[32]:[172.16.1.1]
172.16.1.1 0 65001 i
Route Distinguisher: 172.16.1.2:3
*> [2]:[0]:[0]:[48]:[a2:af:dc:0a:cd:6c]
172.16.1.1 0 65001 i
*> [3]:[0]:[32]:[172.16.1.1]
172.16.1.1 0 65001 i
Route Distinguisher: 172.16.1.3:2
*> [2]:[0]:[0]:[48]:[7a:a1:d2:fa:86:9b]
172.16.1.1 0 65001 i
*> [3]:[0]:[32]:[172.16.1.1]
172.16.1.1 0 65001 i
Route Distinguisher: 172.16.1.3:3
*> [2]:[0]:[0]:[48]:[a2:af:dc:0a:cd:6c]
172.16.1.1 0 65001 i
*> [3]:[0]:[32]:[172.16.1.1]
172.16.1.1 0 65001 i
Route Distinguisher: 172.16.1.5:2
*> [2]:[0]:[0]:[48]:[12:ed:20:2e:b3:1a]
172.16.1.4 0 65002 i
*> [3]:[0]:[32]:[172.16.1.4]
172.16.1.4 0 65002 i
Route Distinguisher: 172.16.1.5:3
*> [2]:[0]:[0]:[48]:[4e:4f:c1:03:32:f3]
172.16.1.4 0 65002 i
*> [3]:[0]:[32]:[172.16.1.4]
172.16.1.4 0 65002 i
Route Distinguisher: 172.16.1.6:2
*> [2]:[0]:[0]:[48]:[12:ed:20:2e:b3:1a]
172.16.1.4 0 65002 i
*> [3]:[0]:[32]:[172.16.1.4]
172.16.1.4 0 65002 i
Route Distinguisher: 172.16.1.6:3
*> [2]:[0]:[0]:[48]:[4e:4f:c1:03:32:f3]
172.16.1.4 0 65002 i
*> [3]:[0]:[32]:[172.16.1.4]
172.16.1.4 0 65002 i
Displayed 16 prefixes (16 paths)
root@BB02:~#
- ControlPlane
route type-2
route type-3
- DataPlane
arp request
vxlanヘッダを見るとsourceアドレスがspine01/02のVTEPアドレス(172.16.1.1)になり、
destinationアドレスがspine03/04のVTEPアドレス(172.16.1.4)となることから、
mlagを組んでいる2台で論理VTEPを使って通信してることがわかります
VTEPがBUM(Broadcast,Unknown unicast,Multicast packet)パケットを受信すると、
1.decapしローカルに接続されているホストに配信
2.再度encapしvxlanの他のVTEPにパケットを送信
いわゆるHead End Replicationの動作ですね
arp reply
ping request
ping reply
参考資料
https://docs.cumulusnetworks.com/display/DOCS/Ethernet+Virtual+Private+Network+-+EVPN
https://docs.cumulusnetworks.com/display/DOCS/Anycast+Design+Guide
https://community.mellanox.com/s/article/howto-configure-vxlan-with-mlag-using-cumulus-linux
http://kakkotetsu.hatenablog.com/entry/2017/05/11/001708#spine%E5%85%A8%E5%8F%B0%E3%81%ABVXLAN-VNI%E8%A8%AD%E5%AE%9A
https://tools.ietf.org/html/rfc7432