やりたいこと

vxlan/evpnはコントローラー無しでoverlayネットワークを実装できる利点
EVPN asymmetric vs symmetric model比較の前に、vxlan/evpnの基本動作、トラフィックフロー確認など
基礎知識も踏まえてroute type 2/3の確認
multi-homing機能を前提としたroute type 1/4はスコープ外とする
なので、MLAGを用いたVXLAN Active-active Modeを使用
route type-5は次会で、、、

環境

GNS3 2.1.15
GNS3 VM (6vCPUs、6G RAM)
Cumulus VX 3.7.3
Ubuntu Docker Guest

ToRはOVSで代用可能(lacpとvlan喋ればなんでもいい)
Ubuntu Docker GuestもvPCで代用可能(IP設定してping送れば)

underlay layer3

各ベンダーの例はほとんどがBGP unnumberedで設定しているが、運用上わかりやすく /30アドレスで設定
環境に合わせてfast-external-failoverとBFDを適用してもいいかも
各loopbackアドレスを広報するだけ、

underlay設定

初期ログインは**cumulus/CumulusLinux!**でログイン
rootアカウントのパスワード変更

cumulus全台

cumulus@cumulus:~$ sudo passwd root
[sudo] password for cumulus:
Enter new UNIX password:
Retype new UNIX password:
passwd: password updated successfully
cumulus@cumulus:~$

rootアカウントでログイン後underlayから設定していく

BB01

#hostname/各IFの設定
net add hostname BB01
net add interface swp1 alias to-spine01-swp1
net add interface swp1 mtu 9000
net add interface swp1 ip address 10.10.10.1/30
net add interface swp2 alias to-spine02-swp1
net add interface swp2 mtu 9000
net add interface swp2 ip address 10.10.10.5/30
net add interface swp3 alias to-spine03-swp1
net add interface swp3 mtu 9000
net add interface swp3 ip address 10.10.10.9/30
net add interface swp4 alias to-spine04-swp1
net add interface swp4 mtu 9000
net add interface swp4 ip address 10.10.10.13/30
net add loopback lo ip address 172.16.222.1/32

#underlay-bgp設定
net add bgp autonomous-system 65000
net add bgp router-id 172.16.222.1
net add bgp redistribute connected
net add bgp bestpath as-path multipath-relax
net add bgp neighbor PEER_SPINE peer-group
net add bgp neighbor 10.10.10.2 remote-as 65001
net add bgp neighbor 10.10.10.2 description spine01
net add bgp neighbor 10.10.10.2 peer-group PEER_SPINE
net add bgp neighbor 10.10.10.6 remote-as 65001
net add bgp neighbor 10.10.10.6 description spine02
net add bgp neighbor 10.10.10.6 peer-group PEER_SPINE
net add bgp neighbor 10.10.10.10 remote-as 65002
net add bgp neighbor 10.10.10.10 description spine03
net add bgp neighbor 10.10.10.10 peer-group PEER_SPINE
net add bgp neighbor 10.10.10.14 remote-as 65002
net add bgp neighbor 10.10.10.14 description spine04
net add bgp neighbor 10.10.10.14 peer-group PEER_SPINE

BB02

#hostname/各IFの設定
net add hostname BB02
net add interface swp1 alias to-spine01-swp2
net add interface swp1 mtu 9000
net add interface swp1 ip address 10.10.10.17/30
net add interface swp2 alias to-spine02-swp2
net add interface swp2 mtu 9000
net add interface swp2 ip address 10.10.10.21/30
net add interface swp3 alias to-spine03-swp2
net add interface swp3 mtu 9000
net add interface swp3 ip address 10.10.10.25/30
net add interface swp4 alias to-spine04-swp2
net add interface swp4 mtu 9000
net add interface swp4 ip address 10.10.10.29/30
net add loopback lo ip address 172.16.222.2/32

#underlay-bgp設定
net add bgp autonomous-system 65000
net add bgp router-id 172.16.222.2
net add bgp redistribute connected
net add bgp bestpath as-path multipath-relax
net add bgp neighbor PEER_SPINE peer-group
net add bgp neighbor 10.10.10.18 remote-as 65001
net add bgp neighbor 10.10.10.18 description spine01
net add bgp neighbor 10.10.10.18 peer-group PEER_SPINE
net add bgp neighbor 10.10.10.22 remote-as 65001
net add bgp neighbor 10.10.10.22 description spine02
net add bgp neighbor 10.10.10.22 peer-group PEER_SPINE
net add bgp neighbor 10.10.10.26 remote-as 65002
net add bgp neighbor 10.10.10.26 description spine03
net add bgp neighbor 10.10.10.26 peer-group PEER_SPINE
net add bgp neighbor 10.10.10.30 remote-as 65002
net add bgp neighbor 10.10.10.30 description spine04
net add bgp neighbor 10.10.10.30 peer-group PEER_SPINE

spine01

#hostname/各IFの設定
net add hostname spine01
net add interface swp1 alias to-BB01-swp1
net add interface swp1 mtu 9000
net add interface swp1 ip address 10.10.10.2/30
net add interface swp2 alias to-BB02-swp1
net add interface swp2 mtu 9000
net add interface swp2 ip address 10.10.10.18/30
net add loopback lo ip address 172.16.1.2/32

#underlay-bgp設定
net add bgp autonomous-system 65001
net add bgp router-id 172.16.1.2
net add bgp redistribute connected
net add bgp bestpath as-path multipath-relax
net add bgp neighbor PEER_BB peer-group
net add bgp neighbor 10.10.10.1 remote-as 65000
net add bgp neighbor 10.10.10.1 description BB01
net add bgp neighbor 10.10.10.1 peer-group PEER_BB
net add bgp neighbor 10.10.10.17 remote-as 65000
net add bgp neighbor 10.10.10.17 description BB02
net add bgp neighbor 10.10.10.17 peer-group PEER_BB

spine02

#hostname/各IFの設定
net add hostname spine02
net add interface swp1 alias to-BB01-swp2
net add interface swp1 mtu 9000
net add interface swp1 ip address 10.10.10.6/30
net add interface swp2 alias to-BB02-swp2
net add interface swp2 mtu 9000
net add interface swp2 ip address 10.10.10.22/30
net add loopback lo ip address 172.16.1.3/32

#underlay-bgp設定
net add bgp autonomous-system 65001
net add bgp router-id 172.16.1.3
net add bgp redistribute connected
net add bgp bestpath as-path multipath-relax
net add bgp neighbor PEER_BB peer-group
net add bgp neighbor 10.10.10.5 remote-as 65000
net add bgp neighbor 10.10.10.5 description BB01
net add bgp neighbor 10.10.10.5 peer-group PEER_BB
net add bgp neighbor 10.10.10.21 remote-as 65000
net add bgp neighbor 10.10.10.21 description BB02
net add bgp neighbor 10.10.10.21 peer-group PEER_BB

spine03

#hostname/各IFの設定
net add hostname spine03
net add interface swp1 alias to-BB01-swp3
net add interface swp1 mtu 9000
net add interface swp1 ip address 10.10.10.10/30
net add interface swp2 alias to-BB02-swp3
net add interface swp2 mtu 9000
net add interface swp2 ip address 10.10.10.26/30
net add loopback lo ip address 172.16.1.5/32

#underlay-bgp設定
net add bgp autonomous-system 65002
net add bgp router-id 172.16.1.5
net add bgp redistribute connected
net add bgp bestpath as-path multipath-relax
net add bgp neighbor PEER_BB peer-group
net add bgp neighbor 10.10.10.9 remote-as 65000
net add bgp neighbor 10.10.10.9 description BB01
net add bgp neighbor 10.10.10.9 peer-group PEER_BB
net add bgp neighbor 10.10.10.25 remote-as 65000
net add bgp neighbor 10.10.10.25 description BB02
net add bgp neighbor 10.10.10.25 peer-group PEER_BB

spine04

#hostname/各IFの設定
net add hostname spine04
net add interface swp1 alias to-BB01-swp4
net add interface swp1 mtu 9000
net add interface swp1 ip address 10.10.10.14/30
net add interface swp2 alias to-BB02-swp4
net add interface swp2 mtu 9000
net add interface swp2 ip address 10.10.10.30/30
net add loopback lo ip address 172.16.1.6/32

#underlay-bgp設定
net add bgp autonomous-system 65002
net add bgp router-id 172.16.1.6
net add bgp redistribute connected
net add bgp bestpath as-path multipath-relax
net add bgp neighbor PEER_BB peer-group
net add bgp neighbor 10.10.10.13 remote-as 65000
net add bgp neighbor 10.10.10.13 description BB01
net add bgp neighbor 10.10.10.13 peer-group PEER_BB
net add bgp neighbor 10.10.10.29 remote-as 65000
net add bgp neighbor 10.10.10.29 description BB02
net add bgp neighbor 10.10.10.29 peer-group PEER_BB

以下のコマンドで確認後、commit

net pending
net commit

続いて、spineのMLAG設定
peer IPアドレスなど手動で設定できるが、今回は自動作成で

spine01

net add clag peer sys-mac 44:38:39:FF:00:00 interface swp3-4 primary
net add vlan 100,200
net add clag port bond bond-to-ToR01 interface swp5 clag-id 1
net add bond bond-to-ToR01 bridge trunk vlans 100,200
net add bond bond-to-ToR01 mtu 9000
#仮想VTEPのIPアドレス設定
net add loopback lo clag vxlan-anycast-ip 172.16.1.1

spine02

net add clag peer sys-mac 44:38:39:FF:00:00 interface swp3-4 secondary
net add vlan 100,200
net add clag port bond bond-to-ToR01 interface swp5 clag-id 1
net add bond bond-to-ToR01 bridge trunk vlans 100,200
net add bond bond-to-ToR01 mtu 9000
#仮想VTEPのIPアドレス設定
net add loopback lo clag vxlan-anycast-ip 172.16.1.1

spine03

net add clag peer sys-mac 44:38:39:FF:00:01 interface swp3-4 primary
net add vlan 100,200
net add clag port bond bond-to-ToR02 interface swp5 clag-id 1
net add bond bond-to-ToR02 bridge trunk vlans 100,200
net add bond bond-to-ToR02 mtu 9000
#仮想VTEPのIPアドレス設定
net add loopback lo clag vxlan-anycast-ip 172.16.1.4

spine04

net add clag peer sys-mac 44:38:39:FF:00:01 interface swp3-4 secondary
net add vlan 100,200
net add clag port bond bond-to-ToR02 interface swp5 clag-id 1
net add bond bond-to-ToR02 bridge trunk vlans 100,200
net add bond bond-to-ToR02 mtu 9000
#仮想VTEPのIPアドレス設定
net add loopback lo clag vxlan-anycast-ip 172.16.1.4

以下のコマンドで確認後、commit

net pending
net commit

続いて、ToRのlacp/vlan設定
OVSでも代用可能なので、使いやすい方で設定すればいいかなーと

ToR01

net add hostname ToR01
net add bond bond-to-SPINE bond slaves swp1-2
net add bond bond-to-SPINE bridge trunk vlans 100,200
net add bond bond-to-SPINE mtu 9000
net add interface swp3 bridge access 100
net add interface swp4 bridge access 200

ToR02

net add hostname ToR02
net add bond bond-to-SPINE bond slaves swp1-2
net add bond bond-to-SPINE bridge trunk vlans 100,200
net add bond bond-to-SPINE mtu 9000
net add interface swp3 bridge access 100
net add interface swp4 bridge access 200

各機器にてルーティングテーブル、IF、mlag状態確認
※spineたちのmlag用IPアドレスが見えちゃうが、気になるならip prefix-listで広報するアドレスを制御

BB01

root@BB01:~# net show route
show ip route
=============
Codes: K - kernel route, C - connected, S - static, R - RIP,
       O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
       T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
       F - PBR,
       > - selected route, * - FIB route

C>* 10.10.10.0/30 is directly connected, swp1, 00:04:14
C>* 10.10.10.4/30 is directly connected, swp2, 00:04:14
C>* 10.10.10.8/30 is directly connected, swp3, 00:04:14
C>* 10.10.10.12/30 is directly connected, swp4, 00:04:14
B>* 10.10.10.16/30 [20/0] via 10.10.10.2, swp1, 00:04:03
B>* 10.10.10.20/30 [20/0] via 10.10.10.6, swp2, 00:03:58
B>* 10.10.10.24/30 [20/0] via 10.10.10.10, swp3, 00:04:05
B>* 10.10.10.28/30 [20/0] via 10.10.10.14, swp4, 00:04:03
B>* 169.254.1.0/30 [20/0] via 10.10.10.10, swp3, 00:03:58
  *                       via 10.10.10.2, swp1, 00:03:58
  *                       via 10.10.10.6, swp2, 00:03:58
  *                       via 10.10.10.14, swp4, 00:03:58
B>* 172.16.1.1/32 [20/0] via 10.10.10.2, swp1, 00:02:23
B>* 172.16.1.2/32 [20/0] via 10.10.10.2, swp1, 00:04:03
B>* 172.16.1.3/32 [20/0] via 10.10.10.6, swp2, 00:03:58
B>* 172.16.1.4/32 [20/0] via 10.10.10.10, swp3, 00:02:23
B>* 172.16.1.5/32 [20/0] via 10.10.10.10, swp3, 00:04:05
B>* 172.16.1.6/32 [20/0] via 10.10.10.14, swp4, 00:04:03
C>* 172.16.222.1/32 is directly connected, lo, 00:04:14



show ipv6 route
===============
Codes: K - kernel route, C - connected, S - static, R - RIPng,
       O - OSPFv3, I - IS-IS, B - BGP, N - NHRP, T - Table,
       v - VNC, V - VNC-Direct, A - Babel, D - SHARP, F - PBR,
       > - selected route, * - FIB route

C * fe80::/64 is directly connected, swp4, 00:04:14
C * fe80::/64 is directly connected, swp3, 00:04:14
C * fe80::/64 is directly connected, swp2, 00:04:14
C * fe80::/64 is directly connected, swp1, 00:04:14
C>* fe80::/64 is directly connected, eth0, 00:04:14
root@BB01:~# net show interface all mac
State  Name  Spd  MAC                MTU    Mode           LLDP            Summary
-----  ----  ---  -----------------  -----  -------------  --------------  -------------------
UP     lo    N/A  00:00:00:00:00:00  65536  Loopback                       IP: 127.0.0.1/8
       lo                                                                  IP: 172.16.222.1/32
       lo                                                                  IP: ::1/128
UP     eth0  1G   0c:1e:ed:fe:04:00  1500   Mgmt
UP     swp1  1G   0c:1e:ed:fe:04:01  9000   Interface/L3   spine01 (swp1)  IP: 10.10.10.1/30
UP     swp2  1G   0c:1e:ed:fe:04:02  9000   Interface/L3   spine02 (swp1)  IP: 10.10.10.5/30
UP     swp3  1G   0c:1e:ed:fe:04:03  9000   Interface/L3   spine03 (swp1)  IP: 10.10.10.9/30
UP     swp4  1G   0c:1e:ed:fe:04:04  9000   Interface/L3   spine04 (swp1)  IP: 10.10.10.13/30
ADMDN  swp5  N/A  0c:1e:ed:fe:04:05  1500   NotConfigured
ADMDN  swp6  N/A  0c:1e:ed:fe:04:06  1500   NotConfigured

root@BB01:~#

BB02

root@BB02:~# net show route
show ip route
=============
Codes: K - kernel route, C - connected, S - static, R - RIP,
       O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
       T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
       F - PBR,
       > - selected route, * - FIB route

B>* 10.10.10.0/30 [20/0] via 10.10.10.18, swp1, 00:04:33
B>* 10.10.10.4/30 [20/0] via 10.10.10.22, swp2, 00:04:27
B>* 10.10.10.8/30 [20/0] via 10.10.10.26, swp3, 00:04:37
B>* 10.10.10.12/30 [20/0] via 10.10.10.30, swp4, 00:04:33
C>* 10.10.10.16/30 is directly connected, swp1, 00:04:46
C>* 10.10.10.20/30 is directly connected, swp2, 00:04:46
C>* 10.10.10.24/30 is directly connected, swp3, 00:04:46
C>* 10.10.10.28/30 is directly connected, swp4, 00:04:46
B>* 169.254.1.0/30 [20/0] via 10.10.10.26, swp3, 00:04:27
  *                       via 10.10.10.18, swp1, 00:04:27
  *                       via 10.10.10.22, swp2, 00:04:27
  *                       via 10.10.10.30, swp4, 00:04:27
B>* 172.16.1.1/32 [20/0] via 10.10.10.18, swp1, 00:02:52
B>* 172.16.1.2/32 [20/0] via 10.10.10.18, swp1, 00:04:33
B>* 172.16.1.3/32 [20/0] via 10.10.10.22, swp2, 00:04:27
B>* 172.16.1.4/32 [20/0] via 10.10.10.26, swp3, 00:02:52
B>* 172.16.1.5/32 [20/0] via 10.10.10.26, swp3, 00:04:37
B>* 172.16.1.6/32 [20/0] via 10.10.10.30, swp4, 00:04:33
C>* 172.16.222.2/32 is directly connected, lo, 00:04:46



show ipv6 route
===============
Codes: K - kernel route, C - connected, S - static, R - RIPng,
       O - OSPFv3, I - IS-IS, B - BGP, N - NHRP, T - Table,
       v - VNC, V - VNC-Direct, A - Babel, D - SHARP, F - PBR,
       > - selected route, * - FIB route

C * fe80::/64 is directly connected, swp4, 00:04:45
C * fe80::/64 is directly connected, swp1, 00:04:45
C * fe80::/64 is directly connected, swp2, 00:04:46
C * fe80::/64 is directly connected, swp3, 00:04:46
C>* fe80::/64 is directly connected, eth0, 00:04:46
root@BB02:~# net show interface all mac
State  Name  Spd  MAC                MTU    Mode           LLDP            Summary
-----  ----  ---  -----------------  -----  -------------  --------------  -------------------
UP     lo    N/A  00:00:00:00:00:00  65536  Loopback                       IP: 127.0.0.1/8
       lo                                                                  IP: 172.16.222.2/32
       lo                                                                  IP: ::1/128
UP     eth0  1G   0c:1e:ed:33:95:00  1500   Mgmt
UP     swp1  1G   0c:1e:ed:33:95:01  9000   Interface/L3   spine01 (swp2)  IP: 10.10.10.17/30
UP     swp2  1G   0c:1e:ed:33:95:02  9000   Interface/L3   spine02 (swp2)  IP: 10.10.10.21/30
UP     swp3  1G   0c:1e:ed:33:95:03  9000   Interface/L3   spine03 (swp2)  IP: 10.10.10.25/30
UP     swp4  1G   0c:1e:ed:33:95:04  9000   Interface/L3   spine04 (swp2)  IP: 10.10.10.29/30
ADMDN  swp5  N/A  0c:1e:ed:33:95:05  1500   NotConfigured
ADMDN  swp6  N/A  0c:1e:ed:33:95:06  1500   NotConfigured

root@BB02:~#

spine01

root@spine01:~# net show route
show ip route
=============
Codes: K - kernel route, C - connected, S - static, R - RIP,
       O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
       T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
       F - PBR,
       > - selected route, * - FIB route

C>* 10.10.10.0/30 is directly connected, swp1, 00:07:16
B>* 10.10.10.4/30 [20/0] via 10.10.10.1, swp1, 00:07:04
B>* 10.10.10.8/30 [20/0] via 10.10.10.1, swp1, 00:07:04
B>* 10.10.10.12/30 [20/0] via 10.10.10.1, swp1, 00:07:04
C>* 10.10.10.16/30 is directly connected, swp2, 00:07:16
B>* 10.10.10.20/30 [20/0] via 10.10.10.17, swp2, 00:07:05
B>* 10.10.10.24/30 [20/0] via 10.10.10.17, swp2, 00:07:05
B>* 10.10.10.28/30 [20/0] via 10.10.10.17, swp2, 00:07:05
C>* 169.254.1.0/30 is directly connected, peerlink.4094, 00:07:07
C>* 172.16.1.1/32 is directly connected, lo, 00:05:25
C>* 172.16.1.2/32 is directly connected, lo, 00:07:16
B>* 172.16.1.4/32 [20/0] via 10.10.10.1, swp1, 00:05:25
  *                      via 10.10.10.17, swp2, 00:05:25
B>* 172.16.1.5/32 [20/0] via 10.10.10.17, swp2, 00:07:04
  *                      via 10.10.10.1, swp1, 00:07:04
B>* 172.16.1.6/32 [20/0] via 10.10.10.17, swp2, 00:07:04
  *                      via 10.10.10.1, swp1, 00:07:04
B>* 172.16.222.1/32 [20/0] via 10.10.10.1, swp1, 00:07:04
B>* 172.16.222.2/32 [20/0] via 10.10.10.17, swp2, 00:07:05



show ipv6 route
===============
Codes: K - kernel route, C - connected, S - static, R - RIPng,
       O - OSPFv3, I - IS-IS, B - BGP, N - NHRP, T - Table,
       v - VNC, V - VNC-Direct, A - Babel, D - SHARP, F - PBR,
       > - selected route, * - FIB route

C * fe80::/64 is directly connected, bridge, 00:06:49
C * fe80::/64 is directly connected, peerlink.4094, 00:07:04
C * fe80::/64 is directly connected, swp2, 00:07:16
C * fe80::/64 is directly connected, swp1, 00:07:16
C>* fe80::/64 is directly connected, eth0, 00:07:16
root@spine01:~# net show interface all mac
State  Name           Spd  MAC                MTU    Mode           LLDP            Summary
-----  -------------  ---  -----------------  -----  -------------  --------------  -------------------------
UP     lo             N/A  00:00:00:00:00:00  65536  Loopback                       IP: 127.0.0.1/8
       lo                                                                           IP: 172.16.1.2/32
       lo                                                                           IP: 172.16.1.1/32
       lo                                                                           IP: ::1/128
UP     eth0           1G   0c:1e:ed:91:74:00  1500   Mgmt
UP     swp1           1G   0c:1e:ed:91:74:01  9000   Interface/L3   BB01 (swp1)     IP: 10.10.10.2/30
UP     swp2           1G   0c:1e:ed:91:74:02  9000   Interface/L3   BB02 (swp1)     IP: 10.10.10.18/30
UP     swp3           1G   0c:1e:ed:91:74:03  1500   BondMember     spine02 (swp3)  Master: peerlink(UP)
UP     swp4           1G   0c:1e:ed:91:74:03  1500   BondMember     spine02 (swp4)  Master: peerlink(UP)
UP     swp5           1G   0c:1e:ed:91:74:05  9000   BondMember     ToR01 (swp1)    Master: bond-to-ToR01(UP)
ADMDN  swp6           N/A  0c:1e:ed:91:74:06  1500   NotConfigured
UP     bond-to-ToR01  1G   0c:1e:ed:91:74:05  9000   802.3ad                        Master: bridge(UP)
       bond-to-ToR01                                                                Bond Members: swp5(UP)
UP     bridge         N/A  0c:1e:ed:91:74:03  1500   Bridge/L2
UP     peerlink       2G   0c:1e:ed:91:74:03  1500   802.3ad                        Master: bridge(UP)
       peerlink                                                                     Bond Members: swp3(UP)
       peerlink                                                                     Bond Members: swp4(UP)
UP     peerlink.4094  2G   0c:1e:ed:91:74:03  1500   SubInt/L3                      IP: 169.254.1.1/30

root@spine01:~# net show clag
The peer is alive
     Our Priority, ID, and Role: 1000 0c:1e:ed:91:74:03 primary
    Peer Priority, ID, and Role: 2000 0c:1e:ed:b9:90:03 secondary
          Peer Interface and IP: peerlink.4094 169.254.1.2
               VxLAN Anycast IP: 172.16.1.1
                      Backup IP:  (inactive)
                     System MAC: 44:38:39:ff:00:00

CLAG Interfaces
Our Interface      Peer Interface     CLAG Id   Conflicts              Proto-Down Reason
----------------   ----------------   -------   --------------------   -----------------
   bond-to-ToR01   bond-to-ToR01      1         -                      -
root@spine01:~#

spine02

root@spine02:~# net show route
show ip route
=============
Codes: K - kernel route, C - connected, S - static, R - RIP,
       O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
       T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
       F - PBR,
       > - selected route, * - FIB route

B>* 10.10.10.0/30 [20/0] via 10.10.10.5, swp1, 00:07:43
C>* 10.10.10.4/30 is directly connected, swp1, 00:07:54
B>* 10.10.10.8/30 [20/0] via 10.10.10.5, swp1, 00:07:43
B>* 10.10.10.12/30 [20/0] via 10.10.10.5, swp1, 00:07:43
B>* 10.10.10.16/30 [20/0] via 10.10.10.21, swp2, 00:07:43
C>* 10.10.10.20/30 is directly connected, swp2, 00:07:54
B>* 10.10.10.24/30 [20/0] via 10.10.10.21, swp2, 00:07:43
B>* 10.10.10.28/30 [20/0] via 10.10.10.21, swp2, 00:07:43
C>* 169.254.1.0/30 is directly connected, peerlink.4094, 00:07:44
C>* 172.16.1.3/32 is directly connected, lo, 00:07:54
B>* 172.16.1.4/32 [20/0] via 10.10.10.5, swp1, 00:06:08
  *                      via 10.10.10.21, swp2, 00:06:08
B>* 172.16.1.5/32 [20/0] via 10.10.10.5, swp1, 00:07:43
  *                      via 10.10.10.21, swp2, 00:07:43
B>* 172.16.1.6/32 [20/0] via 10.10.10.5, swp1, 00:07:43
  *                      via 10.10.10.21, swp2, 00:07:43
B>* 172.16.222.1/32 [20/0] via 10.10.10.5, swp1, 00:07:43
B>* 172.16.222.2/32 [20/0] via 10.10.10.21, swp2, 00:07:43



show ipv6 route
===============
Codes: K - kernel route, C - connected, S - static, R - RIPng,
       O - OSPFv3, I - IS-IS, B - BGP, N - NHRP, T - Table,
       v - VNC, V - VNC-Direct, A - Babel, D - SHARP, F - PBR,
       > - selected route, * - FIB route

C * fe80::/64 is directly connected, bridge, 00:07:32
C * fe80::/64 is directly connected, peerlink.4094, 00:07:42
C * fe80::/64 is directly connected, swp2, 00:07:54
C * fe80::/64 is directly connected, swp1, 00:07:54
C>* fe80::/64 is directly connected, eth0, 00:07:54
root@spine02:~# net show interface all mac
State  Name           Spd  MAC                MTU    Mode           LLDP            Summary
-----  -------------  ---  -----------------  -----  -------------  --------------  -------------------------
UP     lo             N/A  00:00:00:00:00:00  65536  Loopback                       IP: 127.0.0.1/8
       lo                                                                           IP: 172.16.1.3/32
       lo                                                                           IP: ::1/128
UP     eth0           1G   0c:1e:ed:b9:90:00  1500   Mgmt
UP     swp1           1G   0c:1e:ed:b9:90:01  9000   Interface/L3   BB01 (swp2)     IP: 10.10.10.6/30
UP     swp2           1G   0c:1e:ed:b9:90:02  9000   Interface/L3   BB02 (swp2)     IP: 10.10.10.22/30
UP     swp3           1G   0c:1e:ed:b9:90:03  1500   BondMember     spine01 (swp3)  Master: peerlink(UP)
UP     swp4           1G   0c:1e:ed:b9:90:03  1500   BondMember     spine01 (swp4)  Master: peerlink(UP)
UP     swp5           1G   0c:1e:ed:b9:90:05  9000   BondMember     ToR01 (swp2)    Master: bond-to-ToR01(UP)
ADMDN  swp6           N/A  0c:1e:ed:b9:90:06  1500   NotConfigured
UP     bond-to-ToR01  1G   0c:1e:ed:b9:90:05  9000   802.3ad                        Master: bridge(UP)
       bond-to-ToR01                                                                Bond Members: swp5(UP)
UP     bridge         N/A  0c:1e:ed:b9:90:03  1500   Bridge/L2
UP     peerlink       2G   0c:1e:ed:b9:90:03  1500   802.3ad                        Master: bridge(UP)
       peerlink                                                                     Bond Members: swp3(UP)
       peerlink                                                                     Bond Members: swp4(UP)
UP     peerlink.4094  2G   0c:1e:ed:b9:90:03  1500   SubInt/L3                      IP: 169.254.1.2/30

root@spine02:~# net show clag
The peer is alive
    Peer Priority, ID, and Role: 1000 0c:1e:ed:91:74:03 primary
     Our Priority, ID, and Role: 2000 0c:1e:ed:b9:90:03 secondary
          Peer Interface and IP: peerlink.4094 169.254.1.1
               VxLAN Anycast IP: 172.16.1.1
                      Backup IP:  (inactive)
                     System MAC: 44:38:39:ff:00:00

CLAG Interfaces
Our Interface      Peer Interface     CLAG Id   Conflicts              Proto-Down Reason
----------------   ----------------   -------   --------------------   -----------------
   bond-to-ToR01   bond-to-ToR01      1         -                      -
root@spine02:~#

spine03

root@spine03:~# net show route
show ip route
=============
Codes: K - kernel route, C - connected, S - static, R - RIP,
       O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
       T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
       F - PBR,
       > - selected route, * - FIB route

B>* 10.10.10.0/30 [20/0] via 10.10.10.9, swp1, 00:08:09
B>* 10.10.10.4/30 [20/0] via 10.10.10.9, swp1, 00:08:09
C>* 10.10.10.8/30 is directly connected, swp1, 00:08:21
B>* 10.10.10.12/30 [20/0] via 10.10.10.9, swp1, 00:08:09
B>* 10.10.10.16/30 [20/0] via 10.10.10.25, swp2, 00:08:11
B>* 10.10.10.20/30 [20/0] via 10.10.10.25, swp2, 00:08:11
C>* 10.10.10.24/30 is directly connected, swp2, 00:08:21
B>* 10.10.10.28/30 [20/0] via 10.10.10.25, swp2, 00:08:11
C>* 169.254.1.0/30 is directly connected, peerlink.4094, 00:08:17
B>* 172.16.1.1/32 [20/0] via 10.10.10.9, swp1, 00:06:26
  *                      via 10.10.10.25, swp2, 00:06:26
B>* 172.16.1.2/32 [20/0] via 10.10.10.25, swp2, 00:08:06
  *                      via 10.10.10.9, swp1, 00:08:06
B>* 172.16.1.3/32 [20/0] via 10.10.10.9, swp1, 00:08:01
  *                      via 10.10.10.25, swp2, 00:08:01
C>* 172.16.1.4/32 is directly connected, lo, 00:06:26
C>* 172.16.1.5/32 is directly connected, lo, 00:08:21
B>* 172.16.222.1/32 [20/0] via 10.10.10.9, swp1, 00:08:09
B>* 172.16.222.2/32 [20/0] via 10.10.10.25, swp2, 00:08:11



show ipv6 route
===============
Codes: K - kernel route, C - connected, S - static, R - RIPng,
       O - OSPFv3, I - IS-IS, B - BGP, N - NHRP, T - Table,
       v - VNC, V - VNC-Direct, A - Babel, D - SHARP, F - PBR,
       > - selected route, * - FIB route

C * fe80::/64 is directly connected, bridge, 00:07:53
C * fe80::/64 is directly connected, peerlink.4094, 00:08:15
C * fe80::/64 is directly connected, swp2, 00:08:21
C * fe80::/64 is directly connected, swp1, 00:08:21
C>* fe80::/64 is directly connected, eth0, 00:08:21
root@spine03:~# net show interface all mac
State  Name           Spd  MAC                MTU    Mode           LLDP            Summary
-----  -------------  ---  -----------------  -----  -------------  --------------  -------------------------
UP     lo             N/A  00:00:00:00:00:00  65536  Loopback                       IP: 127.0.0.1/8
       lo                                                                           IP: 172.16.1.5/32
       lo                                                                           IP: 172.16.1.4/32
       lo                                                                           IP: ::1/128
UP     eth0           1G   0c:1e:ed:49:da:00  1500   Mgmt
UP     swp1           1G   0c:1e:ed:49:da:01  9000   Interface/L3   BB01 (swp3)     IP: 10.10.10.10/30
UP     swp2           1G   0c:1e:ed:49:da:02  9000   Interface/L3   BB02 (swp3)     IP: 10.10.10.26/30
UP     swp3           1G   0c:1e:ed:49:da:03  1500   BondMember     spine04 (swp3)  Master: peerlink(UP)
UP     swp4           1G   0c:1e:ed:49:da:03  1500   BondMember     spine04 (swp4)  Master: peerlink(UP)
UP     swp5           1G   0c:1e:ed:49:da:05  9000   BondMember     ToR02 (swp1)    Master: bond-to-ToR02(UP)
ADMDN  swp6           N/A  0c:1e:ed:49:da:06  1500   NotConfigured
UP     bond-to-ToR02  1G   0c:1e:ed:49:da:05  9000   802.3ad                        Master: bridge(UP)
       bond-to-ToR02                                                                Bond Members: swp5(UP)
UP     bridge         N/A  0c:1e:ed:49:da:03  1500   Bridge/L2
UP     peerlink       2G   0c:1e:ed:49:da:03  1500   802.3ad                        Master: bridge(UP)
       peerlink                                                                     Bond Members: swp3(UP)
       peerlink                                                                     Bond Members: swp4(UP)
UP     peerlink.4094  2G   0c:1e:ed:49:da:03  1500   SubInt/L3                      IP: 169.254.1.1/30

root@spine03:~# net show clag
The peer is alive
     Our Priority, ID, and Role: 1000 0c:1e:ed:49:da:03 primary
    Peer Priority, ID, and Role: 2000 0c:1e:ed:5a:03:03 secondary
          Peer Interface and IP: peerlink.4094 169.254.1.2
               VxLAN Anycast IP: 172.16.1.4
                      Backup IP:  (inactive)
                     System MAC: 44:38:39:ff:00:01

CLAG Interfaces
Our Interface      Peer Interface     CLAG Id   Conflicts              Proto-Down Reason
----------------   ----------------   -------   --------------------   -----------------
   bond-to-ToR02   bond-to-ToR02      1         -                      -
root@spine03:~#

spine04

root@spine04:~# net show route
net show clagshow ip route
=============
Codes: K - kernel route, C - connected, S - static, R - RIP,
       O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
       T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
       F - PBR,
       > - selected route, * - FIB route

B>* 10.10.10.0/30 [20/0] via 10.10.10.13, swp1, 00:08:19
B>* 10.10.10.4/30 [20/0] via 10.10.10.13, swp1, 00:08:19
B>* 10.10.10.8/30 [20/0] via 10.10.10.13, swp1, 00:08:19
C>* 10.10.10.12/30 is directly connected, swp1, 00:08:31
B>* 10.10.10.16/30 [20/0] via 10.10.10.29, swp2, 00:08:20
B>* 10.10.10.20/30 [20/0] via 10.10.10.29, swp2, 00:08:20
B>* 10.10.10.24/30 [20/0] via 10.10.10.29, swp2, 00:08:20
C>* 10.10.10.28/30 is directly connected, swp2, 00:08:31
C>* 169.254.1.0/30 is directly connected, peerlink.4094, 00:08:21
B>* 172.16.1.1/32 [20/0] via 10.10.10.13, swp1, 00:06:39
  *                      via 10.10.10.29, swp2, 00:06:39
B>* 172.16.1.2/32 [20/0] via 10.10.10.29, swp2, 00:08:19
  *                      via 10.10.10.13, swp1, 00:08:19
B>* 172.16.1.3/32 [20/0] via 10.10.10.13, swp1, 00:08:14
  *                      via 10.10.10.29, swp2, 00:08:14
C>* 172.16.1.6/32 is directly connected, lo, 00:08:31
B>* 172.16.222.1/32 [20/0] via 10.10.10.13, swp1, 00:08:19
B>* 172.16.222.2/32 [20/0] via 10.10.10.29, swp2, 00:08:20



show ipv6 route
===============
Codes: K - kernel route, C - connected, S - static, R - RIPng,
       O - OSPFv3, I - IS-IS, B - BGP, N - NHRP, T - Table,
       v - VNC, V - VNC-Direct, A - Babel, D - SHARP, F - PBR,
       > - selected route, * - FIB route

C * fe80::/64 is directly connected, bridge, 00:08:03
C * fe80::/64 is directly connected, peerlink.4094, 00:08:20
C * fe80::/64 is directly connected, swp2, 00:08:31
C * fe80::/64 is directly connected, swp1, 00:08:31
C>* fe80::/64 is directly connected, eth0, 00:08:31
root@spine04:~# net show interface all mac
State  Name           Spd  MAC                MTU    Mode           LLDP            Summary
-----  -------------  ---  -----------------  -----  -------------  --------------  -------------------------
UP     lo             N/A  00:00:00:00:00:00  65536  Loopback                       IP: 127.0.0.1/8
       lo                                                                           IP: 172.16.1.6/32
       lo                                                                           IP: ::1/128
UP     eth0           1G   0c:1e:ed:5a:03:00  1500   Mgmt
UP     swp1           1G   0c:1e:ed:5a:03:01  9000   Interface/L3   BB01 (swp4)     IP: 10.10.10.14/30
UP     swp2           1G   0c:1e:ed:5a:03:02  9000   Interface/L3   BB02 (swp4)     IP: 10.10.10.30/30
UP     swp3           1G   0c:1e:ed:5a:03:03  1500   BondMember     spine03 (swp3)  Master: peerlink(UP)
UP     swp4           1G   0c:1e:ed:5a:03:03  1500   BondMember     spine03 (swp4)  Master: peerlink(UP)
UP     swp5           1G   0c:1e:ed:5a:03:05  9000   BondMember     ToR02 (swp2)    Master: bond-to-ToR02(UP)
ADMDN  swp6           N/A  0c:1e:ed:5a:03:06  1500   NotConfigured
UP     bond-to-ToR02  1G   0c:1e:ed:5a:03:05  9000   802.3ad                        Master: bridge(UP)
       bond-to-ToR02                                                                Bond Members: swp5(UP)
UP     bridge         N/A  0c:1e:ed:5a:03:03  1500   Bridge/L2
UP     peerlink       2G   0c:1e:ed:5a:03:03  1500   802.3ad                        Master: bridge(UP)
       peerlink                                                                     Bond Members: swp3(UP)
       peerlink                                                                     Bond Members: swp4(UP)
UP     peerlink.4094  2G   0c:1e:ed:5a:03:03  1500   SubInt/L3                      IP: 169.254.1.2/30

root@spine04:~# net show clag
The peer is alive
    Peer Priority, ID, and Role: 1000 0c:1e:ed:49:da:03 primary
     Our Priority, ID, and Role: 2000 0c:1e:ed:5a:03:03 secondary
          Peer Interface and IP: peerlink.4094 169.254.1.1
               VxLAN Anycast IP: 172.16.1.4
                      Backup IP:  (inactive)
                     System MAC: 44:38:39:ff:00:01

CLAG Interfaces
Our Interface      Peer Interface     CLAG Id   Conflicts              Proto-Down Reason
----------------   ----------------   -------   --------------------   -----------------
   bond-to-ToR02   bond-to-ToR02      1         -                      -
root@spine04:~#

ToR01

root@ToR01:~# net show interface all mac
State  Name           Spd  MAC                MTU    Mode           LLDP            Summary
-----  -------------  ---  -----------------  -----  -------------  --------------  -------------------------
UP     lo             N/A  00:00:00:00:00:00  65536  Loopback                       IP: 127.0.0.1/8
       lo                                                                           IP: ::1/128
UP     eth0           1G   0c:1e:ed:9f:32:00  1500   Mgmt
UP     swp1           1G   0c:1e:ed:9f:32:02  9000   BondMember     spine01 (swp5)  Master: bond-to-SPINE(UP)
UP     swp2           1G   0c:1e:ed:9f:32:02  9000   BondMember     spine02 (swp5)  Master: bond-to-SPINE(UP)
UP     swp3           1G   0c:1e:ed:9f:32:03  1500   Access/L2                      Master: bridge(UP)
UP     swp4           1G   0c:1e:ed:9f:32:04  1500   Access/L2                      Master: bridge(UP)
ADMDN  swp5           N/A  0c:1e:ed:9f:32:05  1500   NotConfigured
ADMDN  swp6           N/A  0c:1e:ed:9f:32:06  1500   NotConfigured
UP     bond-to-SPINE  2G   0c:1e:ed:9f:32:02  9000   802.3ad                        Master: bridge(UP)
       bond-to-SPINE                                                                Bond Members: swp1(UP)
       bond-to-SPINE                                                                Bond Members: swp2(UP)
UP     bridge         N/A  0c:1e:ed:9f:32:02  1500   Bridge/L2

root@ToR01:~#

ToR02

root@ToR02:~# net show interface all mac
State  Name           Spd  MAC                MTU    Mode           LLDP            Summary
-----  -------------  ---  -----------------  -----  -------------  --------------  -------------------------
UP     lo             N/A  00:00:00:00:00:00  65536  Loopback                       IP: 127.0.0.1/8
       lo                                                                           IP: ::1/128
UP     eth0           1G   0c:1e:ed:2c:26:00  1500   Mgmt
UP     swp1           1G   0c:1e:ed:2c:26:02  9000   BondMember     spine03 (swp5)  Master: bond-to-SPINE(UP)
UP     swp2           1G   0c:1e:ed:2c:26:02  9000   BondMember     spine04 (swp5)  Master: bond-to-SPINE(UP)
UP     swp3           1G   0c:1e:ed:2c:26:03  1500   Access/L2                      Master: bridge(UP)
UP     swp4           1G   0c:1e:ed:2c:26:04  1500   Access/L2                      Master: bridge(UP)
ADMDN  swp5           N/A  0c:1e:ed:2c:26:05  1500   NotConfigured
ADMDN  swp6           N/A  0c:1e:ed:2c:26:06  1500   NotConfigured
UP     bond-to-SPINE  2G   0c:1e:ed:2c:26:02  9000   802.3ad                        Master: bridge(UP)
       bond-to-SPINE                                                                Bond Members: swp1(UP)
       bond-to-SPINE                                                                Bond Members: swp2(UP)
UP     bridge         N/A  0c:1e:ed:2c:26:02  1500   Bridge/L2

root@ToR02:~#

overlay設定

BB01/02

#vxlanに無関係なBB01/02も転送のためMP-BGP EVPNを有効化
net add bgp l2vpn evpn neighbor PEER_SPINE activate

spine01だけ注釈を付ける

spine01

net add vxlan vxlan10100 vxlan id 10100
net add vxlan vxlan10100 bridge access 100
net add vxlan vxlan10200 vxlan id 10200
net add vxlan vxlan10200 bridge access 200

#Disable Data Plane MAC Learning over VXLAN Tunnels
net add vxlan vxlan10100,vxlan10200 bridge learning off

#The VNIs are configured identically on both peers. However, vxlan-local-tunnelip must be sourced from unique loopback stanza IP address of the switch.
net add vxlan vxlan10100,vxlan10200 vxlan local-tunnelip 172.16.1.2

#MTU設定
net add vxlan vxlan10100 mtu 1550
net add vxlan vxlan10200 mtu 1550

#evpn有効化
net add bgp l2vpn evpn neighbor PEER_BB activate
net add bgp l2vpn evpn advertise-all-vni

spine02

net add vxlan vxlan10100 vxlan id 10100
net add vxlan vxlan10100 bridge access 100
net add vxlan vxlan10200 vxlan id 10200
net add vxlan vxlan10200 bridge access 200
net add vxlan vxlan10100,vxlan10200 bridge learning off
net add vxlan vxlan10100,vxlan10200 vxlan local-tunnelip 172.16.1.3
net add vxlan vxlan10100 mtu 1550
net add vxlan vxlan10200 mtu 1550
net add bgp l2vpn evpn neighbor PEER_BB activate
net add bgp l2vpn evpn advertise-all-vni

spine03

net add vxlan vxlan10100 vxlan id 10100
net add vxlan vxlan10100 bridge access 100
net add vxlan vxlan10200 vxlan id 10200
net add vxlan vxlan10200 bridge access 200
net add vxlan vxlan10100,vxlan10200 bridge learning off
net add vxlan vxlan10100,vxlan10200 vxlan local-tunnelip 172.16.1.5
net add vxlan vxlan10100 mtu 1550
net add vxlan vxlan10200 mtu 1550
net add bgp l2vpn evpn neighbor PEER_BB activate
net add bgp l2vpn evpn advertise-all-vni

spine04

net add vxlan vxlan10100 vxlan id 10100
net add vxlan vxlan10100 bridge access 100
net add vxlan vxlan10200 vxlan id 10200
net add vxlan vxlan10200 bridge access 200
net add vxlan vxlan10100,vxlan10200 bridge learning off
net add vxlan vxlan10100,vxlan10200 vxlan local-tunnelip 172.16.1.6
net add vxlan vxlan10100 mtu 1550
net add vxlan vxlan10200 mtu 1550
net add bgp l2vpn evpn neighbor PEER_BB activate
net add bgp l2vpn evpn advertise-all-vni

確認後、commit

net pending
net commit

nodeにIPアドレス設定

UbuntuDockerGuest-1~4

#UbuntuDockerGuest-1
ifconfig eth0 192.168.1.1 netmask 255.255.255.0

#UbuntuDockerGuest-2
ifconfig eth0 192.168.100.1 netmask 255.255.255.0

#UbuntuDockerGuest-3
ifconfig eth0 192.168.1.2 netmask 255.255.255.0

#UbuntuDockerGuest-4
ifconfig eth0 192.168.100.2 netmask 255.255.255.0

動作確認

ping確認

UbuntuDockerGuest-1

root@UbuntuDockerGuest-1:~# ping 192.168.1.2
PING 192.168.1.2 (192.168.1.2) 56(84) bytes of data.
64 bytes from 192.168.1.2: icmp_seq=1 ttl=64 time=5.60 ms
64 bytes from 192.168.1.2: icmp_seq=2 ttl=64 time=3.53 ms
64 bytes from 192.168.1.2: icmp_seq=3 ttl=64 time=3.41 ms
64 bytes from 192.168.1.2: icmp_seq=4 ttl=64 time=3.51 ms
64 bytes from 192.168.1.2: icmp_seq=5 ttl=64 time=3.42 ms
^C
--- 192.168.1.2 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4008ms
rtt min/avg/max/mdev = 3.412/3.899/5.606/0.857 ms
root@UbuntuDockerGuest-1:~#

UbuntuDockerGuest-2

root@UbuntuDockerGuest-2:~# ping 192.168.100.2
PING 192.168.100.2 (192.168.100.2) 56(84) bytes of data.
64 bytes from 192.168.100.2: icmp_seq=1 ttl=64 time=4.20 ms
64 bytes from 192.168.100.2: icmp_seq=2 ttl=64 time=3.50 ms
64 bytes from 192.168.100.2: icmp_seq=3 ttl=64 time=3.48 ms
64 bytes from 192.168.100.2: icmp_seq=4 ttl=64 time=2.81 ms
64 bytes from 192.168.100.2: icmp_seq=5 ttl=64 time=3.35 ms
^C
--- 192.168.100.2 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4011ms
rtt min/avg/max/mdev = 2.815/3.473/4.200/0.442 ms
root@UbuntuDockerGuest-2:~#

各スイッチのテーブル確認

remote-VTEPのアドレスがroute type-2/3で確認できる

spine01

root@spine01:~# net show evpn
L2 VNIs: 2
L3 VNIs: 0
Advertise gateway mac-ip: No
Duplicate address detection: Enable
  Detection max-moves 5, time 180
root@spine01:~# net show evpn mac vni all
VNI 10200 #MACs (local and remote) 2

MAC               Type   Intf/Remote VTEP      VLAN
12:ed:20:2e:b3:1a remote 172.16.1.4
7a:a1:d2:fa:86:9b local  bond-to-ToR01         200

VNI 10100 #MACs (local and remote) 2

MAC               Type   Intf/Remote VTEP      VLAN
a2:af:dc:0a:cd:6c local  bond-to-ToR01         100
4e:4f:c1:03:32:f3 remote 172.16.1.4
root@spine01:~# net show evpn vni detail
VNI: 10200
 Type: L2
 Tenant VRF: default
 VxLAN interface: vxlan10200
 VxLAN ifIndex: 9
 Local VTEP IP: 172.16.1.1
 Remote VTEPs for this VNI:
  172.16.1.4
 Number of MACs (local and remote) known for this VNI: 2
 Number of ARPs (IPv4 and IPv6, local and remote) known for this VNI: 0
 Advertise-gw-macip: No

VNI: 10100
 Type: L2
 Tenant VRF: default
 VxLAN interface: vxlan10100
 VxLAN ifIndex: 10
 Local VTEP IP: 172.16.1.1
 Remote VTEPs for this VNI:
  172.16.1.4
 Number of MACs (local and remote) known for this VNI: 2
 Number of ARPs (IPv4 and IPv6, local and remote) known for this VNI: 0
 Advertise-gw-macip: No
root@spine01:~# net show bgp
show bgp ipv4 unicast
=====================
BGP table version is 16, local router ID is 172.16.1.2
Status codes: s suppressed, d damped, h history, * valid, > best, = multipath,
              i internal, r RIB-failure, S Stale, R Removed
Origin codes: i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight Path
*  10.10.10.0/30    10.10.10.1               0             0 65000 ?
*>                  0.0.0.0                  0         32768 ?
*> 10.10.10.4/30    10.10.10.1               0             0 65000 ?
*> 10.10.10.8/30    10.10.10.1               0             0 65000 ?
*                   10.10.10.17                            0 65000 65002 ?
*> 10.10.10.12/30   10.10.10.1               0             0 65000 ?
*                   10.10.10.17                            0 65000 65002 ?
*  10.10.10.16/30   10.10.10.17              0             0 65000 ?
*>                  0.0.0.0                  0         32768 ?
*> 10.10.10.20/30   10.10.10.17              0             0 65000 ?
*  10.10.10.24/30   10.10.10.1                             0 65000 65002 ?
*>                  10.10.10.17              0             0 65000 ?
*  10.10.10.28/30   10.10.10.1                             0 65000 65002 ?
*>                  10.10.10.17              0             0 65000 ?
*  169.254.1.0/30   10.10.10.1                             0 65000 65002 ?
*                   10.10.10.17                            0 65000 65002 ?
*>                  0.0.0.0                  0         32768 ?
*> 172.16.1.1/32    0.0.0.0                  0         32768 ?
*> 172.16.1.2/32    0.0.0.0                  0         32768 ?
*> 172.16.1.4/32    10.10.10.1                             0 65000 65002 ?
*=                  10.10.10.17                            0 65000 65002 ?
*> 172.16.1.5/32    10.10.10.1                             0 65000 65002 ?
*=                  10.10.10.17                            0 65000 65002 ?
*> 172.16.1.6/32    10.10.10.1                             0 65000 65002 ?
*=                  10.10.10.17                            0 65000 65002 ?
*> 172.16.222.1/32  10.10.10.1               0             0 65000 ?
*> 172.16.222.2/32  10.10.10.17              0             0 65000 ?

Displayed  16 routes and 27 total paths


show bgp ipv6 unicast
=====================
No BGP prefixes displayed, 0 exist
root@spine01:~# net show bgp evpn route vni all
VNI: 10200

BGP table version is 4, local router ID is 172.16.1.2
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete
EVPN type-2 prefix: [2]:[ESI]:[EthTag]:[MAClen]:[MAC]:[IPlen]:[IP]
EVPN type-3 prefix: [3]:[EthTag]:[IPlen]:[OrigIP]
EVPN type-5 prefix: [5]:[ESI]:[EthTag]:[IPlen]:[IP]

   Network          Next Hop            Metric LocPrf Weight Path
*  [2]:[0]:[0]:[48]:[12:ed:20:2e:b3:1a]
                    172.16.1.4                             0 65000 65002 i
*  [2]:[0]:[0]:[48]:[12:ed:20:2e:b3:1a]
                    172.16.1.4                             0 65000 65002 i
*  [2]:[0]:[0]:[48]:[12:ed:20:2e:b3:1a]
                    172.16.1.4                             0 65000 65002 i
*> [2]:[0]:[0]:[48]:[12:ed:20:2e:b3:1a]
                    172.16.1.4                             0 65000 65002 i
*> [2]:[0]:[0]:[48]:[7a:a1:d2:fa:86:9b]
                    172.16.1.1                         32768 i
*> [3]:[0]:[32]:[172.16.1.1]
                    172.16.1.1                         32768 i
*  [3]:[0]:[32]:[172.16.1.4]
                    172.16.1.4                             0 65000 65002 i
*  [3]:[0]:[32]:[172.16.1.4]
                    172.16.1.4                             0 65000 65002 i
*  [3]:[0]:[32]:[172.16.1.4]
                    172.16.1.4                             0 65000 65002 i
*> [3]:[0]:[32]:[172.16.1.4]
                    172.16.1.4                             0 65000 65002 i

Displayed 4 prefixes (10 paths)
VNI: 10100

BGP table version is 4, local router ID is 172.16.1.2
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete
EVPN type-2 prefix: [2]:[ESI]:[EthTag]:[MAClen]:[MAC]:[IPlen]:[IP]
EVPN type-3 prefix: [3]:[EthTag]:[IPlen]:[OrigIP]
EVPN type-5 prefix: [5]:[ESI]:[EthTag]:[IPlen]:[IP]

   Network          Next Hop            Metric LocPrf Weight Path
*  [2]:[0]:[0]:[48]:[4e:4f:c1:03:32:f3]
                    172.16.1.4                             0 65000 65002 i
*  [2]:[0]:[0]:[48]:[4e:4f:c1:03:32:f3]
                    172.16.1.4                             0 65000 65002 i
*  [2]:[0]:[0]:[48]:[4e:4f:c1:03:32:f3]
                    172.16.1.4                             0 65000 65002 i
*> [2]:[0]:[0]:[48]:[4e:4f:c1:03:32:f3]
                    172.16.1.4                             0 65000 65002 i
*> [2]:[0]:[0]:[48]:[a2:af:dc:0a:cd:6c]
                    172.16.1.1                         32768 i
*> [3]:[0]:[32]:[172.16.1.1]
                    172.16.1.1                         32768 i
*  [3]:[0]:[32]:[172.16.1.4]
                    172.16.1.4                             0 65000 65002 i
*  [3]:[0]:[32]:[172.16.1.4]
                    172.16.1.4                             0 65000 65002 i
*  [3]:[0]:[32]:[172.16.1.4]
                    172.16.1.4                             0 65000 65002 i
*> [3]:[0]:[32]:[172.16.1.4]
                    172.16.1.4                             0 65000 65002 i

Displayed 4 prefixes (10 paths)
root@spine01:~# net show bgp evpn summary
BGP router identifier 172.16.1.2, local AS number 65001 vrf-id 0
BGP table version 0
RIB entries 15, using 2280 bytes of memory
Peers 2, using 39 KiB of memory
Peer groups 1, using 64 bytes of memory

Neighbor          V         AS MsgRcvd MsgSent   TblVer  InQ OutQ  Up/Down State/PfxRcd
BB01(10.10.10.1)  4      65000     387     383        0    0    0 00:18:01            8
BB02(10.10.10.17) 4      65000     387     383        0    0    0 00:18:01            8

Total number of neighbors 2
root@spine01:~# net show bgp evpn vni
Advertise Gateway Macip: Disabled
Advertise All VNI flag: Enabled
BUM flooding: Head-end replication
Number of L2 VNIs: 2
Number of L3 VNIs: 0
Flags: * - Kernel
  VNI        Type RD                    Import RT                 Export RT                 Tenant VRF        
* 10200      L2   172.16.1.2:2          65001:10200               65001:10200              default            
* 10100      L2   172.16.1.2:3          65001:10100               65001:10100              default
root@spine01:~# net show route
show ip route
=============
Codes: K - kernel route, C - connected, S - static, R - RIP,
       O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
       T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
       F - PBR,
       > - selected route, * - FIB route

C>* 10.10.10.0/30 is directly connected, swp1, 00:18:12
B>* 10.10.10.4/30 [20/0] via 10.10.10.1, swp1, 00:18:08
B>* 10.10.10.8/30 [20/0] via 10.10.10.1, swp1, 00:18:08
B>* 10.10.10.12/30 [20/0] via 10.10.10.1, swp1, 00:18:08
C>* 10.10.10.16/30 is directly connected, swp2, 00:18:12
B>* 10.10.10.20/30 [20/0] via 10.10.10.17, swp2, 00:18:08
B>* 10.10.10.24/30 [20/0] via 10.10.10.17, swp2, 00:18:08
B>* 10.10.10.28/30 [20/0] via 10.10.10.17, swp2, 00:18:08
C>* 169.254.1.0/30 is directly connected, peerlink.4094, 00:18:10
C>* 172.16.1.1/32 is directly connected, lo, 00:18:00
C>* 172.16.1.2/32 is directly connected, lo, 00:18:12
B>* 172.16.1.4/32 [20/0] via 10.10.10.1, swp1, 00:18:06
  *                      via 10.10.10.17, swp2, 00:18:06
B>* 172.16.1.5/32 [20/0] via 10.10.10.1, swp1, 00:18:08
  *                      via 10.10.10.17, swp2, 00:18:08
B>* 172.16.1.6/32 [20/0] via 10.10.10.1, swp1, 00:18:08
  *                      via 10.10.10.17, swp2, 00:18:08
B>* 172.16.222.1/32 [20/0] via 10.10.10.1, swp1, 00:18:08
B>* 172.16.222.2/32 [20/0] via 10.10.10.17, swp2, 00:18:08



show ipv6 route
===============
Codes: K - kernel route, C - connected, S - static, R - RIPng,
       O - OSPFv3, I - IS-IS, B - BGP, N - NHRP, T - Table,
       v - VNC, V - VNC-Direct, A - Babel, D - SHARP, F - PBR,
       > - selected route, * - FIB route

C * fe80::/64 is directly connected, bridge, 00:18:06
C * fe80::/64 is directly connected, peerlink.4094, 00:18:09
C * fe80::/64 is directly connected, swp1, 00:18:11
C * fe80::/64 is directly connected, swp2, 00:18:11
C>* fe80::/64 is directly connected, eth0, 00:18:12
root@spine01:~# net show bridge macs

VLAN      Master  Interface      MAC                TunnelDest  State      Flags          LastSeen
--------  ------  -------------  -----------------  ----------  ---------  -------------  --------
1         bridge  bond-to-ToR01  0c:1e:ed:9f:32:02                                        00:00:09
1         bridge  bond-to-ToR01  0e:1e:ed:9f:32:01                                        00:00:03
1         bridge  bond-to-ToR01  0e:1e:ed:9f:32:02                                        00:00:09
100       bridge  bond-to-ToR01  a2:af:dc:0a:cd:6c                                        00:11:11
100       bridge  vxlan10100     4e:4f:c1:03:32:f3                         offload        00:14:23
200       bridge  bond-to-ToR01  7a:a1:d2:fa:86:9b                                        00:00:09
200       bridge  vxlan10200     12:ed:20:2e:b3:1a                         offload        00:14:14
untagged          vxlan10100     00:00:00:00:00:00  172.16.1.4  permanent  self           00:18:15
untagged          vxlan10100     4e:4f:c1:03:32:f3  172.16.1.4             self, offload  00:14:23
untagged          vxlan10200     00:00:00:00:00:00  172.16.1.4  permanent  self           00:18:15
untagged          vxlan10200     12:ed:20:2e:b3:1a  172.16.1.4             self, offload  00:14:14
untagged  bridge  bond-to-ToR01  0c:1e:ed:91:74:05              permanent                 00:18:20
untagged  bridge  peerlink       0c:1e:ed:91:74:03              permanent                 00:18:20
untagged  bridge  vxlan10100     76:ce:61:90:e8:67              permanent                 00:18:20
untagged  bridge  vxlan10200     06:3d:ed:af:51:d5              permanent                 00:18:20
root@spine01:~#

spine02

root@spine02:~# net show evpn
L2 VNIs: 2
L3 VNIs: 0
Advertise gateway mac-ip: No
Duplicate address detection: Enable
  Detection max-moves 5, time 180
root@spine02:~# net show evpn mac vni all
VNI 10200 #MACs (local and remote) 2

MAC               Type   Intf/Remote VTEP      VLAN
12:ed:20:2e:b3:1a remote 172.16.1.4
7a:a1:d2:fa:86:9b local  bond-to-ToR01         200

VNI 10100 #MACs (local and remote) 2

MAC               Type   Intf/Remote VTEP      VLAN
a2:af:dc:0a:cd:6c local  bond-to-ToR01         100
4e:4f:c1:03:32:f3 remote 172.16.1.4
root@spine02:~# net show evpn vni detail
VNI: 10200
 Type: L2
 Tenant VRF: default
 VxLAN interface: vxlan10200
 VxLAN ifIndex: 9
 Local VTEP IP: 172.16.1.1
 Remote VTEPs for this VNI:
  172.16.1.4
 Number of MACs (local and remote) known for this VNI: 2
 Number of ARPs (IPv4 and IPv6, local and remote) known for this VNI: 0
 Advertise-gw-macip: No

VNI: 10100
 Type: L2
 Tenant VRF: default
 VxLAN interface: vxlan10100
 VxLAN ifIndex: 10
 Local VTEP IP: 172.16.1.1
 Remote VTEPs for this VNI:
  172.16.1.4
 Number of MACs (local and remote) known for this VNI: 2
 Number of ARPs (IPv4 and IPv6, local and remote) known for this VNI: 0
 Advertise-gw-macip: No
root@spine02:~# net show bgp
show bgp ipv4 unicast
=====================
BGP table version is 20, local router ID is 172.16.1.3
Status codes: s suppressed, d damped, h history, * valid, > best, = multipath,
              i internal, r RIB-failure, S Stale, R Removed
Origin codes: i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight Path
*> 10.10.10.0/30    10.10.10.5               0             0 65000 ?
*  10.10.10.4/30    10.10.10.5               0             0 65000 ?
*>                  0.0.0.0                  0         32768 ?
*  10.10.10.8/30    10.10.10.21                            0 65000 65002 ?
*>                  10.10.10.5               0             0 65000 ?
*  10.10.10.12/30   10.10.10.21                            0 65000 65002 ?
*>                  10.10.10.5               0             0 65000 ?
*> 10.10.10.16/30   10.10.10.21              0             0 65000 ?
*  10.10.10.20/30   10.10.10.21              0             0 65000 ?
*>                  0.0.0.0                  0         32768 ?
*  10.10.10.24/30   10.10.10.5                             0 65000 65002 ?
*>                  10.10.10.21              0             0 65000 ?
*> 10.10.10.28/30   10.10.10.21              0             0 65000 ?
*                   10.10.10.5                             0 65000 65002 ?
*> 169.254.1.0/30   0.0.0.0                  0         32768 ?
*                   10.10.10.21                            0 65000 65002 ?
*                   10.10.10.5                             0 65000 65002 ?
*> 172.16.1.1/32    0.0.0.0                  0         32768 ?
*> 172.16.1.3/32    0.0.0.0                  0         32768 ?
*> 172.16.1.4/32    10.10.10.5                             0 65000 65002 ?
*=                  10.10.10.21                            0 65000 65002 ?
*> 172.16.1.5/32    10.10.10.5                             0 65000 65002 ?
*=                  10.10.10.21                            0 65000 65002 ?
*= 172.16.1.6/32    10.10.10.21                            0 65000 65002 ?
*>                  10.10.10.5                             0 65000 65002 ?
*> 172.16.222.1/32  10.10.10.5               0             0 65000 ?
*> 172.16.222.2/32  10.10.10.21              0             0 65000 ?

Displayed  16 routes and 27 total paths


show bgp ipv6 unicast
=====================
No BGP prefixes displayed, 0 exist
root@spine02:~# net show bgp evpn route vni all
VNI: 10200

BGP table version is 4, local router ID is 172.16.1.3
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete
EVPN type-2 prefix: [2]:[ESI]:[EthTag]:[MAClen]:[MAC]:[IPlen]:[IP]
EVPN type-3 prefix: [3]:[EthTag]:[IPlen]:[OrigIP]
EVPN type-5 prefix: [5]:[ESI]:[EthTag]:[IPlen]:[IP]

   Network          Next Hop            Metric LocPrf Weight Path
*  [2]:[0]:[0]:[48]:[12:ed:20:2e:b3:1a]
                    172.16.1.4                             0 65000 65002 i
*  [2]:[0]:[0]:[48]:[12:ed:20:2e:b3:1a]
                    172.16.1.4                             0 65000 65002 i
*  [2]:[0]:[0]:[48]:[12:ed:20:2e:b3:1a]
                    172.16.1.4                             0 65000 65002 i
*> [2]:[0]:[0]:[48]:[12:ed:20:2e:b3:1a]
                    172.16.1.4                             0 65000 65002 i
*> [2]:[0]:[0]:[48]:[7a:a1:d2:fa:86:9b]
                    172.16.1.1                         32768 i
*> [3]:[0]:[32]:[172.16.1.1]
                    172.16.1.1                         32768 i
*  [3]:[0]:[32]:[172.16.1.4]
                    172.16.1.4                             0 65000 65002 i
*  [3]:[0]:[32]:[172.16.1.4]
                    172.16.1.4                             0 65000 65002 i
*  [3]:[0]:[32]:[172.16.1.4]
                    172.16.1.4                             0 65000 65002 i
*> [3]:[0]:[32]:[172.16.1.4]
                    172.16.1.4                             0 65000 65002 i

Displayed 4 prefixes (10 paths)
VNI: 10100

BGP table version is 4, local router ID is 172.16.1.3
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete
EVPN type-2 prefix: [2]:[ESI]:[EthTag]:[MAClen]:[MAC]:[IPlen]:[IP]
EVPN type-3 prefix: [3]:[EthTag]:[IPlen]:[OrigIP]
EVPN type-5 prefix: [5]:[ESI]:[EthTag]:[IPlen]:[IP]

   Network          Next Hop            Metric LocPrf Weight Path
*  [2]:[0]:[0]:[48]:[4e:4f:c1:03:32:f3]
                    172.16.1.4                             0 65000 65002 i
*  [2]:[0]:[0]:[48]:[4e:4f:c1:03:32:f3]
                    172.16.1.4                             0 65000 65002 i
*  [2]:[0]:[0]:[48]:[4e:4f:c1:03:32:f3]
                    172.16.1.4                             0 65000 65002 i
*> [2]:[0]:[0]:[48]:[4e:4f:c1:03:32:f3]
                    172.16.1.4                             0 65000 65002 i
*> [2]:[0]:[0]:[48]:[a2:af:dc:0a:cd:6c]
                    172.16.1.1                         32768 i
*> [3]:[0]:[32]:[172.16.1.1]
                    172.16.1.1                         32768 i
*  [3]:[0]:[32]:[172.16.1.4]
                    172.16.1.4                             0 65000 65002 i
*  [3]:[0]:[32]:[172.16.1.4]
                    172.16.1.4                             0 65000 65002 i
*  [3]:[0]:[32]:[172.16.1.4]
                    172.16.1.4                             0 65000 65002 i
*> [3]:[0]:[32]:[172.16.1.4]
                    172.16.1.4                             0 65000 65002 i

Displayed 4 prefixes (10 paths)
root@spine02:~# net show bgp evpn summary
BGP router identifier 172.16.1.3, local AS number 65001 vrf-id 0
BGP table version 0
RIB entries 15, using 2280 bytes of memory
Peers 2, using 39 KiB of memory
Peer groups 1, using 64 bytes of memory

Neighbor          V         AS MsgRcvd MsgSent   TblVer  InQ OutQ  Up/Down State/PfxRcd
BB01(10.10.10.5)  4      65000     393     387        0    0    0 00:18:05            8
BB02(10.10.10.21) 4      65000     390     385        0    0    0 00:18:03            8

Total number of neighbors 2
root@spine02:~# net show bgp evpn vni
Advertise Gateway Macip: Disabled
Advertise All VNI flag: Enabled
BUM flooding: Head-end replication
Number of L2 VNIs: 2
Number of L3 VNIs: 0
Flags: * - Kernel
  VNI        Type RD                    Import RT                 Export RT                 Tenant VRF        
* 10200      L2   172.16.1.3:2          65001:10200               65001:10200              default            
* 10100      L2   172.16.1.3:3          65001:10100               65001:10100              default
root@spine02:~# net show route
show ip route
=============
Codes: K - kernel route, C - connected, S - static, R - RIP,
       O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
       T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
       F - PBR,
       > - selected route, * - FIB route

B>* 10.10.10.0/30 [20/0] via 10.10.10.5, swp1, 00:18:13
C>* 10.10.10.4/30 is directly connected, swp1, 00:18:19
B>* 10.10.10.8/30 [20/0] via 10.10.10.5, swp1, 00:18:13
B>* 10.10.10.12/30 [20/0] via 10.10.10.5, swp1, 00:18:13
B>* 10.10.10.16/30 [20/0] via 10.10.10.21, swp2, 00:18:11
C>* 10.10.10.20/30 is directly connected, swp2, 00:18:19
B>* 10.10.10.24/30 [20/0] via 10.10.10.21, swp2, 00:18:11
B>* 10.10.10.28/30 [20/0] via 10.10.10.21, swp2, 00:18:11
C>* 169.254.1.0/30 is directly connected, peerlink.4094, 00:18:11
C>* 172.16.1.1/32 is directly connected, lo, 00:18:07
C>* 172.16.1.3/32 is directly connected, lo, 00:18:19
B>* 172.16.1.4/32 [20/0] via 10.10.10.5, swp1, 00:18:07
  *                      via 10.10.10.21, swp2, 00:18:07
B>* 172.16.1.5/32 [20/0] via 10.10.10.5, swp1, 00:18:11
  *                      via 10.10.10.21, swp2, 00:18:11
B>* 172.16.1.6/32 [20/0] via 10.10.10.5, swp1, 00:18:11
  *                      via 10.10.10.21, swp2, 00:18:11
B>* 172.16.222.1/32 [20/0] via 10.10.10.5, swp1, 00:18:13
B>* 172.16.222.2/32 [20/0] via 10.10.10.21, swp2, 00:18:11



show ipv6 route
===============
Codes: K - kernel route, C - connected, S - static, R - RIPng,
       O - OSPFv3, I - IS-IS, B - BGP, N - NHRP, T - Table,
       v - VNC, V - VNC-Direct, A - Babel, D - SHARP, F - PBR,
       > - selected route, * - FIB route

C * fe80::/64 is directly connected, bridge, 00:18:09
C * fe80::/64 is directly connected, peerlink.4094, 00:18:09
C * fe80::/64 is directly connected, swp1, 00:18:18
C * fe80::/64 is directly connected, swp2, 00:18:19
C>* fe80::/64 is directly connected, eth0, 00:18:19
root@spine02:~# net show bridge macs

VLAN      Master  Interface      MAC                TunnelDest  State      Flags          LastSeen
--------  ------  -------------  -----------------  ----------  ---------  -------------  --------
1         bridge  bond-to-ToR01  0c:1e:ed:9f:32:02                                        00:00:01
1         bridge  bond-to-ToR01  0e:1e:ed:9f:32:01                                        00:00:16
1         bridge  bond-to-ToR01  0e:1e:ed:9f:32:02                                        00:00:10
100       bridge  bond-to-ToR01  a2:af:dc:0a:cd:6c                                        00:00:16
100       bridge  vxlan10100     4e:4f:c1:03:32:f3                         offload        00:14:23
200       bridge  bond-to-ToR01  7a:a1:d2:fa:86:9b                                        00:14:14
200       bridge  vxlan10200     12:ed:20:2e:b3:1a                         offload        00:14:14
untagged          vxlan10100     00:00:00:00:00:00  172.16.1.4  permanent  self           00:18:15
untagged          vxlan10100     4e:4f:c1:03:32:f3  172.16.1.4             self, offload  00:14:23
untagged          vxlan10200     00:00:00:00:00:00  172.16.1.4  permanent  self           00:18:15
untagged          vxlan10200     12:ed:20:2e:b3:1a  172.16.1.4             self, offload  00:14:14
untagged  bridge  bond-to-ToR01  0c:1e:ed:b9:90:05              permanent                 00:18:27
untagged  bridge  peerlink       0c:1e:ed:b9:90:03              permanent                 00:18:27
untagged  bridge  vxlan10100     f6:01:80:3a:ab:8d              permanent                 00:18:27
untagged  bridge  vxlan10200     4a:c7:eb:94:82:ad              permanent                 00:18:27
root@spine02:~#

spine03

root@spine03:~# net show evpn
L2 VNIs: 2
L3 VNIs: 0
Advertise gateway mac-ip: No
Duplicate address detection: Enable
  Detection max-moves 5, time 180
root@spine03:~# net show evpn mac vni all
VNI 10200 #MACs (local and remote) 2

MAC               Type   Intf/Remote VTEP      VLAN
12:ed:20:2e:b3:1a local  bond-to-ToR02         200
7a:a1:d2:fa:86:9b remote 172.16.1.1

VNI 10100 #MACs (local and remote) 2

MAC               Type   Intf/Remote VTEP      VLAN
a2:af:dc:0a:cd:6c remote 172.16.1.1
4e:4f:c1:03:32:f3 local  bond-to-ToR02         100
root@spine03:~# net show evpn vni detail
VNI: 10200
 Type: L2
 Tenant VRF: default
 VxLAN interface: vxlan10200
 VxLAN ifIndex: 9
 Local VTEP IP: 172.16.1.4
 Remote VTEPs for this VNI:
  172.16.1.1
 Number of MACs (local and remote) known for this VNI: 2
 Number of ARPs (IPv4 and IPv6, local and remote) known for this VNI: 0
 Advertise-gw-macip: No

VNI: 10100
 Type: L2
 Tenant VRF: default
 VxLAN interface: vxlan10100
 VxLAN ifIndex: 11
 Local VTEP IP: 172.16.1.4
 Remote VTEPs for this VNI:
  172.16.1.1
 Number of MACs (local and remote) known for this VNI: 2
 Number of ARPs (IPv4 and IPv6, local and remote) known for this VNI: 0
 Advertise-gw-macip: No
root@spine03:~# net show bgp
show bgp ipv4 unicast
=====================
BGP table version is 18, local router ID is 172.16.1.5
Status codes: s suppressed, d damped, h history, * valid, > best, = multipath,
              i internal, r RIB-failure, S Stale, R Removed
Origin codes: i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight Path
*  10.10.10.0/30    10.10.10.25                            0 65000 65001 ?
*>                  10.10.10.9               0             0 65000 ?
*> 10.10.10.4/30    10.10.10.9               0             0 65000 ?
*                   10.10.10.25                            0 65000 65001 ?
*  10.10.10.8/30    10.10.10.9               0             0 65000 ?
*>                  0.0.0.0                  0         32768 ?
*> 10.10.10.12/30   10.10.10.9               0             0 65000 ?
*  10.10.10.16/30   10.10.10.9                             0 65000 65001 ?
*>                  10.10.10.25              0             0 65000 ?
*  10.10.10.20/30   10.10.10.9                             0 65000 65001 ?
*>                  10.10.10.25              0             0 65000 ?
*  10.10.10.24/30   10.10.10.25              0             0 65000 ?
*>                  0.0.0.0                  0         32768 ?
*> 10.10.10.28/30   10.10.10.25              0             0 65000 ?
*> 169.254.1.0/30   0.0.0.0                  0         32768 ?
*= 172.16.1.1/32    10.10.10.25                            0 65000 65001 ?
*>                  10.10.10.9                             0 65000 65001 ?
*= 172.16.1.2/32    10.10.10.25                            0 65000 65001 ?
*>                  10.10.10.9                             0 65000 65001 ?
*= 172.16.1.3/32    10.10.10.9                             0 65000 65001 ?
*>                  10.10.10.25                            0 65000 65001 ?
*> 172.16.1.4/32    0.0.0.0                  0         32768 ?
*> 172.16.1.5/32    0.0.0.0                  0         32768 ?
*> 172.16.222.1/32  10.10.10.9               0             0 65000 ?
*> 172.16.222.2/32  10.10.10.25              0             0 65000 ?

Displayed  16 routes and 25 total paths


show bgp ipv6 unicast
=====================
No BGP prefixes displayed, 0 exist
root@spine03:~# net show bgp evpn route vni all
VNI: 10200

BGP table version is 4, local router ID is 172.16.1.5
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete
EVPN type-2 prefix: [2]:[ESI]:[EthTag]:[MAClen]:[MAC]:[IPlen]:[IP]
EVPN type-3 prefix: [3]:[EthTag]:[IPlen]:[OrigIP]
EVPN type-5 prefix: [5]:[ESI]:[EthTag]:[IPlen]:[IP]

   Network          Next Hop            Metric LocPrf Weight Path
*> [2]:[0]:[0]:[48]:[12:ed:20:2e:b3:1a]
                    172.16.1.4                         32768 i
*  [2]:[0]:[0]:[48]:[7a:a1:d2:fa:86:9b]
                    172.16.1.1                             0 65000 65001 i
*  [2]:[0]:[0]:[48]:[7a:a1:d2:fa:86:9b]
                    172.16.1.1                             0 65000 65001 i
*  [2]:[0]:[0]:[48]:[7a:a1:d2:fa:86:9b]
                    172.16.1.1                             0 65000 65001 i
*> [2]:[0]:[0]:[48]:[7a:a1:d2:fa:86:9b]
                    172.16.1.1                             0 65000 65001 i
*  [3]:[0]:[32]:[172.16.1.1]
                    172.16.1.1                             0 65000 65001 i
*  [3]:[0]:[32]:[172.16.1.1]
                    172.16.1.1                             0 65000 65001 i
*  [3]:[0]:[32]:[172.16.1.1]
                    172.16.1.1                             0 65000 65001 i
*> [3]:[0]:[32]:[172.16.1.1]
                    172.16.1.1                             0 65000 65001 i
*> [3]:[0]:[32]:[172.16.1.4]
                    172.16.1.4                         32768 i

Displayed 4 prefixes (10 paths)
VNI: 10100

BGP table version is 4, local router ID is 172.16.1.5
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete
EVPN type-2 prefix: [2]:[ESI]:[EthTag]:[MAClen]:[MAC]:[IPlen]:[IP]
EVPN type-3 prefix: [3]:[EthTag]:[IPlen]:[OrigIP]
EVPN type-5 prefix: [5]:[ESI]:[EthTag]:[IPlen]:[IP]

   Network          Next Hop            Metric LocPrf Weight Path
*> [2]:[0]:[0]:[48]:[4e:4f:c1:03:32:f3]
                    172.16.1.4                         32768 i
*  [2]:[0]:[0]:[48]:[a2:af:dc:0a:cd:6c]
                    172.16.1.1                             0 65000 65001 i
*  [2]:[0]:[0]:[48]:[a2:af:dc:0a:cd:6c]
                    172.16.1.1                             0 65000 65001 i
*  [2]:[0]:[0]:[48]:[a2:af:dc:0a:cd:6c]
                    172.16.1.1                             0 65000 65001 i
*> [2]:[0]:[0]:[48]:[a2:af:dc:0a:cd:6c]
                    172.16.1.1                             0 65000 65001 i
*  [3]:[0]:[32]:[172.16.1.1]
                    172.16.1.1                             0 65000 65001 i
*  [3]:[0]:[32]:[172.16.1.1]
                    172.16.1.1                             0 65000 65001 i
*  [3]:[0]:[32]:[172.16.1.1]
                    172.16.1.1                             0 65000 65001 i
*> [3]:[0]:[32]:[172.16.1.1]
                    172.16.1.1                             0 65000 65001 i
*> [3]:[0]:[32]:[172.16.1.4]
                    172.16.1.4                         32768 i

Displayed 4 prefixes (10 paths)
root@spine03:~# net show bgp evpn summary
BGP router identifier 172.16.1.5, local AS number 65002 vrf-id 0
BGP table version 0
RIB entries 15, using 2280 bytes of memory
Peers 2, using 39 KiB of memory
Peer groups 1, using 64 bytes of memory

Neighbor          V         AS MsgRcvd MsgSent   TblVer  InQ OutQ  Up/Down State/PfxRcd
BB01(10.10.10.9)  4      65000     390     385        0    0    0 00:18:03            8
BB02(10.10.10.25) 4      65000     390     385        0    0    0 00:18:03            8

Total number of neighbors 2
root@spine03:~# net show bgp evpn vni
Advertise Gateway Macip: Disabled
Advertise All VNI flag: Enabled
BUM flooding: Head-end replication
Number of L2 VNIs: 2
Number of L3 VNIs: 0
Flags: * - Kernel
  VNI        Type RD                    Import RT                 Export RT                 Tenant VRF        
* 10200      L2   172.16.1.5:2          65002:10200               65002:10200              default            
* 10100      L2   172.16.1.5:3          65002:10100               65002:10100              default
root@spine03:~# net show route
show ip route
=============
Codes: K - kernel route, C - connected, S - static, R - RIP,
       O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
       T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
       F - PBR,
       > - selected route, * - FIB route

B>* 10.10.10.0/30 [20/0] via 10.10.10.9, swp1, 00:18:10
B>* 10.10.10.4/30 [20/0] via 10.10.10.9, swp1, 00:18:10
C>* 10.10.10.8/30 is directly connected, swp1, 00:18:14
B>* 10.10.10.12/30 [20/0] via 10.10.10.9, swp1, 00:18:10
B>* 10.10.10.16/30 [20/0] via 10.10.10.25, swp2, 00:18:11
B>* 10.10.10.20/30 [20/0] via 10.10.10.25, swp2, 00:18:11
C>* 10.10.10.24/30 is directly connected, swp2, 00:18:14
B>* 10.10.10.28/30 [20/0] via 10.10.10.25, swp2, 00:18:11
C>* 169.254.1.0/30 is directly connected, peerlink.4094, 00:18:12
B>* 172.16.1.1/32 [20/0] via 10.10.10.9, swp1, 00:18:07
  *                      via 10.10.10.25, swp2, 00:18:07
B>* 172.16.1.2/32 [20/0] via 10.10.10.9, swp1, 00:18:08
  *                      via 10.10.10.25, swp2, 00:18:08
B>* 172.16.1.3/32 [20/0] via 10.10.10.25, swp2, 00:18:10
  *                      via 10.10.10.9, swp1, 00:18:10
C>* 172.16.1.4/32 is directly connected, lo, 00:18:02
C>* 172.16.1.5/32 is directly connected, lo, 00:18:14
B>* 172.16.222.1/32 [20/0] via 10.10.10.9, swp1, 00:18:10
B>* 172.16.222.2/32 [20/0] via 10.10.10.25, swp2, 00:18:11



show ipv6 route
===============
Codes: K - kernel route, C - connected, S - static, R - RIPng,
       O - OSPFv3, I - IS-IS, B - BGP, N - NHRP, T - Table,
       v - VNC, V - VNC-Direct, A - Babel, D - SHARP, F - PBR,
       > - selected route, * - FIB route

C * fe80::/64 is directly connected, bridge, 00:18:10
C * fe80::/64 is directly connected, peerlink.4094, 00:18:10
C * fe80::/64 is directly connected, swp1, 00:18:13
C * fe80::/64 is directly connected, swp2, 00:18:13
C>* fe80::/64 is directly connected, eth0, 00:18:14
root@spine03:~# net show bridge macs

VLAN      Master  Interface      MAC                TunnelDest  State      Flags          LastSeen
--------  ------  -------------  -----------------  ----------  ---------  -------------  --------
1         bridge  bond-to-ToR02  0c:1e:ed:2c:26:02                                        00:00:11
1         bridge  bond-to-ToR02  0e:1e:ed:2c:26:01                                        00:00:05
1         bridge  bond-to-ToR02  0e:1e:ed:2c:26:02                                        00:00:11
100       bridge  bond-to-ToR02  4e:4f:c1:03:32:f3                                        00:11:11
100       bridge  vxlan10100     a2:af:dc:0a:cd:6c                         offload        00:14:23
200       bridge  bond-to-ToR02  12:ed:20:2e:b3:1a                                        00:10:08
200       bridge  vxlan10200     7a:a1:d2:fa:86:9b                         offload        00:14:14
untagged          vxlan10100     00:00:00:00:00:00  172.16.1.1  permanent  self           00:18:16
untagged          vxlan10100     a2:af:dc:0a:cd:6c  172.16.1.1             self, offload  00:14:23
untagged          vxlan10200     00:00:00:00:00:00  172.16.1.1  permanent  self           00:18:16
untagged          vxlan10200     7a:a1:d2:fa:86:9b  172.16.1.1             self, offload  00:14:14
untagged  bridge  bond-to-ToR02  0c:1e:ed:49:da:05              permanent                 00:18:21
untagged  bridge  peerlink       0c:1e:ed:49:da:03              permanent                 00:18:21
untagged  bridge  vxlan10100     1e:92:4b:a3:78:2e              permanent                 00:18:21
untagged  bridge  vxlan10200     d2:ba:54:1d:df:35              permanent                 00:18:21
root@spine03:~#

spine04

oot@spine04:~# net show evpn
L2 VNIs: 2
L3 VNIs: 0
Advertise gateway mac-ip: No
Duplicate address detection: Enable
  Detection max-moves 5, time 180
root@spine04:~# net show evpn mac vni all
VNI 10200 #MACs (local and remote) 2

MAC               Type   Intf/Remote VTEP      VLAN
12:ed:20:2e:b3:1a local  bond-to-ToR02         200
7a:a1:d2:fa:86:9b remote 172.16.1.1

VNI 10100 #MACs (local and remote) 2

MAC               Type   Intf/Remote VTEP      VLAN
a2:af:dc:0a:cd:6c remote 172.16.1.1
4e:4f:c1:03:32:f3 local  bond-to-ToR02         100
root@spine04:~# net show evpn vni detail
VNI: 10200
 Type: L2
 Tenant VRF: default
 VxLAN interface: vxlan10200
 VxLAN ifIndex: 9
 Local VTEP IP: 172.16.1.4
 Remote VTEPs for this VNI:
  172.16.1.1
 Number of MACs (local and remote) known for this VNI: 2
 Number of ARPs (IPv4 and IPv6, local and remote) known for this VNI: 0
 Advertise-gw-macip: No

VNI: 10100
 Type: L2
 Tenant VRF: default
 VxLAN interface: vxlan10100
 VxLAN ifIndex: 11
 Local VTEP IP: 172.16.1.4
 Remote VTEPs for this VNI:
  172.16.1.1
 Number of MACs (local and remote) known for this VNI: 2
 Number of ARPs (IPv4 and IPv6, local and remote) known for this VNI: 0
 Advertise-gw-macip: No
root@spine04:~# net show bgp
show bgp ipv4 unicast
=====================
BGP table version is 18, local router ID is 172.16.1.6
Status codes: s suppressed, d damped, h history, * valid, > best, = multipath,
              i internal, r RIB-failure, S Stale, R Removed
Origin codes: i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight Path
*  10.10.10.0/30    10.10.10.29                            0 65000 65001 ?
*>                  10.10.10.13              0             0 65000 ?
*  10.10.10.4/30    10.10.10.29                            0 65000 65001 ?
*>                  10.10.10.13              0             0 65000 ?
*> 10.10.10.8/30    10.10.10.13              0             0 65000 ?
*  10.10.10.12/30   10.10.10.13              0             0 65000 ?
*>                  0.0.0.0                  0         32768 ?
*  10.10.10.16/30   10.10.10.13                            0 65000 65001 ?
*>                  10.10.10.29              0             0 65000 ?
*> 10.10.10.20/30   10.10.10.29              0             0 65000 ?
*                   10.10.10.13                            0 65000 65001 ?
*> 10.10.10.24/30   10.10.10.29              0             0 65000 ?
*  10.10.10.28/30   10.10.10.29              0             0 65000 ?
*>                  0.0.0.0                  0         32768 ?
*> 169.254.1.0/30   0.0.0.0                  0         32768 ?
*= 172.16.1.1/32    10.10.10.29                            0 65000 65001 ?
*>                  10.10.10.13                            0 65000 65001 ?
*= 172.16.1.2/32    10.10.10.29                            0 65000 65001 ?
*>                  10.10.10.13                            0 65000 65001 ?
*= 172.16.1.3/32    10.10.10.29                            0 65000 65001 ?
*>                  10.10.10.13                            0 65000 65001 ?
*> 172.16.1.4/32    0.0.0.0                  0         32768 ?
*> 172.16.1.6/32    0.0.0.0                  0         32768 ?
*> 172.16.222.1/32  10.10.10.13              0             0 65000 ?
*> 172.16.222.2/32  10.10.10.29              0             0 65000 ?

Displayed  16 routes and 25 total paths


show bgp ipv6 unicast
=====================
No BGP prefixes displayed, 0 exist
root@spine04:~# net show bgp evpn route vni all
VNI: 10200

BGP table version is 4, local router ID is 172.16.1.6
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete
EVPN type-2 prefix: [2]:[ESI]:[EthTag]:[MAClen]:[MAC]:[IPlen]:[IP]
EVPN type-3 prefix: [3]:[EthTag]:[IPlen]:[OrigIP]
EVPN type-5 prefix: [5]:[ESI]:[EthTag]:[IPlen]:[IP]

   Network          Next Hop            Metric LocPrf Weight Path
*> [2]:[0]:[0]:[48]:[12:ed:20:2e:b3:1a]
                    172.16.1.4                         32768 i
*  [2]:[0]:[0]:[48]:[7a:a1:d2:fa:86:9b]
                    172.16.1.1                             0 65000 65001 i
*  [2]:[0]:[0]:[48]:[7a:a1:d2:fa:86:9b]
                    172.16.1.1                             0 65000 65001 i
*  [2]:[0]:[0]:[48]:[7a:a1:d2:fa:86:9b]
                    172.16.1.1                             0 65000 65001 i
*> [2]:[0]:[0]:[48]:[7a:a1:d2:fa:86:9b]
                    172.16.1.1                             0 65000 65001 i
*  [3]:[0]:[32]:[172.16.1.1]
                    172.16.1.1                             0 65000 65001 i
*  [3]:[0]:[32]:[172.16.1.1]
                    172.16.1.1                             0 65000 65001 i
*  [3]:[0]:[32]:[172.16.1.1]
                    172.16.1.1                             0 65000 65001 i
*> [3]:[0]:[32]:[172.16.1.1]
                    172.16.1.1                             0 65000 65001 i
*> [3]:[0]:[32]:[172.16.1.4]
                    172.16.1.4                         32768 i

Displayed 4 prefixes (10 paths)
VNI: 10100

BGP table version is 4, local router ID is 172.16.1.6
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete
EVPN type-2 prefix: [2]:[ESI]:[EthTag]:[MAClen]:[MAC]:[IPlen]:[IP]
EVPN type-3 prefix: [3]:[EthTag]:[IPlen]:[OrigIP]
EVPN type-5 prefix: [5]:[ESI]:[EthTag]:[IPlen]:[IP]

   Network          Next Hop            Metric LocPrf Weight Path
*> [2]:[0]:[0]:[48]:[4e:4f:c1:03:32:f3]
                    172.16.1.4                         32768 i
*  [2]:[0]:[0]:[48]:[a2:af:dc:0a:cd:6c]
                    172.16.1.1                             0 65000 65001 i
*  [2]:[0]:[0]:[48]:[a2:af:dc:0a:cd:6c]
                    172.16.1.1                             0 65000 65001 i
*  [2]:[0]:[0]:[48]:[a2:af:dc:0a:cd:6c]
                    172.16.1.1                             0 65000 65001 i
*> [2]:[0]:[0]:[48]:[a2:af:dc:0a:cd:6c]
                    172.16.1.1                             0 65000 65001 i
*  [3]:[0]:[32]:[172.16.1.1]
                    172.16.1.1                             0 65000 65001 i
*  [3]:[0]:[32]:[172.16.1.1]
                    172.16.1.1                             0 65000 65001 i
*  [3]:[0]:[32]:[172.16.1.1]
                    172.16.1.1                             0 65000 65001 i
*> [3]:[0]:[32]:[172.16.1.1]
                    172.16.1.1                             0 65000 65001 i
*> [3]:[0]:[32]:[172.16.1.4]
                    172.16.1.4                         32768 i

Displayed 4 prefixes (10 paths)
root@spine04:~# net show bgp evpn summary
BGP router identifier 172.16.1.6, local AS number 65002 vrf-id 0
BGP table version 0
RIB entries 15, using 2280 bytes of memory
Peers 2, using 39 KiB of memory
Peer groups 1, using 64 bytes of memory

Neighbor          V         AS MsgRcvd MsgSent   TblVer  InQ OutQ  Up/Down State/PfxRcd
BB01(10.10.10.13) 4      65000     393     388        0    0    0 00:18:05            8
BB02(10.10.10.29) 4      65000     390     387        0    0    0 00:18:03            8

Total number of neighbors 2
root@spine04:~# net show bgp evpn vni
Advertise Gateway Macip: Disabled
Advertise All VNI flag: Enabled
BUM flooding: Head-end replication
Number of L2 VNIs: 2
Number of L3 VNIs: 0
Flags: * - Kernel
  VNI        Type RD                    Import RT                 Export RT                 Tenant VRF        
* 10200      L2   172.16.1.6:2          65002:10200               65002:10200              default            
* 10100      L2   172.16.1.6:3          65002:10100               65002:10100              default
root@spine04:~# net show route
show ip route
=============
Codes: K - kernel route, C - connected, S - static, R - RIP,
       O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
       T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
       F - PBR,
       > - selected route, * - FIB route

B>* 10.10.10.0/30 [20/0] via 10.10.10.13, swp1, 00:18:12
B>* 10.10.10.4/30 [20/0] via 10.10.10.13, swp1, 00:18:12
B>* 10.10.10.8/30 [20/0] via 10.10.10.13, swp1, 00:18:12
C>* 10.10.10.12/30 is directly connected, swp1, 00:18:19
B>* 10.10.10.16/30 [20/0] via 10.10.10.29, swp2, 00:18:11
B>* 10.10.10.20/30 [20/0] via 10.10.10.29, swp2, 00:18:11
B>* 10.10.10.24/30 [20/0] via 10.10.10.29, swp2, 00:18:11
C>* 10.10.10.28/30 is directly connected, swp2, 00:18:19
C>* 169.254.1.0/30 is directly connected, peerlink.4094, 00:18:12
B>* 172.16.1.1/32 [20/0] via 10.10.10.13, swp1, 00:18:06
  *                      via 10.10.10.29, swp2, 00:18:06
B>* 172.16.1.2/32 [20/0] via 10.10.10.13, swp1, 00:18:08
  *                      via 10.10.10.29, swp2, 00:18:08
B>* 172.16.1.3/32 [20/0] via 10.10.10.13, swp1, 00:18:11
  *                      via 10.10.10.29, swp2, 00:18:11
C>* 172.16.1.4/32 is directly connected, lo, 00:18:06
C>* 172.16.1.6/32 is directly connected, lo, 00:18:19
B>* 172.16.222.1/32 [20/0] via 10.10.10.13, swp1, 00:18:12
B>* 172.16.222.2/32 [20/0] via 10.10.10.29, swp2, 00:18:11



show ipv6 route
===============
Codes: K - kernel route, C - connected, S - static, R - RIPng,
       O - OSPFv3, I - IS-IS, B - BGP, N - NHRP, T - Table,
       v - VNC, V - VNC-Direct, A - Babel, D - SHARP, F - PBR,
       > - selected route, * - FIB route

C * fe80::/64 is directly connected, bridge, 00:18:07
C * fe80::/64 is directly connected, peerlink.4094, 00:18:10
C * fe80::/64 is directly connected, swp1, 00:18:18
C * fe80::/64 is directly connected, swp2, 00:18:18
C>* fe80::/64 is directly connected, eth0, 00:18:19
root@spine04:~# net show bridge macs

VLAN      Master  Interface      MAC                TunnelDest  State      Flags          LastSeen
--------  ------  -------------  -----------------  ----------  ---------  -------------  --------
1         bridge  bond-to-ToR02  0c:1e:ed:2c:26:02                                        <1 sec
1         bridge  bond-to-ToR02  0e:1e:ed:2c:26:01                                        00:00:15
1         bridge  bond-to-ToR02  0e:1e:ed:2c:26:02                                        00:00:10
100       bridge  bond-to-ToR02  4e:4f:c1:03:32:f3                                        00:00:15
100       bridge  vxlan10100     a2:af:dc:0a:cd:6c                         offload        00:14:23
200       bridge  bond-to-ToR02  12:ed:20:2e:b3:1a                                        00:00:15
200       bridge  vxlan10200     7a:a1:d2:fa:86:9b                         offload        00:14:14
untagged          vxlan10100     00:00:00:00:00:00  172.16.1.1  permanent  self           00:18:16
untagged          vxlan10100     a2:af:dc:0a:cd:6c  172.16.1.1             self, offload  00:14:23
untagged          vxlan10200     00:00:00:00:00:00  172.16.1.1  permanent  self           00:18:16
untagged          vxlan10200     7a:a1:d2:fa:86:9b  172.16.1.1             self, offload  00:14:14
untagged  bridge  bond-to-ToR02  0c:1e:ed:5a:03:05              permanent                 00:18:27
untagged  bridge  peerlink       0c:1e:ed:5a:03:03              permanent                 00:18:27
untagged  bridge  vxlan10100     76:42:cb:ab:0a:84              permanent                 00:18:27
untagged  bridge  vxlan10200     2e:70:48:a6:59:a6              permanent                 00:18:27
root@spine04:~#

VNIを持たないBBたちもMP-BGP EVPNへ参加し、各VTEPへのルーティングテーブルが確認できることがわかります

BB01

root@BB01:~# net show bgp evpn route
BGP table version is 2, local router ID is 172.16.222.1
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete
EVPN type-2 prefix: [2]:[ESI]:[EthTag]:[MAClen]:[MAC]:[IPlen]:[IP]
EVPN type-3 prefix: [3]:[EthTag]:[IPlen]:[OrigIP]
EVPN type-5 prefix: [5]:[ESI]:[EthTag]:[IPlen]:[IP]

   Network          Next Hop            Metric LocPrf Weight Path
Route Distinguisher: 172.16.1.2:2
*> [2]:[0]:[0]:[48]:[7a:a1:d2:fa:86:9b]
                    172.16.1.1                             0 65001 i
*> [3]:[0]:[32]:[172.16.1.1]
                    172.16.1.1                             0 65001 i
Route Distinguisher: 172.16.1.2:3
*> [2]:[0]:[0]:[48]:[a2:af:dc:0a:cd:6c]
                    172.16.1.1                             0 65001 i
*> [3]:[0]:[32]:[172.16.1.1]
                    172.16.1.1                             0 65001 i
Route Distinguisher: 172.16.1.3:2
*> [2]:[0]:[0]:[48]:[7a:a1:d2:fa:86:9b]
                    172.16.1.1                             0 65001 i
*> [3]:[0]:[32]:[172.16.1.1]
                    172.16.1.1                             0 65001 i
Route Distinguisher: 172.16.1.3:3
*> [2]:[0]:[0]:[48]:[a2:af:dc:0a:cd:6c]
                    172.16.1.1                             0 65001 i
*> [3]:[0]:[32]:[172.16.1.1]
                    172.16.1.1                             0 65001 i
Route Distinguisher: 172.16.1.5:2
*> [2]:[0]:[0]:[48]:[12:ed:20:2e:b3:1a]
                    172.16.1.4                             0 65002 i
*> [3]:[0]:[32]:[172.16.1.4]
                    172.16.1.4                             0 65002 i
Route Distinguisher: 172.16.1.5:3
*> [2]:[0]:[0]:[48]:[4e:4f:c1:03:32:f3]
                    172.16.1.4                             0 65002 i
*> [3]:[0]:[32]:[172.16.1.4]
                    172.16.1.4                             0 65002 i
Route Distinguisher: 172.16.1.6:2
*> [2]:[0]:[0]:[48]:[12:ed:20:2e:b3:1a]
                    172.16.1.4                             0 65002 i
*> [3]:[0]:[32]:[172.16.1.4]
                    172.16.1.4                             0 65002 i
Route Distinguisher: 172.16.1.6:3
*> [2]:[0]:[0]:[48]:[4e:4f:c1:03:32:f3]
                    172.16.1.4                             0 65002 i
*> [3]:[0]:[32]:[172.16.1.4]
                    172.16.1.4                             0 65002 i

Displayed 16 prefixes (16 paths)
root@BB01:~#

BB02

root@BB02:~# net show bgp evpn route
BGP table version is 2, local router ID is 172.16.222.2
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete
EVPN type-2 prefix: [2]:[ESI]:[EthTag]:[MAClen]:[MAC]:[IPlen]:[IP]
EVPN type-3 prefix: [3]:[EthTag]:[IPlen]:[OrigIP]
EVPN type-5 prefix: [5]:[ESI]:[EthTag]:[IPlen]:[IP]

   Network          Next Hop            Metric LocPrf Weight Path
Route Distinguisher: 172.16.1.2:2
*> [2]:[0]:[0]:[48]:[7a:a1:d2:fa:86:9b]
                    172.16.1.1                             0 65001 i
*> [3]:[0]:[32]:[172.16.1.1]
                    172.16.1.1                             0 65001 i
Route Distinguisher: 172.16.1.2:3
*> [2]:[0]:[0]:[48]:[a2:af:dc:0a:cd:6c]
                    172.16.1.1                             0 65001 i
*> [3]:[0]:[32]:[172.16.1.1]
                    172.16.1.1                             0 65001 i
Route Distinguisher: 172.16.1.3:2
*> [2]:[0]:[0]:[48]:[7a:a1:d2:fa:86:9b]
                    172.16.1.1                             0 65001 i
*> [3]:[0]:[32]:[172.16.1.1]
                    172.16.1.1                             0 65001 i
Route Distinguisher: 172.16.1.3:3
*> [2]:[0]:[0]:[48]:[a2:af:dc:0a:cd:6c]
                    172.16.1.1                             0 65001 i
*> [3]:[0]:[32]:[172.16.1.1]
                    172.16.1.1                             0 65001 i
Route Distinguisher: 172.16.1.5:2
*> [2]:[0]:[0]:[48]:[12:ed:20:2e:b3:1a]
                    172.16.1.4                             0 65002 i
*> [3]:[0]:[32]:[172.16.1.4]
                    172.16.1.4                             0 65002 i
Route Distinguisher: 172.16.1.5:3
*> [2]:[0]:[0]:[48]:[4e:4f:c1:03:32:f3]
                    172.16.1.4                             0 65002 i
*> [3]:[0]:[32]:[172.16.1.4]
                    172.16.1.4                             0 65002 i
Route Distinguisher: 172.16.1.6:2
*> [2]:[0]:[0]:[48]:[12:ed:20:2e:b3:1a]
                    172.16.1.4                             0 65002 i
*> [3]:[0]:[32]:[172.16.1.4]
                    172.16.1.4                             0 65002 i
Route Distinguisher: 172.16.1.6:3
*> [2]:[0]:[0]:[48]:[4e:4f:c1:03:32:f3]
                    172.16.1.4                             0 65002 i
*> [3]:[0]:[32]:[172.16.1.4]
                    172.16.1.4                             0 65002 i

Displayed 16 prefixes (16 paths)
root@BB02:~#

ControlPlane

route type-2

route type-3

DataPlane

arp request
vxlanヘッダを見るとsourceアドレスがspine01/02のVTEPアドレス(172.16.1.1)になり、
destinationアドレスがspine03/04のVTEPアドレス(172.16.1.4)となることから、
mlagを組んでいる2台で論理VTEPを使って通信してることがわかります
VTEPがBUM(Broadcast,Unknown unicast,Multicast packet)パケットを受信すると、
1.decapしローカルに接続されているホストに配信
2.再度encapしvxlanの他のVTEPにパケットを送信
いわゆるHead End Replicationの動作ですね

arp reply

ping request

ping reply

参考資料

https://docs.cumulusnetworks.com/display/DOCS/Ethernet+Virtual+Private+Network+-+EVPN
https://docs.cumulusnetworks.com/display/DOCS/Anycast+Design+Guide
https://community.mellanox.com/s/article/howto-configure-vxlan-with-mlag-using-cumulus-linux
http://kakkotetsu.hatenablog.com/entry/2017/05/11/001708#spine%E5%85%A8%E5%8F%B0%E3%81%ABVXLAN-VNI%E8%A8%AD%E5%AE%9A
https://tools.ietf.org/html/rfc7432