LoginSignup
3
2

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?

@dan-go(dan-go)

【Cisco】EVPN-VXLANってなに？

Last updated at Posted at 2024-09-18

VXLAN?

image.png
https://www.alaxala.com/jp/solution/admin/vxlan/

L3ネットワーク上でL2ネットワーク延伸を実現する “オーバーレイネットワーク” 技術。
データフレームにVXLANヘッダを付けた上でUDPにてカプセル化し、
L3ネットワーク(アンダーレイネットワーク)上をルーティングされながら転送され、L3ネットワークを越えてL2で宛先に到達できる。
これにより物理ネットワークと分離し、スケーラブルに仮想的なネットワーク(オーバーレイネットワーク)を構築できる

VTEP = VXLAN Tunnel End Point。VXLANのカプセル化/カプセル化解除を行うデバイス
NVE = Network Virtualization Edge。上記VTEPに対する設定上の呼び名
VNI = VXLANの識別子。VXLAN ID

EVPN?

image.png
https://techblog.yahoo.co.jp/infrastructure/evpn/

・VXLANにおけるVTEP検出やVNI・ホストMACアドレスの情報を交換するための手法は、MulticastルーティングによるFlood&Learnで学習させる方法もあるが、余計なフラッディングが発生することを防ぐため、コントロールプレーンとしてEVPNの技術を用いてBGP ルーティングプロトコルを活用し情報を交換する方法を取る。

・MP-BGP EVPNのIngress Replication機能(各VTEPの宛先IPに対してUnicastで送信する)によってBUM(Broadcast、Unknown Unicast、Multicast)トラフィックを転送できる。
※Multicastルーティングは使用しない。

VTEPピアの学習方法

1. 各VTEPは、自分の持っているVNIの情報をBGPでアドバタイズする。
2. BGP RRは各VTEPのVNI情報を集約し、他のVTEPにアドバタイズする
3. 各VTEPは他のVTEPのVNI情報を登録する

ホストアドレスの学習方法

1. ホストAがVTEP1に追加される。
2. VTEP1がBGP のRRにホストAの情報をアドバタイズ
3. RRから各VTEPにホストAの情報をアドバタイズ

ホストAが別のVTEP配下に移動すると、移動先のVTEPが検知してホストAの情報をアップデートしBGPのRRにア
ドバタイズすることで、各VTEPでホストAの情報が書き換えられる

BUMトラフィックの動き

1. VTEP1配下のホストAからVTEP2配下のホストBにARPリクエストを送る
2. VTEP1にはホストBの情報がないので、IRのピアのVTEPアドレスを宛先IPにしてARPリクエストを送る
3. ホストBからVTEP2にARPリクエストが返ってくる
4. VTEP2のmacテーブルにホストBの情報が登録され、ARPリプライはVTEP1にユニキャストで送信される
5. EVPN経由でRRより他のVTEPにホストBの情報がアドバタイズされる。

構成概要

image.png

1.	VXLANによってL3アンダーレイ上のL2オーバーレイNWを構成する。VXLAN Routingは使用せず、L2のVXLANとなる。
2.	LeafSWがVXLAN VTEPとして動作し、サービス間通信はLeafSW間L2オーバーレイNWで行う。
3.	VTEP間のVXLAN通信はL3アンダーレイNWのルーティングプロトコルによって伝送される。
4.	L3アンダーレイNWはNexus9000のLeaf-Spineトポロジによって構成される。ルーティングプロトコルはCost変更によるトラフィック制御を行いやすいOSPFを使用。OSPFでは各Leaf SWのLoopback IPの広報を行う。
5.	VXLANのコントロールプレーンとしてMP-BGP EVPNの設定を行う。OSPFにて広報された loopbackアドレスを用いて各Leaf間でiBGPのneighborを構築し、EVPNにてVXLAN通信で必要な情報を交換する。
6.	LeafSWはVTEPピア・VNI・ホストMACアドレスの情報をMP-BGP EVPNによって学習する。iBGPを使用し、SpineSWがRoute Reflector(RR)となる。
7.	vPCを使用するLeafSWのNVE動作に用いるIPアドレスはLoopback1のsecondaryアドレスとなる。vPCを組む2台ではLoopback1のsecondaryアドレスは同一アドレスを使用。

要は...
・オーバーレイ：Leaf間でVTEPを形成し、データ通信を行う

・アンダーレイ：オーバーレイを通るデータ通信のルーティングを行う

・データ通信とは別のコントロールプレーンをMP-BGP EVPNで管理。
Spine機がRRとなり、iBGPで各LeafにVXLAN通信に必要な情報を伝搬

設定例

image.png
CMLで使用できるNXOS9000(version 9.3(8))で確認
Spine上は外部接続用のBorderLeafを用意するのが普通。設定はサーバ向けと基本は同じとなる。

HOST情報

host01 VLAN1000 10.0.0.1/24
host02 VLAN1001 10.10.10.1/24
host03 VLAN1000 10.0.0.2/24
host04 VLAN1001 10.10.10.2/24

Leaf01-02/03-04でvPCを構成。(configは省略)

config

Leaf/Spine共通 
vdc leaf01 id 1    ###VDC設定(デフォルト)
  limit-resource vlan minimum 16 maximum 4094
  limit-resource vrf minimum 2 maximum 4096
  limit-resource port-channel minimum 0 maximum 511
  limit-resource u4route-mem minimum 248 maximum 248
  limit-resource u6route-mem minimum 96 maximum 96
  limit-resource m4route-mem minimum 58 maximum 58
  limit-resource m6route-mem minimum 8 maximum 8

cfs eth distribute                ###CFS over IP機能を使用する(vPC有効化に必要)※Leafのみ
nv overlay evpn                   ###オーバーレイトラフィックをEVPNで制御
feature ospf                      ###アンダーレイNWの構成で使用
feature bgp                       ###EVPNを使用してMACアドレスを伝搬
feature interface-vlan            ###渡りリンク(SVI999)構成のため ※Leafのみ
feature vn-segment-vlan-based     ###VLANとvn-segment紐づけのため ※Leafのみ
feature lacp                      ###ポートチャネル
feature vpc                       ###vPC有効化 ※Leafのみ
feature bfd                       ###双方向フォワーディング検出.1秒以内で障害検知
clock timezone JST 9 0
feature nv overlay                ###VXLAN有効

no password strength-check
username admin password 5 $5$JPKPOH$bM4LoW0uRysvhnRiLDuwD4NcyXtBJBjzG.XXXXX  role network-admin
username cisco password 5 $5$NMCOEL$Cwih0qf0KZVI6okBPdzWi9mq0xDbKG/S7wPkg/XXXXX  role network-admin
username cisco passphrase  lifetime 99999 warntime 14 gracetime 3
no ip domain-lookup               ###名前解決無効化
spanning-tree mode mst            ###Serverを収容するLeafはmst(Leafのみ)
no spanning-tree vlan 1-3967      ###STP無効化(Spineのみ)
copp profile strict               ###COPP設定(デフォルト)
bfd startup-timer 0               ###BFD(障害検知)起動タイマーを無効⇒即時開始
snmp-server user admin network-admin auth md5 0044F75D1791B522A0CA613EB8EC104B7CBC priv 00718A2D77F... localizedV2key
snmp-server user cisco network-admin auth md5 166EBA016594D8418DD1532CF3BB480C46CE priv 165ED1670AC... localizedV2key
rmon event 1 log trap public description FATAL(1) owner PMON@FATAL
rmon event 2 log trap public description CRITICAL(2) owner PMON@CRITICAL
rmon event 3 log trap public description ERROR(3) owner PMON@ERROR
rmon event 4 log trap public description WARNING(4) owner PMON@WARNING
rmon event 5 log trap public description INFORMATION(5) owner PMON@INFO

fabric forwarding anycast-gateway-mac aabb.ccdd.eeff      ###Leafのみ
Leaf01 
vlan 1,999-1001
vlan 999
  name watari
vlan 1000
  name server-segment1
  vn-segment 1000                                   ###VLAN1000をVXLANで透過する
vlan 1001
  name server-segment2
  vn-segment 1001

interface Vlan1
  no ip redirects
  no ipv6 redirects

interface Vlan999
  description Leaf-Leaf_Connect
  no shutdown
  mtu 9216
  no ip redirects                                   ###ip redirectは使用しない
  ip address 192.168.10.9/30
  no ipv6 redirects                                 ###ipv6 redirectは使用しない
  ip ospf network point-to-point                    ###渡りリンクを広報するためOSPFを有効にする
  ip router ospf 1 area 0.0.0.0

interface port-channel1                             ###host01向け
  switchport mode trunk
  storm-control broadcast level 0.10
  storm-control action trap
  switchport isolated
  no lacp graceful-convergence                      ###Nexus機器以外とLACP組む場合に設定する
  vpc 1

interface port-channel2                             ###host02向け
  switchport mode trunk
  storm-control broadcast level 0.10
  storm-control action trap
  switchport isolated
  no lacp graceful-convergence
  vpc 2

interface port-channel50
  switchport mode trunk
  spanning-tree port type network                    ###vpc peer-linkにより自動で設定される
  switchport isolated
  vpc peer-link

interface port-channel110
  description spine01_Po110
  no switchport
  mtu 9216
  bfd interval 300 min_rx 300 multiplier 3
  no ip redirects                                    ###ipv4 redirectしない
  ip address 192.168.1.66/30
  ip ospf network point-to-point
  ip router ospf 1 area 0.0.0.0                      ###アンダーレイNW構成。OSPFで広報する

interface port-channel210
  description spine02_Po210
  no switchport
  mtu 9216
  bfd interval 300 min_rx 300 multiplier 3
  no ip redirects
  ip address 192.168.2.66/30
  ip ospf network point-to-point
  ip router ospf 1 area 0.0.0.0

interface port-channel310
  description spine03_Po310
  no switchport
  mtu 9216
  bfd interval 300 min_rx 300 multiplier 3
  no ip redirects
  ip address 192.168.3.66/30
  ip ospf network point-to-point
  ip router ospf 1 area 0.0.0.0

interface nve1                                       ###nve(vxlan)インタフェース有効化
  no shutdown
  host-reachability protocol bgp                     ###コントロールプレーンで使用するプロトコル。ホスト到達可能性のアドバタイズメント機構としてBGPを指定
  source-interface loopback1                         ###VXLANを使用するためのIF。secondary addressを設定したloopbackインターフェースを指定する
  source-interface hold-down-time 240
  member vni 1000                                    ###interface nve 1のこれ以降の設定でVLANとVNIの紐付けを行う。VLAN1000をVXLANで透過する
    ingress-replication protocol bgp
  member vni 1001
    ingress-replication protocol bgp

interface Ethernet1/1
  description host01
  lacp rate fast
  no cdp enable                                      ###物理IFのみに入る
  switchport mode trunk
  storm-control broadcast level 0.10                 ###ブロードキャストストームが0.1%(10Gなら10M)受け取ったら3.9秒間ドロップする。流れ続けている場合、3.9秒ごとに判定
  storm-control action trap                          ###ブロードキャストがしきい値を超過した場合のフィルタリング、及び超過時のSNMP Trap送信を有効
  switchport isolated
  channel-group 1 mode active

interface Ethernet1/2
  description host02
  no cdp enable
  switchport mode trunk
  storm-control broadcast level 0.10
  storm-control action trap
  switchport isolated
  channel-group 2 mode active

interface Ethernet1/10
  description Spine01_Eth1/1
  no switchport
  mtu 9216
  channel-group 110 mode active
  no shutdown

interface Ethernet1/11
  description Spine02_Eth1/1
  no switchport

  mtu 9216
  channel-group 210 mode active
  no shutdown

interface Ethernet1/12
  description Spine03_Eth1/1
  no switchport
  mtu 9216
  channel-group 310 mode active
  no shutdown

interface Ethernet1/15
  description Leaf02_Eth1/15
  switchport mode trunk
  switchport isolated
  channel-group 50 mode active

interface Ethernet1/16
  description Leaf02_Eth1/16
  switchport mode trunk
  switchport isolated
  channel-group 50 mode active

interface mgmt0
  vrf member management
  ip address 172.21.227.1/24

interface loopback0
  ip address 192.168.0.27/32
  ip router ospf 1 area 0.0.0.0                         ###OSPFでループバックインターフェースを広報するため指定

interface loopback1
  ip address 192.168.0.87/32
  ip address 192.168.0.139/32 secondary                 ###interfac nveが使用するVTEPのアドレスのためセカンダリアドレスの設定が必要。vPCピアで同じ値を指定
  ip router ospf 1 area 0.0.0.0                         ###OSPFでループバックインターフェースを広報するため指定
icam monitor scale

line console
line vty
boot nxos bootflash:/nxos.9.3.8.bin sup-1

router ospf 1                                            ###各IFで有効化にしているため、networkコマンドはなし
  bfd                                                    ###1秒以内に断検知するための各IFの設定と合わせて必要
  router-id 192.168.0.27                                 ###loopback0と同じ
  log-adjacency-changes                                  ###ステータス変化時にログを出す
router bgp 65500
  router-id 192.168.0.27                                 ###loopback0と同じ
  timers bgp 20 60
  log-neighbor-changes                                   ###ネイバー状態変化時にログを出力するため必要
  address-family ipv4 unicast                            ###router bgpセクションにあるaddress-familyは、そのルーターがどのようなアドレスファミリーをサポートしているのかを指定する。この場合、IPv4 unicastとl2vpn evpnの両方をサポートを指示。
  address-family l2vpn evpn
  neighbor 192.168.0.1                                   ###Spine01のrouter-id
    remote-as 65500                                      ###iBGPを使用するためlocal-asと同じものを指定
    update-source loopback0                              ###ループバックインターフェースを使用するため送信元を指定
    address-family ipv4 unicast                          ###neighborセクションにあるaddress-familyは、特定のBGPネイバー（つまり、ピアリングする他のBGPルーター）との間でどのアドレスファミリーを使用してルート情報を交換するのかを指定する。これにより、特定のネイバーとの間でのみ特定のアドレスファミリーを使用。
    address-family l2vpn evpn
      send-community                                     ###address-family l2vpn evpnで基本コミュニティを使用する
      send-community extended                            ###address-family l2vpn evpnで拡張コミュニティを使用する
  neighbor 192.168.0.2                                   ###Spine02のrouter-id
    remote-as 65500
    update-source loopback0
    address-family ipv4 unicast
    address-family l2vpn evpn
      send-community
      send-community extended
  neighbor 192.168.0.3                                   ###Spine03のrouter-id
    remote-as 65500
    update-source loopback0
    address-family ipv4 unicast
    address-family l2vpn evpn
      send-community
      send-community extended
evpn                                                     ###EVPN設定
  vni 1000 l2                                            ###VNI設定
    rd auto
    route-target import auto
    route-target export auto
  vni 1001 l2
    rd auto
    route-target import auto
    route-target export auto
ip tcp path-mtu-discovery                                ###経路上の最低MTU確認のため
Leaf02 
vlan 1,999-1001
vlan 999
  name watari
vlan 1000
  name server-segment1
  vn-segment 1000
vlan 1001
  name server-segment2
  vn-segment 1001

interface Vlan1
  no ip redirects
  no ipv6 redirects

interface Vlan999
  description Leaf-Leaf_Connect
  no shutdown
  mtu 9216
  no ip redirects
  ip address 192.168.10.10/30
  no ipv6 redirects
  ip ospf network point-to-point
  ip router ospf 1 area 0.0.0.0

interface port-channel1
  switchport mode trunk
  storm-control broadcast level 0.10
  storm-control action trap
  switchport isolated
  no lacp graceful-convergence
  vpc 1

interface port-channel2
  switchport mode trunk
  storm-control broadcast level 0.10
  storm-control action trap
  switchport isolated
  no lacp graceful-convergence
  vpc 2

interface port-channel50
  switchport mode trunk
  spanning-tree port type network
  switchport isolated
  vpc peer-link

interface port-channel111
  description spine01_Po111
  no switchport
  mtu 9216
  bfd interval 300 min_rx 300 multiplier 3
  no ip redirects
  ip address 192.168.1.70/30
  ip ospf network point-to-point
  ip router ospf 1 area 0.0.0.0

interface port-channel211
  description spine02_Po211
  no switchport
  mtu 9216
  bfd interval 300 min_rx 300 multiplier 3
  no ip redirects
  ip address 192.168.2.70/30
  ip ospf network point-to-point
  ip router ospf 1 area 0.0.0.0

interface port-channel311
  description spine03_Po311
  no switchport
  mtu 9216
  bfd interval 300 min_rx 300 multiplier 3
  no ip redirects
  ip address 192.168.3.70/30
  ip ospf network point-to-point
  ip router ospf 1 area 0.0.0.0

interface nve1
  no shutdown
  host-reachability protocol bgp
  source-interface loopback1
  source-interface hold-down-time 240
  member vni 1000
    ingress-replication protocol bgp
  member vni 1001
    ingress-replication protocol bgp

interface Ethernet1/1
  description host01
  lacp rate fast
  no cdp enable
  switchport mode trunk
  storm-control broadcast level 0.10
  storm-control action trap
  switchport isolated
  channel-group 1 mode active

interface Ethernet1/2
  description host02
  lacp rate fast
  no cdp enable
  switchport mode trunk
  storm-control broadcast level 0.10
  storm-control action trap
  switchport isolated
  channel-group 2 mode active

interface Ethernet1/10
  description Spine01_Eth1/2
  no switchport
  mtu 9216
  channel-group 111 mode active
  no shutdown

interface Ethernet1/11
  description Spine02_Eth1/2
  no switchport
  mtu 9216
  channel-group 211 mode active
  no shutdown

interface Ethernet1/12
  description Spine03_Eth1/2
  no switchport
  mtu 9216
  channel-group 311 mode active
  no shutdown

interface Ethernet1/15
  description Leaf01_Eth1/15
  switchport mode trunk
  switchport isolated
  channel-group 50 mode active

interface Ethernet1/16
  description Leaf01_Eth1/16
  switchport mode trunk
  switchport isolated
  channel-group 50 mode active

interface mgmt0
  vrf member management
  ip address 172.21.227.2/24

interface loopback0
  ip address 192.168.0.28/32
  ip router ospf 1 area 0.0.0.0

interface loopback1
  ip address 192.168.0.88/32
  ip address 192.168.0.139/32 secondary
  ip router ospf 1 area 0.0.0.0
icam monitor scale

line console
line vty
boot nxos bootflash:/nxos.9.3.8.bin sup-1

router ospf 1
  bfd
  router-id 192.168.0.28
  log-adjacency-changes
router bgp 65500
  router-id 192.168.0.28
  timers bgp 20 60
  log-neighbor-changes
  address-family ipv4 unicast
  address-family l2vpn evpn
  neighbor 192.168.0.1
    remote-as 65500
    update-source loopback0
    address-family ipv4 unicast
    address-family l2vpn evpn
      send-community
      send-community extended
  neighbor 192.168.0.2
    remote-as 65500
    update-source loopback0
    address-family ipv4 unicast
    address-family l2vpn evpn
      send-community
      send-community extended
  neighbor 192.168.0.3
    remote-as 65500
    update-source loopback0
    address-family ipv4 unicast
    address-family l2vpn evpn
      send-community
      send-community extended
evpn
  vni 1000 l2
    rd auto
    route-target import auto
    route-target export auto
  vni 1001 l2
    rd auto
    route-target import auto
    route-target export auto
Leaf03 
vlan 1,999-1001
vlan 999
  name watari
vlan 1000
  name server-segment1
  vn-segment 1000
vlan 1001
  name server-segment2
  vn-segment 1001

interface Vlan1
  no ip redirects
  no ipv6 redirects

interface Vlan999
  description Leaf-Leaf_Connect
  no shutdown
  mtu 9216
  no ip redirects
  ip address 192.168.10.13/30
  no ipv6 redirects
  ip ospf network point-to-point
  ip router ospf 1 area 0.0.0.0

interface port-channel1
  switchport mode trunk
  storm-control broadcast level 0.10
  storm-control action trap
  switchport isolated
  vpc 1

interface port-channel2
  switchport mode trunk
  storm-control broadcast level 0.10
  storm-control action trap
  switchport isolated
  vpc 2

interface port-channel50
  switchport mode trunk
  spanning-tree port type network
  switchport isolated
  vpc peer-link

interface port-channel112
  no switchport
  mtu 9216
  bfd interval 300 min_rx 300 multiplier 3
  no ip redirects
  ip address 192.168.1.74/30
  ip ospf network point-to-point
  ip router ospf 1 area 0.0.0.0

interface port-channel212
  no switchport
  mtu 9216
  bfd interval 300 min_rx 300 multiplier 3
  no ip redirects
  ip address 192.168.2.74/30
  ip ospf network point-to-point
  ip router ospf 1 area 0.0.0.0

interface port-channel312
  no switchport
  mtu 9216
  bfd interval 300 min_rx 300 multiplier 3
  no ip redirects
  ip address 192.168.3.74/30
  ip ospf network point-to-point
  ip router ospf 1 area 0.0.0.0

interface nve1
  no shutdown
  host-reachability protocol bgp
  source-interface loopback1
  source-interface hold-down-time 240
  member vni 1000
    ingress-replication protocol bgp
  member vni 1001
    ingress-replication protocol bgp

interface Ethernet1/1
  description host03
  lacp rate fast
  no cdp enable
  switchport mode trunk
  storm-control broadcast level 0.10
  storm-control action trap
  switchport isolated
  channel-group 1 mode active

interface Ethernet1/2
  description host04
  lacp rate fast
  no cdp enable
  switchport mode trunk
  storm-control broadcast level 0.10
  storm-control action trap
  switchport isolated
  channel-group 2 mode active

interface Ethernet1/10
  description spine01_Eth1/3
  mtu 9216
  channel-group 112 mode active

interface Ethernet1/11
  description spine02_Eth1/3
  mtu 9216
  channel-group 212 mode active

interface Ethernet1/12
  description spine03_Eth1/3
  mtu 9216
  channel-group 312 mode active

interface Ethernet1/15
  description Leaf04_Eth1/15
  switchport mode trunk
  switchport isolated
  channel-group 50 mode active

interface Ethernet1/16
  description Leaf04_Eth1/16
  switchport mode trunk
  switchport isolated
  channel-group 50 mode active

interface mgmt0
  vrf member management
  ip address 172.21.227.3/24

interface loopback0
  ip address 192.168.0.29/32
  ip router ospf 1 area 0.0.0.0

interface loopback1
  ip address 192.168.0.89/32
  ip address 192.168.0.140/32 secondary
  ip router ospf 1 area 0.0.0.0
icam monitor scale

line console
line vty
boot nxos bootflash:/nxos.9.3.8.bin sup-1

router ospf 1
  bfd
  router-id 192.168.0.29
  log-adjacency-changes
router bgp 65500
  router-id 192.168.0.29
  timers bgp 20 60
  log-neighbor-changes
  address-family ipv4 unicast
  address-family l2vpn evpn
  neighbor 192.168.0.1
    remote-as 65500
    update-source loopback0
    address-family ipv4 unicast
    address-family l2vpn evpn
      send-community
      send-community extended
  neighbor 192.168.0.2
    remote-as 65500
    update-source loopback0
    address-family ipv4 unicast
    address-family l2vpn evpn
      send-community
      send-community extended
  neighbor 192.168.0.3
    remote-as 65500
    update-source loopback0
    address-family ipv4 unicast
    address-family l2vpn evpn
      send-community
      send-community extended
evpn
  vni 1000 l2
    rd auto
    route-target import auto
    route-target export auto
  vni 1001 l2
    rd auto
    route-target import auto
    route-target export aut
Leaf04 
vlan 1,999-1001
vlan 999
  name watari
vlan 1000
  name server-segment1
  vn-segment 1000
vlan 1001
  name server-segment2
  vn-segment 1001

interface Vlan1
  no ip redirects
  no ipv6 redirects

interface Vlan999
  description Leaf-Leaf_Connect
  no shutdown
  mtu 9216
  no ip redirects
  ip address 192.168.10.14/30
  no ipv6 redirects
  ip ospf network point-to-point
  ip router ospf 1 area 0.0.0.0

interface port-channel1
  switchport mode trunk
  storm-control broadcast level 0.10
  storm-control action trap
  switchport isolated
  vpc 1

interface port-channel2
  switchport mode trunk
  storm-control broadcast level 0.10
  storm-control action trap
  switchport isolated
  vpc 2

interface port-channel50
  switchport mode trunk
  spanning-tree port type network
  switchport isolated
  vpc peer-link

interface port-channel113
  description spine01_Po113
  no switchport
  mtu 9216
  bfd interval 300 min_rx 300 multiplier 3
  no ip redirects
  ip address 192.168.1.78/30
  ip ospf network point-to-point
  ip router ospf 1 area 0.0.0.0

interface port-channel213
  description spine02_Po213
  no switchport
  mtu 9216
  bfd interval 300 min_rx 300 multiplier 3
  no ip redirects
  ip address 192.168.2.78/30
  ip ospf network point-to-point
  ip router ospf 1 area 0.0.0.0

interface port-channel313
  description spine03_Po313
  no switchport
  mtu 9216
  bfd interval 300 min_rx 300 multiplier 3
  no ip redirects
  ip address 192.168.3.78/30
  ip ospf network point-to-point
  ip router ospf 1 area 0.0.0.0

interface nve1
  no shutdown
  host-reachability protocol bgp
  source-interface loopback1
  source-interface hold-down-time 240
  member vni 1000
    ingress-replication protocol bgp
  member vni 1001
    ingress-replication protocol bgp

interface Ethernet1/1
  description host03
  lacp rate fast
  no cdp enable
  switchport mode trunk
  storm-control broadcast level 0.10
  storm-control action trap
  switchport isolated
  channel-group 1 mode active

interface Ethernet1/2
  description host04
  lacp rate fast
  no cdp enable
  switchport mode trunk
  storm-control broadcast level 0.10
  storm-control action trap
  switchport isolated
  channel-group 2 mode active

interface Ethernet1/10
  description spine01_Eth1/4
  no switchport
  mtu 9216
  channel-group 113 mode active
  no shutdown

interface Ethernet1/11
  description spine02_Eth1/4
  no switchport
  mtu 9216
  channel-group 213 mode active
  no shutdown

interface Ethernet1/12
  description spine03_Eth1/4
  no switchport
  mtu 9216
  channel-group 313 mode active
  no shutdown

interface Ethernet1/15
  description Leaf03_Eth1/15
  switchport mode trunk
  switchport isolated
  channel-group 50 mode active

interface Ethernet1/16
  description Leaf03_Eth1/16
  switchport mode trunk
  switchport isolated
  channel-group 50 mode active

interface mgmt0
  vrf member management
  ip address 172.21.227.4/24

interface loopback0
  ip address 192.168.0.30/32
  ip router ospf 1 area 0.0.0.0

interface loopback1
  ip address 192.168.0.90/32
  ip address 192.168.0.140/32 secondary
  ip router ospf 1 area 0.0.0.0
icam monitor scale

line console
line vty
boot nxos bootflash:/nxos.9.3.8.bin sup-1

router ospf 1
  bfd
  router-id 192.168.0.30
  log-adjacency-changes
router bgp 65500
  router-id 192.168.0.30
  timers bgp 20 60
  log-neighbor-changes
  address-family ipv4 unicast
  address-family l2vpn evpn
  neighbor 192.168.0.1
    remote-as 65500
    update-source loopback0
    address-family ipv4 unicast
    address-family l2vpn evpn
      send-community
      send-community extended
  neighbor 192.168.0.2
    remote-as 65500
    update-source loopback0
    address-family ipv4 unicast
    address-family l2vpn evpn
      send-community
      send-community extended
  neighbor 192.168.0.3
    remote-as 65500
    update-source loopback0
    address-family ipv4 unicast
    address-family l2vpn evpn
      send-community
      send-community extended
evpn
  vni 1000 l2
    rd auto
    route-target import auto
    route-target export auto
  vni 1001 l2
    rd auto
    route-target import auto
    route-target export auto
Spine01 
interface port-channel110
  description Leaf01_Po110
  no switchport
  mtu 9216
  bfd interval 300 min_rx 300 multiplier 3
  no ip redirects                                          ###ip redirectは使用しない
  ip address 192.168.1.65/30
  ip ospf network point-to-point
  ip router ospf 1 area 0.0.0.0                      ###アンダーレイNW構成。OSPFで広報する

interface port-channel111
  description Leaf02_Po111
  no switchport
  mtu 9216
  bfd interval 300 min_rx 300 multiplier 3
  no ip redirects
  ip address 192.168.1.69/30
  ip ospf network point-to-point
  ip router ospf 1 area 0.0.0.0

interface port-channel112
  description Leaf03_Po112
  no switchport
  mtu 9216
  bfd interval 300 min_rx 300 multiplier 3
  no ip redirects
  ip address 192.168.1.73/30
  ip ospf network point-to-point
  ip router ospf 1 area 0.0.0.0

interface port-channel113
  description Leaf04_Po113
  no switchport
  mtu 9216
  bfd interval 300 min_rx 300 multiplier 3
  no ip redirects
  ip address 192.168.1.77/30
  ip ospf network point-to-point
  ip router ospf 1 area 0.0.0.0

interface Ethernet1/1
  description Leaf01_Eth1/10
  no switchport
  mtu 9216
  channel-group 110 mode active
  no shutdown

interface Ethernet1/2
  description Leaf02_Eth1/10
  no switchport
  mtu 9216
  channel-group 111 mode active
  no shutdown

interface Ethernet1/3
  description Leaf03_Eth1/10
  no switchport
  mtu 9216
  channel-group 112 mode active
  no shutdown

interface Ethernet1/4
  description Leaf04_Eth1/10
  no switchport
  mtu 9216
  channel-group 113 mode active
  no shutdown

interface mgmt0
  vrf member management

interface loopback0
  ip address 192.168.0.1/32
  ip router ospf 1 area 0.0.0.0                         ###OSPFでループバックインターフェースを広報するため指定
icam monitor scale

line console
line vty
boot nxos bootflash:/nxos.9.3.8.bin sup-1

router ospf 1                                                 ###各IFで有効化にしているため、networkコマンドはなし
  bfd                                                             ###1秒以内に断検知するための各IFの設定と合わせて必要
  router-id 192.168.0.1                                  ###loopback0と同じ
  log-adjacency-changes                                ###ステータス変化時にログを出す
router bgp 65500
  router-id 192.168.0.1                                  ###loopback0と同じ
  cluster-id 192.168.0.1
  timers bgp 20 60
  log-neighbor-changes                                 ###ネイバー状態変化時にログを出力するため必要
  address-family ipv4 unicast                         ###ipv4を使用するためenable
  address-family l2vpn evpn                          ###l2vpn evpnを使用する
  neighbor 192.168.0.27                               ###Leaf01のrouter-id
    remote-as 65500                                     ###iBGPを使用するためlocal-asと同じものを指定
    update-source loopback0                          ###ループバックインターフェースを使用するため送信元を指定
    address-family ipv4 unicast
      route-reflector-client                              ###IBGPピアから受信するルート情報を他のIBGPピアに反射させてアドバタイズする
    address-family l2vpn evpn
      send-community                                    ###address-family l2vpn evpnで基本コミュニティを使用する
      send-community extended                      ###address-family l2vpn evpnで拡張コミュニティを使用する
      route-reflector-client
  neighbor 192.168.0.28                               ###Leaf02のrouter-id
    remote-as 65500
    update-source loopback0
    address-family ipv4 unicast
      route-reflector-client
    address-family l2vpn evpn
      send-community
      send-community extended
      route-reflector-client
  neighbor 192.168.0.29                               ###Leaf03のrouter-id
    remote-as 65500
    update-source loopback0
    address-family ipv4 unicast
      route-reflector-client
    address-family l2vpn evpn
      send-community
      send-community extended
      route-reflector-client
  neighbor 192.168.0.30                               ###Leaf04のrouter-id
    remote-as 65500
    update-source loopback0
    address-family ipv4 unicast
      route-reflector-client
    address-family l2vpn evpn
      send-community
      send-community extended
      route-reflector-client
ip tcp path-mtu-discovery                             ###経路上の最低MTU確認のため
Spine02 
interface port-channel210
  description Leaf01_Po210
  no switchport
  mtu 9216
  bfd interval 300 min_rx 300 multiplier 3
  no ip redirects
  ip address 192.168.2.65/30
  ip ospf network point-to-point
  ip router ospf 1 area 0.0.0.0

interface port-channel211
  description Leaf02_Po211
  no switchport
  mtu 9216
  bfd interval 300 min_rx 300 multiplier 3
  no ip redirects
  ip address 192.168.2.69/30
  ip ospf network point-to-point
  ip router ospf 1 area 0.0.0.0

interface port-channel212
  description Leaf03_Po212
  no switchport
  mtu 9216
  bfd interval 300 min_rx 300 multiplier 3
  no ip redirects
  ip address 192.168.2.73/30
  ip ospf network point-to-point
  ip router ospf 1 area 0.0.0.0

interface port-channel213
  description Leaf04_Po213
  no switchport
  mtu 9216
  bfd interval 300 min_rx 300 multiplier 3
  no ip redirects
  ip address 192.168.2.77/30
  ip ospf network point-to-point
  ip router ospf 1 area 0.0.0.0

interface Ethernet1/1
  description Leaf01_Eth1/11
  no switchport
  mtu 9216
  channel-group 210 mode active
  no shutdown

interface Ethernet1/2
  description Leaf02_Eth1/11
  no switchport
  mtu 9216
  channel-group 211 mode active
  no shutdown

interface Ethernet1/3
  description Leaf03_Eth1/11
  no switchport
  mtu 9216
  channel-group 212 mode active
  no shutdown

interface Ethernet1/4
  description Leaf04_Eth1/11
  no switchport
  mtu 9216
  channel-group 213 mode active
  no shutdown

interface mgmt0
  vrf member management

interface loopback0
  ip address 192.168.0.2/32
  ip router ospf 1 area 0.0.0.0
icam monitor scale

line console
line vty
boot nxos bootflash:/nxos.9.3.8.bin sup-1

router ospf 1
  bfd
  router-id 192.168.0.2
  log-adjacency-changes
router bgp 65500
  router-id 192.168.0.2
  cluster-id 192.168.0.2
  timers bgp 20 60
  log-neighbor-changes
  address-family ipv4 unicast
  address-family l2vpn evpn
  neighbor 192.168.0.27
    remote-as 65500
    update-source loopback0
    address-family ipv4 unicast
      route-reflector-client
    address-family l2vpn evpn
      send-community
      send-community extended
      route-reflector-client
  neighbor 192.168.0.28
    remote-as 65500
    update-source loopback0
    address-family ipv4 unicast
      route-reflector-client
    address-family l2vpn evpn
      send-community
      send-community extended
      route-reflector-client
  neighbor 192.168.0.29
    remote-as 65500
    update-source loopback0
    address-family ipv4 unicast
      route-reflector-client
    address-family l2vpn evpn
      send-community
      send-community extended
      route-reflector-client
  neighbor 192.168.0.30
    remote-as 65500
    update-source loopback0
    address-family ipv4 unicast
      route-reflector-client
    address-family l2vpn evpn
      send-community
      send-community extended
      route-reflector-client
ip tcp path-mtu-discovery
Spine03 
interface port-channel310
  description Leaf01_Po310
  no switchport
  mtu 9216
  bfd interval 300 min_rx 300 multiplier 3
  no ip redirects
  ip address 192.168.3.65/30
  ip ospf network point-to-point
  ip router ospf 1 area 0.0.0.0

interface port-channel311
  description Leaf02_Po311
  no switchport
  mtu 9216
  bfd interval 300 min_rx 300 multiplier 3
  no ip redirects
  ip address 192.168.3.69/30
  ip ospf network point-to-point
  ip router ospf 1 area 0.0.0.0

interface port-channel312
  description Leaf03_Po312
  no switchport
  mtu 9216
  bfd interval 300 min_rx 300 multiplier 3
  no ip redirects
  ip address 192.168.3.73/30
  ip ospf network point-to-point
  ip router ospf 1 area 0.0.0.0

interface port-channel313
  description Leaf04_Po313
  no switchport
  mtu 9216
  bfd interval 300 min_rx 300 multiplier 3
  no ip redirects
  ip address 192.168.3.77/30
  ip ospf network point-to-point
  ip router ospf 1 area 0.0.0.0

interface Ethernet1/1
  description Leaf01_Eth1/12
  no switchport
  mtu 9216
  channel-group 310 mode active
  no shutdown

interface Ethernet1/2
  description Leaf02_Eth1/12
  no switchport
  mtu 9216
  channel-group 311 mode active
  no shutdown

interface Ethernet1/3
  description Leaf03_Eth1/12
  no switchport
  mtu 9216
  channel-group 312 mode active
  no shutdown

interface Ethernet1/4
  description Leaf04_Eth1/12
  no switchport
  mtu 9216
  channel-group 313 mode active
  no shutdown

interface mgmt0
  vrf member management

interface loopback0
  ip address 192.168.0.3/32
  ip router ospf 1 area 0.0.0.0
icam monitor scale

line console
line vty
boot nxos bootflash:/nxos.9.3.8.bin sup-1

router ospf 1
  bfd
  router-id 192.168.0.3
  log-adjacency-changes
router bgp 65500
  router-id 192.168.0.3
  cluster-id 192.168.0.3
  timers bgp 20 60
  log-neighbor-changes
  address-family ipv4 unicast
  address-family l2vpn evpn
  neighbor 192.168.0.27
    remote-as 65500
    update-source loopback0
    address-family ipv4 unicast
      route-reflector-client
    address-family l2vpn evpn
      send-community
      send-community extended
      route-reflector-client
  neighbor 192.168.0.28
    remote-as 65500
    update-source loopback0
    address-family ipv4 unicast
      route-reflector-client
    address-family l2vpn evpn
      send-community
      send-community extended
      route-reflector-client
  neighbor 192.168.0.29
    remote-as 65500
    update-source loopback0
    address-family ipv4 unicast
      route-reflector-client
    address-family l2vpn evpn
      send-community
      send-community extended
      route-reflector-client
  neighbor 192.168.0.30
    remote-as 65500
    update-source loopback0
    address-family ipv4 unicast
      route-reflector-client
    address-family l2vpn evpn
      send-community
      send-community extended
      route-reflector-client
ip tcp path-mtu-discovery

確認

ピア情報

Leaf

各Spine、Leaf02とOSPFネイバーが組めていることを確認。ネイバーIDはloopback0のアドレスと同じだが、ネイバー用としてはIFに付与されたアドレス。

leaf01#
leaf01# show ip ospf neighbors
 OSPF Process ID 1 VRF default
 Total number of neighbors: 4
 Neighbor ID     Pri State            Up Time  Address         Interface
 192.168.0.28      1 FULL/ -          1w0d     192.168.10.10   Vlan999
 192.168.0.1       1 FULL/ -          6d22h    192.168.1.65    Po110
 192.168.0.2       1 FULL/ -          6d22h    192.168.2.65    Po210
 192.168.0.3       1 FULL/ -          6d22h    192.168.3.65    Po310
leaf01#

各SpineとiBGPが組めていることを確認。Lo0アドレス同士で疎通している。(updatesourceをLo0で指定)

leaf01#
leaf01# show ip bgp summary
BGP summary information for VRF default, address family IPv4 Unicast
BGP router identifier 192.168.0.27, local AS number 65500
BGP table version is 5, IPv4 Unicast config peers 3, capable peers 3
0 network entries and 0 paths using 0 bytes of memory
BGP attribute entries [0/0], BGP AS path entries [0/0]
BGP community entries [0/0], BGP clusterlist entries [6/24]

Neighbor        V    AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
192.168.0.1     4 65500   30699   30691        5    0    0    6d22h 0
192.168.0.2     4 65500   30049   30039        5    0    0    6d22h 0
192.168.0.3     4 65500   30049   30039        5    0    0    6d22h 0
leaf01#

ルーティングテーブル確認。※わかりやすく区切ってコメントを入れている

leaf01#
leaf01# show ip route
IP Route Table for VRF "default"
'*' denotes best ucast next-hop
'**' denotes best mcast next-hop
'[x/y]' denotes [preference/metric]
'%<string>' in via output denotes VRF <string>

#各Spine Lo0 via IF(Po) address
192.168.0.1/32, ubest/mbest: 1/0
    *via 192.168.1.65, Po110, [110/41], 6d22h, ospf-1, intra
192.168.0.2/32, ubest/mbest: 1/0
    *via 192.168.2.65, Po210, [110/41], 6d22h, ospf-1, intra
192.168.0.3/32, ubest/mbest: 1/0
    *via 192.168.3.65, Po310, [110/41], 6d22h, ospf-1, intra

#自Lo0
192.168.0.27/32, ubest/mbest: 2/0, attached
    *via 192.168.0.27, Lo0, [0/0], 2w6d, local
    *via 192.168.0.27, Lo0, [0/0], 2w6d, direct
#Leaf02 Lo0
192.168.0.28/32, ubest/mbest: 1/0
    *via 192.168.10.10, Vlan999, [110/41], 1w0d, ospf-1, intra

#Leaf03/04 Lo0
192.168.0.29/32, ubest/mbest: 3/0
    *via 192.168.1.65, Po110, [110/121], 6d22h, ospf-1, intra
    *via 192.168.2.65, Po210, [110/121], 6d22h, ospf-1, intra
    *via 192.168.3.65, Po310, [110/121], 6d22h, ospf-1, intra
192.168.0.30/32, ubest/mbest: 3/0
    *via 192.168.1.65, Po110, [110/81], 6d22h, ospf-1, intra
    *via 192.168.2.65, Po210, [110/81], 6d22h, ospf-1, intra
    *via 192.168.3.65, Po310, [110/81], 6d22h, ospf-1, intra

#Leaf01-04 Lo1
192.168.0.87/32, ubest/mbest: 2/0, attached
    *via 192.168.0.87, Lo1, [0/0], 2w6d, local
    *via 192.168.0.87, Lo1, [0/0], 2w6d, direct
192.168.0.88/32, ubest/mbest: 1/0
    *via 192.168.10.10, Vlan999, [110/41], 1w0d, ospf-1, intra
192.168.0.89/32, ubest/mbest: 3/0
    *via 192.168.1.65, Po110, [110/121], 6d22h, ospf-1, intra
    *via 192.168.2.65, Po210, [110/121], 6d22h, ospf-1, intra
    *via 192.168.3.65, Po310, [110/121], 6d22h, ospf-1, intra
192.168.0.90/32, ubest/mbest: 3/0
    *via 192.168.1.65, Po110, [110/81], 6d22h, ospf-1, intra
    *via 192.168.2.65, Po210, [110/81], 6d22h, ospf-1, intra
    *via 192.168.3.65, Po310, [110/81], 6d22h, ospf-1, intra

#Leaf01/02 Leaf03/04 secondary
192.168.0.139/32, ubest/mbest: 2/0, attached
    *via 192.168.0.139, Lo1, [0/0], 2w6d, local
    *via 192.168.0.139, Lo1, [0/0], 2w6d, direct
192.168.0.140/32, ubest/mbest: 3/0
    *via 192.168.1.65, Po110, [110/81], 6d22h, ospf-1, intra
    *via 192.168.2.65, Po210, [110/81], 6d22h, ospf-1, intra
    *via 192.168.3.65, Po310, [110/81], 6d22h, ospf-1, intra

#Leaf01-Spine01 LINK
192.168.1.64/30, ubest/mbest: 1/0, attached
    *via 192.168.1.66, Po110, [0/0], 6d22h, direct

#Leaf01-Spine01 LINK local address
192.168.1.66/32, ubest/mbest: 1/0, attached
    *via 192.168.1.66, Po110, [0/0], 6d22h, local

#Leaf02-Spine01 LINK
192.168.1.68/30, ubest/mbest: 2/0
    *via 192.168.1.65, Po110, [110/80], 6d22h, ospf-1, intra
    *via 192.168.10.10, Vlan999, [110/80], 6d22h, ospf-1, intra

#Leaf03-Spine01 LINK
192.168.1.72/30, ubest/mbest: 1/0
    *via 192.168.1.65, Po110, [110/80], 02:33:52, ospf-1, intra

#Leaf04-Spine01 LINK
192.168.1.76/30, ubest/mbest: 1/0
    *via 192.168.1.65, Po110, [110/80], 6d22h, ospf-1, intra

#Leaf01-Spine02 LINK
192.168.2.64/30, ubest/mbest: 1/0, attached
    *via 192.168.2.66, Po210, [0/0], 6d22h, direct

#Leaf01-Spine02 LINK local address
192.168.2.66/32, ubest/mbest: 1/0, attached
    *via 192.168.2.66, Po210, [0/0], 6d22h, local

#Leaf02-Spine02 LINK
192.168.2.68/30, ubest/mbest: 2/0
    *via 192.168.2.65, Po210, [110/80], 6d22h, ospf-1, intra
    *via 192.168.10.10, Vlan999, [110/80], 6d22h, ospf-1, intra

#Leaf03-Spine02 LINK
192.168.2.72/30, ubest/mbest: 1/0
    *via 192.168.2.65, Po210, [110/80], 02:33:33, ospf-1, intra

#Leaf04-Spine02 LINK
192.168.2.76/30, ubest/mbest: 1/0
    *via 192.168.2.65, Po210, [110/80], 6d22h, ospf-1, intra

#Leaf01-Spine03 LINK
192.168.3.64/30, ubest/mbest: 1/0, attached
    *via 192.168.3.66, Po310, [0/0], 6d22h, direct

#Leaf01-Spine03 LINK local address
192.168.3.66/32, ubest/mbest: 1/0, attached
    *via 192.168.3.66, Po310, [0/0], 6d22h, local

#Leaf02-Spine03 LINK
192.168.3.68/30, ubest/mbest: 2/0
    *via 192.168.3.65, Po310, [110/80], 6d22h, ospf-1, intra
    *via 192.168.10.10, Vlan999, [110/80], 6d22h, ospf-1, intra

#Leaf03-Spine03 LINK
192.168.3.72/30, ubest/mbest: 1/0
    *via 192.168.3.65, Po310, [110/80], 02:31:36, ospf-1, intra

#Leaf04-Spine03 LINK
192.168.3.76/30, ubest/mbest: 1/0
    *via 192.168.3.65, Po310, [110/80], 6d22h, ospf-1, intra

#Leaf01-Leaf02 WATARI VLAN999
192.168.10.8/30, ubest/mbest: 1/0, attached
    *via 192.168.10.9, Vlan999, [0/0], 1w0d, direct

#Leaf01-Leaf02 WATARI VLAN999 local
192.168.10.9/32, ubest/mbest: 1/0, attached
    *via 192.168.10.9, Vlan999, [0/0], 1w0d, local

#Leaf03-Leaf04 WATARI VLAN999
192.168.10.12/30, ubest/mbest: 3/0
    *via 192.168.1.65, Po110, [110/120], 6d22h, ospf-1, intra
    *via 192.168.2.65, Po210, [110/120], 6d22h, ospf-1, intra
    *via 192.168.3.65, Po310, [110/120], 6d22h, ospf-1, intra

leaf01#

・NVE ピアのステータスに関する情報

leaf01#
leaf01# show nve peers
Interface Peer-IP                                 State LearnType Uptime   Route
r-Mac
--------- --------------------------------------  ----- --------- -------- -----
------------
nve1      192.168.0.140                           Up    CP        6d22h    n/a

leaf01#

Peer-IP ⇒VXLANで通信を行う対向のVTEP(peer)のIPアドレスが表示される。※ここではLeaf03-04のLo1セカンダリアドレスが表示されている。
State ⇒Peerが正常に認識していればこちらのStateがUPと表示される
Learn Type ⇒Peer のHost Learning Modeを表示。今回はBGP EVPNベースのVXLANになるためCP(Contorole Plane)と表示される。
Uptime ⇒Peerとの接続性が取れてからの時間が表示される


対向先のLeafセットが増えるごとに行も増えることになる

・Peerの情報の詳細確認

leaf01#
leaf01# show nve peers detail
Details of nve Peers:
----------------------------------------
Peer-Ip: 192.168.0.140
    NVE Interface       : nve1
    Peer State          : Up
    Peer Uptime         : 1w0d
    Router-Mac          : n/a
    Peer First VNI      : 1000
    Time since Create   : 1w0d
    Configured VNIs     : 1000-1001
    Provision State     : peer-add-complete
    Learnt CP VNIs      : 1000-1001
    vni assignment mode : SYMMETRIC
    Peer Location       : N/A

leaf01#

・VXLAN VNIのステータス

leaf01#
leaf01# show vxlan
Vlan            VN-Segment
====            ==========
1000            1000
1001            1001
leaf01#
leaf01#
leaf01# show nve vni
Codes: CP - Control Plane        DP - Data Plane
       UC - Unconfigured         SA - Suppress ARP
       SU - Suppress Unknown Unicast
       Xconn - Crossconnect
       MS-IR - Multisite Ingress Replication

Interface VNI      Multicast-group   State Mode Type [BD/VRF]      Flags
--------- -------- ----------------- ----- ---- ------------------ -----
nve1      1000     UnicastBGP        Up    CP   L2 [1000]
nve1      1001     UnicastBGP        Up    CP   L2 [1001]

leaf01#

VNI ⇒設定しているVNI情報を表示。
Multicast-group ⇒ここではマルチキャストではなくMP-BGP Ingres replicationによるユニキャストでBUMトラフィックを処理するため、UnicastBGPと表示される。
State⇒VNIが正常に動作していればUPと表示される。
Mode⇒今回はBGP EVPNベースのVXLANになるためCP(Control Plane)と表示される。
Type [BD/VRF] ⇒L2もしくはL3が表示される。ここではL2用途。また[]の中は対象のVNIに対応するVLAN番号が表示される。ここではVNI1000とVLAN1000が変換される。
Flags ⇒VNEの設定でSuppress Arpを設定しているVNIについてはこの項目にSAと表示される。

・VXLAN EVPNのL2 MAC address情報

VXLAN経由での通信の疎通確認時には、このテーブルで対象のMAC addressのNext-hopが想定されるVTEPのアドレスとなっていることを確認する。

leaf01#
leaf01# show l2route evpn mac all

Flags -(Rmac):Router MAC (Stt):Static (L):Local (R):Remote (V):vPC link
(Dup):Duplicate (Spl):Split (Rcv):Recv (AD):Auto-Delete (D):Del Pending
(S):Stale (C):Clear, (Ps):Peer Sync (O):Re-Originated (Nho):NH-Override
(Pf):Permanently-Frozen, (Orp): Orphan

Topology    Mac Address    Prod   Flags         Seq No     Next-Hops

----------- -------------- ------ ------------- ---------- ---------------------
------------------
1000        5212.3c23.1b08 Local  L,            0          Po1

1000        5214.bd53.1b08 BGP    Rcv           0          192.168.0.140 (Label:1000)
1001        5202.3531.1b08 Local  L,            0          Po2

1001        5204.d741.1b08 BGP    Rcv           0          192.168.0.140 (Label:1001)
leaf01#



leaf03#
leaf03#  show l2route evpn mac all

Flags -(Rmac):Router MAC (Stt):Static (L):Local (R):Remote (V):vPC link
(Dup):Duplicate (Spl):Split (Rcv):Recv (AD):Auto-Delete (D):Del Pending
(S):Stale (C):Clear, (Ps):Peer Sync (O):Re-Originated (Nho):NH-Override
(Pf):Permanently-Frozen, (Orp): Orphan

Topology    Mac Address    Prod   Flags         Seq No     Next-Hops

----------- -------------- ------ ------------- ---------- ---------------------
------------------
1000        5212.3c23.1b08 BGP    Rcv           0          192.168.0.139 (Label:1000)
1000        5214.bd53.1b08 Local  L,            0          Po1

1001        5202.3531.1b08 BGP    Rcv           0          192.168.0.139 (Label:1001)
1001        5204.d741.1b08 Local  L,            0          Po2

leaf03#

Topology ⇒vlan IDが表示される。
MAC Address ⇒通信によって自動で登録されたMac Addressが表示される。
Prod ⇒通知のタイプが表示される。自身のIFから登録されMACであれば Local。対向のVTEPからVXLAN経由で登録されたMACについてはBGPと表示。
Next-Hop ⇒対象のMACアドレスへの送信先の情報を表示。


・VXLANのVNI毎のBGPルーティング情報

leaf01
leaf01#
leaf01# show bgp l2vpn evpn vni-id 1000
BGP routing table information for VRF default, address family L2VPN EVPN
BGP table version is 126, Local Router ID is 192.168.0.27
Status: s-suppressed, x-deleted, S-stale, d-dampened, h-history, *-valid, >-best
Path type: i-internal, e-external, c-confed, l-local, a-aggregate, r-redist, I-i
njected
Origin codes: i - IGP, e - EGP, ? - incomplete, | - multipath, & - backup, 2 - b
est2

   Network            Next Hop            Metric     LocPrf     Weight Path
Route Distinguisher: 192.168.0.27:33767    (L2VNI 1000)
*>l[2]:[0]:[0]:[48]:[5212.3c23.1b08]:[0]:[0.0.0.0]/216
                      192.168.0.139                     100      32768 i
*>i[2]:[0]:[0]:[48]:[5214.bd53.1b08]:[0]:[0.0.0.0]/216
                      192.168.0.140                     100          0 i
* i                   192.168.0.140                     100          0 i
*>l[3]:[0]:[32]:[192.168.0.139]/88
                      192.168.0.139                     100      32768 i
*>i[3]:[0]:[32]:[192.168.0.140]/88
                      192.168.0.140                     100          0 i
* i                   192.168.0.140                     100          0 i

leaf01#
leaf01# show bgp l2vpn evpn vni-id 1001
BGP routing table information for VRF default, address family L2VPN EVPN
BGP table version is 126, Local Router ID is 192.168.0.27
Status: s-suppressed, x-deleted, S-stale, d-dampened, h-history, *-valid, >-best
Path type: i-internal, e-external, c-confed, l-local, a-aggregate, r-redist, I-i
njected
Origin codes: i - IGP, e - EGP, ? - incomplete, | - multipath, & - backup, 2 - b
est2

   Network            Next Hop            Metric     LocPrf     Weight Path
Route Distinguisher: 192.168.0.27:33768    (L2VNI 1001)
*>l[2]:[0]:[0]:[48]:[5202.3531.1b08]:[0]:[0.0.0.0]/216
                      192.168.0.139                     100      32768 i
*>i[2]:[0]:[0]:[48]:[5204.d741.1b08]:[0]:[0.0.0.0]/216
                      192.168.0.140                     100          0 i
* i                   192.168.0.140                     100          0 i
*>l[3]:[0]:[32]:[192.168.0.139]/88
                      192.168.0.139                     100      32768 i
*>i[3]:[0]:[32]:[192.168.0.140]/88
                      192.168.0.140                     100          0 i
* i                   192.168.0.140                     100          0 i

leaf01#
leaf01#

5204.d741.1b08のMACアドレスへの経路情報が192.168.0.140よりBGPで受け取りベストパスとして選択されている。

leaf03
leaf03#
leaf03# show bgp l2vpn evpn vni-id 1000
BGP routing table information for VRF default, address family L2VPN EVPN
BGP table version is 84, Local Router ID is 192.168.0.29
Status: s-suppressed, x-deleted, S-stale, d-dampened, h-history, *-valid, >-best
Path type: i-internal, e-external, c-confed, l-local, a-aggregate, r-redist, I-i
njected
Origin codes: i - IGP, e - EGP, ? - incomplete, | - multipath, & - backup, 2 - b
est2

   Network            Next Hop            Metric     LocPrf     Weight Path
Route Distinguisher: 192.168.0.29:33767    (L2VNI 1000)
* i[2]:[0]:[0]:[48]:[5212.3c23.1b08]:[0]:[0.0.0.0]/216
                      192.168.0.139                     100          0 i
*>i                   192.168.0.139                     100          0 i
*>l[2]:[0]:[0]:[48]:[5214.bd53.1b08]:[0]:[0.0.0.0]/216
                      192.168.0.140                     100      32768 i
*>i[3]:[0]:[32]:[192.168.0.139]/88
                      192.168.0.139                     100          0 i
* i                   192.168.0.139                     100          0 i
*>l[3]:[0]:[32]:[192.168.0.140]/88
                      192.168.0.140                     100      32768 i

leaf03#
leaf03# show bgp l2vpn evpn vni-id 1001
BGP routing table information for VRF default, address family L2VPN EVPN
BGP table version is 84, Local Router ID is 192.168.0.29
Status: s-suppressed, x-deleted, S-stale, d-dampened, h-history, *-valid, >-best
Path type: i-internal, e-external, c-confed, l-local, a-aggregate, r-redist, I-i
njected
Origin codes: i - IGP, e - EGP, ? - incomplete, | - multipath, & - backup, 2 - b
est2

   Network            Next Hop            Metric     LocPrf     Weight Path
Route Distinguisher: 192.168.0.29:33768    (L2VNI 1001)
*>i[2]:[0]:[0]:[48]:[5202.3531.1b08]:[0]:[0.0.0.0]/216
                      192.168.0.139                     100          0 i
* i                   192.168.0.139                     100          0 i
*>l[2]:[0]:[0]:[48]:[5204.d741.1b08]:[0]:[0.0.0.0]/216
                      192.168.0.140                     100      32768 i
*>i[3]:[0]:[32]:[192.168.0.139]/88
                      192.168.0.139                     100          0 i
* i                   192.168.0.139                     100          0 i
*>l[3]:[0]:[32]:[192.168.0.140]/88
                      192.168.0.140                     100      32768 i

leaf03#
leaf03#

Spine

OSPF/BGPネイバー

spine01#
spine01# show ip ospf neighbors
 OSPF Process ID 1 VRF default
 Total number of neighbors: 4
 Neighbor ID     Pri State            Up Time  Address         Interface
 192.168.0.27      1 FULL/ -          1w0d     192.168.1.66    Po110
 192.168.0.28      1 FULL/ -          1w0d     192.168.1.70    Po111
 192.168.0.29      1 FULL/ -          00:06:14 192.168.1.74    Po112
 192.168.0.30      1 FULL/ -          6d23h    192.168.1.78    Po113
spine01#
spine01#
spine01# show ip bgp summary
BGP summary information for VRF default, address family IPv4 Unicast
BGP router identifier 192.168.0.1, local AS number 65500
BGP table version is 8, IPv4 Unicast config peers 4, capable peers 4
0 network entries and 0 paths using 0 bytes of memory
BGP attribute entries [0/0], BGP AS path entries [0/0]
BGP community entries [0/0], BGP clusterlist entries [0/0]

Neighbor        V    AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
192.168.0.27    4 65500   30254   30253        8    0    0     1w0d 0
192.168.0.28    4 65500   30253   30252        8    0    0     1w0d 0
192.168.0.29    4 65500   30123   30123        8    0    0 00:09:00 0
192.168.0.30    4 65500   30247   30247        8    0    0     1w0d 0
spine01#

ルーティング情報

spine01#
spine01# show ip route
IP Route Table for VRF "default"
'*' denotes best ucast next-hop
'**' denotes best mcast next-hop
'[x/y]' denotes [preference/metric]
'%<string>' in via output denotes VRF <string>

192.168.0.1/32, ubest/mbest: 2/0, attached
    *via 192.168.0.1, Lo0, [0/0], 1w0d, local
    *via 192.168.0.1, Lo0, [0/0], 1w0d, direct
192.168.0.2/32, ubest/mbest: 4/0
    *via 192.168.1.66, Po110, [110/81], 1w0d, ospf-1, intra
    *via 192.168.1.70, Po111, [110/81], 1w0d, ospf-1, intra
    *via 192.168.1.74, Po112, [110/81], 06:03:37, ospf-1, intra
    *via 192.168.1.78, Po113, [110/81], 1w0d, ospf-1, intra
192.168.0.3/32, ubest/mbest: 4/0
    *via 192.168.1.66, Po110, [110/81], 1w0d, ospf-1, intra
    *via 192.168.1.70, Po111, [110/81], 1w0d, ospf-1, intra
    *via 192.168.1.74, Po112, [110/81], 06:03:27, ospf-1, intra
    *via 192.168.1.78, Po113, [110/81], 1w0d, ospf-1, intra
192.168.0.27/32, ubest/mbest: 1/0
    *via 192.168.1.66, Po110, [110/41], 1w0d, ospf-1, intra
192.168.0.28/32, ubest/mbest: 1/0
    *via 192.168.1.70, Po111, [110/41], 1w0d, ospf-1, intra
192.168.0.29/32, ubest/mbest: 1/0
    *via 192.168.1.74, Po112, [110/41], 06:03:40, ospf-1, intra
192.168.0.30/32, ubest/mbest: 1/0
    *via 192.168.1.78, Po113, [110/41], 1w0d, ospf-1, intra
192.168.0.87/32, ubest/mbest: 1/0
    *via 192.168.1.66, Po110, [110/41], 1w0d, ospf-1, intra
192.168.0.88/32, ubest/mbest: 1/0
    *via 192.168.1.70, Po111, [110/41], 1w0d, ospf-1, intra
192.168.0.89/32, ubest/mbest: 1/0
    *via 192.168.1.74, Po112, [110/41], 00:47:19, ospf-1, intra
192.168.0.90/32, ubest/mbest: 1/0
    *via 192.168.1.78, Po113, [110/41], 1w0d, ospf-1, intra
192.168.0.139/32, ubest/mbest: 2/0
    *via 192.168.1.66, Po110, [110/41], 1w0d, ospf-1, intra
    *via 192.168.1.70, Po111, [110/41], 1w0d, ospf-1, intra
192.168.0.140/32, ubest/mbest: 2/0
    *via 192.168.1.74, Po112, [110/41], 00:47:19, ospf-1, intra
    *via 192.168.1.78, Po113, [110/41], 1w0d, ospf-1, intra
192.168.1.64/30, ubest/mbest: 1/0, attached
    *via 192.168.1.65, Po110, [0/0], 1w0d, direct
192.168.1.65/32, ubest/mbest: 1/0, attached
    *via 192.168.1.65, Po110, [0/0], 1w0d, local
192.168.1.68/30, ubest/mbest: 1/0, attached
    *via 192.168.1.69, Po111, [0/0], 1w0d, direct
192.168.1.69/32, ubest/mbest: 1/0, attached
    *via 192.168.1.69, Po111, [0/0], 1w0d, local
192.168.1.72/30, ubest/mbest: 1/0, attached
    *via 192.168.1.73, Po112, [0/0], 06:43:37, direct
192.168.1.73/32, ubest/mbest: 1/0, attached
    *via 192.168.1.73, Po112, [0/0], 06:43:37, local
192.168.1.76/30, ubest/mbest: 1/0, attached
    *via 192.168.1.77, Po113, [0/0], 1w0d, direct
192.168.1.77/32, ubest/mbest: 1/0, attached
    *via 192.168.1.77, Po113, [0/0], 1w0d, local
192.168.2.64/30, ubest/mbest: 1/0
    *via 192.168.1.66, Po110, [110/80], 1w0d, ospf-1, intra
192.168.2.68/30, ubest/mbest: 1/0
    *via 192.168.1.70, Po111, [110/80], 1w0d, ospf-1, intra
192.168.2.72/30, ubest/mbest: 1/0
    *via 192.168.1.74, Po112, [110/80], 06:03:40, ospf-1, intra
192.168.2.76/30, ubest/mbest: 1/0
    *via 192.168.1.78, Po113, [110/80], 1w0d, ospf-1, intra
192.168.3.64/30, ubest/mbest: 1/0
    *via 192.168.1.66, Po110, [110/80], 1w0d, ospf-1, intra
192.168.3.68/30, ubest/mbest: 1/0
    *via 192.168.1.70, Po111, [110/80], 1w0d, ospf-1, intra
192.168.3.72/30, ubest/mbest: 1/0
    *via 192.168.1.74, Po112, [110/80], 06:03:36, ospf-1, intra
192.168.3.76/30, ubest/mbest: 1/0
    *via 192.168.1.78, Po113, [110/80], 1w0d, ospf-1, intra
192.168.10.8/30, ubest/mbest: 2/0
    *via 192.168.1.66, Po110, [110/80], 1w0d, ospf-1, intra
    *via 192.168.1.70, Po111, [110/80], 1w0d, ospf-1, intra
192.168.10.12/30, ubest/mbest: 2/0
    *via 192.168.1.74, Po112, [110/80], 00:48:39, ospf-1, intra
    *via 192.168.1.78, Po113, [110/80], 01:41:57, ospf-1, intra

spine01#
3
2
0

Register as a new user and use Qiita more conveniently

  1. You get articles that match your needs
  2. You can efficiently read back useful information
  3. You can use dark theme
What you can do with signing up
Sign upLogin
3
2

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?