はじめに

MXシリーズでMPLS/VPLSを利用したサービスを提供しているのですが、ある日新規に設定したVPLSがUpしない事象に遭遇しました。
原因や対策についての結果をまとめました。

結論

結論からお話しすると、MXシリーズのルータではLSPのnext-hop ECMPがデフォルト16で設定されています。
そのため、同一宛先に設定されたLSPが16を超過すると、超過したLSPはroute tableに乗らないため通信できないという事象が発生します。

検証環境

下記、Juniper vLabs環境で実施しております。
vMX:21.1R3.11
vMX1 - vMX2 : LSP65本
vMX1 - vMX3 : LSP65本

同一宛先16本しか通信できないことの確認

vMX1

show route table inet.3

vMX2(10.100.100.2)向け：65本中16本が確認できる。残り49本は確認できない。
vMX3(10.100.100.3)向け：65本中16本が確認できる。残り49本は確認できない。

jcluser@vMX1> show route table inet.3    

inet.3: 2 destinations, 2 routes (2 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

10.100.100.2/32    *[RSVP/7/1] 01:18:53, metric 1
                       to 10.100.12.2 via ge-0/0/0.0, label-switched-path vMX1-to-vMX2-00
                       to 10.100.12.2 via ge-0/0/0.0, label-switched-path vMX1-to-vMX2-01
                       to 10.100.12.2 via ge-0/0/0.0, label-switched-path vMX1-to-vMX2-02
                       to 10.100.12.2 via ge-0/0/0.0, label-switched-path vMX1-to-vMX2-03
                       to 10.100.12.2 via ge-0/0/0.0, label-switched-path vMX1-to-vMX2-04
                       to 10.100.12.2 via ge-0/0/0.0, label-switched-path vMX1-to-vMX2-05
                       to 10.100.12.2 via ge-0/0/0.0, label-switched-path vMX1-to-vMX2-06
                       to 10.100.12.2 via ge-0/0/0.0, label-switched-path vMX1-to-vMX2-07
                    >  to 10.100.12.2 via ge-0/0/0.0, label-switched-path vMX1-to-vMX2-08
                       to 10.100.12.2 via ge-0/0/0.0, label-switched-path vMX1-to-vMX2-09
                       to 10.100.12.2 via ge-0/0/0.0, label-switched-path vMX1-to-vMX2-10
                       to 10.100.12.2 via ge-0/0/0.0, label-switched-path vMX1-to-vMX2-11
                       to 10.100.12.2 via ge-0/0/0.0, label-switched-path vMX1-to-vMX2-12
                       to 10.100.12.2 via ge-0/0/0.0, label-switched-path vMX1-to-vMX2-13
                       to 10.100.12.2 via ge-0/0/0.0, label-switched-path vMX1-to-vMX2-14
                       to 10.100.12.2 via ge-0/0/0.0, label-switched-path vMX1-to-vMX2-15
10.100.100.3/32    *[RSVP/7/1] 00:40:37, metric 1
                       to 10.100.13.2 via ge-0/0/2.0, label-switched-path vMX1-to-vMX3-00
                       to 10.100.13.2 via ge-0/0/2.0, label-switched-path vMX1-to-vMX3-01
                       to 10.100.13.2 via ge-0/0/2.0, label-switched-path vMX1-to-vMX3-02
                       to 10.100.13.2 via ge-0/0/2.0, label-switched-path vMX1-to-vMX3-03
                       to 10.100.13.2 via ge-0/0/2.0, label-switched-path vMX1-to-vMX3-04
                       to 10.100.13.2 via ge-0/0/2.0, label-switched-path vMX1-to-vMX3-05
                       to 10.100.13.2 via ge-0/0/2.0, label-switched-path vMX1-to-vMX3-06
                       to 10.100.13.2 via ge-0/0/2.0, label-switched-path vMX1-to-vMX3-07
                       to 10.100.13.2 via ge-0/0/2.0, label-switched-path vMX1-to-vMX3-08
                    >  to 10.100.13.2 via ge-0/0/2.0, label-switched-path vMX1-to-vMX3-09
                       to 10.100.13.2 via ge-0/0/2.0, label-switched-path vMX1-to-vMX3-10
                       to 10.100.13.2 via ge-0/0/2.0, label-switched-path vMX1-to-vMX3-11
                       to 10.100.13.2 via ge-0/0/2.0, label-switched-path vMX1-to-vMX3-12
                       to 10.100.13.2 via ge-0/0/2.0, label-switched-path vMX1-to-vMX3-13
                       to 10.100.13.2 via ge-0/0/2.0, label-switched-path vMX1-to-vMX3-14
                       to 10.100.13.2 via ge-0/0/2.0, label-switched-path vMX1-to-vMX3-15

jcluser@vMX1>

show vpls connections

各LSPに紐づいているVPLSの状態
vMX2(10.100.100.2)向け：65本中16本がUp、残り49本がVC-Dn。
vMX3(10.100.100.3)向け：65本中16本がUp、残り49本がVC-Dn。

jcluser@vMX1> show vpls connections 
Layer-2 VPN connections:

Legend for connection status (St)   
EI -- encapsulation invalid      NC -- interface encapsulation not CCC/TCC/VPLS
EM -- encapsulation mismatch     WE -- interface and instance encaps not same
VC-Dn -- Virtual circuit down    NP -- interface hardware not present 
CM -- control-word mismatch      -> -- only outbound connection is up
CN -- circuit not provisioned    <- -- only inbound connection is up
OR -- out of range               Up -- operational
OL -- no outgoing label          Dn -- down                      
LD -- local site signaled down   CF -- call admission control failure      
RD -- remote site signaled down  SC -- local and remote site ID collision
LN -- local site not designated  LM -- local site ID not minimum designated
RN -- remote site not designated RM -- remote site ID not minimum designated
XX -- unknown connection status  IL -- no incoming label
MM -- MTU mismatch               MI -- Mesh-Group ID not available
BK -- Backup connection         ST -- Standby connection
PF -- Profile parse failure      PB -- Profile busy
RS -- remote site standby SN -- Static Neighbor
LB -- Local site not best-site   RB -- Remote site not best-site
VM -- VLAN ID mismatch           HS -- Hot-standby Connection

Legend for interface status 
Up -- operational           
Dn -- down

Instance: rt00000
Edge protection: Not-Primary
  Local site: rt00000 (1)
    connection-site           Type  St     Time last up          # Up trans
    2                         rmt   Up     Dec 25 04:34:39 2023           1
      Remote PE: 10.100.100.2, Negotiated control-word: No
      Incoming label: 262154, Outgoing label: 262153
      Local interface: lsi.1048576, Status: Up, Encapsulation: VPLS
        Description: Intf - vpls rt00000 local site 1 remote site 2
      Flow Label Transmit: No, Flow Label Receive: No

Instance: rt00001
Edge protection: Not-Primary
  Local site: rt00001 (1)
    connection-site           Type  St     Time last up          # Up trans
    2                         rmt   Up     Dec 25 05:54:22 2023           1
      Remote PE: 10.100.100.2, Negotiated control-word: No
      Incoming label: 262162, Outgoing label: 262161
      Local interface: lsi.1048642, Status: Up, Encapsulation: VPLS
        Description: Intf - vpls rt00001 local site 1 remote site 2
      Flow Label Transmit: No, Flow Label Receive: No

~

Instance: rt00015
Edge protection: Not-Primary            
  Local site: rt00015 (1)
    connection-site           Type  St     Time last up          # Up trans
    2                         rmt   Up     Dec 25 05:54:23 2023           1
      Remote PE: 10.100.100.2, Negotiated control-word: No
      Incoming label: 262274, Outgoing label: 262273
      Local interface: lsi.1048692, Status: Up, Encapsulation: VPLS
        Description: Intf - vpls rt00015 local site 1 remote site 2
      Flow Label Transmit: No, Flow Label Receive: No

Instance: rt00016
Edge protection: Not-Primary
  Local site: rt00016 (1)
    connection-site           Type  St     Time last up          # Up trans
    2                         rmt   VC-Dn  -----                          0
      Remote PE: 10.100.100.2, Negotiated control-word: No
      Incoming label: 262282, Outgoing label: 262281
      Local interface: lsi.1048652, Status: Up, Encapsulation: VPLS
        Description: Intf - vpls rt00016 local site 1 remote site 2
      Flow Label Transmit: No, Flow Label Receive: No
~      

Instance: rt00064
Edge protection: Not-Primary
  Local site: rt00064 (1)
    connection-site           Type  St     Time last up          # Up trans
    2                         rmt   VC-Dn  -----                          0
      Remote PE: 10.100.100.2, Negotiated control-word: No
      Incoming label: 327690, Outgoing label: 327689
      Local interface: lsi.1048705, Status: Up, Encapsulation: VPLS
        Description: Intf - vpls rt00064 local site 1 remote site 2
      Flow Label Transmit: No, Flow Label Receive: No

Instance: rt10000
Edge protection: Not-Primary
  Local site: rt10000 (1)
    connection-site           Type  St     Time last up          # Up trans
    3                         rmt   Up     Dec 25 05:43:09 2023           1
      Remote PE: 10.100.100.3, Negotiated control-word: No
      Incoming label: 327699, Outgoing label: 262145
      Local interface: lsi.1048577, Status: Up, Encapsulation: VPLS
        Description: Intf - vpls rt10000 local site 1 remote site 3
      Flow Label Transmit: No, Flow Label Receive: No

Instance: rt10001
Edge protection: Not-Primary
  Local site: rt10001 (1)
    connection-site           Type  St     Time last up          # Up trans
    3                         rmt   Up     Dec 25 05:43:09 2023           1
      Remote PE: 10.100.100.3, Negotiated control-word: No
      Incoming label: 327707, Outgoing label: 262153
      Local interface: lsi.1048578, Status: Up, Encapsulation: VPLS
        Description: Intf - vpls rt10001 local site 1 remote site 3
      Flow Label Transmit: No, Flow Label Receive: No

~

Instance: rt10015
Edge protection: Not-Primary
  Local site: rt10015 (1)
    connection-site           Type  St     Time last up          # Up trans
    3                         rmt   Up     Dec 25 05:43:09 2023           1
      Remote PE: 10.100.100.3, Negotiated control-word: No
      Incoming label: 327819, Outgoing label: 262265
      Local interface: lsi.1048592, Status: Up, Encapsulation: VPLS
        Description: Intf - vpls rt10015 local site 1 remote site 3
      Flow Label Transmit: No, Flow Label Receive: No

Instance: rt10016
Edge protection: Not-Primary
  Local site: rt10016 (1)
    connection-site           Type  St     Time last up          # Up trans
    3                         rmt   VC-Dn  -----                          0
      Remote PE: 10.100.100.3, Negotiated control-word: No
      Incoming label: 327827, Outgoing label: 262273
      Local interface: lsi.1048593, Status: Up, Encapsulation: VPLS
        Description: Intf - vpls rt10016 local site 1 remote site 3
      Flow Label Transmit: No, Flow Label Receive: No

~
 
Instance: rt10064
Edge protection: Not-Primary
  Local site: rt10064 (1)
    connection-site           Type  St     Time last up          # Up trans
    3                         rmt   VC-Dn  -----                          0
      Remote PE: 10.100.100.3, Negotiated control-word: No
      Incoming label: 393243, Outgoing label: 327689
      Local interface: lsi.1048641, Status: Up, Encapsulation: VPLS
        Description: Intf - vpls rt10064 local site 1 remote site 3
      Flow Label Transmit: No, Flow Label Receive: No

jcluser@vMX1>

show route forwarding-table table "table-name"

rt00000,rt10000(Up) : lsi及びvMX2向けInterfaceが確認できる
rt00016,rt10016(VC-Dn) : lsiが確認できない

jcluser@vMX1> show route forwarding-table table rt00000    
Routing table: rt00000.vpls
VPLS:
Enabled protocols: ACKed by all peers, 
Destination        Type RtRef Next hop           Type Index    NhRef Netif
default            perm     0                    dscd      580     1
lsi.1048576        intf     0                    indr  1048574     4
                              10.100.12.2       Push 262153      598     2 ge-0/0/0.0
0x30004/51         user     0                    comp      602     2
ge-0/0/1.1000      intf     0                    ucst      592     3 ge-0/0/1.1000
0x30005/51         user     0                    comp      596     2
0x30003/51         user     0                    comp      597     2

jcluser@vMX1> show route forwarding-table table rt00016    
Routing table: rt00016.vpls
VPLS:
Enabled protocols: ACKed by all peers, 
Destination        Type RtRef Next hop           Type Index    NhRef Netif
default            perm     0                    dscd      669     1
ge-0/0/1.1016      intf     0                    ucst      762     3 ge-0/0/1.1016
0x30065/51         user     0                    comp     1074     2
0x30015/51         user     0                    comp     1075     2

jcluser@vMX1> 

jcluser@vMX1> show route forwarding-table table rt10000    
Routing table: rt10000.vpls
VPLS:
Enabled protocols: ACKed by all peers, 
Destination        Type RtRef Next hop           Type Index    NhRef Netif
default            perm     0                    dscd     1179     1
lsi.1048577        intf     0                    indr  1048575     4
                              10.100.13.2       Push 262145     1764     2 ge-0/0/2.0
0x30107/51         user     0                    comp     1787     2
ge-0/0/1.2000      intf     0                    ucst     1195     3 ge-0/0/1.2000
0x30108/51         user     0                    comp     1634     2
0x300c6/51         user     0                    comp     1635     2

jcluser@vMX1> show route forwarding-table table rt10016    
Routing table: rt10016.vpls
VPLS:
Enabled protocols: ACKed by all peers, 
Destination        Type RtRef Next hop           Type Index    NhRef Netif
default            perm     0                    dscd     1250     1
ge-0/0/1.2016      intf     0                    ucst     1339     3 ge-0/0/1.2016
0x30128/51         user     0                    comp     1666     2
0x300d6/51         user     0                    comp     1667     2

jcluser@vMX1> 
ge-0/0/1.1002      intf     0

※ vMX2,vMX3も同様に16本で制限されています。長くなるため本記事ではログは省略しています。　

制限拡張

ECMPの値による制限のため、ECMPの値を変更すれば拡張することができます。
設定可能なパラメータは下記ヘルプの通りです。
今回は64で設定して、有効であることを確認しました。

jcluser@vMX1# set chassis maximum-ecmp ? 
Possible completions:
  16                   Maximum 16 ECMP routes
  32                   Maximum 32 ECMP routes
  64                   Maximum 64 ECMP routes
  96                   Maximum 96 ECMP routes
  128                  Maximum 128 ECMP routes
[edit]
jcluser@vMX1# set chassis maximum-ecmp 64    

[edit]
jcluser@vMX1# commit 
commit complete

[edit]
jcluser@vMX1# 

jcluser@vMX1# run show configuration chassis maximum-ecmp | display set    
set chassis maximum-ecmp 64

[edit]
jcluser@vMX1#

vMX1

show route table inet.3

vMX2(10.100.100.2)向け：65本中64本が確認できる。1本(65本目)は確認できない。
vMX3(10.100.100.3)向け：65本中64本が確認できる。1本(65本目)は確認できない。

jcluser@vMX1> show route table inet.3 | no-more 

inet.3: 2 destinations, 2 routes (2 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

10.100.100.2/32    *[RSVP/7/1] 00:10:01, metric 1
                       to 10.100.12.2 via ge-0/0/0.0, label-switched-path vMX1-to-vMX2-00
                       to 10.100.12.2 via ge-0/0/0.0, label-switched-path vMX1-to-vMX2-01
                       to 10.100.12.2 via ge-0/0/0.0, label-switched-path vMX1-to-vMX2-02
                       to 10.100.12.2 via ge-0/0/0.0, label-switched-path vMX1-to-vMX2-03
                       to 10.100.12.2 via ge-0/0/0.0, label-switched-path vMX1-to-vMX2-04
                       to 10.100.12.2 via ge-0/0/0.0, label-switched-path vMX1-to-vMX2-05
                       to 10.100.12.2 via ge-0/0/0.0, label-switched-path vMX1-to-vMX2-06
                       to 10.100.12.2 via ge-0/0/0.0, label-switched-path vMX1-to-vMX2-07
                       to 10.100.12.2 via ge-0/0/0.0, label-switched-path vMX1-to-vMX2-08
                       to 10.100.12.2 via ge-0/0/0.0, label-switched-path vMX1-to-vMX2-09
                       to 10.100.12.2 via ge-0/0/0.0, label-switched-path vMX1-to-vMX2-10
                       to 10.100.12.2 via ge-0/0/0.0, label-switched-path vMX1-to-vMX2-11
                       to 10.100.12.2 via ge-0/0/0.0, label-switched-path vMX1-to-vMX2-12
                       to 10.100.12.2 via ge-0/0/0.0, label-switched-path vMX1-to-vMX2-13
                       to 10.100.12.2 via ge-0/0/0.0, label-switched-path vMX1-to-vMX2-14
                       to 10.100.12.2 via ge-0/0/0.0, label-switched-path vMX1-to-vMX2-15
                       to 10.100.12.2 via ge-0/0/0.0, label-switched-path vMX1-to-vMX2-16
                       to 10.100.12.2 via ge-0/0/0.0, label-switched-path vMX1-to-vMX2-17
                       to 10.100.12.2 via ge-0/0/0.0, label-switched-path vMX1-to-vMX2-18
                       to 10.100.12.2 via ge-0/0/0.0, label-switched-path vMX1-to-vMX2-19
                       to 10.100.12.2 via ge-0/0/0.0, label-switched-path vMX1-to-vMX2-20
                       to 10.100.12.2 via ge-0/0/0.0, label-switched-path vMX1-to-vMX2-21
                       to 10.100.12.2 via ge-0/0/0.0, label-switched-path vMX1-to-vMX2-22
                       to 10.100.12.2 via ge-0/0/0.0, label-switched-path vMX1-to-vMX2-23
                       to 10.100.12.2 via ge-0/0/0.0, label-switched-path vMX1-to-vMX2-24
                       to 10.100.12.2 via ge-0/0/0.0, label-switched-path vMX1-to-vMX2-25
                       to 10.100.12.2 via ge-0/0/0.0, label-switched-path vMX1-to-vMX2-26
                       to 10.100.12.2 via ge-0/0/0.0, label-switched-path vMX1-to-vMX2-27
                       to 10.100.12.2 via ge-0/0/0.0, label-switched-path vMX1-to-vMX2-28
                       to 10.100.12.2 via ge-0/0/0.0, label-switched-path vMX1-to-vMX2-29
                       to 10.100.12.2 via ge-0/0/0.0, label-switched-path vMX1-to-vMX2-30
                       to 10.100.12.2 via ge-0/0/0.0, label-switched-path vMX1-to-vMX2-31
                    >  to 10.100.12.2 via ge-0/0/0.0, label-switched-path vMX1-to-vMX2-32
                       to 10.100.12.2 via ge-0/0/0.0, label-switched-path vMX1-to-vMX2-33
                       to 10.100.12.2 via ge-0/0/0.0, label-switched-path vMX1-to-vMX2-34
                       to 10.100.12.2 via ge-0/0/0.0, label-switched-path vMX1-to-vMX2-35
                       to 10.100.12.2 via ge-0/0/0.0, label-switched-path vMX1-to-vMX2-36
                       to 10.100.12.2 via ge-0/0/0.0, label-switched-path vMX1-to-vMX2-37
                       to 10.100.12.2 via ge-0/0/0.0, label-switched-path vMX1-to-vMX2-38
                       to 10.100.12.2 via ge-0/0/0.0, label-switched-path vMX1-to-vMX2-39
                       to 10.100.12.2 via ge-0/0/0.0, label-switched-path vMX1-to-vMX2-40
                       to 10.100.12.2 via ge-0/0/0.0, label-switched-path vMX1-to-vMX2-41
                       to 10.100.12.2 via ge-0/0/0.0, label-switched-path vMX1-to-vMX2-42
                       to 10.100.12.2 via ge-0/0/0.0, label-switched-path vMX1-to-vMX2-43
                       to 10.100.12.2 via ge-0/0/0.0, label-switched-path vMX1-to-vMX2-44
                       to 10.100.12.2 via ge-0/0/0.0, label-switched-path vMX1-to-vMX2-45
                       to 10.100.12.2 via ge-0/0/0.0, label-switched-path vMX1-to-vMX2-46
                       to 10.100.12.2 via ge-0/0/0.0, label-switched-path vMX1-to-vMX2-47
                       to 10.100.12.2 via ge-0/0/0.0, label-switched-path vMX1-to-vMX2-48
                       to 10.100.12.2 via ge-0/0/0.0, label-switched-path vMX1-to-vMX2-49
                       to 10.100.12.2 via ge-0/0/0.0, label-switched-path vMX1-to-vMX2-50
                       to 10.100.12.2 via ge-0/0/0.0, label-switched-path vMX1-to-vMX2-51
                       to 10.100.12.2 via ge-0/0/0.0, label-switched-path vMX1-to-vMX2-52
                       to 10.100.12.2 via ge-0/0/0.0, label-switched-path vMX1-to-vMX2-53
                       to 10.100.12.2 via ge-0/0/0.0, label-switched-path vMX1-to-vMX2-54
                       to 10.100.12.2 via ge-0/0/0.0, label-switched-path vMX1-to-vMX2-55
                       to 10.100.12.2 via ge-0/0/0.0, label-switched-path vMX1-to-vMX2-56
                       to 10.100.12.2 via ge-0/0/0.0, label-switched-path vMX1-to-vMX2-57
                       to 10.100.12.2 via ge-0/0/0.0, label-switched-path vMX1-to-vMX2-58
                       to 10.100.12.2 via ge-0/0/0.0, label-switched-path vMX1-to-vMX2-59
                       to 10.100.12.2 via ge-0/0/0.0, label-switched-path vMX1-to-vMX2-60
                       to 10.100.12.2 via ge-0/0/0.0, label-switched-path vMX1-to-vMX2-61
                       to 10.100.12.2 via ge-0/0/0.0, label-switched-path vMX1-to-vMX2-62
                       to 10.100.12.2 via ge-0/0/0.0, label-switched-path vMX1-to-vMX2-63
10.100.100.3/32    *[RSVP/7/1] 00:10:01, metric 1
                       to 10.100.13.2 via ge-0/0/2.0, label-switched-path vMX1-to-vMX3-00
                       to 10.100.13.2 via ge-0/0/2.0, label-switched-path vMX1-to-vMX3-01
                       to 10.100.13.2 via ge-0/0/2.0, label-switched-path vMX1-to-vMX3-02
                       to 10.100.13.2 via ge-0/0/2.0, label-switched-path vMX1-to-vMX3-03
                       to 10.100.13.2 via ge-0/0/2.0, label-switched-path vMX1-to-vMX3-04
                       to 10.100.13.2 via ge-0/0/2.0, label-switched-path vMX1-to-vMX3-05
                       to 10.100.13.2 via ge-0/0/2.0, label-switched-path vMX1-to-vMX3-06
                       to 10.100.13.2 via ge-0/0/2.0, label-switched-path vMX1-to-vMX3-07
                       to 10.100.13.2 via ge-0/0/2.0, label-switched-path vMX1-to-vMX3-08
                       to 10.100.13.2 via ge-0/0/2.0, label-switched-path vMX1-to-vMX3-09
                       to 10.100.13.2 via ge-0/0/2.0, label-switched-path vMX1-to-vMX3-10
                       to 10.100.13.2 via ge-0/0/2.0, label-switched-path vMX1-to-vMX3-11
                       to 10.100.13.2 via ge-0/0/2.0, label-switched-path vMX1-to-vMX3-12
                       to 10.100.13.2 via ge-0/0/2.0, label-switched-path vMX1-to-vMX3-13
                       to 10.100.13.2 via ge-0/0/2.0, label-switched-path vMX1-to-vMX3-14
                       to 10.100.13.2 via ge-0/0/2.0, label-switched-path vMX1-to-vMX3-15
                       to 10.100.13.2 via ge-0/0/2.0, label-switched-path vMX1-to-vMX3-16
                       to 10.100.13.2 via ge-0/0/2.0, label-switched-path vMX1-to-vMX3-17
                       to 10.100.13.2 via ge-0/0/2.0, label-switched-path vMX1-to-vMX3-18
                       to 10.100.13.2 via ge-0/0/2.0, label-switched-path vMX1-to-vMX3-19
                       to 10.100.13.2 via ge-0/0/2.0, label-switched-path vMX1-to-vMX3-20
                       to 10.100.13.2 via ge-0/0/2.0, label-switched-path vMX1-to-vMX3-21
                       to 10.100.13.2 via ge-0/0/2.0, label-switched-path vMX1-to-vMX3-22
                       to 10.100.13.2 via ge-0/0/2.0, label-switched-path vMX1-to-vMX3-23
                       to 10.100.13.2 via ge-0/0/2.0, label-switched-path vMX1-to-vMX3-24
                       to 10.100.13.2 via ge-0/0/2.0, label-switched-path vMX1-to-vMX3-25
                       to 10.100.13.2 via ge-0/0/2.0, label-switched-path vMX1-to-vMX3-26
                       to 10.100.13.2 via ge-0/0/2.0, label-switched-path vMX1-to-vMX3-27
                       to 10.100.13.2 via ge-0/0/2.0, label-switched-path vMX1-to-vMX3-28
                       to 10.100.13.2 via ge-0/0/2.0, label-switched-path vMX1-to-vMX3-29
                       to 10.100.13.2 via ge-0/0/2.0, label-switched-path vMX1-to-vMX3-30
                       to 10.100.13.2 via ge-0/0/2.0, label-switched-path vMX1-to-vMX3-31
                       to 10.100.13.2 via ge-0/0/2.0, label-switched-path vMX1-to-vMX3-32
                       to 10.100.13.2 via ge-0/0/2.0, label-switched-path vMX1-to-vMX3-33
                       to 10.100.13.2 via ge-0/0/2.0, label-switched-path vMX1-to-vMX3-34
                       to 10.100.13.2 via ge-0/0/2.0, label-switched-path vMX1-to-vMX3-35
                    >  to 10.100.13.2 via ge-0/0/2.0, label-switched-path vMX1-to-vMX3-36
                       to 10.100.13.2 via ge-0/0/2.0, label-switched-path vMX1-to-vMX3-37
                       to 10.100.13.2 via ge-0/0/2.0, label-switched-path vMX1-to-vMX3-38
                       to 10.100.13.2 via ge-0/0/2.0, label-switched-path vMX1-to-vMX3-39
                       to 10.100.13.2 via ge-0/0/2.0, label-switched-path vMX1-to-vMX3-40
                       to 10.100.13.2 via ge-0/0/2.0, label-switched-path vMX1-to-vMX3-41
                       to 10.100.13.2 via ge-0/0/2.0, label-switched-path vMX1-to-vMX3-42
                       to 10.100.13.2 via ge-0/0/2.0, label-switched-path vMX1-to-vMX3-43
                       to 10.100.13.2 via ge-0/0/2.0, label-switched-path vMX1-to-vMX3-44
                       to 10.100.13.2 via ge-0/0/2.0, label-switched-path vMX1-to-vMX3-45
                       to 10.100.13.2 via ge-0/0/2.0, label-switched-path vMX1-to-vMX3-46
                       to 10.100.13.2 via ge-0/0/2.0, label-switched-path vMX1-to-vMX3-47
                       to 10.100.13.2 via ge-0/0/2.0, label-switched-path vMX1-to-vMX3-48
                       to 10.100.13.2 via ge-0/0/2.0, label-switched-path vMX1-to-vMX3-49
                       to 10.100.13.2 via ge-0/0/2.0, label-switched-path vMX1-to-vMX3-50
                       to 10.100.13.2 via ge-0/0/2.0, label-switched-path vMX1-to-vMX3-51
                       to 10.100.13.2 via ge-0/0/2.0, label-switched-path vMX1-to-vMX3-52
                       to 10.100.13.2 via ge-0/0/2.0, label-switched-path vMX1-to-vMX3-53
                       to 10.100.13.2 via ge-0/0/2.0, label-switched-path vMX1-to-vMX3-54
                       to 10.100.13.2 via ge-0/0/2.0, label-switched-path vMX1-to-vMX3-55
                       to 10.100.13.2 via ge-0/0/2.0, label-switched-path vMX1-to-vMX3-56
                       to 10.100.13.2 via ge-0/0/2.0, label-switched-path vMX1-to-vMX3-57
                       to 10.100.13.2 via ge-0/0/2.0, label-switched-path vMX1-to-vMX3-58
                       to 10.100.13.2 via ge-0/0/2.0, label-switched-path vMX1-to-vMX3-59
                       to 10.100.13.2 via ge-0/0/2.0, label-switched-path vMX1-to-vMX3-60
                       to 10.100.13.2 via ge-0/0/2.0, label-switched-path vMX1-to-vMX3-61
                       to 10.100.13.2 via ge-0/0/2.0, label-switched-path vMX1-to-vMX3-62
                       to 10.100.13.2 via ge-0/0/2.0, label-switched-path vMX1-to-vMX3-63

jcluser@vMX1>

show vpls connections

各LSPに紐づいているVPLSの状態
vMX2(10.100.100.2)向け：65本中64本がUp,1本がVC-Dn
vMX3(10.100.100.3)向け：65本中64本がUp,1本がVC-Dn

jcluser@vMX1> show vpls connections 
Layer-2 VPN connections:

Legend for connection status (St)   
EI -- encapsulation invalid      NC -- interface encapsulation not CCC/TCC/VPLS
EM -- encapsulation mismatch     WE -- interface and instance encaps not same
VC-Dn -- Virtual circuit down    NP -- interface hardware not present 
CM -- control-word mismatch      -> -- only outbound connection is up
CN -- circuit not provisioned    <- -- only inbound connection is up
OR -- out of range               Up -- operational
OL -- no outgoing label          Dn -- down                      
LD -- local site signaled down   CF -- call admission control failure      
RD -- remote site signaled down  SC -- local and remote site ID collision
LN -- local site not designated  LM -- local site ID not minimum designated
RN -- remote site not designated RM -- remote site ID not minimum designated
XX -- unknown connection status  IL -- no incoming label
MM -- MTU mismatch               MI -- Mesh-Group ID not available
BK -- Backup connection         ST -- Standby connection
PF -- Profile parse failure      PB -- Profile busy
RS -- remote site standby SN -- Static Neighbor
LB -- Local site not best-site   RB -- Remote site not best-site
VM -- VLAN ID mismatch           HS -- Hot-standby Connection

Legend for interface status 
Up -- operational           
Dn -- down

Instance: rt00000
Edge protection: Not-Primary
  Local site: rt00000 (1)
    connection-site           Type  St     Time last up          # Up trans
    2                         rmt   Up     Dec 25 04:34:39 2023           1
      Remote PE: 10.100.100.2, Negotiated control-word: No
      Incoming label: 262154, Outgoing label: 262153
      Local interface: lsi.1048576, Status: Up, Encapsulation: VPLS
        Description: Intf - vpls rt00000 local site 1 remote site 2
      Flow Label Transmit: No, Flow Label Receive: No

Instance: rt00001
Edge protection: Not-Primary
  Local site: rt00001 (1)
    connection-site           Type  St     Time last up          # Up trans
    2                         rmt   Up     Dec 25 05:54:22 2023           1
      Remote PE: 10.100.100.2, Negotiated control-word: No
      Incoming label: 262162, Outgoing label: 262161
      Local interface: lsi.1048642, Status: Up, Encapsulation: VPLS
        Description: Intf - vpls rt00001 local site 1 remote site 2
      Flow Label Transmit: No, Flow Label Receive: No

~

Instance: rt00063
Edge protection: Not-Primary
  Local site: rt00063 (1)
    connection-site           Type  St     Time last up          # Up trans
    2                         rmt   Up     Dec 25 06:55:54 2023           1
      Remote PE: 10.100.100.2, Negotiated control-word: No
      Incoming label: 327682, Outgoing label: 327681
      Local interface: lsi.1048704, Status: Up, Encapsulation: VPLS
        Description: Intf - vpls rt00063 local site 1 remote site 2
      Flow Label Transmit: No, Flow Label Receive: No

Instance: rt00064
Edge protection: Not-Primary
  Local site: rt00064 (1)
    connection-site           Type  St     Time last up          # Up trans
    2                         rmt   VC-Dn  -----                          0
      Remote PE: 10.100.100.2, Negotiated control-word: No
      Incoming label: 327690, Outgoing label: 327689
      Local interface: lsi.1048705, Status: Up, Encapsulation: VPLS
        Description: Intf - vpls rt00064 local site 1 remote site 2
      Flow Label Transmit: No, Flow Label Receive: No


Instance: rt10000
Edge protection: Not-Primary
  Local site: rt10000 (1)
    connection-site           Type  St     Time last up          # Up trans
    3                         rmt   Up     Dec 25 05:43:09 2023           1
      Remote PE: 10.100.100.3, Negotiated control-word: No
      Incoming label: 327699, Outgoing label: 262145
      Local interface: lsi.1048577, Status: Up, Encapsulation: VPLS
        Description: Intf - vpls rt10000 local site 1 remote site 3
      Flow Label Transmit: No, Flow Label Receive: No

~

Instance: rt10063
Edge protection: Not-Primary
  Local site: rt10063 (1)
    connection-site           Type  St     Time last up          # Up trans
    3                         rmt   Up     Dec 25 06:55:54 2023           1
      Remote PE: 10.100.100.3, Negotiated control-word: No
      Incoming label: 393235, Outgoing label: 327681
      Local interface: lsi.1048640, Status: Up, Encapsulation: VPLS
        Description: Intf - vpls rt10063 local site 1 remote site 3
      Flow Label Transmit: No, Flow Label Receive: No

Instance: rt10064
Edge protection: Not-Primary
  Local site: rt10064 (1)
    connection-site           Type  St     Time last up          # Up trans
    3                         rmt   VC-Dn  -----                          0
      Remote PE: 10.100.100.3, Negotiated control-word: No
      Incoming label: 393243, Outgoing label: 327689
      Local interface: lsi.1048641, Status: Up, Encapsulation: VPLS
        Description: Intf - vpls rt10064 local site 1 remote site 3
      Flow Label Transmit: No, Flow Label Receive: No

jcluser@vMX1>

show route forwarding-table table "table-name"

rt00000,rt00063,rt10000,rt10063(Up) : lsi及びvMX2向けInterfaceが確認できる
rt00064,rt10064(VC-Dn) : lsiが確認できない

jcluser@vMX1> show route forwarding-table table rt00000 
Routing table: rt00000.vpls
VPLS:
Enabled protocols: ACKed by all peers, 
Destination        Type RtRef Next hop           Type Index    NhRef Netif
default            perm     0                    dscd      580     1
lsi.1048576        intf     0                    indr  1048574     4
                              10.100.12.2       Push 262153      598     2 ge-0/0/0.0
0x30004/51         user     0                    comp      602     2
ge-0/0/1.1000      intf     0                    ucst      592     3 ge-0/0/1.1000
0x30005/51         user     0                    comp      596     2
0x30003/51         user     0                    comp      597     2

jcluser@vMX1> show route forwarding-table table rt00063    
Routing table: rt00063.vpls
VPLS:
Enabled protocols: ACKed by all peers, 
Destination        Type RtRef Next hop           Type Index    NhRef Netif
default            perm     0                    dscd      879     1
lsi.1048704        intf     0                    indr  1048606     4
                              10.100.12.2       Push 327681     2115     2 ge-0/0/0.0
0x300c2/51         user     0                    comp     2307     2
ge-0/0/1.1063      intf     0                    ucst      921     3 ge-0/0/1.1063
0x300c3/51         user     0                    comp     1175     2
0x30044/51         user     0                    comp     1176     2

jcluser@vMX1> show route forwarding-table table rt00064    
Routing table: rt00064.vpls
VPLS:
Enabled protocols: ACKed by all peers, 
Destination        Type RtRef Next hop           Type Index    NhRef Netif
default            perm     0                    dscd      884     1
ge-0/0/1.1064      intf     0                    ucst      922     3 ge-0/0/1.1064
0x300c5/51         user     0                    comp     1177     2
0x30045/51         user     0                    comp     1178     2

jcluser@vMX1> show route forwarding-table table rt10000    
Routing table: rt10000.vpls
VPLS:
Enabled protocols: ACKed by all peers, 
Destination        Type RtRef Next hop           Type Index    NhRef Netif
default            perm     0                    dscd     1179     1
lsi.1048577        intf     0                    indr  1048575     4
                              10.100.13.2       Push 262145     1764     2 ge-0/0/2.0
0x30107/51         user     0                    comp     1787     2
ge-0/0/1.2000      intf     0                    ucst     1195     3 ge-0/0/1.2000
0x30108/51         user     0                    comp     1634     2
0x300c6/51         user     0                    comp     1635     2

jcluser@vMX1> show route forwarding-table table rt10063    
Routing table: rt10063.vpls
VPLS:
Enabled protocols: ACKed by all peers, 
Destination        Type RtRef Next hop           Type Index    NhRef Netif
default            perm     0                    dscd     1463     1
lsi.1048640        intf     0                    indr  1048654     4
                              10.100.13.2       Push 327681     2163     2 ge-0/0/2.0
0x30185/51         user     0                    comp     2355     2
ge-0/0/1.2063      intf     0                    ucst     1502     3 ge-0/0/1.2063
0x30186/51         user     0                    comp     1760     2
0x30105/51         user     0                    comp     1761     2

jcluser@vMX1> show route forwarding-table table rt10064    
Routing table: rt10064.vpls
VPLS:
Enabled protocols: ACKed by all peers, 
Destination        Type RtRef Next hop           Type Index    NhRef Netif
default            perm     0                    dscd     1468     1
ge-0/0/1.2064      intf     0                    ucst     1503     3 ge-0/0/1.2064
0x30188/51         user     0                    comp     1762     2
0x30106/51         user     0                    comp     1763     2

jcluser@vMX1>

まとめ

機器の導入やOSバージョン検証時には、スケール検証も大切であると体感したトラブルでした。

参考記事

https://www.juniper.net/documentation/us/en/software/junos/cli-reference/topics/ref/statement/maximum-ecmp-edit-chassis.html
https://jlabs.juniper.net/vlabs/
https://www.juniper.net/documentation/jp/ja/software/junos/vpn-l2/topics/example/vpls-bgp-configuring-detailed-solutions.html
https://www.infraexpert.com/infra/junosr01.html

その他

本検証で使ったコンフィグやshowコマンドの結果の完全版は別途下記の記事に記載しております。