LoginSignup
0
0
@s5uishida(Shigeru Ishida)in株式会社インテック

Open5GS EPCとZeroMQ対応srsRAN 4G UE/RANでeUPF(eBPF/XDP UPF)を試す

Posted at

本記事では、プライベートLTEを想定して、Open5GS EPCとeBPF/XDP対応UPFのeUPFを連携動作させるための簡単な構成について説明します。 eUPFのインストールと設定については、こちらを参照してください。

Open5GS CU分離EPCのシミュレーション携帯網の概要

今回、Open5GS EPCのC-Plane、eUPF、およびData Network Gatewayの簡単な構成について説明します。この構成はVirtualbox VMを使用して実装されることに注意して下さい。
このシミュレーション環境は以下の条件を満たす最小構成になります。

  • 1つのSGW-U/UPF(PGW-U)とData Network Gateway
  • 1つのUEと1つのAPN

構築したシミュレーション環境は以下の通りです。

network-overview.png

EPC、eUPFとUE/RANに使用しているOSSは以下の通りです。

各VMは以下の通りです。

VM SW & Role IP address OS CPU
(Min)		 Memory
(Min)		 HDD
(Min)
VM1 Open5GS EPC
C-Plane		 192.168.0.111/24 Ubuntu
22.04		 1 1GB 20GB
VM2 Open5GS EPC
U-Plane(SGW-U)		 192.168.0.112/24 Ubuntu
22.04		 1 1GB 20GB
VM-UP eUPF
U-Plane(PGW-U)		 192.168.0.151/24 Ubuntu 22.04 1 2GB 20GB
VM-DN Data Network
Gateway		 192.168.0.152/24 Ubuntu
22.04		 1 1GB 10GB
VM3 srsRAN 4G
ZMQ RAN (eNodeB)		 192.168.0.121/24 Ubuntu
22.04		 1 2GB 10GB
VM4 srsRAN 4G
ZMQ UE		 192.168.0.122/24 Ubuntu
22.04		 1 2GB 10GB

各VMのネットワークインターフェースは以下の通りです。

VM Device Network Adapter IP address Interface XDP
VM1 enp0s3 NAT(default) 10.0.2.15/24 (VM default NW) --
enp0s8 Bridged Adapter 192.168.0.111/24 (Mgmt NW) --
enp0s9 NAT Network 192.168.14.111/24 Sxb (N4 for 5GC) --
VM2 enp0s3 NAT(default) 10.0.2.15/24 (VM default NW) --
enp0s8 Bridged Adapter 192.168.0.112/24 (Mgmt NW) --
enp0s9 NAT Network 192.168.13.112/24 S1-U,S5u (N3 for 5GC) --
VM-UP enp0s3 NAT(default) 10.0.2.15/24 (VM default NW)
down 		--
enp0s8 Bridged Adapter 192.168.0.151/24 (Mgmt NW) --
enp0s9 NAT Network 192.168.13.151/24 S5u (N3 for 5GC) x
enp0s10 NAT Network 192.168.14.151/24 Sxb (N4 for 5GC) --
enp0s16 NAT Network 192.168.16.151/24 SGi (N6 for 5GC) x
VM-DN enp0s3 NAT(default) 10.0.2.15/24 (VM default NW) --
enp0s8 Bridged Adapter 192.168.0.152/24 (Mgmt NW) --
enp0s9 NAT Network 192.168.16.152/24 N6,
default GW
for VM-UP 		--
VM3 enp0s3 NAT(default) 10.0.2.15/24 (VM default NW) --
enp0s8 Bridged Adapter 192.168.0.121/24 (Mgmt NW) --
enp0s9 NAT Network 192.168.13.121/24 S1-U (N3 for 5GC) --
VM4 enp0s3 NAT(default) 10.0.2.15/24 (VM default NW) --
enp0s8 Bridged Adapter 192.168.0.122/24 (Mgmt NW) --

EPCのNF間のインターフェースの内、S1-U、S5u、Sxb、SGiについては、VirtualboxのNATネットワークで以下の通りに設定しました。

Network Name Network CIDR Note
N3 192.168.13.0/24 S1-U,S5u for EPC
N4 192.168.14.0/24 Sxb for EPC
N6 192.168.16.0/24 SGi for EPC

加入者情報は以下の通りです。(その他の項目はデフォルト値)
srsRAN 4GのUEの設定に合わせて、OPあるいはOPcを選択してください。

UE IMSI APN OP/OPc
UE 001010000000100 internet OPc

これらの情報の登録には、Open5GSが提供するWebUIを使用しました。なお、3GPP認証とキー生成機能(MILENAGE)の試験データとして、3GPP TS 35.208 "4.3 Test Sets"が公開されています。

PDNは以下の通りです。

PDN APN TUNnel interface of UE
10.45.0.0/16 internet tun_srsue

eNodeBの主な情報は以下の通りです。

MCC MNC TAC eNodeB ID Cell ID E-UTRAN Cell ID
001 01 1 0x19b 0x01 0x19b01

Open5GS EPC、eUPFとZMQ対応srsRAN 4G UE/RANの設定ファイルの変更点

Open5GS、eUPFとZMQ対応srsRAN 4G UE/RANのビルド方法は以下を参照して下さい。

Open5GS EPC C-Planeの設定ファイルの変更点

Open5GSでは、PFCPの接続先としてSGW-Uを選択するロジックに使用可能なパラメータは以下の通りです。PFCP(Packet Forwarding Control Protocol)とは、5GCでCU分離のために導入されたプロトコルです。

  • APN
  • TAC (Tracking Area Code)
  • e_CellID

なお、TACとe_CellIDはRANの設定項目です。話を簡単にするために、今回はAPNのみ使用しています。Open5GSでSGW-Uを選択するロジックについては、こちらを参照して下さい。

  • open5gs/install/etc/open5gs/mme.yaml
--- mme.yaml.orig       2023-12-05 03:49:16.000000000 +0900
+++ mme.yaml    2023-12-05 04:32:50.429030682 +0900
@@ -11,7 +11,7 @@
   freeDiameter: /root/open5gs/install/etc/freeDiameter/mme.conf
   s1ap:
     server:
-      - address: 127.0.0.2
+      - address: 192.168.0.111
   gtpc:
     server:
       - address: 127.0.0.2
@@ -26,14 +26,14 @@
         port: 9090
   gummei:
     - plmn_id:
-        mcc: 999
-        mnc: 70
+        mcc: 001
+        mnc: 01
       mme_gid: 2
       mme_code: 1
   tai:
     - plmn_id:
-        mcc: 999
-        mnc: 70
+        mcc: 001
+        mnc: 01
       tac: 1
   security:
     integrity_order : [ EIA2, EIA1, EIA0 ]
  • open5gs/install/etc/open5gs/sgwc.yaml
--- sgwc.yaml.orig      2023-12-05 03:49:16.000000000 +0900
+++ sgwc.yaml   2023-12-05 04:34:51.785067121 +0900
@@ -13,10 +13,11 @@
       - address: 127.0.0.3
   pfcp:
     server:
-      - address: 127.0.0.3
+      - address: 192.168.0.111
     client:
       sgwu:
-        - address: 127.0.0.6
+        - address: 192.168.0.112
+          apn: internet
 
 ################################################################################
 # GTP-C Server
  • open5gs/install/etc/open5gs/smf.yaml
--- smf.yaml.orig       2023-12-05 03:49:16.000000000 +0900
+++ smf.yaml    2023-12-05 04:45:38.004295776 +0900
@@ -8,39 +8,29 @@
 #    peer: 64
 
 smf:
-  sbi:
-    server:
-      - address: 127.0.0.4
-        port: 7777
-    client:
-#      nrf:
-#        - uri: http://127.0.0.10:7777
-      scp:
-        - uri: http://127.0.0.200:7777
   pfcp:
     server:
-      - address: 127.0.0.4
+      - address: 192.168.14.111
     client:
       upf:
-        - address: 127.0.0.7
+        - address: 192.168.14.151
+          dnn: internet
   gtpc:
     server:
       - address: 127.0.0.4
   gtpu:
     server:
-      - address: 127.0.0.4
+      - address: 192.168.14.111
   metrics:
     server:
       - address: 127.0.0.4
         port: 9090
   session:
     - subnet: 10.45.0.1/16
-    - subnet: 2001:db8:cafe::1/48
+      dnn: internet
   dns:
     - 8.8.8.8
     - 8.8.4.4
-    - 2001:4860:4860::8888
-    - 2001:4860:4860::8844
   mtu: 1400
 #  p-cscf:
 #    - 127.0.0.1

Open5GS EPC U-Planeの設定ファイルの変更点

  • open5gs/install/etc/open5gs/sgwu.yaml
--- sgwu.yaml.orig      2023-12-05 03:49:16.000000000 +0900
+++ sgwu.yaml   2023-12-05 04:55:03.425607103 +0900
@@ -10,13 +10,13 @@
 sgwu:
   pfcp:
     server:
-      - address: 127.0.0.6
+      - address: 192.168.0.112
     client:
 #      sgwc:    # SGW-U PFCP Client try to associate SGW-C PFCP Server
 #        - address: 127.0.0.3
   gtpu:
     server:
-      - address: 127.0.0.6
+      - address: 192.168.13.112
 
 ################################################################################
 # PFCP Server

eUPFの設定ファイルの変更点

元の設定ファイルは、こちらを参照して下さい。

  • eupf/config.yml
    元ファイルからの変更はありません。

ZMQ対応srsRAN 4G UE/RANの設定ファイルの変更点

RAN(eNodeB)の設定ファイルの変更点

  • srsRAN_4G/build/srsenb/enb.conf
--- enb.conf.example    2023-12-02 14:18:34.000000000 +0900
+++ enb.conf    2023-12-02 15:06:53.357631639 +0900
@@ -22,9 +22,9 @@
 enb_id = 0x19B
 mcc = 001
 mnc = 01
-mme_addr = 127.0.1.100
-gtp_bind_addr = 127.0.1.1
-s1c_bind_addr = 127.0.1.1
+mme_addr = 192.168.0.111
+gtp_bind_addr = 192.168.13.121
+s1c_bind_addr = 192.168.0.121
 s1c_bind_port = 0
 n_prb = 50
 #tm = 4
@@ -80,8 +80,8 @@
 #time_adv_nsamples = auto
 
 # Example for ZMQ-based operation with TCP transport for I/Q samples
-#device_name = zmq
-#device_args = fail_on_disconnect=true,tx_port=tcp://*:2000,rx_port=tcp://localhost:2001,id=enb,base_srate=23.04e6
+device_name = zmq
+device_args = fail_on_disconnect=true,tx_port=tcp://192.168.0.121:2000,rx_port=tcp://192.168.0.122:2001,id=enb,base_srate=23.04e6
 
 #####################################################################
 # Packet capture configuration
  • srsRAN_4G/build/srsenb/rr.conf
--- rr.conf.example     2023-12-02 14:18:34.000000000 +0900
+++ rr.conf     2023-05-02 11:52:54.000000000 +0900
@@ -55,7 +55,7 @@
   {
     // rf_port = 0;
     cell_id = 0x01;
-    tac = 0x0007;
+    tac = 0x0001;
     pci = 1;
     // root_seq_idx = 204;
     dl_earfcn = 3350;

UE(IMSI-001010000000100)の設定ファイルの変更点

  • srsRAN_4G/build/srsue/ue.conf
--- ue.conf.example     2023-05-02 10:51:20.000000000 +0900
+++ ue.conf     2023-05-02 12:01:28.000000000 +0900
@@ -42,8 +42,8 @@
 #continuous_tx     = auto
 
 # Example for ZMQ-based operation with TCP transport for I/Q samples
-#device_name = zmq
-#device_args = tx_port=tcp://*:2001,rx_port=tcp://localhost:2000,id=ue,base_srate=23.04e6
+device_name = zmq
+device_args = tx_port=tcp://192.168.0.122:2001,rx_port=tcp://192.168.0.121:2000,id=ue,base_srate=23.04e6
 
 #####################################################################
 # EUTRA RAT configuration
@@ -139,9 +139,9 @@
 [usim]
 mode = soft
 algo = milenage
-opc  = 63BFA50EE6523365FF14C1F45F88737D
-k    = 00112233445566778899aabbccddeeff
-imsi = 001010123456780
+opc  = E8ED289DEBA952E4283B54E88E6183CA
+k    = 465B5CE8B199B49FAA5F0A2EE238A6BC
+imsi = 001010000000100
 imei = 353490069873319
 #reader =
 #pin  = 1234
@@ -180,8 +180,8 @@
 #                      Supported: 0 - NULL, 1 - Snow3G, 2 - AES, 3 - ZUC
 #####################################################################
 [nas]
-#apn = internetinternet
-#apn_protocol = ipv4
+apn = internet
+apn_protocol = ipv4
 #user = srsuser
 #pass = srspass
 #force_imsi_attach = false

Open5GS EPC、eUPFとZMQ対応srsRAN 4G UE/RANのネットワーク設定

eUPFとData Network Gatewayのネットワーク設定

こちらとこちらを参照して下さい。

Open5GS、eUPFとUERANSIMのビルド

Open5GS、eUPFとUERANSIMのビルド方法は以下を参考にして下さい。

Open5GS 5GC C-PlaneマシンにMongoDBをインストールして下さい。なお、MongoDBのデータベースを簡単に確認するには、MongoDB Compassが便利です。

Open5GS EPC、eUPFとZMQ対応srsRAN 4G UE/RANを実行

最初にeUPFを起動し、次にOpen5GS EPC U-Plane(SGW-U)とC-Plane、ZMQ対応srsRAN 4G RANとUEを起動します。

eUPFを実行

こちらを参考にして下さい。

Open5GS EPC U-Planeを実行

次に、Open5GS EPC U-Planeを起動します。

./install/bin/open5gs-sgwud &

Open5GS EPC C-Planeを実行

次に、Open5GS EPC C-Planeを起動します。

./install/bin/open5gs-mmed &
./install/bin/open5gs-sgwcd &
./install/bin/open5gs-smfd &
./install/bin/open5gs-hssd &
./install/bin/open5gs-pcrfd &

eUPFとOpen5GS SMF間のPFCP Associationログは以下の通りです。

2023/12/05 20:21:41 INF Got Association Setup Request from: 192.168.14.111. 

2023/12/05 20:21:41 INF 
Association Setup Request:
  Node ID: 192.168.14.111
  Recovery Time: 2023-12-05 20:21:40 +0900 JST

2023/12/05 20:21:41 INF Saving new association: &{ID:192.168.14.111 Addr:192.168.14.111 NextSessionID:1 NextSequenceID:1 Sessions:map[] HeartbeatRetries:0 cancelRetries:<nil>}

ZMQ対応srsRAN 4G RAN(eNodeB)を起動

ZMQ対応srsRAN 4G RAN(eNodeB)を起動し、Open5GS EPCに接続します。

# cd srsRAN_4G/build/srsenb
# ./src/srsenb enb.conf
---  Software Radio Systems LTE eNodeB  ---

Reading configuration file enb.conf...

Built in Release mode using commit eea87b1d8 on branch master.

Opening 1 channels in RF device=zmq with args=fail_on_disconnect=true,tx_port=tcp://192.168.0.121:2000,rx_port=tcp://192.168.0.122:2001,id=enb,base_srate=23.04e6
Supported RF device list: zmq file
CHx base_srate=23.04e6
CHx id=enb
Current sample rate is 1.92 MHz with a base rate of 23.04 MHz (x12 decimation)
CH0 rx_port=tcp://192.168.0.122:2001
CH0 tx_port=tcp://192.168.0.121:2000
CH0 fail_on_disconnect=true

==== eNodeB started ===
Type <t> to view trace
Current sample rate is 11.52 MHz with a base rate of 23.04 MHz (x2 decimation)
Current sample rate is 11.52 MHz with a base rate of 23.04 MHz (x2 decimation)
Setting frequency: DL=2680.0 Mhz, UL=2560.0 MHz for cc_idx=0 nof_prb=50

The Open5GS C-Plane log when executed is as follows.

12/05 20:22:10.188: [mme] INFO: eNB-S1 accepted[192.168.0.121]:47130 in s1_path module (../src/mme/s1ap-sctp.c:114)
12/05 20:22:10.188: [mme] INFO: eNB-S1 accepted[192.168.0.121] in master_sm module (../src/mme/mme-sm.c:108)
12/05 20:22:10.188: [mme] INFO: [Added] Number of eNBs is now 1 (../src/mme/mme-context.c:2808)
12/05 20:22:10.188: [mme] INFO: eNB-S1[192.168.0.121] max_num_of_ostreams : 30 (../src/mme/mme-sm.c:150)

Open5GS C-Planeのログは以下の通りです。

12/05 20:22:10.188: [mme] INFO: eNB-S1 accepted[192.168.0.121]:47130 in s1_path module (../src/mme/s1ap-sctp.c:114)
12/05 20:22:10.188: [mme] INFO: eNB-S1 accepted[192.168.0.121] in master_sm module (../src/mme/mme-sm.c:108)
12/05 20:22:10.188: [mme] INFO: [Added] Number of eNBs is now 1 (../src/mme/mme-context.c:2808)
12/05 20:22:10.188: [mme] INFO: eNB-S1[192.168.0.121] max_num_of_ostreams : 30 (../src/mme/mme-sm.c:150)

ZMQ対応srsRAN 4G UEを起動

ZMQ対応srsRAN 4G UEを起動し、eNodeB、Open5GS EPCに接続します。

# cd srsRAN_4G/build/srsue
# ./src/srsue ue.conf
Reading configuration file ue.conf...

Built in Release mode using commit eea87b1d8 on branch master.

Opening 1 channels in RF device=zmq with args=tx_port=tcp://192.168.0.122:2001,rx_port=tcp://192.168.0.121:2000,id=ue,base_srate=23.04e6
Supported RF device list: zmq file
CHx base_srate=23.04e6
CHx id=ue
Current sample rate is 1.92 MHz with a base rate of 23.04 MHz (x12 decimation)
CH0 rx_port=tcp://192.168.0.121:2000
CH0 tx_port=tcp://192.168.0.122:2001
Waiting PHY to initialize ... done!
Attaching UE...
Current sample rate is 1.92 MHz with a base rate of 23.04 MHz (x12 decimation)
Current sample rate is 1.92 MHz with a base rate of 23.04 MHz (x12 decimation)
.
Found Cell:  Mode=FDD, PCI=1, PRB=50, Ports=1, CP=Normal, CFO=-0.2 KHz
Current sample rate is 11.52 MHz with a base rate of 23.04 MHz (x2 decimation)
Current sample rate is 11.52 MHz with a base rate of 23.04 MHz (x2 decimation)
Found PLMN:  Id=00101, TAC=1
Random Access Transmission: seq=38, tti=341, ra-rnti=0x2
RRC Connected
Random Access Complete.     c-rnti=0x46, ta=0
Network attach successful. IP: 10.45.0.2
 nTp) ((t) 5/12/2023 11:23:6 TZ:99

Open5GS C-Planeのログは以下の通りです。

12/05 20:23:05.989: [mme] INFO: InitialUEMessage (../src/mme/s1ap-handler.c:406)
12/05 20:23:05.989: [mme] INFO: [Added] Number of eNB-UEs is now 1 (../src/mme/mme-context.c:4684)
12/05 20:23:05.989: [mme] INFO: Unknown UE by S_TMSI[G:2,C:1,M_TMSI:0xc0000672] (../src/mme/s1ap-handler.c:486)
12/05 20:23:05.989: [mme] INFO:     ENB_UE_S1AP_ID[1] MME_UE_S1AP_ID[1] TAC[1] CellID[0x19b01] (../src/mme/s1ap-handler.c:582)
12/05 20:23:05.989: [mme] INFO: Unknown UE by GUTI[G:2,C:1,M_TMSI:0xc0000672] (../src/mme/mme-context.c:3538)
12/05 20:23:05.989: [mme] INFO: [Added] Number of MME-UEs is now 1 (../src/mme/mme-context.c:3340)
12/05 20:23:05.989: [emm] INFO: [] Attach request (../src/mme/emm-sm.c:412)
12/05 20:23:05.990: [emm] INFO:     GUTI[G:2,C:1,M_TMSI:0xc0000672] IMSI[Unknown IMSI] (../src/mme/emm-handler.c:236)
12/05 20:23:06.028: [emm] INFO: Identity response (../src/mme/emm-sm.c:382)
12/05 20:23:06.028: [emm] INFO:     IMSI[001010000000100] (../src/mme/emm-handler.c:425)
12/05 20:23:06.144: [mme] INFO: [Added] Number of MME-Sessions is now 1 (../src/mme/mme-context.c:4698)
12/05 20:23:06.223: [sgwc] INFO: [Added] Number of SGWC-UEs is now 1 (../src/sgwc/context.c:239)
12/05 20:23:06.223: [sgwc] INFO: [Added] Number of SGWC-Sessions is now 1 (../src/sgwc/context.c:881)
12/05 20:23:06.223: [sgwc] INFO: UE IMSI[001010000000100] APN[internet] (../src/sgwc/s11-handler.c:237)
12/05 20:23:06.224: [gtp] INFO: gtp_connect() [127.0.0.4]:2123 (../lib/gtp/path.c:60)
12/05 20:23:06.225: [smf] INFO: [Added] Number of SMF-UEs is now 1 (../src/smf/context.c:1019)
12/05 20:23:06.225: [smf] INFO: [Added] Number of SMF-Sessions is now 1 (../src/smf/context.c:3068)
12/05 20:23:06.225: [smf] INFO: UE IMSI[001010000000100] APN[internet] IPv4[10.45.0.2] IPv6[] (../src/smf/s5c-handler.c:275)
12/05 20:23:06.230: [gtp] INFO: gtp_connect() [192.168.13.151]:2152 (../lib/gtp/path.c:60)
12/05 20:23:06.540: [emm] INFO: [001010000000100] Attach complete (../src/mme/emm-sm.c:1298)
12/05 20:23:06.540: [emm] INFO:     IMSI[001010000000100] (../src/mme/emm-handler.c:274)
12/05 20:23:06.540: [emm] INFO:     UTC [2023-12-05T11:23:06] Timezone[0]/DST[0] (../src/mme/emm-handler.c:281)
12/05 20:23:06.540: [emm] INFO:     LOCAL [2023-12-05T20:23:06] Timezone[32400]/DST[0] (../src/mme/emm-handler.c:285)

Open5GS U-Planeのログは以下の通りです。

12/05 20:23:06.187: [sgwu] INFO: UE F-SEID[UP:0x970 CP:0x2a7] (../src/sgwu/context.c:171)
12/05 20:23:06.187: [sgwu] INFO: [Added] Number of SGWU-Sessions is now 1 (../src/sgwu/context.c:176)
12/05 20:23:06.194: [gtp] INFO: gtp_connect() [192.168.13.151]:2152 (../lib/gtp/path.c:60)
12/05 20:23:06.504: [gtp] INFO: gtp_connect() [192.168.13.121]:2152 (../lib/gtp/path.c:60)

eUPFとOpen5GS SMF(PGW-C)のPFCP associationのステータスは次のように表示されます。

2023/12/05 20:23:06 INF Got Session Establishment Request from: 192.168.14.111.
2023/12/05 20:23:06 INF 
Session Establishment Request:
  CreatePDR ID: 1 
    FAR ID: 1 
    QER ID: 1 
    Source Interface: 1 
    UE IPv4 Address: 10.45.0.2 
  CreatePDR ID: 2 
    Outer Header Removal: 0 
    FAR ID: 2 
    QER ID: 1 
    Source Interface: 0 
    TEID: 0 
    Ipv4: <nil> 
    Ipv6: <nil> 
    UE IPv4 Address: 10.45.0.2 
  CreatePDR ID: 3 
    Outer Header Removal: 0 
    FAR ID: 1 
    Source Interface: 3 
    TEID: 0 
    Ipv4: <nil> 
    Ipv6: <nil> 
  CreatePDR ID: 4 
    Outer Header Removal: 0 
    FAR ID: 3 
    Source Interface: 0 
    TEID: 0 
    Ipv4: <nil> 
    Ipv6: <nil> 
    SDF Filter: permit out 58 from ff02::2/128 to assigned 
  CreateFAR ID: 1 
    Apply Action: [2 0] 
    Forwarding Parameters:
      Network Instance:internet 
      Outer Header Creation: &{OuterHeaderCreationDescription:256 TEID:60524 IPv4Address:192.168.13.112 IPv6Address:<nil> PortNumber:0 CTag:0 STag:0} 
  CreateFAR ID: 2 
    Apply Action: [2 0] 
    Forwarding Parameters:
      Network Instance:internet 
  CreateFAR ID: 3 
    Apply Action: [2 0] 
    Forwarding Parameters:
      Network Instance:internet 
      Outer Header Creation: &{OuterHeaderCreationDescription:256 TEID:1 IPv4Address:192.168.14.111 IPv6Address:<nil> PortNumber:0 CTag:0 STag:0} 
  CreateQER ID: 1 
    Gate Status DL: 0 
    Gate Status UL: 0 
    Max Bitrate DL: 1000000 
    Max Bitrate UL: 1000000 
  CreateBAR ID: 1

2023/12/05 20:23:06 INF Saving FAR info to session: 1, {Action:2 OuterHeaderCreation:1 Teid:60524 RemoteIP:1879943360 LocalIP:2534254784 TransportLevelMarking:0}
2023/12/05 20:23:06 INF WARN: No OuterHeaderCreation
2023/12/05 20:23:06 INF Saving FAR info to session: 2, {Action:2 OuterHeaderCreation:0 Teid:0 RemoteIP:0 LocalIP:2534254784 TransportLevelMarking:0}
2023/12/05 20:23:06 INF Saving FAR info to session: 3, {Action:2 OuterHeaderCreation:1 Teid:1 RemoteIP:1863231680 LocalIP:2534254784 TransportLevelMarking:0}
2023/12/05 20:23:06 INF Saving QER info to session: 1, {GateStatusUL:0 GateStatusDL:0 Qfi:0 MaxBitrateUL:1000000000 MaxBitrateDL:1000000000 StartUL:0 StartDL:0}
2023/12/05 20:23:06 Matched groups: [permit out 58 from ff02::2/128 to assigned 58 ff02::2 128  assigned  ]
2023/12/05 20:23:06 INF Session Establishment Request from 192.168.14.111 accepted.

VM4(UE)でのip addr showは以下の通りです。TUNnelインターフェースtun_srsueは次のように作成され、IPアドレス10.45.0.2がUEに割り当てられたことが分かります。

5: tun_srsue: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UNKNOWN group default qlen 500
    link/none 
    inet 10.45.0.2/24 scope global tun_srsue
       valid_lft forever preferred_lft forever

google.comにpingする

UEのTUNインターフェースを指定して、pingしてみます。

DN=10.45.0.0/16を経由する場合

VM-DNでtcpdumpを実行し、パケットがSGiインターフェースenp0s9を通過することを確認します。

  • VM4(UE)でping google.com 
# ping google.com -I tun_srsue -n
PING google.com (142.251.222.14) from 10.45.0.2 tun_srsue: 56(84) bytes of data.
64 bytes from 142.251.222.14: icmp_seq=1 ttl=61 time=101 ms
64 bytes from 142.251.222.14: icmp_seq=2 ttl=61 time=73.8 ms
64 bytes from 142.251.222.14: icmp_seq=3 ttl=61 time=58.0 ms
  • VM-DNでtcpdumpを実行
# tcpdump -i enp0s9 -n
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on enp0s9, link-type EN10MB (Ethernet), snapshot length 262144 bytes
20:25:50.944272 IP 10.45.0.2 > 142.251.222.14: ICMP echo request, id 3, seq 1, length 64
20:25:50.973107 IP 142.251.222.14 > 10.45.0.2: ICMP echo reply, id 3, seq 1, length 64
20:25:51.933226 IP 10.45.0.2 > 142.251.222.14: ICMP echo request, id 3, seq 2, length 64
20:25:51.955657 IP 142.251.222.14 > 10.45.0.2: ICMP echo reply, id 3, seq 2, length 64
20:25:52.919980 IP 10.45.0.2 > 142.251.222.14: ICMP echo request, id 3, seq 3, length 64
20:25:52.938727 IP 142.251.222.14 > 10.45.0.2: ICMP echo reply, id 3, seq 3, length 64
  • VM-UPで/sys/kernel/debug/tracing/trace_pipeを確認
# cat /sys/kernel/debug/tracing/trace_pipe
...
          <idle>-0       [000] d.s31  3184.066978: bpf_trace_printk: upf: gtp-u received
          <idle>-0       [000] d.s31  3184.066982: bpf_trace_printk: SDF: filter protocol: 4
          <idle>-0       [000] d.s31  3184.066985: bpf_trace_printk: SDF: filter source ip: 0.0.0.2, destination ip: 0.0.0.0
          <idle>-0       [000] d.s31  3184.066987: bpf_trace_printk: SDF: filter source ip mask: 255.255.255.255, destination ip mask: 0.0.0.0
          <idle>-0       [000] d.s31  3184.066988: bpf_trace_printk: SDF: filter source port lower bound: 0, source port upper bound: 65535
          <idle>-0       [000] d.s31  3184.066989: bpf_trace_printk: SDF: filter destination port lower bound: 0, destination port upper bound: 65535
          <idle>-0       [000] d.s31  3184.066989: bpf_trace_printk: SDF: packet protocol: 0
          <idle>-0       [000] d.s31  3184.066990: bpf_trace_printk: SDF: packet source ip: 10.45.0.2, destination ip: 142.251.222.14
          <idle>-0       [000] d.s31  3184.066991: bpf_trace_printk: SDF: packet source port: 0, destination port: 0
          <idle>-0       [000] d.s31  3184.066992: bpf_trace_printk: upf: sdf filter doesn't match teid:1
          <idle>-0       [000] d.s31  3184.066993: bpf_trace_printk: upf: far:1 action:2 outer_header_creation:0
          <idle>-0       [000] d.s31  3184.066994: bpf_trace_printk: upf: qer:0 gate_status:0 mbr:1000000000
          <idle>-0       [000] d.s31  3184.066995: bpf_trace_printk: upf: session for teid:1 far:1 outer_header_removal:0
          <idle>-0       [000] d.s31  3184.067003: bpf_trace_printk: upf: bpf_fib_lookup 10.45.0.2 -> 142.251.222.14: nexthop: 192.168.16.152
          <idle>-0       [000] d.s31  3184.096132: bpf_trace_printk: upf: downlink session for ip:10.45.0.2  far:0 action:2
          <idle>-0       [000] d.s31  3184.096135: bpf_trace_printk: upf: qer:0 gate_status:0 mbr:1000000000
          <idle>-0       [000] d.s31  3184.096136: bpf_trace_printk: upf: use mapping 10.45.0.2 -> TEID:60524
          <idle>-0       [000] d.s31  3184.096138: bpf_trace_printk: upf: send gtp pdu 192.168.13.151 -> 192.168.13.112
          <idle>-0       [000] d.s31  3184.096146: bpf_trace_printk: upf: bpf_fib_lookup 192.168.13.151 -> 192.168.13.112: nexthop: 192.168.13.112
          <idle>-0       [000] d.s31  3185.055908: bpf_trace_printk: upf: gtp-u received
          <idle>-0       [000] d.s31  3185.055912: bpf_trace_printk: SDF: filter protocol: 4
          <idle>-0       [000] d.s31  3185.055915: bpf_trace_printk: SDF: filter source ip: 0.0.0.2, destination ip: 0.0.0.0
          <idle>-0       [000] d.s31  3185.055916: bpf_trace_printk: SDF: filter source ip mask: 255.255.255.255, destination ip mask: 0.0.0.0
          <idle>-0       [000] d.s31  3185.055917: bpf_trace_printk: SDF: filter source port lower bound: 0, source port upper bound: 65535
          <idle>-0       [000] d.s31  3185.055918: bpf_trace_printk: SDF: filter destination port lower bound: 0, destination port upper bound: 65535
          <idle>-0       [000] d.s31  3185.055918: bpf_trace_printk: SDF: packet protocol: 0
          <idle>-0       [000] d.s31  3185.055938: bpf_trace_printk: SDF: packet source ip: 10.45.0.2, destination ip: 142.251.222.14
          <idle>-0       [000] d.s31  3185.055939: bpf_trace_printk: SDF: packet source port: 0, destination port: 0
          <idle>-0       [000] d.s31  3185.055940: bpf_trace_printk: upf: sdf filter doesn't match teid:1
          <idle>-0       [000] d.s31  3185.055941: bpf_trace_printk: upf: far:1 action:2 outer_header_creation:0
          <idle>-0       [000] d.s31  3185.055942: bpf_trace_printk: upf: qer:0 gate_status:0 mbr:1000000000
          <idle>-0       [000] d.s31  3185.055943: bpf_trace_printk: upf: session for teid:1 far:1 outer_header_removal:0
          <idle>-0       [000] d.s31  3185.055950: bpf_trace_printk: upf: bpf_fib_lookup 10.45.0.2 -> 142.251.222.14: nexthop: 192.168.16.152
          <idle>-0       [000] d.s31  3185.078676: bpf_trace_printk: upf: downlink session for ip:10.45.0.2  far:0 action:2
          <idle>-0       [000] d.s31  3185.078680: bpf_trace_printk: upf: qer:0 gate_status:0 mbr:1000000000
          <idle>-0       [000] d.s31  3185.078681: bpf_trace_printk: upf: use mapping 10.45.0.2 -> TEID:60524
          <idle>-0       [000] d.s31  3185.078683: bpf_trace_printk: upf: send gtp pdu 192.168.13.151 -> 192.168.13.112
          <idle>-0       [000] d.s31  3185.078691: bpf_trace_printk: upf: bpf_fib_lookup 192.168.13.151 -> 192.168.13.112: nexthop: 192.168.13.112
          <idle>-0       [000] d.s31  3186.042681: bpf_trace_printk: upf: gtp-u received
          <idle>-0       [000] d.s31  3186.042685: bpf_trace_printk: SDF: filter protocol: 4
          <idle>-0       [000] d.s31  3186.042688: bpf_trace_printk: SDF: filter source ip: 0.0.0.2, destination ip: 0.0.0.0
          <idle>-0       [000] d.s31  3186.042690: bpf_trace_printk: SDF: filter source ip mask: 255.255.255.255, destination ip mask: 0.0.0.0
          <idle>-0       [000] d.s31  3186.042691: bpf_trace_printk: SDF: filter source port lower bound: 0, source port upper bound: 65535
          <idle>-0       [000] d.s31  3186.042692: bpf_trace_printk: SDF: filter destination port lower bound: 0, destination port upper bound: 65535
          <idle>-0       [000] d.s31  3186.042692: bpf_trace_printk: SDF: packet protocol: 0
          <idle>-0       [000] d.s31  3186.042693: bpf_trace_printk: SDF: packet source ip: 10.45.0.2, destination ip: 142.251.222.14
          <idle>-0       [000] d.s31  3186.042694: bpf_trace_printk: SDF: packet source port: 0, destination port: 0
          <idle>-0       [000] d.s31  3186.042695: bpf_trace_printk: upf: sdf filter doesn't match teid:1
          <idle>-0       [000] d.s31  3186.042696: bpf_trace_printk: upf: far:1 action:2 outer_header_creation:0
          <idle>-0       [000] d.s31  3186.042697: bpf_trace_printk: upf: qer:0 gate_status:0 mbr:1000000000
          <idle>-0       [000] d.s31  3186.042698: bpf_trace_printk: upf: session for teid:1 far:1 outer_header_removal:0
          <idle>-0       [000] d.s31  3186.042705: bpf_trace_printk: upf: bpf_fib_lookup 10.45.0.2 -> 142.251.222.14: nexthop: 192.168.16.152
          <idle>-0       [000] d.s31  3186.061671: bpf_trace_printk: upf: downlink session for ip:10.45.0.2  far:0 action:2
          <idle>-0       [000] d.s31  3186.061674: bpf_trace_printk: upf: qer:0 gate_status:0 mbr:1000000000
          <idle>-0       [000] d.s31  3186.061676: bpf_trace_printk: upf: use mapping 10.45.0.2 -> TEID:60524
          <idle>-0       [000] d.s31  3186.061677: bpf_trace_printk: upf: send gtp pdu 192.168.13.151 -> 192.168.13.112
          <idle>-0       [000] d.s31  3186.061684: bpf_trace_printk: upf: bpf_fib_lookup 192.168.13.151 -> 192.168.13.112: nexthop: 192.168.13.112
...

ping以外にも、以下のようにcurlでTUNnelインターフェースを指定してWebにアクセスしてみることもできます。

  • curl google.com on VM4 (UE)
# curl --interface tun_srsue google.com
<HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8">
<TITLE>301 Moved</TITLE></HEAD><BODY>
<H1>301 Moved</H1>
The document has moved
<A HREF="http://www.google.com/">here</A>.
</BODY></HTML>
  • VM-DNでtcpdumpを実行
20:27:46.955340 IP 10.45.0.2.54964 > 172.217.161.78.80: Flags [S], seq 718929327, win 64240, options [mss 1460,sackOK,TS val 1478275073 ecr 0,nop,wscale 7], length 0
20:27:46.971398 IP 172.217.161.78.80 > 10.45.0.2.54964: Flags [S.], seq 192001, ack 718929328, win 65535, options [mss 1460], length 0
20:27:47.095296 IP 10.45.0.2.54964 > 172.217.161.78.80: Flags [.], ack 1, win 64240, length 0
20:27:47.095296 IP 10.45.0.2.54964 > 172.217.161.78.80: Flags [P.], seq 1:75, ack 1, win 64240, length 74: HTTP: GET / HTTP/1.1
20:27:47.095530 IP 172.217.161.78.80 > 10.45.0.2.54964: Flags [.], ack 75, win 65535, length 0
20:27:47.173318 IP 172.217.161.78.80 > 10.45.0.2.54964: Flags [P.], seq 1:774, ack 75, win 65535, length 773: HTTP: HTTP/1.1 301 Moved Permanently
20:27:47.234283 IP 10.45.0.2.54964 > 172.217.161.78.80: Flags [.], ack 774, win 63467, length 0
20:27:47.234283 IP 10.45.0.2.54964 > 172.217.161.78.80: Flags [F.], seq 75, ack 774, win 63467, length 0
20:27:47.234527 IP 172.217.161.78.80 > 10.45.0.2.54964: Flags [.], ack 76, win 65535, length 0
20:27:47.264013 IP 172.217.161.78.80 > 10.45.0.2.54964: Flags [F.], seq 774, ack 76, win 65535, length 0
20:27:47.332814 IP 10.45.0.2.54964 > 172.217.161.78.80: Flags [.], ack 775, win 63467, length 0

以上で、eBPF/XDP対応UPFのeUPFのPDNに接続し、PDNに任意のパケットを流せるようになり、プライベートLTEを想定したEPCのシミュレーション携帯網として、MECを試す環境を構築できました。

最後に

eBPF/XDP対応UPFのeUPFは、U-Planeのデータネットワークを高速化する技術と期待しています。本記事では、Virtualbox VM上の設定例を書いてみましたが、高性能な物理NICを搭載したサーバを設定する際のご参考になれれば幸いです。

最後に、元記事はGithubに書いたものです。

主な変更履歴

  • [2024.02.22] 初版。
0
0
0

Register as a new user and use Qiita more conveniently

  1. You get articles that match your needs
  2. You can efficiently read back useful information
  3. You can use dark theme
What you can do with signing up
Sign upLogin
0
0