Go to Qiita Advent Calendar 2024 Top
LoginSignup
5
3

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?

More than 3 years have passed since last update.

@ishizaghi

Lagopus Routerで iOS/Android VPN接続できる IPsecサーバーを構築する

Last updated at Posted at 2020-01-20

#イメージ図
lagopus-router-ipsec-network.png

#目的
Lagopus Routerに IPsec終端機能が実装されているので、スマホ端末を接続できるような IPsec/XAUTH PSKサーバーを構築してみました。

#Lagopus Routerとは
O3プロジェクトで開発され、OSS公開されている高速パケット処理が可能なソフトウェアルーター。
I/OにDPDKを採用し、スペックシートでは IPsecスループット 10Gbps(!)、 2,048セッション(!!) と記載されている。

詳しくは、リンク先を参照ください。[*1][*2]

#セットアップ環境
使用したOS, ソフトウェア

  • OS / HWスペック

  • Ubuntu 16.04.3 LTS

  • CPU 4コア, Memory 4GB, NIC * 2

  • Software

  • Lagopus Router v19.07.1

  • DPDK 18.11.5

  • Go v1.10.4

  • vsw (Dataplane of Lagopus Router)

  • openconfigd

  • iked

  • strongSwan

  • Zebra/VRF library

#インストール手順
インストール手順は Lagopus software router book [*3] を参考におこないます。

##1, DPDKのインストール
vsw (Dataplane of Lagopus router)は DPDK 18.11.1以上が必要のため、現時点のDPDK 18系最新の DPDK 18.11.5をインストールする。

$ wget http://static.dpdk.org/rel/dpdk-18.11.5.tar.xz
$ tar xvf dpdk-18.11.5.tar.xz
$ cd dpdk-stable-18.11.5/
~dpdk-stable-18.11.5$ cp config/common_base config/common_base.original
~dpdk-stable-18.11.5$ vi config/common_base
CONFIG_RTE_BUILD_SHARED_LIB=y
CONFIG_RTE_IXGBE_INC_VECTOR=n
CONFIG_RTE_LIBRTE_PMD_BOND=n
CONFIG_RTE_LIBRTE_PMD_OPENSSL=y

~dpdk-stable-18.11.5$ make T=x86_64-native-linuxapp-gcc config
~dpdk-stable-18.11.5$ make
~dpdk-stable-18.11.5$ sudo make instlal

##2, Goのインストール

$ sudo apt install golang-go go-dep
$ vi ~/.bashrc
export GOPATH=$HOME/go
export PATH=$GOPATH/bin:$PATH

~$ mkdir -p go/src
~$ mkdir -p go/pkg
~$ mkdir -p go/bin

##3, openconfigd, cliのインストール
openconfigd/README.md を参考にすすめる

$ go get github.com/coreswitch/openconfigd/openconfigd
$ go get github.com/coreswitch/openconfigd/cli_command
$ cd $GOPATH/src/github.com/coreswitch/openconfigd/cli
$ ./configure
$ make
$ sudo make install
$ cd $GOPATH/src/github.com/coreswitch/openconfigd/bash_completion.d
$ sudo cp cli /etc/bash_completion.d/

##4, vsw(Dataplane of Lagopus router)のインストール
lagopus/vsw/README.md を参考にしてすすめる

$ mkdir -p go/src/github.com/lagopus
$ cd go/src/github.com/lagopus
$ ~/go/src/github.com/lagopus$ git clone https://github.com/lagopus/vsw.git

% git clone https://github.com/lagopus/utils ~/utils
% cd ~/utils/
% ./configure
% make
% sudo make install

~/go/src/github.com/lagopus/vsw$ make 
~/go/src/github.com/lagopus/vsw$ sudo make install 
~/go/src/github.com/lagopus/vsw$ go install

##5, Lagopus Routerのインストール

$ cd ~/go/src/github.com/lagopus/
~/go/src/github.com/lagopus$ git clone https://github.com/lagopus/lagopus-router.git

##6, Lagopus iked/strongSwanのインストール

必要ディレクトリの作成
~$ sudo mkdir -p /usr/local/ipsec/etc/ipsec.d/run
~$ sudo mkdir -p /usr/local/ipsec/etc/ipsec.d/cacerts
~$ sudo mkdir -p /usr/local/ipsec/etc/ipsec.d/certs
~$ sudo mkdir -p /usr/local/ipsec/etc/ipsec.d/private

~$ git clone https://github.com/lagopus/iked.git
~$ cd iked/strongswan
~iked/strongswan$ ./autogen.sh
~iked/strongswan$ ./configure --enable-openssl --enable-lagopus-pfkey --enable-xauth-eap --enable-eap-radius --sysconfdir=/usr/local/ipsec/etc --with-piddir=/usr/local/ipsec/etc/ipsec.d/run
~iked/strongswan$ make, sudo make install

##7, Lagopus iked/ipsec_vrfのインストール

~$ cd iked/ipsec_vrf
~iked/ipsec_vrf$ sudo make install

##8, ZebraのVRF Library(vrf_socket.so)のインストール

~$ git clone https://github.com/coreswitch/zebra
~$ cd zebra/tools/vrf
~zebra/tools/vrf$ make
~zebra/tools/vrf$ sudo mkdir -p /usr/local/ipsec/lib
~zebra/tools/vrf$ sudo cp vrf_socket.so /usr/local/ipsec/lib

#設定方法
設定ファイルをそれぞれ以下のパスに作成します。

  • openconfigd
  • /usr/local/etc/openconfigd.conf
  • vsw
  • /usr/local/etc/vsw.conf
  • iked/strongswan
  • /usr/local/ipsec/etc/strongswan.conf
  • /usr/local/ipsec/etc/strongswan.d/charon/socket-default.conf
  • ipsec
  • /usr/local/ipsec/etc/ipsec.conf
  • /usr/local/ipsec/etc/ipsec.secrets
  • vrf (vrf1を作成する場合)
  • /usr/local/ipsec/vrf1/strongswan.d/charon.conf
  • /usr/local/ipsec/vrf1/strongswan.d/charon/lagopus-pfkey.conf

##1, openconfigd 設定
事前に設定ファイルの作成は必要なし。後述の openconfigdを起動後 cliから設定していく。

##2, vsw 設定
以下の設定ファイルを作成編集する。

  • /usr/local/etc/vsw.conf

デフォルト設定から以下の部分を変更する。 (デフォルト設定は $GOPATH/src/github.com/lagopus/vsw/vsw.conf)

  • [dpdk] core_mask をCPUコア数に合わせて変更
  • [dpdk] vdevs=["crypto_openssl"], [agents] enable=[ "ipsec" ], [router] rr_process_mode = "enable" を追加。
/usr/local/etc/vsw.conf
# VSW configuration file
[agents]
enable = [ "netlink", "config", "debugsh", "ipsec" ]

# Logging settings
[logging]
log = "stdout" # "file", "stdout", or "syslog"

# DPDK configuration section
[dpdk]
core_mask = 0x0e
# core_list = "1,2,3,4,5,6,7"
memory_channel = 2
pmd_path = "/usr/local/lib"
num_elements = 131072
cache_size = 256
vdevs=["crypto_openssl"] 

# Openconfigd section
[openconfig]
server_host = "localhost"       # Openconfigd server host
server_port = 2650              # Openconfigd server port
listen_port = 2653              # Port to listen for show command

# ethdev configuration section
[ethdev]
rx_core = 2     # Slave core to use for RX
tx_core = 3     # Slave core to use for TX

# Set force_linearize to true, if the link device doesn't
# support multi-segment mbuf.
force_linearize = false

# bridge configuration section
[bridge]
core = 2
# RIF configuration section
[rif]
core = 3

# tunnel configuration section
[tunnel]
  # IPsec tunnel
  [tunnel.ipsec]
  # Enable/Disable CPU core binding for IPsec.
  core_bind = false
  # Set CPU core mask.
  # It is better not to overlap with dpdk.core_mask.
  inbound_core_mask = 0x10  # core ID: 4
  outbound_core_mask = 0x20 # core ID: 5

  # IP in IP tunnel
  [tunnel.ipip]
  inbound_core = 2
  outbound_core = 3

  # GRE tunnel
  [tunnel.gre]
  inbound_core = 2
  outbound_core = 3

  # L2GRE tunnel
  [tunnel.l2gre]
  inbound_core = 2
  outbound_core = 3

  # VXLAN tunnel
  [tunnel.vxlan]
  inbound_core = 2
  outbound_core = 3
  aging_time = 300

# router configuration section
#
# rr_process_mode is processing mode for IPv4 options that require
# to record a route. The following modes are available.
#   * enable: process and forward packets as per RFC791
#   * disable: drops packets
#   * ignore: forward packets as if they did not contain the options
[router]
core = 3
rr_process_mode = "enable"

##3, strongSwan 設定
以下の設定ファイルを作成編集する。

  • /usr/local/ipsec/strongswan.conf
  • /usr/local/ipsec/etc/strongswan.d/charon/socket-default.conf
/usr/local/ipsec/strongswan.conf
# strongswan.conf - strongSwan configuration file
#
# Refer to the strongswan.conf(5) manpage for details
#
# Configuration changes should be made in the included files

charon {
        load_modular = yes
        plugins {
                include strongswan.d/charon/*.conf
        }
}

include strongswan.d/*.conf
/usr/local/ipsec/etc/strongswan.d/charon/socket-default.conf
socket-default {

    # Firewall mark to set on outbound packets.
    # fwmark =

    # Whether to load the plugin. Can also be an integer to increase the
    # priority of this plugin.
    load = yes

    # Set source address on outbound packets, if possible.
    # set_source = yes

    # Force sending interface on outbound packets, if possible.
    set_sourceif = yes    # ← yesに変更

    # Listen on IPv4, if possible.
    # use_ipv4 = yes

    # Listen on IPv6, if possible.
    # use_ipv6 = yes

}

##4, vrf(vrf1)向け strongSwan/ipsec 設定
/usr/local/ipsec/etc を /usr/local/ipsec/vrf1 へコピーしてから、以下の設定ファイルを作成編集する。

  • /usr/local/ipsec/vrf1/ipsec.conf
  • /usr/local/ipsec/vrf1/ipsec.secrets
  • /usr/local/ipsec/vrf1/strongswan.d/charon.conf
  • /usr/local/ipsec/vrf1/strongswan.d/charon/lagopus-pfkey.conf
/usr/local/ipsec/vrf1/ipsec.conf
#ipsec.conf - strongSwan IPsec configuration file
config setup
#         charondebug="cfg 2, dmn 2, ike 2, net 2"

conn ipsec-xauth
    auto=add
    dpdaction=clear
    keyexchange=ikev1
    ike=aes128-sha1-modp1024
    esp=aes128-sha1
    left=%any
    leftauth=psk
    leftsubnet=0.0.0.0/0
    leftid=203.0.113.1
    right=%any
    rightsourceip=192.0.2.0/24
    rightdns=8.8.8.8
    dpdaction=clear
    rightauth=psk
    rightauth2=xauth
/usr/local/ipsec/vrf1/ipsec.secrets
: PSK "lagopus"
test1 : XAUTH "test1-lagopus"
test2 : XAUTH "test2-lagopus"
/usr/local/ipsec/vrf1/strongswan.d/charon.conf
..
    # A comma-separated list of network interfaces that should be used by
    # charon. All other interfaces are ignored.
    # interfaces_use =
    interfaces_use = if0-0    # 鍵交換に使うインターフェースを追加
..
/usr/local/ipsec/vrf1/strongswan.d/charon/lagopus-pfkey.conf
lagopus-pfkey {

    # Size of the receive buffer for the event socket (0 for default size).
    # events_buffer_size = 0

    # Whether to load the plugin. Can also be an integer to increase the
    # priority of this plugin.
    load = yes

    # UNIX domain socket path for lagopus
    socket_path = /var/tmp/lagopus-vrf1.sock # ← 変更

}

#起動方法
##事前準備
DPDKのセットアップ、NICバインドをおこなう。

###1, DPDKのセットアップ
DPDKを使うためにはHugepagesの有効化をおこなう。

Hugepagesの設定 & reboot

$ sudo vi /etc/sysctl.conf
vm.nr_hugepages = 1024
$ sudo mkdir -p /mnt/huge
$ sudo vi /etc/fstab
nodev /mnt/huge hugetlbfs defaults 0 0
$ sudo reboot
..

Hugepagesの有効化確認

$ grep -i "HugePages" /proc/meminfo
AnonHugePages:         0 kB
ShmemHugePages:        0 kB
HugePages_Total:    1024
HugePages_Free:     1024
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB

$ mount | grep huge
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime,pagesize=2M)
nodev on /mnt/huge type hugetlbfs (rw,relatime,pagesize=2M)

###2, NICを DPDKへバインド

  • kernel moduleの有効化 (uio, DPDKのigp_uio.ko)
  • if downさせてからバインドする
~dpdk-stable-18.11.5$ sudo -s
~dpdk-stable-18.11.5# modprobe uio
~dpdk-stable-18.11.5# insmod build/kmod/igb_uio.ko
~dpdk-stable-18.11.5# lsmod | grep uio
igb_uio                16384  2
uio                    20480  5 igb_uio

~dpdk-stable-18.11.5# ip link set ens160 down
~dpdk-stable-18.11.5# ip link set ens192 down
~dpdk-stable-18.11.5# ./usertools/dpdk-devbind.py --bind=igb_uio ens160 ens192

###3, NICがDPDKに適用されたかを確認

~dpdk-stable-18.11.5# ./usertools/dpdk-devbind.py -s

Network devices using DPDK-compatible driver
============================================
0000:03:00.0 'VMXNET3 Ethernet Controller 07b0' drv=igb_uio unused=vmxnet3
0000:0b:00.0 'VMXNET3 Ethernet Controller 07b0' drv=igb_uio unused=vmxnet3

Network devices using kernel driver
===================================
0000:13:00.0 'VMXNET3 Ethernet Controller 07b0' if=ens224 drv=vmxnet3 unused=igb_uio *Active*

##起動
openconfigdの起動 → vswの起動 → cliでinterface, vrf設定 → ipsec daemonの順番で起動する。
注意事項は、

  • vsw起動の前に Openconfigdが動いていないと設定が読み込めない。
  • ipsec daemonを起動する前に、interface, vrf設定がされていないと、起動できないためです。

###1, openconfigdの起動

$ cd ~/go/src/github.com/lagopus/lagopus-router/yang
~/go/src/github.com/lagopus/lagopus-router/yang$ sudo openconfigd -y modules:modules/policy:modules/bgp:modules/interfaces:modules/local-routing:modules/vlan:modules/rib:modules/network-instance:modules/types lagopus-router.yang

###2, vswの起動

$ sudo -s
# DPDKDIR=~/dpdk-stable-18.11.5
 env LD_LIBRARY_PATH=$DPDKDIR/build/lib vsw

###3, openconfigd cliから interface, vrf設定
commitすると設定が適用され、設定ファイル /usr/local/etc/openconfigd.conf に保存される

$ cli
> config
# set interfaces interface if0
# set interfaces interface if0 config device "0000:03:00.0"
# set interfaces interface if0 config driver dpdk
# set interfaces interface if0 config enabled true
# set interfaces interface if0 config type ethernetCsmacd
# set interfaces interface if0 subinterfaces subinterface 0
# set interfaces interface if0 subinterfaces subinterface 0 config enabled true
# set interfaces interface if0 subinterfaces subinterface 0 ipv4 addresses address 203.0.113.1
# set interfaces interface if0 subinterfaces subinterface 0 ipv4 addresses address 203.0.113.1 config prefix-length 24
# set interfaces interface if1
# set interfaces interface if1 config device "0000:0b:00.0"
# set interfaces interface if1 config driver dpdk
# set interfaces interface if1 config enabled true
# set interfaces interface if1 config type ethernetCsmacd
# set interfaces interface if1 subinterfaces subinterface 0
# set interfaces interface if1 subinterfaces subinterface 0 config enabled true
# set interfaces interface if1 subinterfaces subinterface 0 ipv4 addresses address 198.51.100.1
# set interfaces interface if1 subinterfaces subinterface 0 ipv4 addresses address 198.51.100.1 config prefix-length 24
# set network-instances network-instance vrf1
# set network-instances network-instance vrf1 interfaces interface if0
# set network-instances network-instance vrf1 interfaces interface if0 subinterface 0
# set network-instances network-instance vrf1 interfaces interface if1
# set network-instances network-instance vrf1 interfaces interface if1 subinterface 0
# set network-instances network-instance vrf1 config enabled true
# set network-instances network-instance vrf1 config enabled-address-families IPV4
# set network-instances network-instance vrf1 config type L3VRF
# commit

###4, ipsec daemonの起動

$ sudo /usr/local/ipsec/bin/ipsec_vrf start
$ sudo /usr/local/ipsec/bin/ipsec_vrf statusall
vrf1
Status of IKE charon daemon (strongSwan 5.6.2, Linux 4.15.0-74-generic, x86_64):
  uptime: 1 days, since Jan 16 16:11:34 2020
  malloc: sbrk 2686976, mmap 0, used 913488, free 1773488
  worker threads: 11 of 16 idle, 5/0/0/0 working, job queue: 0/0/0/0, scheduled: 20
  loaded plugins: charon aes des rc2 sha2 sha1 md5 mgf1 random nonce x509 revocation constraints pubkey pkcs1 pkcs7 pkcs8 pkcs12 pgp dnskey sshkey pem openssl fips-prf gmp curve25519 xcbc cmac hmac attr kernel-netlink resolve socket-default stroke vici updown eap-radius xauth-generic xauth-eap counters
Virtual IP pools (size/online/offline):
  192.0.2.0/24: 254/0/1
Listening IP addresses:
  203.0.113.1
Connections:
 ipsec-xauth:  %any...%any  IKEv1, dpddelay=3s
 ipsec-xauth:   local:  [203.0.113.1] uses pre-shared key authentication
 ipsec-xauth:   remote: uses pre-shared key authentication
 ipsec-xauth:   remote: uses XAuth authentication: any
 ipsec-xauth:   child:  0.0.0.0/0 === dynamic TUNNEL, dpdaction=clear
Security Associations (0 up, 0 connecting):
  none

##その他設定
###1, 追加設定

  • Policy Based Routingは現状ARP解決の機能がないため、Static ARPを設定。
  • vrfのrouting設定を追加。
$ sudo arp -s 203.0.113.2 00:00:5e:00:53:00 -i if0-0
$ sudo arp -s 198.51.100.2 00:00:5e:00:53:ff -i if1-0
$ arp -na

$ sudo ip rule add iif vrf1 from 192.0.2.0/24 table 2
$ sudo ip route add table 1 default via 203.0.113.2 dev if0-0
$ sudo ip route add table 2 default via 198.51.100.2 dev if1-0
$ ip rule
0:      from all lookup local 
219:    from 192.0.2.0/24 iif vrf1 lookup 2 
220:    from all lookup 220 
1000:   from all lookup [l3mdev-table] 
1000:   from all iif vrf1 lookup 1 
1000:   from all oif vrf1 lookup 1 
32766:  from all lookup main 
32767:  from all lookup default
$ routel

#スマホ設定

  • iOS : 設定 → VPN
  • Android : 設定 → ネットワークとインタネット → 詳細設定 → VPN
    device-ipsec-config.png

完成

接続できたところで満足して、細かいテストなどはまだおこなっておりません。。
ご興味がある方は、色々と試してみてください。
コメントなど歓迎します。

参考資料

*1 Lagopus Switch and Router - http://www.lagopus.org/
*2 Lagopus Router v19.07.1 SlideShare - https://www.slideshare.net/tomoyahibi/lagopus-router-v19071
*3 Lagopus software router book - http://www.lagopus.org/lagopus-book/en/router/html/
[](
*4 FD.io VPP/IPSec and IKEv2
*5 TNSR - TNSR Secure Networking Software Platform - Router VPN Firewall
)

5
3
0

Register as a new user and use Qiita more conveniently

  1. You get articles that match your needs
  2. You can efficiently read back useful information
  3. You can use dark theme
What you can do with signing up
Sign upLogin
5
3

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?