VMware Fusion on Mac OS X環境にて、vMXソフトウェアルータをセットアップした備忘録です。
- MacBook Pro (Retina, 15-inch, Mid 2015)
- プロセッサ： 2.2 GHz Intel Core i7
- メモリ：16 GB 1600 MHz DDR3
基本的には、"vMX Getting Started Guide for KVM"の記載に従って、vMX環境を整備しました。
■ vMXトライアル版の入手
次のURLから、vMXトライアル版をダウンロードします。
http://www.juniper.net/us/en/dm/free-vmx-trial
たぶん、ここの工程が、一番難易度が高いのかもしれません
■ vMXのセットアップ備忘録
最小構成となる、"Lab simulation"環境のセットアップを目指します。
次なる難関は、最小構成といえども、4CPU, 5GB (+数GB)の仮想インスタンスが必要になる点ですね
（ちなみに、今回は、4CPU, 8GBの仮想インスタンスで作業を進めました。）
(1) Ubuntu環境確認
VMware Fusion上に、仮想インスタンス"Ubuntu Server版 16.04.1"をデプロイしました。
- Ubuntuバージョンの確認
$ cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=16.04
DISTRIB_CODENAME=xenial
DISTRIB_DESCRIPTION="Ubuntu 16.04.1 LTS"
- 搭載nicの確認
$ lspci|grep Ethernet
02:01.0 Ethernet controller: Intel Corporation 82545EM Gigabit Ethernet Controller (Copper) (rev 01)
02:02.0 Ethernet controller: Intel Corporation 82545EM Gigabit Ethernet Controller (Copper) (rev 01)
02:03.0 Ethernet controller: Intel Corporation 82545EM Gigabit Ethernet Controller (Copper) (rev 01)
- network設定の確認
$ cat /etc/network/interfaces
/etc/network/interfaces
# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).
source /etc/network/interfaces.d/*
# The loopback network interface
auto lo
iface lo inet loopback
# The primary network interface
auto ens33
iface ens33 inet static
address 192.168.100.201
netmask 255.255.255.0
gateway 192.168.100.1
$ ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 00:0c:29:f3:16:bb brd ff:ff:ff:ff:ff:ff
inet 192.168.100.201/24 brd 192.168.100.255 scope global ens33
valid_lft forever preferred_lft forever
inet6 240b:11:53a0:400:20c:29ff:fef3:16bb/64 scope global mngtmpaddr dynamic
valid_lft 14378sec preferred_lft 12578sec
inet6 fe80::20c:29ff:fef3:16bb/64 scope link
valid_lft forever preferred_lft forever
3: ens34: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether 00:0c:29:f3:16:c5 brd ff:ff:ff:ff:ff:ff
4: ens35: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether 00:0c:29:f3:16:cf brd ff:ff:ff:ff:ff:ff
- kernel バージョン確認
$ uname -a
Linux vMX-1 4.4.0-31-generic #50-Ubuntu SMP Wed Jul 13 00:07:12 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
- "vMX-1"仮想インスタンス上で、vt-xが有効になっていることを確認する
$ cat /proc/cpuinfo | grep vmx
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts mmx fxsr sse sse2 ss syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts nopl xtopology tsc_reliable nonstop_tsc aperfmperf eagerfpu pni pclmulqdq vmx ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm epb tpr_shadow vnmi ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 invpcid xsaveopt dtherm ida arat pln pts
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts mmx fxsr sse sse2 ss syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts nopl xtopology tsc_reliable nonstop_tsc aperfmperf eagerfpu pni pclmulqdq vmx ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm epb tpr_shadow vnmi ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 invpcid xsaveopt dtherm ida arat pln pts
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts mmx fxsr sse sse2 ss syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts nopl xtopology tsc_reliable nonstop_tsc aperfmperf eagerfpu pni pclmulqdq vmx ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm epb tpr_shadow vnmi ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 invpcid xsaveopt dtherm ida arat pln pts
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts mmx fxsr sse sse2 ss syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts nopl xtopology tsc_reliable nonstop_tsc aperfmperf eagerfpu pni pclmulqdq vmx ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm epb tpr_shadow vnmi ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 invpcid xsaveopt dtherm ida arat pln pts
(2) vMXインストールに向けての事前準備
- 各種パッケージのインストール
$ sudo apt-get update
$ sudo apt-get install -y bridge-utils qemu-kvm libvirt-bin
$ sudo apt-get install -y libyaml-dev python-yaml numactl libparted0-dev libpciaccess-dev
$ sudo apt-get install -y libnuma-dev libyajl-dev libxml2-dev libglib2.0-dev
$ sudo apt-get install -y python-pip python-dev libxml2-dev libxslt-dev libnl-3-dev
$ sudo apt-get install -y python python-netifaces vnc4server
- libvirtのバージョン確認
$ libvirtd --version
libvirtd (libvirt) 1.3.1
- virtshのバージョン確認
$ virsh --version
1.3.1
(3) Ubuntu環境の設定変更
- KSMを無効にする
$ sudo vi /etc/default/qemu-kvm
qemu-kvm
KSM_ENABLED=0
- APICを無効にする
$ sudo vi /etc/modprobe.d/qemu-system-x86.conf
qemu-system-x86.conf
options kvm_intel enable_apicv=0 nested=1 enable_apicv=0
- 再起動 ..
$ sudo reboot
- "kvm-intel"カーネルモジュールを組み込む
$ sudo modprobe kvm_intel
$ lsmod|grep kvm
kvm_intel 172032 0
kvm 536576 1 kvm_intel
irqbypass 16384 1 kvm
(4) vMXパッケージのインストール
- vMXパッケージを展開する（ここでは、/home/tsubo配下に展開したものとする）
$ ls -l /home/tsubo
total 4
drwxrwxr-x 8 tsubo tsubo 4096 Dec 31 04:30 vmx-16.1R1.7
- vMX設定ファイルを配布する
$ vi /home/tsubo/vmx-16.1R1.7/config/vmx.conf
vmx.conf
##############################################################
#
# vmx.conf
# Config file for vmx on the hypervisor.
# Uses YAML syntax.
# Leave a space after ":" to specify the parameter value.
#
##############################################################
---
#Configuration on the host side - management interface, VM images etc.
HOST:
identifier : vmx1 # Maximum 4 characters
host-management-interface : ens33
routing-engine-image : "/home/tsubo/vmx-16.1R1.7/images/junos-vmx-x86-64-16.1R1.7.qcow2"
routing-engine-hdd : "/home/tsubo/vmx-16.1R1.7/images/vmxhdd.img"
forwarding-engine-image : "/home/tsubo/vmx-16.1R1.7/images/vFPC-20160617.img"
---
#External bridge configuration
BRIDGES:
- type : external
name : br-ext # Max 10 characters
---
#vRE VM parameters
CONTROL_PLANE:
vcpus : 1
memory-mb : 1024
console_port: 8601
interfaces :
- type : static
ipaddr : 10.102.144.94
macaddr : "0A:00:DD:C0:DE:0E"
---
#vPFE VM parameters
FORWARDING_PLANE:
memory-mb : 4096
vcpus : 3
console_port: 8602
device-type : virtio
interfaces :
- type : static
ipaddr : 10.102.144.98
macaddr : "0A:00:DD:C0:DE:10"
---
#Interfaces
JUNOS_DEVICES:
- interface : ge-0/0/0
mac-address : "02:06:0A:0E:FF:F0"
description : "ge-0/0/0 interface"
- interface : ge-0/0/1
mac-address : "02:06:0A:0E:FF:F1"
description : "ge-0/0/1 interface"
- vMXパッケージをインストールする
$ cd /home/tsubo/vmx-16.1R1.7
$ sudo ./vmx.sh -lv --install
==================================================
Welcome to VMX
==================================================
Date..............................................12/31/16 04:46:07
VMX Identifier....................................vmx1
Config file......................................./home/tsubo/vmx-16.1R1.7/config/vmx.conf
Build Directory.................................../home/tsubo/vmx-16.1R1.7/build/vmx1
Environment file................................../home/tsubo/vmx-16.1R1.7/env/ubuntu_virtio.env
Junos Device Type.................................virtio
Initialize scripts................................[OK]
Copy images to build directory....................[OK]
==================================================
VMX Environment Setup Completed
==================================================
==================================================
VMX Install & Start
==================================================
Linux distribution................................ubuntu
Check GRUB........................................[Disabled]
Installation status of qemu-kvm...................[OK]
Installation status of libvirt-bin................[OK]
Installation status of bridge-utils...............[OK]
Installation status of python.....................[OK]
Installation status of libyaml-dev................[OK]
Installation status of python-yaml................[OK]
Installation status of numactl....................[OK]
Installation status of libnuma-dev................[OK]
Installation status of libparted0-dev.............[OK]
Installation status of libpciaccess-dev...........[OK]
Installation status of libyajl-dev................[OK]
Installation status of libxml2-dev................[OK]
Installation status of libglib2.0-dev.............[OK]
Installation status of libnl-dev..................[OK]
Check Kernel Version..............................[Disabled]
Check Qemu Version................................[Disabled]
Check libvirt Version.............................[Disabled]
Check virsh connectivity..........................[OK]
IXGBE Enabled.....................................[Disabled]
==================================================
Pre-Install Checks Completed
==================================================
Check for VM vcp-vmx1.............................[Not Running]
Check for VM vfp-vmx1.............................[Not Running]
Cleanup VM states.................................[OK]
Check if bridge br-ext exists.....................[No]
Cleanup VM bridge br-ext..........................[OK]
Cleanup VM bridge br-int-vmx1.....................[OK]
==================================================
VMX Stop Completed
==================================================
Check VCP image...................................[OK]
Check VFP image...................................[OK]
VMX Model.........................................FPC
Check VCP Config image............................[OK]
Check management interface........................[OK]
Check interface ens34.............................[OK]
Check interface ens35.............................[OK]
Setup huge pages to 8192..........................[OK]
Attempt to kill libvirt...........................[OK]
Attempt to start libvirt..........................[OK]
Sleep 2 secs......................................[OK]
Check libvirt support for hugepages...............[OK]
==================================================
System Setup Completed
==================================================
Get Management Address of ens33...................[OK]
Generate libvirt files............................[OK]
Sleep 2 secs......................................[OK]
Find configured management interface..............ens33
Find existing management gateway..................ens33
Check if ens33 is already enslaved to br-ext......[No]
Gateway interface needs change....................[Yes]
Create br-ext.....................................[OK]
Get Management Gateway............................192.168.100.1
Flush ens33.......................................[OK]
Start br-ext......................................[OK]
Bind ens33 to br-ext..............................[OK]
Get Management MAC................................00:0c:29:f3:16:bb
Assign Management MAC 00:0c:29:f3:16:bb...........[OK]
Add default gw 192.168.100.1......................[OK]
Create br-int-vmx1................................[OK]
Start br-int-vmx1.................................[OK]
Check and start default bridge....................[OK]
Define vcp-vmx1...................................[OK]
Define vfp-vmx1...................................[OK]
Wait 2 secs.......................................[OK]
Start vcp-vmx1....................................[OK]
Start vfp-vmx1....................................[OK]
Wait 2 secs.......................................[OK]
==================================================
VMX Bringup Completed
==================================================
Check if br-ext is created........................[Created]
Check if br-int-vmx1 is created...................[Created]
Check if VM vcp-vmx1 is running...................[Running]
Check if VM vfp-vmx1 is running...................[Running]
Check if tap interface vcp_ext-vmx1 exists........[OK]
Check if tap interface vcp_int-vmx1 exists........[OK]
Check if tap interface vfp_ext-vmx1 exists........[OK]
Check if tap interface vfp_int-vmx1 exists........[OK]
==================================================
VMX Status Verification Completed.
==================================================
Log file........................................../home/tsubo/vmx-16.1R1.7/build/vmx1/logs/vmx_1483127167.log
==================================================
Thank you for using VMX
==================================================
(5) vMXインストール後の初期コンフィグ設定
- VCPにtelnetログインする
$ telnet localhost 8601
Trying ::1...
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
FreeBSD/amd64 (Amnesiac) (ttyu0)
login: root
--- JUNOS 16.1R1.7 Kernel 64-bit JNPR-10.1-20160624.329953_builder_stable_10
root@:~ # cli
root> configure
Entering configuration mode
[edit]
root#
- rootユーザのパスワードを設定する
root# set system root-authentication plain-text-password
New password:
Retype new password:
[edit]
- ssh経由でのリモートログインを許容する
root# set system services ssh root-login allow
[edit]
- マネージメントポートのアドレス設定を行う
root# set interfaces fxp0 unit 0 family inet address 192.168.100.101/24
[edit]
- lite modeを有効にする
root# set chassis fpc 0 lite-mode
- 初期コンフィグ設定をコミットする
root# commit
Generating RSA key /etc/ssh/ssh_host_key
Generating DSA key /etc/ssh/ssh_host_dsa_key
Generating RSA2 key /etc/ssh/ssh_host_rsa_key
Generating ECDSA key /etc/ssh/ssh_host_ecdsa_key
Generating ED25519 key /etc/ssh/ssh_host_ed25519_key
commit complete
[edit]
- telnetを終了する（ctrl + "]"でloginセッションを終了したのち、"quit"にて終了する）
telnet> quit
Connection closed.
- リモートログインできることを確認する
$ ssh root@192.168.100.101
Password:
Last login: Fri Dec 30 20:01:47 2016
--- JUNOS 16.1R1.7 Kernel 64-bit JNPR-10.1-20160624.329953_builder_stable_10
root@:~ # cli
root>
- vMXの初期コンフィグ内容を確認してみる
root> show configuration |display set
set version 16.1R1.7
set system root-authentication encrypted-password "$5$hb1aiak5$ffPgYwaiom.2CADb.NARz/mLcM4PivxkYCRDU/lSVxC"
set system services ssh root-login allow
set system syslog user * any emergency
set system syslog file messages any notice
set system syslog file messages authorization info
set system syslog file interactive-commands interactive-commands any
set chassis fpc 0 lite-mode
set interfaces fxp0 unit 0 family inet address 192.168.100.101/24
- vMX構成を確認してみる
root> show chassis hardware
Hardware inventory:
Item Version Part number Serial number Description
Chassis VM5866BB5829 VMX
Midplane
Routing Engine 0 RE-VMX
CB 0 VMX SCB
CB 1 VMX SCB
FPC 0 Virtual FPC
CPU Rev. 1.0 RIOT 123XYZ987
MIC 0 Virtual
PIC 0 BUILTIN BUILTIN Virtual
- interface動作状態を確認してみる
root> show interfaces terse
Interface Admin Link Proto Local Remote
ge-0/0/0 up up
lc-0/0/0 up up
lc-0/0/0.32769 up up vpls
pfe-0/0/0 up up
pfe-0/0/0.16383 up up inet
inet6
pfh-0/0/0 up up
pfh-0/0/0.16383 up up inet
pfh-0/0/0.16384 up up inet
ge-0/0/1 up up
ge-0/0/2 up down
ge-0/0/3 up down
ge-0/0/4 up down
ge-0/0/5 up down
ge-0/0/6 up down
ge-0/0/7 up down
ge-0/0/8 up down
ge-0/0/9 up down
cbp0 up up
demux0 up up
dsc up up
em1 up up
em1.0 up up inet 10.0.0.4/8
128.0.0.1/2
128.0.0.4/2
inet6 fe80::5254:ff:fe61:3c61/64
fec0::a:0:0:4/64
tnp 0x4
esi up up
fxp0 up up
fxp0.0 up up inet 192.168.100.101/24
gre up up
ipip up up
irb up up
jsrv up up
jsrv.1 up up inet 128.0.0.127/2
lo0 up up
lo0.16384 up up inet 127.0.0.1 --> 0/0
lo0.16385 up up inet
lsi up up
mtun up up
pimd up up
pime up up
pip0 up up
pp0 up up
rbeb up up
tap up up
vtep up up
以上、vMXの初期セットアップが、完了できました。
■ 最後に
そもそも、MacBook Proに、vMXソフトルータをセットアップするのは、無謀なチャレンジでした。
本来、Intel Xeon搭載のマルチコアCPUを多数搭載しているサーバ / ワークステーションに、セットアップするべきでした。
ただ、セットアップ作業は、だいぶ熟知できたので、今後、新たなサーバ入手する機会があれば、本格的に、vMXソフトウェアルータの検証を行ってみたいです。