OpenStack仮想ネットワークの基本動作を体験してみたいと思います。

■ やってみたこと

OpenStaxk仮想ネットワークで、試してみたことは、

サーバ機器２台を、HUB接続する
サーバ機器に、IPアドレスを付与する
サーバ機器の間で、ping通信してみる

という、非常にシンプルな内容です。

■ OpenStack実験環境

"Nested KVM環境でのNewton版OpenStack構築メモ"で作成したOpenStack環境を使用します。

nova仮想マシン一覧

[root@newton ~(keystone_demo)]# nova list
+--------------------------------------+---------+--------+------------+-------------+----------------------------------+
| ID                                   | Name    | Status | Task State | Power State | Networks                         |
+--------------------------------------+---------+--------+------------+-------------+----------------------------------+
| ab70f784-385d-44a8-b010-1322e7be94a7 | server1 | ACTIVE | -          | Running     | admin_net=10.0.0.4, 10.79.5.191  |
| 06521269-bddb-4def-8369-3f92fd2f4622 | server2 | ACTIVE | -          | Running     | admin_net=10.0.0.11, 10.79.5.195 |
+--------------------------------------+---------+--------+------------+-------------+----------------------------------+

neutronフローティングIP一覧

[root@newton ~(keystone_demo)]# neutron floatingip-list
+--------------------------------------+------------------+---------------------+--------------------------------------+
| id                                   | fixed_ip_address | floating_ip_address | port_id                              |
+--------------------------------------+------------------+---------------------+--------------------------------------+
| affd1370-760d-4148-9d48-5afb9f97eaf5 | 10.0.0.4         | 10.79.5.191         | 3155cbf3-ddcb-4acd-b5a1-dc37f4a49a4c |
| cb83e8bc-47c2-438a-8c74-2f1654ddc09c | 10.0.0.11        | 10.79.5.195         | 75f7e4ac-600b-4a45-9948-d45b3ec7169b |
+--------------------------------------+------------------+---------------------+--------------------------------------+

OpenStack仮想ネットワーク構成イメージ

■ 実験１「サーバ機器２台を、HUB接続してみる」

userネットワークに仮想マシンを配備して、ping疎通確認を行ってみます。

(1) userネットワーク用ポートを作成する

仮想マシン"server1"用ポートの作成

[root@newton ~(keystone_demo)]# neutron port-create --name user_port1 user_net
Created a new port:
+-----------------------+------------------------------------------------------------------------------------+
| Field                 | Value                                                                              |
+-----------------------+------------------------------------------------------------------------------------+
| admin_state_up        | True                                                                               |
| allowed_address_pairs |                                                                                    |
| binding:vnic_type     | normal                                                                             |
| created_at            | 2017-02-03T00:29:03Z                                                               |
| description           |                                                                                    |
| device_id             |                                                                                    |
| device_owner          |                                                                                    |
| extra_dhcp_opts       |                                                                                    |
| fixed_ips             | {"subnet_id": "9f0e59df-d6c8-4c2d-9de5-8979798e5350", "ip_address": "172.16.0.11"} |
| id                    | 4fbca26b-95bd-4d74-8f56-20b1acfa8e89                                               |
| mac_address           | fa:16:3e:b3:33:ae                                                                  |
| name                  | user_port1                                                                         |
| network_id            | 80feacbb-6bd5-4a3a-adff-a715b78c5d3e                                               |
| project_id            | 67d85efd952f4b6a8c03c6f86fecb8ea                                                   |
| revision_number       | 4                                                                                  |
| security_groups       | bff315ad-0f08-45f8-8b14-5d19a9f95aa6                                               |
| status                | DOWN                                                                               |
| tenant_id             | 67d85efd952f4b6a8c03c6f86fecb8ea                                                   |
| updated_at            | 2017-02-03T00:29:04Z                                                               |
+-----------------------+------------------------------------------------------------------------------------+

仮想マシン"server2"用ポートの作成

[root@newton ~(keystone_demo)]# neutron port-create --name user_port2 user_net
Created a new port:
+-----------------------+------------------------------------------------------------------------------------+
| Field                 | Value                                                                              |
+-----------------------+------------------------------------------------------------------------------------+
| admin_state_up        | True                                                                               |
| allowed_address_pairs |                                                                                    |
| binding:vnic_type     | normal                                                                             |
| created_at            | 2017-02-03T00:29:12Z                                                               |
| description           |                                                                                    |
| device_id             |                                                                                    |
| device_owner          |                                                                                    |
| extra_dhcp_opts       |                                                                                    |
| fixed_ips             | {"subnet_id": "9f0e59df-d6c8-4c2d-9de5-8979798e5350", "ip_address": "172.16.0.12"} |
| id                    | e1776e0d-9f64-4684-b050-c8749f8c9a88                                               |
| mac_address           | fa:16:3e:38:dc:ef                                                                  |
| name                  | user_port2                                                                         |
| network_id            | 80feacbb-6bd5-4a3a-adff-a715b78c5d3e                                               |
| project_id            | 67d85efd952f4b6a8c03c6f86fecb8ea                                                   |
| revision_number       | 4                                                                                  |
| security_groups       | bff315ad-0f08-45f8-8b14-5d19a9f95aa6                                               |
| status                | DOWN                                                                               |
| tenant_id             | 67d85efd952f4b6a8c03c6f86fecb8ea                                                   |
| updated_at            | 2017-02-03T00:29:13Z                                                               |
+-----------------------+------------------------------------------------------------------------------------+

登録済みのport一覧の確認

[root@newton ~(keystone_demo)]# neutron port-list 
+--------------------------------------+------------+-------------------+---------------------------------------------------+
| id                                   | name       | mac_address       | fixed_ips                                         |
+--------------------------------------+------------+-------------------+---------------------------------------------------+
| 3155cbf3-ddcb-4acd-b5a1-dc37f4a49a4c |            | fa:16:3e:49:48:8e | {"subnet_id": "31a40092-042b-4f27-a00c-           |
|                                      |            |                   | 54c9324db2ac", "ip_address": "10.0.0.4"}          |
| 3c173d98-c714-492d-8c6f-ef330c0e5d12 |            | fa:16:3e:6a:16:02 | {"subnet_id": "31a40092-042b-4f27-a00c-           |
|                                      |            |                   | 54c9324db2ac", "ip_address": "10.0.0.2"}          |
| 4fbca26b-95bd-4d74-8f56-20b1acfa8e89 | user_port1 | fa:16:3e:b3:33:ae | {"subnet_id": "9f0e59df-d6c8-4c2d-                |
|                                      |            |                   | 9de5-8979798e5350", "ip_address": "172.16.0.11"}  |
| 5da3eb5e-d1ff-41ec-a583-0cefc19e7973 |            | fa:16:3e:4e:58:bc | {"subnet_id": "31a40092-042b-4f27-a00c-           |
|                                      |            |                   | 54c9324db2ac", "ip_address": "10.0.0.1"}          |
| 680fcfb5-2e04-4ec5-a77c-3c6fd29d9ea9 |            | fa:16:3e:0f:0d:f1 | {"subnet_id": "9f0e59df-d6c8-4c2d-                |
|                                      |            |                   | 9de5-8979798e5350", "ip_address": "172.16.0.2"}   |
| 75f7e4ac-600b-4a45-9948-d45b3ec7169b |            | fa:16:3e:7c:92:60 | {"subnet_id": "31a40092-042b-4f27-a00c-           |
|                                      |            |                   | 54c9324db2ac", "ip_address": "10.0.0.11"}         |
| e1776e0d-9f64-4684-b050-c8749f8c9a88 | user_port2 | fa:16:3e:38:dc:ef | {"subnet_id": "9f0e59df-d6c8-4c2d-                |
|                                      |            |                   | 9de5-8979798e5350", "ip_address": "172.16.0.12"}  |
+--------------------------------------+------------+-------------------+---------------------------------------------------+

(2) 仮想マシンに、userネットワーク用ポートを取り付ける

仮想マシン"server1"側でのポート取り付け

[root@newton ~(keystone_demo)]# nova interface-attach --port-id 4fbca26b-95bd-4d74-8f56-20b1acfa8e89 server1

仮想マシン"server2"側でのポート取り付け

[root@newton ~(keystone_demo)]# nova interface-attach --port-id e1776e0d-9f64-4684-b050-c8749f8c9a88 server2

仮想マシン"server1"側での登録済みのインタフェース一覧の確認

[root@newton ~(keystone_demo)]# nova interface-list server1
+------------+--------------------------------------+--------------------------------------+--------------+-------------------+
| Port State | Port ID                              | Net ID                               | IP addresses | MAC Addr          |
+------------+--------------------------------------+--------------------------------------+--------------+-------------------+
| ACTIVE     | 3155cbf3-ddcb-4acd-b5a1-dc37f4a49a4c | 9ea97f2d-7ead-44b1-91e1-afc7ca22e21e | 10.0.0.4     | fa:16:3e:49:48:8e |
| ACTIVE     | 4fbca26b-95bd-4d74-8f56-20b1acfa8e89 | 80feacbb-6bd5-4a3a-adff-a715b78c5d3e | 172.16.0.11  | fa:16:3e:b3:33:ae |
+------------+--------------------------------------+--------------------------------------+--------------+-------------------+

仮想マシン"server2"側での登録済みのインタフェース一覧の確認

[root@newton ~(keystone_demo)]# nova interface-list server2
+------------+--------------------------------------+--------------------------------------+--------------+-------------------+
| Port State | Port ID                              | Net ID                               | IP addresses | MAC Addr          |
+------------+--------------------------------------+--------------------------------------+--------------+-------------------+
| ACTIVE     | 75f7e4ac-600b-4a45-9948-d45b3ec7169b | 9ea97f2d-7ead-44b1-91e1-afc7ca22e21e | 10.0.0.11    | fa:16:3e:7c:92:60 |
| ACTIVE     | e1776e0d-9f64-4684-b050-c8749f8c9a88 | 80feacbb-6bd5-4a3a-adff-a715b78c5d3e | 172.16.0.12  | fa:16:3e:38:dc:ef |
+------------+--------------------------------------+--------------------------------------+--------------+-------------------+

(3) 仮想マシン"server1"上で、取り付け済みインタフェースを有効にする

現在のインタフェース状態の確認（ "ens6" が新たに追加になった）

ubuntu@server1:~$ ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc pfifo_fast state UP group default qlen 1000
    link/ether fa:16:3e:49:48:8e brd ff:ff:ff:ff:ff:ff
    inet 10.0.0.4/24 brd 10.0.0.255 scope global ens3
       valid_lft forever preferred_lft forever
    inet6 fe80::f816:3eff:fe49:488e/64 scope link 
       valid_lft forever preferred_lft forever
3: ens6: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether fa:16:3e:b3:33:ae brd ff:ff:ff:ff:ff:ff

PCI接続デバイスの確認

root@server1:~# lspci |grep Ether
00:03.0 Ethernet controller: Red Hat, Inc Virtio network device
00:06.0 Ethernet controller: Red Hat, Inc Virtio network device

暫定として、IPアドレス値を付与

ubuntu@server1:~$ sudo -i
root@server1:~# ip addr add 172.16.0.11/24 dev ens6
root@server1:~# ip link set ens6 up

再度、インタフェース状態の確認

root@server1:~# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc pfifo_fast state UP group default qlen 1000
    link/ether fa:16:3e:49:48:8e brd ff:ff:ff:ff:ff:ff
    inet 10.0.0.4/24 brd 10.0.0.255 scope global ens3
       valid_lft forever preferred_lft forever
    inet6 fe80::f816:3eff:fe49:488e/64 scope link 
       valid_lft forever preferred_lft forever
3: ens6: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether fa:16:3e:b3:33:ae brd ff:ff:ff:ff:ff:ff
    inet 172.16.0.11/24 scope global ens6
       valid_lft forever preferred_lft forever
    inet6 fe80::f816:3eff:feb3:33ae/64 scope link 
       valid_lft forever preferred_lft forever

(4) 仮想マシン"server2"上で、取り付け済みインタフェースを有効にする

現在のインタフェース状態の確認（"eth1"が新たに追加になった）

$ ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc pfifo_fast qlen 1000
    link/ether fa:16:3e:7c:92:60 brd ff:ff:ff:ff:ff:ff
    inet 10.0.0.11/24 brd 10.0.0.255 scope global eth0
    inet6 fe80::f816:3eff:fe7c:9260/64 scope link 
       valid_lft forever preferred_lft forever
3: eth1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop qlen 1000
    link/ether fa:16:3e:38:dc:ef brd ff:ff:ff:ff:ff:ff

暫定として、IPアドレス値を付与

$ sudo -i
# ip addr add 172.16.0.12/24 dev eth1
# ip link set eth1 up

再度、インタフェース状態の確認

# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc pfifo_fast qlen 1000
    link/ether fa:16:3e:7c:92:60 brd ff:ff:ff:ff:ff:ff
    inet 10.0.0.11/24 brd 10.0.0.255 scope global eth0
    inet6 fe80::f816:3eff:fe7c:9260/64 scope link 
       valid_lft forever preferred_lft forever
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast qlen 1000
    link/ether fa:16:3e:38:dc:ef brd ff:ff:ff:ff:ff:ff
    inet 172.16.0.12/24 scope global eth1
    inet6 fe80::f816:3eff:fe38:dcef/64 scope link 
       valid_lft forever preferred_lft forever

(4) 仮想マシン間での疎通性を確認してみる

仮想マシン"server1"上で、対向の仮想マシン宛てに、pingを打ってみる

root@server1:~# ping 172.16.0.12
PING 172.16.0.12 (172.16.0.12) 56(84) bytes of data.
64 bytes from 172.16.0.12: icmp_seq=1 ttl=64 time=2.28 ms
64 bytes from 172.16.0.12: icmp_seq=2 ttl=64 time=2.19 ms
64 bytes from 172.16.0.12: icmp_seq=3 ttl=64 time=0.708 ms
^C
--- 172.16.0.12 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2003ms
rtt min/avg/max/mdev = 0.708/1.730/2.288/0.724 ms

最終的に、作成したトポロジ構成

ここまでは、順調に、OpenStack仮想ネットワークが作成できました。

■ 実験２「HUB接続したサーバ機器のLANケーブルを抜き差ししてみる」

続いて、仮想マシン"server1"側のインタフェースの取り外し/取り付けを試してみます。

(1) neuton上でのポート取り外し/取り付け

仮想マシン"server1"側での登録済みのインタフェース一覧の確認

[root@newton ~(keystone_demo)]# nova interface-list server1
+------------+--------------------------------------+--------------------------------------+--------------+-------------------+
| Port State | Port ID                              | Net ID                               | IP addresses | MAC Addr          |
+------------+--------------------------------------+--------------------------------------+--------------+-------------------+
| ACTIVE     | 3155cbf3-ddcb-4acd-b5a1-dc37f4a49a4c | 9ea97f2d-7ead-44b1-91e1-afc7ca22e21e | 10.0.0.4     | fa:16:3e:49:48:8e |
| ACTIVE     | 4fbca26b-95bd-4d74-8f56-20b1acfa8e89 | 80feacbb-6bd5-4a3a-adff-a715b78c5d3e | 172.16.0.11  | fa:16:3e:b3:33:ae |
+------------+--------------------------------------+--------------------------------------+--------------+-------------------+

仮想マシン"server1"側のポート取り外し

[root@newton ~(keystone_demo)]# nova interface-detach server1 4fbca26b-95bd-4d74-8f56-20b1acfa8e89

仮想マシン"server1"側での登録済みのインタフェース一覧の確認

[root@newton ~(keystone_demo)]# nova interface-list server1
+------------+--------------------------------------+--------------------------------------+--------------+-------------------+
| Port State | Port ID                              | Net ID                               | IP addresses | MAC Addr          |
+------------+--------------------------------------+--------------------------------------+--------------+-------------------+
| ACTIVE     | 3155cbf3-ddcb-4acd-b5a1-dc37f4a49a4c | 9ea97f2d-7ead-44b1-91e1-afc7ca22e21e | 10.0.0.4     | fa:16:3e:49:48:8e |
+------------+--------------------------------------+--------------------------------------+--------------+-------------------+

再度、仮想マシン"server1"側でのポート取り付け

[root@newton ~(keystone_demo)]# nova interface-attach --port-id 4fbca26b-95bd-4d74-8f56-20b1acfa8e89 server1

仮想マシン"server1"側での登録済みのインタフェース一覧の確認

[root@newton ~(keystone_demo)]# nova interface-list server1
+------------+--------------------------------------+--------------------------------------+--------------+-------------------+
| Port State | Port ID                              | Net ID                               | IP addresses | MAC Addr          |
+------------+--------------------------------------+--------------------------------------+--------------+-------------------+
| ACTIVE     | 3155cbf3-ddcb-4acd-b5a1-dc37f4a49a4c | 9ea97f2d-7ead-44b1-91e1-afc7ca22e21e | 10.0.0.4     | fa:16:3e:49:48:8e |
| ACTIVE     | 4fbca26b-95bd-4d74-8f56-20b1acfa8e89 | 80feacbb-6bd5-4a3a-adff-a715b78c5d3e | 172.16.0.11  | fa:16:3e:b3:33:ae |
+------------+--------------------------------------+--------------------------------------+--------------+-------------------+

(2) 仮想マシン"server1"上での確認

現在のインタフェース状態の確認（ "ens7" にNIC名が変わってしまった）

root@server1:~# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc pfifo_fast state UP group default qlen 1000
    link/ether fa:16:3e:49:48:8e brd ff:ff:ff:ff:ff:ff
    inet 10.0.0.4/24 brd 10.0.0.255 scope global ens3
       valid_lft forever preferred_lft forever
    inet6 fe80::f816:3eff:fe49:488e/64 scope link 
       valid_lft forever preferred_lft forever
4: ens7: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether fa:16:3e:b3:33:ae brd ff:ff:ff:ff:ff:ff

PCI接続デバイスの確認

root@server1:~# lspci |grep Ether
00:03.0 Ethernet controller: Red Hat, Inc Virtio network device
00:07.0 Ethernet controller: Red Hat, Inc Virtio network device

[実験結果]
neuton上でのポート取り外し/取り付けを行うと、暫定で設定していたIPアドレス値が無効になってしまいました。さらに、Etherデバイスカードの接続PCIポートが変更されてしまいました。ちなみに、MACアドレスは、特に、変化しませんでした。

■ 終わりに

今回は、実環境では、発生しえない動作を、経験することとなりました。
OpenStack仮想ネットワーク構成は、二つの異なるOpenStackコミュニティを基盤として構成されるためですが、いまいち、有効な解決方法が見つかっていない現状です。

OpenStack仮想ネットワークを活用して、クラウドオーケストレーションの自動化が一般化しつつありますし、NFVのような柔軟なトラフィック制御を実現する際には、阻害要因になりそうな気がします。みなさん、どうやって、解決しているか気になるところです。