INSTALL
INSTALL.Docker.md を手引きに、Ubuntu 16.10 でインストールしてみた。192.168.0.40 が CENTRAL_IP で、192.168.0.41, 192.168.0.42 という docker host という構成にする。
ovn-northd は ovn-central package に入っている。ovn-central package を入れると ovn-northd も起動する。systemd での unit name は ovn-central だが、意味のあるログは /var/log/openvswitch 以下に出力される。
ovn-controller は ovn-host package に入っている。ovn-host package を入れると ovn-controller も起動する。unit name は ovn-host だが、意味のあるログは /var/log/openvswitch 以下に出力される。次のコマンドを入れると ovn-central に繋がって、初期フローまで入る。
ovs-vsctl set Open_vSwitch . external_ids:ovn-remote="tcp:$CENTRAL_IP:6642" \
external_ids:ovn-nb="tcp:$CENTRAL_IP:6641" \
external_ids:ovn-encap-ip=$LOCAL_IP \
external_ids:ovn-encap-type="$ENCAP_TYPE"
NXST_FLOW reply (xid=0x4):
cookie=0x0, duration=1025.214s, table=0, n_packets=0, n_bytes=0, idle_age=1025, priority=100,in_port=1 actions=move:NXM_NX_TUN_ID[0..23]->OXM_OF_METADATA[0..23],move:NXM_NX_TUN_METADATA0[16..30]->NXM_NX_REG14[0..14],move:NXM_NX_TUN_METADATA0[0..15]->NXM_NX_REG15[0..15],resubmit(,33)
cookie=0x0, duration=1025.215s, table=32, n_packets=0, n_bytes=0, idle_age=1025, priority=0 actions=resubmit(,33)
cookie=0x0, duration=1025.215s, table=34, n_packets=0, n_bytes=0, idle_age=1025, priority=0 actions=load:0->NXM_NX_REG0[],load:0->NXM_NX_REG1[],load:0->NXM_NX_REG2[],load:0->NXM_NX_REG3[],load:0->NXM_NX_REG4[],load:0->NXM_NX_REG5[],load:0->NXM_NX_REG6[],load:0->NXM_NX_REG7[],load:0->NXM_NX_REG8[],load:0->NXM_NX_REG9[],resubmit(,48)
cookie=0x0, duration=1025.215s, table=64, n_packets=0, n_bytes=0, idle_age=1025, priority=0 actions=resubmit(,65)
ovn-docker package を入れる。systemd ファイルは無い様子。ovn-docker-overlay-driver を起動させると、/etc/docker/plugins/openvswitch.spec が書き込まれる。これで docker から openvswitch プラグインとして認識される準備が整う。
ovn-docker-overlay-driver --detach
docker をクラスタリングさせるために consul package を入れる。いつも通り bootstrap-expect=3 で 3 台のクラスタを組んでみた。3 台それぞれで agent を起動させる。
consul agent -bind 192.168.0.40 -data-dir=/var/lib/consul/data -server -bootstrap-expect=3 | logger &
## 下は 192.168.0.40 で実行すれば事足りる。
consul join 192.168.0.41
consul join 192.168.0.42
docker.io package を入れる。/etc/defaults/docker に DOCKER_OPTS を書いて再起動しておく。
DOCKER_OPTS=--cluster-store=consul://127.0.0.1:8500 --cluster-advertise=192.168.0.41:0
動作確認
$ docker network create -d openvswitch --subnet=192.168.1.0/24 foo
ffbdc7c12ae8e885dcf87f12992e7c508de2b5073d071c9def78be56eba4cc21
$ docker network ls
NETWORK ID NAME DRIVER SCOPE
9e8ac92d1448 bridge bridge local
ffbdc7c12ae8 foo openvswitch global
80421cffc1f0 host host local
6219ee0ca5c9 none null local
INSTALL.Docker.md は :6640 と書いてるけど、たぶん typo で、:6641 に問い合わせる。
ovn-nbctl --db=tcp:192.168.0.40:6641 ls-list
コンテナデプロイ
2 台の docker host でそれぞれ 1 コンテナデプロイさせた後のフローはこんな感じになった。
NXST_FLOW reply (xid=0x4):
cookie=0x0, duration=5338.201s, table=0, n_packets=7, n_bytes=578, idle_age=4, priority=100,in_port=2 actions=move:NXM_NX_TUN_ID[0..23]->OXM_OF_METADATA[0..23],move:NXM_NX_TUN_METADATA0[16..30]->NXM_NX_REG14[0..14],move:NXM_NX_TUN_METADATA0[0..15]->NXM_NX_REG15[0..15],resubmit(,33)
cookie=0x0, duration=41.260s, table=0, n_packets=8, n_bytes=648, idle_age=30, priority=100,in_port=4 actions=load:0x1->NXM_NX_REG13[],load:0x2->OXM_OF_METADATA[],load:0x1->NXM_NX_REG14[],resubmit(,16)
cookie=0x0, duration=41.260s, table=16, n_packets=0, n_bytes=0, idle_age=41, priority=100,metadata=0x2,vlan_tci=0x1000/0x1000 actions=drop
cookie=0x0, duration=41.260s, table=16, n_packets=0, n_bytes=0, idle_age=41, priority=100,metadata=0x2,dl_src=01:00:00:00:00:00/01:00:00:00:00:00 actions=drop
cookie=0x0, duration=41.260s, table=16, n_packets=8, n_bytes=648, idle_age=30, priority=50,reg14=0x1,metadata=0x2 actions=resubmit(,17)
cookie=0x0, duration=11.153s, table=16, n_packets=0, n_bytes=0, idle_age=11, priority=50,reg14=0x2,metadata=0x2 actions=resubmit(,17)
cookie=0x0, duration=41.260s, table=17, n_packets=8, n_bytes=648, idle_age=30, priority=0,metadata=0x2 actions=resubmit(,18)
cookie=0x0, duration=41.260s, table=18, n_packets=8, n_bytes=648, idle_age=30, priority=0,metadata=0x2 actions=resubmit(,19)
cookie=0x0, duration=41.260s, table=19, n_packets=8, n_bytes=648, idle_age=30, priority=0,metadata=0x2 actions=resubmit(,20)
cookie=0x0, duration=41.260s, table=20, n_packets=8, n_bytes=648, idle_age=30, priority=0,metadata=0x2 actions=resubmit(,21)
cookie=0x0, duration=41.260s, table=21, n_packets=0, n_bytes=0, idle_age=41, priority=100,ip,reg0=0x1/0x1,metadata=0x2 actions=ct(table=22,zone=NXM_NX_REG13[0..15])
cookie=0x0, duration=41.260s, table=21, n_packets=0, n_bytes=0, idle_age=41, priority=100,ipv6,reg0=0x1/0x1,metadata=0x2 actions=ct(table=22,zone=NXM_NX_REG13[0..15])
cookie=0x0, duration=41.260s, table=21, n_packets=8, n_bytes=648, idle_age=30, priority=0,metadata=0x2 actions=resubmit(,22)
cookie=0x0, duration=41.260s, table=22, n_packets=8, n_bytes=648, idle_age=30, priority=0,metadata=0x2 actions=resubmit(,23)
cookie=0x0, duration=41.260s, table=23, n_packets=8, n_bytes=648, idle_age=30, priority=0,metadata=0x2 actions=resubmit(,24)
cookie=0x0, duration=41.260s, table=24, n_packets=0, n_bytes=0, idle_age=41, priority=100,ipv6,reg0=0x2/0x2,metadata=0x2 actions=ct(commit,zone=NXM_NX_REG13[0..15],exec(load:0->NXM_NX_CT_LABEL[0])),resubmit(,25)
cookie=0x0, duration=41.260s, table=24, n_packets=0, n_bytes=0, idle_age=41, priority=100,ip,reg0=0x2/0x2,metadata=0x2 actions=ct(commit,zone=NXM_NX_REG13[0..15],exec(load:0->NXM_NX_CT_LABEL[0])),resubmit(,25)
cookie=0x0, duration=41.260s, table=24, n_packets=0, n_bytes=0, idle_age=41, priority=100,ip,reg0=0x4/0x4,metadata=0x2 actions=ct(table=25,zone=NXM_NX_REG13[0..15],nat)
cookie=0x0, duration=41.260s, table=24, n_packets=0, n_bytes=0, idle_age=41, priority=100,ipv6,reg0=0x4/0x4,metadata=0x2 actions=ct(table=25,zone=NXM_NX_REG13[0..15],nat)
cookie=0x0, duration=41.260s, table=24, n_packets=8, n_bytes=648, idle_age=30, priority=0,metadata=0x2 actions=resubmit(,25)
cookie=0x0, duration=41.259s, table=25, n_packets=0, n_bytes=0, idle_age=41, priority=50,arp,metadata=0x2,arp_tpa=192.168.1.3,arp_op=1 actions=move:NXM_OF_ETH_SRC[]->NXM_OF_ETH_DST[],mod_dl_src:02:1b:5c:3e:66:87,load:0x2->NXM_OF_ARP_OP[],move:NXM_NX_ARP_SHA[]->NXM_NX_ARP_THA[],load:0x21b5c3e6687->NXM_NX_ARP_SHA[],move:NXM_OF_ARP_SPA[]->NXM_OF_ARP_TPA[],load:0xc0a80103->NXM_OF_ARP_SPA[],move:NXM_NX_REG14[]->NXM_NX_REG15[],load:0x1->NXM_NX_REG10[0],resubmit(,32)
cookie=0x0, duration=10.820s, table=25, n_packets=0, n_bytes=0, idle_age=10, priority=50,arp,metadata=0x2,arp_tpa=192.168.1.4,arp_op=1 actions=move:NXM_OF_ETH_SRC[]->NXM_OF_ETH_DST[],mod_dl_src:02:3b:98:39:5c:70,load:0x2->NXM_OF_ARP_OP[],move:NXM_NX_ARP_SHA[]->NXM_NX_ARP_THA[],load:0x23b98395c70->NXM_NX_ARP_SHA[],move:NXM_OF_ARP_SPA[]->NXM_OF_ARP_TPA[],load:0xc0a80104->NXM_OF_ARP_SPA[],move:NXM_NX_REG14[]->NXM_NX_REG15[],load:0x1->NXM_NX_REG10[0],resubmit(,32)
cookie=0x0, duration=41.260s, table=25, n_packets=8, n_bytes=648, idle_age=30, priority=0,metadata=0x2 actions=resubmit(,26)
cookie=0x0, duration=41.260s, table=26, n_packets=8, n_bytes=648, idle_age=30, priority=0,metadata=0x2 actions=resubmit(,27)
cookie=0x0, duration=41.260s, table=27, n_packets=8, n_bytes=648, idle_age=30, priority=0,metadata=0x2 actions=resubmit(,28)
cookie=0x0, duration=41.260s, table=28, n_packets=8, n_bytes=648, idle_age=30, priority=100,metadata=0x2,dl_dst=01:00:00:00:00:00/01:00:00:00:00:00 actions=load:0xffff->NXM_NX_REG15[],resubmit(,32)
cookie=0x0, duration=41.260s, table=28, n_packets=0, n_bytes=0, idle_age=41, priority=50,metadata=0x2,dl_dst=02:1b:5c:3e:66:87 actions=load:0x1->NXM_NX_REG15[],resubmit(,32)
cookie=0x0, duration=11.145s, table=28, n_packets=0, n_bytes=0, idle_age=11, priority=50,metadata=0x2,dl_dst=02:3b:98:39:5c:70 actions=load:0x2->NXM_NX_REG15[],resubmit(,32)
cookie=0x0, duration=10.821s, table=32, n_packets=0, n_bytes=0, idle_age=10, priority=150,reg10=0x2/0x2 actions=resubmit(,33)
cookie=0x0, duration=10.821s, table=32, n_packets=0, n_bytes=0, idle_age=10, priority=100,reg15=0xffff,metadata=0x2 actions=load:0x2->NXM_NX_TUN_ID[0..23],set_field:0xffff/0xffffffff->tun_metadata0,move:NXM_NX_REG14[0..14]->NXM_NX_TUN_METADATA0[16..30],output:2,resubmit(,33)
cookie=0x0, duration=10.821s, table=32, n_packets=0, n_bytes=0, idle_age=10, priority=100,reg15=0x2,metadata=0x2 actions=load:0x2->NXM_NX_TUN_ID[0..23],set_field:0x2/0xffffffff->tun_metadata0,move:NXM_NX_REG14[0..14]->NXM_NX_TUN_METADATA0[16..30],output:2
cookie=0x0, duration=8822.393s, table=32, n_packets=16, n_bytes=1296, idle_age=30, priority=0 actions=resubmit(,33)
cookie=0x0, duration=41.260s, table=33, n_packets=0, n_bytes=0, idle_age=41, priority=100,reg15=0x1,metadata=0x2 actions=load:0x1->NXM_NX_REG13[],resubmit(,34)
cookie=0x0, duration=41.260s, table=33, n_packets=15, n_bytes=1226, idle_age=4, priority=100,reg15=0xffff,metadata=0x2 actions=load:0x1->NXM_NX_REG13[],load:0x1->NXM_NX_REG15[],resubmit(,34),load:0xffff->NXM_NX_REG15[]
cookie=0x0, duration=41.260s, table=34, n_packets=8, n_bytes=648, idle_age=30, priority=100,reg10=0/0x1,reg14=0x1,reg15=0x1,metadata=0x2 actions=drop
cookie=0x0, duration=8822.393s, table=34, n_packets=7, n_bytes=578, idle_age=4, priority=0 actions=load:0->NXM_NX_REG0[],load:0->NXM_NX_REG1[],load:0->NXM_NX_REG2[],load:0->NXM_NX_REG3[],load:0->NXM_NX_REG4[],load:0->NXM_NX_REG5[],load:0->NXM_NX_REG6[],load:0->NXM_NX_REG7[],load:0->NXM_NX_REG8[],load:0->NXM_NX_REG9[],resubmit(,48)
cookie=0x0, duration=41.260s, table=48, n_packets=7, n_bytes=578, idle_age=4, priority=0,metadata=0x2 actions=resubmit(,49)
cookie=0x0, duration=41.260s, table=49, n_packets=7, n_bytes=578, idle_age=4, priority=0,metadata=0x2 actions=resubmit(,50)
cookie=0x0, duration=41.260s, table=50, n_packets=0, n_bytes=0, idle_age=41, priority=100,ipv6,reg0=0x1/0x1,metadata=0x2 actions=ct(table=51,zone=NXM_NX_REG13[0..15])
cookie=0x0, duration=41.260s, table=50, n_packets=0, n_bytes=0, idle_age=41, priority=100,ip,reg0=0x1/0x1,metadata=0x2 actions=ct(table=51,zone=NXM_NX_REG13[0..15])
cookie=0x0, duration=41.260s, table=50, n_packets=7, n_bytes=578, idle_age=4, priority=0,metadata=0x2 actions=resubmit(,51)
cookie=0x0, duration=41.260s, table=51, n_packets=7, n_bytes=578, idle_age=4, priority=0,metadata=0x2 actions=resubmit(,52)
cookie=0x0, duration=41.260s, table=52, n_packets=7, n_bytes=578, idle_age=4, priority=0,metadata=0x2 actions=resubmit(,53)
cookie=0x0, duration=41.260s, table=53, n_packets=0, n_bytes=0, idle_age=41, priority=100,ipv6,reg0=0x4/0x4,metadata=0x2 actions=ct(table=54,zone=NXM_NX_REG13[0..15],nat)
cookie=0x0, duration=41.260s, table=53, n_packets=0, n_bytes=0, idle_age=41, priority=100,ip,reg0=0x4/0x4,metadata=0x2 actions=ct(table=54,zone=NXM_NX_REG13[0..15],nat)
cookie=0x0, duration=41.260s, table=53, n_packets=0, n_bytes=0, idle_age=41, priority=100,ip,reg0=0x2/0x2,metadata=0x2 actions=ct(commit,zone=NXM_NX_REG13[0..15],exec(load:0->NXM_NX_CT_LABEL[0])),resubmit(,54)
cookie=0x0, duration=41.260s, table=53, n_packets=0, n_bytes=0, idle_age=41, priority=100,ipv6,reg0=0x2/0x2,metadata=0x2 actions=ct(commit,zone=NXM_NX_REG13[0..15],exec(load:0->NXM_NX_CT_LABEL[0])),resubmit(,54)
cookie=0x0, duration=41.260s, table=53, n_packets=7, n_bytes=578, idle_age=4, priority=0,metadata=0x2 actions=resubmit(,54)
cookie=0x0, duration=41.260s, table=54, n_packets=7, n_bytes=578, idle_age=4, priority=0,metadata=0x2 actions=resubmit(,55)
cookie=0x0, duration=41.260s, table=55, n_packets=7, n_bytes=578, idle_age=4, priority=100,metadata=0x2,dl_dst=01:00:00:00:00:00/01:00:00:00:00:00 actions=resubmit(,64)
cookie=0x0, duration=41.260s, table=55, n_packets=0, n_bytes=0, idle_age=41, priority=50,reg15=0x1,metadata=0x2 actions=resubmit(,64)
cookie=0x0, duration=11.153s, table=55, n_packets=0, n_bytes=0, idle_age=11, priority=50,reg15=0x2,metadata=0x2 actions=resubmit(,64)
cookie=0x0, duration=41.260s, table=64, n_packets=0, n_bytes=0, idle_age=41, priority=100,reg10=0x1/0x1,reg15=0x1,metadata=0x2 actions=push:NXM_OF_IN_PORT[],load:0->NXM_OF_IN_PORT[],resubmit(,65),pop:NXM_OF_IN_PORT[]
cookie=0x0, duration=8822.393s, table=64, n_packets=7, n_bytes=578, idle_age=4, priority=0 actions=resubmit(,65)
cookie=0x0, duration=41.260s, table=65, n_packets=7, n_bytes=578, idle_age=4, priority=100,reg15=0x1,metadata=0x2 actions=output:4