LoginSignup
0
0

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?

@so_ma(Souma Sakaguchi)

nerdctlでCalicoを動かす（実行編）

Last updated at Posted at 2024-08-17

前回までのあらすじ

nerdctlをつかってCalico動かしたい。
Calicoのアーキテクチャを確認した。
環境構築した。こっから動かす。
前回はこちら↓↓

etcdの起動

今回はetcdクラスタを組んで見ました。
どこかでetcdを動かしさえすればいいので、片方のノードでetcdを実行するとか、別のノード用意してetcdを動かすとかでもOK。

node1
$ etcd \
  --name calico-node1 \
  --listen-peer-urls http://192.168.22.10:2380 \
  --listen-client-urls http://0.0.0.0:2379 \
  --initial-advertise-peer-urls http://192.168.22.10:2380 \
  --initial-cluster calico-node1=http://192.168.22.10:2380,calico-node2=http://192.168.22.11:2380 \
  --initial-cluster-state new \
  --initial-cluster-token etcd-cluster \
  --advertise-client-urls http://192.168.22.10:2379
node2
$ etcd \
  --name calico-node2 \
  --listen-peer-urls http://192.168.22.11:2380 \
  --listen-client-urls http://0.0.0.0:2379 \
  --initial-advertise-peer-urls http://192.168.22.11:2380 \
  --initial-cluster calico-node1=http://192.168.22.10:2380,calico-node2=http://192.168.22.11:2380 \
  --initial-cluster-state new \
  --initial-cluster-token etcd-cluster \
  --advertise-client-urls http://192.168.22.11:2379

calico-nodeの実行

calicoctl用設定ファイルの作成

デフォルトでは、/etc/calico/calicoctl.cfgを読み込む。--configオプションを指定することで、読み込むファイルを指定もできる。

node1
apiVersion: projectcalico.org/v3
kind: CalicoAPIConfig
metadata:
spec:
  datastoreType: "etcdv3"
  etcdEndpoints: "http://127.0.0.1:2379,http://192.168.22.11:2379"
node2
apiVersion: projectcalico.org/v3
kind: CalicoAPIConfig
metadata:
spec:
  datastoreType: "etcdv3"
  etcdEndpoints: "http://192.168.22.10:2379,http://127.0.0.1:2379"

calicoctlの実行

node1 & node2
$ sudo calicoctl node run
結果確認
$ sudo calicoctl node status

Calico process is running.

IPv4 BGP status
+---------------+-------------------+-------+----------+-------------+
| PEER ADDRESS  |     PEER TYPE     | STATE |  SINCE   |    INFO     |
+---------------+-------------------+-------+----------+-------------+
| 192.168.22.11 | node-to-node mesh | up    | 10:06:57 | Established |
+---------------+-------------------+-------+----------+-------------+

IPv6 BGP status
No IPv6 peers found.

etcdにも設定が入っていることが分かる。

etcdの結果
$ etcdctl get "" --prefix --keys-only
/calico/resources/v3/projectcalico.org/clusterinformations/default

/calico/resources/v3/projectcalico.org/felixconfigurations/default

/calico/resources/v3/projectcalico.org/felixconfigurations/node.node1

/calico/resources/v3/projectcalico.org/felixconfigurations/node.node2

/calico/resources/v3/projectcalico.org/ippools/default-ipv4-ippool

/calico/resources/v3/projectcalico.org/ippools/default-ipv6-ippool

/calico/resources/v3/projectcalico.org/nodes/node1

/calico/resources/v3/projectcalico.org/nodes/node2

いろいろ準備完了！動かしましょう。

コンテナ実行

nerdctlからcalicoプラグインとコンテナランタイムを叩き、コンテナとそのネットワークを構築する。

calico用設定ファイルの作成

cniプラグイン用の設定ファイルは、/etc/cni/net.d/hogehoge.conflistで設定する。

node1
{
	"cniVersion": "1.0.0",
	"name": "99-calico",
	"plugins": [
		{
			"datastore_type": "etcdv3",
			"etcd_endpoints": "http://127.0.0.1:2379,http://192.168.22.11:2379",
			"log_level": "DEBUG",
			"log_file_path": "/var/log/calico/cni/cni.log",
			"type": "calico",
			"ipam": {
				"type": "calico-ipam"
			}
		}
	]
}
node2
{
	"cniVersion": "1.0.0",
	"name": "99-calico",
	"plugins": [
		{
			"datastore_type": "etcdv3",
			"etcd_endpoints": "http://192.168.22.10:2379,http://127.0.0.1:2379",
			"log_level": "DEBUG",
			"log_file_path": "/var/log/calico/cni/cni.log",
			"type": "calico",
			"ipam": {
				"type": "calico-ipam"
			}
		}
	]
}

containerdの実行

node1 & node2
$ sudo containerd

nerdctlの実行

node1 & node2
$ sudo nerdctl run -it --security-opt="seccomp=unconfined" --network=99-calico weibeld/ubuntu-networking:latest /bin/bash

vagrant@node1:~$のようなプロンプトからroot@000000000000:/#みたいなのに変化したらコンテナ構築完了です。

コンテナ間通信の確認

IPアドレスの割当

まずは、IPアドレスが割り当てられていることを確認しましょう。

node1
root@39334a563058:/# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0@if7: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether a6:ce:aa:b1:70:e9 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 172.16.166.128/32 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::a4ce:aaff:feb1:70e9/64 scope link 
       valid_lft forever preferred_lft forever
node2
root@f09ee008b45d:/# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0@if7: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 8e:69:cf:77:e6:78 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 172.16.104.0/32 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::8c69:cfff:fe77:e678/64 scope link 
       valid_lft forever preferred_lft forever

eth0というvlanが作られていて、一意のIPアドレスが割り振られています。図にするとこんな感じ。
node-container.png

ping・tcpdumpによる通信の確認

calico-node1コンテナ -> calico-node2コンテナ

calico-node1の上にあるコンテナから、calico-node2の上にあるコンテナに対して通信ができるか確認します。

node1
root@39334a563058:/# ping -c 5 172.16.104.0
PING 172.16.104.0 (172.16.104.0) 56(84) bytes of data.
64 bytes from 172.16.104.0: icmp_seq=1 ttl=62 time=0.994 ms
64 bytes from 172.16.104.0: icmp_seq=2 ttl=62 time=0.650 ms
64 bytes from 172.16.104.0: icmp_seq=3 ttl=62 time=0.710 ms
64 bytes from 172.16.104.0: icmp_seq=4 ttl=62 time=0.770 ms
64 bytes from 172.16.104.0: icmp_seq=5 ttl=62 time=0.635 ms

--- 172.16.104.0 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4056ms
rtt min/avg/max/mdev = 0.635/0.751/0.994/0.134 ms
node2
root@f09ee008b45d:/# tcpdump -i eth0 icmp
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
09:56:06.263429 IP 172.16.166.128 > f09ee008b45d: ICMP echo request, id 19, seq 1, length 64
09:56:06.263528 IP f09ee008b45d > 172.16.166.128: ICMP echo reply, id 19, seq 1, length 64
09:56:07.267801 IP 172.16.166.128 > f09ee008b45d: ICMP echo request, id 19, seq 2, length 64
09:56:07.267866 IP f09ee008b45d > 172.16.166.128: ICMP echo reply, id 19, seq 2, length 64
09:56:08.302871 IP 172.16.166.128 > f09ee008b45d: ICMP echo request, id 19, seq 3, length 64
09:56:08.302958 IP f09ee008b45d > 172.16.166.128: ICMP echo reply, id 19, seq 3, length 64
09:56:09.305760 IP 172.16.166.128 > f09ee008b45d: ICMP echo request, id 19, seq 4, length 64
09:56:09.305850 IP f09ee008b45d > 172.16.166.128: ICMP echo reply, id 19, seq 4, length 64
09:56:10.320056 IP 172.16.166.128 > f09ee008b45d: ICMP echo request, id 19, seq 5, length 64
09:56:10.320112 IP f09ee008b45d > 172.16.166.128: ICMP echo reply, id 19, seq 5, length 64

calico-node2コンテナ -> calico-node1コンテナ

先程の逆向きも一応試します。

node1
root@39334a563058:/# tcpdump -i eth0 icmp
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
09:58:42.058563 IP 172.16.104.0 > 39334a563058: ICMP echo request, id 20, seq 1, length 64
09:58:42.058633 IP 39334a563058 > 172.16.104.0: ICMP echo reply, id 20, seq 1, length 64
09:58:43.059954 IP 172.16.104.0 > 39334a563058: ICMP echo request, id 20, seq 2, length 64
09:58:43.060018 IP 39334a563058 > 172.16.104.0: ICMP echo reply, id 20, seq 2, length 64
09:58:44.089402 IP 172.16.104.0 > 39334a563058: ICMP echo request, id 20, seq 3, length 64
09:58:44.089469 IP 39334a563058 > 172.16.104.0: ICMP echo reply, id 20, seq 3, length 64
09:58:45.113388 IP 172.16.104.0 > 39334a563058: ICMP echo request, id 20, seq 4, length 64
09:58:45.113465 IP 39334a563058 > 172.16.104.0: ICMP echo reply, id 20, seq 4, length 64
09:58:46.137444 IP 172.16.104.0 > 39334a563058: ICMP echo request, id 20, seq 5, length 64
09:58:46.137552 IP 39334a563058 > 172.16.104.0: ICMP echo reply, id 20, seq 5, length 64
node2
root@f09ee008b45d:/# ping -c 5 172.16.166.128
PING 172.16.166.128 (172.16.166.128) 56(84) bytes of data.
64 bytes from 172.16.166.128: icmp_seq=1 ttl=62 time=0.856 ms
64 bytes from 172.16.166.128: icmp_seq=2 ttl=62 time=0.603 ms
64 bytes from 172.16.166.128: icmp_seq=3 ttl=62 time=0.659 ms
64 bytes from 172.16.166.128: icmp_seq=4 ttl=62 time=0.610 ms
64 bytes from 172.16.166.128: icmp_seq=5 ttl=62 time=0.725 ms

--- 172.16.166.128 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4078ms
rtt min/avg/max/mdev = 0.603/0.690/0.856/0.097 ms

コンテナから外への通信

こちらはおまけ、ホスト外への通信ができることを確認します。

node1
root@39334a563058:/# ping -c 5 google.com
PING google.com (172.217.161.78) 56(84) bytes of data.
64 bytes from nrt20s09-in-f14.1e100.net (172.217.161.78): icmp_seq=1 ttl=61 time=18.3 ms
64 bytes from nrt20s09-in-f14.1e100.net (172.217.161.78): icmp_seq=2 ttl=61 time=18.5 ms
64 bytes from nrt20s09-in-f14.1e100.net (172.217.161.78): icmp_seq=3 ttl=61 time=18.4 ms
64 bytes from nrt20s09-in-f14.1e100.net (172.217.161.78): icmp_seq=4 ttl=61 time=18.4 ms
64 bytes from nrt20s09-in-f14.1e100.net (172.217.161.78): icmp_seq=5 ttl=61 time=18.3 ms

--- google.com ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4026ms
rtt min/avg/max/mdev = 18.380/18.440/18.503/0.156 ms

全部通信できたーーー。
お片付けは、Vagrant destroy -fでいいでしょう。

おわり

以上でおしまいです。長いことお疲れ様でした。
nerdctlを使ってCalicoを動かせることが確認できたので、自分の研究のテスト環境となっていただきましょう。
「Kubernetesがよしなにやってくれてることを自分でやった」みたいな気分。Kubernetesすげーってなりました。

0
0
0

Register as a new user and use Qiita more conveniently

  1. You get articles that match your needs
  2. You can efficiently read back useful information
  3. You can use dark theme
What you can do with signing up
Sign upLogin
0
0

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?