AWS
kubernetes
kops

kopsでamazon-vpc-cni-k8sを利用する

kopsでamazon-vpc-cni-k8sを利用してみました。kopsの利用方法自体は参考URLを参考にしてください。

本記事では省略しています。podがVPC内のIPを利用するのでk8sクラスタ外のEC2がpodにNodePortなどを利用せずにアクセスできることがポイントです。


参考URL


kops create cluster

オプションで--networking amazon-vpc-routed-eniを指定。

$ kops create cluster \

--name paper2-cni.cluster.k8s.local \
--zones us-east-1a,us-east-1c \
--master-size="t2.micro" --node-size="t2.micro" \
--networking amazon-vpc-routed-eni \
--yes
I0729 16:55:35.622015 51569 create_cluster.go:1318] Using SSH public key: /Users/paper2/.ssh/id_rsa.pub
I0729 16:55:37.923294 51569 create_cluster.go:472] Inferred --cloud=aws from zone "us-east-1a"
I0729 16:55:39.634860 51569 subnets.go:184] Assigned CIDR 172.20.32.0/19 to subnet us-east-1a
I0729 16:55:39.634927 51569 subnets.go:184] Assigned CIDR 172.20.64.0/19 to subnet us-east-1c
I0729 16:55:45.861707 51569 apply_cluster.go:456] Gossip DNS: skipping DNS validation
I0729 16:55:47.768441 51569 executor.go:91] Tasks: 0 done / 81 total; 31 can run
I0729 16:55:50.086276 51569 vfs_castore.go:731] Issuing new certificate: "apiserver-aggregator-ca"
I0729 16:55:50.101474 51569 vfs_castore.go:731] Issuing new certificate: "ca"
I0729 16:55:52.641220 51569 executor.go:91] Tasks: 31 done / 81 total; 25 can run
I0729 16:55:57.438697 51569 vfs_castore.go:731] Issuing new certificate: "kubelet-api"
I0729 16:55:57.524915 51569 vfs_castore.go:731] Issuing new certificate: "kops"
I0729 16:55:57.553610 51569 vfs_castore.go:731] Issuing new certificate: "kube-controller-manager"
I0729 16:55:57.621873 51569 vfs_castore.go:731] Issuing new certificate: "kube-proxy"
I0729 16:55:57.656320 51569 vfs_castore.go:731] Issuing new certificate: "apiserver-aggregator"
I0729 16:55:57.847304 51569 vfs_castore.go:731] Issuing new certificate: "kubelet"
I0729 16:55:58.005469 51569 vfs_castore.go:731] Issuing new certificate: "kubecfg"
I0729 16:55:58.200209 51569 vfs_castore.go:731] Issuing new certificate: "kube-scheduler"
I0729 16:55:58.528400 51569 vfs_castore.go:731] Issuing new certificate: "apiserver-proxy-client"
I0729 16:56:00.733555 51569 executor.go:91] Tasks: 56 done / 81 total; 21 can run
I0729 16:56:00.902358 51569 logging_retryer.go:60] Retryable error (RequestError: send request failed
caused by: Post https://ec2.us-east-1.amazonaws.com/: EOF) from ec2/DescribeSecurityGroups - will retry after delay of 56ms
I0729 16:56:09.965480 51569 logging_retryer.go:60] Retryable error (RequestError: send request failed
caused by: Post https://elasticloadbalancing.us-east-1.amazonaws.com/: EOF) from elasticloadbalancing/DescribeTags - will retry after delay of 30ms
I0729 16:56:37.412321 51569 executor.go:91] Tasks: 77 done / 81 total; 3 can run
I0729 16:56:40.602256 51569 vfs_castore.go:731] Issuing new certificate: "master"
I0729 16:56:41.881540 51569 executor.go:91] Tasks: 80 done / 81 total; 1 can run
I0729 16:56:42.818618 51569 executor.go:91] Tasks: 81 done / 81 total; 0 can run
I0729 16:56:43.495941 51569 update_cluster.go:291] Exporting kubecfg for cluster
kops has set your kubectl context to paper2-cni.cluster.k8s.local

Cluster is starting. It should be ready in a few minutes.

Suggestions:
* validate cluster: kops validate cluster
* list nodes: kubectl get nodes --show-labels
* ssh to the master: ssh -i ~/.ssh/id_rsa admin@api.paper2-cni.cluster.k8s.local
* the admin user is specific to Debian. If not using Debian please use the appropriate user based on your OS.
* read about installing addons at: https://github.com/kubernetes/kops/blob/master/docs/addons.md.


podをデプロイしてみる


deployment.yaml

apiVersion: extensions/v1beta1

kind: Deployment
metadata:
name: nginx-deployment
spec:
replicas: 5
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:1.12.1
ports:
- containerPort: 80
- containerPort: 443

$ kubectl apply -f deployment.yaml

deployment.extensions/nginx-deployment created


podにVPCのIPが割り振られているか確認

$ kubectl get all -o wide

NAME READY STATUS RESTARTS AGE IP NODE
pod/nginx-deployment-7f786f684b-4cgqh 1/1 Running 0 31s 172.20.80.188 ip-172-20-95-59.ec2.internal
pod/nginx-deployment-7f786f684b-6fpq2 1/1 Running 0 32s 172.20.40.146 ip-172-20-36-86.ec2.internal
pod/nginx-deployment-7f786f684b-bgd2j 1/1 Running 0 32s 172.20.81.121 ip-172-20-95-59.ec2.internal
pod/nginx-deployment-7f786f684b-cz848 1/1 Running 0 32s 172.20.59.80 ip-172-20-36-86.ec2.internal
pod/nginx-deployment-7f786f684b-xbjfr 1/1 Running 0 32s 172.20.51.46 ip-172-20-36-86.ec2.internal

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
service/kubernetes ClusterIP 172.20.0.1 <none> 443/TCP 7m <none>

NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR
deployment.extensions/nginx-deployment 5 5 5 5 32s nginx nginx:1.12.1 app=nginx

NAME DESIRED CURRENT READY AGE CONTAINERS IMAGES SELECTOR
replicaset.extensions/nginx-deployment-7f786f684b 5 5 5 32s nginx nginx:1.12.1 app=nginx,pod-template-hash=3934292406

NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR
deployment.apps/nginx-deployment 5 5 5 5 32s nginx nginx:1.12.1 app=nginx

NAME DESIRED CURRENT READY AGE CONTAINERS IMAGES SELECTOR
replicaset.apps/nginx-deployment-7f786f684b 5 5 5 32s nginx nginx:1.12.1 app=nginx,pod-template-hash=3934292406

podのIPに172.20/16が割り振られていますね。


k8sクラスタに所属しないEC2からアクセスしてみる

通常k8sのpodへはingressやNodePortを使用しない限りk8sクラスタの外部からはアクセスできません。しかしamazon-vpc-cni-k8sを利用するとpodのIPはVPCのIPになるので、同じVPC内のインスタンスからpodにアクセスできます。

それではEC2をk8sクラスタのサブネット上に作成してアクセスしてみます。

kopsはセキュリティグループでアクセスを制限しているので作成したEC2にk8sクラスタを構築するEC2にアクセスできるセキュリティグループを割り当てる必要があります。

[ec2-user@ip-172-20-89-195 ~]$ ip route

default via 172.20.64.1 dev eth0
169.254.169.254 dev eth0
172.20.64.0/19 dev eth0 proto kernel scope link src 172.20.89.195
[ec2-user@ip-172-20-89-195 ~]$ curl 172.20.80.188
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
body {
width: 35em;
margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif;
}
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>

k8sクラスタ外のEC2からpodにアクセスできました。