L7のロードバランサをオンプレ(ベアメタル)のk8sで使うには、いろいろ設定せねばならぬ。k8s本体にはそんな機能はないんだよ。
CNIにCanal, metallb(L2 modeとして), L7LBにnginx-ingress-controllerを使う。
nginx-ingress-controllerを入れて設定しておかないとKind Ingressのyamlを食わせてもうまく動かないので注意。
※クラウドとかでEKSやGKEなどを使っている場合には良しなにやってくれるので必要なし。
※minikubeの人はアドオンがあるので別途ググってください。簡単です。
TL;DR
v1.19.2で試しています。
Calico for policy and flannel (aka Canal), metal-lb をインストールして Ingress nginxを設定して動作確認しています。
初期セットアップ
https://qiita.com/murata-tomohide/items/be2171ac459f8f7d54f0 を参考にセットアップしておきます。 https://qiita.com/murata-tomohide/items/be2171ac459f8f7d54f0#%E5%8B%95%E4%BD%9C%E7%A2%BA%E8%AA%8D この辺りまでできていればおｋ。
Canalまで終わったところだとこんな感じ。
murata:~ $ kubectl version
Client Version: version.Info{Major:"1", Minor:"19", GitVersion:"v1.19.2", GitCommit:"f5743093fd1c663cb0cbc89748f730662345d44d", GitTreeState:"clean", BuildDate:"2020-09-16T13:41:02Z", GoVersion:"go1.15", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"19", GitVersion:"v1.19.2", GitCommit:"f5743093fd1c663cb0cbc89748f730662345d44d", GitTreeState:"clean", BuildDate:"2020-09-16T13:32:58Z", GoVersion:"go1.15", Compiler:"gc", Platform:"linux/amd64"}
murata:~ $ kubectl get all -A
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system pod/calico-kube-controllers-c9784d67d-49g6t 1/1 Running 0 2m20s
kube-system pod/canal-wwmvf 2/2 Running 0 2m20s
kube-system pod/coredns-f9fd979d6-ccgm5 1/1 Running 0 7m43s
kube-system pod/coredns-f9fd979d6-ztwgs 1/1 Running 0 7m43s
kube-system pod/etcd-k8s-tmp.dev.deroris.local 1/1 Running 0 7m53s
kube-system pod/kube-apiserver-k8s-tmp.dev.deroris.local 1/1 Running 0 7m53s
kube-system pod/kube-controller-manager-k8s-tmp.dev.deroris.local 1/1 Running 0 7m53s
kube-system pod/kube-proxy-rfd7c 1/1 Running 0 7m43s
kube-system pod/kube-scheduler-k8s-tmp.dev.deroris.local 1/1 Running 0 7m53s
NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
default service/kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 8m2s
kube-system service/kube-dns ClusterIP 10.96.0.10 <none> 53/UDP,53/TCP,9153/TCP 8m
NAMESPACE NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE
kube-system daemonset.apps/canal 1 1 1 1 1 kubernetes.io/os=linux 2m21s
kube-system daemonset.apps/kube-proxy 1 1 1 1 1 kubernetes.io/os=linux 8m
NAMESPACE NAME READY UP-TO-DATE AVAILABLE AGE
kube-system deployment.apps/calico-kube-controllers 1/1 1 1 2m21s
kube-system deployment.apps/coredns 2/2 2 2 8m
NAMESPACE NAME DESIRED CURRENT READY AGE
kube-system replicaset.apps/calico-kube-controllers-c9784d67d 1 1 1 2m21s
kube-system replicaset.apps/coredns-f9fd979d6 2 2 2 7m43s
corednsのPodが動いていることと、ホストのNICが増えていることを確認します。(CanalはFlannnelとcaricoを使うらしい（よくわかってない）
murata:~ $ ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: ens3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
link/ether 52:54:00:81:7f:82 brd ff:ff:ff:ff:ff:ff
altname enp0s3
inet 172.30.203.203/24 brd 172.30.203.255 scope global noprefixroute ens3
valid_lft forever preferred_lft forever
inet6 fe80::5054:ff:fe81:7f82/64 scope link
valid_lft forever preferred_lft forever
3: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default
link/ether 02:42:ec:bb:c4:47 brd ff:ff:ff:ff:ff:ff
inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
valid_lft forever preferred_lft forever
4: flannel.1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UNKNOWN group default
link/ether ca:7e:00:d4:d8:c7 brd ff:ff:ff:ff:ff:ff
inet 10.244.0.0/32 scope global flannel.1
valid_lft forever preferred_lft forever
inet6 fe80::c87e:ff:fed4:d8c7/64 scope link
valid_lft forever preferred_lft forever
7: cali700ca5a0c12@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP group default
link/ether ee:ee:ee:ee:ee:ee brd ff:ff:ff:ff:ff:ff link-netnsid 0
inet6 fe80::ecee:eeff:feee:eeee/64 scope link
valid_lft forever preferred_lft forever
8: cali70c270b17c5@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP group default
link/ether ee:ee:ee:ee:ee:ee brd ff:ff:ff:ff:ff:ff link-netnsid 1
inet6 fe80::ecee:eeff:feee:eeee/64 scope link
valid_lft forever preferred_lft forever
9: cali7867aceb346@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP group default
link/ether ee:ee:ee:ee:ee:ee brd ff:ff:ff:ff:ff:ff link-netnsid 2
inet6 fe80::ecee:eeff:feee:eeee/64 scope link
valid_lft forever preferred_lft forever
MetalLB インストール
MetalLBについてはこちら
KubernetesロードバランサーのMetalLBを導入した話(Necoプロジェクト体験入部)
要するにTypeがLoadBalancerの場合にIPアドレスを付与してくれるやつ。
https://metallb.universe.tf/installation/
の通りにやっていきます。
kube-proxyの設定を変える
editで変えても下にあるシェルを流してもOKっぽいです。今回はシェルを流しました。
murata:~ $ # see what changes would be made, returns nonzero returncode if different
ube-system -o yaml | \
sed -e "s/strictARP: false/strictARP: true/" | \
kubectl diff -f - -n kube-systemmurata:~ $ kubectl get configmap kube-proxy -n kube-system -o yaml | \
> sed -e "s/strictARP: false/strictARP: true/" | \
> kubectl diff -f - -n kube-system
diff -u -N /tmp/LIVE-375254102/v1.ConfigMap.kube-system.kube-proxy /tmp/MERGED-128212413/v1.ConfigMap.kube-system.kube-proxy
--- /tmp/LIVE-375254102/v1.ConfigMap.kube-system.kube-proxy 2020-10-05 16:55:11.169956454 +0900
+++ /tmp/MERGED-128212413/v1.ConfigMap.kube-system.kube-proxy 2020-10-05 16:55:11.175956620 +0900
@@ -30,7 +30,7 @@
excludeCIDRs: null
minSyncPeriod: 0s
scheduler: ""
- strictARP: false
+ strictARP: true
syncPeriod: 0s
tcpFinTimeout: 0s
tcpTimeout: 0s
@@ -79,7 +79,6 @@
fieldsV1:
f:data:
.: {}
- f:config.conf: {}
f:kubeconfig.conf: {}
f:metadata:
f:annotations:
@@ -91,6 +90,14 @@
manager: kubeadm
operation: Update
time: "2020-10-05T07:44:04Z"
+ - apiVersion: v1
+ fieldsType: FieldsV1
+ fieldsV1:
+ f:data:
+ f:config.conf: {}
+ manager: kubectl-client-side-apply
+ operation: Update
+ time: "2020-10-05T07:55:11Z"
name: kube-proxy
namespace: kube-system
resourceVersion: "191"
murata:~ $ kubectl get configmap kube-proxy -n kube-system -o yaml | \
> sed -e "s/strictARP: false/strictARP: true/" | \
> kubectl apply -f - -n kube-system
Warning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply
configmap/kube-proxy configured
マニフェストを食う
完全にコピペです。
kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.9.3/manifests/namespace.yaml
kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.9.3/manifests/metallb.yaml
# On first install only
kubectl create secret generic -n metallb-system memberlist --from-literal=secretkey="$(openssl rand -base64 128)"
こんな感じです。
murata:~ $ kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.9.3/manifests/namespace.yaml
y -f https://raw.githubusercontent.com/metallb/metallb/v0.9.3/manifests/metallb.yaml
# On first install only
kubectl create secret generic -n metallb-system memberlist --from-literal=secretkey="$(openssl rand -base64 128)"
namespace/metallb-system created
murata:~ $ kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.9.3/manifests/metallb.yaml
podsecuritypolicy.policy/controller created
podsecuritypolicy.policy/speaker created
serviceaccount/controller created
serviceaccount/speaker created
clusterrole.rbac.authorization.k8s.io/metallb-system:controller created
clusterrole.rbac.authorization.k8s.io/metallb-system:speaker created
role.rbac.authorization.k8s.io/config-watcher created
role.rbac.authorization.k8s.io/pod-lister created
clusterrolebinding.rbac.authorization.k8s.io/metallb-system:controller created
clusterrolebinding.rbac.authorization.k8s.io/metallb-system:speaker created
rolebinding.rbac.authorization.k8s.io/config-watcher created
rolebinding.rbac.authorization.k8s.io/pod-lister created
daemonset.apps/speaker created
deployment.apps/controller created
murata:~ $ # On first install only
murata:~ $ kubectl create secret generic -n metallb-system memberlist --from-literal=secretkey="$(openssl rand -base64 128)"
secret/memberlist created
利用するIPアドレスのconfigmapを作ります。
https://metallb.universe.tf/configuration/#layer-2-configuration
Layer 2 Configuration の部分を行います。
addressの部分は設定時にバインドできるIPアドレスにします。今回はノードのIPアドレスの帯域で空いているところ。
※そのIPアドレスでクライアントPCからアクセスします。
この場合ノードサーバのIPアドレスは172.30.203.203なので、空いていた172.30.203.203-172.30.203.205を設定しました。（必ず使っていないことを確認すること！＆自分自身はいけるみたい？
アドレスのところ、うまくバリデーションしてくれないので注意してください。変にコメント入れると動かない気がします。
cat <<EOF | kubectl create -f -
apiVersion: v1
kind: ConfigMap
metadata:
namespace: metallb-system
name: config
data:
config: |
address-pools:
- name: default
protocol: layer2
addresses:
- 172.30.203.203-172.30.203.205
EOF
```
現段階ではこんな感じになる。
```
murata:~ $ kubectl get all,configmaps -A
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system pod/calico-kube-controllers-c9784d67d-49g6t 1/1 Running 0 14m
kube-system pod/canal-wwmvf 2/2 Running 0 14m
kube-system pod/coredns-f9fd979d6-ccgm5 1/1 Running 0 19m
kube-system pod/coredns-f9fd979d6-ztwgs 1/1 Running 0 19m
kube-system pod/etcd-k8s-tmp.dev.deroris.local 1/1 Running 0 19m
kube-system pod/kube-apiserver-k8s-tmp.dev.deroris.local 1/1 Running 0 19m
kube-system pod/kube-controller-manager-k8s-tmp.dev.deroris.local 1/1 Running 0 19m
kube-system pod/kube-proxy-rfd7c 1/1 Running 0 19m
kube-system pod/kube-scheduler-k8s-tmp.dev.deroris.local 1/1 Running 0 19m
metallb-system pod/controller-fb659dc8-75t6h 1/1 Running 0 6m18s
metallb-system pod/speaker-jb8rb 1/1 Running 0 6m18s
NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
default service/kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 19m
kube-system service/kube-dns ClusterIP 10.96.0.10 <none> 53/UDP,53/TCP,9153/TCP 19m
NAMESPACE NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE
kube-system daemonset.apps/canal 1 1 1 1 1 kubernetes.io/os=linux 14m
kube-system daemonset.apps/kube-proxy 1 1 1 1 1 kubernetes.io/os=linux 19m
metallb-system daemonset.apps/speaker 1 1 1 1 1 beta.kubernetes.io/os=linux 6m18s
NAMESPACE NAME READY UP-TO-DATE AVAILABLE AGE
kube-system deployment.apps/calico-kube-controllers 1/1 1 1 14m
kube-system deployment.apps/coredns 2/2 2 2 19m
metallb-system deployment.apps/controller 1/1 1 1 6m18s
NAMESPACE NAME DESIRED CURRENT READY AGE
kube-system replicaset.apps/calico-kube-controllers-c9784d67d 1 1 1 14m
kube-system replicaset.apps/coredns-f9fd979d6 2 2 2 19m
metallb-system replicaset.apps/controller-fb659dc8 1 1 1 6m18s
NAMESPACE NAME DATA AGE
kube-public configmap/cluster-info 2 19m
kube-system configmap/canal-config 6 14m
kube-system configmap/coredns 1 19m
kube-system configmap/extension-apiserver-authentication 6 19m
kube-system configmap/kube-proxy 2 19m
kube-system configmap/kubeadm-config 2 19m
kube-system configmap/kubelet-config-1.19 1 19m
metallb-system configmap/config 1 2m37s
```
# nginx-ingress-controller インストール
下記のマニフェストを利用します。BareMetal用だとNordPortになってしまうためちゃんと動きません。
https://kubernetes.github.io/ingress-nginx/deploy/#docker-for-mac
```
murata:~ $ kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v0.40.1/deploy/static/provider/cloud/deploy.yaml
namespace/ingress-nginx created
serviceaccount/ingress-nginx created
configmap/ingress-nginx-controller created
clusterrole.rbac.authorization.k8s.io/ingress-nginx created
clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx created
role.rbac.authorization.k8s.io/ingress-nginx created
rolebinding.rbac.authorization.k8s.io/ingress-nginx created
service/ingress-nginx-controller-admission created
service/ingress-nginx-controller created
deployment.apps/ingress-nginx-controller created
validatingwebhookconfiguration.admissionregistration.k8s.io/ingress-nginx-admission created
serviceaccount/ingress-nginx-admission created
clusterrole.rbac.authorization.k8s.io/ingress-nginx-admission created
clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx-admission created
role.rbac.authorization.k8s.io/ingress-nginx-admission created
rolebinding.rbac.authorization.k8s.io/ingress-nginx-admission created
job.batch/ingress-nginx-admission-create created
job.batch/ingress-nginx-admission-patch created
```
Ingressが産まれています。
Serviceの `ingress-nginx service/ingress-nginx-controller LoadBalancer 10.99.216.235 172.30.203.203 80:31980/TCP,443:32708/TCP 30s`
でEXTERNAL-IPにちゃんとIPアドレスが入っているかを確認してください。
```
murata:~ $ kubectl get all,ing,configmaps -A
Warning: extensions/v1beta1 Ingress is deprecated in v1.14+, unavailable in v1.22+; use networking.k8s.io/v1 Ingress
NAMESPACE NAME READY STATUS RESTARTS AGE
ingress-nginx pod/ingress-nginx-admission-create-hfs2f 0/1 Completed 0 30s
ingress-nginx pod/ingress-nginx-admission-patch-xdg52 0/1 Completed 1 30s
ingress-nginx pod/ingress-nginx-controller-59859f77c7-xnvcf 1/1 Running 0 30s
kube-system pod/calico-kube-controllers-c9784d67d-49g6t 1/1 Running 0 16m
kube-system pod/canal-wwmvf 2/2 Running 0 16m
kube-system pod/coredns-f9fd979d6-ccgm5 1/1 Running 0 22m
kube-system pod/coredns-f9fd979d6-ztwgs 1/1 Running 0 22m
kube-system pod/etcd-k8s-tmp.dev.deroris.local 1/1 Running 0 22m
kube-system pod/kube-apiserver-k8s-tmp.dev.deroris.local 1/1 Running 0 22m
kube-system pod/kube-controller-manager-k8s-tmp.dev.deroris.local 1/1 Running 0 22m
kube-system pod/kube-proxy-rfd7c 1/1 Running 0 22m
kube-system pod/kube-scheduler-k8s-tmp.dev.deroris.local 1/1 Running 0 22m
metallb-system pod/controller-fb659dc8-75t6h 1/1 Running 0 8m56s
metallb-system pod/speaker-jb8rb 1/1 Running 0 8m56s
NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
default service/kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 22m
ingress-nginx service/ingress-nginx-controller LoadBalancer 10.99.216.235 172.30.203.203 80:31980/TCP,443:32708/TCP 30s
ingress-nginx service/ingress-nginx-controller-admission ClusterIP 10.99.223.96 <none> 443/TCP 30s
kube-system service/kube-dns ClusterIP 10.96.0.10 <none> 53/UDP,53/TCP,9153/TCP 22m
NAMESPACE NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE
kube-system daemonset.apps/canal 1 1 1 1 1 kubernetes.io/os=linux 16m
kube-system daemonset.apps/kube-proxy 1 1 1 1 1 kubernetes.io/os=linux 22m
metallb-system daemonset.apps/speaker 1 1 1 1 1 beta.kubernetes.io/os=linux 8m56s
NAMESPACE NAME READY UP-TO-DATE AVAILABLE AGE
ingress-nginx deployment.apps/ingress-nginx-controller 1/1 1 1 30s
kube-system deployment.apps/calico-kube-controllers 1/1 1 1 16m
kube-system deployment.apps/coredns 2/2 2 2 22m
metallb-system deployment.apps/controller 1/1 1 1 8m56s
NAMESPACE NAME DESIRED CURRENT READY AGE
ingress-nginx replicaset.apps/ingress-nginx-controller-59859f77c7 1 1 1 30s
kube-system replicaset.apps/calico-kube-controllers-c9784d67d 1 1 1 16m
kube-system replicaset.apps/coredns-f9fd979d6 2 2 2 22m
metallb-system replicaset.apps/controller-fb659dc8 1 1 1 8m56s
NAMESPACE NAME COMPLETIONS DURATION AGE
ingress-nginx job.batch/ingress-nginx-admission-create 1/1 2s 30s
ingress-nginx job.batch/ingress-nginx-admission-patch 1/1 4s 30s
NAMESPACE NAME DATA AGE
ingress-nginx configmap/ingress-controller-leader-nginx 0 17s
ingress-nginx configmap/ingress-nginx-controller 0 30s
kube-public configmap/cluster-info 2 22m
kube-system configmap/canal-config 6 16m
kube-system configmap/coredns 1 22m
kube-system configmap/extension-apiserver-authentication 6 22m
kube-system configmap/kube-proxy 2 22m
kube-system configmap/kubeadm-config 2 22m
kube-system configmap/kubelet-config-1.19 1 22m
metallb-system configmap/config 1 5m15s
```
# 動作確認
試しにこんな感じのマニフェストを食わせる。
/apacheでapacheに、/nginxでnginxに行く。
※ Kubernetes 1.19からIngressAPIがGAとなりました。併せて書式がちょっと変わっているので注意してください。
https://qiita.com/tkusumi/items/ee1e100d34465ff2d764#whats-new-%E6%96%B0%E6%83%85%E5%A0%B1
↓は1.18までのものです。
```
cat <<EOF | kubectl apply -f -
# namespace
apiVersion: v1
kind: Namespace
metadata:
name: ns-test
---
# ingress
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: lb
namespace: ns-test
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
nginx.ingress.kubernetes.io/ssl-redirect: "false"
spec:
rules:
- host: ns-test.172.30.203.203.nip.io
http:
paths:
- path: /apache
backend:
serviceName: apache-svc
servicePort: 80
- path: /nginx
backend:
serviceName: nginx-svc
servicePort: 80
- path: /
backend:
serviceName: blackhole
servicePort: 80
---
# apache
apiVersion: v1
kind: Service
metadata:
name: apache-svc
namespace: ns-test
spec:
ports:
- port: 80
protocol: TCP
targetPort: 80
selector:
app: httpd
type: NodePort
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: httpd
namespace: ns-test
spec:
replicas: 1
selector:
matchLabels:
app: httpd
template:
metadata:
labels:
app: httpd
spec:
containers:
- image: httpd:alpine
name: httpd
ports:
- containerPort: 80
---
# nginx
apiVersion: v1
kind: Service
metadata:
name: nginx-svc
namespace: ns-test
spec:
ports:
- port: 80
protocol: TCP
targetPort: 80
selector:
app: nginx
type: NodePort
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx
namespace: ns-test
spec:
replicas: 1
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- image: nginx:alpine
name: nginx
ports:
- containerPort: 80
EOF
```
ServiceとIngressのところにIPアドレスが入っています。
```
murata:~ $ kubectl get all,ing -A
Warning: extensions/v1beta1 Ingress is deprecated in v1.14+, unavailable in v1.22+; use networking.k8s.io/v1 Ingress
NAMESPACE NAME READY STATUS RESTARTS AGE
ingress-nginx pod/ingress-nginx-admission-create-hfs2f 0/1 Completed 0 5m28s
ingress-nginx pod/ingress-nginx-admission-patch-xdg52 0/1 Completed 1 5m28s
ingress-nginx pod/ingress-nginx-controller-59859f77c7-xnvcf 1/1 Running 0 5m28s
kube-system pod/calico-kube-controllers-c9784d67d-49g6t 1/1 Running 0 21m
kube-system pod/canal-wwmvf 2/2 Running 0 21m
kube-system pod/coredns-f9fd979d6-ccgm5 1/1 Running 0 27m
kube-system pod/coredns-f9fd979d6-ztwgs 1/1 Running 0 27m
kube-system pod/etcd-k8s-tmp.dev.deroris.local 1/1 Running 0 27m
kube-system pod/kube-apiserver-k8s-tmp.dev.deroris.local 1/1 Running 0 27m
kube-system pod/kube-controller-manager-k8s-tmp.dev.deroris.local 1/1 Running 0 27m
kube-system pod/kube-proxy-rfd7c 1/1 Running 0 27m
kube-system pod/kube-scheduler-k8s-tmp.dev.deroris.local 1/1 Running 0 27m
metallb-system pod/controller-fb659dc8-75t6h 1/1 Running 0 13m
metallb-system pod/speaker-jb8rb 1/1 Running 0 13m
ns-test pod/httpd-6cd65b68c6-r4cpx 1/1 Running 0 2m17s
ns-test pod/nginx-7fb7fd49b4-wq7zl 1/1 Running 0 2m17s
NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
default service/kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 27m
ingress-nginx service/ingress-nginx-controller LoadBalancer 10.99.216.235 172.30.203.203 80:31980/TCP,443:32708/TCP 5m28s
ingress-nginx service/ingress-nginx-controller-admission ClusterIP 10.99.223.96 <none> 443/TCP 5m28s
kube-system service/kube-dns ClusterIP 10.96.0.10 <none> 53/UDP,53/TCP,9153/TCP 27m
ns-test service/apache-svc NodePort 10.103.168.10 <none> 80:32324/TCP 2m17s
ns-test service/nginx-svc NodePort 10.104.26.107 <none> 80:32278/TCP 2m17s
NAMESPACE NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE
kube-system daemonset.apps/canal 1 1 1 1 1 kubernetes.io/os=linux 21m
kube-system daemonset.apps/kube-proxy 1 1 1 1 1 kubernetes.io/os=linux 27m
metallb-system daemonset.apps/speaker 1 1 1 1 1 beta.kubernetes.io/os=linux 13m
NAMESPACE NAME READY UP-TO-DATE AVAILABLE AGE
ingress-nginx deployment.apps/ingress-nginx-controller 1/1 1 1 5m28s
kube-system deployment.apps/calico-kube-controllers 1/1 1 1 21m
kube-system deployment.apps/coredns 2/2 2 2 27m
metallb-system deployment.apps/controller 1/1 1 1 13m
ns-test deployment.apps/httpd 1/1 1 1 2m17s
ns-test deployment.apps/nginx 1/1 1 1 2m17s
NAMESPACE NAME DESIRED CURRENT READY AGE
ingress-nginx replicaset.apps/ingress-nginx-controller-59859f77c7 1 1 1 5m28s
kube-system replicaset.apps/calico-kube-controllers-c9784d67d 1 1 1 21m
kube-system replicaset.apps/coredns-f9fd979d6 2 2 2 27m
metallb-system replicaset.apps/controller-fb659dc8 1 1 1 13m
ns-test replicaset.apps/httpd-6cd65b68c6 1 1 1 2m17s
ns-test replicaset.apps/nginx-7fb7fd49b4 1 1 1 2m17s
NAMESPACE NAME COMPLETIONS DURATION AGE
ingress-nginx job.batch/ingress-nginx-admission-create 1/1 2s 5m28s
ingress-nginx job.batch/ingress-nginx-admission-patch 1/1 4s 5m28s
NAMESPACE NAME CLASS HOSTS ADDRESS PORTS AGE
ns-test ingress.extensions/lb <none> ns-test.172.30.203.203.nip.io 172.30.203.203 80 2m17s
```
ほかのPCから動作確認。
```
murata:~ $ curl http://ns-test.172.30.203.203.nip.io/apache
<html><body><h1>It works!</h1></body></html>
murata:~ $ curl http://ns-test.172.30.203.203.nip.io/nginx
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
```
apacheはapacheに、nginxはnginxにバランシングされています。（ヘッダーはIngressのものになってしまうのでペイロード部分で確認した
# まとめ
nginx-ingress-controllerはDaemonSetでありServiceでもIngressでもないので気を付けて。
複数台のノードでクラスタを組んでいる場合には特定のノード(router node)にIngressコントローラーを配置したかったりするかもしれません。その場合にはyamlファイルを編集すればOKみたいです。(Router Node)
SSL設定などもありますので、↑をそのまま使うのはまだまだかな。
それでは！