OKDとは
Red Hat OpenShiftのコミュニティ版
https://github.com/openshift/okd
環境情報
- OKD: 4.10.0-0.okd-2022-03-07-13121
- OS：Fedra CoreOS (FCOS) 35
- ノード：OpenStack(Victoria)のインスタンス
- プロキシ環境
構成図
作業ログ
インストール
以下を参照して、インストールを進める。
https://docs.okd.io/latest/installing/index.html
OKDには、以下2種類のインストール方法がある。
- Installer-provisioned infrastructure (IPI)：インストーラーがサーバー等のインフラも配備する方法
- User-provisioned infrastructure (UPI)：サーバー等のインフラはユーザーがあらかじめ用意する方法
更に、インフラごとにインストール手順が用意されている。
OpenStack上にOKDをインストールする手順もあるが、RHOSPが前提のため今回は使用しない。
- https://docs.okd.io/latest/installing/installing_openstack/preparing-to-install-on-openstack.html#choosing-an-method-to-install-ocp-on-openstack-installer-provisioned
- https://docs.okd.io/latest/installing/installing_openstack/preparing-to-install-on-openstack.html#choosing-an-method-to-install-ocp-on-openstack-user-provisioned
今回はインフラを問わない以下の方法でインストールを進める。
https://docs.okd.io/latest/installing/installing_platform_agnostic/installing-platform-agnostic.html
FCOSのイメージをopenstackに作成
fcosのqcow2は以下からダウンロードする。
https://getfedora.org/ja/coreos/download?tab=cloud_operators&stream=stable&arch=x86_64
[centos@rdo ~]$ export https_proxy=172.16.183.1:63128/
[centos@rdo ~]$ curl -O https://builds.coreos.fedoraproject.org/prod/streams/stable/builds/35.20220213.3.0/x86_64/fedora-coreos-35.20220213.3.0-openstack.x86_64.qcow2.xz
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 640M 100 640M 0 0 10.0M 0 0:01:03 0:01:03 --:--:-- 10.9M
ブラウザからダウンロードしたチェックサムファイルを配置する。（curlでダウンロードできなかったため）
[centos@rdo ~]$ ll fedora-coreos-35.20220213.3.0-openstack.x86_64.qcow2.xz*
-rw-rw-r-- 1 centos centos 671742324 Mar 14 14:20 fedora-coreos-35.20220213.3.0-openstack.x86_64.qcow2.xz
-rw-rw-r-- 1 centos centos 131 Mar 14 14:35 fedora-coreos-35.20220213.3.0-openstack.x86_64.qcow2.xz-CHECKSUM
署名とチェックサムを検証する。
[centos@rdo ~]$ curl -O https://builds.coreos.fedoraproject.org/prod/streams/stable/builds/35.20220213.3.0/x86_64/fedora-coreos-35.20220213.3.0-openstack.x86_64.qcow2.xz.sig
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 566 100 566 0 0 510 0 0:00:01 0:00:01 --:--:-- 510
[centos@rdo ~]$ ll fedora-coreos-35.20220213.3.0-openstack.x86_64.qcow2.xz.sig
-rw-rw-r-- 1 centos centos 566 Mar 14 14:38 fedora-coreos-35.20220213.3.0-openstack.x86_64.qcow2.xz.sig
[centos@rdo ~]$ curl https://getfedora.org/static/fedora.gpg | gpg --import
...
gpg: /home/centos/.gnupg/trustdb.gpg: trustdb created
gpg: key F55AD3FB5323552A: public key "Fedora (37) <fedora-37-primary@fedoraproject.org>" imported
gpg: key 999F7CBF38AB71F4: public key "Fedora (36) <fedora-36-primary@fedoraproject.org>" imported
gpg: key DB4639719867C58F: public key "Fedora (35) <fedora-35-primary@fedoraproject.org>" imported
gpg: key 1161AE6945719A39: public key "Fedora (34) <fedora-34-primary@fedoraproject.org>" imported
gpg: key 49FD77499570FF31: public key "Fedora (33) <fedora-33-primary@fedoraproject.org>" imported
gpg: key 7BB90722DBBDCF7C: public key "Fedora (iot 2019) <fedora-iot-2019@fedoraproject.org>" imported
gpg: key 8A3872BF3228467C: public key "Fedora (epel9) <epel@fedoraproject.org>" imported
gpg: key 21EA45AB2F86D6A1: public key "Fedora EPEL (8) <epel@fedoraproject.org>" imported
gpg: key 6A2FAEA2352C64E5: public key "Fedora EPEL (7) <epel@fedoraproject.org>" imported
gpg: Total number processed: 9
gpg: imported: 9
[centos@rdo ~]$ gpg --verify fedora-coreos-35.20220213.3.0-openstack.x86_64.qcow2.xz.sig fedora-coreos-35.20220213.3.0-openstack.x86_64.qcow2.xz
gpg: Signature made Tue 01 Mar 2022 12:37:37 PM JST
gpg: using RSA key 787EA6AE1147EEE56C40B30CDB4639719867C58F
gpg: Good signature from "Fedora (35) <fedora-35-primary@fedoraproject.org>" [unknown]
gpg: WARNING: This key is not certified with a trusted signature!
gpg: There is no indication that the signature belongs to the owner.
Primary key fingerprint: 787E A6AE 1147 EEE5 6C40 B30C DB46 3971 9867 C58F
[centos@rdo ~]$ sha256sum -c fedora-coreos-35.20220213.3.0-openstack.x86_64.qcow2.xz-CHECKSUM
fedora-coreos-35.20220213.3.0-openstack.x86_64.qcow2.xz: OK
[centos@rdo ~]$ unset https_proxy
解凍して、qcow2形式のファイルにする。
[centos@rdo ~(keystone_admin)]$ unxz fedora-coreos-35.20220213.3.0-openstack.x86_64.qcow2.xz
[centos@rdo ~(keystone_admin)]$ ll fedora-coreos-35.20220213.3.0-openstack.x86_64.qcow2
-rw-rw-r-- 1 centos centos 1590951936 Mar 14 14:20 fedora-coreos-35.20220213.3.0-openstack.x86_64.qcow2
openstackにイメージを登録
[centos@rdo ~(keystone_admin)]$ openstack image create \
> --disk-format qcow2 \
> --container-format bare \
> --public \
> --file ./fedora-coreos-35.20220213.3.0-openstack.x86_64.qcow2 \
> fcos-35
+------------------+---------------------------------------------------------------------------------------------------------------------------------------------+
| Field | Value |
+------------------+---------------------------------------------------------------------------------------------------------------------------------------------+
| container_format | bare |
| created_at | 2022-03-18T08:32:41Z |
| disk_format | qcow2 |
| file | /v2/images/193df775-6184-438e-973f-cd5cd76c56b6/file |
| id | 193df775-6184-438e-973f-cd5cd76c56b6 |
| min_disk | 0 |
| min_ram | 0 |
| name | fcos-35 |
| owner | c2b7c1d87c204b5aa1913603ae926ac8 |
| properties | os_hidden='False', owner_specified.openstack.md5='', owner_specified.openstack.object='images/fcos-35', owner_specified.openstack.sha256='' |
| protected | False |
| schema | /v2/schemas/image |
| status | queued |
| tags | |
| updated_at | 2022-03-18T08:32:41Z |
| visibility | public |
+------------------+---------------------------------------------------------------------------------------------------------------------------------------------+
フレーバーの作成
ドキュメントに記載の最小スペックを使用。
https://docs.okd.io/latest/installing/installing_platform_agnostic/installing-platform-agnostic.html#installation-minimum-resource-requirements_installing-platform-agnostic
[centos@rdo ~(keystone_admin)]$ openstack flavor create bootstrap_controlplane --ram 16384 --disk 100 --vcpus 4
+----------------------------+--------------------------------------+
| Field | Value |
+----------------------------+--------------------------------------+
| OS-FLV-DISABLED:disabled | False |
| OS-FLV-EXT-DATA:ephemeral | 0 |
| disk | 100 |
| id | 33f64211-3a83-412a-a0c6-2e70e254717c |
| name | bootstrap_controlplane |
| os-flavor-access:is_public | True |
| properties | |
| ram | 16384 |
| rxtx_factor | 1.0 |
| swap | |
| vcpus | 4 |
+----------------------------+--------------------------------------+
[centos@rdo ~(keystone_admin)]$ openstack flavor create compute --ram 8192 --disk 100 --vcpus 2
+----------------------------+--------------------------------------+
| Field | Value |
+----------------------------+--------------------------------------+
| OS-FLV-DISABLED:disabled | False |
| OS-FLV-EXT-DATA:ephemeral | 0 |
| disk | 100 |
| id | 194065c1-ea33-4766-b88b-e204796f35f4 |
| name | compute |
| os-flavor-access:is_public | True |
| properties | |
| ram | 8192 |
| rxtx_factor | 1.0 |
| swap | |
| vcpus | 2 |
+----------------------------+--------------------------------------+
セキュリティグループ
今回はdefaultのセキュリティグループに、全てのIPv4のingressを許可して使用する。
DNSの構築
以下を参考に、ubuntu 20.04のインスタンスにdnsmasqをインストールする。
ubuntu@util:~$ sudo apt -y install dnsmasq
...
Job for dnsmasq.service failed because the control process exited with error code.
See "systemctl status dnsmasq.service" and "journalctl -xe" for details.
invoke-rc.d: initscript dnsmasq, action "start" failed.
● dnsmasq.service - dnsmasq - A lightweight DHCP and caching DNS server
Loaded: loaded (/lib/systemd/system/dnsmasq.service; enabled; vendor preset: enabled)
Active: failed (Result: exit-code) since Fri 2022-03-18 05:54:59 UTC; 5ms ago
Process: 2236 ExecStartPre=/usr/sbin/dnsmasq --test (code=exited, status=0/SUCCESS)
Process: 2237 ExecStart=/etc/init.d/dnsmasq systemd-exec (code=exited, status=2)
Mar 18 05:54:59 util systemd[1]: Starting dnsmasq - A lightweight DHCP and caching DNS server...
Mar 18 05:54:59 util dnsmasq[2236]: dnsmasq: syntax check OK.
Mar 18 05:54:59 util dnsmasq[2237]: dnsmasq: failed to create listening socket for port 53: Address already in use
Mar 18 05:54:59 util systemd[1]: dnsmasq.service: Control process exited, code=exited, status=2/INVALIDARGUMENT
Mar 18 05:54:59 util dnsmasq[2237]: failed to create listening socket for port 53: Address already in use
Mar 18 05:54:59 util systemd[1]: dnsmasq.service: Failed with result 'exit-code'.
Mar 18 05:54:59 util dnsmasq[2237]: FAILED to start up
Mar 18 05:54:59 util systemd[1]: Failed to start dnsmasq - A lightweight DHCP and caching DNS server.
Processing triggers for libc-bin (2.31-0ubuntu9.2) ...
Processing triggers for systemd (245.4-4ubuntu3.15) ...
Processing triggers for man-db (2.9.1-1) ...
Processing triggers for dbus (1.12.16-2ubuntu2.1) ...
失敗した。
53番ポートが既に使用されているようである。
ubuntu@util:~$ ss -atnu|grep :53
udp ESTAB 0 0 127.0.0.1:36704 127.0.0.53:53
udp ESTAB 0 0 127.0.0.1:53269 127.0.0.53:53
udp UNCONN 0 0 127.0.0.53:53 0.0.0.0:*
tcp LISTEN 0 4096 127.0.0.53%lo:53 0.0.0.0:*
systemd-resolvedが、localhost(127.0.0.53)にスタブDNSサーバを立てるためのようである。
https://moguno.hatenablog.jp/entry/2018/06/17/155921
スタブDNSは不要なため、無効にする。
ubuntu@util:~$ sudo sed s/#DNSStubListener=yes/DNSStubListener=no/ /etc/systemd/resolved.conf
# This file is part of systemd.
#
# systemd is free software; you can redistribute it and/or modify it
# under the terms of the GNU Lesser General Public License as published by
# the Free Software Foundation; either version 2.1 of the License, or
# (at your option) any later version.
#
# Entries in this file show the compile time defaults.
# You can change settings by editing this file.
# Defaults can be restored by simply deleting this file.
#
# See resolved.conf(5) for details
[Resolve]
#DNS=
#FallbackDNS=
#Domains=
#LLMNR=no
#MulticastDNS=no
#DNSSEC=no
#DNSOverTLS=no
#Cache=no-negative
DNSStubListener=no
#ReadEtcHosts=yes
ubuntu@util:~$ systemctl status dnsmasq.service
● dnsmasq.service - dnsmasq - A lightweight DHCP and caching DNS server
Loaded: loaded (/lib/systemd/system/dnsmasq.service; enabled; vendor preset: enabled)
Active: active (running) since Fri 2022-03-18 06:35:16 UTC; 23s ago
Process: 2737 ExecStartPre=/usr/sbin/dnsmasq --test (code=exited, status=0/SUCCESS)
Process: 2738 ExecStart=/etc/init.d/dnsmasq systemd-exec (code=exited, status=0/SUCCESS)
Process: 2750 ExecStartPost=/etc/init.d/dnsmasq systemd-start-resolvconf (code=exited, status=0/SUCCESS)
Main PID: 2749 (dnsmasq)
Tasks: 1 (limit: 4726)
Memory: 1.0M
CGroup: /system.slice/dnsmasq.service
└─2749 /usr/sbin/dnsmasq -x /run/dnsmasq/dnsmasq.pid -u dnsmasq -7 /etc/dnsmasq.d,.dpkg-dist,.dpkg-old,.dpkg-new --local-service --trust-anchor>
Mar 18 06:35:16 util systemd[1]: Starting dnsmasq - A lightweight DHCP and caching DNS server...
Mar 18 06:35:16 util dnsmasq[2737]: dnsmasq: syntax check OK.
Mar 18 06:35:16 util dnsmasq[2749]: started, version 2.80 cachesize 150
Mar 18 06:35:16 util dnsmasq[2749]: DNS service limited to local subnets
Mar 18 06:35:16 util dnsmasq[2749]: compile time options: IPv6 GNU-getopt DBus i18n IDN DHCP DHCPv6 no-Lua TFTP conntrack ipset auth nettlehash DNSSEC loop->
Mar 18 06:35:16 util dnsmasq[2749]: reading /etc/resolv.conf
Mar 18 06:35:16 util dnsmasq[2749]: using nameserver 127.0.0.53#53
Mar 18 06:35:16 util dnsmasq[2749]: read /etc/hosts - 7 addresses
Mar 18 06:35:16 util systemd[1]: Started dnsmasq - A lightweight DHCP and caching DNS server.
dnsmasqが起動した。
/etc/dnsmasq.confに以下定義を追加する。
port=53
domain-needed
bogus-priv
no-poll
address=/apps.test.example.com/10.0.0.137
expand-hosts
domain=test.example.com
/etc/hostsに以下定義を追加する。
10.0.0.137 api.test.example.com api-int.test.example.com
10.0.0.21 bootstrap.test.example.com
10.0.0.31 master0.test.example.com
10.0.0.32 master1.test.example.com
10.0.0.33 master2.test.example.com
10.0.0.41 worker0.test.example.com
10.0.0.42 worker1.test.example.com
dnsmasqを再起動して、設定を反映
ubuntu@util:~$ sudo systemctl restart dnsmasq
ubuntu@util:~$
動作確認
ubuntu@util:~$ dig @10.0.0.137 master0.test.example.com
; <<>> DiG 9.16.1-Ubuntu <<>> @10.0.0.137 master0.test.example.com
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 39444
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;master0.test.example.com. IN A
;; ANSWER SECTION:
master0.test.example.com. 0 IN A 10.0.0.31
;; Query time: 0 msec
;; SERVER: 10.0.0.137#53(10.0.0.137)
;; WHEN: Thu Mar 24 05:43:18 UTC 2022
;; MSG SIZE rcvd: 69
ubuntu@util:~$ dig @10.0.0.137 -x 10.0.0.31
; <<>> DiG 9.16.1-Ubuntu <<>> @10.0.0.137 -x 10.0.0.31
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 7676
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;31.0.0.10.in-addr.arpa. IN PTR
;; ANSWER SECTION:
31.0.0.10.in-addr.arpa. 0 IN PTR master0.test.example.com.
;; Query time: 0 msec
;; SERVER: 10.0.0.137#53(10.0.0.137)
;; WHEN: Thu Mar 24 05:43:34 UTC 2022
;; MSG SIZE rcvd: 89
ロードバランサーの構築
以下を参考に、dnsmasqをインストールしたマシンにhaproxyをインストールする。
ubuntu@util:~$ sudo apt update
...
ubuntu@util:~$ sudo apt -y install haproxy
...
/etc/haproxy/haproxy.cfgに以下定義を追加
frontend stats
bind *:1936
mode http
log global
maxconn 10
stats enable
stats hide-version
stats refresh 30s
stats show-node
stats show-desc Stats for test cluster
stats auth admin:test
stats uri /stats
listen api-server-6443
bind *:6443
mode tcp
server bootstrap bootstrap.test.example.com:6443 check inter 1s backup
server master0 master0.test.example.com:6443 check inter 1s
server master1 master1.test.example.com:6443 check inter 1s
server master2 master2.test.example.com:6443 check inter 1s
listen machine-config-server-22623
bind *:22623
mode tcp
server bootstrap bootstrap.test.example.com:22623 check inter 1s backup
server master0 master0.test.example.com:22623 check inter 1s
server master1 master1.test.example.com:22623 check inter 1s
server master2 master2.test.example.com:22623 check inter 1s
listen ingress-router-443
bind *:443
mode tcp
balance source
server worker0 worker0.test.example.com:443 check inter 1s
server worker1 worker1.test.example.com:443 check inter 1s
listen ingress-router-80
bind *:80
mode tcp
balance source
server worker0 worker0.test.example.com:80 check inter 1s
server worker1 worker1.test.example.com:80 check inter 1s
設定を反映させるためにhaproxyを再起動
ubuntu@util:~$ sudo systemctl restart haproxy.service
ubuntu@util:~$
OKDインストールモジュールのダウンロード
ubuntu@util:~$ https_proxy=172.16.184.1:63128/ curl -LO https://github.com/openshift/okd/releases/download/4.10.0-0.okd-2022-03-07-131213/openshift-install-linux-4.10.0-0.okd-2022-03-07-131213.tar.gz
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 700 100 700 0 0 2811 0 --:--:-- --:--:-- --:--:-- 2811
100 124M 100 124M 0 0 8355k 0 0:00:15 0:00:15 --:--:-- 8069k
ubuntu@util:~$ https_proxy=172.16.184.1:63128/ curl -LO https://github.com/openshift/okd/releases/download/4.10.0-0.okd-2022-03-07-131213/sha256sum.txt
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 652 100 652 0 0 2608 0 --:--:-- --:--:-- --:--:-- 2608
100 710 100 710 0 0 1085 0 --:--:-- --:--:-- --:--:-- 3169
ubuntu@util:~$ sha256sum -c --ignore-missing sha256sum.txt
openshift-install-linux-4.10.0-0.okd-2022-03-07-131213.tar.gz: OK
ubuntu@util:~$ tar -xvf openshift-install-linux-4.10.0-0.okd-2022-03-07-131213.tar.gz
README.md
openshift-install
ubuntu@util:~$ ls
README.md openshift-install openshift-install-linux-4.10.0-0.okd-2022-03-07-131213.tar.gz sha256sum.txt
OCコマンドのインストール
ubuntu@util:~$ https_proxy=172.16.184.1:63128/ curl -O https://mirror.openshift.com/pub/openshift-v4/clients/oc/latest/linux/oc.tar.gz
...
ubuntu@util:~$ tar xvzf oc.tar.gz
README.md
oc
kubectl
ubuntu@util:~$ sudo mv kubectl /usr/local/bin/
ubuntu@util:~$ sudo mv oc /usr/local/bin/
ubuntu@util:~$ oc version
Client Version: 4.8.11
インストール用コンフィグファイルの作成
ubuntu@util:~$ mkdir okd
install-config.yamlの作成
apiVersion: v1
baseDomain: example.com
proxy:
httpProxy: http://172.16.184.1:63128/
httpsProxy: http://172.16.184.1:63128/
noProxy: example.com
compute:
- hyperthreading: Enabled
name: worker
replicas: 0
controlPlane:
hyperthreading: Enabled
name: master
replicas: 3
metadata:
name: test
networking:
clusterNetwork:
- cidr: 10.128.0.0/14
hostPrefix: 23
networkType: OVNKubernetes
serviceNetwork:
- 172.30.0.0/16
platform:
none: {}
pullSecret: '{"auths":{"fake":{"auth":"aWQ6cGFzcwo="}}}'
sshKey: 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAA...'
UPIの場合、computeのreplicasは0に指定する必要がある。
マニフェストの作成
ubuntu@util:~$ ./openshift-install create manifests --dir okd
INFO Consuming Install Config from target directory
WARNING Making control-plane schedulable by setting MastersSchedulable to true for Scheduler cluster settings
INFO Manifests created in: okd/manifests and okd/openshift
ubuntu@util:~$ tree okd
okd
├── manifests
│ ├── cluster-config.yaml
│ ├── cluster-dns-02-config.yml
│ ├── cluster-infrastructure-02-config.yml
│ ├── cluster-ingress-02-config.yml
│ ├── cluster-network-01-crd.yml
│ ├── cluster-network-02-config.yml
│ ├── cluster-proxy-01-config.yaml
│ ├── cluster-scheduler-02-config.yml
│ ├── cvo-overrides.yaml
│ ├── kube-cloud-config.yaml
│ ├── kube-system-configmap-root-ca.yaml
│ ├── machine-config-server-tls-secret.yaml
│ └── openshift-config-secret-pull-secret.yaml
└── openshift
├── 99_kubeadmin-password-secret.yaml
├── 99_openshift-cluster-api_master-user-data-secret.yaml
├── 99_openshift-cluster-api_worker-user-data-secret.yaml
├── 99_openshift-machineconfig_99-master-ssh.yaml
├── 99_openshift-machineconfig_99-worker-ssh.yaml
└── openshift-install-manifests.yaml
2 directories, 19 files
mastersSchedulableの確認
ubuntu@util:~$ cat okd/manifests/cluster-scheduler-02-config.yml
apiVersion: config.openshift.io/v1
kind: Scheduler
metadata:
creationTimestamp: null
name: cluster
spec:
mastersSchedulable: true
policy:
name: ""
status: {}
trueが指定されているので、falseに変更する。
ignitionファイルを作成
ubuntu@util:~$ ./openshift-install create ignition-configs --dir okd
INFO Consuming Master Machines from target directory
INFO Consuming Common Manifests from target directory
INFO Consuming OpenShift Install (Manifests) from target directory
INFO Consuming Openshift Manifests from target directory
INFO Consuming Worker Machines from target directory
INFO Ignition-Configs created in: okd and okd/auth
ubuntu@util:~$ tree okd/
okd/
├── auth
│ ├── kubeadmin-password
│ └── kubeconfig
├── bootstrap.ign
├── master.ign
├── metadata.json
└── worker.ign
1 directory, 6 files
FCOSのインストール
openstackでFCOSのインスタンスををインストールすると、ホスト名にはインスタンス名ではなくIPアドレスが設定される。
そのため、ignitionでホスト名を設定する。
参考：https://coreos.github.io/ignition/examples/#set-the-hostname
以下の定義をignitionに追加する。
{
"storage": {
"files": [{
"path": "/etc/hostname",
"mode": 420,
"overwrite": true,
"contents": { "source": "data:,bootstrap.test.example.com" }
}]
}
}
ネットワークのポート作成
あらかじめサブネットのDNSにdnsmasqを指定しておく必要がある。
[centos@rdo ~(keystone_admin)]$ openstack port create --network 4507ad6f-361d-4b4e-bc20-2327dca81756 --fixed-ip subnet=e87d8830-c075-4359-9907-182d7d17fadd,ip-address=10.0.0.21 bootstrap
+-------------------------+--------------------------------------------------------------------------+
| Field | Value |
+-------------------------+--------------------------------------------------------------------------+
| admin_state_up | UP |
| allowed_address_pairs | |
| binding_host_id | |
| binding_profile | |
| binding_vif_details | |
| binding_vif_type | unbound |
| binding_vnic_type | normal |
| created_at | 2022-03-23T04:42:50Z |
| data_plane_status | None |
| description | |
| device_id | |
| device_owner | |
| dns_assignment | None |
| dns_domain | None |
| dns_name | None |
| extra_dhcp_opts | |
| fixed_ips | ip_address='10.0.0.21', subnet_id='e87d8830-c075-4359-9907-182d7d17fadd' |
| id | 61fed2e0-1e82-46ab-a164-1bd1e0390aa6 |
| ip_allocation | None |
| mac_address | fa:16:3e:08:34:f8 |
| name | bootstrap |
| network_id | 4507ad6f-361d-4b4e-bc20-2327dca81756 |
| numa_affinity_policy | None |
| port_security_enabled | True |
| project_id | c2b7c1d87c204b5aa1913603ae926ac8 |
| propagate_uplink_status | None |
| qos_network_policy_id | None |
| qos_policy_id | None |
| resource_request | None |
| revision_number | 1 |
| security_group_ids | 4f975a13-0638-45ce-ad80-213be2ce275d |
| status | DOWN |
| tags | |
| trunk_details | None |
| updated_at | 2022-03-23T04:42:50Z |
+-------------------------+--------------------------------------------------------------------------+
以下を参考にインスタンスを作成
https://docs.fedoraproject.org/en-US/fedora-coreos/provisioning-openstack/#_launching_a_vm_instance
[centos@rdo ~(keystone_admin)]$ openstack server create \
> --port=61fed2e0-1e82-46ab-a164-1bd1e0390aa6 \
> --flavor=33f64211-3a83-412a-a0c6-2e70e254717c \
> --image=193df775-6184-438e-973f-cd5cd76c56b6 \
> --security-group 4f975a13-0638-45ce-ad80-213be2ce275d \
> --user-data ./ignition/bootstrap.ign \
> bootstrap.test.example.com
Request is too large. Larger than max_request_body_size (114688).<br /><br />
(HTTP 413) (Request-ID: req-94be941b-e04c-41fd-8421-6bfe27255dc9)
ignitionファイルのサイズが大き過ぎて失敗した。
novaのパラメータチューニングが必要そう。
ignitionファイルのサイズを確認
[centos@rdo ~(keystone_admin)]$ du ignition/bootstrap.ign -h
264K ignition/bootstrap.ign
nova.confを確認すると、以下のサイズがデフォルトのようである。
# The maximum body size for each request, in bytes (integer value)
# Deprecated group;name - DEFAULT;osapi_max_request_body_size
# Deprecated group;name - DEFAULT;max_request_body_size
#max_request_body_size=114688
以下のように変更する。
max_request_body_size=1000000
設定変更の反映
[centos@rdo ~(keystone_admin)]$ sudo systemctl restart httpd
インスタンス作成を再実施したところ、413のエラーは解消したようだが、今度は400のエラーが発生。
Invalid input for field/attribute user_data. Value: eyJp ... dfX0K' is too long (HTTP 400) (Request-ID: req-13954903-17bc-4da0-b16c-3a8f8ecd47b9)
以下によると、openstackのuser dataの上限は65,535B（base64エンコード後）とのこと。
https://docs.openstack.org/nova/latest/user/metadata.html#user-data
user dataのサイズ上限を上げることはできなそう。
ISOからのインストールであれば、ignitionをwebサーバー経由で渡せることができるため、bootstrapノードだけはISOからインストールする。
まずはdnsmasqとhaproxyをインストールしたマシンにnginxをインストールして、bootstrap nodeのignitionファイルを公開する。
ubuntu@util:~$ sudo apt install nginx
...
ubuntu@util:~$ systemctl status nginx.service
● nginx.service - A high performance web server and a reverse proxy server
Loaded: loaded (/lib/systemd/system/nginx.service; enabled; vendor preset: enabled)
Active: failed (Result: exit-code) since Thu 2022-03-24 03:05:39 UTC; 1h 18min ago
Docs: man:nginx(8)
Process: 35946 ExecStartPre=/usr/sbin/nginx -t -q -g daemon on; master_process on; (code=exited, status=0/SUCCESS)
Process: 35947 ExecStart=/usr/sbin/nginx -g daemon on; master_process on; (code=exited, status=1/FAILURE)
Mar 24 03:05:36 util systemd[1]: Starting A high performance web server and a reverse proxy server...
Mar 24 03:05:36 util nginx[35947]: nginx: [emerg] bind() to 0.0.0.0:80 failed (98: Address already in use)
Mar 24 03:05:37 util nginx[35947]: nginx: [emerg] bind() to 0.0.0.0:80 failed (98: Address already in use)
Mar 24 03:05:37 util nginx[35947]: nginx: [emerg] bind() to 0.0.0.0:80 failed (98: Address already in use)
Mar 24 03:05:38 util nginx[35947]: nginx: [emerg] bind() to 0.0.0.0:80 failed (98: Address already in use)
Mar 24 03:05:38 util nginx[35947]: nginx: [emerg] bind() to 0.0.0.0:80 failed (98: Address already in use)
Mar 24 03:05:39 util nginx[35947]: nginx: [emerg] still could not bind()
Mar 24 03:05:39 util systemd[1]: nginx.service: Control process exited, code=exited, status=1/FAILURE
Mar 24 03:05:39 util systemd[1]: nginx.service: Failed with result 'exit-code'.
Mar 24 03:05:39 util systemd[1]: Failed to start A high performance web server and a reverse proxy server.
ubuntu@util:~$ sudo lsof -i :80
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
haproxy 34102 haproxy 12u IPv4 1057064 0t0 TCP *:http (LISTEN)
ポート80番はhaproxyに使われているので、別のポートを使用する。
/etc/nginx/sites-enabled/defaultを以下のように修正
server {
#listen 80 default_server;
#listen [::]:80 default_server;
listen 60080 default_server;
listen [::]:60080 default_server;
ubuntu@util:~$ sudo systemctl start nginx.service
ubuntu@util:~$ systemctl status nginx.service
● nginx.service - A high performance web server and a reverse proxy server
Loaded: loaded (/lib/systemd/system/nginx.service; enabled; vendor preset: enabled)
Active: active (running) since Thu 2022-03-24 04:28:26 UTC; 5s ago
...
ubuntu@util:~$ curl http://127.0.0.1:60080/
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
body {
width: 35em;
margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif;
}
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>
/var/www/html/に公開するignitionファイルを配置する。
ubuntu@util:~$ sudo cp okd/bootstrap.ign /var/www/html/
ubuntu@util:~$ curl 127.0.0.1:60080/bootstrap.ign
ubuntu@util:~$ curl 127.0.0.1:60080/bootstrap.ign
<html>
<head><title>403 Forbidden</title></head>
<body>
<center><h1>403 Forbidden</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
ubuntu@util:~$ sudo ls -l /var/www/html/bootstrap.ign
-rw-r----- 1 root root 268862 Mar 24 04:34 /var/www/html/bootstrap.ign
ubuntu@util:~$ sudo chmod +r /var/www/html/bootstrap.ign
...
次に、FCOSのisoをopenstackに登録する。
isoのダウンロード
[centos@rdo ~]$ https_proxy=http://172.16.183.1:63128/ curl -O https://builds.coreos.fedoraproject.org/prod/streams/stable/builds/35.20220227.3.0/x86_64/fedor
a-coreos-35.20220227.3.0-live.x86_64.iso
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 772M 100 772M 0 0 9691k 0 0:01:21 0:01:21 --:--:-- 8166k
isoの検証（チェックサムとシグネイチャをダウンロードしておく）
[centos@rdo ~]$ gpg --verify fedora-coreos-35.20220227.3.0-live.x86_64.iso.sig fedora-coreos-35.20220227.3.0-live.x86_64.iso
gpg: Signature made Tue 15 Mar 2022 06:01:18 AM JST
gpg: using RSA key 787EA6AE1147EEE56C40B30CDB4639719867C58F
gpg: Good signature from "Fedora (35) <fedora-35-primary@fedoraproject.org>" [unknown]
gpg: WARNING: This key is not certified with a trusted signature!
gpg: There is no indication that the signature belongs to the owner.
Primary key fingerprint: 787E A6AE 1147 EEE5 6C40 B30C DB46 3971 9867 C58F
[centos@rdo ~]$ sha256sum -c fedora-coreos-35.20220227.3.0-live.x86_64.iso-CHECKSUM
fedora-coreos-35.20220227.3.0-live.x86_64.iso: OK
openstackにイメージを登録
[centos@rdo ~(keystone_admin)]$ openstack image create \
> --disk-format iso \
> --container-format bare \
> --public \
> --file ./fedora-coreos-35.20220227.3.0-live.x86_64.iso \
> fcos-35-iso
+------------------+-------------------------------------------------------------------------------------------------------------------------------------------------+
| Field | Value |
+------------------+-------------------------------------------------------------------------------------------------------------------------------------------------+
| container_format | bare |
| created_at | 2022-03-24T02:52:35Z |
| disk_format | iso |
| file | /v2/images/1410641e-92b9-465d-8ef2-2261a145c18e/file |
| id | 1410641e-92b9-465d-8ef2-2261a145c18e |
| min_disk | 0 |
| min_ram | 0 |
| name | fcos-35-iso |
| owner | c2b7c1d87c204b5aa1913603ae926ac8 |
| properties | os_hidden='False', owner_specified.openstack.md5='', owner_specified.openstack.object='images/fcos-35-iso', owner_specified.openstack.sha256='' |
| protected | False |
| schema | /v2/schemas/image |
| status | queued |
| tags | |
| updated_at | 2022-03-24T02:52:35Z |
| visibility | public |
+------------------+-------------------------------------------------------------------------------------------------------------------------------------------------+
以下を参考に、isoイメージからインスタンスを作成する。
https://docs.openstack.org/ja/user-guide/cli-nova-launch-instance-using-ISO-image.html
[centos@rdo ~(keystone_admin)]$ openstack server create \
> --network=openstack_net \
> --flavor=33f64211-3a83-412a-a0c6-2e70e254717c \
> --image=fcos-35-iso \
> --security-group 4f975a13-0638-45ce-ad80-213be2ce275d \
> bootstrap-temp
+-------------------------------------+---------------------------------------------------------------+
| Field | Value |
+-------------------------------------+---------------------------------------------------------------+
| OS-DCF:diskConfig | MANUAL |
| OS-EXT-AZ:availability_zone | |
| OS-EXT-SRV-ATTR:host | None |
| OS-EXT-SRV-ATTR:hypervisor_hostname | None |
| OS-EXT-SRV-ATTR:instance_name | |
| OS-EXT-STS:power_state | NOSTATE |
| OS-EXT-STS:task_state | scheduling |
| OS-EXT-STS:vm_state | building |
| OS-SRV-USG:launched_at | None |
| OS-SRV-USG:terminated_at | None |
| accessIPv4 | |
| accessIPv6 | |
| addresses | |
| adminPass | YS4nYfbgVug7 |
| config_drive | |
| created | 2022-03-24T06:23:12Z |
| flavor | bootstrap_controlplane (33f64211-3a83-412a-a0c6-2e70e254717c) |
| hostId | |
| id | ae8e2a3b-ded8-4987-b77d-3595a46ff9d2 |
| image | fcos-35-iso (1410641e-92b9-465d-8ef2-2261a145c18e) |
| key_name | None |
| name | bootstrap-temp |
| progress | 0 |
| project_id | c2b7c1d87c204b5aa1913603ae926ac8 |
| properties | |
| security_groups | name='4f975a13-0638-45ce-ad80-213be2ce275d' |
| status | BUILD |
| updated | 2022-03-24T06:23:12Z |
| user_id | 973053e4bb1f4bae95f1ad4264f73487 |
| volumes_attached | |
+-------------------------------------+---------------------------------------------------------------+
[centos@rdo ~(keystone_admin)]$ openstack volume create \
> --size 20 \
> --bootable bootstrap-temp
+---------------------+--------------------------------------+
| Field | Value |
+---------------------+--------------------------------------+
| attachments | [] |
| availability_zone | nova |
| bootable | false |
| consistencygroup_id | None |
| created_at | 2022-03-24T06:23:54.000000 |
| description | None |
| encrypted | False |
| id | 77b92d9a-82a9-4f58-a6ea-2bac028d8b33 |
| migration_status | None |
| multiattach | False |
| name | bootstrap-temp |
| properties | |
| replication_status | None |
| size | 20 |
| snapshot_id | None |
| source_volid | None |
| status | creating |
| type | iscsi |
| updated_at | None |
| user_id | 973053e4bb1f4bae95f1ad4264f73487 |
+---------------------+--------------------------------------+
[centos@rdo ~(keystone_admin)]$ openstack server add volume \
> bootstrap-temp \
> bootstrap-temp \
> --device /dev/vda
horizonのコンソールからfcosのインストールを進める。
[centos@rdo ~(keystone_admin)]$ openstack server delete bootstrap-temp
[centos@rdo ~(keystone_admin)]$ openstack image create --volume bootstrap-temp bootstrap
+---------------------+--------------------------------------+
| Field | Value |
+---------------------+--------------------------------------+
| container_format | bare |
| disk_format | raw |
| display_description | None |
| id | 77b92d9a-82a9-4f58-a6ea-2bac028d8b33 |
| image_id | 6ef09886-88e3-41be-bdf2-c8a8858a7e72 |
| image_name | bootstrap |
| protected | False |
| size | 20 |
| status | uploading |
| updated_at | 2022-03-24T06:29:12.000000 |
| visibility | shared |
| volume_type | iscsi |
+---------------------+--------------------------------------+
[centos@rdo ~(keystone_admin)]$ openstack server create \
> --port=61fed2e0-1e82-46ab-a164-1bd1e0390aa6 \
> --flavor=33f64211-3a83-412a-a0c6-2e70e254717c \
> --image=bootstrap \
> --security-group 4f975a13-0638-45ce-ad80-213be2ce275d \
> bootstrap.test.example.com
+-------------------------------------+---------------------------------------------------------------+
| Field | Value |
+-------------------------------------+---------------------------------------------------------------+
| OS-DCF:diskConfig | MANUAL |
| OS-EXT-AZ:availability_zone | |
| OS-EXT-SRV-ATTR:host | None |
| OS-EXT-SRV-ATTR:hypervisor_hostname | None |
| OS-EXT-SRV-ATTR:instance_name | |
| OS-EXT-STS:power_state | NOSTATE |
| OS-EXT-STS:task_state | scheduling |
| OS-EXT-STS:vm_state | building |
| OS-SRV-USG:launched_at | None |
| OS-SRV-USG:terminated_at | None |
| accessIPv4 | |
| accessIPv6 | |
| addresses | |
| adminPass | 7gRMHjrnVoWp |
| config_drive | |
| created | 2022-03-24T06:31:00Z |
| flavor | bootstrap_controlplane (33f64211-3a83-412a-a0c6-2e70e254717c) |
| hostId | |
| id | 45bc184f-b759-498b-91ae-22cbad006774 |
| image | bootstrap (6ef09886-88e3-41be-bdf2-c8a8858a7e72) |
| key_name | None |
| name | bootstrap.test.example.com |
| progress | 0 |
| project_id | c2b7c1d87c204b5aa1913603ae926ac8 |
| properties | |
| security_groups | name='4f975a13-0638-45ce-ad80-213be2ce275d' |
| status | BUILD |
| updated | 2022-03-24T06:31:00Z |
| user_id | 973053e4bb1f4bae95f1ad4264f73487 |
| volumes_attached | |
+-------------------------------------+---------------------------------------------------------------+
controlplaneの3ノードはqcow2のfcosから作成する。
[centos@rdo ~(keystone_admin)]$ openstack server create \
> --port=2068243d-0039-4cd3-9663-6a869f62ccef \
> --flavor=33f64211-3a83-412a-a0c6-2e70e254717c \
> --image=193df775-6184-438e-973f-cd5cd76c56b6 \
> --security-group 4f975a13-0638-45ce-ad80-213be2ce275d \
> --user-data ./ignition/master0.ign \
> master0.test.example.com
+-------------------------------------+---------------------------------------------------------------+
| Field | Value |
+-------------------------------------+---------------------------------------------------------------+
| OS-DCF:diskConfig | MANUAL |
| OS-EXT-AZ:availability_zone | |
| OS-EXT-SRV-ATTR:host | None |
| OS-EXT-SRV-ATTR:hypervisor_hostname | None |
| OS-EXT-SRV-ATTR:instance_name | |
| OS-EXT-STS:power_state | NOSTATE |
| OS-EXT-STS:task_state | scheduling |
| OS-EXT-STS:vm_state | building |
| OS-SRV-USG:launched_at | None |
| OS-SRV-USG:terminated_at | None |
| accessIPv4 | |
| accessIPv6 | |
| addresses | |
| adminPass | B8ung9mr6CoZ |
| config_drive | |
| created | 2022-03-24T06:31:34Z |
| flavor | bootstrap_controlplane (33f64211-3a83-412a-a0c6-2e70e254717c) |
| hostId | |
| id | 7fbd80e0-deec-4330-a394-6b7e0569f5f8 |
| image | fcos-35 (193df775-6184-438e-973f-cd5cd76c56b6) |
| key_name | None |
| name | master0.test.example.com |
| progress | 0 |
| project_id | c2b7c1d87c204b5aa1913603ae926ac8 |
| properties | |
| security_groups | name='4f975a13-0638-45ce-ad80-213be2ce275d' |
| status | BUILD |
| updated | 2022-03-24T06:31:34Z |
| user_id | 973053e4bb1f4bae95f1ad4264f73487 |
| volumes_attached | |
+-------------------------------------+---------------------------------------------------------------+
...
ブートストラッププロセスの完了を待つ
ubuntu@util:~$ ./openshift-install --dir okd wait-for bootstrap-complete --log-level=info
INFO Waiting up to 20m0s (until 6:52AM) for the Kubernetes API at https://api.test.example.com:6443...
INFO API v1.23.3-2003+e419edff267ffa-dirty up
INFO Waiting up to 30m0s (until 7:08AM) for bootstrapping to complete...
INFO It is now safe to remove the bootstrap resources
INFO Time elapsed: 30m2s
完了したようである。
controlplne nodeの確認
ubuntu@util:~$ echo "export KUBECONFIG=~/okd/auth/kubeconfig" >> ~/.bashrc
ubuntu@util:~$ source ~/.bashrc
ubuntu@util:~$ oc whoami
system:admin
ubuntu@util:~$ oc get csr
NAME AGE SIGNERNAME REQUESTOR REQUESTEDDURATION CONDITION
csr-f42fn 24m kubernetes.io/kubelet-serving system:node:master1.test.example.com <none> Approved,Issued
csr-k87lm 24m kubernetes.io/kube-apiserver-client-kubelet system:serviceaccount:openshift-machine-config-operator:node-bootstrapper <none> Approved,Issued
csr-lvnl9 23m kubernetes.io/kubelet-serving system:node:master0.test.example.com <none> Approved,Issued
csr-lwgzc 24m kubernetes.io/kube-apiserver-client-kubelet system:serviceaccount:openshift-machine-config-operator:node-bootstrapper <none> Approved,Issued
csr-mfzn7 23m kubernetes.io/kubelet-serving system:node:master2.test.example.com <none> Approved,Issued
csr-r9hhv 25m kubernetes.io/kube-apiserver-client-kubelet system:serviceaccount:openshift-machine-config-operator:node-bootstrapper <none> Approved,Issued
csr-rnm7k 24m kubernetes.io/kube-apiserver-client-kubelet system:serviceaccount:openshift-machine-config-operator:node-bootstrapper <none> Approved,Issued
csr-tqj6v 23m kubernetes.io/kube-apiserver-client-kubelet system:serviceaccount:openshift-machine-config-operator:node-bootstrapper <none> Approved,Issued
csr-w7f8f 23m kubernetes.io/kube-apiserver-client-kubelet system:serviceaccount:openshift-machine-config-operator:node-bootstrapper <none> Approved,Issued
system:openshift:openshift-authenticator-h2bpt 13m kubernetes.io/kube-apiserver-client system:serviceaccount:openshift-authentication-operator:authentication-operator <none> Approved,Issued
system:openshift:openshift-monitoring-zh4dr 12m kubernetes.io/kube-apiserver-client system:serviceaccount:openshift-monitoring:cluster-monitoring-operator <none> Approved,Issued
全てのCSRが承認されている。
worker nodeの追加
作成したignitionには、ssh公開鍵の設定が入っていないため、以下のような定義を追加しておく。
{
"passwd": {
"users": [
{
"name": "core",
"sshAuthorizedKeys": [
"ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD..."
]
}
]
},
}
[centos@rdo ~(keystone_admin)]$ openstack port create --network 4507ad6f-361d-4b4e-bc20-2327dca81756 --fixed-ip subnet=e87d8830-c075-4359-9907-182d7d17fadd,ip-address=10.0.0.41 worker0
+-------------------------+--------------------------------------------------------------------------+
| Field | Value |
+-------------------------+--------------------------------------------------------------------------+
| admin_state_up | UP |
| allowed_address_pairs | |
| binding_host_id | |
| binding_profile | |
| binding_vif_details | |
| binding_vif_type | unbound |
| binding_vnic_type | normal |
| created_at | 2022-03-24T07:23:03Z |
| data_plane_status | None |
| description | |
| device_id | |
| device_owner | |
| dns_assignment | None |
| dns_domain | None |
| dns_name | None |
| extra_dhcp_opts | |
| fixed_ips | ip_address='10.0.0.41', subnet_id='e87d8830-c075-4359-9907-182d7d17fadd' |
| id | 525f121f-3673-405e-bca5-3f120eede580 |
| ip_allocation | None |
| mac_address | fa:16:3e:d3:60:6f |
| name | worker0 |
| network_id | 4507ad6f-361d-4b4e-bc20-2327dca81756 |
| numa_affinity_policy | None |
| port_security_enabled | True |
| project_id | c2b7c1d87c204b5aa1913603ae926ac8 |
| propagate_uplink_status | None |
| qos_network_policy_id | None |
| qos_policy_id | None |
| resource_request | None |
| revision_number | 1 |
| security_group_ids | 4f975a13-0638-45ce-ad80-213be2ce275d |
| status | DOWN |
| tags | |
| trunk_details | None |
| updated_at | 2022-03-24T07:23:03Z |
+-------------------------+--------------------------------------------------------------------------+
[centos@rdo ~(keystone_admin)]$ openstack server create \
> --port=525f121f-3673-405e-bca5-3f120eede580 \
> --flavor=compute \
> --image=193df775-6184-438e-973f-cd5cd76c56b6 \
> --security-group 4f975a13-0638-45ce-ad80-213be2ce275d \
> --user-data ./ignition/worker0.ign \
> worker0.test.example.com
+-------------------------------------+------------------------------------------------+
| Field | Value |
+-------------------------------------+------------------------------------------------+
| OS-DCF:diskConfig | MANUAL |
| OS-EXT-AZ:availability_zone | |
| OS-EXT-SRV-ATTR:host | None |
| OS-EXT-SRV-ATTR:hypervisor_hostname | None |
| OS-EXT-SRV-ATTR:instance_name | |
| OS-EXT-STS:power_state | NOSTATE |
| OS-EXT-STS:task_state | scheduling |
| OS-EXT-STS:vm_state | building |
| OS-SRV-USG:launched_at | None |
| OS-SRV-USG:terminated_at | None |
| accessIPv4 | |
| accessIPv6 | |
| addresses | |
| adminPass | BZJYr7YPt787 |
| config_drive | |
| created | 2022-03-24T07:23:45Z |
| flavor | compute (194065c1-ea33-4766-b88b-e204796f35f4) |
| hostId | |
| id | 5e8f6fbc-34d3-45ce-a88a-42fe926ee006 |
| image | fcos-35 (193df775-6184-438e-973f-cd5cd76c56b6) |
| key_name | None |
| name | worker0.test.example.com |
| progress | 0 |
| project_id | c2b7c1d87c204b5aa1913603ae926ac8 |
| properties | |
| security_groups | name='4f975a13-0638-45ce-ad80-213be2ce275d' |
| status | BUILD |
| updated | 2022-03-24T07:23:46Z |
| user_id | 973053e4bb1f4bae95f1ad4264f73487 |
| volumes_attached | |
+-------------------------------------+------------------------------------------------+
しばらくするとpendingのCSRが作成された。
ubuntu@util:~$ oc get csr
NAME AGE SIGNERNAME REQUESTOR REQUESTEDDURATION CONDITION
csr-7v7xj 11s kubernetes.io/kube-apiserver-client-kubelet system:serviceaccount:openshift-machine-config-operator:node-bootstrapper <none> Pending
...
ubuntu@util:~$ oc get node
NAME STATUS ROLES AGE VERSION
master0.test.example.com Ready master 43m v1.23.3+759c22b
master1.test.example.com Ready master 44m v1.23.3+759c22b
master2.test.example.com Ready master 43m v1.23.3+759c22b
ubuntu@util:~$ oc adm certificate approve csr-7v7xj
certificatesigningrequest.certificates.k8s.io/csr-7v7xj approved
ubuntu@util:~$ oc get csr csr-7v7xj
NAME AGE SIGNERNAME REQUESTOR REQUESTEDDURATION CONDITION
csr-7v7xj 116s kubernetes.io/kube-apiserver-client-kubelet system:serviceaccount:openshift-machine-config-operator:node-bootstrapper <none> Approved,Issued
ubuntu@util:~$ oc get node
NAME STATUS ROLES AGE VERSION
master0.test.example.com Ready master 44m v1.23.3+759c22b
master1.test.example.com Ready master 45m v1.23.3+759c22b
master2.test.example.com Ready master 44m v1.23.3+759c22b
worker0.test.example.com NotReady worker 5s v1.23.3+759c22b
...
ubuntu@util:~$ oc get node
NAME STATUS ROLES AGE VERSION
master0.test.example.com Ready master 50m v1.23.3+759c22b
master1.test.example.com Ready master 51m v1.23.3+759c22b
master2.test.example.com Ready master 50m v1.23.3+759c22b
worker0.test.example.com Ready worker 5m22s v1.23.3+759c22b
同じようにworker nodeをもう一台追加。
ubuntu@util:~$ oc get node
NAME STATUS ROLES AGE VERSION
master0.test.example.com Ready master 7d18h v1.23.3+759c22b
master1.test.example.com Ready master 7d18h v1.23.3+759c22b
master2.test.example.com Ready master 7d18h v1.23.3+759c22b
worker0.test.example.com Ready worker 7d17h v1.23.3+759c22b
worker1.test.example.com Ready worker 4m25s v1.23.3+759c22b
webコンソールへのアクセス
URLの確認
ubuntu@util:~$ oc whoami --show-console
https://console-openshift-console.apps.test.example.com
ブラウザを実行するマシンに以下のhostsを設定
172.16.184.197 console-openshift-console.apps.test.example.com
172.16.184.197はhaproxyをインストールしたマシンに付与したfloating IP
ブラウザからアクセスすると、以下にリダイレクトされた。
https://oauth-openshift.apps.test.example.com/oauth/authorize?client_id=console&redirect_uri=https%3A%2F%2Fconsole-openshift-console.apps.test.example.com%2Fauth%2Fcallback&response_type=code&scope=user%3Afull&state=4e6e8409
以下をhostsに追加して再トライ。
172.16.184.197 oauth-openshift.apps.test.example.com
以下のようにログイン画面が開いた。
初期ユーザーとしてkubeadminが使えるようだが、パスワードがわからない。
インストールディレクトリのauth/kubeadmin-passwordに記載された文字列を入れたらログインできた！
アプリのデプロイ
ubuntu@util:~$ oc new-app docker.io/nginx
--> Found container image 12766a6 (2 days old) from docker.io for "docker.io/nginx"
* An image stream tag will be created as "nginx:latest" that will track this image
--> Creating resources ...
imagestream.image.openshift.io "nginx" created
deployment.apps "nginx" created
service "nginx" created
--> Success
WARNING: No container image registry has been configured with the server. Automatic builds and deployments may not function.
Application is not exposed. You can expose services to the outside world by executing one or more of the commands below:
'oc expose service/nginx'
Run 'oc status' to view your app.
ubuntu@util:~$ oc get pod
NAME READY STATUS RESTARTS AGE
nginx-6778f7495f-qwn8f 1/1 Running 0 34s
クラスタ外にserviceを公開する。
ubuntu@util:~$ oc expose service nginx
route.route.openshift.io/nginx exposed
ubuntu@util:~$ oc get routes.route.openshift.io
NAME HOST/PORT PATH SERVICES PORT TERMINATION WILDCARD
nginx nginx-default.apps.test.example.com nginx 80-tcp None
ubuntu@util:~$ curl http://nginx-default.apps.test.example.com
curl: (6) Could not resolve host: nginx-default.apps.test.example.com
ubuntu@util:~$ curl http://nginx-default.apps.test.example.com --resolv nginx-default.apps.test.example.com:80:127.0.0.1
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>
クラスタ外からopenshift上にデプロイしたnginxにアクセスすることができた！
本記事の内容は以上です。