Go to Qiita Advent Calendar 2024 Top
LoginSignup
3
2

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?

More than 1 year has passed since last update.

@nakkoh

TripleOでRDOをインストールする

Last updated at Posted at 2022-05-19

環境情報

マシンのスペック

  • CPUコア数：32
  • RAM容量：384 GB
  • ローカルストレージ（SSD）容量： 1.92 TB（RAID1）

ソフトウェア

  • OpenStack Yoga
  • python3-tripleoclient 17.1.1
  • CentOS Stream 8

ネットワーク構成

tripleo_network.png

ネットワーク種別 VLAN Type VLAN ID サブネット
踏み台からundercloudホストに
SSH接続する用		 tag 11 172.16.11.0/24
IPMI untag 1 :zipper_mouth:
Provisioning untag 10 172.16.10.0/24
Storage tag 12 172.16.12.0/24
StorageMgmt tag 13 172.16.13.0/24
InternalApi tag 14 172.16.14.0/24
Tenant tag 15 172.16.15.0/24
External tag 16 172.16.16.0/24

外のネットワークに出るためにはproxyを経由する必要がある。

作業ログ

Undercloudのインストール

以下を参考にインストールを進める。
https://docs.openstack.org/tripleo-docs/latest/install/index.html

tripleoによるインストールには大きく以下の4つの構成があるようである。

本記事では、Baremetal Environmentを試す。

undercloudをインストールするマシンにcentos 8 stream x86_64をインストールする。（手順は省略）

OSインストール後の事前準備

[centos@undercloud ~]$ cat /etc/os-release 
NAME="CentOS Stream"
VERSION="8"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="8"
PLATFORM_ID="platform:el8"
PRETTY_NAME="CentOS Stream 8"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:centos:centos:8"
HOME_URL="https://centos.org/"
BUG_REPORT_URL="https://bugzilla.redhat.com/"
REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
[centos@undercloud ~]$ localectl status
   System Locale: LANG=ja_JP.UTF-8
       VC Keymap: jp
      X11 Layout: jp

packstackでインストールする場合はロケールが英語であることが必須だが、tripleoの場合は不明。
念のため、英語に変えておく。

[root@undercloud ~]# localectl list-locales

...

en_US.utf8

...

[centos@undercloud ~]$ sudo localectl set-locale LANG=en_US.utf8
[centos@undercloud ~]$ localectl status
   System Locale: LANG=en_US.utf8
       VC Keymap: jp
      X11 Layout: jp

環境変数のproxy設定

/etc/environmentを以下のように編集。

export http_proxy=http://172.16.11.1:63128/
export https_proxy=http://172.16.11.1:63128/
export HTTP_PROXY=http://172.16.11.1:63128/
export HTTPS_PROXY=http://172.16.11.1:63128/
export no_proxy=localhost,127.0.0.1

dnfのproxy設定

[centos@undercloud ~]$ sudo vi /etc/dnf/dnf.conf
[centos@undercloud ~]$ cat /etc/dnf/dnf.conf 
[main]

...

proxy=http://172.16.11.1:63128/

ntpの設定

[centos@undercloud ~]$ dnf list --installed |grep chrony
[centos@undercloud ~]$ sudo dnf install chrony

...
[centos@undercloud ~]$ sudo vim /etc/chrony.conf
[centos@undercloud ~]$ cat /etc/chrony.conf

...

#pool 2.centos.pool.ntp.org iburst
pool <ntp server host> iburst

...
[centos@undercloud ~]$ sudo systemctl enable --now chronyd
[centos@undercloud ~]$ chronyc sources
MS Name/IP address         Stratum Poll Reach LastRx Last sample               
===============================================================================
^* <ntp server host>>     4   6    17    41  -1221ns[  +15us] +/-  101ms

NTPの設定はundercloudのインストールコマンドで実施されるようなので、ここで設定する必要はなさそう。。

stackユーザーを作成

[centos@undercloud ~]$ sudo useradd stack
[centos@undercloud ~]$ sudo passwd stack
Changing password for user stack.
New password: 
BAD PASSWORD: The password fails the dictionary check - it is based on a dictionary word
Retype new password: 
passwd: all authentication tokens updated successfully.
[centos@undercloud ~]$ echo "stack ALL=(root) NOPASSWD:ALL" | sudo tee -a /etc/sudoers.d/stack
stack ALL=(root) NOPASSWD:ALL
[centos@undercloud ~]$ sudo chmod 0440 /etc/sudoers.d/stack

undercloudのデプロイ

以下を参考に進める。
https://docs.openstack.org/project-deploy-guide/tripleo-docs/latest/deployment/install_undercloud.html

stackユーザーに変わる。

[centos@undercloud ~]$ su stack
Password: 
[stack@undercloud centos]$ cd
[stack@undercloud ~]$

ホスト名の確認

[stack@undercloud ~]$ echo $HOSTNAME
undercloud.try-tripleo.example.com

/etc/hostsに追加

[stack@undercloud ~]$ cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4 undercloud.try-tripleo.example.com undercloud
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6

python3のインストール

[stack@undercloud ~]$ python3 -V
bash: python3: command not found
[stack@undercloud ~]$ sudo dnf install python3-pip

...

[stack@undercloud ~]$ python3 -V
Python 3.6.8
[stack@undercloud ~]$ pip3 -V
pip 9.0.3 from /usr/lib/python3.6/site-packages (python 3.6)
[root@undercloud ~]$ sudo python3 -m pip install --upgrade pip
WARNING: Running pip install with root privileges is generally not a good idea. Try `__main__.py install --user` instead.
Collecting pip
  Downloading https://files.pythonhosted.org/packages/a4/6d/6463d49a933f547439d6b5b98b46af8742cc03ae83543e4d7688c2420f8b/pip-21.3.1-py3-none-any.whl (1.7MB)
    100% |████████████████████████████████| 1.7MB 907kB/s 
Installing collected packages: pip
Successfully installed pip-21.3.1
[stack@undercloud ~]$ pip -V
pip 21.3.1 from /usr/local/lib/python3.6/site-packages/pip (python 3.6)

python-tripleo-reposRPMをインストール

以下からRPMをダウンロードする。
https://trunk.rdoproject.org/centos8/component/tripleo/current/

[root@undercloud ~]$ sudo dnf install -y https://trunk.rdoproject.org/centos8/component/tripleo/current/python3-tripleo-repos-0.1.1-0.20220214194848.cbbdde6.el8.noarch.rpm

...

Complete!

リポジトリのインストール

[stack@undercloud ~]$ sudo -E tripleo-repos current-tripleo-dev ceph
Installed repo delorean-master-testing to /etc/yum.repos.d/delorean-master-testing.repo
Installed repo delorean-current-tripleo to /etc/yum.repos.d/delorean-current-tripleo.repo
Installed repo delorean to /etc/yum.repos.d/delorean.repo
Installed repo tripleo-centos-ceph-pacific to /etc/yum.repos.d/tripleo-centos-ceph-pacific.repo
Installed repo tripleo-centos-highavailability to /etc/yum.repos.d/tripleo-centos-highavailability.repo
Installed repo tripleo-centos-powertools to /etc/yum.repos.d/tripleo-centos-powertools.repo
Cache was expired
27 files removed

tripleo cliのインストール

[stack@undercloud ~]$ sudo dnf install -y python*-tripleoclient

...


Complete!

コンテナイメージの定義を生成

(undercloud) [stack@undercloud ~]$ openstack tripleo container image prepare default \
>   --local-push-destination \
>   --output-env-file ~/containers-prepare-parameter.yaml

...

undercloud.confをコピー

[stack@undercloud ~]$ cp /usr/share/python-tripleoclient/undercloud.conf.sample ~/undercloud.conf
[stack@undercloud ~]$ ll undercloud.conf 
-rw-r--r--. 1 stack stack 15897 Apr  6 06:41 undercloud.conf

環境に合わせて、編集

[stack@undercloud ~]$ diff /usr/share/python-tripleoclient/undercloud.conf.sample undercloud.conf 
52c52
< #container_images_file = <None>
---
> container_images_file = /home/stack/containers-prepare-parameter.yaml
223c223
< #local_interface = eth1
---
> local_interface = eno1
230c230
< #local_ip = 192.168.24.1/24
---
> local_ip = 172.16.10.1/24
324c324
< #undercloud_admin_host = 192.168.24.3
---
> undercloud_admin_host = 172.16.10.3
338c338
< #undercloud_hostname = <None>
---
> undercloud_hostname = undercloud.try-tripleo.example.com
348c348
< #undercloud_nameservers =
---
> undercloud_nameservers = xx.xx.xx.xx
351c351
< #undercloud_ntp_servers = 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org
---
> undercloud_ntp_servers = xxx.yyy.com
356c356
< #undercloud_public_host = 192.168.24.2
---
> undercloud_public_host = 172.16.10.2
365c365
< #undercloud_timezone = <None>
---
> undercloud_timezone = UTC
377c377
< #cidr = 192.168.24.0/24
---
> cidr = 172.16.10.0/24
382c382
< #dhcp_end = 192.168.24.24
---
> dhcp_end = 172.16.10.24
392c392
< #dhcp_start = 192.168.24.5
---
> dhcp_start = 172.16.10.5
403c403
< #gateway = 192.168.24.1
---
> gateway = 172.16.10.1
412c412
< #host_routes = [{destination: 10.10.10.0/24, nexthop: 192.168.24.1}]
---
> host_routes = [{destination: 0.0.0.0/0, nexthop: 172.16.11.1}]
419c419
< #inspection_iprange = 192.168.24.100,192.168.24.120
---
> inspection_iprange = 172.16.10.100,172.16.10.120

no_proxyに以下を追加する。

.localdomain,172.16.10.1,172.16.10.2,172.16.10.3,172.16.10.5,172.16.10.6,172.16.10.7,172.16.10.8,172.16.10.9,172.16.10.10,172.16.10.11,172.16.10.12,172.16.10.13,172.16.10.14,172.16.10.15,172.16.10.16,172.16.10.17,172.16.10.18,172.16.10.19,172.16.10.20,172.16.10.21,172.16.10.22,172.16.10.23,172.16.10.24
  • .localdomainはundercloudのホストにデプロイされるローカルレジストリへの通信に必要
  • 172.16.10.5 ~ 172.16.10.24は、DHCPで各overcloudノードに割り当てるIPアドレス

undercloudのインストール

[stack@undercloud ~]$ openstack undercloud install

...

########################################################

Deployment successful!

########################################################

Writing the stack virtual update mark file /var/lib/tripleo-heat-installer/update_mark_undercloud
Erase is control-H (^H).

##########################################################

The Undercloud has been successfully installed.

Useful files:

Password file is at /home/stack/tripleo-undercloud-passwords.yaml
The stackrc file is at ~/stackrc

Use these files to interact with OpenStack services, and
ensure they are secured.

##########################################################

成功したようである。

しかし、作成されたコンテナの内の2つはヘルスチェックに失敗していた。

(undercloud) [stack@undercloud ~]$ sudo podman ps
CONTAINER ID  IMAGE                                                                                                   COMMAND               CREATED             STATUS                        PORTS       NAMES
f584173b6c03  undercloud.ctlplane.localdomain:8787/tripleomaster/openstack-haproxy:current-tripleo                    kolla_start           6 minutes ago       Up 5 minutes ago                          haproxy
0435ce07ff3b  undercloud.ctlplane.localdomain:8787/tripleomaster/openstack-rabbitmq:current-tripleo                   kolla_start           6 minutes ago       Up 5 minutes ago (healthy)                rabbitmq
6d4c2a094e98  undercloud.ctlplane.localdomain:8787/tripleomaster/openstack-mariadb:current-tripleo                    kolla_start           5 minutes ago       Up 5 minutes ago (healthy)                mysql
c5a161774275  undercloud.ctlplane.localdomain:8787/tripleomaster/openstack-iscsid:current-tripleo                     kolla_start           4 minutes ago       Up 4 minutes ago (healthy)                iscsid
b42fef268805  undercloud.ctlplane.localdomain:8787/tripleomaster/openstack-neutron-server:current-tripleo             kolla_start           4 minutes ago       Up 2 minutes ago (healthy)                neutron_api
1b27d2234699  undercloud.ctlplane.localdomain:8787/tripleomaster/openstack-cron:current-tripleo                       kolla_start           4 minutes ago       Up 3 minutes ago (healthy)                logrotate_crond
09ead95c1834  undercloud.ctlplane.localdomain:8787/tripleomaster/openstack-neutron-l3-agent:current-tripleo           kolla_start           4 minutes ago       Up 2 minutes ago (healthy)                neutron_l3_agent
3cc0bd4628a7  undercloud.ctlplane.localdomain:8787/tripleomaster/openstack-neutron-openvswitch-agent:current-tripleo  kolla_start           4 minutes ago       Up 2 minutes ago (healthy)                neutron_ovs_agent
5046be1c99ed  undercloud.ctlplane.localdomain:8787/tripleomaster/openstack-neutron-dhcp-agent:current-tripleo         kolla_start           4 minutes ago       Up 2 minutes ago (healthy)                neutron_dhcp
101a1d88a392  undercloud.ctlplane.localdomain:8787/tripleomaster/openstack-ironic-api:current-tripleo                 kolla_start           4 minutes ago       Up 3 minutes ago (healthy)                ironic_api
42e460aac772  undercloud.ctlplane.localdomain:8787/tripleomaster/openstack-ironic-neutron-agent:current-tripleo       kolla_start           4 minutes ago       Up 3 minutes ago (healthy)                ironic_neutron_agent
e0cd730170af  undercloud.ctlplane.localdomain:8787/tripleomaster/openstack-ironic-conductor:current-tripleo           kolla_start           4 minutes ago       Up 3 minutes ago (unhealthy)              ironic_conductor
02fbbe91743f  undercloud.ctlplane.localdomain:8787/tripleomaster/openstack-ironic-pxe:current-tripleo                 kolla_start           4 minutes ago       Up 3 minutes ago (unhealthy)              ironic_pxe_tftp
800e6a0588ab  undercloud.ctlplane.localdomain:8787/tripleomaster/openstack-ironic-pxe:current-tripleo                 kolla_start           4 minutes ago       Up 3 minutes ago (healthy)                ironic_pxe_http
bdbd5dd139fe  undercloud.ctlplane.localdomain:8787/tripleomaster/openstack-ironic-inspector:current-tripleo           kolla_start           4 minutes ago       Up 3 minutes ago (healthy)                ironic_inspector
79478e92847f  undercloud.ctlplane.localdomain:8787/tripleomaster/openstack-ironic-inspector:current-tripleo           kolla_start           4 minutes ago       Up 3 minutes ago (healthy)                ironic_inspector_dnsmasq
b630d9974189  undercloud.ctlplane.localdomain:8787/tripleomaster/openstack-neutron-dhcp-agent:current-tripleo         /usr/sbin/dnsmasq...  About a minute ago  Up About a minute ago                     neutron-dnsmasq-qdhcp-a835ded8-0a20-4d31-9f4d-4cc23ef7debc

原因を突きとめられなかったので、とりあえず先に進む。

overcloudのインストール

以下を参考に進める。
https://docs.openstack.org/project-deploy-guide/tripleo-docs/latest/deployment/install_overcloud.html

overcloud用イメージのビルド

[stack@undercloud ~]$ source stackrc
(undercloud) [stack@undercloud ~]$ cat /usr/share/openstack-tripleo-common/image-yaml/overcloud-images-centos8.yaml
disk_images:
  -
    imagename: overcloud-full
    type: qcow2
    distro: centos
    release: 8
    elements:
      - selinux-permissive
      - interface-names
    packages:
      - kernel-modules-extra
    environment:
      FS_TYPE: xfs
    options:
      - "--mkfs-options '-s size=4096'"
  -
    imagename: ironic-python-agent
    type: qcow2
    distro: centos
    release: 8
    elements:
      - selinux-permissive
      - interface-names
(undercloud) [stack@undercloud ~]$ export OS_YAML="/usr/share/openstack-tripleo-common/image-yaml/overcloud-images-centos8.yaml"
(undercloud) [stack@undercloud ~]$ sudo dnf install -y https://trunk.rdoproject.org/centos8/component/tripleo/current/python3-tripleo-repos-0.1.1-0.20220214194848.cbbdde6
.el8.noarch.rpm
Last metadata expiration check: 2:07:17 ago on Mon 11 Apr 2022 11:42:17 PM UTC.
python3-tripleo-repos-0.1.1-0.20220214194848.cbbdde6.el8.noarch.rpm                                                           38 kB/s |  60 kB     00:01    
Package python3-tripleo-repos-0.1.1-0.20220214194848.cbbdde6.el8.noarch is already installed.
Dependencies resolved.
Nothing to do.
Complete!
(undercloud) [stack@undercloud ~]$ sudo -E tripleo-repos current-tripleo-dev ceph
Removed old repo "/etc/yum.repos.d/delorean-master-testing.repo"
Removed old repo "/etc/yum.repos.d/delorean-current-tripleo.repo"
Removed old repo "/etc/yum.repos.d/delorean.repo"
Removed old repo "/etc/yum.repos.d/tripleo-centos-ceph-pacific.repo"
Removed old repo "/etc/yum.repos.d/tripleo-centos-highavailability.repo"
Removed old repo "/etc/yum.repos.d/tripleo-centos-powertools.repo"
Installed repo delorean-master-testing to /etc/yum.repos.d/delorean-master-testing.repo
Installed repo delorean-current-tripleo to /etc/yum.repos.d/delorean-current-tripleo.repo
Installed repo delorean to /etc/yum.repos.d/delorean.repo
Installed repo tripleo-centos-ceph-pacific to /etc/yum.repos.d/tripleo-centos-ceph-pacific.repo
Installed repo tripleo-centos-highavailability to /etc/yum.repos.d/tripleo-centos-highavailability.repo
Installed repo tripleo-centos-powertools to /etc/yum.repos.d/tripleo-centos-powertools.repo
Cache was expired
245 files removed

cephは使わないつもりなので、上記手順は不要かも。

(undercloud) [stack@undercloud ~]$ export DIB_YUM_REPO_CONF="/etc/yum.repos.d/delorean* /etc/yum.repos.d/tripleo-centos-*"
(undercloud) [stack@undercloud ~]$ export STABLE_RELEASE="victoria"
(undercloud) [stack@undercloud ~]$ openstack overcloud image build --config-file /usr/share/openstack-tripleo-common/image-yaml/overcloud-images-python3.yaml --config-file $OS_YAML

...

2022-04-12 02:27:51.348 | Error: Failed to download metadata for repo 'appstream': Cannot prepare internal mirrorlist: No URLs in mirrorlist
2022-04-12 02:27:51.393 | Unmount /home/stack/dib_build.AD3HNFsC/mnt/tmp/yum
2022-04-12 02:27:51.426 | Unmount /home/stack/dib_build.AD3HNFsC/mnt/tmp/in_target.d
2022-04-12 02:27:51.447 | Unmount /home/stack/dib_build.AD3HNFsC/mnt/sys
2022-04-12 02:27:51.473 | Unmount /home/stack/dib_build.AD3HNFsC/mnt/proc
2022-04-12 02:27:51.509 | Unmount /home/stack/dib_build.AD3HNFsC/mnt/dev/pts
2022-04-12 02:27:51.537 | Unmount /home/stack/dib_build.AD3HNFsC/mnt/dev
2022-04-12 02:27:51.997 | INFO diskimage_builder.block_device.blockdevice [-] State already cleaned - no way to do anything here
Exception occured while running the command
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/tripleoclient/command.py", line 34, in run
    super(Command, self).run(parsed_args)
  File "/usr/lib/python3.6/site-packages/osc_lib/command/command.py", line 39, in run
    return super(Command, self).run(parsed_args)
  File "/usr/lib/python3.6/site-packages/cliff/command.py", line 186, in run
    return_code = self.take_action(parsed_args) or 0
  File "/usr/lib/python3.6/site-packages/tripleoclient/v1/overcloud_image.py", line 128, in take_action
    manager.build()
  File "/usr/lib/python3.6/site-packages/tripleo_common/image/build.py", line 85, in build
    elements, options, packages, extra_options)
  File "/usr/lib/python3.6/site-packages/tripleo_common/image/image_builder.py", line 138, in build_image
    raise subprocess.CalledProcessError(process.returncode, cmd)
subprocess.CalledProcessError: Command '['disk-image-create', '-a', 'amd64', '-o', './overcloud-full', '-t', 'qcow2', '-p', 'python3-psutil,python3-debtcollector,sos,device-mapper-multipath,openstack-heat-agents,os-net-config,jq,python3-dbus,kernel-modules-extra', '--min-tmpfs=7', '--mkfs-options', '-s size=4096', 'centos', 'baremetal', 'openvswitch', 'overcloud-agent', 'overcloud-base', 'overcloud-controller', 'overcloud-compute', 'overcloud-ceph-storage', 'puppet-modules', 'stable-interface-names', 'grub2', 'element-manifest', 'dynamic-login', 'iptables', 'enable-packages-install', 'override-pip-and-virtualenv', 'dracut-regenerate', 'remove-resolvconf', 'openssh', 'disable-nouveau', 'selinux-permissive', 'interface-names']' returned non-zero exit status 1.
Command '['disk-image-create', '-a', 'amd64', '-o', './overcloud-full', '-t', 'qcow2', '-p', 'python3-psutil,python3-debtcollector,sos,device-mapper-multipath,openstack-heat-agents,os-net-config,jq,python3-dbus,kernel-modules-extra', '--min-tmpfs=7', '--mkfs-options', '-s size=4096', 'centos', 'baremetal', 'openvswitch', 'overcloud-agent', 'overcloud-base', 'overcloud-controller', 'overcloud-compute', 'overcloud-ceph-storage', 'puppet-modules', 'stable-interface-names', 'grub2', 'element-manifest', 'dynamic-login', 'iptables', 'enable-packages-install', 'override-pip-and-virtualenv', 'dracut-regenerate', 'remove-resolvconf', 'openssh', 'disable-nouveau', 'selinux-permissive', 'interface-names']' returned non-zero exit status 1.

失敗した。

以下が原因と思われる。

2022-04-12 02:27:51.348 | Error: Failed to download metadata for repo 'appstream': Cannot prepare internal mirrorlist: No URLs in mirrorlist

ググったところ、以下がヒットした。
https://qiita.com/yamada-hakase/items/cb1b6124e11ca65e2a2b

centos8のミラーサイトが停止したことが原因のようである。
undercloudはcentos stream 8だったので、overcloud用のイメージのosもcentos stream 8かと思っていたが、centos8でビルドしようとしていたようである。
OS_YAMLを指定しなければcentos7のイメージをビルドするようなので、OS_YAMLを指定せずに再実施する。

centos stream 8のqcow2イメージをダウンロード

(undercloud) [stack@undercloud ~]$ curl -O https://cloud.centos.org/centos/8-stream/x86_64/images/CentOS-Stream-GenericCloud-8-20220125.1.x86_64.qcow2
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 1421M  100 1421M    0     0   815k      0  0:29:45  0:29:45 --:--:-- 1052k
(undercloud) [stack@undercloud ~]$ curl -O https://cloud.centos.org/centos/8-stream/x86_64/images/CHECKSUM
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  4840  100  4840    0     0   2903      0  0:00:01  0:00:01 --:--:--  2905
(undercloud) [stack@undercloud ~]$ sha256sum -c CHECKSUM |grep CentOS-Stream-GenericCloud-8-20220125.1.x86_64.qcow2

...

CentOS-Stream-GenericCloud-8-20220125.1.x86_64.qcow2: OK

(undercloud) [stack@undercloud ~]$ export DIB_LOCAL_IMAGE=/home/stack/CentOS-Stream-GenericCloud-8-20220125.1.x86_64.qcow2
(undercloud) [stack@undercloud ~]$ export OS_YAML="/usr/share/openstack-tripleo-common/image-yaml/overcloud-images-centos8.yaml"
(undercloud) [stack@undercloud ~]$ export DIB_YUM_REPO_CONF="/etc/yum.repos.d/delorean* /etc/yum.repos.d/tripleo-centos-*"
(undercloud) [stack@undercloud ~]$ openstack overcloud image build --config-file /usr/share/openstack-tripleo-common/image-yaml/overcloud-images-python3.yaml --config-file $OS_YAML

...

2022-04-12 08:52:41.519 | [MIRROR] librados2-16.2.7-1.el8s.x86_64.rpm: Status code: 503 for http://mirror.centos.org/centos/8-stream/storage/x86_64/ceph-pacific/Packages/l/librados2-16.2.7-1.el8s.x86_64.rpm (IP: 172.16.11.1)
2022-04-12 08:52:41.551 | [MIRROR] librbd1-16.2.7-1.el8s.x86_64.rpm: Status code: 503 for http://mirror.centos.org/centos/8-stream/storage/x86_64/ceph-pacific/Packages/l/librbd1-16.2.7-1.el8s.x86_64.rpm (IP: 172.16.11.1)
2022-04-12 08:52:42.656 | [MIRROR] librados2-16.2.7-1.el8s.x86_64.rpm: Status code: 503 for http://mirror.centos.org/centos/8-stream/storage/x86_64/ceph-pacific/Packages/l/librados2-16.2.7-1.el8s.x86_64.rpm (IP: 172.16.11.1)
2022-04-12 08:52:42.687 | [MIRROR] librbd1-16.2.7-1.el8s.x86_64.rpm: Status code: 503 for http://mirror.centos.org/centos/8-stream/storage/x86_64/ceph-pacific/Packages/l/librbd1-16.2.7-1.el8s.x86_64.rpm (IP: 172.16.11.1)
2022-04-12 08:52:42.687 | [FAILED] librbd1-16.2.7-1.el8s.x86_64.rpm: No more mirrors to try - All mirrors were already tried without success
2022-04-12 08:52:42.694 | Error: Error downloading packages:
2022-04-12 08:52:42.694 |   librbd1-2:16.2.7-1.el8s.x86_64: Cannot download, all mirrors were already tried without success

...

centosのリポジトリのメタデータダウンロードはうまくいあったが、別の箇所で失敗。
overcloud-fullのビルドは成功したが、ironic-python-agentのビルドに失敗。
503のエラーで、librbd1-2:16.2.7-1.el8s.x86_64のダウンロードに失敗したようである。

試しにダウンロードできるか確認

(undercloud) [stack@undercloud ~]$ curl -O http://mirror.centos.org/centos/8-stream/storage/x86_64/ceph-pacific/Packages/l/librbd1-16.2.7-1.el8s.x86_64.rpm
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 4304k  100 4304k    0     0  4853k      0 --:--:-- --:--:-- --:--:-- 4847k
(undercloud) [stack@undercloud ~]$ ll librbd1-16.2.7-1.el8s.x86_64.rpm 
-rw-rw-r--. 1 stack stack 4408168 Apr 13 01:15 librbd1-16.2.7-1.el8s.x86_64.rpm

普通にダウンロードできてしまった。
また、~/.cache/image-create/yumにrpmがキャッシュされているようだが、そこにも対象のrpmが存在した。何でうまくいかないんだ？？

(undercloud) [stack@undercloud ~]$ find ~/.cache/image-create/yum/ -name librbd1-16.2.7-1*
/home/stack/.cache/image-create/yum/tripleo-centos-ceph-pacific-373fc707964594d1/packages/librbd1-16.2.7-1.el8s.x86_64.rpm

--image-name ironic-python-agentを指定して、ironic-python-agentのビルドを再実施する。
すると、以下のように対象パッケージがダウンロード済となっていた。

2022-04-13 01:33:38.004 | [SKIPPED] librbd1-16.2.7-1.el8s.x86_64.rpm: Already downloaded

しかし、別のパッケージlibrados2-2:16.2.7-1.el8s.x86_64のダウンロードに失敗。

2022-04-13 01:45:42.575 | [MIRROR] librados2-16.2.7-1.el8s.x86_64.rpm: Curl error (28): Timeout was reached for http://mirror.centos.org/centos/8-stream/storage/x86_64/ceph-pacific/Packages/l/librados2-16.2.7-1.el8s.x86_64.rpm [Operation too slow. Less than 1000 bytes/sec transferred the last 30 seconds]
2022-04-13 01:56:47.103 | [MIRROR] librados2-16.2.7-1.el8s.x86_64.rpm: Curl error (28): Timeout was reached for http://mirror.centos.org/centos/8-stream/storage/x86_64/ceph-pacific/Packages/l/librados2-16.2.7-1.el8s.x86_64.rpm [Operation too slow. Less than 1000 bytes/sec transferred the last 30 seconds]
2022-04-13 01:57:25.113 | [MIRROR] librados2-16.2.7-1.el8s.x86_64.rpm: Curl error (28): Timeout was reached for http://mirror.centos.org/centos/8-stream/storage/x86_64/ceph-pacific/Packages/l/librados2-16.2.7-1.el8s.x86_64.rpm [Operation too slow. Less than 1000 bytes/sec transferred the last 30 seconds]
2022-04-13 01:58:02.122 | [MIRROR] librados2-16.2.7-1.el8s.x86_64.rpm: Curl error (28): Timeout was reached for http://mirror.centos.org/centos/8-stream/storage/x86_64/ceph-pacific/Packages/l/librados2-16.2.7-1.el8s.x86_64.rpm [Operation too slow. Less than 1000 bytes/sec transferred the last 30 seconds]
2022-04-13 01:58:02.122 | [FAILED] librados2-16.2.7-1.el8s.x86_64.rpm: No more mirrors to try - All mirrors were already tried without success
2022-04-13 01:58:02.131 | Error: Error downloading packages:
2022-04-13 01:58:02.131 |   librados2-2:16.2.7-1.el8s.x86_64: Cannot download, all mirrors were already tried without success

先程のエラーとは異なり、ダウンロードの速度が遅いことが原因のようである。

ビルドを再度実行してみる。

(undercloud) [stack@undercloud ~]$ openstack overcloud image build --config-file /usr/share/openstack-tripleo-common/image-yaml/overcloud-images-python3.yaml --config-file $OS_YAML --image-name ironic-python-agent

...

2022-04-13 02:37:43.180 | Build completed successfully
(undercloud) [stack@undercloud ~]$ ll ironic-python-agent.*
-rw-rw-r--. 1 stack stack 447314997 Apr 13 02:37 ironic-python-agent.initramfs
-rwxr-xr-x. 1 stack stack  10460296 Apr 13 02:37 ironic-python-agent.kernel
-rw-rw-r--. 1 stack stack    569173 Apr 13 02:37 ironic-python-agent.log

ironic-python-agent.d:
total 0
drwxr-xr-x. 2 stack stack 78 Apr 13 02:36 dib-manifests

ビルドできた！

overcloud用イメージのアップロード

(undercloud) [stack@undercloud ~]$ openstack overcloud image upload
/usr/lib64/python3.6/site-packages/_yaml/__init__.py:23: DeprecationWarning: The _yaml extension module is now located at yaml._yaml and its location is subject to change.  To use the LibYAML-based parser and emitter, import from `yaml`: `from yaml import CLoader as Loader, CDumper as Dumper`.
  DeprecationWarning
Image "file:///var/lib/ironic/images/overcloud-full.vmlinuz" was copied.
+------------------------------------------------------+----------------+----------+
|                         Path                         |      Name      |   Size   |
+------------------------------------------------------+----------------+----------+
| file:///var/lib/ironic/images/overcloud-full.vmlinuz | overcloud-full | 10460296 |
+------------------------------------------------------+----------------+----------+
Image "file:///var/lib/ironic/images/overcloud-full.initrd" was copied.
+-----------------------------------------------------+----------------+----------+
|                         Path                        |      Name      |   Size   |
+-----------------------------------------------------+----------------+----------+
| file:///var/lib/ironic/images/overcloud-full.initrd | overcloud-full | 70275515 |
+-----------------------------------------------------+----------------+----------+
Image "file:///var/lib/ironic/images/overcloud-full.raw" was copied.
+--------------------------------------------------+----------------+------------+
|                       Path                       |      Name      |    Size    |
+--------------------------------------------------+----------------+------------+
| file:///var/lib/ironic/images/overcloud-full.raw | overcloud-full | 5105647616 |
+--------------------------------------------------+----------------+------------+

overcloud-full.qcow2とironic-python-agentのイメージがアップロードされていないように見えるがコマンドは成功したので、とりあえず先に進む。

ノード登録

以下を参考に、overcloudをインストールするノードの情報をinstackenv.jsonに定義する。

{
  "nodes": [
    {
      "name": "controller01",
      "pm_type": "ipmi",
      "ports": [
        {
          "address": "xx:xx:xx:xx:xx:xx"
        }
      ],
      "pm_user": "admin",
      "pm_password": "admin",
      "pm_addr": "yy.yy.yy.yy"
    },

...

作成したinstackenv.jsonを指定して、ノードを登録する。

(undercloud) [stack@undercloud ~]$ openstack overcloud image upload
/usr/lib64/python3.6/site-packages/_yaml/__init__.py:23: DeprecationWarning: The _yaml extension module is now located at yaml._yaml and its location is subject to change.  To use the LibYAML-based parser and emitter, import from `yaml`: `from yaml import CLoader as Loader, CDumper as Dumper`.
  DeprecationWarning
Image "/var/lib/ironic/images/overcloud-full.vmlinuz" is up-to-date, skipping.
Image "/var/lib/ironic/images/overcloud-full.initrd" is up-to-date, skipping.
Image "/var/lib/ironic/images/overcloud-full.raw" is up-to-date, skipping.
Image file "/var/lib/ironic/httpboot/agent.kernel" is up-to-date, skipping.
Image file "/var/lib/ironic/httpboot/agent.ramdisk" is up-to-date, skipping.
(undercloud) [stack@undercloud ~]$ openstack overcloud node import instackenv.json
/usr/lib64/python3.6/site-packages/_yaml/__init__.py:23: DeprecationWarning: The _yaml extension module is now located at yaml._yaml and its location is subject to change.  To use the LibYAML-based parser and emitter, import from `yaml`: `from yaml import CLoader as Loader, CDumper as Dumper`.
  DeprecationWarning
Successfully registered node UUID e6c03feb-9e7b-4f56-bd1b-85a8f0f2598c
Successfully registered node UUID 9a36e8b4-7434-465c-90b5-8d4d832e0b54
Successfully registered node UUID db744892-09b4-4b52-b91a-91f2e547f503
Successfully registered node UUID 96d91e7a-0b1f-4e76-978c-9aa29e4630b3
(undercloud) [stack@undercloud ~]$ openstack baremetal node list
/usr/lib64/python3.6/site-packages/_yaml/__init__.py:23: DeprecationWarning: The _yaml extension module is now located at yaml._yaml and its location is subject to change.  To use the LibYAML-based parser and emitter, import from `yaml`: `from yaml import CLoader as Loader, CDumper as Dumper`.
  DeprecationWarning
+--------------------------------------+--------------+---------------+-------------+--------------------+-------------+
| UUID                                 | Name         | Instance UUID | Power State | Provisioning State | Maintenance |
+--------------------------------------+--------------+---------------+-------------+--------------------+-------------+
| e6c03feb-9e7b-4f56-bd1b-85a8f0f2598c | controller01 | None          | power off   | manageable         | False       |
| 9a36e8b4-7434-465c-90b5-8d4d832e0b54 | controller02 | None          | power off   | manageable         | False       |
| db744892-09b4-4b52-b91a-91f2e547f503 | controller03 | None          | power off   | manageable         | False       |
| 96d91e7a-0b1f-4e76-978c-9aa29e4630b3 | compute01    | None          | power off   | manageable         | False       |
+--------------------------------------+--------------+---------------+-------------+--------------------+-------------+

ノードのイントロスペクト

pre introspectionを実施

(undercloud) [stack@undercloud ~]$ openstack tripleo validator run --group pre-introspection
/usr/lib64/python3.6/site-packages/_yaml/__init__.py:23: DeprecationWarning: The _yaml extension module is now located at yaml._yaml and its location is subject to change.  To use the LibYAML-based parser and emitter, import from `yaml`: `from yaml import CLoader as Loader, CDumper as Dumper`.
  DeprecationWarning
'ArgumentParser' object has no attribute 'output_dir'

output_dirオプションを指定していないにも関わらず、output_dirは存在しないと怒られた。
バグ？このコマンドはundercloudを検証するコマンドで、必須ではないので次に進む。

ノードのイントロスペクト

(undercloud) [stack@undercloud ~]$ openstack overcloud node introspect --all-manageable --provide

...

Successfully introspected nodes: ['e6c03feb-9e7b-4f56-bd1b-85a8f0f2598c', '9a36e8b4-7434-465c-90b5-8d4d832e0b54', 'db744892-09b4-4b52-b91a-91f2e547f503', '96d91e7a-0b1f-4e76-978c-9aa29e4630b3']
(undercloud) [stack@undercloud ~]$ openstack baremetal node list
/usr/lib64/python3.6/site-packages/_yaml/__init__.py:23: DeprecationWarning: The _yaml extension module is now located at yaml._yaml and its location is subject to change.  To use the LibYAML-based parser and emitter, import from `yaml`: `from yaml import CLoader as Loader, CDumper as Dumper`.
  DeprecationWarning
+--------------------------------------+--------------+---------------+-------------+--------------------+-------------+
| UUID                                 | Name         | Instance UUID | Power State | Provisioning State | Maintenance |
+--------------------------------------+--------------+---------------+-------------+--------------------+-------------+
| e6c03feb-9e7b-4f56-bd1b-85a8f0f2598c | controller01 | None          | power off   | available          | False       |
| 9a36e8b4-7434-465c-90b5-8d4d832e0b54 | controller02 | None          | power off   | available          | False       |
| db744892-09b4-4b52-b91a-91f2e547f503 | controller03 | None          | power off   | available          | False       |
| 96d91e7a-0b1f-4e76-978c-9aa29e4630b3 | compute01    | None          | power off   | available          | False       |
+--------------------------------------+--------------+---------------+-------------+--------------------+-------------+

ネットワーク定義

以下を参考に進める。

ネットワーク定義のテンプレートをコピー

(undercloud) [stack@undercloud ~]$ mkdir -p overcloud/templates
(undercloud) [stack@undercloud ~]$ cp /usr/share/openstack-tripleo-heat-templates/network-data-samples/default-network-isolation.yaml ~/overcloud/templates

各ファイルを以下のように編集

- name: Storage
  name_lower: storage
  vip: true
  mtu: 1500
  subnets:
    storage_subnet:
      ip_subnet: 172.16.12.0/24
      allocation_pools:
        - start: 172.16.12.40
          end: 172.16.12.99
      vlan: 12
- name: StorageMgmt
  name_lower: storage_mgmt
  vip: true
  mtu: 1500
  subnets:
    storage_mgmt_subnet:
      ip_subnet: 172.16.13.0/24
      allocation_pools:
        - start: 172.16.13.40
          end: 172.16.13.99
      vlan: 13
- name: InternalApi
  name_lower: internal_api
  vip: true
  mtu: 1500
  subnets:
    internal_api_subnet:
      ip_subnet: 172.16.14.0/24
      allocation_pools:
        - start: 172.16.14.40
          end: 172.16.14.99
      vlan: 14
- name: Tenant
  vip: false  # Tenant network does not use VIPs
  mtu: 1500
  name_lower: tenant
  subnets:
    tenant_subnet:
      ip_subnet: 172.16.15.0/24
      allocation_pools:
        - start: 172.16.15.40
          end: 172.16.15.99
      vlan: 15
- name: External
  name_lower: external
  vip: true
  mtu: 1500
  subnets:
    external_subnet:
      ip_subnet: 172.16.16.0/24
      allocation_pools:
        - start: 172.16.16.40
          end: 172.16.16.99
      gateway_ip: 172.16.16.1
      vlan: 16

VIP定義

VIPの定義テンプレートをコピー

(undercloud) [stack@undercloud ~]$ cp /usr/share/openstack-tripleo-heat-templates/network-data-samples/vip-data-fixed-ip.yaml ~/overcloud/templates

以下のように編集

- network: ctlplane
  dns_name: overcloud
  ip_address: 172.16.10.30
- network: storage
  dns_name: overcloud
  ip_address: 172.16.12.30
- network: storage_mgmt
  dns_name: overcloud
  ip_address: 172.16.13.30
- network: internal_api
  dns_name: overcloud
  ip_address: 172.16.14.30
- network: external
  dns_name: overcloud
  ip_address: 172.16.16.30

externalネットワークのVIPがパブリックなAPIエンドポイントに使用されるため、undecloudのno_proxyに追加する必要がある。

ノード定義

以下を参考にベアメタルの定義を作成する。

各ノードにproxy設定するためにansibleのplaybookを作成する。

作成したyamlは以下の通り

  • ~/overcloud/playbooks/set_environment.yaml
(undercloud) [stack@undercloud ~]$ cat overcloud/playbooks/set_environment.yaml 
- name: Set environment valuable
  hosts: allovercloud
  any_errors_fatal: true
  gather_facts: false
  tasks:
  - name: Set http proxy
    become: true
    template:
      src: environment.j2
      dest: /etc/environment
  • ~/overcloud/playbooks/templates/environment.j2
http_proxy={{ http_proxy }}
https_proxy={{ http_proxy }}
HTTP_PROXY={{ http_proxy }}
HTTPS_PROXY={{ http_proxy }}
no_proxy={{ no_proxy }}
NO_PROXY={{ no_proxy }}

次にnicの定義を作成

/usr/share/ansible/roles/tripleo_network_config/templates/bonds_vlans/bonds_vlans.j2を使ったが、パラメータを入れる処理で改行されない問題（バグと思われる）があったため、修正して使った。

(undercloud) [stack@undercloud ~]$ mkdir ~/overcloud/templates/nic-configs
(undercloud) [stack@undercloud ~]$ cp /usr/share/ansible/roles/tripleo_network_config/templates/bonds_vlans/bonds_vlans.j2 ~/overcloud/templates/nic-configs/
  • 修正前
...

{% for network in role_networks %}
  - type: vlan

...
  • 修正後
...

{% for network in role_networks %}

  - type: vlan

...

ノードの定義を作成

- name: Controller
  count: 3
  defaults:
    networks:
    - network: ctlplane
      vif: true
    - network: external
      subnet: external_subnet
    - network: internal_api
      subnet: internal_api_subnet
    - network: storage
      subnet: storage_subnet
    - network: storage_mgmt
      subnet: storage_mgmt_subnet
    - network: tenant
      subnet: tenant_subnet
    network_config:
      template: /home/stack/overcloud/templates/nic-configs/bonds_vlans.j2
      default_route_network:
      - external
      bond_interface_ovs_options: "lacp=active other-config:lacp-fallback-ab=true"
    config_drive:
      cloud_config:
        chpasswd:
          list: heat-admin:password
          expire: false
  ansible_playbooks:
  - playbook: ../playbooks/set_environment.yaml
    extra_vars:
      http_proxy: http://172.16.11.1:63128/
      no_proxy: localhost,127.0.0.1,.localdomain,.ctlplane,.internalapi,.external,.storage,.storagemgmt,.tenant,undercloud,overcloud-controller-0,overcloud-controller-1,overcloud-controller-2,172.16.10.30,172.16.12.30,172.16.13.30,172.16.14.30,172.16.16.30
  instances:
  - hostname: overcloud-controller-0
    name: controller01
  - hostname: overcloud-controller-1
    name: controller02
  - hostname: overcloud-controller-2
    name: controller03
- name: Compute
  count: 1
  defaults:
    networks:
    - network: ctlplane
      vif: true
    - network: internal_api
      subnet: internal_api_subnet
    - network: tenant
      subnet: tenant_subnet
    - network: storage
      subnet: storage_subnet
    - network: external
      subnet: external_subnet
    network_config:
      template: /home/stack/overcloud/templates/nic-configs/bonds_vlans.j2
      default_route_network:
      - external
      bond_interface_ovs_options: "lacp=active other-config:lacp-fallback-ab=true"
    config_drive:
      cloud_config:
        chpasswd:
          list: heat-admin:password
          expire: false
  ansible_playbooks:
  - playbook: ../playbooks/set_environment.yaml
    extra_vars:
      http_proxy: http://172.16.11.1:63128/
      no_proxy: localhost,127.0.0.1,.localdomain,.ctlplane,.internalapi,.external,.storage,.storagemgmt,.tenant,undercloud,overcloud-controller-0,overcloud-controller-1,overcloud-controller-2,172.16.10.30,172.16.12.30,172.16.13.30,172.16.14.30,172.16.16.30
  instances:
  - hostname: overcloud-novacompute-0
    name: compute01

調査時、リモートコンソールからログインできるようにlinuxユーザーのパスワードをcloud-initで設定しておく。

コンテナレジストリ定義

デフォルトだとredhatのレジストリから直接イメージをpullするようだが、今回はundercloudのレジストリからpullするように設定する。

以下を参考に設定する。
https://docs.openstack.org/project-deploy-guide/tripleo-docs/latest/deployment/container_image_prepare.html#running-container-image-prepare

undercloudインストール時に作成したcontainers-prepare-parameters.yamlをコピーする。

(undercloud) [stack@undercloud ~]$ mkdir ~/overcloud/environment
(undercloud) [stack@undercloud ~]$ cp containers-prepare-parameter.yaml ~/overcloud/environment/

overcloudのデプロイ

作成したovercloudの定義ファイルは以下の通り。

(undercloud) [stack@undercloud ~]$ tree overcloud
overcloud
├── environment
│   └── containers-prepare-parameter.yaml
├── playbooks
│   ├── set_environment.yaml
│   ├── templates
│   │   └── environment.j2
└── templates
    ├── baremetal_deployment.yaml
    ├── default-network-isolation.yaml
    ├── nic-configs
    │   └── bonds_vlans.j2
    └── vip-data-fixed-ip.yaml

5 directories, 7 files

上記ファイルを指定してovercloudをデプロイする。

(undercloud) [stack@undercloud ~]$ openstack overcloud deploy \
>   --templates \
>   --network-config \
>   --networks-file ~/overcloud/templates/default-network-isolation.yaml \
>   --vip-file ~/overcloud/templates/vip-data-fixed-ip.yaml \
>   --baremetal-deployment ~/overcloud/templates/baremetal_deployment.yaml \
>   -e ~/overcloud/environment/containers-prepare-parameter.yaml \
>   --ntp-server <NTPサーバーのFQDN>

...

PLAY [Set environment valuable] ************************************************
2022-05-18 01:58:33.580738 | 40a6b717-34a8-365e-0e88-00000000000c |       TASK | Set http proxy
2022-05-18 01:59:01.047957 | 40a6b717-34a8-365e-0e88-00000000000c | UNREACHABLE | Set http proxy | overcloud-controller-2
2022-05-18 01:59:01.049610 | 40a6b717-34a8-365e-0e88-00000000000c |     TIMING | Set http proxy | overcloud-controller-2 | 0:00:27.478103 | 27.46s
2022-05-18 01:59:01.050183 | 40a6b717-34a8-365e-0e88-00000000000c | UNREACHABLE | Set http proxy | overcloud-controller-0
2022-05-18 01:59:01.050651 | 40a6b717-34a8-365e-0e88-00000000000c |     TIMING | Set http proxy | overcloud-controller-0 | 0:00:27.479154 | 27.47s
2022-05-18 01:59:01.051145 | 40a6b717-34a8-365e-0e88-00000000000c | UNREACHABLE | Set http proxy | overcloud-controller-1
2022-05-18 01:59:01.051540 | 40a6b717-34a8-365e-0e88-00000000000c |     TIMING | Set http proxy | overcloud-controller-1 | 0:00:27.480043 | 27.47s

NO MORE HOSTS LEFT *************************************************************

PLAY RECAP *********************************************************************
overcloud-controller-0     : ok=0    changed=0    unreachable=1    failed=0    skipped=0    rescued=0    ignored=0   
overcloud-controller-1     : ok=0    changed=0    unreachable=1    failed=0    skipped=0    rescued=0    ignored=0   
overcloud-controller-2     : ok=0    changed=0    unreachable=1    failed=0    skipped=0    rescued=0    ignored=0

作成したplaybookを実行する処理で、各ノードに通信できず失敗。
プロビジョニング後、ノードが起動する前にansibleを実行しにいったためと思われる。
ノードの起動を待つタイムアウト値がありそうだが、よくわからなかったため、同じコマンドを再実行する。

(undercloud) [stack@undercloud ~]$ openstack overcloud deploy \
>   --templates \
>   --network-config \
>   --networks-file ~/overcloud/templates/default-network-isolation.yaml \
>   --vip-file ~/overcloud/templates/vip-data-fixed-ip.yaml \
>   --baremetal-deployment ~/overcloud/templates/baremetal_deployment.yaml \
>   -e ~/overcloud/environment/containers-prepare-parameter.yaml \
>   --ntp-server <NTPサーバーのFQDN>

...

Overcloud Endpoint: http://172.16.16.30:5000
Overcloud Horizon Dashboard URL: http://172.16.16.30:80/dashboard
Overcloud rc file: /home/stack/overcloud-deploy/overcloud/overcloudrc and /home/stack/overcloudrc
Overcloud Deployed successfully
2022-05-18 07:18:39.364 182445 INFO tripleoclient.v1.overcloud_deploy.DeployOvercloud [-] Stopping ephemeral heat.
2022-05-18 07:18:39.535 182445 INFO tripleoclient.heat_launcher [-] Killing pod: ephemeral-heat
92b97a25384e69859f6a9d305b9bd34a894b0eb430734816a34d65552195b72a
2022-05-18 07:18:39.700 182445 INFO tripleoclient.heat_launcher [-] Killed pod: ephemeral-heat
2022-05-18 07:18:39.982 182445 INFO tripleoclient.heat_launcher [-] Starting back up of heat db
2022-05-18 07:18:45.999 182445 INFO tripleoclient.heat_launcher [-] Created tarfile /home/stack/overcloud-deploy/overcloud/heat-launcher/heat-db.sql-1652857088.5547752.tar.bzip2
2022-05-18 07:18:46.001 182445 INFO tripleoclient.heat_launcher [-] Deleting /home/stack/overcloud-deploy/overcloud/heat-launcher/heat-db.sql
2022-05-18 07:18:46.614 182445 INFO tripleoclient.heat_launcher [-] Removing pod: ephemeral-heat
92b97a25384e69859f6a9d305b9bd34a894b0eb430734816a34d65552195b72a
2022-05-18 07:18:47.283 182445 INFO tripleoclient.heat_launcher [-] Created tarfile /home/stack/overcloud-deploy/overcloud/heat-launcher/log/heat-1652857088.5547752.log-1652857088.5547752.tar.bzip2
2022-05-18 07:18:47.284 182445 INFO tripleoclient.heat_launcher [-] Deleting /home/stack/overcloud-deploy/overcloud/heat-launcher/log/heat-1652857088.5547752.log
2022-05-18 07:18:48.146 182445 INFO osc_lib.shell [-] END return value: None

デプロイに成功した！

overcloudの動作検証

まずはコマンドを試してみる。

[stack@undercloud ~]$ source ~/overcloudrc 
(overcloud) [stack@undercloud ~]$ openstack project list
/usr/lib64/python3.6/site-packages/_yaml/__init__.py:23: DeprecationWarning: The _yaml extension module is now located at yaml._yaml and its location is subject to change.  To use the LibYAML-based parser and emitter, import from `yaml`: `from yaml import CLoader as Loader, CDumper as Dumper`.
  DeprecationWarning
+----------------------------------+---------+
| ID                               | Name    |
+----------------------------------+---------+
| 10bf429c596e48d986436745ec9ba614 | service |
| 76dd48b465f949deb0806018b925d4e1 | admin   |
+----------------------------------+---------+

次に、ブラウザからhttp://172.16.16.30:80/dashboardにアクセスし、horizonを試してみる。

ログインユーザーはadminが用意されており、パスワードは~/overcloudrcの中身を参照して確認する。
以下の画像のようにログインに成功し、horizonのトップ画面を参照できた。

horizon.png

次にインスタンスを作成するために、ネットワーク等のリソースを作成する。

まずはパブリックネットワークを作成する。
サブネットにはexternalネットワークと同様の172.16.16.0/24を使用し、externalネットワークの範囲（172.16.16.40 ~ 172.16.16.99）と重複しないように指定する。

[stack@undercloud ~]$ source overcloudrc 
(overcloud) [stack@undercloud ~]$ openstack network create public --external --provider-network-type vlan \
> --provider-physical-network datacentre --provider-segment 16

...

(overcloud) [stack@undercloud ~]$ openstack subnet create --allocation-pool start=172.16.16.200,end=172.16.16.240 \
> --network public --gateway 172.16.16.1 --no-dhcp --subnet-range \
> 172.16.16.0/24 public

...

プライベートネットワークの作成

(overcloud) [stack@undercloud ~]$ openstack network create private-net

...

(overcloud) [stack@undercloud ~]$ openstack subnet create private-subnet --network private-net \
> --subnet-range 10.0.0.0/24

...

仮想ルーターの作成とネットワークの接続

(overcloud) [stack@undercloud ~]$ openstack router create admin-router

...

(overcloud) [stack@undercloud ~]$ openstack router set admin-router --external-gateway public

...

(overcloud) [stack@undercloud ~]$ openstack router add subnet admin-router private-subnet

...

defaultセキュリティグループにSSHとICMPを許可するルールを追加

(overcloud) [stack@undercloud ~]$ openstack security group rule create default \
> --protocol tcp --dst-port 22:22 --remote-ip 0.0.0.0/0

...

(overcloud) [stack@undercloud ~]$ openstack security group rule create default \
> --protocol icmp --remote-ip 0.0.0.0/0

...

キーペア登録

(overcloud) [stack@undercloud ~]$ openstack keypair create --public-key ~/.ssh/id_rsa.pub admin-key

...

イメージ作成

(overcloud) [stack@undercloud ~]$ openstack image create \
> --disk-format qcow2 \
> --container-format bare \
> --public \
> --file ~/images/jammy-server-cloudimg-amd64-disk-kvm.img \
> ubuntu-2204

...

フレーバー作成

(overcloud) [stack@undercloud ~]$ openstack flavor create m1.small --ram 2048 --disk 20 --vcpus 1

...

インスタンス作成

(overcloud) [stack@undercloud ~]$ openstack server create \
> --flavor m1.small \
> --image ubuntu-2204 \
> --boot-from-volume 20 \
> --network private-net \
> --key-name admin-key \
> test-server

...

floating IPの割当

(overcloud) [stack@undercloud ~]$ openstack floating ip create public

...

(overcloud) [stack@undercloud ~]$ openstack server add floating ip test-server 172.16.16.223

...

インスタンスへの疎通確認

(overcloud) [stack@undercloud ~]$ ping 172.16.16.223
PING 172.16.16.223 (172.16.16.223) 56(84) bytes of data.
64 bytes from 172.16.16.223: icmp_seq=1 ttl=62 time=2.05 ms
64 bytes from 172.16.16.223: icmp_seq=2 ttl=62 time=1.02 ms
^C
--- 172.16.16.223 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1001ms
rtt min/avg/max/mdev = 1.022/1.534/2.046/0.512 ms
(overcloud) [stack@undercloud ~]$ ssh ubuntu@172.16.16.223
The authenticity of host '172.16.16.223 (172.16.16.223)' can't be established.
ECDSA key fingerprint is SHA256:4EHJxshmTiwV15xUqqPGlzLO5F2ym59PPHuIh1yqa9Y.
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added '172.16.16.223' (ECDSA) to the list of known hosts.
Welcome to Ubuntu 22.04 LTS (GNU/Linux 5.15.0-1004-kvm x86_64)

 * Documentation:  https://help.ubuntu.com
 * Management:     https://landscape.canonical.com
 * Support:        https://ubuntu.com/advantage

  System information as of Thu May 19 02:12:26 UTC 2022

  System load:  0.001953125       Processes:               88
  Usage of /:   6.6% of 19.21GB   Users logged in:         0
  Memory usage: 7%                IPv4 address for enp3s0: 10.0.0.153
  Swap usage:   0%

0 updates can be applied immediately.


The list of available updates is more than a week old.
To check for new updates run: sudo apt update


The programs included with the Ubuntu system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Ubuntu comes with ABSOLUTELY NO WARRANTY, to the extent permitted by
applicable law.

To run a command as administrator (user "root"), use "sudo <command>".
See "man sudo_root" for details.

ubuntu@test-server:~$

作成したインスタンスにSSHで接続することができた。

設定変更

試しにnovaのログレベルをdebugに変更する。

デフォルトでは以下のようにdebugモードは無効になっている。

[heat-admin@overcloud-controller-0 ~]$ sudo cat /var/lib/config-data/puppet-generated/nova/etc/nova/nova.conf |grep debug=
#debug=false
debug=False
#connection_debug=0
#debug=false
#connection_debug=0
#debug=false
#debug=false

上記のファイルを直接編集することで設定を変更することが可能だが、何らかのtripleoの操作で設定変更が元に戻ってしまうため、tripleoを使って設定変更することが推奨されている。

RHOSPのドキュメントに変更すべきパラメータを見つける方法があったので、これを参考に試す。
https://access.redhat.com/documentation/ja-jp/red_hat_openstack_platform/16.2/html/advanced_overcloud_customization/proc_identifying-parameters-that-you-want-to-modify_heat-parameters

まずは、novapuppetモジュールからdebugを検索する。

[stack@undercloud ~]$ grep debug -r /etc/puppet/modules/nova/

...

/etc/puppet/modules/nova/manifests/logging.pp:# [*debug*]

...

上記結果から、hieradataにおけるパラメータ名はnova::logging::debugだとわかる。

次に、heatテンプレートから、特定したhieradataのパラメータ名を検索する。

[stack@undercloud ~]$ grep nova::logging::debug -r /usr/share/openstack-tripleo-heat-templates/deployment/nova/

...

/usr/share/openstack-tripleo-heat-templates/deployment/nova/nova-base-puppet.yaml:          nova::logging::debug:

...

[stack@undercloud ~]$ cat /usr/share/openstack-tripleo-heat-templates/deployment/nova/nova-base-puppet.yaml

...

          nova::logging::debug:
            if:
            - {get_param: NovaDebug}
            - true
            - {get_param: Debug}

...

上記からheatテンプレートにおけるパラメータ名がNovaDebugであることがわかる。
次に特定したパラメータをenvironmentファイルに定義する。

~/overcloud/environment/nova-config.yamlを新規作成して、以下のように定義

parameter_defaults:
  NovaDebug: true

overcloudデプロイ時のコマンドに上記environmentファイルの指定を追加して実行し、設定変更を反映する。

(undercloud) [stack@undercloud ~]$ openstack overcloud deploy \
>   --templates \
>   --network-config \
>   --networks-file ~/overcloud/templates/default-network-isolation.yaml \
>   --vip-file ~/overcloud/templates/vip-data-fixed-ip.yaml \
>   --baremetal-deployment ~/overcloud/templates/baremetal_deployment.yaml \
>   -e ~/overcloud/environment/containers-prepare-parameter.yaml \
>   -e ~/overcloud/environment/nova-config.yaml \
>   --ntp-server <ntp-server-host>

...

デプロイ時と同様のコマンドだとノードのプロビジョニングなどovercloudのデプロイに必要な全てを実行するので時間がかかってしまう。
ドキュメントからは見つけられなかったが、設定変更だけ実施する方法がありそう。。

設定変更が反映されているかを確認する。

[heat-admin@overcloud-controller-0 ~]$ sudo cat /var/lib/config-data/puppet-generated/nova/etc/nova/nova.conf |grep debug=
#debug=false
debug=True
#connection_debug=0
#debug=false
#connection_debug=0
#debug=false
#debug=false

trueに変わった。

/var/log/containers/nova/nova-api.logを参照し、debugレベルのログが出力されていることも確認できた。

本記事は以上になります。

3
2
0

Register as a new user and use Qiita more conveniently

  1. You get articles that match your needs
  2. You can efficiently read back useful information
  3. You can use dark theme
What you can do with signing up
Sign upLogin
3
2

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?