LoginSignup
0
0

More than 1 year has passed since last update.

LXD上でのpacemakerのクラスタのインストール

Last updated at Posted at 2021-11-29

LXD(システムコンテナ)上でPacemakerをセットアップしてみました。

環境 - まずは単ノードで

KVM上のCentOS 8
CentOS-Stream-8-x86_64-2021102

LXDのインストール

以下のページを参考にしました。

手順

以下の操作は途中まではrootユーザで実行しています

KVMホスト上にCentOSのインストールを実行

インストール後にselinuxをdisabledに設定。firewallも停止。
EPELの有効化とアップデート

# yum install -y https://dl.fedoraproject.org/pub/epel/epel-release-latest-8.noarch.rpm
(snip)
# yum -y update
(snip)

snapdのインストール前の設定と再起動

[root@cent8-01 ~]# yum search snapd
メタデータの期限切れの最終確認: 0:10:26 時間前の 2021年11月29日 16時29分31秒 に実施しました。
=============================================================================== 名前 完全一致: snapd ================================================================================
snapd.x86_64 : A transactional software package manager
============================================================================== 名前 & 概要 一致: snapd ==============================================================================
snapd-devel.noarch : Development files for snapd
snapd-glib.x86_64 : Library providing a GLib interface to snapd
snapd-glib-devel.x86_64 : Development files for snapd-glib
snapd-glib-tests.x86_64 : Installed tests for snapd-glib
snapd-qt.x86_64 : Library providing a Qt5 interface to snapd
snapd-qt-devel.x86_64 : Development files for snapd-qt
snapd-qt-qml.x86_64 : Library providing a Qt5 QML interface to snapd
snapd-qt-tests.x86_64 : Installed tests for snapd-qt
snapd-selinux.noarch : SELinux module for snapd

[root@cent8-01 ~]# grubby --args="namespace.unpriv_enable=1" --update-kernel="$(grubby --default-kernel)"

[root@cent8-01 ~]# sh -c 'echo "user.max_user_namespaces=2147483647" >> /etc/sysctl.d/99-userns.conf'

[root@cent8-01 ~]# shutdown -r now 

設定の確認とsnapdのインストールと再起動

[root@cent8-01 ~]# cat /proc/cmdline
BOOT_IMAGE=(hd0,msdos1)/vmlinuz-4.18.0-348.2.1.el8_5.x86_64 root=/dev/mapper/cs-root ro resume=/dev/mapper/cs-swap rd.lvm.lv=cs/root rd.lvm.lv=cs/swap rhgb quiet namespace.unpriv_enable=1

[root@cent8-01 ~]# yum -y install snapd

[root@cent8-01 ~]# ln -s /var/lib/snapd/snap /snap
[root@cent8-01 ~]# 
[root@cent8-01 ~]# systemctl enable --now snapd.socket
Created symlink /etc/systemd/system/sockets.target.wants/snapd.socket → /usr/lib/systemd/system/snapd.socket.
[root@cent8-01 ~]# systemctl status snapd.socket
● snapd.socket - Socket activation for snappy daemon
   Loaded: loaded (/usr/lib/systemd/system/snapd.socket; enabled; vendor preset: disabled)
   Active: active (listening) since Mon 2021-11-29 16:46:49 JST; 6s ago
   Listen: /run/snapd.socket (Stream)
           /run/snapd-snap.socket (Stream)
    Tasks: 0 (limit: 49678)
   Memory: 4.0K
   CGroup: /system.slice/snapd.socket

11月 29 16:46:49 cent8-01 systemd[1]: Starting Socket activation for snappy daemon.
11月 29 16:46:49 cent8-01 systemd[1]: Listening on Socket activation for snappy daemon.
[root@cent8-01 ~]# systemctl status snapd.service
● snapd.service - Snap Daemon
   Loaded: loaded (/usr/lib/systemd/system/snapd.service; disabled; vendor preset: disabled)
   Active: inactive (dead)

[root@cent8-01 ~]# systemctl enable snapd.service
Created symlink /etc/systemd/system/multi-user.target.wants/snapd.service → /usr/lib/systemd/system/snapd.service.

[root@cent8-01 ~]# systemctl start snapd.service

[root@cent8-01 ~]# shutdown -r now

lxdのインストール

[root@cent8-01 ~]# snap install lxd
2021-11-29T16:50:24+09:00 INFO Waiting for automatic snapd restart...
lxd 4.20 from Canonical✓ installed

[root@cent8-01 ~]# useradd yamauchi
[root@cent8-01 ~]# usermod -a -G lxd yamauchi
[root@cent8-01 ~]# su - yamauchi
[yamauchi@cent8-01 ~]$ newgrp lxd

[yamauchi@cent8-01 ~]$ id
uid=1001(yamauchi) gid=971(lxd) groups=971(lxd),1001(yamauchi)

[yamauchi@cent8-01 ~]$ lxc list
If this is your first time running LXD on this machine, you should also run: lxd init
To start your first container, try: lxc launch ubuntu:20.04
Or for a virtual machine: lxc launch ubuntu:20.04 --vm

+------+-------+------+------+------+-----------+
| NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS |
+------+-------+------+------+------+-----------+

lxdのconfigurationの実行

途中storage backendだけ、OSインストール時のファイルシステムのlvmで指定しています。

[yamauchi@cent8-01 ~]$ lxd init
Would you like to use LXD clustering? (yes/no) [default=no]: 
Do you want to configure a new storage pool? (yes/no) [default=yes]: 
Name of the new storage pool [default=default]: 
Name of the storage backend to use (ceph, btrfs, dir, lvm) [default=btrfs]: lvm
Create a new LVM pool? (yes/no) [default=yes]: 
Would you like to use an existing empty block device (e.g. a disk or partition)? (yes/no) [default=no]: 
Size in GB of the new loop device (1GB minimum) [default=5GB]: 
Would you like to connect to a MAAS server? (yes/no) [default=no]: 
Would you like to create a new local network bridge? (yes/no) [default=yes]: 
What should the new bridge be called? [default=lxdbr0]: 
What IPv4 address should be used? (CIDR subnet notation, “auto” or “none”) [default=auto]: 
What IPv6 address should be used? (CIDR subnet notation, “auto” or “none”) [default=auto]: 
Would you like the LXD server to be available over the network? (yes/no) [default=no]: 
Would you like stale cached images to be updated automatically? (yes/no) [default=yes] 
Would you like a YAML "lxd init" preseed to be printed? (yes/no) [default=no]: 

イメージの確認

多くの公式イメージが出力される。

[yamauchi@cent8-01 ~] lxc image list images:

とりあえず、centos8のイメージを取得

[yamauchi@cent8-01 ~]$ lxc launch images:centos/8/amd64 cent8-01
Creating cent8-01
Starting cent8-01  

[yamauchi@cent8-01 ~]$ lxc list
+----------+---------+-----------------------+-----------------------------------------------+-----------+-----------+
|   NAME   |  STATE  |         IPV4          |                     IPV6                      |   TYPE    | SNAPSHOTS |
+----------+---------+-----------------------+-----------------------------------------------+-----------+-----------+
| cent8-01 | RUNNING | 10.144.107.190 (eth0) | fd42:26e0:cb93:870a:216:3eff:fe63:367d (eth0) | CONTAINER | 0         |
+----------+---------+-----------------------+-----------------------------------------------+-----------+-----------+

起動しているコンテナへ接続

[yamauchi@cent8-01 ~]$ lxc exec cent8-01 bash

High Availabilityを有効にしてコンテナにpcs,pacemaker,fence-agentsをインストール

[root@cent8-01 ~]# dnf --enablerepo=ha -y install pacemaker pcs fence-agents-all
CentOS Linux 8 - AppStream                                                                                                                           3.7 MB/s | 8.1 MB     00:02    
CentOS Linux 8 - BaseOS                                                                                                                              2.9 MB/s | 3.5 MB     00:01    
CentOS Linux 8 - Extras                                                                                                                               17 kB/s |  10 kB     00:00    
CentOS Linux 8 - HighAvailability                                                                                                                    528 kB/s | 521 kB     00:00    
Dependencies resolved.
=====================================================================================================================================================================================
 Package                                             Architecture                Version                                                        Repository                      Size
=====================================================================================================================================================================================
Installing:
 fence-agents-all                                    x86_64                      4.2.1-75.el8                                                   appstream                       22 k
 pacemaker                                           x86_64                      2.1.0-8.el8                                                    ha                             456 k
 pcs                                                 x86_64                      0.10.8-1.el8                                                   ha                              13 M
(snip)

一旦、コンテナから抜けて、コンテナを停止

[root@cent8-01 ~]# exit
exit

[yamauchi@cent8-01 ~]$ lxc stop  cent8-01
[yamauchi@cent8-01 ~]$ lxc list
+----------+---------+------+------+-----------+-----------+
|   NAME   |  STATE  | IPV4 | IPV6 |   TYPE    | SNAPSHOTS |
+----------+---------+------+------+-----------+-----------+
| cent8-01 | STOPPED |      |      | CONTAINER | 0         |
+----------+---------+------+------+-----------+-----------+

cont8-01をpublishしてから、cent8-02のイメージを作成

[yamauchi@cent8-01 ~]$ lxc publish cent8-01 --alias pacemaker
Instance published with fingerprint: f6716cdfd9643b59c6f96786b8d2306726dae386259cd9cfd3e77dfce78e8900
[yamauchi@cent8-01 ~]$ lxc list
+----------+---------+------+------+-----------+-----------+
|   NAME   |  STATE  | IPV4 | IPV6 |   TYPE    | SNAPSHOTS |
+----------+---------+------+------+-----------+-----------+
| cent8-01 | STOPPED |      |      | CONTAINER | 0         |
+----------+---------+------+------+-----------+-----------+

[yamauchi@cent8-01 ~]$  lxc image list
+-----------+--------------+--------+----------------------------------+--------------+-----------+----------+------------------------------+
|   ALIAS   | FINGERPRINT  | PUBLIC |           DESCRIPTION            | ARCHITECTURE |   TYPE    |   SIZE   |         UPLOAD DATE          |
+-----------+--------------+--------+----------------------------------+--------------+-----------+----------+------------------------------+
| pacemaker | f6716cdfd964 | no     | Centos 8 x86_64 (20211127_07:08) | x86_64       | CONTAINER | 308.77MB | Nov 29, 2021 at 8:04am (UTC) |
+-----------+--------------+--------+----------------------------------+--------------+-----------+----------+------------------------------+
|           | 6b5db6ae52c3 | no     | Centos 8 amd64 (20211127_07:08)  | x86_64       | CONTAINER | 127.49MB | Nov 29, 2021 at 7:56am (UTC) |
+-----------+--------------+--------+----------------------------------+--------------+-----------+----------+------------------------------+

[yamauchi@cent8-01 ~]$ lxc launch pacemaker cent8-02
Creating cent8-02
Starting cent8-02                          
[yamauchi@cent8-01 ~]$ lxc list
+----------+---------+----------------------+-----------------------------------------------+-----------+-----------+
|   NAME   |  STATE  |         IPV4         |                     IPV6                      |   TYPE    | SNAPSHOTS |
+----------+---------+----------------------+-----------------------------------------------+-----------+-----------+
| cent8-01 | STOPPED |                      |                                               | CONTAINER | 0         |
+----------+---------+----------------------+-----------------------------------------------+-----------+-----------+
| cent8-02 | RUNNING | 10.144.107.86 (eth0) | fd42:26e0:cb93:870a:216:3eff:fe88:8ae1 (eth0) | CONTAINER | 0         |
+----------+---------+----------------------+-----------------------------------------------+-----------+-----------+

cont8-01も再起動

[yamauchi@cent8-01 ~]$ lxc start cent8-01
[yamauchi@cent8-01 ~]$ lxc list
+----------+---------+-----------------------+-----------------------------------------------+-----------+-----------+
|   NAME   |  STATE  |         IPV4          |                     IPV6                      |   TYPE    | SNAPSHOTS |
+----------+---------+-----------------------+-----------------------------------------------+-----------+-----------+
| cent8-01 | RUNNING | 10.144.107.190 (eth0) | fd42:26e0:cb93:870a:216:3eff:fe63:367d (eth0) | CONTAINER | 0         |
+----------+---------+-----------------------+-----------------------------------------------+-----------+-----------+
| cent8-02 | RUNNING | 10.144.107.86 (eth0)  | fd42:26e0:cb93:870a:216:3eff:fe88:8ae1 (eth0) | CONTAINER | 0         |
+----------+---------+-----------------------+-----------------------------------------------+-----------+-----------+

クラスタの構築

2つのコンソールからそれぞれのコンテナへ接続

[yamauchi@cent8-01 ~]$ lxc exec cent8-01 bash
[root@cent8-01 ~]#

[yamauchi@cent8-01 ~]$ lxc exec cent8-02 bash
[root@cent8-02 ~]# 

それぞれのコンテナでhaclusterユーザのパスワードを設定し、pcsdサービスを起動

[root@cent8-01 ~]# passwd hacluster
Changing password for user hacluster.
New password: 
Retype new password: 
passwd: all authentication tokens updated successfully.

[root@cent8-01 ~]# systemctl enable pcsd --now
Created symlink /etc/systemd/system/multi-user.target.wants/pcsd.service → /usr/lib/systemd/system/pcsd.service.
[root@cent8-01 ~]# systemctl status  pcsd
● pcsd.service - PCS GUI and remote configuration interface
   Loaded: loaded (/usr/lib/systemd/system/pcsd.service; enabled; vendor preset: disabled)
  Drop-In: /run/systemd/system/pcsd.service.d
           └─zzz-lxc-service.conf
   Active: active (running) since Mon 2021-11-29 08:10:11 UTC; 5s ago
     Docs: man:pcsd(8)
           man:pcs(8)
 Main PID: 1367 (pcsd)
    Tasks: 1 (limit: 49678)
   Memory: 38.8M
   CGroup: /system.slice/pcsd.service
           └─1367 /usr/libexec/platform-python -Es /usr/sbin/pcsd

Nov 29 08:10:10 cent8-01 systemd[1]: Starting PCS GUI and remote configuration interface...
Nov 29 08:10:11 cent8-01 systemd[1]: Started PCS GUI and remote configuration interface.

片方のコンテナで、pcsコマンドで各コンテナを認証

[root@cent8-01 ~]# pcs host auth cent8-01 cent8-02
Username: hacluster
Password: 
cent8-01: Authorized
cent8-02: Authorized

一旦、pcsコマンドでクラスタをセットアップ
インターフェースは、最初は既存のインターフェースを指定

[root@cent8-01 ~]# pcs cluster setup cluster_name cent8-01 addr=10.144.107.190 cent8-02 addr=10.144.107.86
Destroying cluster on hosts: 'cent8-01', 'cent8-02'...
cent8-01: Successfully destroyed cluster
cent8-02: Successfully destroyed cluster
Requesting remove 'pcsd settings' from 'cent8-01', 'cent8-02'
cent8-01: successful removal of the file 'pcsd settings'
cent8-02: successful removal of the file 'pcsd settings'
Sending 'corosync authkey', 'pacemaker authkey' to 'cent8-01', 'cent8-02'
cent8-01: successful distribution of the file 'corosync authkey'
cent8-01: successful distribution of the file 'pacemaker authkey'
cent8-02: successful distribution of the file 'corosync authkey'
cent8-02: successful distribution of the file 'pacemaker authkey'
Sending 'corosync.conf' to 'cent8-01', 'cent8-02'
cent8-01: successful distribution of the file 'corosync.conf'
cent8-02: successful distribution of the file 'corosync.conf'
Cluster has been successfully set up.

pcsでクラスタを起動しようとするが失敗

[root@cent8-01 ~]# pcs cluster start --all
cent8-01: Error connecting to cent8-01 - (HTTP error: 400)
cent8-02: Error connecting to cent8-02 - (HTTP error: 400)
Error: unable to start all nodes
cent8-01: Error connecting to cent8-01 - (HTTP error: 400)
cent8-02: Error connecting to cent8-02 - (HTTP error: 400)

どうやら、問題は、knet指定がうまく動作していないようなので、両コンテナのcorosync.confのtotemブロックを以下のように変更
transportをudpに、crypto_cipher/crypto_hashをnoneに、interfaceブロックを追加

totem {
    version: 2
    cluster_name: cluster_name
    transport: udp
    crypto_cipher: none
    crypto_hash: none
    interface {
        ringnumber: 0
    }
}
(snip)

再度、クラスタを起動

[root@cent8-01 ~]# pcs cluster start --all
cent8-01: Starting Cluster...
cent8-02: Starting Cluster...

無事に起動

[root@cent8-01 ~]# crm_mon -rfA1
Cluster Summary:
  * Stack: corosync
  * Current DC: cent8-02 (version 2.1.0-8.el8-7c3f660707) - partition with quorum
  * Last updated: Mon Nov 29 08:18:54 2021
  * Last change:  Mon Nov 29 08:18:49 2021 by hacluster via crmd on cent8-02
  * 2 nodes configured
  * 0 resource instances configured

Node List:
  * Online: [ cent8-01 cent8-02 ]

Full List of Resources:
  * No resources

Migration Summary:

関連リンク

https://linuxcontainers.org/ja/lxd/introduction/
https://computingforgeeks.com/run-linux-containers-with-lxc-lxd-on-rocky-almalinux/

以上、

0
0
1

Register as a new user and use Qiita more conveniently

  1. You get articles that match your needs
  2. You can efficiently read back useful information
  3. You can use dark theme
What you can do with signing up
0
0