Galera Clusterを使えば簡単にマルチマスタのDBを構築できたのでその時の構築メモです。
LB配下にインスタンスを複数ぶら下げて各インスタンスにAPサーバとDBサーバを同居させて、各インスタンスで更新・参照をしなければならなかったので、試してみました。
PostgreSQLでマルチマスタの構成を組んでみたかったけど、難しそうだったので簡単そうなMariaDBで組んでみました。
- OS: CentOS7, 最小3台で構築
- MariaDB: 10.3.8
とても参考になった記事。
というかほぼコピペになってしまった。
http://www.orangetakam.com/blog/archives/1403
OS準備
Vagrantfileの設定
# *- mode: ruby -*-
# vi: set ft=ruby :
Vagrant.configure("2") do |config|
config.vm.define "db1" do |node|
node.vm.provider "virtualbox" do |vm|
vm.name = "db1"
vm.customize ["modifyvm", :id, "--memory", "2048"]
end
node.vm.box = "centos/7"
node.vm.hostname = "db1"
node.vm.network "private_network", ip: "192.168.56.10"
end
config.vm.define "db2" do |node|
node.vm.provider "virtualbox" do |vm|
vm.name = "db2"
vm.customize ["modifyvm", :id, "--memory", "2048"]
end
node.vm.box = "centos/7"
node.vm.hostname = "db2"
node.vm.network "private_network", ip: "192.168.56.20"
end
config.vm.define "db3" do |node|
node.vm.provider "virtualbox" do |vm|
vm.name = "db3"
vm.customize ["modifyvm", :id, "--memory", "2048"]
end
node.vm.box = "centos/7"
node.vm.hostname = "db3"
node.vm.network "private_network", ip: "192.168.56.30"
end
end
OS起動
vagrant up db1 db2 db3
SELINUXの無効化
/etc/sysconfig/selinuxを以下のように編集して再起動する。
SELINUX=disabled
SELINUXTYPE=targeted
MariaDBのインストール
以下の作業をdb1-db3まですべて実施する。
バンドルされているMariaDBを削除する
sudo yum info mariadb-libs.x86_64
sudo yum -y remove mariadb-libs.x86_64
リポジトリの設定
curl -sS https://downloads.mariadb.com/MariaDB/mariadb_repo_setup | sudo bash
MariaDB最新版インストール
sudo yum -y install MariaDB-server MariaDB-client
MariaDBの初期セットアップ
sudo systemctl disable mariadb
sudo systemctl start mariadb
sudo mysql_secure_installation
NOTE: RUNNING ALL PARTS OF THIS SCRIPT IS RECOMMENDED FOR ALL MariaDB
SERVERS IN PRODUCTION USE! PLEASE READ EACH STEP CAREFULLY!
In order to log into MariaDB to secure it, we'll need the current
password for the root user. If you've just installed MariaDB, and
you haven't set the root password yet, the password will be blank,
so you should just press enter here.
Enter current password for root (enter for none):
OK, successfully used password, moving on...
Setting the root password ensures that nobody can log into the MariaDB
root user without the proper authorisation.
Set root password? [Y/n] Y
New password: Pass@0077
Re-enter new password: Pass@0077
Password updated successfully!
Reloading privilege tables..
... Success!
By default, a MariaDB installation has an anonymous user, allowing anyone
to log into MariaDB without having to have a user account created for
them. This is intended only for testing, and to make the installation
go a bit smoother. You should remove them before moving into a
production environment.
Remove anonymous users? [Y/n] Y
... Success!
Normally, root should only be allowed to connect from 'localhost'. This
ensures that someone cannot guess at the root password from the network.
Disallow root login remotely? [Y/n] Y
... Success!
By default, MariaDB comes with a database named 'test' that anyone can
access. This is also intended only for testing, and should be removed
before moving into a production environment.
Remove test database and access to it? [Y/n] Y
- Dropping test database...
... Success!
- Removing privileges on test database...
... Success!
Reloading the privilege tables will ensure that all changes made so far
will take effect immediately.
Reload privilege tables now? [Y/n] Y
... Success!
Cleaning up...
All done! If you've completed all of the above steps, your MariaDB
installation should now be secure.
Thanks for using MariaDB!
MariaDBの停止
あとでGalera Clusterで起動するので、停止しておく。
sudo systemctl stop mariadb
Galera Clusterの初期設定(/etc/my.cnf.d/server.cnf)
db1の初期設定
sudo vi /etc/my.cnf.d/server.cnf
sudo mkdir -p /var/log/mariadb
sudo chown -R mysql:mysql /var/log/mariadb
[server]
[mysqld]
character-set-server=utf8
log-output=FILE
# Error log
log-error="/var/log/mariadb/mysqld.log"
# Query log
general-log=0
general_log_file="/var/log/mariadb/sql.log"
# Slow Query log
slow-query-log=1
slow_query_log_file="/var/log/mariadb/slow.log"
log_queries_not_using_indexes
log_slow_admin_statements
long_query_time=10
[galera]
wsrep_node_address=192.168.56.10
wsrep_node_name=db1
wsrep_cluster_address='gcomm://192.168.56.10,192.168.56.20,192.168.56.30'
wsrep_cluster_name=DevelopCluster
wsrep_provider=/usr/lib64/galera/libgalera_smm.so
binlog_format=row
default_storage_engine=InnoDB
innodb_autoinc_lock_mode=2
wsrep_on=ON
wsrep_sst_method=rsync
wsrep_slave_threads=2
innodb_locks_unsafe_for_binlog=1
[embedded]
[mariadb]
[mariadb-10.3]
db2の初期設定
sudo vi /etc/my.cnf.d/server.cnf
sudo mkdir /var/log/mariadb
sudo chown -R mysql:mysql /var/log/mariadb
[server]
[mysqld]
character-set-server=utf8
log-output=FILE
# Error log
log-error="/var/log/mariadb/mysqld.log"
# Query log
general-log=0
general_log_file="/var/log/mariadb/sql.log"
# Slow Query log
slow-query-log=1
slow_query_log_file="/var/log/mariadb/slow.log"
log_queries_not_using_indexes
log_slow_admin_statements
long_query_time=10
[galera]
wsrep_node_address=192.168.56.20
wsrep_node_name=db2
wsrep_cluster_address='gcomm://192.168.56.10,192.168.56.20,192.168.56.30'
wsrep_cluster_name=DevelopCluster
wsrep_provider=/usr/lib64/galera/libgalera_smm.so
binlog_format=row
default_storage_engine=InnoDB
innodb_autoinc_lock_mode=2
wsrep_on=ON
wsrep_sst_method=rsync
wsrep_slave_threads=2
innodb_locks_unsafe_for_binlog=1
[embedded]
[mariadb]
[mariadb-10.3]
db3の初期設定
sudo vi /etc/my.cnf.d/server.cnf
sudo mkdir -p /var/log/mariadb
sudo chown -R mysql:mysql /var/log/mariadb
[server]
[mysqld]
character-set-server=utf8
log-output=FILE
# Error log
log-error="/var/log/mariadb/mysqld.log"
# Query log
general-log=0
general_log_file="/var/log/mariadb/sql.log"
# Slow Query log
slow-query-log=1
slow_query_log_file="/var/log/mariadb/slow.log"
log_queries_not_using_indexes
log_slow_admin_statements
long_query_time=10
[galera]
wsrep_node_address=192.168.56.30
wsrep_node_name=db3
wsrep_cluster_address='gcomm://192.168.56.10,192.168.56.20,192.168.56.30'
wsrep_cluster_name=DevelopCluster
wsrep_provider=/usr/lib64/galera/libgalera_smm.so
binlog_format=row
default_storage_engine=InnoDB
innodb_autoinc_lock_mode=2
wsrep_on=ON
wsrep_sst_method=rsync
wsrep_slave_threads=2
innodb_locks_unsafe_for_binlog=1
[embedded]
[mariadb]
[mariadb-10.3]
Galera Clusterの起動
db1のMariaDB Galera Cluster起動確認
db1の起動確認です。
このとき、db2とdb3は停止状態です。
※ Galera Clusterでは、初期設定ファイルの「wsrep_cluster_address」に記述されたノードのどれかが起動していないと、起動できません。
1台目（ファーストノード）を起動する際には、他のノードは停止状態です。
ファーストノードはgalera_new_clusterコマンドで起動します。
sudo galera_new_cluster
show status like 'wsrep_%';を実行して以下の事項を確認してください。
[確認事項]
- wsrep_cluster_status: Primary
- wsrep_incoming_addresses: 設定したアドレス
- wsrep_local_state_comment: Synced
[vagrant@db1 my.cnf.d]$ mysql -u root -p
Enter password:
MariaDB [(none)]> show status like 'wsrep_%';
+------------------------------+--------------------------------------+
| Variable_name | Value |
+------------------------------+--------------------------------------+
| wsrep_apply_oooe | 0.000000 |
| wsrep_apply_oool | 0.000000 |
| wsrep_apply_window | 0.000000 |
| wsrep_causal_reads | 0 |
| wsrep_cert_deps_distance | 0.000000 |
| wsrep_cert_index_size | 0 |
| wsrep_cert_interval | 0.000000 |
| wsrep_cluster_conf_id | 1 |
| wsrep_cluster_size | 1 |
| wsrep_cluster_state_uuid | 5ea4eb1f-8bff-11e8-934d-7aa2eaa06ac1 |
| wsrep_cluster_status | Primary |
| wsrep_commit_oooe | 0.000000 |
| wsrep_commit_oool | 0.000000 |
| wsrep_commit_window | 0.000000 |
| wsrep_connected | ON |
| wsrep_desync_count | 0 |
| wsrep_evs_delayed | |
| wsrep_evs_evict_list | |
| wsrep_evs_repl_latency | 0/0/0/0/0 |
| wsrep_evs_state | OPERATIONAL |
| wsrep_flow_control_paused | 0.000000 |
| wsrep_flow_control_paused_ns | 0 |
| wsrep_flow_control_recv | 0 |
| wsrep_flow_control_sent | 0 |
| wsrep_gcomm_uuid | 5ea35535-8bff-11e8-90ef-6b5cfed70296 |
| wsrep_incoming_addresses | 192.168.56.10:3306 |
| wsrep_last_committed | 0 |
| wsrep_local_bf_aborts | 0 |
| wsrep_local_cached_downto | 18446744073709551615 |
| wsrep_local_cert_failures | 0 |
| wsrep_local_commits | 0 |
| wsrep_local_index | 0 |
| wsrep_local_recv_queue | 0 |
| wsrep_local_recv_queue_avg | 0.500000 |
| wsrep_local_recv_queue_max | 2 |
| wsrep_local_recv_queue_min | 0 |
| wsrep_local_replays | 0 |
| wsrep_local_send_queue | 0 |
| wsrep_local_send_queue_avg | 0.000000 |
| wsrep_local_send_queue_max | 1 |
| wsrep_local_send_queue_min | 0 |
| wsrep_local_state | 4 |
| wsrep_local_state_comment | Synced |
| wsrep_local_state_uuid | 5ea4eb1f-8bff-11e8-934d-7aa2eaa06ac1 |
| wsrep_protocol_version | 8 |
| wsrep_provider_name | Galera |
| wsrep_provider_vendor | Codership Oy <info@codership.com> |
| wsrep_provider_version | 25.3.23(r3789) |
| wsrep_ready | ON |
| wsrep_received | 2 |
| wsrep_received_bytes | 140 |
| wsrep_repl_data_bytes | 0 |
| wsrep_repl_keys | 0 |
| wsrep_repl_keys_bytes | 0 |
| wsrep_repl_other_bytes | 0 |
| wsrep_replicated | 0 |
| wsrep_replicated_bytes | 0 |
| wsrep_thread_count | 3 |
+------------------------------+--------------------------------------+
58 rows in set (0.001 sec)
MariaDB [(none)]> exit
db2のMariaDB Galera Cluster起動確認
db2の起動確認になります。
このときは、db1は起動状態で、db3は停止状態です。
sudo systemctl start mariadb
クラスタに参加できていることを確認してください。
$ mysql -u root -p
Enter password:
Welcome to the MariaDB monitor. Commands end with ; or \g.
Your MariaDB connection id is 11
Server version: 10.3.8-MariaDB MariaDB Server
Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
MariaDB [(none)]> show status like 'wsrep_%';
+------------------------------+---------------------------------------+
| Variable_name | Value |
+------------------------------+---------------------------------------+
| wsrep_apply_oooe | 0.000000 |
| wsrep_apply_oool | 0.000000 |
| wsrep_apply_window | 0.000000 |
| wsrep_causal_reads | 0 |
| wsrep_cert_deps_distance | 0.000000 |
| wsrep_cert_index_size | 0 |
| wsrep_cert_interval | 0.000000 |
| wsrep_cluster_conf_id | 2 |
| wsrep_cluster_size | 2 |
| wsrep_cluster_state_uuid | 5ea4eb1f-8bff-11e8-934d-7aa2eaa06ac1 |
| wsrep_cluster_status | Primary |
| wsrep_commit_oooe | 0.000000 |
| wsrep_commit_oool | 0.000000 |
| wsrep_commit_window | 0.000000 |
| wsrep_connected | ON |
| wsrep_desync_count | 0 |
| wsrep_evs_delayed | |
| wsrep_evs_evict_list | |
| wsrep_evs_repl_latency | 0/0/0/0/0 |
| wsrep_evs_state | OPERATIONAL |
| wsrep_flow_control_paused | 0.000000 |
| wsrep_flow_control_paused_ns | 0 |
| wsrep_flow_control_recv | 0 |
| wsrep_flow_control_sent | 0 |
| wsrep_gcomm_uuid | 58f28b22-8c03-11e8-b913-db00a1269d15 |
| wsrep_incoming_addresses | 192.168.56.10:3306,192.168.56.20:3306 |
| wsrep_last_committed | 0 |
| wsrep_local_bf_aborts | 0 |
| wsrep_local_cached_downto | 18446744073709551615 |
| wsrep_local_cert_failures | 0 |
| wsrep_local_commits | 0 |
| wsrep_local_index | 1 |
| wsrep_local_recv_queue | 0 |
| wsrep_local_recv_queue_avg | 0.000000 |
| wsrep_local_recv_queue_max | 1 |
| wsrep_local_recv_queue_min | 0 |
| wsrep_local_replays | 0 |
| wsrep_local_send_queue | 0 |
| wsrep_local_send_queue_avg | 0.000000 |
| wsrep_local_send_queue_max | 1 |
| wsrep_local_send_queue_min | 0 |
| wsrep_local_state | 4 |
| wsrep_local_state_comment | Synced |
| wsrep_local_state_uuid | 5ea4eb1f-8bff-11e8-934d-7aa2eaa06ac1 |
| wsrep_protocol_version | 8 |
| wsrep_provider_name | Galera |
| wsrep_provider_vendor | Codership Oy <info@codership.com> |
| wsrep_provider_version | 25.3.23(r3789) |
| wsrep_ready | ON |
| wsrep_received | 3 |
| wsrep_received_bytes | 216 |
| wsrep_repl_data_bytes | 0 |
| wsrep_repl_keys | 0 |
| wsrep_repl_keys_bytes | 0 |
| wsrep_repl_other_bytes | 0 |
| wsrep_replicated | 0 |
| wsrep_replicated_bytes | 0 |
| wsrep_thread_count | 3 |
+------------------------------+---------------------------------------+
58 rows in set (0.001 sec)
MariaDB [(none)]> exit
db3のMariaDB Galera Cluster起動確認
db3の起動確認になります。
sudo systemctl start mariadb
mysql -u root -p
Enter password:
Welcome to the MariaDB monitor. Commands end with ; or \g.
Your MariaDB connection id is 11
Server version: 10.3.8-MariaDB MariaDB Server
Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
MariaDB [(none)]> show status like 'wsrep_%';
+------------------------------+----------------------------------------------------------+
| Variable_name | Value |
+------------------------------+----------------------------------------------------------+
| wsrep_apply_oooe | 0.000000 |
| wsrep_apply_oool | 0.000000 |
| wsrep_apply_window | 0.000000 |
| wsrep_causal_reads | 0 |
| wsrep_cert_deps_distance | 0.000000 |
| wsrep_cert_index_size | 0 |
| wsrep_cert_interval | 0.000000 |
| wsrep_cluster_conf_id | 3 |
| wsrep_cluster_size | 3 |
| wsrep_cluster_state_uuid | 5ea4eb1f-8bff-11e8-934d-7aa2eaa06ac1 |
| wsrep_cluster_status | Primary |
| wsrep_commit_oooe | 0.000000 |
| wsrep_commit_oool | 0.000000 |
| wsrep_commit_window | 0.000000 |
| wsrep_connected | ON |
| wsrep_desync_count | 0 |
| wsrep_evs_delayed | |
| wsrep_evs_evict_list | |
| wsrep_evs_repl_latency | 0.000878738/0.00319743/0.00973027/0.003294/5 |
| wsrep_evs_state | OPERATIONAL |
| wsrep_flow_control_paused | 0.000000 |
| wsrep_flow_control_paused_ns | 0 |
| wsrep_flow_control_recv | 0 |
| wsrep_flow_control_sent | 0 |
| wsrep_gcomm_uuid | f93e0362-8c03-11e8-b220-a3600473017a |
| wsrep_incoming_addresses | 192.168.56.10:3306,192.168.56.20:3306,192.168.56.30:3306 |
| wsrep_last_committed | 0 |
| wsrep_local_bf_aborts | 0 |
| wsrep_local_cached_downto | 18446744073709551615 |
| wsrep_local_cert_failures | 0 |
| wsrep_local_commits | 0 |
| wsrep_local_index | 2 |
| wsrep_local_recv_queue | 0 |
| wsrep_local_recv_queue_avg | 0.000000 |
| wsrep_local_recv_queue_max | 1 |
| wsrep_local_recv_queue_min | 0 |
| wsrep_local_replays | 0 |
| wsrep_local_send_queue | 0 |
| wsrep_local_send_queue_avg | 0.000000 |
| wsrep_local_send_queue_max | 1 |
| wsrep_local_send_queue_min | 0 |
| wsrep_local_state | 4 |
| wsrep_local_state_comment | Synced |
| wsrep_local_state_uuid | 5ea4eb1f-8bff-11e8-934d-7aa2eaa06ac1 |
| wsrep_protocol_version | 8 |
| wsrep_provider_name | Galera |
| wsrep_provider_vendor | Codership Oy <info@codership.com> |
| wsrep_provider_version | 25.3.23(r3789) |
| wsrep_ready | ON |
| wsrep_received | 3 |
| wsrep_received_bytes | 284 |
| wsrep_repl_data_bytes | 0 |
| wsrep_repl_keys | 0 |
| wsrep_repl_keys_bytes | 0 |
| wsrep_repl_other_bytes | 0 |
| wsrep_replicated | 0 |
| wsrep_replicated_bytes | 0 |
| wsrep_thread_count | 3 |
+------------------------------+----------------------------------------------------------+
58 rows in set (0.001 sec)
MariaDB [(none)]> exit
データ同期の確認
- db1でデータベース作成
[vagrant@db1 ~]$ mysql -u root -p
Enter password:
Welcome to the MariaDB monitor. Commands end with ; or \g.
Your MariaDB connection id is 11
Server version: 10.3.8-MariaDB MariaDB Server
Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
MariaDB [(none)]> create database dev_nagakuray;
Query OK, 1 row affected (0.002 sec)
MariaDB [(none)]> show databases;
+--------------------+
| Database |
+--------------------+
| dev_nagakuray |
| information_schema |
| mysql |
| performance_schema |
+--------------------+
4 rows in set (0.001 sec)
MariaDB [(none)]> use dev_nagakuray
Database changed
MariaDB [dev_nagakuray]> create table test (id int, name varchar(100));
Query OK, 0 rows affected (0.008 sec)
MariaDB [dev_nagakuray]> insert into test (id, name) values (1, 'hogehoge');
Query OK, 1 row affected (0.003 sec)
MariaDB [dev_nagakuray]>
- db2で確認
[vagrant@db2 ~]$ mysql -u root -p
Enter password:
Welcome to the MariaDB monitor. Commands end with ; or \g.
Your MariaDB connection id is 11
Server version: 10.3.8-MariaDB MariaDB Server
Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
MariaDB [(none)]> show databases;
+--------------------+
| Database |
+--------------------+
| dev_nagakuray |
| information_schema |
| mysql |
| performance_schema |
+--------------------+
4 rows in set (0.001 sec)
MariaDB [(none)]> use dev_nagakuray
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
Database changed
MariaDB [dev_nagakuray]> select * from test;
+------+----------+
| id | name |
+------+----------+
| 1 | hogehoge |
+------+----------+
1 row in set (0.000 sec)
MariaDB [dev_nagakuray]> insert into test (id, name) values (2, 'fugafuga');
Query OK, 1 row affected (0.005 sec)
- db3で確認
[vagrant@db3 ~]$ mysql -u root -p
Enter password:
Welcome to the MariaDB monitor. Commands end with ; or \g.
Your MariaDB connection id is 12
Server version: 10.3.8-MariaDB MariaDB Server
Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
MariaDB [(none)]> show databases;
+--------------------+
| Database |
+--------------------+
| dev_nagakuray |
| information_schema |
| mysql |
| performance_schema |
+--------------------+
4 rows in set (0.000 sec)
MariaDB [(none)]> use dev_nagakuray
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
Database changed
MariaDB [dev_nagakuray]> select * from test;
+------+----------+
| id | name |
+------+----------+
| 1 | hogehoge |
| 2 | fugafuga |
+------+----------+
2 rows in set (0.000 sec)
MariaDB [dev_nagakuray]> insert into test (id, name) values (3, 'hanako');
Query OK, 1 row affected (0.003 sec)
注意事項: MariaDB Galera Clusterの停止→起動の順序について
- 起動状態時のdb1
sudo cat /var/lib/mysql/grastate.dat
# GALERA saved state
version: 2.1
uuid: 5ea4eb1f-8bff-11e8-934d-7aa2eaa06ac1
seqno: -1
safe_to_bootstrap: 0
- 起動状態時のdb2
sudo cat /var/lib/mysql/grastate.dat
# GALERA saved state
version: 2.1
uuid: 5ea4eb1f-8bff-11e8-934d-7aa2eaa06ac1
seqno: -1
safe_to_bootstrap: 0
- 起動状態時のdb3
sudo cat /var/lib/mysql/grastate.dat
# GALERA saved state
version: 2.1
uuid: 5ea4eb1f-8bff-11e8-934d-7aa2eaa06ac1
seqno: -1
safe_to_bootstrap: 0
上記の通り、該当ファイルの「safe_to_bootstrap」の値は「0」となっています。
では、1号機 → 2号機 → 3号機 の順番に停止します。
[vagrant@db1 ~]$ sudo systemctl stop mariadb
[vagrant@db2 ~]$ sudo systemctl stop mariadb
[vagrant@db3 ~]$ sudo systemctl stop mariadb
この時の/var/lib/mysql/grastate.datを確認します。
- 停止状態時のdb1
[vagrant@db1 ~]$ sudo cat /var/lib/mysql/grastate.dat
# GALERA saved state
version: 2.1
uuid: 5ea4eb1f-8bff-11e8-934d-7aa2eaa06ac1
seqno: 0
safe_to_bootstrap: 0
- 停止状態時のdb2
[vagrant@db2 ~]$ sudo cat /var/lib/mysql/grastate.dat
# GALERA saved state
version: 2.1
uuid: 5ea4eb1f-8bff-11e8-934d-7aa2eaa06ac1
seqno: 0
safe_to_bootstrap: 0
- 停止状態時のdb3
[vagrant@db3 ~]$ sudo cat /var/lib/mysql/grastate.dat
# GALERA saved state
version: 2.1
uuid: 5ea4eb1f-8bff-11e8-934d-7aa2eaa06ac1
seqno: 0
safe_to_bootstrap: 1
この時、1号機から起動すると、以下の通りエラーになります。
[vagrant@db1 ~]$ sudo galera_new_cluster
Job for mariadb.service failed because the control process exited with error code. See "systemctl status mariadb.service" and "journalctl -xe" for details.
safe_to_bootstrap: 1となっている3号機から起動する必要があります
[vagrant@db3 ~]$ sudo galera_new_cluster
[vagrant@db1 ~]$ sudo systemctl start mariadb
[vagrant@db2 ~]$ sudo systemctl start mariadb