参考
1.構成
- CentOS5.11 x86_64
- IP
- drbd1.local: 192.168.100.199
- drbd2.local: 192.168.100.200
- drbd仮想IP: 192.168.100.198
- DRBD
- /dev/vdb1
- リソース
- mysqldata : /var/lib/mysql/
2.リポジトリ追加
epel-release
yum install -y epel-release
clusterlabs.repo
cd /etc/yum.repos.d/
wget http://clusterlabs.org/rpm/epel-5/clusterlabs.repo
パッケージインストール
yum install {corosync,heartbeat,pacemaker}.x86_64
- i386のパッケージが入らないように、x86_64を指定。
3.corosync設定
/etc/corosync/corosync.conf
aisexec {
user: root
group: root
}
service {
name: pacemaker
ver: 0
use_mgmtd: yes
}
totem {
version: 2
secauth: off
threads: 0
rrp_mode: active
clear_node_high_bit: yes
token: 4000
consensus: 10000
rrp_problem_count_timeout: 3000
interface {
ringnumber: 0
bindnetaddr: 192.168.100.0
mcastaddr: 226.94.1.1
mcastport: 5405
}
}
logging {
fileline: on
to_syslog: yes
syslog_facility: local1
syslog_priority: info
debug: off
timestamp: on
}
4.syslog
/etc/syslog.conf
- *.info;mail.none;authpriv.none;cron.none /var/log/messages
+ *.info;mail.none;authpriv.none;cron.none;local1.none /var/log/messages
+ local1.* /var/log/pacemaker.log
- /var/log/pacemaker.log にログが記録されます。
5.corosync
5.1. 起動
/sbin/service corosync start
5.2. STONITH無効化
PATHを通す
export PATH=$PATH:/usr/sbin:/sbin
以下は一台だけで実施
確認
# /usr/sbin/crm_verify -L
crm_verify[2097]: 2015/03/16_16:42:10 ERROR: unpack_resources: Resource start-up disabled since no STONITH resources have been defined
crm_verify[2097]: 2015/03/16_16:42:10 ERROR: unpack_resources: Either configure some or disable STONITH with the stonith-enabled option
crm_verify[2097]: 2015/03/16_16:42:10 ERROR: unpack_resources: NOTE: Clusters with shared data need STONITH to ensure data integrity
Errors found during check: config not valid
-V may provide more details
STONITH無効化
# crm configure property stonith-enabled=false
INFO: building help index
確認
# crm_verify -L
5.3. 仮想IPの設定
これは一台だけで実施
# crm configure \
primitive \
ClusterIP \
ocf:heartbeat:IPaddr2 \
params \
ip=192.168.100.198 \
cidr_netmask=24 \
nic=eth0 \
iflabel=0 \
op \
monitor \
interval=30s \
nic=eth0 \
iflabel=:0
crm_verify[2171]: 2015/03/16_16:46:44 WARN: cluster_status: We do not have quorum - fencing and resource management disabled
5.4. quorum無効化
これは一台だけで実施
# crm configure property no-quorum-policy=ignore
ノード復帰時に自動でリソースを元のノード上に移そうとするのを止める
# crm configure rsc_defaults resource-stickiness=100
6. DRBD
6.1. インストール
yum install drbd83 drbd83-kmod
chkconfig drbd off
6.2. 設定
/etc/drbd.conf
global {
usage-count no;
}
common {
protocol C;
}
resource mysqldata{
meta-disk internal;
device /dev/drbd0;
syncer {
verify-alg sha1;
rate 80M;
}
on drbd1.local {
disk /dev/vdb1;
address 192.168.100.199:7789;
}
on drbd2.local {
disk /dev/vdb1;
address 192.168.100.200:7789;
}
}
6.3 hostsにて名前解決
/etc/hosts
+ 192.168.100.199 drbd1 drbd1.local
+ 192.168.100.200 drbd2 drbd2.local
6.4. create-md
# drbdadm create-md mysqldata
Writing meta data...
initializing activity log
NOT initialized bitmap
New drbd meta data block successfully created.
modprobe drbd
drbdadm up mysqldata
6.5. DRBDデバイスの状態確認
/proc/drbd
# cat /proc/drbd
version: 8.3.15 (api:88/proto:86-97)
GIT-hash: 0ce4d235fc02b5c53c1c52c53433d11a694eab8c build by mockbuild@builder10.centos.org, 2013-03-27 16:01:26
1: cs:WFConnection ro:Secondary/Unknown ds:Inconsistent/DUnknown C r----s
ns:0 nr:0 dw:0 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b oos:4194092
6.6. DRBDデバイスの初期化
初期化
drbdadm -- --overwrite-data-of-peer primary mysqldata
mkfs.ext3 /dev/drbd0
同期が終わるまで待つ。
mkdir /var/lib/mysql/
mount /dev/drbd0 /var/lib/mysql/
確認
# df -h /var/lib/mysql/
Filesystem サイズ 使用 残り 使用% マウント位置
/dev/drbd0 4.0G 73M 3.7G 2% /var/lib/mysql
7. crm (pacemaker)
- corosyncが起動していること
- 2台目も動作していること
7.1. DRBDの登録
# crm
crm(live)# cib new drbd
INFO: building help index
INFO: drbd shadow CIB created
rm(drbd)# configure primitive MySQLData ocf:heartbeat:drbd params drbd_resource=mysqldata op monitor interval=60s
WARNING: MySQLData: default timeout 20s for start is smaller than the advised 240
WARNING: MySQLData: default timeout 20s for stop is smaller than the advised 100
WARNING: MySQLData: action monitor not advertised in meta-data, it may not be supported by the RA
WARNING: MySQLData: default timeout 20s for start is smaller than the advised 240
WARNING: MySQLData: default timeout 20s for stop is smaller than the advised 100
WARNING: MySQLData: action monitor not advertised in meta-data, it may not be supported by the RA
crm(drbd)# configure ms MySQLDataClone MySQLData meta master-max=1 master-node-max=1 clone-max=2 clone-node-max=1 notify=true
crm(drbd)# cib commit drbd
INFO: commited 'drbd' shadow CIB to the cluster
crm(drbd)# quit
7.2. /var/lib/mysql/の登録
# crm
crm(live)# cib new fs
INFO: fs shadow CIB created
crm(fs)# configure primitive MySQLFS ocf:heartbeat:Filesystem params device="/dev/drbd0" directory="/var/lib/mysql" fstype="ext3"
WARNING: MySQLFS: default timeout 20s for start is smaller than the advised 60
WARNING: MySQLFS: default timeout 20s for stop is smaller than the advised 60
WARNING: MySQLFS: default timeout 20s for start is smaller than the advised 60
WARNING: MySQLFS: default timeout 20s for stop is smaller than the advised 60
DRBDと同じノード上で動作させる(colocation)
crm(fs)# configure colocation fs_on_drbd inf: MySQLFS MySQLDataClone:Master
DRBDの後にmount(order)
crm(fs)# configure order MySQLFS-after-MySQLData inf: MySQLDataClone:promote MySQLFS:start
crm(fs)# cib commit fs
crm(fs)# quit
7.3. MySQL
yum install mysql-server
初期設定するためにstart,stopする
service mysqld start
service mysqld stop
# ls /var/lib/mysql
ib_logfile0 ib_logfile1 ibdata1 lost+found mysql test
umount /var/lib/mysql
7.4. pacemakerにMySQL登録
# crm
crm(live)# configure
crm(live)configure# primitive MySQL ocf:heartbeat:mysql params binary=/usr/bin/mysqld_safe pid=/var/run/mysqld/mysqld.pid op monitor interval=60s
WARNING: MySQL: default timeout 20s for start is smaller than the advised 120
WARNING: MySQL: default timeout 20s for stop is smaller than the advised 120
WARNING: MySQL: default timeout 20s for monitor is smaller than the advised 30
DRBDと同じノード上で動作させる(colocation)
crm(live)configure# colocation MySQL-with-MySQLFS inf: MySQL MySQLFS
DRBDの後にmount(order)
crm(live)configure# order MySQL-after-MySQLFS inf: MySQLFS MySQL
DRBDと同じノード上で動作させる(colocation)
crm(live)configure# colocation MySQL-with-ClusterIP inf: MySQL ClusterIP
DRBDの後にmount(order)
crm(live)configure# order MySQL-after-ClusterIP inf: ClusterIP MySQL
crm(live)configure# commit
WARNING: MySQL: default timeout 20s for start is smaller than the advised 120
WARNING: MySQL: default timeout 20s for stop is smaller than the advised 120
WARNING: MySQL: default timeout 20s for monitor is smaller than the advised 30
7.5. クラスタグループの作成
# crm configure group MySQLGroup MySQLFS ClusterIP MySQL
INFO: resource references in colocation:fs_on_drbd updated
INFO: resource references in colocation:MySQL-with-MySQLFS updated
INFO: resource references in order:MySQLFS-after-MySQLData updated
INFO: resource references in order:MySQL-after-MySQLFS updated
INFO: resource references in colocation:MySQL-with-ClusterIP updated
INFO: resource references in order:MySQL-after-ClusterIP updated
INFO: resource references in colocation:MySQL-with-MySQLFS updated
INFO: resource references in colocation:MySQL-with-ClusterIP updated
INFO: resource references in order:MySQL-after-MySQLFS updated
INFO: resource references in order:MySQL-after-ClusterIP updated
- crm configure show
8. 動作確認
OS起動後、
drbd1, drbd2 両方にて/sbin/service corosync start
8.1. drbd1(Master), drbd2(Slave)
crm_mon -A
Online: [ drbd1.local drbd2.local ]
Master/Slave Set: MySQLDataClone
Masters: [ drbd1.local ]
Slaves: [ drbd2.local ]
Resource Group: MySQLGroup
MySQLFS (ocf::heartbeat:Filesystem): Started drbd1.local
ClusterIP (ocf::heartbeat:IPaddr2): Started drbd1.local
MySQL (ocf::heartbeat:mysql): Started drbd1.local
Node Attributes:
* Node drbd1.local:
+ master-MySQLData:0 : 10
* Node drbd2.local:
+ master-MySQLData:1 : 10
8.2. drbd1(Slave), drbd2(Master)に切替
drbd1をオフラインにする
crm node standby drbd1.local
crm_mon -A
Node drbd1.local: standby
Online: [ drbd2.local ]
Master/Slave Set: MySQLDataClone
Masters: [ drbd2.local ]
Stopped: [ MySQLData:0 ]
Resource Group: MySQLGroup
MySQLFS (ocf::heartbeat:Filesystem): Started drbd2.local
ClusterIP (ocf::heartbeat:IPaddr2): Started drbd2.local
MySQL (ocf::heartbeat:mysql): Started drbd2.local
Node Attributes:
* Node drbd1.local:
* Node drbd2.local:
+ master-MySQLData:1 : 5
drbd1をオンラインにする
crm node online drbd1.local
Online: [ drbd1.local drbd2.local ]
Master/Slave Set: MySQLDataClone
Masters: [ drbd2.local ]
Slaves: [ drbd1.local ]
Resource Group: MySQLGroup
MySQLFS (ocf::heartbeat:Filesystem): Started drbd2.local
ClusterIP (ocf::heartbeat:IPaddr2): Started drbd2.local
MySQL (ocf::heartbeat:mysql): Started drbd2.local
Node Attributes:
* Node drbd1.local:
+ master-MySQLData:0 : 10
* Node drbd2.local:
+ master-MySQLData:1 : 5
8.3. drbd1(Master), drbd2(Slave)に戻す
crm node standby drbd1.local
crm node online drbd1.local
9. 障害テスト
- drbd2がMasterの時、drbd2を再起動
- drbd1がMasterに切り替わる