Go to Qiita Advent Calendar 2024 Top
LoginSignup
5
5

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?

More than 5 years have passed since last update.

@ochiba(Hiroki KUNITAKE)

iSCSIデバイスでmultipathを組んでLVMを作成する

Last updated at Posted at 2019-02-05

仮想環境でiSCSI target/initiator、multipath、LVMを試してみた。

環境

CentOS 7.6 (VirtualBox, vagrant box を利用)

Hostname IP address Description
svr00 192.168.33.100, 192.168.33.110 iSCSI ターゲット
svr01 192.168.33.101 iSCSI イニシエータ

svr00: iSCSIターゲットの構成

targetcli の導入と target.service の起動

# yum -y install targetcli
# systemctl enable target.service
# systemctl start target.service
# systemctl status target.service
● target.service - Restore LIO kernel target configuration
   Loaded: loaded (/usr/lib/systemd/system/target.service; enabled; vendor preset: disabled)
   Active: active (exited) since Tue 2019-02-05 12:22:22 UTC; 8min ago
  Process: 24553 ExecStart=/usr/bin/targetctl restore (code=exited, status=0/SUCCESS)
 Main PID: 24553 (code=exited, status=0/SUCCESS)

iSCSIターゲット初期状態

# targetcli ls /
o- / ............................................................................. [...]
  o- backstores .................................................................. [...]
  | o- block ...................................................... [Storage Objects: 0]
  | o- fileio ..................................................... [Storage Objects: 0]
  | o- pscsi ...................................................... [Storage Objects: 0]
  | o- ramdisk .................................................... [Storage Objects: 0]
  o- iscsi ................................................................ [Targets: 0]
  o- loopback ............................................................. [Targets: 0]

backstoreを作成（面倒だったのでFILEIOで作成、ディスク付ける場合はBLOCKIOで作成する）

# mkdir /var/iscsi
# targetcli /backstores/fileio create file00 /var/iscsi/disk00.img 10MB write_back=false
# targetcli ls /
o- / ............................................................................. [...]
  o- backstores .................................................................. [...]
  | o- block ...................................................... [Storage Objects: 0]
  | o- fileio ..................................................... [Storage Objects: 1]
  | | o- file00 ............... [/var/iscsi/disk00.img (10.0MiB) write-thru deactivated]
  | |   o- alua ....................................................... [ALUA Groups: 1]
  | |     o- default_tg_pt_gp ........................... [ALUA state: Active/optimized]
  | o- pscsi ...................................................... [Storage Objects: 0]
  | o- ramdisk .................................................... [Storage Objects: 0]
  o- iscsi ................................................................ [Targets: 0

IQNの定義（命名規則はRFC 3721を参照）

# targetcli /iscsi create iqn.2000-08.jp.co.example:svr00
# targetcli ls /
o- / ............................................................................. [...]
  o- backstores .................................................................. [...]
  | o- block ...................................................... [Storage Objects: 0]
  | o- fileio ..................................................... [Storage Objects: 1]
  | | o- file00 ............... [/var/iscsi/disk00.img (10.0MiB) write-thru deactivated]
  | |   o- alua ....................................................... [ALUA Groups: 1]
  | |     o- default_tg_pt_gp ........................... [ALUA state: Active/optimized]
  | o- pscsi ...................................................... [Storage Objects: 0]
  | o- ramdisk .................................................... [Storage Objects: 0]
  o- iscsi ................................................................ [Targets: 1]
  | o- iqn.2000-08.jp.co.example:svr00 ....................................... [TPGs: 1]
  |   o- tpg1 ................................................... [no-gen-acls, no-auth]
  |     o- acls .............................................................. [ACLs: 0]
  |     o- luns .............................................................. [LUNs: 0]
  |     o- portals ........................................................ [Portals: 1]
  |       o- 0.0.0.0:3260 ......................................................... [OK]
  o- loopback ............................................................. [Targets: 0]

IQN定義時にTPG（Target Portal Group)が1つ作成される。
Portalは「0.0.0.0:3260」が定義されているのでこのまま利用。
LUNは先に作成したbackstore(file00)を紐付ける。

# targetcli /iscsi/iqn.2000-08.jp.co.example:svr00/tpg1/luns create /backstores/fileio/file00
# targetcli ls /
o- / ............................................................................. [...]
  o- backstores .................................................................. [...]
  | o- block ...................................................... [Storage Objects: 0]
  | o- fileio ..................................................... [Storage Objects: 1]
  | | o- file00 ................. [/var/iscsi/disk00.img (10.0MiB) write-thru activated]
  | |   o- alua ....................................................... [ALUA Groups: 1]
  | |     o- default_tg_pt_gp ........................... [ALUA state: Active/optimized]
  | o- pscsi ...................................................... [Storage Objects: 0]
  | o- ramdisk .................................................... [Storage Objects: 0]
  o- iscsi ................................................................ [Targets: 1]
  | o- iqn.2000-08.jp.co.example:svr00 ....................................... [TPGs: 1]
  |   o- tpg1 ................................................... [no-gen-acls, no-auth]
  |     o- acls .............................................................. [ACLs: 0]
  |     o- luns .............................................................. [LUNs: 1]
  |     | o- lun0 ........... [fileio/file00 (/var/iscsi/disk00.img) (default_tg_pt_gp)]
  |     o- portals ........................................................ [Portals: 1]
  |       o- 0.0.0.0:3260 ......................................................... [OK]
  o- loopback ............................................................. [Targets: 0]

ACLを定義（svr01のiSCSIイニシエータ名はiqn.2000-08.jp.co.example:svr01とする）

# targetcli /iscsi/iqn.2000-08.jp.co.example:svr00/tpg1/acls create iqn.2000-08.jp.co.example:svr01
# targetcli ls /
o- / ............................................................................. [...]
  o- backstores .................................................................. [...]
  | o- block ...................................................... [Storage Objects: 0]
  | o- fileio ..................................................... [Storage Objects: 1]
  | | o- file00 ................. [/var/iscsi/disk00.img (10.0MiB) write-thru activated]
  | |   o- alua ....................................................... [ALUA Groups: 1]
  | |     o- default_tg_pt_gp ........................... [ALUA state: Active/optimized]
  | o- pscsi ...................................................... [Storage Objects: 0]
  | o- ramdisk .................................................... [Storage Objects: 0]
  o- iscsi ................................................................ [Targets: 1]
  | o- iqn.2000-08.jp.co.example:svr00 ....................................... [TPGs: 1]
  |   o- tpg1 ................................................... [no-gen-acls, no-auth]
  |     o- acls .............................................................. [ACLs: 1]
  |     | o- iqn.2000-08.jp.co.example:svr01 .......................... [Mapped LUNs: 1]
  |     |   o- mapped_lun0 ................................... [lun0 fileio/file00 (rw)]
  |     o- luns .............................................................. [LUNs: 1]
  |     | o- lun0 ........... [fileio/file00 (/var/iscsi/disk00.img) (default_tg_pt_gp)]
  |     o- portals ........................................................ [Portals: 1]
  |       o- 0.0.0.0:3260 ......................................................... [OK]
  o- loopback ............................................................. [Targets: 0]

設定を保存

# targetcli saveconfig
Configuration saved to /etc/target/saveconfig.json

svr01: iSCSイニシエータの構成

iscsi-initiator-utils のインストール、イニシエータ名の設定

# yum -y install iscsi-initiator-utils
# echo "InitiatorName=iqn.2000-08.jp.co.example:svr01" > /etc/iscsi/initiatorname.iscsi

iSCSIデーモン(iscsid.service)の起動

# systemctl enable iscsid.service
# systemctl start iscsid.service
# systemctl status iscsid.service
● iscsid.service - Open-iSCSI
   Loaded: loaded (/usr/lib/systemd/system/iscsid.service; enabled; vendor preset: disabled)
   Active: active (running) since Tue 2019-02-05 13:20:14 UTC; 3s ago
     Docs: man:iscsid(8)
           man:iscsiadm(8)
  Process: 24840 ExecStart=/usr/sbin/iscsid (code=exited, status=0/SUCCESS)
 Main PID: 24842 (iscsid)
   CGroup: /system.slice/iscsid.service
           ├─24841 /usr/sbin/iscsid
           └─24842 /usr/sbin/iscsid

登録済みiSCSIターゲットの自動認識(iscsi.service)の有効化確認 ※ServiceTypeはoneshotなのでenableのみ

# systemctl status iscsi.service
● iscsi.service - Login and scanning of iSCSI devices
   Loaded: loaded (/usr/lib/systemd/system/iscsi.service; enabled; vendor preset: disabled)
   Active: inactive (dead)
     Docs: man:iscsid(8)
           man:iscsiadm(8)

block deviceの作業前確認

# lsblk -S
NAME HCTL       TYPE VENDOR   MODEL             REV TRAN
sda  0:0:0:0    disk ATA      VBOX HARDDISK    1.0  ata

iSCSIターゲットで指定したIQNを指定してログイン

# iscsiadm -m discovery -t st -p 192.168.33.100:3260
# iscsiadm -m discovery -t st -p 192.168.33.110:3260

# iscsiadm -m node -T iqn.2000-08.jp.co.example:svr00 -p 192.168.33.100:3260 --login
# iscsiadm -m node -T iqn.2000-08.jp.co.example:svr00 -p 192.168.33.110:3260 --login

# iscsiadm -m node
192.168.33.100:3260,1 iqn.2000-08.jp.co.example:svr00
192.168.33.110:3260,1 iqn.2000-08.jp.co.example:svr00

block deviceが増えたのを確認

# lsblk -S
NAME HCTL       TYPE VENDOR   MODEL             REV TRAN
sda  0:0:0:0    disk ATA      VBOX HARDDISK    1.0  ata
sdb  2:0:0:0    disk LIO-ORG  file00           4.0  iscsi
sdc  3:0:0:0    disk LIO-ORG  file00           4.0  iscsi

WWIDは揃っているので同じデバイスを見ている

# /usr/lib/udev/scsi_id --whitelisted --device=/dev/sdb
360014053ca1100494274260830c9109f
# /usr/lib/udev/scsi_id --whitelisted --device=/dev/sdc
360014053ca1100494274260830c9109f

svr01: multipathを組む

DM-Multipath をインストール

# yum -y install device-mapper-multipath

multipath.confを作成（mpathXという名前で自動作成する設定）

/etc/multipath.conf
defaults {
    user_friendly_names yes
    find_multipaths yes
}
blacklist {
}

dry-runして構成を確認

# multipath -v2 -d
create: mpatha (360014053ca1100494274260830c9109f) undef LIO-ORG ,file00
size=10M features='0' hwhandler='0' wp=undef
|-+- policy='service-time 0' prio=1 status=undef
| `- 2:0:0:0 sdb 8:16 undef ready running
`-+- policy='service-time 0' prio=1 status=undef
  `- 3:0:0:0 sdc 8:32 undef ready running

multipathd.serviceを起動してmultipath deviceが作成されたのを確認

# systemctl enable multipathd.service
# systemctl start multipathd.service
# systemctl status multipathd.service
● multipathd.service - Device-Mapper Multipath Device Controller
   Loaded: loaded (/usr/lib/systemd/system/multipathd.service; enabled; vendor preset: enabled)
   Active: active (running) since Tue 2019-02-05 13:52:17 UTC; 17s ago
  Process: 25312 ExecStart=/sbin/multipathd (code=exited, status=0/SUCCESS)
  Process: 25310 ExecStartPre=/sbin/multipath -A (code=exited, status=0/SUCCESS)
  Process: 25308 ExecStartPre=/sbin/modprobe dm-multipath (code=exited, status=0/SUCCESS)
 Main PID: 25315 (multipathd)
   CGroup: /system.slice/multipathd.service
           └─25315 /sbin/multipathd

# multipath -ll
mpatha (360014053ca1100494274260830c9109f) dm-0 LIO-ORG ,file00
size=10M features='0' hwhandler='0' wp=rw
|-+- policy='service-time 0' prio=1 status=enabled
| `- 2:0:0:0 sdb 8:16 active ready running
`-+- policy='service-time 0' prio=1 status=enabled
  `- 3:0:0:0 sdc 8:32 active ready running
Feb 05 13:52:20 | multipath device maps are present, but 'multipathd' service is not running
Feb 05 13:52:20 | IO failover/failback will not work without 'multipathd' service running

# ls -l /dev/mapper/mpatha
lrwxrwxrwx. 1 root root 7 Feb  5 13:52 /dev/mapper/mpatha -> ../dm-0

svr01: LVMを作成する

事前準備

折角なので multipath device を追加しておく。

  1. svr00: backstore 作成
  2. svr00: lun に追加
  3. svr01: iSCSI デバイスの再スキャン iscsiadm -m session --rescan
  4. svr01: (multipathd 起動していない場合) multipath device を作成

とりあえず今回は３つ用意（10MB x 2だとPE size変更してもファイルシステム作成時にblock数が足りなかったので後からmpathc足したのは内緒）

# multipath -ll
mpathc (360014058d5fa0d9125b45ec9aa4491d4) dm-2 LIO-ORG ,file02
size=100M features='0' hwhandler='0' wp=rw
|-+- policy='service-time 0' prio=1 status=active
| `- 3:0:0:2 sdg 8:96 active ready running
`-+- policy='service-time 0' prio=1 status=enabled
  `- 2:0:0:2 sdf 8:80 active ready running
mpathb (360014056b1f65811723430ebc736012b) dm-1 LIO-ORG ,file01
size=10M features='0' hwhandler='0' wp=rw
|-+- policy='service-time 0' prio=1 status=active
| `- 2:0:0:1 sde 8:64 active ready running
`-+- policy='service-time 0' prio=1 status=enabled
  `- 3:0:0:1 sdd 8:48 active ready running
mpatha (360014053ca1100494274260830c9109f) dm-0 LIO-ORG ,file00
size=10M features='0' hwhandler='0' wp=rw
|-+- policy='service-time 0' prio=1 status=active
| `- 2:0:0:0 sdb 8:16 active ready running
`-+- policy='service-time 0' prio=1 status=enabled
  `- 3:0:0:0 sdc 8:32 active ready running

PV,VG,LVの作成

PVの作成

# pvcreate /dev/mapper/mpatha
  Physical volume "/dev/mapper/mpatha" successfully created.

# pvcreate /dev/mapper/mpathb
  Physical volume "/dev/mapper/mpathb" successfully created.

# pvcreate /dev/mapper/mpathc
  Physical volume "/dev/mapper/mpathc" successfully created.

# pvs
  PV                 VG Fmt  Attr PSize  PFree
  /dev/mapper/mpatha    lvm2 ---  10.00m 10.00m
  /dev/mapper/mpathb    lvm2 ---  10.00m 10.00m
  /dev/mapper/mpathc    lvm2 ---  100.00m 100.00m

VGの作成（PE Size　を1MBに変更）

# vgcreate -s 1M VG0 /dev/mapper/mpatha /dev/mapper/mpathb
  Volume group "VG0" successfully created

# vgextend VG0 /dev/mapper/mpathc
  Volume group "VG0" successfully extended

# vgs
  VG  #PV #LV #SN Attr   VSize   VFree
  VG0   3   0   0 wz--n- 108.00m 108.00m

LVの作成

# lvcreate -l 100%FREE -n lv00 VG0
  Logical volume "lv00" created.

# lvs
  LV   VG  Attr       LSize   Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert
  lv00 VG0 -wi-ao---- 108.00m

ファイルシステムの作成

ファイルシステム作成

# mkfs.xfs /dev/VG0/lv00
meta-data=/dev/VG0/lv00          isize=512    agcount=4, agsize=6912 blks
         =                       sectsz=512   attr=2, projid32bit=1
         =                       crc=1        finobt=0, sparse=0
data     =                       bsize=4096   blocks=27648, imaxpct=25
         =                       sunit=0      swidth=0 blks
naming   =version 2              bsize=4096   ascii-ci=0 ftype=1
log      =internal log           bsize=4096   blocks=855, version=2
         =                       sectsz=512   sunit=0 blks, lazy-count=1
realtime =none                   extsz=4096   blocks=0, rtextents=0

UUIDを確認、fstabに追記してmount

# blkid /dev/VG0/lv00
/dev/VG0/lv00: UUID="9351d7da-a701-4011-9553-15feb27ecc5c" TYPE="xfs"

# echo "UUID=9351d7da-a701-4011-9553-15feb27ecc5c /work xfs defaults 0 0" >> /etc/fstab
# mkdir /work
# mount /work
# df -h | grep /work
/dev/mapper/VG0-lv00  105M  5.7M  100M   6% /work

おしまい。

5
5
0

Register as a new user and use Qiita more conveniently

  1. You get articles that match your needs
  2. You can efficiently read back useful information
  3. You can use dark theme
What you can do with signing up
Sign upLogin
5
5

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?