1. はじめに
IBM Cloud Classic Infrastructureでは、10 IOPS/GB x 4000 GB = 40000 IOPSのBlock Storage(iSCSI)が利用可能である。
本稿では、一世代前のBMSサーバー(Intel Xeon-Skylake (4110-SILVER))およびOS(Windows Server 2016)であっても、NICが10Gbpsさえあれば、問題なくこの性能が出ることを確認したい。
なお、このIOPS性能は、ここにも以下のように説明されているように、16KBブロックで、read/writeが50%のランダムアクセスであることを前提としている。また、あくまでストレージそのものが提供するIOPS性能であり、例えばストレージにアクセスするサーバーのOS設定やNICスピードがボトルネックになる可能性がある。
IOPS is measured based on a load profile of 16-KB blocks with random 50 percent read and 50 percent writes. Workloads that differ from this profile can experience inferior performance.
2. HW/Storage構成
-
サーバー環境(public側のNICはセキュリティーの都合上、停止）
-
ストレージ環境(40000 IOPS Block Storage)
-
OS設定
systeminfo
C:\Users\Administrator>systeminfo
Host Name: BMSBLK01
OS Name: Microsoft Windows Server 2016 Standard
OS Version: 10.0.14393 N/A Build 14393
OS Manufacturer: Microsoft Corporation
OS Configuration: Standalone Server
OS Build Type: Multiprocessor Free
Registered Owner: NetworkLayer
Registered Organization: NetworkLayer
Product ID: 00377-60000-00000-AA934
Original Install Date: 3/18/2024, 12:44:32 PM
System Boot Time: 3/19/2024, 12:31:05 PM
System Manufacturer: Supermicro
System Model: SYS-6029U-E1CR4T
System Type: x64-based PC
Processor(s): 2 Processor(s) Installed.
[01]: Intel64 Family 6 Model 85 Stepping 4 GenuineIntel ~700 Mhz
[02]: Intel64 Family 6 Model 85 Stepping 4 GenuineIntel ~700 Mhz
BIOS Version: American Megatrends Inc. 4.0, 8/11/2023
Windows Directory: C:\windows
System Directory: C:\windows\system32
Boot Device: \Device\HarddiskVolume1
System Locale: en-us;English (United States)
Input Locale: en-us;English (United States)
Time Zone: (UTC+09:00) Osaka, Sapporo, Tokyo
Total Physical Memory: 65,187 MB
Available Physical Memory: 63,003 MB
Virtual Memory: Max Size: 74,915 MB
Virtual Memory: Available: 72,808 MB
Virtual Memory: In Use: 2,107 MB
Page File Location(s): C:\pagefile.sys
Domain: WORKGROUP
Logon Server: \\BMSBLK01
Hotfix(s): 13 Hotfix(s) Installed.
[01]: KB4514354
[02]: KB3192137
[03]: KB4013418
[04]: KB4091664
[05]: KB4132216
[06]: KB4346087
[07]: KB4494175
[08]: KB4516115
[09]: KB4520724
[10]: KB4521858
[11]: KB4589210
[12]: KB5035962
[13]: KB4343887
Network Card(s): 6 NIC(s) Installed.
[01]: Intel(R) Ethernet Controller X710/X557-AT 10GBASE-T
Connection Name: PublicNetwork-B
Status: Media disconnected
[02]: Intel(R) Ethernet Controller X710/X557-AT 10GBASE-T
Connection Name: PrivateNetwork-A
DHCP Enabled: No
IP address(es)
[03]: Intel(R) Ethernet Controller X710/X557-AT 10GBASE-T
Connection Name: PrivateNetwork-B
DHCP Enabled: Yes
DHCP Server: N/A
IP address(es)
[04]: Intel(R) Ethernet Controller X710/X557-AT 10GBASE-T
Connection Name: PublicNetwork-A
Status: Media disconnected
[05]: Microsoft Network Adapter Multiplexor Driver
Connection Name: PublicNetwork-Teamed
Status: Media disconnected
[06]: Microsoft Network Adapter Multiplexor Driver
Connection Name: PrivateNetwork-Teamed
DHCP Enabled: No
IP address(es)
[01]: 10.192.27.6
[02]: fe80::7852:47a:cd6f:6a08
Hyper-V Requirements: VM Monitor Mode Extensions: Yes
Virtualization Enabled In Firmware: Yes
Second Level Address Translation: Yes
Data Execution Prevention Available: Yes
NIC情報
PS C:\Users\Administrator> Get-NetAdapter
Name InterfaceDescription ifIndex Status MacAddress LinkSpeed
---- -------------------- ------- ------ ---------- ---------
PublicNetwork-Teamed Microsoft Network Adapter Multiplexo... 13 Disconnected 3C-EC-EF-1C-4B-4F 0 bps
PublicNetwork-B Intel(R) Ethernet Controller X710/X5... 7 Disconnected 3C-EC-EF-1C-4B-51 10 Gbps
PrivateNetwork-B Intel(R) Ethernet Controller X710/...#3 12 Up 3C-EC-EF-1C-4B-50 10 Gbps
PublicNetwork-A Intel(R) Ethernet Controller X710/...#4 4 Disconnected 3C-EC-EF-1C-4B-4F 10 Gbps
PrivateNetwork-Teamed Microsoft Network Adapter Multiple...#2 6 Up 3C-EC-EF-1C-4B-4E 20 Gbps
PrivateNetwork-A Intel(R) Ethernet Controller X710/...#2 9 Up 3C-EC-EF-1C-4B-4E 10 Gbps
-
Windows Firewall
- デフォルトのまま、停止した状態である。（有効化しても大きな差は見られなかったが）
- デフォルトのまま、停止した状態である。（有効化しても大きな差は見られなかったが）
-
iSCSI構成
- IBM Cloud docsによる構成方法に従ってBlock StorageをMPIO冗長化パスでマウント。特別な追加構成はしていない。
-
JumboFrameを有効にするなどの設定はしていない。
3. 計測ツール
- fioを利用。
- 本計測のタイミングで最新のfio-3.36-x64.msiを利用。
4. 性能計測
4-1. ブロックサイズ＝4K
D:\>fio.exe --randrepeat=1 --ioengine=windowsaio --direct=1 --sync=1 --name=fiotest --filename=fio-test-file.fio --overwrite=1 --iodepth=64 --size=5GB --readwrite=randrw --rwmixwrite=50 --rwmixread=50 --bs=4k --time_based --thread --runtime=120 --ramp_time=15
fiotest: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=windowsaio, iodepth=64
fio-3.36
Starting 1 thread
Jobs: 1 (f=1): [m(1)][100.0%][r=78.9MiB/s,w=80.6MiB/s][r=20.2k,w=20.6k IOPS][eta 00m:00s]
fiotest: (groupid=0, jobs=1): err= 0: pid=4608: Tue Mar 19 14:21:55 2024
read: IOPS=20.5k, BW=80.1MiB/s (84.0MB/s)(9608MiB/120002msec)
slat (usec): min=4, max=492, avg=14.37, stdev=11.85
clat (usec): min=31, max=46403, avg=1282.48, stdev=1150.95
lat (usec): min=272, max=46418, avg=1296.85, stdev=1150.35
clat percentiles (usec):
| 1.00th=[ 562], 5.00th=[ 701], 10.00th=[ 783], 20.00th=[ 889],
| 30.00th=[ 971], 40.00th=[ 1045], 50.00th=[ 1106], 60.00th=[ 1188],
| 70.00th=[ 1270], 80.00th=[ 1369], 90.00th=[ 1549], 95.00th=[ 1778],
| 99.00th=[ 9372], 99.50th=[10552], 99.90th=[11207], 99.95th=[11469],
| 99.99th=[11994]
bw ( KiB/s): min=68673, max=95090, per=100.00%, avg=82075.05, stdev=1655.15, samples=238
iops : min=17168, max=23772, avg=20518.49, stdev=413.79, samples=238
write: IOPS=20.5k, BW=80.0MiB/s (83.9MB/s)(9601MiB/120002msec); 0 zone resets
slat (usec): min=5, max=572, avg=21.27, stdev=13.40
clat (usec): min=59, max=13843, avg=1367.62, stdev=1173.36
lat (usec): min=331, max=13858, avg=1388.90, stdev=1172.33
clat percentiles (usec):
| 1.00th=[ 586], 5.00th=[ 734], 10.00th=[ 832], 20.00th=[ 963],
| 30.00th=[ 1057], 40.00th=[ 1139], 50.00th=[ 1205], 60.00th=[ 1270],
| 70.00th=[ 1352], 80.00th=[ 1450], 90.00th=[ 1631], 95.00th=[ 1844],
| 99.00th=[ 9634], 99.50th=[10945], 99.90th=[11731], 99.95th=[11994],
| 99.99th=[12387]
bw ( KiB/s): min=68593, max=95098, per=100.00%, avg=82014.73, stdev=1673.36, samples=238
iops : min=17148, max=23774, avg=20503.42, stdev=418.34, samples=238
lat (usec) : 50=0.01%, 100=0.01%, 250=0.01%, 500=0.27%, 750=6.56%
lat (usec) : 1000=22.29%
lat (msec) : 2=67.61%, 4=1.29%, 10=1.11%, 20=0.88%, 50=0.01%
cpu : usr=7.50%, sys=62.50%, ctx=0, majf=0, minf=0
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.2%, 16=5.8%, 32=81.4%, >=64=12.6%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=94.8%, 8=2.3%, 16=1.5%, 32=0.8%, 64=0.6%, >=64=0.0%
issued rwts: total=2459575,2457919,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=64
Run status group 0 (all jobs):
READ: bw=80.1MiB/s (84.0MB/s), 80.1MiB/s-80.1MiB/s (84.0MB/s-84.0MB/s), io=9608MiB (10.1GB), run=120002-120002msec
WRITE: bw=80.0MiB/s (83.9MB/s), 80.0MiB/s-80.0MiB/s (83.9MB/s-83.9MB/s), io=9601MiB (10.1GB), run=120002-120002msec
4-2. ブロックサイズ＝16K
D:\>fio.exe --randrepeat=1 --ioengine=windowsaio --direct=1 --sync=1 --name=fiotest --filename=fio-test-file.fio --overwrite=1 --iodepth=64 --size=5GB --readwrite=randrw --rwmixwrite=50 --rwmixread=50 --bs=16k --time_based --thread --runtime=120 --ramp_time=15
fiotest: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=windowsaio, iodepth=64
fio-3.36
Starting 1 thread
Jobs: 1 (f=1): [m(1)][100.0%][r=314MiB/s,w=320MiB/s][r=20.1k,w=20.5k IOPS][eta 00m:00s]
fiotest: (groupid=0, jobs=1): err= 0: pid=1212: Tue Mar 19 12:37:34 2024
read: IOPS=20.5k, BW=320MiB/s (336MB/s)(37.5GiB/120001msec)
slat (usec): min=4, max=389, avg=14.37, stdev=10.80
clat (usec): min=137, max=14469, avg=1263.95, stdev=948.27
lat (usec): min=278, max=14484, avg=1278.32, stdev=947.96
clat percentiles (usec):
| 1.00th=[ 586], 5.00th=[ 742], 10.00th=[ 824], 20.00th=[ 930],
| 30.00th=[ 1004], 40.00th=[ 1074], 50.00th=[ 1139], 60.00th=[ 1205],
| 70.00th=[ 1270], 80.00th=[ 1385], 90.00th=[ 1565], 95.00th=[ 1795],
| 99.00th=[ 6325], 99.50th=[10421], 99.90th=[11207], 99.95th=[11338],
| 99.99th=[11731]
bw ( KiB/s): min=313809, max=353045, per=100.00%, avg=328459.27, stdev=5064.91, samples=238
iops : min=19613, max=22065, avg=20528.25, stdev=316.52, samples=238
write: IOPS=20.5k, BW=320MiB/s (336MB/s)(37.5GiB/120001msec); 0 zone resets
slat (usec): min=6, max=421, avg=22.98, stdev=13.28
clat (usec): min=108, max=14596, avg=1419.20, stdev=1018.25
lat (usec): min=371, max=14614, avg=1442.19, stdev=1017.53
clat percentiles (usec):
| 1.00th=[ 635], 5.00th=[ 816], 10.00th=[ 922], 20.00th=[ 1045],
| 30.00th=[ 1139], 40.00th=[ 1205], 50.00th=[ 1287], 60.00th=[ 1352],
| 70.00th=[ 1434], 80.00th=[ 1549], 90.00th=[ 1762], 95.00th=[ 2040],
| 99.00th=[ 6915], 99.50th=[10945], 99.90th=[11994], 99.95th=[12256],
| 99.99th=[12649]
bw ( KiB/s): min=312396, max=346170, per=100.00%, avg=328204.25, stdev=4995.11, samples=238
iops : min=19524, max=21635, avg=20512.34, stdev=312.14, samples=238
lat (usec) : 250=0.01%, 500=0.20%, 750=4.08%, 1000=18.31%
lat (msec) : 2=73.06%, 4=2.99%, 10=0.72%, 20=0.64%
cpu : usr=10.00%, sys=61.67%, ctx=0, majf=0, minf=0
IO depths : 1=0.0%, 2=0.1%, 4=0.1%, 8=0.2%, 16=4.6%, 32=79.4%, >=64=15.8%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=94.8%, 8=2.5%, 16=1.5%, 32=0.7%, 64=0.5%, >=64=0.0%
issued rwts: total=2460316,2458563,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=64
Run status group 0 (all jobs):
READ: bw=320MiB/s (336MB/s), 320MiB/s-320MiB/s (336MB/s-336MB/s), io=37.5GiB (40.3GB), run=120001-120001msec
WRITE: bw=320MiB/s (336MB/s), 320MiB/s-320MiB/s (336MB/s-336MB/s), io=37.5GiB (40.3GB), run=120001-120001msec
4-3. ブロックサイズ＝32K
D:\>fio.exe --randrepeat=1 --ioengine=windowsaio --direct=1 --sync=1 --name=fiotest --filename=fio-test-file.fio --overwrite=1 --iodepth=64 --size=5GB --readwrite=randrw --rwmixwrite=50 --rwmixread=50 --bs=32k --time_based --thread --runtime=120 --ramp_time=15
fiotest: (g=0): rw=randrw, bs=(R) 32.0KiB-32.0KiB, (W) 32.0KiB-32.0KiB, (T) 32.0KiB-32.0KiB, ioengine=windowsaio, iodepth=64
fio-3.36
Starting 1 thread
Jobs: 1 (f=1): [m(1)][100.0%][r=658MiB/s,w=646MiB/s][r=21.0k,w=20.7k IOPS][eta 00m:00s]
fiotest: (groupid=0, jobs=1): err= 0: pid=6300: Wed Mar 20 09:19:57 2024
read: IOPS=20.3k, BW=635MiB/s (665MB/s)(74.4GiB/120002msec)
slat (usec): min=4, max=766, avg=11.72, stdev= 9.50
clat (usec): min=348, max=23848, avg=1425.23, stdev=709.11
lat (usec): min=358, max=23887, avg=1436.95, stdev=709.18
clat percentiles (usec):
| 1.00th=[ 709], 5.00th=[ 848], 10.00th=[ 930], 20.00th=[ 1037],
| 30.00th=[ 1106], 40.00th=[ 1188], 50.00th=[ 1254], 60.00th=[ 1352],
| 70.00th=[ 1483], 80.00th=[ 1663], 90.00th=[ 2024], 95.00th=[ 2474],
| 99.00th=[ 3884], 99.50th=[ 5145], 99.90th=[10552], 99.95th=[11076],
| 99.99th=[11600]
bw ( KiB/s): min=376432, max=741835, per=100.00%, avg=650553.12, stdev=41538.21, samples=235
iops : min=11763, max=23182, avg=20329.33, stdev=1298.08, samples=235
write: IOPS=20.3k, BW=634MiB/s (665MB/s)(74.3GiB/120002msec); 0 zone resets
slat (usec): min=5, max=1314, avg=18.50, stdev=11.47
clat (usec): min=344, max=24120, avg=1502.18, stdev=734.82
lat (usec): min=380, max=24138, avg=1520.68, stdev=734.69
clat percentiles (usec):
| 1.00th=[ 734], 5.00th=[ 881], 10.00th=[ 979], 20.00th=[ 1090],
| 30.00th=[ 1188], 40.00th=[ 1270], 50.00th=[ 1352], 60.00th=[ 1450],
| 70.00th=[ 1565], 80.00th=[ 1762], 90.00th=[ 2114], 95.00th=[ 2540],
| 99.00th=[ 3982], 99.50th=[ 5211], 99.90th=[11076], 99.95th=[11731],
| 99.99th=[12649]
bw ( KiB/s): min=374380, max=742541, per=100.00%, avg=649893.56, stdev=41961.83, samples=235
iops : min=11699, max=23204, avg=20308.71, stdev=1311.26, samples=235
lat (usec) : 500=0.01%, 750=1.52%, 1000=12.40%
lat (msec) : 2=74.72%, 4=10.40%, 10=0.81%, 20=0.14%, 50=0.01%
cpu : usr=9.17%, sys=50.00%, ctx=0, majf=0, minf=0
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.9%, 32=77.6%, >=64=21.3%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=94.1%, 8=3.5%, 16=1.7%, 32=0.6%, 64=0.1%, >=64=0.0%
issued rwts: total=2437066,2434644,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=64
Run status group 0 (all jobs):
READ: bw=635MiB/s (665MB/s), 635MiB/s-635MiB/s (665MB/s-665MB/s), io=74.4GiB (79.9GB), run=120002-120002msec
WRITE: bw=634MiB/s (665MB/s), 634MiB/s-634MiB/s (665MB/s-665MB/s), io=74.3GiB (79.8GB), run=120002-120002msec
4-4. ブロックサイズ＝64K
D:\>fio.exe --randrepeat=1 --ioengine=windowsaio --direct=1 --sync=1 --name=fiotest --filename=fio-test-file.fio --overwrite=1 --iodepth=64 --size=5GB --readwrite=randrw --rwmixwrite=50 --rwmixread=50 --bs=64k --time_based --thread --runtime=120 --ramp_time=15
fiotest: (g=0): rw=randrw, bs=(R) 64.0KiB-64.0KiB, (W) 64.0KiB-64.0KiB, (T) 64.0KiB-64.0KiB, ioengine=windowsaio, iodepth=64
fio-3.36
Starting 1 thread
Jobs: 1 (f=1): [m(1)][100.0%][r=549MiB/s,w=543MiB/s][r=8787,w=8682 IOPS][eta 00m:00s]
fiotest: (groupid=0, jobs=1): err= 0: pid=4468: Tue Mar 19 14:29:06 2024
read: IOPS=8677, BW=542MiB/s (569MB/s)(63.6GiB/120003msec)
slat (usec): min=7, max=527, avg=20.45, stdev=11.56
clat (usec): min=488, max=16740, avg=3601.53, stdev=1277.24
lat (usec): min=503, max=16756, avg=3621.98, stdev=1278.90
clat percentiles (usec):
| 1.00th=[ 1975], 5.00th=[ 2311], 10.00th=[ 2507], 20.00th=[ 2737],
| 30.00th=[ 2933], 40.00th=[ 3130], 50.00th=[ 3326], 60.00th=[ 3523],
| 70.00th=[ 3818], 80.00th=[ 4228], 90.00th=[ 4948], 95.00th=[ 5800],
| 99.00th=[ 8717], 99.50th=[10421], 99.90th=[13435], 99.95th=[14222],
| 99.99th=[15401]
bw ( KiB/s): min=431488, max=741707, per=100.00%, avg=555877.32, stdev=57805.44, samples=238
iops : min= 6742, max=11589, avg=8685.26, stdev=903.20, samples=238
write: IOPS=8671, BW=542MiB/s (568MB/s)(63.5GiB/120003msec); 0 zone resets
slat (usec): min=8, max=500, avg=25.30, stdev=13.35
clat (usec): min=551, max=17566, avg=3677.17, stdev=1308.41
lat (usec): min=575, max=17582, avg=3702.47, stdev=1310.26
clat percentiles (usec):
| 1.00th=[ 2024], 5.00th=[ 2376], 10.00th=[ 2540], 20.00th=[ 2802],
| 30.00th=[ 2999], 40.00th=[ 3163], 50.00th=[ 3392], 60.00th=[ 3621],
| 70.00th=[ 3884], 80.00th=[ 4293], 90.00th=[ 5080], 95.00th=[ 5932],
| 99.00th=[ 8979], 99.50th=[10683], 99.90th=[13566], 99.95th=[14353],
| 99.99th=[15533]
bw ( KiB/s): min=419014, max=754276, per=100.00%, avg=555510.02, stdev=58003.07, samples=238
iops : min= 6547, max=11785, avg=8679.45, stdev=906.30, samples=238
lat (usec) : 500=0.01%, 750=0.01%, 1000=0.01%
lat (msec) : 2=1.01%, 4=73.00%, 10=25.34%, 20=0.64%
cpu : usr=7.50%, sys=34.17%, ctx=0, majf=0, minf=0
IO depths : 1=0.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.1%, 32=50.1%, >=64=49.9%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=97.9%, 8=1.9%, 16=0.2%, 32=0.1%, 64=0.1%, >=64=0.0%
issued rwts: total=1041329,1040585,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=64
Run status group 0 (all jobs):
READ: bw=542MiB/s (569MB/s), 542MiB/s-542MiB/s (569MB/s-569MB/s), io=63.6GiB (68.2GB), run=120003-120003msec
WRITE: bw=542MiB/s (568MB/s), 542MiB/s-542MiB/s (568MB/s-568MB/s), io=63.5GiB (68.2GB), run=120003-120003msec
4-5. ブロックサイズ＝128K
D:\>fio.exe --randrepeat=1 --ioengine=windowsaio --direct=1 --sync=1 --name=fiotest --filename=fio-test-file.fio --overwrite=1 --iodepth=64 --size=5GB --readwrite=randrw --rwmixwrite=50 --rwmixread=50 --bs=128k --time_based --thread --runtime=120 --ramp_time=15
fiotest: (g=0): rw=randrw, bs=(R) 128KiB-128KiB, (W) 128KiB-128KiB, (T) 128KiB-128KiB, ioengine=windowsaio, iodepth=64
fio-3.36
Starting 1 thread
Jobs: 1 (f=0): [f(1)][100.0%][r=722MiB/s,w=734MiB/s][r=5774,w=5875 IOPS][eta 00m:00s]
fiotest: (groupid=0, jobs=1): err= 0: pid=6296: Tue Mar 19 14:32:26 2024
read: IOPS=5850, BW=731MiB/s (767MB/s)(85.7GiB/120007msec)
slat (usec): min=7, max=376, avg=30.32, stdev=16.32
clat (usec): min=612, max=286636, avg=3865.61, stdev=5240.34
lat (usec): min=657, max=286656, avg=3895.93, stdev=5240.64
clat percentiles (msec):
| 1.00th=[ 3], 5.00th=[ 3], 10.00th=[ 3], 20.00th=[ 3],
| 30.00th=[ 4], 40.00th=[ 4], 50.00th=[ 4], 60.00th=[ 4],
| 70.00th=[ 5], 80.00th=[ 5], 90.00th=[ 5], 95.00th=[ 6],
| 99.00th=[ 9], 99.50th=[ 10], 99.90th=[ 17], 99.95th=[ 37],
| 99.99th=[ 239]
bw ( KiB/s): min=31367, max=995014, per=100.00%, avg=753353.26, stdev=153117.92, samples=237
iops : min= 245, max= 7773, avg=5885.07, stdev=1196.23, samples=237
write: IOPS=5842, BW=730MiB/s (766MB/s)(85.6GiB/120007msec); 0 zone resets
slat (usec): min=9, max=752, avg=48.69, stdev=21.44
clat (usec): min=1346, max=1150.9k, avg=6910.57, stdev=13019.43
lat (usec): min=1392, max=1151.0k, avg=6959.26, stdev=13019.81
clat percentiles (msec):
| 1.00th=[ 4], 5.00th=[ 5], 10.00th=[ 5], 20.00th=[ 6],
| 30.00th=[ 6], 40.00th=[ 6], 50.00th=[ 7], 60.00th=[ 7],
| 70.00th=[ 7], 80.00th=[ 8], 90.00th=[ 9], 95.00th=[ 10],
| 99.00th=[ 15], 99.50th=[ 18], 99.90th=[ 41], 99.95th=[ 241],
| 99.99th=[ 300]
bw ( KiB/s): min=13154, max=991422, per=100.00%, avg=752364.50, stdev=153225.64, samples=237
iops : min= 102, max= 7745, avg=5877.38, stdev=1197.10, samples=237
lat (usec) : 750=0.01%, 1000=0.01%
lat (msec) : 2=0.38%, 4=35.36%, 10=61.65%, 20=2.40%, 50=0.14%
lat (msec) : 100=0.01%, 250=0.05%, 500=0.01%, 2000=0.01%
cpu : usr=11.67%, sys=33.33%, ctx=0, majf=0, minf=0
IO depths : 1=0.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.1%, 32=46.3%, >=64=53.7%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=97.9%, 8=1.7%, 16=0.3%, 32=0.1%, 64=0.1%, >=64=0.0%
issued rwts: total=702102,701160,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=64
Run status group 0 (all jobs):
READ: bw=731MiB/s (767MB/s), 731MiB/s-731MiB/s (767MB/s-767MB/s), io=85.7GiB (92.0GB), run=120007-120007msec
WRITE: bw=730MiB/s (766MB/s), 730MiB/s-730MiB/s (766MB/s-766MB/s), io=85.6GiB (91.9GB), run=120007-120007msec
4-6. ブロックサイズ＝256K
D:\>fio.exe --randrepeat=1 --ioengine=windowsaio --direct=1 --sync=1 --name=fiotest --filename=fio-test-file.fio --overwrite=1 --iodepth=64 --size=5GB --readwrite=randrw --rwmixwrite=50 --rwmixread=50 --bs=256k --time_based --thread --runtime=120 --ramp_time=15
fiotest: (g=0): rw=randrw, bs=(R) 256KiB-256KiB, (W) 256KiB-256KiB, (T) 256KiB-256KiB, ioengine=windowsaio, iodepth=64
fio-3.36
Starting 1 thread
Jobs: 1 (f=1): [m(1)][100.0%][r=588MiB/s,w=604MiB/s][r=2352,w=2417 IOPS][eta 00m:00s]
fiotest: (groupid=0, jobs=1): err= 0: pid=6844: Tue Mar 19 13:00:27 2024
read: IOPS=2764, BW=691MiB/s (725MB/s)(81.0GiB/120012msec)
slat (usec): min=9, max=7987, avg=55.79, stdev=179.18
clat (usec): min=3, max=249740, avg=8276.41, stdev=5380.75
lat (usec): min=847, max=249752, avg=8332.20, stdev=5379.98
clat percentiles (msec):
| 1.00th=[ 4], 5.00th=[ 5], 10.00th=[ 6], 20.00th=[ 7],
| 30.00th=[ 7], 40.00th=[ 8], 50.00th=[ 8], 60.00th=[ 9],
| 70.00th=[ 9], 80.00th=[ 10], 90.00th=[ 12], 95.00th=[ 14],
| 99.00th=[ 19], 99.50th=[ 22], 99.90th=[ 33], 99.95th=[ 47],
| 99.99th=[ 245]
bw ( KiB/s): min=300544, max=964535, per=100.00%, avg=708659.40, stdev=118761.43, samples=234
iops : min= 1174, max= 3767, avg=2767.88, stdev=463.87, samples=234
write: IOPS=2760, BW=690MiB/s (724MB/s)(80.9GiB/120012msec); 0 zone resets
slat (usec): min=13, max=11597, avg=91.76, stdev=214.39
clat (usec): min=1113, max=268285, avg=13994.41, stdev=7325.54
lat (usec): min=1533, max=268401, avg=14086.16, stdev=7322.05
clat percentiles (msec):
| 1.00th=[ 6], 5.00th=[ 9], 10.00th=[ 10], 20.00th=[ 11],
| 30.00th=[ 12], 40.00th=[ 13], 50.00th=[ 14], 60.00th=[ 14],
| 70.00th=[ 15], 80.00th=[ 17], 90.00th=[ 19], 95.00th=[ 22],
| 99.00th=[ 32], 99.50th=[ 38], 99.90th=[ 52], 99.95th=[ 245],
| 99.99th=[ 257]
bw ( KiB/s): min=265234, max=1014929, per=100.00%, avg=707990.54, stdev=123579.24, samples=234
iops : min= 1036, max= 3964, avg=2765.26, stdev=482.73, samples=234
lat (usec) : 4=0.01%, 10=0.01%, 250=0.01%, 750=0.01%, 1000=0.01%
lat (msec) : 2=0.04%, 4=1.01%, 10=46.54%, 20=48.67%, 50=3.67%
lat (msec) : 100=0.03%, 250=0.03%, 500=0.01%
cpu : usr=5.83%, sys=10.83%, ctx=0, majf=0, minf=0
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.2%, 16=1.5%, 32=57.9%, >=64=40.4%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=96.8%, 8=1.9%, 16=0.9%, 32=0.3%, 64=0.1%, >=64=0.0%
issued rwts: total=331763,331313,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=64
Run status group 0 (all jobs):
READ: bw=691MiB/s (725MB/s), 691MiB/s-691MiB/s (725MB/s-725MB/s), io=81.0GiB (87.0GB), run=120012-120012msec
WRITE: bw=690MiB/s (724MB/s), 690MiB/s-690MiB/s (724MB/s-724MB/s), io=80.9GiB (86.9GB), run=120012-120012msec
5. まとめ
以上の結果より、4K~32Kブロックにおいて、特にチューニングなしで40000 IOPS性能(read 20000 IOPS, write 20000 IOPS)が出ていることが確認できた。