クラウドでのネットワーク レイテンシの測定より
クラウド アーキテクトによく寄せられる質問の一つに、「2 つのエンドポイント間でリクエストとレスポンスをどの程度まで迅速に交換できるのですか？」というのがあります。ネットワークのラウンドトリップ レイテンシを測定するツールには ping、iperf、netperf などがありますが、すべてが同じように実装、構成されるわけではないため、ツールによって結果が異なる場合があります。ほとんどの場合、この質問に対する代表的な回答が得られるツールは、netperf であると考えられます。
netperfコマンドは、単一リージョン内の単一のゾーンにある 2 台のマシン間でゾーン内レイテンシ ベンチマークを実行します。このようなゾーン内ベンチマークを使用することで、緊密に連携するマシン間の非常に低いレイテンシをマイクロ秒単位で示すことができます。
ということで、詳しい内容は この情報を読むということで、netperf を macOS と Linuxへインストールして実行してみてみます。
■ Macbook へのインストール
● Homebrewホームページ
・Homebrew netperf URL: https://formulae.brew.sh/formula/netperf
● netperfインストール
shirok@macbook ~ % brew install netperf
==> Auto-updating Homebrew...
Adjust how often this is run with HOMEBREW_AUTO_UPDATE_SECS or disable with
HOMEBREW_NO_AUTO_UPDATE. Hide these hints with HOMEBREW_NO_ENV_HINTS (see `man brew`).
==> Downloading https://ghcr.io/v2/homebrew/portable-ruby/portable-ruby/blobs/sha256:bbb73a9d86fa37128c54c74b020096a646c46c525fd5eb0c4a2467551fb2d377
########################################################################################################################################## 100.0%
==> Pouring portable-ruby-3.3.2.arm64_big_sur.bottle.tar.gz
==> homebrew/core is old and unneeded, untapping to save space...
Untapping homebrew/core...
Untapped 3 commands and 7096 formulae (7,477 files, 822.8MB).
==> homebrew/cask is old and unneeded, untapping to save space...
Untapping homebrew/cask...
Untapped 6927 casks (4,510 files, 414.7MB).
==> Auto-updated Homebrew!
Updated 2 taps (homebrew/core and homebrew/cask).
==> New Formulae
ansible-creator fern-api kuzu oj stripe-cli
apache-flink-cdc ffmpeg@6 lexido openfa stripe-mock
autotrace flawz liblc3 parsedmarc superfile
・・・
font-hedvig-letters-sans font-open-iconic zulu@17
font-hedvig-letters-serif font-open-sans zulu@21
font-heebo font-open-sans-hebrew zulu@8
You have 8 outdated formulae installed.
==> Downloading https://ghcr.io/v2/homebrew/core/netperf/manifests/2.7.0-1
########################################################################################################################################## 100.0%
==> Fetching netperf
==> Downloading https://ghcr.io/v2/homebrew/core/netperf/blobs/sha256:dbb69e91e2bbb449c4b6ce47a4a3e624461262ed275327236afd7b83af588778
########################################################################################################################################## 100.0%
==> Pouring netperf--2.7.0.arm64_sonoma.bottle.1.tar.gz
🍺 /opt/homebrew/Cellar/netperf/2.7.0: 12 files, 608.3KB
==> Running `brew cleanup netperf`...
Disable this behaviour by setting HOMEBREW_NO_INSTALL_CLEANUP.
Hide these hints with HOMEBREW_NO_ENV_HINTS (see `man brew`).
==> `brew cleanup` has not been run in the last 30 days, running now...
Disable this behaviour by setting HOMEBREW_NO_INSTALL_CLEANUP.
Hide these hints with HOMEBREW_NO_ENV_HINTS (see `man brew`).
● 確認
shirok@macbook~ % netperf -h
Usage: netperf [global options] -- [test options]
Global options:
-a send,recv Set the local send,recv buffer alignment
-A send,recv Set the remote send,recv buffer alignment
-B brandstr Specify a string to be emitted with brief output
-c [cpu_rate] Report local CPU usage
-C [cpu_rate] Report remote CPU usage
-d Increase debugging output
-D time,[units] * Display interim results at least every time interval
using units as the initial guess for units per second
A negative value for time will make heavy use of the
system's timestamping functionality
-f G|M|K|g|m|k Set the output units
-F lfill[,rfill]* Pre-fill buffers with data from specified file
-h Display this text
-H name|ip,fam * Specify the target machine and/or local ip and family
-i max,min Specify the max and min number of iterations (15,1)
-I lvl[,intvl] Specify confidence level (95 or 99) (99)
and confidence interval in percentage (10)
-j Keep additional timing statistics
-l testlen Specify test duration (>0 secs) (<0 bytes|trans)
-L name|ip,fam * Specify the local ip|name and address family
-o send,recv Set the local send,recv buffer offsets
-O send,recv Set the remote send,recv buffer offset
-n numcpu Set the number of processors for CPU util
-N Establish no control connection, do 'send' side only
-p port,lport* Specify netserver port number and/or local port
-P 0|1 Don't/Do display test headers
-r Allow confidence to be hit on result only
-s seconds Wait seconds between test setup and test start
-S Set SO_KEEPALIVE on the data connection
-t testname Specify test to perform
-T lcpu,rcpu Request netperf/netserver be bound to local/remote cpu
-v verbosity Specify the verbosity level
-W send,recv Set the number of send,recv buffers
-v level Set the verbosity level (default 1, min 0)
-V Display the netperf version and exit
-y local,remote Set the socket priority
-Y local,remote Set the IP_TOS. Use hexadecimal.
-Z passphrase Set and pass to netserver a passphrase
For those options taking two parms, at least one must be specified;
specifying one value without a comma will set both parms to that
value, specifying a value with a leading comma will set just the second
parm, a value with a trailing comma will set just the first. To set
each parm to unique values, specify both and separate them with a
comma.
* For these options taking two parms, specifying one value with no comma
will only set the first parms and will leave the second at the default
value. To set the second value it must be preceded with a comma or be a
comma-separated pair. This is to retain previous netperf behaviour.
■ Oracle Linux へのインストール
yumリポジトリにない場合、Webサイトからダウンロードしてインストールします。
・参考: The Netperf Homepage
Oracle Linuxでは エンタープライズ Linux 用の拡張パッケージ Extra Packages for Enterprise Linux(EPEL) の Webサイトに netperfが RPMで提供されています。
● Install実行
[root@tokyo-inst01 ~]# yum install https://yum.oracle.com/repo/OracleLinux/OL8/developer/EPEL/x86_64/getPackage/netperf-2.7.0-1.20210803git3bc455b.el8.x86_64.rpm
This system is receiving updates from OSMS server.
Last metadata expiration check: 2:24:44 ago on Wed 17 Jan 2024 11:52:39 AM GMT.
netperf-2.7.0-1.20210803git3bc455b.el8.x86_64.rpm 2.9 MB/s | 677 kB 00:00
Dependencies resolved.
===================================================================================================================================================
Package Architecture Version Repository Size
===================================================================================================================================================
Installing:
netperf x86_64 2.7.0-1.20210803git3bc455b.el8 @commandline 677 k
Transaction Summary
===================================================================================================================================================
Install 1 Package
Total size: 677 k
Installed size: 1.5 M
Is this ok [y/N]: y
Downloading Packages:
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
Preparing : 1/1
Installing : netperf-2.7.0-1.20210803git3bc455b.el8.x86_64 1/1
Running scriptlet: netperf-2.7.0-1.20210803git3bc455b.el8.x86_64 1/1
Verifying : netperf-2.7.0-1.20210803git3bc455b.el8.x86_64 1/1
Installed:
netperf-2.7.0-1.20210803git3bc455b.el8.x86_64
Complete!
● インストール確認
1) Version 情報確認
[root@tokyo-inst01 ~]# netperf -V
Netperf version 2.7.1
2) Help 情報確認
[root@tokyo-inst01 ~]# netperf -h
Usage: netperf [global options] -- [test options]
Global options:
-a send,recv Set the local send,recv buffer alignment
-A send,recv Set the remote send,recv buffer alignment
-B brandstr Specify a string to be emitted with brief output
-c [cpu_rate] Report local CPU usage
-C [cpu_rate] Report remote CPU usage
-d Increase debugging output
-D time,[units] * Display interim results at least every time interval
using units as the initial guess for units per second
A negative value for time will make heavy use of the
system's timestamping functionality
-f G|M|K|g|m|k Set the output units
-F lfill[,rfill]* Pre-fill buffers with data from specified file
-h Display this text
-H name|ip,fam * Specify the target machine and/or local ip and family
-i max,min Specify the max and min number of iterations (15,1)
-I lvl[,intvl] Specify confidence level (95 or 99) (99)
and confidence interval in percentage (10)
-j Keep additional timing statistics
-l testlen Specify test duration (>0 secs) (<0 bytes|trans)
-L name|ip,fam * Specify the local ip|name and address family
-o send,recv Set the local send,recv buffer offsets
-O send,recv Set the remote send,recv buffer offset
-n numcpu Set the number of processors for CPU util
-N Establish no control connection, do 'send' side only
-p port,lport* Specify netserver port number and/or local port
-P 0|1 Don't/Do display test headers
-r Allow confidence to be hit on result only
-s seconds Wait seconds between test setup and test start
-S Set SO_KEEPALIVE on the data connection
-t testname Specify test to perform
-T lcpu,rcpu Request netperf/netserver be bound to local/remote cpu
-v verbosity Specify the verbosity level
-W send,recv Set the number of send,recv buffers
-v level Set the verbosity level (default 1, min 0)
-V Display the netperf version and exit
-y local,remote Set the socket priority
-Y local,remote Set the IP_TOS. Use hexadecimal.
-Z passphrase Set and pass to netserver a passphrase
For those options taking two parms, at least one must be specified;
specifying one value without a comma will set both parms to that
value, specifying a value with a leading comma will set just the second
parm, a value with a trailing comma will set just the first. To set
each parm to unique values, specify both and separate them with a
comma.
* For these options taking two parms, specifying one value with no comma
will only set the first parms and will leave the second at the default
value. To set the second value it must be preceded with a comma or be a
comma-separated pair. This is to retain previous netperf behaviour.
■ netpwerf での TCP, UDP レイテンシ測定
TCPとUDPプロトコルでの測定を実施してみます。
● Server(受信側)設定
[root@osaka-inst01 ~]# netserver
Starting netserver with host 'IN(6)ADDR_ANY' port '12865' and family AF_UNSPEC
● Client(送信側)実行
・ TCP Laitency
単位は 1μs (0.001ms)です。
[root@tokyo-inst01 ~]# netperf -H 10.20.0.2 -t TCP_RR -- -k MIN_LATENCY,MEAN_LATENCY,MAX_LATENCY
MIGRATED TCP REQUEST/RESPONSE TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 10.20.0.2 () port 0 AF_INET : histogram : interval : dirty data : demo : first burst 0
MIN_LATENCY=7985
MEAN_LATENCY=8071.47
MAX_LATENCY=16252
・ UDP Laitency
単位は 1μs (0.001ms)です。
[root@tokyo-inst01 ~]# netperf -H 10.20.0.2 -t UDP_RR -- -k MIN_LATENCY,MEAN_LATENCY,MAX_LATENCY
MIGRATED UDP REQUEST/RESPONSE TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 10.20.0.2 () port 0 AF_INET : histogram : interval : dirty data : demo : first burst 0
MIN_LATENCY=8353
MEAN_LATENCY=8415.86
MAX_LATENCY=10508
■ ping での ICMP レイテンシ測定
-i オプションで ping実行間隔を 1μs (0.001ms) の単位に制御できます。
200ms以下は、root権限で実行する必要があります。
● root権限ない場合(失敗)
[opc@tokyo-inst01 ~]$ ping 10.20.0.2 -i 0.001 -c 10
PING 10.20.0.2 (10.20.0.2) 56(84) bytes of data.
ping: cannot flood; minimal interval allowed for user is 200ms
● root権限で実行(成功)
[opc@tokyo-inst01 ~]$ sudo ping 10.20.0.2 -i 0.001 -c 10
PING 10.20.0.2 (10.20.0.2) 56(84) bytes of data.
64 bytes from 10.20.0.2: icmp_seq=1 ttl=62 time=8.52 ms
64 bytes from 10.20.0.2: icmp_seq=2 ttl=62 time=8.40 ms
64 bytes from 10.20.0.2: icmp_seq=3 ttl=62 time=8.40 ms
64 bytes from 10.20.0.2: icmp_seq=4 ttl=62 time=8.35 ms
64 bytes from 10.20.0.2: icmp_seq=5 ttl=62 time=8.40 ms
64 bytes from 10.20.0.2: icmp_seq=6 ttl=62 time=8.34 ms
64 bytes from 10.20.0.2: icmp_seq=7 ttl=62 time=8.39 ms
64 bytes from 10.20.0.2: icmp_seq=8 ttl=62 time=8.35 ms
64 bytes from 10.20.0.2: icmp_seq=9 ttl=62 time=8.40 ms
64 bytes from 10.20.0.2: icmp_seq=10 ttl=62 time=8.35 ms
--- 10.20.0.2 ping statistics ---
10 packets transmitted, 10 received, 0% packet loss, time 76ms
rtt min/avg/max/mdev = 8.342/8.388/8.519/0.049 ms
■ 参考
・The Netperf Homepage
・クラウドでのネットワーク レイテンシの測定
・【Linux】pingコマンドの使い方