Help us understand the problem. What is going on with this article?

Redis Sentinel構成のdown-after-millisecondsをどこまで減らせるか実験

概要

Redis Sentinel構成を取った際に、検討すべきパラメータとしてdown-after-millisecondsというのがあります。

down-after-milliseconds is the time in milliseconds an instance should not be reachable (either does not reply to our PINGs or it is replying with an error) for a Sentinel starting to think it is down.

Redis Sentinel Documentation – Redis
SentinelプロセスがRedisプロセスがダウンしたと判断する時間[ミリ秒]とあります。つまり、これが短いほどフェイルオーバーが速く完了して、クライアントのエラーになる可能性が減るということになります。
デフォルトは30秒ですが、どこまで減らせるかを実験してみました。

前提

サーバ:Amazon Linux2
インスタンスタイプ:t2.micro
Redis:5.0.8

1台のサーバにRedis(master)x1、Redis(replica)x2、Sentinelx2を起動する構成。
フェイルオーバーはmasterをshutdownして発生させる。

結論

10秒くらいが妥当ではないかと判断します。(down-after-milliseconds=10000)
それ以上下げると、ネットワーク瞬断等でフェイルオーバーしてしまう可能性が高まります。フェイルオーバーするとマニュアルオペレーションで戻す必要があり、Redis Sentinel構成の運用負荷が上がるので、自動でフェイルオーバーする仕組みをスポイルするのではと思います。

実験

以下の秒数を実験した。

30秒、20秒、10秒、1秒、1ミリ秒、0ミリ秒

まとめ

秒数 フェイルオーバーにかかった秒数(※1) OK/NG
30秒 32.3秒 OK
20秒 22.5秒 OK
10秒 12.1秒 OK
1秒 3.3秒 OK(※2)
1ミリ秒 4.8秒 NG
0ミリ秒 - NG

※1・・User requested shutdown... と +switch-master mymasterの間。
※2・・今回は同一サーバ内でネットワークレイテンシがない状態だったが、分散配置した時はNGになると思われる。

実験詳細

30秒

[ec2-user@ip-172-30-2-184 redis-5.0.8]$ src/redis-cli -p 6379 shutdown
3457:M 12 Jul 2020 12:41:44.639 # User requested shutdown...
3457:M 12 Jul 2020 12:41:44.639 * Saving the final RDB snapshot before exiting.
3457:M 12 Jul 2020 12:41:44.641 * DB saved on disk
3457:M 12 Jul 2020 12:41:44.641 * Removing the pid file.
3457:M 12 Jul 2020 12:41:44.641 # Redis is now ready to exit, bye bye...
3466:S 12 Jul 2020 12:41:44.641 # Connection with master lost.
3466:S 12 Jul 2020 12:41:44.641 * Caching the disconnected master state.
3461:S 12 Jul 2020 12:41:44.641 # Connection with master lost.
3461:S 12 Jul 2020 12:41:44.641 * Caching the disconnected master state.
[1]   Done                    ./src/redis-server ./redis1.conf
[ec2-user@ip-172-30-2-184 redis-5.0.8]$ 3461:S 12 Jul 2020 12:41:44.643 * Connecting to MASTER 172.30.2.184:6379
3461:S 12 Jul 2020 12:41:44.643 * MASTER <-> REPLICA sync started
3461:S 12 Jul 2020 12:41:44.643 # Error condition on socket for SYNC: Connection refused
3466:S 12 Jul 2020 12:41:45.318 * Connecting to MASTER 172.30.2.184:6379
3466:S 12 Jul 2020 12:41:45.318 * MASTER <-> REPLICA sync started
3466:S 12 Jul 2020 12:41:45.318 # Error condition on socket for SYNC: Connection refused
3461:S 12 Jul 2020 12:41:45.646 * Connecting to MASTER 172.30.2.184:6379
3461:S 12 Jul 2020 12:41:45.646 * MASTER <-> REPLICA sync started
3461:S 12 Jul 2020 12:41:45.646 # Error condition on socket for SYNC: Connection refused
3466:S 12 Jul 2020 12:41:46.320 * Connecting to MASTER 172.30.2.184:6379
3466:S 12 Jul 2020 12:41:46.320 * MASTER <-> REPLICA sync started
3466:S 12 Jul 2020 12:41:46.320 # Error condition on socket for SYNC: Connection refused
3461:S 12 Jul 2020 12:41:46.647 * Connecting to MASTER 172.30.2.184:6379
3461:S 12 Jul 2020 12:41:46.648 * MASTER <-> REPLICA sync started
3461:S 12 Jul 2020 12:41:46.648 # Error condition on socket for SYNC: Connection refused
3466:S 12 Jul 2020 12:41:47.323 * Connecting to MASTER 172.30.2.184:6379
3466:S 12 Jul 2020 12:41:47.323 * MASTER <-> REPLICA sync started
3466:S 12 Jul 2020 12:41:47.323 # Error condition on socket for SYNC: Connection refused
3461:S 12 Jul 2020 12:41:47.651 * Connecting to MASTER 172.30.2.184:6379
3461:S 12 Jul 2020 12:41:47.651 * MASTER <-> REPLICA sync started
3461:S 12 Jul 2020 12:41:47.651 # Error condition on socket for SYNC: Connection refused
3466:S 12 Jul 2020 12:41:48.326 * Connecting to MASTER 172.30.2.184:6379
3466:S 12 Jul 2020 12:41:48.326 * MASTER <-> REPLICA sync started
3466:S 12 Jul 2020 12:41:48.326 # Error condition on socket for SYNC: Connection refused
3461:S 12 Jul 2020 12:41:48.655 * Connecting to MASTER 172.30.2.184:6379
3461:S 12 Jul 2020 12:41:48.655 * MASTER <-> REPLICA sync started
3461:S 12 Jul 2020 12:41:48.655 # Error condition on socket for SYNC: Connection refused
3466:S 12 Jul 2020 12:41:49.329 * Connecting to MASTER 172.30.2.184:6379
3466:S 12 Jul 2020 12:41:49.330 * MASTER <-> REPLICA sync started
3466:S 12 Jul 2020 12:41:49.330 # Error condition on socket for SYNC: Connection refused
3461:S 12 Jul 2020 12:41:49.659 * Connecting to MASTER 172.30.2.184:6379
3461:S 12 Jul 2020 12:41:49.659 * MASTER <-> REPLICA sync started
3461:S 12 Jul 2020 12:41:49.659 # Error condition on socket for SYNC: Connection refused

(中略)

3466:S 12 Jul 2020 12:42:11.389 * Connecting to MASTER 172.30.2.184:6379
3466:S 12 Jul 2020 12:42:11.389 * MASTER <-> REPLICA sync started
3466:S 12 Jul 2020 12:42:11.389 # Error condition on socket for SYNC: Connection refused
3461:S 12 Jul 2020 12:42:11.718 * Connecting to MASTER 172.30.2.184:6379
3461:S 12 Jul 2020 12:42:11.718 * MASTER <-> REPLICA sync started
3461:S 12 Jul 2020 12:42:11.718 # Error condition on socket for SYNC: Connection refused
3466:S 12 Jul 2020 12:42:12.391 * Connecting to MASTER 172.30.2.184:6379
3466:S 12 Jul 2020 12:42:12.391 * MASTER <-> REPLICA sync started
3466:S 12 Jul 2020 12:42:12.391 # Error condition on socket for SYNC: Connection refused
3461:S 12 Jul 2020 12:42:12.720 * Connecting to MASTER 172.30.2.184:6379
3461:S 12 Jul 2020 12:42:12.720 * MASTER <-> REPLICA sync started
3461:S 12 Jul 2020 12:42:12.720 # Error condition on socket for SYNC: Connection refused
3466:S 12 Jul 2020 12:42:13.394 * Connecting to MASTER 172.30.2.184:6379
3466:S 12 Jul 2020 12:42:13.394 * MASTER <-> REPLICA sync started
3466:S 12 Jul 2020 12:42:13.395 # Error condition on socket for SYNC: Connection refused
3461:S 12 Jul 2020 12:42:13.723 * Connecting to MASTER 172.30.2.184:6379
3461:S 12 Jul 2020 12:42:13.723 * MASTER <-> REPLICA sync started
3461:S 12 Jul 2020 12:42:13.723 # Error condition on socket for SYNC: Connection refused
3466:S 12 Jul 2020 12:42:14.396 * Connecting to MASTER 172.30.2.184:6379
3466:S 12 Jul 2020 12:42:14.396 * MASTER <-> REPLICA sync started
3466:S 12 Jul 2020 12:42:14.397 # Error condition on socket for SYNC: Connection refused
3478:X 12 Jul 2020 12:42:14.710 # +sdown master mymaster 172.30.2.184 6379
3474:X 12 Jul 2020 12:42:14.719 # +sdown master mymaster 172.30.2.184 6379
3461:S 12 Jul 2020 12:42:14.725 * Connecting to MASTER 172.30.2.184:6379
3461:S 12 Jul 2020 12:42:14.725 * MASTER <-> REPLICA sync started
3461:S 12 Jul 2020 12:42:14.725 # Error condition on socket for SYNC: Connection refused
3474:X 12 Jul 2020 12:42:14.785 # +odown master mymaster 172.30.2.184 6379 #quorum 2/2
3474:X 12 Jul 2020 12:42:14.785 # +new-epoch 13
3474:X 12 Jul 2020 12:42:14.785 # +try-failover master mymaster 172.30.2.184 6379
3474:X 12 Jul 2020 12:42:14.787 # +vote-for-leader 9a72799337b637763343e01e971a580d51c7d4be 13
3478:X 12 Jul 2020 12:42:14.789 # +new-epoch 13
3478:X 12 Jul 2020 12:42:14.790 # +vote-for-leader 9a72799337b637763343e01e971a580d51c7d4be 13
3474:X 12 Jul 2020 12:42:14.790 # 9a72799337b637763343e01e971a580d51c7d4bf voted for 9a72799337b637763343e01e971a580d51c7d4be 13
3474:X 12 Jul 2020 12:42:14.849 # +elected-leader master mymaster 172.30.2.184 6379
3474:X 12 Jul 2020 12:42:14.849 # +failover-state-select-slave master mymaster 172.30.2.184 6379
3474:X 12 Jul 2020 12:42:14.950 # +selected-slave slave 172.30.2.184:6381 172.30.2.184 6381 @ mymaster 172.30.2.184 6379
3474:X 12 Jul 2020 12:42:14.950 * +failover-state-send-slaveof-noone slave 172.30.2.184:6381 172.30.2.184 6381 @ mymaster 172.30.2.184 6379
3474:X 12 Jul 2020 12:42:15.051 * +failover-state-wait-promotion slave 172.30.2.184:6381 172.30.2.184 6381 @ mymaster 172.30.2.184 6379
3466:M 12 Jul 2020 12:42:15.051 # Setting secondary replication ID to a99b1062bf55c59dc92efaa28d15629759725025, valid up to offset: 37809. New replication ID is 2a8a8c8bca5ece9b0df8dce813e00eb5dcb18803
3466:M 12 Jul 2020 12:42:15.051 * Discarding previously cached master state.
3466:M 12 Jul 2020 12:42:15.051 * MASTER MODE enabled (user request from 'id=5 addr=172.30.2.184:42552 fd=9 name=sentinel-9a727993-cmd age=304 idle=0 flags=x db=0 sub=0 psub=0 multi=3 qbuf=140 qbuf-free=32628 obl=36 oll=0 omem=0 events=r cmd=exec')
3466:M 12 Jul 2020 12:42:15.052 # CONFIG REWRITE executed with success.
3461:S 12 Jul 2020 12:42:15.727 * Connecting to MASTER 172.30.2.184:6379
3461:S 12 Jul 2020 12:42:15.727 * MASTER <-> REPLICA sync started
3461:S 12 Jul 2020 12:42:15.727 # Error condition on socket for SYNC: Connection refused
3478:X 12 Jul 2020 12:42:15.782 # +odown master mymaster 172.30.2.184 6379 #quorum 2/2
3478:X 12 Jul 2020 12:42:15.782 # Next failover delay: I will not start a failover before Sun Jul 12 12:48:14 2020
3474:X 12 Jul 2020 12:42:15.806 # +promoted-slave slave 172.30.2.184:6381 172.30.2.184 6381 @ mymaster 172.30.2.184 6379
3474:X 12 Jul 2020 12:42:15.806 # +failover-state-reconf-slaves master mymaster 172.30.2.184 6379
3474:X 12 Jul 2020 12:42:15.870 * +slave-reconf-sent slave 172.30.2.184:6380 172.30.2.184 6380 @ mymaster 172.30.2.184 6379
3461:S 12 Jul 2020 12:42:15.870 * REPLICAOF 172.30.2.184:6381 enabled (user request from 'id=5 addr=172.30.2.184:40246 fd=9 name=sentinel-9a727993-cmd age=304 idle=0 flags=x db=0 sub=0 psub=0 multi=3 qbuf=293 qbuf-free=32475 obl=36 oll=0 omem=0 events=r cmd=exec')
3461:S 12 Jul 2020 12:42:15.871 # CONFIG REWRITE executed with success.
3478:X 12 Jul 2020 12:42:15.872 # +config-update-from sentinel 9a72799337b637763343e01e971a580d51c7d4be 172.30.2.184 26379 @ mymaster 172.30.2.184 6379
3478:X 12 Jul 2020 12:42:15.872 # +switch-master mymaster 172.30.2.184 6379 172.30.2.184 6381
3478:X 12 Jul 2020 12:42:15.872 * +slave slave 172.30.2.184:6380 172.30.2.184 6380 @ mymaster 172.30.2.184 6381
3478:X 12 Jul 2020 12:42:15.872 * +slave slave 172.30.2.184:6379 172.30.2.184 6379 @ mymaster 172.30.2.184 6381
3461:S 12 Jul 2020 12:42:16.730 * Connecting to MASTER 172.30.2.184:6381
3461:S 12 Jul 2020 12:42:16.730 * MASTER <-> REPLICA sync started
3461:S 12 Jul 2020 12:42:16.730 * Non blocking connect for SYNC fired the event.
3461:S 12 Jul 2020 12:42:16.730 * Master replied to PING, replication can continue...
3461:S 12 Jul 2020 12:42:16.730 * Trying a partial resynchronization (request a99b1062bf55c59dc92efaa28d15629759725025:37809).
3466:M 12 Jul 2020 12:42:16.730 * Replica 172.30.2.184:6380 asks for synchronization
3466:M 12 Jul 2020 12:42:16.730 * Partial resynchronization request from 172.30.2.184:6380 accepted. Sending 446 bytes of backlog starting from offset 37809.
3461:S 12 Jul 2020 12:42:16.730 * Successful partial resynchronization with master.
3461:S 12 Jul 2020 12:42:16.731 # Master replication ID changed to 2a8a8c8bca5ece9b0df8dce813e00eb5dcb18803
3461:S 12 Jul 2020 12:42:16.731 * MASTER <-> REPLICA sync: Master accepted a Partial Resynchronization.
3474:X 12 Jul 2020 12:42:16.868 * +slave-reconf-inprog slave 172.30.2.184:6380 172.30.2.184 6380 @ mymaster 172.30.2.184 6379
3474:X 12 Jul 2020 12:42:16.868 * +slave-reconf-done slave 172.30.2.184:6380 172.30.2.184 6380 @ mymaster 172.30.2.184 6379
3474:X 12 Jul 2020 12:42:16.929 # -odown master mymaster 172.30.2.184 6379
3474:X 12 Jul 2020 12:42:16.929 # +failover-end master mymaster 172.30.2.184 6379
3474:X 12 Jul 2020 12:42:16.929 # +switch-master mymaster 172.30.2.184 6379 172.30.2.184 6381
3474:X 12 Jul 2020 12:42:16.930 * +slave slave 172.30.2.184:6380 172.30.2.184 6380 @ mymaster 172.30.2.184 6381
3474:X 12 Jul 2020 12:42:16.930 * +slave slave 172.30.2.184:6379 172.30.2.184 6379 @ mymaster 172.30.2.184 6381
3478:X 12 Jul 2020 12:42:45.889 # +sdown slave 172.30.2.184:6379 172.30.2.184 6379 @ mymaster 172.30.2.184 6381
3474:X 12 Jul 2020 12:42:46.972 # +sdown slave 172.30.2.184:6379 172.30.2.184 6379 @ mymaster 172.30.2.184 6381

[ec2-user@ip-172-30-2-184 redis-5.0.8]$

20秒

[ec2-user@ip-172-30-2-184 redis-5.0.8]$ src/redis-cli shutdown
4284:M 12 Jul 2020 13:47:29.290 # User requested shutdown...
4284:M 12 Jul 2020 13:47:29.290 * Saving the final RDB snapshot before exiting.
4284:M 12 Jul 2020 13:47:29.292 * DB saved on disk
4284:M 12 Jul 2020 13:47:29.292 * Removing the pid file.
4284:M 12 Jul 2020 13:47:29.292 # Redis is now ready to exit, bye bye...
4285:S 12 Jul 2020 13:47:29.292 # Connection with master lost.
4285:S 12 Jul 2020 13:47:29.292 * Caching the disconnected master state.
4286:S 12 Jul 2020 13:47:29.292 # Connection with master lost.
4286:S 12 Jul 2020 13:47:29.292 * Caching the disconnected master state.
[ec2-user@ip-172-30-2-184 redis-5.0.8]$ 4286:S 12 Jul 2020 13:47:30.258 * Connecting to MASTER 172.30.2.184:6379
4286:S 12 Jul 2020 13:47:30.258 * MASTER <-> REPLICA sync started
4286:S 12 Jul 2020 13:47:30.258 # Error condition on socket for SYNC: Connection refused
4285:S 12 Jul 2020 13:47:30.258 * Connecting to MASTER 172.30.2.184:6379
4285:S 12 Jul 2020 13:47:30.258 * MASTER <-> REPLICA sync started
4285:S 12 Jul 2020 13:47:30.258 # Error condition on socket for SYNC: Connection refused
4286:S 12 Jul 2020 13:47:31.261 * Connecting to MASTER 172.30.2.184:6379
4285:S 12 Jul 2020 13:47:31.261 * Connecting to MASTER 172.30.2.184:6379
4285:S 12 Jul 2020 13:47:31.261 * MASTER <-> REPLICA sync started
4286:S 12 Jul 2020 13:47:31.261 * MASTER <-> REPLICA sync started
4286:S 12 Jul 2020 13:47:31.261 # Error condition on socket for SYNC: Connection refused
4285:S 12 Jul 2020 13:47:31.261 # Error condition on socket for SYNC: Connection refused
4285:S 12 Jul 2020 13:47:32.263 * Connecting to MASTER 172.30.2.184:6379
4285:S 12 Jul 2020 13:47:32.263 * MASTER <-> REPLICA sync started
4285:S 12 Jul 2020 13:47:32.263 # Error condition on socket for SYNC: Connection refused
4286:S 12 Jul 2020 13:47:32.263 * Connecting to MASTER 172.30.2.184:6379
4286:S 12 Jul 2020 13:47:32.263 * MASTER <-> REPLICA sync started
4286:S 12 Jul 2020 13:47:32.263 # Error condition on socket for SYNC: Connection refused
4285:S 12 Jul 2020 13:47:33.265 * Connecting to MASTER 172.30.2.184:6379
4285:S 12 Jul 2020 13:47:33.265 * MASTER <-> REPLICA sync started
4285:S 12 Jul 2020 13:47:33.265 # Error condition on socket for SYNC: Connection refused

(中略)

4286:S 12 Jul 2020 13:47:46.301 * Connecting to MASTER 172.30.2.184:6379
4286:S 12 Jul 2020 13:47:46.301 * MASTER <-> REPLICA sync started
4286:S 12 Jul 2020 13:47:46.301 # Error condition on socket for SYNC: Connection refused
4285:S 12 Jul 2020 13:47:47.303 * Connecting to MASTER 172.30.2.184:6379
4285:S 12 Jul 2020 13:47:47.303 * MASTER <-> REPLICA sync started
4285:S 12 Jul 2020 13:47:47.303 # Error condition on socket for SYNC: Connection refused
4286:S 12 Jul 2020 13:47:47.303 * Connecting to MASTER 172.30.2.184:6379
4286:S 12 Jul 2020 13:47:47.304 * MASTER <-> REPLICA sync started
4286:S 12 Jul 2020 13:47:47.304 # Error condition on socket for SYNC: Connection refused
4286:S 12 Jul 2020 13:47:48.307 * Connecting to MASTER 172.30.2.184:6379
4286:S 12 Jul 2020 13:47:48.307 * MASTER <-> REPLICA sync started
4286:S 12 Jul 2020 13:47:48.307 # Error condition on socket for SYNC: Connection refused
4285:S 12 Jul 2020 13:47:48.307 * Connecting to MASTER 172.30.2.184:6379
4285:S 12 Jul 2020 13:47:48.307 * MASTER <-> REPLICA sync started
4285:S 12 Jul 2020 13:47:48.307 # Error condition on socket for SYNC: Connection refused
4285:S 12 Jul 2020 13:47:49.309 * Connecting to MASTER 172.30.2.184:6379
4285:S 12 Jul 2020 13:47:49.309 * MASTER <-> REPLICA sync started
4285:S 12 Jul 2020 13:47:49.309 # Error condition on socket for SYNC: Connection refused
4286:S 12 Jul 2020 13:47:49.310 * Connecting to MASTER 172.30.2.184:6379
4286:S 12 Jul 2020 13:47:49.310 * MASTER <-> REPLICA sync started
4286:S 12 Jul 2020 13:47:49.310 # Error condition on socket for SYNC: Connection refused
4287:X 12 Jul 2020 13:47:49.324 # +sdown master mymaster 172.30.2.184 6379
4288:X 12 Jul 2020 13:47:49.391 # +sdown master mymaster 172.30.2.184 6379
4288:X 12 Jul 2020 13:47:49.467 # +odown master mymaster 172.30.2.184 6379 #quorum 2/2
4288:X 12 Jul 2020 13:47:49.467 # +new-epoch 1
4288:X 12 Jul 2020 13:47:49.467 # +try-failover master mymaster 172.30.2.184 6379
4288:X 12 Jul 2020 13:47:49.469 # +vote-for-leader 9a72799337b637763343e01e971a580d51c7d4bf 1
4287:X 12 Jul 2020 13:47:49.471 # +new-epoch 1
4287:X 12 Jul 2020 13:47:49.472 # +vote-for-leader 9a72799337b637763343e01e971a580d51c7d4bf 1
4288:X 12 Jul 2020 13:47:49.472 # 9a72799337b637763343e01e971a580d51c7d4be voted for 9a72799337b637763343e01e971a580d51c7d4bf 1
4288:X 12 Jul 2020 13:47:49.552 # +elected-leader master mymaster 172.30.2.184 6379
4288:X 12 Jul 2020 13:47:49.552 # +failover-state-select-slave master mymaster 172.30.2.184 6379
4288:X 12 Jul 2020 13:47:49.619 # +selected-slave slave 172.30.2.184:6380 172.30.2.184 6380 @ mymaster 172.30.2.184 6379
4288:X 12 Jul 2020 13:47:49.619 * +failover-state-send-slaveof-noone slave 172.30.2.184:6380 172.30.2.184 6380 @ mymaster 172.30.2.184 6379
4288:X 12 Jul 2020 13:47:49.674 * +failover-state-wait-promotion slave 172.30.2.184:6380 172.30.2.184 6380 @ mymaster 172.30.2.184 6379
4285:M 12 Jul 2020 13:47:49.674 # Setting secondary replication ID to dea4e6d9164f1ca225b8f4d5277fa3a98400283f, valid up to offset: 14064. New replication ID is 177f6a24d9489e62371172980a6e3c99b7a38a7e
4285:M 12 Jul 2020 13:47:49.674 * Discarding previously cached master state.
4285:M 12 Jul 2020 13:47:49.674 * MASTER MODE enabled (user request from 'id=6 addr=172.30.2.184:48484 fd=12 name=sentinel-9a727993-cmd age=122 idle=0 flags=x db=0 sub=0 psub=0 multi=3 qbuf=140 qbuf-free=32628 obl=36 oll=0 omem=0 events=r cmd=exec')
4285:M 12 Jul 2020 13:47:49.676 # CONFIG REWRITE executed with success.
4286:S 12 Jul 2020 13:47:50.312 * Connecting to MASTER 172.30.2.184:6379
4286:S 12 Jul 2020 13:47:50.312 * MASTER <-> REPLICA sync started
4286:S 12 Jul 2020 13:47:50.312 # Error condition on socket for SYNC: Connection refused
4287:X 12 Jul 2020 13:47:50.482 # +odown master mymaster 172.30.2.184 6379 #quorum 2/2
4287:X 12 Jul 2020 13:47:50.482 # Next failover delay: I will not start a failover before Sun Jul 12 13:53:49 2020
4288:X 12 Jul 2020 13:47:50.668 # +promoted-slave slave 172.30.2.184:6380 172.30.2.184 6380 @ mymaster 172.30.2.184 6379
4288:X 12 Jul 2020 13:47:50.668 # +failover-state-reconf-slaves master mymaster 172.30.2.184 6379
4288:X 12 Jul 2020 13:47:50.725 * +slave-reconf-sent slave 172.30.2.184:6381 172.30.2.184 6381 @ mymaster 172.30.2.184 6379
4286:S 12 Jul 2020 13:47:50.725 * REPLICAOF 172.30.2.184:6380 enabled (user request from 'id=7 addr=172.30.2.184:50798 fd=12 name=sentinel-9a727993-cmd age=123 idle=0 flags=x db=0 sub=0 psub=0 multi=3 qbuf=291 qbuf-free=32477 obl=36 oll=0 omem=0 events=r cmd=exec')
4286:S 12 Jul 2020 13:47:50.726 # CONFIG REWRITE executed with success.
4287:X 12 Jul 2020 13:47:50.726 # +config-update-from sentinel 9a72799337b637763343e01e971a580d51c7d4bf 172.30.2.184 26380 @ mymaster 172.30.2.184 6379
4287:X 12 Jul 2020 13:47:50.726 # +switch-master mymaster 172.30.2.184 6379 172.30.2.184 6380
4287:X 12 Jul 2020 13:47:50.726 * +slave slave 172.30.2.184:6381 172.30.2.184 6381 @ mymaster 172.30.2.184 6380
4287:X 12 Jul 2020 13:47:50.726 * +slave slave 172.30.2.184:6379 172.30.2.184 6379 @ mymaster 172.30.2.184 6380
4286:S 12 Jul 2020 13:47:51.314 * Connecting to MASTER 172.30.2.184:6380
4286:S 12 Jul 2020 13:47:51.314 * MASTER <-> REPLICA sync started
4286:S 12 Jul 2020 13:47:51.314 * Non blocking connect for SYNC fired the event.
4286:S 12 Jul 2020 13:47:51.314 * Master replied to PING, replication can continue...
4286:S 12 Jul 2020 13:47:51.315 * Trying a partial resynchronization (request dea4e6d9164f1ca225b8f4d5277fa3a98400283f:14064).
4285:M 12 Jul 2020 13:47:51.315 * Replica 172.30.2.184:6381 asks for synchronization
4285:M 12 Jul 2020 13:47:51.315 * Partial resynchronization request from 172.30.2.184:6381 accepted. Sending 579 bytes of backlog starting from offset 14064.
4286:S 12 Jul 2020 13:47:51.315 * Successful partial resynchronization with master.
4286:S 12 Jul 2020 13:47:51.315 # Master replication ID changed to 177f6a24d9489e62371172980a6e3c99b7a38a7e
4286:S 12 Jul 2020 13:47:51.315 * MASTER <-> REPLICA sync: Master accepted a Partial Resynchronization.
4288:X 12 Jul 2020 13:47:51.600 # -odown master mymaster 172.30.2.184 6379
4288:X 12 Jul 2020 13:47:51.676 * +slave-reconf-inprog slave 172.30.2.184:6381 172.30.2.184 6381 @ mymaster 172.30.2.184 6379
4288:X 12 Jul 2020 13:47:51.677 * +slave-reconf-done slave 172.30.2.184:6381 172.30.2.184 6381 @ mymaster 172.30.2.184 6379
4288:X 12 Jul 2020 13:47:51.747 # +failover-end master mymaster 172.30.2.184 6379
4288:X 12 Jul 2020 13:47:51.747 # +switch-master mymaster 172.30.2.184 6379 172.30.2.184 6380
4288:X 12 Jul 2020 13:47:51.747 * +slave slave 172.30.2.184:6381 172.30.2.184 6381 @ mymaster 172.30.2.184 6380
4288:X 12 Jul 2020 13:47:51.747 * +slave slave 172.30.2.184:6379 172.30.2.184 6379 @ mymaster 172.30.2.184 6380

[ec2-user@ip-172-30-2-184 redis-5.0.8]$ 4287:X 12 Jul 2020 13:48:10.778 # +sdown slave 172.30.2.184:6379 172.30.2.184 6379 @ mymaster 172.30.2.184 6380
4288:X 12 Jul 2020 13:48:11.788 # +sdown slave 172.30.2.184:6379 172.30.2.184 6379 @ mymaster 172.30.2.184 6380

[ec2-user@ip-172-30-2-184 redis-5.0.8]$

10秒

[ec2-user@ip-172-30-2-184 redis-5.0.8]$ src/redis-cli shutdown
4385:M 12 Jul 2020 13:52:48.652 # User requested shutdown...
4385:M 12 Jul 2020 13:52:48.652 * Saving the final RDB snapshot before exiting.
4385:M 12 Jul 2020 13:52:48.654 * DB saved on disk
4385:M 12 Jul 2020 13:52:48.654 * Removing the pid file.
4385:M 12 Jul 2020 13:52:48.654 # Redis is now ready to exit, bye bye...
4386:S 12 Jul 2020 13:52:48.654 # Connection with master lost.
4386:S 12 Jul 2020 13:52:48.654 * Caching the disconnected master state.
4387:S 12 Jul 2020 13:52:48.654 # Connection with master lost.
4387:S 12 Jul 2020 13:52:48.654 * Caching the disconnected master state.
[ec2-user@ip-172-30-2-184 redis-5.0.8]$ 4387:S 12 Jul 2020 13:52:48.965 * Connecting to MASTER 172.30.2.184:6379
4387:S 12 Jul 2020 13:52:48.965 * MASTER <-> REPLICA sync started
4387:S 12 Jul 2020 13:52:48.965 # Error condition on socket for SYNC: Connection refused
4386:S 12 Jul 2020 13:52:48.967 * Connecting to MASTER 172.30.2.184:6379
4386:S 12 Jul 2020 13:52:48.967 * MASTER <-> REPLICA sync started
4386:S 12 Jul 2020 13:52:48.967 # Error condition on socket for SYNC: Connection refused
4387:S 12 Jul 2020 13:52:49.967 * Connecting to MASTER 172.30.2.184:6379
4387:S 12 Jul 2020 13:52:49.967 * MASTER <-> REPLICA sync started
4387:S 12 Jul 2020 13:52:49.967 # Error condition on socket for SYNC: Connection refused
4386:S 12 Jul 2020 13:52:49.969 * Connecting to MASTER 172.30.2.184:6379
4386:S 12 Jul 2020 13:52:49.969 * MASTER <-> REPLICA sync started
4386:S 12 Jul 2020 13:52:49.969 # Error condition on socket for SYNC: Connection refused
4387:S 12 Jul 2020 13:52:50.968 * Connecting to MASTER 172.30.2.184:6379
4387:S 12 Jul 2020 13:52:50.968 * MASTER <-> REPLICA sync started
4387:S 12 Jul 2020 13:52:50.969 # Error condition on socket for SYNC: Connection refused
4386:S 12 Jul 2020 13:52:50.970 * Connecting to MASTER 172.30.2.184:6379
4386:S 12 Jul 2020 13:52:50.970 * MASTER <-> REPLICA sync started
4386:S 12 Jul 2020 13:52:50.970 # Error condition on socket for SYNC: Connection refused
4386:S 12 Jul 2020 13:52:51.972 * Connecting to MASTER 172.30.2.184:6379
4386:S 12 Jul 2020 13:52:51.972 * MASTER <-> REPLICA sync started
4386:S 12 Jul 2020 13:52:51.972 # Error condition on socket for SYNC: Connection refused
4387:S 12 Jul 2020 13:52:51.972 * Connecting to MASTER 172.30.2.184:6379
4387:S 12 Jul 2020 13:52:51.972 * MASTER <-> REPLICA sync started
4387:S 12 Jul 2020 13:52:51.972 # Error condition on socket for SYNC: Connection refused
4386:S 12 Jul 2020 13:52:52.975 * Connecting to MASTER 172.30.2.184:6379
4386:S 12 Jul 2020 13:52:52.975 * MASTER <-> REPLICA sync started
4386:S 12 Jul 2020 13:52:52.975 # Error condition on socket for SYNC: Connection refused
4387:S 12 Jul 2020 13:52:52.975 * Connecting to MASTER 172.30.2.184:6379
4387:S 12 Jul 2020 13:52:52.975 * MASTER <-> REPLICA sync started
4387:S 12 Jul 2020 13:52:52.975 # Error condition on socket for SYNC: Connection refused
4386:S 12 Jul 2020 13:52:53.977 * Connecting to MASTER 172.30.2.184:6379
4386:S 12 Jul 2020 13:52:53.977 * MASTER <-> REPLICA sync started
4386:S 12 Jul 2020 13:52:53.977 # Error condition on socket for SYNC: Connection refused
4387:S 12 Jul 2020 13:52:53.977 * Connecting to MASTER 172.30.2.184:6379
4387:S 12 Jul 2020 13:52:53.977 * MASTER <-> REPLICA sync started
4387:S 12 Jul 2020 13:52:53.977 # Error condition on socket for SYNC: Connection refused
4386:S 12 Jul 2020 13:52:54.979 * Connecting to MASTER 172.30.2.184:6379
4386:S 12 Jul 2020 13:52:54.979 * MASTER <-> REPLICA sync started
4386:S 12 Jul 2020 13:52:54.979 # Error condition on socket for SYNC: Connection refused
4387:S 12 Jul 2020 13:52:54.980 * Connecting to MASTER 172.30.2.184:6379
4387:S 12 Jul 2020 13:52:54.980 * MASTER <-> REPLICA sync started
4387:S 12 Jul 2020 13:52:54.980 # Error condition on socket for SYNC: Connection refused
4386:S 12 Jul 2020 13:52:55.981 * Connecting to MASTER 172.30.2.184:6379
4386:S 12 Jul 2020 13:52:55.981 * MASTER <-> REPLICA sync started
4386:S 12 Jul 2020 13:52:55.981 # Error condition on socket for SYNC: Connection refused
4387:S 12 Jul 2020 13:52:55.982 * Connecting to MASTER 172.30.2.184:6379
4387:S 12 Jul 2020 13:52:55.982 * MASTER <-> REPLICA sync started
4387:S 12 Jul 2020 13:52:55.982 # Error condition on socket for SYNC: Connection refused
4386:S 12 Jul 2020 13:52:56.983 * Connecting to MASTER 172.30.2.184:6379
4386:S 12 Jul 2020 13:52:56.983 * MASTER <-> REPLICA sync started
4386:S 12 Jul 2020 13:52:56.983 # Error condition on socket for SYNC: Connection refused
4387:S 12 Jul 2020 13:52:56.984 * Connecting to MASTER 172.30.2.184:6379
4387:S 12 Jul 2020 13:52:56.984 * MASTER <-> REPLICA sync started
4387:S 12 Jul 2020 13:52:56.984 # Error condition on socket for SYNC: Connection refused
4386:S 12 Jul 2020 13:52:57.985 * Connecting to MASTER 172.30.2.184:6379
4386:S 12 Jul 2020 13:52:57.985 * MASTER <-> REPLICA sync started
4386:S 12 Jul 2020 13:52:57.985 # Error condition on socket for SYNC: Connection refused
4387:S 12 Jul 2020 13:52:57.986 * Connecting to MASTER 172.30.2.184:6379
4387:S 12 Jul 2020 13:52:57.986 * MASTER <-> REPLICA sync started
4387:S 12 Jul 2020 13:52:57.986 # Error condition on socket for SYNC: Connection refused
4388:X 12 Jul 2020 13:52:58.721 # +sdown master mymaster 172.30.2.184 6379
4389:X 12 Jul 2020 13:52:58.722 # +sdown master mymaster 172.30.2.184 6379
4389:X 12 Jul 2020 13:52:58.785 # +odown master mymaster 172.30.2.184 6379 #quorum 2/2
4389:X 12 Jul 2020 13:52:58.785 # +new-epoch 1
4389:X 12 Jul 2020 13:52:58.785 # +try-failover master mymaster 172.30.2.184 6379
4389:X 12 Jul 2020 13:52:58.787 # +vote-for-leader 9a72799337b637763343e01e971a580d51c7d4bf 1
4388:X 12 Jul 2020 13:52:58.789 # +new-epoch 1
4388:X 12 Jul 2020 13:52:58.790 # +vote-for-leader 9a72799337b637763343e01e971a580d51c7d4bf 1
4389:X 12 Jul 2020 13:52:58.790 # 9a72799337b637763343e01e971a580d51c7d4be voted for 9a72799337b637763343e01e971a580d51c7d4bf 1
4389:X 12 Jul 2020 13:52:58.877 # +elected-leader master mymaster 172.30.2.184 6379
4389:X 12 Jul 2020 13:52:58.877 # +failover-state-select-slave master mymaster 172.30.2.184 6379
4389:X 12 Jul 2020 13:52:58.949 # +selected-slave slave 172.30.2.184:6380 172.30.2.184 6380 @ mymaster 172.30.2.184 6379
4389:X 12 Jul 2020 13:52:58.949 * +failover-state-send-slaveof-noone slave 172.30.2.184:6380 172.30.2.184 6380 @ mymaster 172.30.2.184 6379
4386:S 12 Jul 2020 13:52:58.987 * Connecting to MASTER 172.30.2.184:6379
4386:S 12 Jul 2020 13:52:58.987 * MASTER <-> REPLICA sync started
4386:S 12 Jul 2020 13:52:58.987 # Error condition on socket for SYNC: Connection refused
4387:S 12 Jul 2020 13:52:58.987 * Connecting to MASTER 172.30.2.184:6379
4387:S 12 Jul 2020 13:52:58.987 * MASTER <-> REPLICA sync started
4387:S 12 Jul 2020 13:52:58.987 # Error condition on socket for SYNC: Connection refused
4389:X 12 Jul 2020 13:52:59.039 * +failover-state-wait-promotion slave 172.30.2.184:6380 172.30.2.184 6380 @ mymaster 172.30.2.184 6379
4386:M 12 Jul 2020 13:52:59.039 # Setting secondary replication ID to 09763946986da00e53793f0e69a0650277b1840c, valid up to offset: 19402. New replication ID is a1b501dea123c55bc1e1bb992e3a5cb6d52ddb57
4386:M 12 Jul 2020 13:52:59.039 * Discarding previously cached master state.
4386:M 12 Jul 2020 13:52:59.039 * MASTER MODE enabled (user request from 'id=4 addr=172.30.2.184:49574 fd=10 name=sentinel-9a727993-cmd age=152 idle=1 flags=x db=0 sub=0 psub=0 multi=3 qbuf=140 qbuf-free=32628 obl=36 oll=0 omem=0 events=r cmd=exec')
4386:M 12 Jul 2020 13:52:59.040 # CONFIG REWRITE executed with success.
4389:X 12 Jul 2020 13:52:59.669 # +promoted-slave slave 172.30.2.184:6380 172.30.2.184 6380 @ mymaster 172.30.2.184 6379
4389:X 12 Jul 2020 13:52:59.669 # +failover-state-reconf-slaves master mymaster 172.30.2.184 6379
4389:X 12 Jul 2020 13:52:59.738 * +slave-reconf-sent slave 172.30.2.184:6381 172.30.2.184 6381 @ mymaster 172.30.2.184 6379
4387:S 12 Jul 2020 13:52:59.738 * REPLICAOF 172.30.2.184:6380 enabled (user request from 'id=6 addr=172.30.2.184:51888 fd=12 name=sentinel-9a727993-cmd age=152 idle=0 flags=x db=0 sub=0 psub=0 multi=3 qbuf=291 qbuf-free=32477 obl=36 oll=0 omem=0 events=r cmd=exec')
4387:S 12 Jul 2020 13:52:59.739 # CONFIG REWRITE executed with success.
4388:X 12 Jul 2020 13:52:59.739 # +config-update-from sentinel 9a72799337b637763343e01e971a580d51c7d4bf 172.30.2.184 26380 @ mymaster 172.30.2.184 6379
4388:X 12 Jul 2020 13:52:59.739 # +switch-master mymaster 172.30.2.184 6379 172.30.2.184 6380
4388:X 12 Jul 2020 13:52:59.740 * +slave slave 172.30.2.184:6381 172.30.2.184 6381 @ mymaster 172.30.2.184 6380
4388:X 12 Jul 2020 13:52:59.740 * +slave slave 172.30.2.184:6379 172.30.2.184 6379 @ mymaster 172.30.2.184 6380
4389:X 12 Jul 2020 13:52:59.881 # -odown master mymaster 172.30.2.184 6379
4387:S 12 Jul 2020 13:52:59.990 * Connecting to MASTER 172.30.2.184:6380
4387:S 12 Jul 2020 13:52:59.990 * MASTER <-> REPLICA sync started
4387:S 12 Jul 2020 13:52:59.990 * Non blocking connect for SYNC fired the event.
4387:S 12 Jul 2020 13:52:59.990 * Master replied to PING, replication can continue...
4387:S 12 Jul 2020 13:52:59.990 * Trying a partial resynchronization (request 09763946986da00e53793f0e69a0650277b1840c:19402).
4386:M 12 Jul 2020 13:52:59.990 * Replica 172.30.2.184:6381 asks for synchronization
4386:M 12 Jul 2020 13:52:59.990 * Partial resynchronization request from 172.30.2.184:6381 accepted. Sending 440 bytes of backlog starting from offset 19402.
4387:S 12 Jul 2020 13:52:59.990 * Successful partial resynchronization with master.
4387:S 12 Jul 2020 13:52:59.990 # Master replication ID changed to a1b501dea123c55bc1e1bb992e3a5cb6d52ddb57
4387:S 12 Jul 2020 13:52:59.990 * MASTER <-> REPLICA sync: Master accepted a Partial Resynchronization.
4389:X 12 Jul 2020 13:53:00.679 * +slave-reconf-inprog slave 172.30.2.184:6381 172.30.2.184 6381 @ mymaster 172.30.2.184 6379
4389:X 12 Jul 2020 13:53:00.679 * +slave-reconf-done slave 172.30.2.184:6381 172.30.2.184 6381 @ mymaster 172.30.2.184 6379
4389:X 12 Jul 2020 13:53:00.754 # +failover-end master mymaster 172.30.2.184 6379
4389:X 12 Jul 2020 13:53:00.754 # +switch-master mymaster 172.30.2.184 6379 172.30.2.184 6380
4389:X 12 Jul 2020 13:53:00.755 * +slave slave 172.30.2.184:6381 172.30.2.184 6381 @ mymaster 172.30.2.184 6380
4389:X 12 Jul 2020 13:53:00.755 * +slave slave 172.30.2.184:6379 172.30.2.184 6379 @ mymaster 172.30.2.184 6380
4388:X 12 Jul 2020 13:53:09.804 # +sdown slave 172.30.2.184:6379 172.30.2.184 6379 @ mymaster 172.30.2.184 6380
4389:X 12 Jul 2020 13:53:10.770 # +sdown slave 172.30.2.184:6379 172.30.2.184 6379 @ mymaster 172.30.2.184 6380

[ec2-user@ip-172-30-2-184 redis-5.0.8]$

1秒

src/redis-cli shutdown
4586:M 12 Jul 2020 14:03:42.757 # User requested shutdown...
4586:M 12 Jul 2020 14:03:42.757 * Saving the final RDB snapshot before exiting.
4586:M 12 Jul 2020 14:03:42.758 * DB saved on disk
4586:M 12 Jul 2020 14:03:42.758 * Removing the pid file.
4586:M 12 Jul 2020 14:03:42.758 # Redis is now ready to exit, bye bye...
4588:S 12 Jul 2020 14:03:42.759 # Connection with master lost.
4588:S 12 Jul 2020 14:03:42.759 * Caching the disconnected master state.
4587:S 12 Jul 2020 14:03:42.759 # Connection with master lost.
4587:S 12 Jul 2020 14:03:42.759 * Caching the disconnected master state.
[ec2-user@ip-172-30-2-184 redis-5.0.8]$ 4590:X 12 Jul 2020 14:03:42.843 # +sdown master mymaster 172.30.2.184 6379
4589:X 12 Jul 2020 14:03:42.863 # +sdown master mymaster 172.30.2.184 6379
4587:S 12 Jul 2020 14:03:42.874 * Connecting to MASTER 172.30.2.184:6379
4587:S 12 Jul 2020 14:03:42.874 * MASTER <-> REPLICA sync started
4587:S 12 Jul 2020 14:03:42.874 # Error condition on socket for SYNC: Connection refused
4588:S 12 Jul 2020 14:03:42.875 * Connecting to MASTER 172.30.2.184:6379
4588:S 12 Jul 2020 14:03:42.875 * MASTER <-> REPLICA sync started
4588:S 12 Jul 2020 14:03:42.875 # Error condition on socket for SYNC: Connection refused
4589:X 12 Jul 2020 14:03:42.963 # +odown master mymaster 172.30.2.184 6379 #quorum 2/2
4589:X 12 Jul 2020 14:03:42.963 # +new-epoch 1
4589:X 12 Jul 2020 14:03:42.963 # +try-failover master mymaster 172.30.2.184 6379
4589:X 12 Jul 2020 14:03:42.965 # +vote-for-leader 9a72799337b637763343e01e971a580d51c7d4be 1
4590:X 12 Jul 2020 14:03:42.967 # +new-epoch 1
4590:X 12 Jul 2020 14:03:42.968 # +vote-for-leader 9a72799337b637763343e01e971a580d51c7d4be 1
4589:X 12 Jul 2020 14:03:42.968 # 9a72799337b637763343e01e971a580d51c7d4bf voted for 9a72799337b637763343e01e971a580d51c7d4be 1
4589:X 12 Jul 2020 14:03:43.027 # +elected-leader master mymaster 172.30.2.184 6379
4589:X 12 Jul 2020 14:03:43.027 # +failover-state-select-slave master mymaster 172.30.2.184 6379
4589:X 12 Jul 2020 14:03:43.127 # +selected-slave slave 172.30.2.184:6380 172.30.2.184 6380 @ mymaster 172.30.2.184 6379
4589:X 12 Jul 2020 14:03:43.127 * +failover-state-send-slaveof-noone slave 172.30.2.184:6380 172.30.2.184 6380 @ mymaster 172.30.2.184 6379
4589:X 12 Jul 2020 14:03:43.227 * +failover-state-wait-promotion slave 172.30.2.184:6380 172.30.2.184 6380 @ mymaster 172.30.2.184 6379
4587:M 12 Jul 2020 14:03:43.227 # Setting secondary replication ID to fbc14afa8e4edbac6ef51b8c05d04ad7ba822ba1, valid up to offset: 13772. New replication ID is 59f3c39228e700cc9ef8d7d8e8af625f54704301
4587:M 12 Jul 2020 14:03:43.227 * Discarding previously cached master state.
4587:M 12 Jul 2020 14:03:43.227 * MASTER MODE enabled (user request from 'id=12 addr=172.30.2.184:53188 fd=9 name=sentinel-9a727993-cmd age=25 idle=0 flags=x db=0 sub=0 psub=0 multi=3 qbuf=154 qbuf-free=32614 obl=36 oll=0 omem=0 events=r cmd=exec')
4587:M 12 Jul 2020 14:03:43.228 # CONFIG REWRITE executed with success.
4588:S 12 Jul 2020 14:03:43.881 * Connecting to MASTER 172.30.2.184:6379
4588:S 12 Jul 2020 14:03:43.881 * MASTER <-> REPLICA sync started
4588:S 12 Jul 2020 14:03:43.881 # Error condition on socket for SYNC: Connection refused
4590:X 12 Jul 2020 14:03:43.971 # +odown master mymaster 172.30.2.184 6379 #quorum 2/2
4590:X 12 Jul 2020 14:03:43.971 # Next failover delay: I will not start a failover before Sun Jul 12 14:09:43 2020
4589:X 12 Jul 2020 14:03:44.007 # +promoted-slave slave 172.30.2.184:6380 172.30.2.184 6380 @ mymaster 172.30.2.184 6379
4589:X 12 Jul 2020 14:03:44.007 # +failover-state-reconf-slaves master mymaster 172.30.2.184 6379
4589:X 12 Jul 2020 14:03:44.088 * +slave-reconf-sent slave 172.30.2.184:6381 172.30.2.184 6381 @ mymaster 172.30.2.184 6379
4588:S 12 Jul 2020 14:03:44.088 * REPLICAOF 172.30.2.184:6380 enabled (user request from 'id=10 addr=172.30.2.184:55490 fd=9 name=sentinel-9a727993-cmd age=68 idle=0 flags=x db=0 sub=0 psub=0 multi=3 qbuf=305 qbuf-free=32463 obl=36 oll=0 omem=0 events=r cmd=exec')
4588:S 12 Jul 2020 14:03:44.089 # CONFIG REWRITE executed with success.
4590:X 12 Jul 2020 14:03:44.089 # +config-update-from sentinel 9a72799337b637763343e01e971a580d51c7d4be 172.30.2.184 26379 @ mymaster 172.30.2.184 6379
4590:X 12 Jul 2020 14:03:44.089 # +switch-master mymaster 172.30.2.184 6379 172.30.2.184 6380
4590:X 12 Jul 2020 14:03:44.089 * +slave slave 172.30.2.184:6381 172.30.2.184 6381 @ mymaster 172.30.2.184 6380
4590:X 12 Jul 2020 14:03:44.090 * +slave slave 172.30.2.184:6379 172.30.2.184 6379 @ mymaster 172.30.2.184 6380
4590:X 12 Jul 2020 14:03:44.118 # +sdown master mymaster 172.30.2.184 6380
4590:X 12 Jul 2020 14:03:44.119 # +odown master mymaster 172.30.2.184 6380 #quorum 2/2
4590:X 12 Jul 2020 14:03:44.119 # +new-epoch 2
4590:X 12 Jul 2020 14:03:44.119 # +try-failover master mymaster 172.30.2.184 6380
4590:X 12 Jul 2020 14:03:44.120 # +vote-for-leader 9a72799337b637763343e01e971a580d51c7d4bf 2
4590:X 12 Jul 2020 14:03:44.121 # +sdown slave 172.30.2.184:6379 172.30.2.184 6379 @ mymaster 172.30.2.184 6380
4590:X 12 Jul 2020 14:03:44.121 # +sdown slave 172.30.2.184:6381 172.30.2.184 6381 @ mymaster 172.30.2.184 6380
4590:X 12 Jul 2020 14:03:44.180 # -odown master mymaster 172.30.2.184 6380
4590:X 12 Jul 2020 14:03:44.180 # -sdown slave 172.30.2.184:6381 172.30.2.184 6381 @ mymaster 172.30.2.184 6380
4589:X 12 Jul 2020 14:03:44.453 # +new-epoch 2
4588:S 12 Jul 2020 14:03:44.883 * Connecting to MASTER 172.30.2.184:6380
4588:S 12 Jul 2020 14:03:44.883 * MASTER <-> REPLICA sync started
4588:S 12 Jul 2020 14:03:44.883 * Non blocking connect for SYNC fired the event.
4588:S 12 Jul 2020 14:03:44.883 * Master replied to PING, replication can continue...
4588:S 12 Jul 2020 14:03:44.884 * Trying a partial resynchronization (request fbc14afa8e4edbac6ef51b8c05d04ad7ba822ba1:13772).
4587:M 12 Jul 2020 14:03:44.884 * Replica 172.30.2.184:6381 asks for synchronization
4587:M 12 Jul 2020 14:03:44.884 * Partial resynchronization request from 172.30.2.184:6381 accepted. Sending 301 bytes of backlog starting from offset 13772.
4588:S 12 Jul 2020 14:03:44.884 * Successful partial resynchronization with master.
4588:S 12 Jul 2020 14:03:44.884 # Master replication ID changed to 59f3c39228e700cc9ef8d7d8e8af625f54704301
4588:S 12 Jul 2020 14:03:44.884 * MASTER <-> REPLICA sync: Master accepted a Partial Resynchronization.
4590:X 12 Jul 2020 14:03:44.915 # -sdown master mymaster 172.30.2.184 6380
4589:X 12 Jul 2020 14:03:45.091 * +slave-reconf-inprog slave 172.30.2.184:6381 172.30.2.184 6381 @ mymaster 172.30.2.184 6379
4589:X 12 Jul 2020 14:03:45.091 * +slave-reconf-done slave 172.30.2.184:6381 172.30.2.184 6381 @ mymaster 172.30.2.184 6379
4589:X 12 Jul 2020 14:03:45.157 # +failover-end master mymaster 172.30.2.184 6379
4589:X 12 Jul 2020 14:03:45.157 # +sdown sentinel 9a72799337b637763343e01e971a580d51c7d4bf 172.30.2.184 26380 @ mymaster 172.30.2.184 6379
4589:X 12 Jul 2020 14:03:45.157 # +switch-master mymaster 172.30.2.184 6379 172.30.2.184 6380
4589:X 12 Jul 2020 14:03:45.157 * +slave slave 172.30.2.184:6381 172.30.2.184 6381 @ mymaster 172.30.2.184 6380
4589:X 12 Jul 2020 14:03:45.158 * +slave slave 172.30.2.184:6379 172.30.2.184 6379 @ mymaster 172.30.2.184 6380
4589:X 12 Jul 2020 14:03:45.242 # +sdown master mymaster 172.30.2.184 6380
4589:X 12 Jul 2020 14:03:45.242 # +odown master mymaster 172.30.2.184 6380 #quorum 2/2
4589:X 12 Jul 2020 14:03:45.242 # +new-epoch 3
4589:X 12 Jul 2020 14:03:45.242 # +try-failover master mymaster 172.30.2.184 6380
4589:X 12 Jul 2020 14:03:45.244 # +vote-for-leader 9a72799337b637763343e01e971a580d51c7d4be 3
4589:X 12 Jul 2020 14:03:45.244 # +sdown slave 172.30.2.184:6381 172.30.2.184 6381 @ mymaster 172.30.2.184 6380
4589:X 12 Jul 2020 14:03:45.244 # +sdown slave 172.30.2.184:6379 172.30.2.184 6379 @ mymaster 172.30.2.184 6380
4589:X 12 Jul 2020 14:03:45.244 # -sdown sentinel 9a72799337b637763343e01e971a580d51c7d4bf 172.30.2.184 26380 @ mymaster 172.30.2.184 6380
4590:X 12 Jul 2020 14:03:45.246 # +new-epoch 3
4590:X 12 Jul 2020 14:03:45.247 # +vote-for-leader 9a72799337b637763343e01e971a580d51c7d4be 3
4589:X 12 Jul 2020 14:03:45.247 # 9a72799337b637763343e01e971a580d51c7d4bf voted for 9a72799337b637763343e01e971a580d51c7d4be 3
4589:X 12 Jul 2020 14:03:45.310 # -odown master mymaster 172.30.2.184 6380
4589:X 12 Jul 2020 14:03:45.310 # +elected-leader master mymaster 172.30.2.184 6380
4589:X 12 Jul 2020 14:03:45.310 # +failover-state-select-slave master mymaster 172.30.2.184 6380
4589:X 12 Jul 2020 14:03:45.310 # -sdown slave 172.30.2.184:6381 172.30.2.184 6381 @ mymaster 172.30.2.184 6380
4589:X 12 Jul 2020 14:03:45.393 # +selected-slave slave 172.30.2.184:6381 172.30.2.184 6381 @ mymaster 172.30.2.184 6380
4589:X 12 Jul 2020 14:03:45.393 * +failover-state-send-slaveof-noone slave 172.30.2.184:6381 172.30.2.184 6381 @ mymaster 172.30.2.184 6380
4589:X 12 Jul 2020 14:03:45.477 * +failover-state-wait-promotion slave 172.30.2.184:6381 172.30.2.184 6381 @ mymaster 172.30.2.184 6380
4588:M 12 Jul 2020 14:03:45.477 # Setting secondary replication ID to 59f3c39228e700cc9ef8d7d8e8af625f54704301, valid up to offset: 14073. New replication ID is c8bdae796932d3bd3426ccebb62cae11a76abe89
4588:M 12 Jul 2020 14:03:45.477 # Connection with master lost.
4588:M 12 Jul 2020 14:03:45.477 * Caching the disconnected master state.
4588:M 12 Jul 2020 14:03:45.477 * Discarding previously cached master state.
4588:M 12 Jul 2020 14:03:45.477 * MASTER MODE enabled (user request from 'id=15 addr=172.30.2.184:55570 fd=9 name=sentinel-9a727993-cmd age=0 idle=0 flags=x db=0 sub=0 psub=0 multi=3 qbuf=154 qbuf-free=32614 obl=36 oll=0 omem=0 events=r cmd=exec')
4587:M 12 Jul 2020 14:03:45.478 # Connection with replica 172.30.2.184:6381 lost.
4588:M 12 Jul 2020 14:03:45.478 # CONFIG REWRITE executed with success.
4589:X 12 Jul 2020 14:03:46.041 # -sdown master mymaster 172.30.2.184 6380
4589:X 12 Jul 2020 14:03:46.250 # +promoted-slave slave 172.30.2.184:6381 172.30.2.184 6381 @ mymaster 172.30.2.184 6380
4589:X 12 Jul 2020 14:03:46.250 # +failover-state-reconf-slaves master mymaster 172.30.2.184 6380
4589:X 12 Jul 2020 14:03:46.303 # +failover-end master mymaster 172.30.2.184 6380
4589:X 12 Jul 2020 14:03:46.303 # +switch-master mymaster 172.30.2.184 6380 172.30.2.184 6381
4589:X 12 Jul 2020 14:03:46.303 * +slave slave 172.30.2.184:6379 172.30.2.184 6379 @ mymaster 172.30.2.184 6381
4589:X 12 Jul 2020 14:03:46.304 * +slave slave 172.30.2.184:6380 172.30.2.184 6380 @ mymaster 172.30.2.184 6381
4590:X 12 Jul 2020 14:03:46.305 # +config-update-from sentinel 9a72799337b637763343e01e971a580d51c7d4be 172.30.2.184 26379 @ mymaster 172.30.2.184 6380
4590:X 12 Jul 2020 14:03:46.305 # +switch-master mymaster 172.30.2.184 6380 172.30.2.184 6381
4590:X 12 Jul 2020 14:03:46.305 * +slave slave 172.30.2.184:6379 172.30.2.184 6379 @ mymaster 172.30.2.184 6381
4590:X 12 Jul 2020 14:03:46.306 * +slave slave 172.30.2.184:6380 172.30.2.184 6380 @ mymaster 172.30.2.184 6381
4590:X 12 Jul 2020 14:03:46.366 # +sdown master mymaster 172.30.2.184 6381
4590:X 12 Jul 2020 14:03:46.366 # +sdown slave 172.30.2.184:6380 172.30.2.184 6380 @ mymaster 172.30.2.184 6381
4590:X 12 Jul 2020 14:03:46.366 # +sdown slave 172.30.2.184:6379 172.30.2.184 6379 @ mymaster 172.30.2.184 6381
4589:X 12 Jul 2020 14:03:46.389 # +sdown master mymaster 172.30.2.184 6381
4589:X 12 Jul 2020 14:03:46.389 # +sdown slave 172.30.2.184:6380 172.30.2.184 6380 @ mymaster 172.30.2.184 6381
4589:X 12 Jul 2020 14:03:46.389 # +sdown slave 172.30.2.184:6379 172.30.2.184 6379 @ mymaster 172.30.2.184 6381
4589:X 12 Jul 2020 14:03:46.466 # +odown master mymaster 172.30.2.184 6381 #quorum 2/2
4589:X 12 Jul 2020 14:03:46.466 # +new-epoch 4
4589:X 12 Jul 2020 14:03:46.466 # +try-failover master mymaster 172.30.2.184 6381
4590:X 12 Jul 2020 14:03:46.467 # -sdown master mymaster 172.30.2.184 6381
4590:X 12 Jul 2020 14:03:46.467 # -sdown slave 172.30.2.184:6380 172.30.2.184 6380 @ mymaster 172.30.2.184 6381
4589:X 12 Jul 2020 14:03:46.467 # +vote-for-leader 9a72799337b637763343e01e971a580d51c7d4be 4
4589:X 12 Jul 2020 14:03:46.468 # -sdown slave 172.30.2.184:6380 172.30.2.184 6380 @ mymaster 172.30.2.184 6381
4590:X 12 Jul 2020 14:03:46.469 # +new-epoch 4
4590:X 12 Jul 2020 14:03:46.470 # +vote-for-leader 9a72799337b637763343e01e971a580d51c7d4be 4
4589:X 12 Jul 2020 14:03:46.470 # 9a72799337b637763343e01e971a580d51c7d4bf voted for 9a72799337b637763343e01e971a580d51c7d4be 4
4589:X 12 Jul 2020 14:03:46.521 # -odown master mymaster 172.30.2.184 6381
4589:X 12 Jul 2020 14:03:46.521 # +elected-leader master mymaster 172.30.2.184 6381
4589:X 12 Jul 2020 14:03:46.521 # +failover-state-select-slave master mymaster 172.30.2.184 6381
4589:X 12 Jul 2020 14:03:46.573 # +selected-slave slave 172.30.2.184:6380 172.30.2.184 6380 @ mymaster 172.30.2.184 6381
4589:X 12 Jul 2020 14:03:46.573 * +failover-state-send-slaveof-noone slave 172.30.2.184:6380 172.30.2.184 6380 @ mymaster 172.30.2.184 6381
4589:X 12 Jul 2020 14:03:46.625 * +failover-state-wait-promotion slave 172.30.2.184:6380 172.30.2.184 6380 @ mymaster 172.30.2.184 6381
4587:M 12 Jul 2020 14:03:46.626 # CONFIG REWRITE executed with success.
4590:X 12 Jul 2020 14:03:46.649 # +sdown slave 172.30.2.184:6380 172.30.2.184 6380 @ mymaster 172.30.2.184 6381
4589:X 12 Jul 2020 14:03:47.070 # -sdown master mymaster 172.30.2.184 6381
4590:X 12 Jul 2020 14:03:47.402 # -sdown slave 172.30.2.184:6380 172.30.2.184 6380 @ mymaster 172.30.2.184 6381
4589:X 12 Jul 2020 14:03:47.423 # +promoted-slave slave 172.30.2.184:6380 172.30.2.184 6380 @ mymaster 172.30.2.184 6381
4589:X 12 Jul 2020 14:03:47.423 # +failover-state-reconf-slaves master mymaster 172.30.2.184 6381
4589:X 12 Jul 2020 14:03:47.512 # +failover-end master mymaster 172.30.2.184 6381
4589:X 12 Jul 2020 14:03:47.512 # +switch-master mymaster 172.30.2.184 6381 172.30.2.184 6380
4589:X 12 Jul 2020 14:03:47.512 * +slave slave 172.30.2.184:6379 172.30.2.184 6379 @ mymaster 172.30.2.184 6380
4589:X 12 Jul 2020 14:03:47.512 * +slave slave 172.30.2.184:6381 172.30.2.184 6381 @ mymaster 172.30.2.184 6380
4590:X 12 Jul 2020 14:03:47.514 # +config-update-from sentinel 9a72799337b637763343e01e971a580d51c7d4be 172.30.2.184 26379 @ mymaster 172.30.2.184 6381
4590:X 12 Jul 2020 14:03:47.514 # +switch-master mymaster 172.30.2.184 6381 172.30.2.184 6380
4590:X 12 Jul 2020 14:03:47.514 * +slave slave 172.30.2.184:6379 172.30.2.184 6379 @ mymaster 172.30.2.184 6380
4590:X 12 Jul 2020 14:03:47.514 * +slave slave 172.30.2.184:6381 172.30.2.184 6381 @ mymaster 172.30.2.184 6380
4589:X 12 Jul 2020 14:03:47.580 # +sdown master mymaster 172.30.2.184 6380
4589:X 12 Jul 2020 14:03:47.580 # +sdown slave 172.30.2.184:6381 172.30.2.184 6381 @ mymaster 172.30.2.184 6380
4589:X 12 Jul 2020 14:03:47.580 # +sdown slave 172.30.2.184:6379 172.30.2.184 6379 @ mymaster 172.30.2.184 6380
4590:X 12 Jul 2020 14:03:47.594 # +sdown master mymaster 172.30.2.184 6380
4590:X 12 Jul 2020 14:03:47.594 # +sdown slave 172.30.2.184:6381 172.30.2.184 6381 @ mymaster 172.30.2.184 6380
4590:X 12 Jul 2020 14:03:47.594 # +sdown slave 172.30.2.184:6379 172.30.2.184 6379 @ mymaster 172.30.2.184 6380
4589:X 12 Jul 2020 14:03:47.633 # -sdown slave 172.30.2.184:6381 172.30.2.184 6381 @ mymaster 172.30.2.184 6380
4590:X 12 Jul 2020 14:03:47.656 # -sdown master mymaster 172.30.2.184 6380
4590:X 12 Jul 2020 14:03:47.656 # -sdown slave 172.30.2.184:6381 172.30.2.184 6381 @ mymaster 172.30.2.184 6380
4589:X 12 Jul 2020 14:03:48.117 # -sdown master mymaster 172.30.2.184 6380

[ec2-user@ip-172-30-2-184 redis-5.0.8]$
[ec2-user@ip-172-30-2-184 redis-5.0.8]$
[ec2-user@ip-172-30-2-184 redis-5.0.8]$ 4589:X 12 Jul 2020 14:03:57.594 * +convert-to-slave slave 172.30.2.184:6381 172.30.2.184 6381 @ mymaster 172.30.2.184 6380
4588:S 12 Jul 2020 14:03:57.594 * Before turning into a replica, using my master parameters to synthesize a cached master: I may be able to synchronize with the new master with just a partial transfer.
4588:S 12 Jul 2020 14:03:57.594 * REPLICAOF 172.30.2.184:6380 enabled (user request from 'id=22 addr=172.30.2.184:55626 fd=8 name=sentinel-9a727993-cmd age=10 idle=0 flags=x db=0 sub=0 psub=0 multi=3 qbuf=152 qbuf-free=32616 obl=36 oll=0 omem=0 events=r cmd=exec')
4588:S 12 Jul 2020 14:03:57.595 # CONFIG REWRITE executed with success.
4588:S 12 Jul 2020 14:03:57.911 * Connecting to MASTER 172.30.2.184:6380
4588:S 12 Jul 2020 14:03:57.911 * MASTER <-> REPLICA sync started
4588:S 12 Jul 2020 14:03:57.911 * Non blocking connect for SYNC fired the event.
4588:S 12 Jul 2020 14:03:57.911 * Master replied to PING, replication can continue...
4588:S 12 Jul 2020 14:03:57.911 * Trying a partial resynchronization (request c8bdae796932d3bd3426ccebb62cae11a76abe89:15625).
4587:M 12 Jul 2020 14:03:57.911 * Replica 172.30.2.184:6381 asks for synchronization
4587:M 12 Jul 2020 14:03:57.911 * Partial resynchronization not accepted: Replication ID mismatch (Replica asked for 'c8bdae796932d3bd3426ccebb62cae11a76abe89', my replication IDs are '59f3c39228e700cc9ef8d7d8e8af625f54704301' and 'fbc14afa8e4edbac6ef51b8c05d04ad7ba822ba1')
4587:M 12 Jul 2020 14:03:57.911 * Starting BGSAVE for SYNC with target: disk
4587:M 12 Jul 2020 14:03:57.912 * Background saving started by pid 4627
4588:S 12 Jul 2020 14:03:57.912 * Full resync from master: 59f3c39228e700cc9ef8d7d8e8af625f54704301:15601
4588:S 12 Jul 2020 14:03:57.912 * Discarding previously cached master state.
4627:C 12 Jul 2020 14:03:57.914 * DB saved on disk
4627:C 12 Jul 2020 14:03:57.914 * RDB: 0 MB of memory used by copy-on-write
4587:M 12 Jul 2020 14:03:58.012 * Background saving terminated with success
4587:M 12 Jul 2020 14:03:58.012 * Synchronization with replica 172.30.2.184:6381 succeeded
4588:S 12 Jul 2020 14:03:58.012 * MASTER <-> REPLICA sync: receiving 197 bytes from master
4588:S 12 Jul 2020 14:03:58.012 * MASTER <-> REPLICA sync: Flushing old data
4588:S 12 Jul 2020 14:03:58.012 * MASTER <-> REPLICA sync: Loading DB in memory
4588:S 12 Jul 2020 14:03:58.012 * MASTER <-> REPLICA sync: Finished with success

[ec2-user@ip-172-30-2-184 redis-5.0.8]$ 

1ミリ秒

failover前から下記のようなログが出続けていて不安定。

[ec2-user@ip-172-30-2-184 redis-5.0.8]$ 4589:X 12 Jul 2020 14:02:28.378 # +sdown master mymaster 172.30.2.184 6379
4589:X 12 Jul 2020 14:02:28.433 # -sdown master mymaster 172.30.2.184 6379
4589:X 12 Jul 2020 14:02:32.453 # +sdown sentinel 9a72799337b637763343e01e971a580d51c7d4bf 172.30.2.184 26380 @ mymaster 172.30.2.184 6379
4589:X 12 Jul 2020 14:02:32.544 # -sdown sentinel 9a72799337b637763343e01e971a580d51c7d4bf 172.30.2.184 26380 @ mymaster 172.30.2.184 6379
4589:X 12 Jul 2020 14:02:36.272 # +sdown slave 172.30.2.184:6380 172.30.2.184 6380 @ mymaster 172.30.2.184 6379
4589:X 12 Jul 2020 14:02:36.356 # -sdown slave 172.30.2.184:6380 172.30.2.184 6380 @ mymaster 172.30.2.184 6379
4589:X 12 Jul 2020 14:02:36.843 # +sdown slave 172.30.2.184:6381 172.30.2.184 6381 @ mymaster 172.30.2.184 6379
4589:X 12 Jul 2020 14:02:36.896 # -sdown slave 172.30.2.184:6381 172.30.2.184 6381 @ mymaster 172.30.2.184 6379
4590:X 12 Jul 2020 14:02:59.727 # +sdown master mymaster 172.30.2.184 6379
4590:X 12 Jul 2020 14:02:59.793 # -sdown master mymaster 172.30.2.184 6379
4589:X 12 Jul 2020 14:03:04.176 # +sdown master mymaster 172.30.2.184 6379
4589:X 12 Jul 2020 14:03:04.267 # -sdown master mymaster 172.30.2.184 6379
4590:X 12 Jul 2020 14:03:05.428 # +sdown slave 172.30.2.184:6380 172.30.2.184 6380 @ mymaster 172.30.2.184 6379
4590:X 12 Jul 2020 14:03:05.481 # -sdown slave 172.30.2.184:6380 172.30.2.184 6380 @ mymaster 172.30.2.184 6379

[ec2-user@ip-172-30-2-184 redis-5.0.8]$

0ミリ秒

Sentinelプロセスが起動しない。

[ec2-user@ip-172-30-2-184 redis-5.0.8]$ src/redis-sentinel ./sentinel1.conf

*** FATAL CONFIG FILE ERROR ***
Reading the configuration file, at line 108
>>> 'sentinel down-after-milliseconds mymaster 0'
negative or zero time parameter.
[ec2-user@ip-172-30-2-184 redis-5.0.8]$  
Why not register and get more from Qiita?
  1. We will deliver articles that match you
    By following users and tags, you can catch up information on technical fields that you are interested in as a whole
  2. you can read useful information later efficiently
    By "stocking" the articles you like, you can search right away