WindowsでDockerを用いてRedisCluster

環境

項目	バージョン
OS	Windows 10
Redis-server	6.2.1
redis-cli	6.2.1
Docker Hub	3.1.0

構築手順

クラスタノード作成

クラスタ用の設定ファイル配置フォルダ作成

mkdir 7000 7001 7002 7003 7004 7005

redis.confをそれぞれのフォルダに配置

port 7000 # ここだけフォルダ名（ポート）と合わせる
cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 5000
appendonly yes

docker create network at CMD

クラスタを所属させるためのネットワーク作成

docker network create --driver bridge redis-network

redis cluster mode で起動 at CMD

docker run -d -v $PWD\7000:/usr/local/etc/redis -p 7000:7000 --net=redis-network --name myredis-7000 redis redis-server /usr/local/etc/redis/redis.conf
docker run -d -v $PWD\7001:/usr/local/etc/redis -p 7001:7001 --net=redis-network --name myredis-7001 redis redis-server /usr/local/etc/redis/redis.conf
docker run -d -v $PWD\7002:/usr/local/etc/redis -p 7002:7002 --net=redis-network --name myredis-7002 redis redis-server /usr/local/etc/redis/redis.conf
docker run -d -v $PWD\7003:/usr/local/etc/redis -p 7003:7003 --net=redis-network --name myredis-7003 redis redis-server /usr/local/etc/redis/redis.conf
docker run -d -v $PWD\7004:/usr/local/etc/redis -p 7004:7004 --net=redis-network --name myredis-7004 redis redis-server /usr/local/etc/redis/redis.conf
docker run -d -v $PWD\7005:/usr/local/etc/redis -p 7005:7005 --net=redis-network --name myredis-7005 redis redis-server /usr/local/etc/redis/redis.conf

redis cluster 作成

redis cluster 作成指示用コンテナ(redis-cli クライアント環境)を作成 at CMD

Windwosで使えるredis-cliは、MS製のものでv3のものしかないため、最新のRedisをCluster化できない。（少なくとも見つからなかったというか、redisはWindows公式サポートないっぽい。
なので、Dockerでもう一台Redis環境を用意し、このコンテナを操作クライアントとして、各ノードを束ねてクラスター化する。

docker run -d --name myredis-cluster --net=redis-network -d redis

Docker Hub よりbashコンソールへ

CLIボタンを押下して、bashコンソールへ入る

update, install network tools at Docker Bash

これ要らないかも。一応これで、他コンテナへpingできるようになる。

apt-get update
apt-get install iputils-ping net-tools dnsutils

redis-networkの確認 at CMD

docker network inspect redis-network

実行結果

[
    {
        "Name": "redis-network",
        "Id": "295279efbf106260495527f053c212233e1f8168ae8ba6a71210b5cb601f5747",
        "Created": "2021-04-03T07:49:53.1882328Z",
        "Scope": "local",
        "Driver": "bridge",
        "EnableIPv6": false,
        "IPAM": {
            "Driver": "default",
            "Options": {},
            "Config": [
                {
                    "Subnet": "172.18.0.0/16",
                    "Gateway": "172.18.0.1"
                }
            ]
        },
        "Internal": false,
        "Attachable": false,
        "Ingress": false,
        "ConfigFrom": {
            "Network": ""
        },
        "ConfigOnly": false,
        "Containers": {
            "0a9e8a5e1b37f4e63ead66a5d698f6fa37aa317aa5dc13e972b048896c1d812f": {
                "Name": "myredis-7000",
                "EndpointID": "8f239705f7bea8a5e7422f51dc8869731a9f14d110524b98c53a3b35a51e69d5",
                "MacAddress": "02:42:ac:12:00:02",
                "IPv4Address": "172.18.0.2/16",
                "IPv6Address": ""
            },
            "3e0b146e38e830045a2bbfb5ca29549f5f14f8dc4be9cf5194e57685e6d86fd5": {
                "Name": "myredis-7005",
                "EndpointID": "e91e21febdc18553fb98bba7777bee2612005c9f9e961a04f1759619ea3c23a7",
                "MacAddress": "02:42:ac:12:00:07",
                "IPv4Address": "172.18.0.7/16",
                "IPv6Address": ""
            },
            "751f02e9d23fd3d1ee85c7721fccd6219e8af6416dcf467c492005adffcabdee": {
                "Name": "myredis-cluster",
                "EndpointID": "b7be844cf2b1e225c113b907cc4588b71723dec8fff3360a91826fa6e304fdde",
                "MacAddress": "02:42:ac:12:00:08",
                "IPv4Address": "172.18.0.8/16",
                "IPv6Address": ""
            },
            "aadca156eacd9b29d1781e3cb99e104c255978471830b71cad9bf52abc8237a7": {
                "Name": "myredis-7003",
                "EndpointID": "51efba56048e50a7f269b513d6eacb4c6c2902b358e13fde9c839d7406026c79",
                "MacAddress": "02:42:ac:12:00:05",
                "IPv4Address": "172.18.0.5/16",
                "IPv6Address": ""
            },
            "b9d7ce4adbd56638035eb4b137a39c16dbaf739e7f53d3565ba5cc17d8322d69": {
                "Name": "myredis-7002",
                "EndpointID": "fa3a8b093b91a2f3e349ca0afe05f71da904a0d24407de58e338bffd6b4b7049",
                "MacAddress": "02:42:ac:12:00:04",
                "IPv4Address": "172.18.0.4/16",
                "IPv6Address": ""
            },
            "d0c843bbd7f18aa4c109d278b72e9bf3a3a412da64e05885bb6a4d5fad256ff7": {
                "Name": "hungry_cannon",
                "EndpointID": "673c549d1a08766e35cf9c79e125c9cf3e49b8dd79586a0e1d593f28dd2300fd",
                "MacAddress": "02:42:ac:12:00:09",
                "IPv4Address": "172.18.0.9/16",
                "IPv6Address": ""
            },
            "e9e27c9470eec229f7192615cb51e975490f5edffe78c92cdf0a654863d5d055": {
                "Name": "myredis-7001",
                "EndpointID": "fd06001d70918ae60ade0f88416be19733283ae8577fdf99eeb1af9a0c097889",
                "MacAddress": "02:42:ac:12:00:03",
                "IPv4Address": "172.18.0.3/16",
                "IPv6Address": ""
            },
            "ed7e7636fdfc96502139b6f72b78966ece0d4386b345113e0bb42e9e169cc31c": {
                "Name": "myredis-7004",
                "EndpointID": "9443253ef09aa72eff9be068aeb91ec11a748af64a7daa00143f98a7abfbc48e",
                "MacAddress": "02:42:ac:12:00:06",
                "IPv4Address": "172.18.0.6/16",
                "IPv6Address": ""
            }
        },
        "Options": {},
        "Labels": {}
    }
]

redis cluster 化のコマンド作成

上記の結果を用いて、Cluster化用コマンドを作成

redis-cli --cluster create 172.18.0.2:7000 172.18.0.3:7001 172.18.0.4:7002 172.18.0.5:7003 172.18.0.6:7004 172.18.0.7:7005 --cluster-replicas 1

myredis-clusterへ入り、クラスター化コマンドを実行 at Docker Bash

先ほど作成した、クラスター化指示用コンテナへ入り、上記のクラスター化コマンドを実行

実行結果

ログイメージ（上の回でやったものを取り忘れたので。基本は同じ）

>>> Performing hash slots allocation on 6 nodes...
Master[0] -> Slots 0 - 5460
Master[1] -> Slots 5461 - 10922
Master[2] -> Slots 10923 - 16383
Adding replica 172.17.0.7:7004 to 172.17.0.3:7000
Adding replica 172.17.0.8:7005 to 172.17.0.4:7001
Adding replica 172.17.0.6:7003 to 172.17.0.5:7002
M: 406a7d98f36e636b94c45ad0e60d37548d4de4c3 172.17.0.3:7000
   slots:[0-5460] (5461 slots) master
M: 33d1f3204acab84cad3b8138027a0d0bde746e86 172.17.0.4:7001
   slots:[5461-10922] (5462 slots) master
M: aa33ab09e2ccb434bad22aa39213c78a046bf184 172.17.0.5:7002
   slots:[10923-16383] (5461 slots) master
S: 53d3615478fb7df2aa0d473799e76df65688152a 172.17.0.6:7003
   replicates aa33ab09e2ccb434bad22aa39213c78a046bf184
S: 476450f1c55bc6e5f148a8304d748825ef76e149 172.17.0.7:7004
   replicates 406a7d98f36e636b94c45ad0e60d37548d4de4c3
S: 620f14cd5f1cabb71e6c34447cbc40177d74fdba 172.17.0.8:7005
   replicates 33d1f3204acab84cad3b8138027a0d0bde746e86
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join
.
>>> Performing Cluster Check (using node 172.17.0.3:7000)
M: 406a7d98f36e636b94c45ad0e60d37548d4de4c3 172.17.0.3:7000
   slots:[0-5460] (5461 slots) master
   1 additional replica(s)
M: 33d1f3204acab84cad3b8138027a0d0bde746e86 172.17.0.4:7001
   slots:[5461-10922] (5462 slots) master
   1 additional replica(s)
M: aa33ab09e2ccb434bad22aa39213c78a046bf184 172.17.0.5:7002
   slots:[10923-16383] (5461 slots) master
   1 additional replica(s)
S: 620f14cd5f1cabb71e6c34447cbc40177d74fdba 172.17.0.8:7005
   slots: (0 slots) slave
   replicates 33d1f3204acab84cad3b8138027a0d0bde746e86
S: 476450f1c55bc6e5f148a8304d748825ef76e149 172.17.0.7:7004
   slots: (0 slots) slave
   replicates 406a7d98f36e636b94c45ad0e60d37548d4de4c3
S: 53d3615478fb7df2aa0d473799e76df65688152a 172.17.0.6:7003
   slots: (0 slots) slave
   replicates aa33ab09e2ccb434bad22aa39213c78a046bf184
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

myredis-clsuter からRedis Cluster Nodes 確認 at Docker Bash

# redis-cli -h 172.18.0.7 -p 7005 cluster nodes

結果

master 3台, slave 3台であることがわかる。

4150775dda2b33c4b859ea121a757fc87c7d3e85 172.18.0.4:7002@17002 master - 0 1617465545010 18 connected 10923-16383
b398e6508a574e545e8adbfd1ffd74e5b025a93e 172.18.0.6:7004@17004 master - 0 1617465546013 16 connected 0-5460
238c00a365c3e9ef8acb6a6f3857e7ec7dc3b55c 172.18.0.7:7005@17005 myself,master - 0 1617465543000 14 connected 5461-10922
da467da010c24917e115be9db0c45135b704db97 172.18.0.3:7001@17001 slave 238c00a365c3e9ef8acb6a6f3857e7ec7dc3b55c 0 1617465546514 14 connected
7d62b7d57a5788374fe2571925fde685d80d1dc7 172.18.0.2:7000@17000 slave b398e6508a574e545e8adbfd1ffd74e5b025a93e 0 1617465545010 16 connected
a79b76ba663d28bd39b13a5954207a93778d2f86 172.18.0.5:7003@17003 slave 4150775dda2b33c4b859ea121a757fc87c7d3e85 0 1617465545000 18 connected

myredis-clusterから、Redis ノードへログイン at Docker Bash

redis-cli -c -h 172.18.0.7 -p 7005

Test at Redis on Docker Bash

172.18.0.7:7005> set test dekita
OK
172.18.0.7:7005> get test
"dekita"
172.18.0.7:7005>

フェイルオーバーテスト

事前状態 at Redis on Docker Bash

# redis-cli -h 172.18.0.7 -p 7005 cluster nodes
4150775dda2b33c4b859ea121a757fc87c7d3e85 172.18.0.4:7002@17002 master - 0 1617497582000 18 connected 10923-16383
b398e6508a574e545e8adbfd1ffd74e5b025a93e 172.18.0.6:7004@17004 slave 7d62b7d57a5788374fe2571925fde685d80d1dc7 0 1617497582847 20 connected
238c00a365c3e9ef8acb6a6f3857e7ec7dc3b55c 172.18.0.7:7005@17005 myself,slave da467da010c24917e115be9db0c45135b704db97 0 1617497583000 19 connected
da467da010c24917e115be9db0c45135b704db97 172.18.0.3:7001@17001 master - 0 1617497582000 19 connected 5461-10922
7d62b7d57a5788374fe2571925fde685d80d1dc7 172.18.0.2:7000@17000 master - 0 1617497583551 20 connected 0-5460
a79b76ba663d28bd39b13a5954207a93778d2f86 172.18.0.5:7003@17003 slave 4150775dda2b33c4b859ea121a757fc87c7d3e85 0 1617497582000 18 connected

master破壊 at Redis on Docker Bash

2つある模様
1. ノードを破壊する方法（redis-cli -h 172.18.0.4 -p 7002 debug segfault、プロセスkillなど）
2. Slaveノードをfailoverする方法（例：redis-cli -h 172.18.0.3 -p 7001 cluster failover takeover）

2の挙動はコマンド発行と同時に即座にmasterとslaveが切り替わっていた（あまりちゃんと理解していない）。
以下では、1で。

# redis-cli -h 172.18.0.4 -p 7002 debug segfault
Error: Server closed the connection

その後 at Redis on Docker Bash

• 7002がfailしている

# redis-cli -h 172.18.0.7 -p 7005 cluster nodes
4150775dda2b33c4b859ea121a757fc87c7d3e85 172.18.0.4:7002@17002 master,fail - 1617497606424 1617497603915 18 connected 10923-16383
b398e6508a574e545e8adbfd1ffd74e5b025a93e 172.18.0.6:7004@17004 slave 7d62b7d57a5788374fe2571925fde685d80d1dc7 0 1617497611538 20 connected
238c00a365c3e9ef8acb6a6f3857e7ec7dc3b55c 172.18.0.7:7005@17005 myself,slave da467da010c24917e115be9db0c45135b704db97 0 1617497610000 19 connected
da467da010c24917e115be9db0c45135b704db97 172.18.0.3:7001@17001 master - 0 1617497611940 19 connected 5461-10922
7d62b7d57a5788374fe2571925fde685d80d1dc7 172.18.0.2:7000@17000 master - 0 1617497610937 20 connected 0-5460
a79b76ba663d28bd39b13a5954207a93778d2f86 172.18.0.5:7003@17003 slave 4150775dda2b33c4b859ea121a757fc87c7d3e85 0 1617497610000 18 connected

しばらくすると

• 7003がmaster昇格

# redis-cli -h 172.18.0.7 -p 7005 cluster nodes
4150775dda2b33c4b859ea121a757fc87c7d3e85 172.18.0.4:7002@17002 master,fail - 1617497606424 1617497603915 18 connected
b398e6508a574e545e8adbfd1ffd74e5b025a93e 172.18.0.6:7004@17004 slave 7d62b7d57a5788374fe2571925fde685d80d1dc7 0 1617497773418 20 connected
238c00a365c3e9ef8acb6a6f3857e7ec7dc3b55c 172.18.0.7:7005@17005 myself,slave da467da010c24917e115be9db0c45135b704db97 0 1617497772000 19 connected
da467da010c24917e115be9db0c45135b704db97 172.18.0.3:7001@17001 master - 0 1617497772415 19 connected 5461-10922
7d62b7d57a5788374fe2571925fde685d80d1dc7 172.18.0.2:7000@17000 master - 0 1617497773418 20 connected 0-5460
a79b76ba663d28bd39b13a5954207a93778d2f86 172.18.0.5:7003@17003 master - 0 1617497772000 21 connected 10923-16383

破壊した7002を再起動

Docker Hubから再起動

再起動後

7002は復帰し、今度はslaveとして参加している。

# redis-cli -h 172.18.0.7 -p 7005 cluster nodes
4150775dda2b33c4b859ea121a757fc87c7d3e85 172.18.0.4:7002@17002 slave a79b76ba663d28bd39b13a5954207a93778d2f86 0 1617498276013 21 connected
b398e6508a574e545e8adbfd1ffd74e5b025a93e 172.18.0.6:7004@17004 slave 7d62b7d57a5788374fe2571925fde685d80d1dc7 0 1617498278017 20 connected
238c00a365c3e9ef8acb6a6f3857e7ec7dc3b55c 172.18.0.7:7005@17005 myself,slave da467da010c24917e115be9db0c45135b704db97 0 1617498276000 19 connected
da467da010c24917e115be9db0c45135b704db97 172.18.0.3:7001@17001 master - 0 1617498277516 19 connected 5461-10922
7d62b7d57a5788374fe2571925fde685d80d1dc7 172.18.0.2:7000@17000 master - 0 1617498277516 20 connected 0-5460
a79b76ba663d28bd39b13a5954207a93778d2f86 172.18.0.5:7003@17003 master - 0 1617498277015 21 connected 10923-16383

取得テスト at Redis on Docker Bash

#  redis-cli -c -h 172.18.0.5 -p 7003
172.18.0.5:7003> get test
-> Redirected to slot [6918] located at 172.18.0.3:7001
"dekita"
172.18.0.3:7001>