Qiita Teams that are logged in
You are not logged in to any team

Log in to Qiita Team
Community
OrganizationEventAdvent CalendarQiitadon (β)
Service
Qiita JobsQiita ZineQiita Blog
3
Help us understand the problem. What are the problem?

More than 5 years have passed since last update.

@yuukigoodman

クラスタにノードが出入りするときの挙動について

普段の挙動

Riakにデータがコミットされると、どのvnodeに格納するかが計算されて、そのあとに実行される。
複数台構成を前提として、Riakの前にLBを置いてアクセスを分散させている場合、どのノードにいつデータがコミットされるか分からない状態だ。
該当のデータを持っていない(持つべきでない)ノードは、持っているノードに問い合わせることで処理を実行する。

riak-admin cluster leave

今回は、devrelで作成した開発環境でleaveコマンドを試してみる。
本当は物理4台でriakクラスタを構成し、LANケーブルをハサミで切って確かめたかったが、時間もお金もなかったので…

➜  riak git:(a12d5a2) dev/dev1/bin/riak-admin member-status
================================= Membership ==================================
Status     Ring    Pending    Node
-------------------------------------------------------------------------------
valid      25.0%      --      'dev1@127.0.0.1'
valid      25.0%      --      'dev3@127.0.0.1'
valid      25.0%      --      'dev4@127.0.0.1'
valid      25.0%      --      'dev5@127.0.0.1'
-------------------------------------------------------------------------------
Valid:4 / Leaving:0 / Exiting:0 / Joining:0 / Down:0

このような4台のクラスタだとする。
で、dev4を操作し、クラスタから離脱させる。

➜  riak git:(a12d5a2) dev/dev4/bin/riak-admin cluster leave
Success: staged leave request for 'dev4@127.0.0.1'
###############################################################################
                         After cluster transition 1/2
###############################################################################

================================= Membership ==================================
Status     Ring    Pending    Node
-------------------------------------------------------------------------------
leaving    25.0%      0.0%    'dev4@127.0.0.1'
valid      25.0%     34.4%    'dev1@127.0.0.1'
valid      25.0%     32.8%    'dev3@127.0.0.1'
valid      25.0%     32.8%    'dev5@127.0.0.1'
-------------------------------------------------------------------------------
Valid:3 / Leaving:1 / Exiting:0 / Joining:0 / Down:0

###############################################################################
                         After cluster transition 2/2
###############################################################################

================================= Membership ==================================
Status     Ring    Pending    Node
-------------------------------------------------------------------------------
valid      34.4%      --      'dev1@127.0.0.1'
valid      32.8%      --      'dev3@127.0.0.1'
valid      32.8%      --      'dev5@127.0.0.1'
-------------------------------------------------------------------------------
Valid:3 / Leaving:0 / Exiting:0 / Joining:0 / Down:0

そして変更をコミットさせると、データが移動していくのがわかる。
大体1分くらいで、移動が完了する。

➜  riak git:(a12d5a2) dev/dev1/bin/riak-admin cluster commit
Cluster changes committed

➜  riak git:(a12d5a2) dev/dev1/bin/riak-admin member-status
================================= Membership ==================================
Status     Ring    Pending    Node
-------------------------------------------------------------------------------
valid      25.0%      --      'dev1@127.0.0.1'
valid      25.0%      --      'dev3@127.0.0.1'
valid      25.0%      --      'dev4@127.0.0.1'
valid      25.0%      --      'dev5@127.0.0.1'
-------------------------------------------------------------------------------
Valid:4 / Leaving:0 / Exiting:0 / Joining:0 / Down:0
➜  riak git:(a12d5a2) dev/dev1/bin/riak-admin cluster commit
Cluster changes committed
➜  riak git:(a12d5a2) dev/dev1/bin/riak-admin member-status
================================= Membership ==================================
Status     Ring    Pending    Node
-------------------------------------------------------------------------------
leaving    21.9%      0.0%    'dev4@127.0.0.1'
valid      23.4%     34.4%    'dev1@127.0.0.1'
valid      25.0%     32.8%    'dev3@127.0.0.1'
valid      29.7%     32.8%    'dev5@127.0.0.1'
-------------------------------------------------------------------------------
Valid:3 / Leaving:1 / Exiting:0 / Joining:0 / Down:0
➜  riak git:(a12d5a2) dev/dev1/bin/riak-admin member-status
================================= Membership ==================================
Status     Ring    Pending    Node
-------------------------------------------------------------------------------
leaving    14.1%      0.0%    'dev4@127.0.0.1'
valid      25.0%     34.4%    'dev1@127.0.0.1'
valid      28.1%     32.8%    'dev3@127.0.0.1'
valid      32.8%     32.8%    'dev5@127.0.0.1'
-------------------------------------------------------------------------------
Valid:3 / Leaving:1 / Exiting:0 / Joining:0 / Down:0
➜  riak git:(a12d5a2) dev/dev1/bin/riak-admin member-status
================================= Membership ==================================
Status     Ring    Pending    Node
-------------------------------------------------------------------------------
leaving     9.4%      0.0%    'dev4@127.0.0.1'
valid      28.1%     34.4%    'dev1@127.0.0.1'
valid      29.7%     32.8%    'dev3@127.0.0.1'
valid      32.8%     32.8%    'dev5@127.0.0.1'
-------------------------------------------------------------------------------
Valid:3 / Leaving:1 / Exiting:0 / Joining:0 / Down:0
➜  riak git:(a12d5a2) dev/dev1/bin/riak-admin member-status
================================= Membership ==================================
Status     Ring    Pending    Node
-------------------------------------------------------------------------------
leaving     0.0%      --      'dev4@127.0.0.1'
valid      34.4%      --      'dev1@127.0.0.1'
valid      32.8%      --      'dev3@127.0.0.1'
valid      32.8%      --      'dev5@127.0.0.1'
-------------------------------------------------------------------------------
Valid:3 / Leaving:1 / Exiting:0 / Joining:0 / Down:0

このように、Leaveコマンドを使った場合は段階的にそのノードが持っているデータが他のクラスタに移されることがわかる。

追加したとき

追加するノードが動いていることを確認したら、クラスタにplanで確認してcommitsする。

➜  riak git:(a12d5a2) dev/dev4/bin/riak-admin cluster join dev1@127.0.0.1
Success: staged join request for 'dev4@127.0.0.1' to 'dev1@127.0.0.1'
➜  riak git:(a12d5a2) dev/dev1/bin/riak-admin cluster plan
=============================== Staged Changes ================================
Action         Details(s)
-------------------------------------------------------------------------------
join           'dev4@127.0.0.1'
-------------------------------------------------------------------------------


NOTE: Applying these changes will result in 1 cluster transition

###############################################################################
                         After cluster transition 1/1
###############################################################################

================================= Membership ==================================
Status     Ring    Pending    Node
-------------------------------------------------------------------------------
valid      34.4%     25.0%    'dev1@127.0.0.1'
valid      32.8%     25.0%    'dev3@127.0.0.1'
valid       0.0%     25.0%    'dev4@127.0.0.1'
valid      32.8%     25.0%    'dev5@127.0.0.1'
-------------------------------------------------------------------------------
Valid:4 / Leaving:0 / Exiting:0 / Joining:0 / Down:0

Transfers resulting from cluster changes: 47
  5 transfers from 'dev3@127.0.0.1' to 'dev4@127.0.0.1'
  5 transfers from 'dev1@127.0.0.1' to 'dev3@127.0.0.1'
  5 transfers from 'dev5@127.0.0.1' to 'dev1@127.0.0.1'
  5 transfers from 'dev3@127.0.0.1' to 'dev5@127.0.0.1'
  5 transfers from 'dev1@127.0.0.1' to 'dev4@127.0.0.1'
  5 transfers from 'dev5@127.0.0.1' to 'dev3@127.0.0.1'
  5 transfers from 'dev3@127.0.0.1' to 'dev1@127.0.0.1'
  6 transfers from 'dev1@127.0.0.1' to 'dev5@127.0.0.1'
  6 transfers from 'dev5@127.0.0.1' to 'dev4@127.0.0.1'

➜  riak git:(a12d5a2) dev/dev1/bin/riak-admin cluster commit
Cluster changes committed
➜  riak git:(a12d5a2) dev/dev1/bin/riak-admin member-status
================================= Membership ==================================
Status     Ring    Pending    Node
-------------------------------------------------------------------------------
valid      34.4%     25.0%    'dev1@127.0.0.1'
valid      32.8%     25.0%    'dev3@127.0.0.1'
valid       0.0%     25.0%    'dev4@127.0.0.1'
valid      32.8%     25.0%    'dev5@127.0.0.1'
-------------------------------------------------------------------------------
Valid:4 / Leaving:0 / Exiting:0 / Joining:0 / Down:0

あとは先ほどと同様の挙動を示す。

➜  riak git:(a12d5a2) dev/dev1/bin/riak-admin cluster commit
Cluster changes committed
➜  riak git:(a12d5a2) dev/dev1/bin/riak-admin member-status
================================= Membership ==================================
Status     Ring    Pending    Node
-------------------------------------------------------------------------------
valid      34.4%     25.0%    'dev1@127.0.0.1'
valid      32.8%     25.0%    'dev3@127.0.0.1'
valid       0.0%     25.0%    'dev4@127.0.0.1'
valid      32.8%     25.0%    'dev5@127.0.0.1'
-------------------------------------------------------------------------------
Valid:4 / Leaving:0 / Exiting:0 / Joining:0 / Down:0
➜  riak git:(a12d5a2) dev/dev1/bin/riak-admin member-status
================================= Membership ==================================
Status     Ring    Pending    Node
-------------------------------------------------------------------------------
valid      31.3%     25.0%    'dev1@127.0.0.1'
valid      32.8%     25.0%    'dev3@127.0.0.1'
valid       3.1%     25.0%    'dev4@127.0.0.1'
valid      32.8%     25.0%    'dev5@127.0.0.1'
-------------------------------------------------------------------------------
Valid:4 / Leaving:0 / Exiting:0 / Joining:0 / Down:0
➜  riak git:(a12d5a2) dev/dev1/bin/riak-admin member-status
================================= Membership ==================================
Status     Ring    Pending    Node
-------------------------------------------------------------------------------
valid      25.0%      --      'dev1@127.0.0.1'
valid      25.0%      --      'dev3@127.0.0.1'
valid      25.0%      --      'dev4@127.0.0.1'
valid      25.0%      --      'dev5@127.0.0.1'
-------------------------------------------------------------------------------
Valid:4 / Leaving:0 / Exiting:0 / Joining:0 / Down:0
Why not register and get more from Qiita?
  1. We will deliver articles that match you
    By following users and tags, you can catch up information on technical fields that you are interested in as a whole
  2. you can read useful information later efficiently
    By "stocking" the articles you like, you can search right away
3
Help us understand the problem. What are the problem?