課題
ReplicaSetでread_secondary => tureにしている時、読みに行くmongoはランダムに決定される。
これが失敗するとMongo::ConnectionFailure
が発生するのだが、デフォルトだとそのままリトライせずにエラーとなる。
failover復旧時やindex作成時などsecondaryがblock状態の時にも、何度かリトライするようにしたい。
※注: background => trueで実行したときprimaryは確かにbackground実行になっているが、secondaryはblockされる
解決
mongoid.ymlにmax_retries_on_connection_failure
オプションを追加する。
production:
database: myapp
hosts: [[100.100.100.100, 27017],
[200.200.200.200, 27017],
[300.300.300.300, 27017]]
pool_size: 99
read_secondary: true
max_retries_on_connection_failure: 5
参考
Connection Failures
Imagine that either the master node or one of the read nodes goes offline. How will the driver respond?
If any read operation fails, the driver will raise a
ConnectionFailure
exception. It then becomes the client's responsibility to decide how to handle this.If the client decides to retry, it's not guaranteed that another member of the replica set will have been promoted to master right away, so it's still possible that the driver will raise another
ConnectionFailure
. However, once a member has been promoted to master, typically within a few seconds, subsequent operations will succeed. Note that this does not prevent exception in the event of a primary failover.The driver will essentially cycle through all known seed addresses until a node identifies itself as master.
If you would also like Mongoid to retry operations if a Mongo::ConnectionFailure occurs you may specify this option in your config. Mongoid will retry the operation every half second up to the limit that is set. This defaults to 0.
max_retries_on_connection_failure: 3
Note that Mongoid does not handle reconnection at the time of this documentation, so you'll have to catch the connection errors and retry yourself in the meantime.
その他ソース
http://groups.google.com/group/mongoid/browse_thread/thread/700b5e0c5249baed?pli=1
http://stackoverflow.com/questions/10221198/mongooperationfailure-repeatedly-happening-in-ec2-what-to-do