1
2

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?

More than 3 years have passed since last update.

MongoDBAdvent Calendar 2019

Day 17

MongoDB で getMore が発生するタイミング

Posted at

大量の結果を返す find()aggregate() に対して、 MongoDB は結果全件を一気に取得するのではなく、いくつかに分割して取得を行う。

getMore を使う条件は、以下の通り。

  • 初回は101ドキュメントを取得する
  • 未取得のドキュメントがある場合は getMore を使って1回につき16MBを超えない範囲でドキュメントを取得する

以下は101件の取得には getMore を使っていないが102件の取得になると getMore が使われる様子。
(これはおそらく MongoDB 3.4 以降)
cf. https://docs.mongodb.com/manual/tutorial/iterate-a-cursor/#cursor-batches

irb(main):001:0> Item.limit(101).map(&:name); nil
MONGODB | [7] localhost:28001 #1 | sandbox.find | STARTED | {"find"=>"items", "filter"=>{}, "limit"=>101, "lsid"=>{"id"=><BSON::Binary:0x70306682318860 type=uuid data=0xf2b72d35df7c417a...>}}
MONGODB | [7] localhost:28001 | sandbox.find | SUCCEEDED | 0.002s
=> nil
irb(main):002:0> Item.limit(102).map(&:name); nil
MONGODB | [8] localhost:28001 #1 | sandbox.find | STARTED | {"find"=>"items", "filter"=>{}, "limit"=>102, "lsid"=>{"id"=><BSON::Binary:0x70306682318860 type=uuid data=0xf2b72d35df7c417a...>}}
MONGODB | [8] localhost:28001 | sandbox.find | SUCCEEDED | 0.003s
MONGODB | [9] localhost:28001 #1 | sandbox.getMore | STARTED | {"getMore"=>#<BSON::Int64:0x00007fe3196c0f30 @value=8225986221740535645>, "collection"=>"items", "lsid"=>{"id"=><BSON::Binary:0x70306682318860 type=uuid data=0xf2b72d35df7c417a...>}}
MONGODB | [9] localhost:28001 | sandbox.getMore | SUCCEEDED | 0.002s
=> nil

ちなみに、 getMore が2回走るようになるタイミングを調べてみたところ、92914件取得したタイミングで2回目が実行されるようになった。
当然これは1件ごとのドキュメントのサイズに依存するため、この数値自体には意味はない。

irb(main):043:0> Item.limit(92913).map(&:name); nil
MONGODB | [94] localhost:28001 #3 | sandbox.find | STARTED | {"find"=>"items", "filter"=>{}, "limit"=>92913, "lsid"=>{"id"=><BSON::Binary:0x70306433028540 type=uuid data=0x085ec2a138c44df1...>}}
MONGODB | [94] localhost:28001 | sandbox.find | SUCCEEDED | 0.002s
MONGODB | [95] localhost:28001 #3 | sandbox.getMore | STARTED | {"getMore"=>#<BSON::Int64:0x00007fe33960dc58 @value=5867813388408376440>, "collection"=>"items", "lsid"=>{"id"=><BSON::Binary:0x70306433028540 type=uuid data=0x085ec2a138c44df1...>}}
MONGODB | [95] localhost:28001 | sandbox.getMore | SUCCEEDED | 0.621s
=> nil
irb(main):044:0> Item.limit(92914).map(&:name); nil
MONGODB | [96] localhost:28001 #3 | sandbox.find | STARTED | {"find"=>"items", "filter"=>{}, "limit"=>92914, "lsid"=>{"id"=><BSON::Binary:0x70306433028540 type=uuid data=0x085ec2a138c44df1...>}}
MONGODB | [96] localhost:28001 | sandbox.find | SUCCEEDED | 0.002s
MONGODB | [97] localhost:28001 #3 | sandbox.getMore | STARTED | {"getMore"=>#<BSON::Int64:0x00007fe2fb922dc8 @value=2566893090831268646>, "collection"=>"items", "lsid"=>{"id"=><BSON::Binary:0x70306433028540 type=uuid data=0x085ec2a138c44df1...>}}
MONGODB | [97] localhost:28001 | sandbox.getMore | SUCCEEDED | 0.533s
MONGODB | [98] localhost:28001 #3 | sandbox.getMore | STARTED | {"getMore"=>#<BSON::Int64:0x00007fe339546518 @value=2566893090831268646>, "collection"=>"items", "lsid"=>{"id"=><BSON::Binary:0x70306433028540 type=uuid data=0x085ec2a138c44df1...>}}
MONGODB | [98] localhost:28001 | sandbox.getMore | SUCCEEDED | 0.002s
=> nil

検証に使ったドキュメントは以下のような形。

num が数値(1件目から順にインクリメント)、 name が item_* の形で末尾に num と同じ数値(整数)、そして description にランダムな100bytesの文字列を格納している。

irb(main):045:0> Item.first
MONGODB | [99] localhost:28001 #3 | sandbox.find | STARTED | {"find"=>"items", "filter"=>{}, "sort"=>{"_id"=>1}, "limit"=>1, "singleBatch"=>true, "lsid"=>{"id"=><BSON::Binary:0x70306433028540 type=uuid data=0x085ec2a138c44df1...>}}
MONGODB | [99] localhost:28001 | sandbox.find | SUCCEEDED | 0.002s
=> #<Item _id: 5e38e59f4bc718a1495d8e4d, created_at: nil, updated_at: nil, num: 1.0, name: "item_1", description: "lbregl1u4pkk95yn2qzg22988jjlotgfendf41hdfuw68gg4f7dbos8tvf1qxcq2qpjvzqll7sl47cwsdt6y7hdewmz35wbu0174">
1
2
1

Register as a new user and use Qiita more conveniently

  1. You get articles that match your needs
  2. You can efficiently read back useful information
  3. You can use dark theme
What you can do with signing up
1
2

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?