Go to Qiita Advent Calendar 2024 Top
LoginSignup
35
27

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?

More than 5 years have passed since last update.

@nakamura-tsuyoshi(中村 剛)

elasticsearchのField datatypes(ArrayとNested)の使い方では検索結果が変わる

Posted at

はじめに

  • 色々とelasticsearctにデータを入れて検索をしていたけど、親子関係のあるデータを作って検索したら自分の勝手な期待値と異なる挙動をしてたのであたらめて学習した

elasticsearctのバージョン

elasticsearct2.1.1

ポイントとなるのはField datatypesで軽く整理

Array datatype

  • 0以上の値が必要
  • 配列の要素は全て同じ型になってる必要がある

an array of strings: [ "one", "two" ]
an array of integers: [ 1, 2 ]
an array of arrays: [ 1, [ 2, 3 ]] which is the equivalent of [ 1, 2, 3 ]
an array of objects: [ { "name": "Mary", "age": 12 }, { "name": "John", "age": 10 }]

参照）https://www.elastic.co/guide/en/elasticsearch/reference/2.x/array.html

Nested datatype

  • オブジェクトの配列であること
  • インナーオブジェクトは利用不可っぽい

参照）https://www.elastic.co/guide/en/elasticsearch/reference/2.x/nested.html#_how_arrays_of_objects_are_flattened

結論からまとめる

  1. Array datatypeはデータの親子関係が考慮されないので検索条件を複数設定した場合、結果が異なる
  2. Nested datatypeを利用するとデータの親子関係が維持されるのでRDBっぽく使える。んーgroup_byみたいなことができてaggregationが活躍する
  • aggregationは検索結果に対して集計ができる機能

実データを元づいて説明する

下記のようなテストデータを作ります

パターン１

  • ECをやっているショップAがあります
  • ショップAに商品A（500円）、商品B(800円)があります。

パターン２

  • ECをやっているショップBがあります
  • ショップBに商品A（500円）、商品B(800円)があります。

テンプレートファイル(Array datatype).設定

[root@test_elasticsearch ~]# curl -XPUT localhost:9200/_template/shop01 -d '
> {
>   "order": 0,
>   "template": "shop01-*",
>   "settings": {
>     "index": {
>       "number_of_shards": "1",
>       "number_of_replicas": "0"
>     }
>   },
>   "mappings": {
>     "shop01": {
>       "_source": {
>         "enabled": true
>       },
>       "_all": {
>         "analyzer": "ja",
>         "enabled": true
>       },
>       "properties": {
>         "title": {
>           "analyzer": "ja",
>           "type": "string"
>         },
>         "description": {
>           "analyzer": "ja",
>           "type": "string"
>         },
>         "products": {
>           "properties" : {
>             "shop_id": {
>               "type": "integer"
>             },
>             "item_id": {
>               "type": "integer"
>             },
>             "title": {
>               "analyzer": "ja",
>               "type": "string"
>             },
>             "description": {
>               "analyzer": "ja",
>               "type": "string"
>             },
>             "price": {
>               "type": "integer"
>             }
>           }
>         }
>       }
>     }
>   }
> }'

テンプレートファイル(Nested datatype).設定

[root@test_elasticsearch ~]# curl -XPUT localhost:9200/_template/shop02 -d '
> {
>   "order": 0,
>   "template": "shop02-*",
>   "settings": {
>     "index": {
>       "number_of_shards": "1",
>       "number_of_replicas": "0"
>     }                      
>   },
>   "mappings": {
>     "shop02": {
>       "_source": {
>         "enabled": true
>       },
>       "_all": {
>         "analyzer": "ja",
>         "enabled": true
>       },
>       "properties": {
>         "title": {
>           "analyzer": "ja",
>           "type": "string"
>         },
>         "description": {
>           "analyzer": "ja",
>           "type": "string"
>         },
>         "products": {
>           "type": "nested",
>           "properties" : {
>             "shop_id": {
>               "type": "integer"
>             },
>             "item_id": {
>               "type": "integer"
>             },
>             "title": {
>               "analyzer": "ja",
>               "type": "string"
>             },
>             "description": {
>               "analyzer": "ja",
>               "type": "string"
>             },
>             "price": {
>               "type": "integer"
>             }
>           }
>         }
>       }
>     }
>   }
> }'
{"acknowledged":true}[root@test_elasticsearch ~]#

Array datatypeへデータを投入

[root@test_elasticsearch ~]# curl -X POST http://localhost:9200/shop01-20160111/shop01/1  -d '
> {
>     "id" : 1,
>     "title" : "ショップA",
>     "description" : "東京都渋谷区道玄坂にあるお花屋さんです。",
>     "products" : [
>       {
>         "shop_id" : 1,
>         "item_id" : 1,
>         "title" : "商品A",
>         "description" : "商品Aの説明",
>         "price"       : 500
>       },
>       {
>         "shop_id": 1,
>         "item_id" : 2,
>         "title" : "商品B",
>         "description" : "商品Bの説明",
>         "price"       : 800
>       }
>     ]
> }'
{"_index":"shop01-20160111","_type":"shop01","_id":"1","_version":1,"_shards":{"total":1,"successful":1,"failed":0},"created":true}

[root@test_elasticsearch ~]# curl -X POST http://localhost:9200/shop01-20160111/shop01/2  -d '
> {
>     "title" : "ショップB",
>     "description" : "東京都赤坂にあるお花屋さんです。",
>     "products" : [
>       {
>         "shop_id" : 2,
>         "item_id" : 1,
>         "title" : "商品A",
>         "description" : "商品Aの説明",
>         "price"       : 500
>       },
>       {
>         "shop_id": 2,
>         "item_id" : 2,
>         "title" : "商品B",
>         "description" : "商品Bの説明",
>         "price"       : 800
>       }
>     ]
> }'
{"_index":"shop01-20160111","_type":"shop01","_id":"2","_version":1,"_shards":{"total":1,"successful":1,"failed":0},"created":true}

Nested datatypeへデータを投入

[root@test_elasticsearch ~]# curl -X POST http://localhost:9200/shop02-20160111/shop02/1  -d '
> {
>     "title" : "ショップA",
>     "description" : "東京都渋谷区道玄坂にあるお花屋さんです。",
>     "products" : [
>       {
>         "shop_id": 1,
>         "item_id" : 1,   
>         "title" : "商品A",    
>         "description" : "商品Aの説明",
>         "price"       : 500
>       },
>       {
>         "shop_id": 1,
>         "item_id" : 2,
>         "title" : "商品B",
>         "description" : "商品Bの説明",
>         "price"       : 800
>       }
>     ]
> }'
{"_index":"shop02-20160111","_type":"shop02","_id":"1","_version":1,"_shards":{"total":1,"successful":1,"failed":0},"created":true}



[root@test_elasticsearch ~]# curl -X POST http://localhost:9200/shop02-20160111/shop02/2  -d '
> {
>     "title" : "ショップB",
>     "description" : "東京都赤坂にあるお花屋さんです。",
>     "products" : [                                             
>       {           
>         "shop_id" : 2,
>         "item_id" : 1,
>         "title" : "商品A",
>         "description" : "商品Aの説明",
>         "price"       : 500
>       },                   
>       {
>         "shop_id": 2,
>         "item_id" : 2,
>         "title" : "商品B",
>         "description" : "商品Bの説明",
>         "price"       : 800
>       }                    
>     ]                
> }'
{"_index":"shop02-20160111","_type":"shop02","_id":"2","_version":1,"_shards":{"total":1,"successful":1,"failed":0},"created":true}

結論1の実例

Array datatypeへのクエリーを投げる

  • mustはANDの意味なのでitem_id=1 AND shop_id=1 AND price=800という条件になり、期待値は0件だけどヒットしてしまう。
[root@test_elasticsearch ~]# curl -XGET 'localhost:9200/shop01-20160111/shop01/_search?pretty' -d'
> {
>     "query": {
>         "bool": {
>             "must": [
>                 {"term": {"products.item_id": "1"}},
>                 {"term": {"products.shop_id": "1"}},
>                 {"term": {"products.price": 800}}
>             ]
>         }
>     }
> }'
{
  "took" : 1,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "failed" : 0
  },
  "hits" : {
    "total" : 1,
    "max_score" : 1.3065004,
    "hits" : [ {
      "_index" : "shop01-20160111",
      "_type" : "shop01",
      "_id" : "1",
      "_score" : 1.3065004,
      "_source":
{
    "id" : 1,
    "title" : "ショップA",
    "description" : "東京都渋谷区道玄坂にあるお花屋さんです。",
    "products" : [
      {
        "shop_id" : 1,
        "item_id" : 1,
        "title" : "商品A",
        "description" : "商品Aの説明",
        "price"       : 500
      },
      {
        "shop_id": 1,
        "item_id" : 2,
        "title" : "商品B",
        "description" : "商品Bの説明",
        "price"       : 800
      }
    ]
}
    } ]
  }
}

これは下記のようなデータのindexされ方をしている為だと思われる

{
    "id": "1",
    "products.shop_id": ["1"],
    "products.item_id": [1,2],
    "products.price": [500, 800]
}

Nested datatypeへのクエリーを投げる

  • mustはANDの意味なのでitem_id=1 AND shop_id=1 AND price=800という条件になり、期待値は0件で期待通り。
[root@test_elasticsearch ~]# curl -XGET 'localhost:9200/shop02-20160111/shop02/_search?pretty' -d'
> {
>   "query": {
>     "nested": {
>       "path": "products",
>       "query": {
>         "bool": {
>           "must": [
>                 {"term": {"products.item_id": "1"}},
>                 {"term": {"products.shop_id": "1"}},
>                 {"term": {"products.price": 800}}
>           ]
>         }
>       }
>     }
>   }
> }'
{
  "took" : 1,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "failed" : 0
  },
  "hits" : {
    "total" : 0,
    "max_score" : null,
    "hits" : [ ]
  }
}

ちゃんと親子関係が維持されて検索ができているので0件になる

結論2の実例

各ショップ毎に最安値を取得する場合、一発で取れる
shop_idをgroup byしてその中で最安値を取得する

[root@test_elasticsearch ~]# curl -XGET 'localhost:9200/shop02-20160111/shop02/_search?pretty' -d'
> {
>     "aggs": {
>         "aaa": {
>             "nested": {
>                 "path": "products"
>             },
>             "aggs": {
>                 "group_by_customer": {
>                     "terms": {
>                         "field": "products.shop_id"
>                     },
>                     "aggs": {
>                       "min_price" : { "min" : { "field" : "products.price" } }
>                     }
>                 }
>             }
>         }
>     }
> }'
...
  "aggregations" : {
    "aaa" : {
      "doc_count" : 4,
      "group_by_customer" : {
        "doc_count_error_upper_bound" : 0,
        "sum_other_doc_count" : 0,
        "buckets" : [ {
          "key" : 1,
          "doc_count" : 2,
          "min_price" : {
            "value" : 500.0
          }
        }, {
          "key" : 2,
          "doc_count" : 2,
          "min_price" : {
            "value" : 500.0
          }
        } ]
      }
    }
  }
}

感想とか

はじめに公式ドキュメントを読めって話かもしれないけど、色々と動くもので実現したい事を調べながら実装してみて、あれ思った挙動じゃないってなって、そこから色々と掘り調べる方法でこのtipsに辿りついた。

35
27
2

Register as a new user and use Qiita more conveniently

  1. You get articles that match your needs
  2. You can efficiently read back useful information
  3. You can use dark theme
What you can do with signing up
Sign upLogin
35
27

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?