Elasticsearch index,typeについて

何

• Elasticsearchのindexとtypeのところがわからなかったので調べました
• version5.5に基づいています。6以降は、概要の追記部分からキャッチアップしてください。

概要

• 検索すると見つかるindex=database, type=tableのDB用語による例えはかえってわかりにくかった、と言っている
• (https://www.elastic.co/blog/index-vs-type)
• 複数の構造をindexとtypeでどう使い分けるかのポイントは上に書いてある。要約すると間違えそうなので、各自で頑張って読もう。
• 構造をどう考えたらいいのか理解するために動かしてみる
• 2018/11/02追記）重要な参考情報：https://qiita.com/mkisono/items/4bcaedeced3b70312976
• 2020/03/31追記) version6以降のtypeについてhttps://www.elastic.co/jp/blog/removal-of-mapping-types-elasticsearch

環境

• AWS Elasticsearch service(Elasticsearch5.5)

実験開始

index作成

• この時はtypeもmappingもない

$ curl -X PUT https://search-my-es-xxx.ap-northeast-1.es.amazonaws.com/my_index?pretty
{
  "acknowledged" : true,
  "shards_acknowledged" : true
}

$ curl -X GET https://search-my-es-xxx.ap-northeast-1.es.amazonaws.com/my_index?pretty
{
  "my_index" : {
    "aliases" : { },
    "mappings" : { },
    "settings" : {
      "index" : {
        "creation_date" : "1508770567930",
        "number_of_shards" : "5",
        "number_of_replicas" : "1",
        "uuid" : "HkK-3dIlTNOw9rX_7tlHQw",
        "version" : {
          "created" : "5050299"
        },
        "provided_name" : "my_index"
      }
    }
  }
}

mapping作成

• この書き方は新規作成用だからエラー (index .. already exists..)

$ curl -XPUT https://search-my-es-xxx.ap-northeast-1.es.amazonaws.com/my_index -d '
{
    "mappings" : {
        "my_type1" : {
            "properties" : {
                "field1" : { "type" : "text" }
            }
        }
    }
}'

mapping作成

• これはOK

$ curl -XPUT https://search-my-es-xxx.ap-northeast-1.es.amazonaws.com/my_index/_mapping/my_type1 -d '
{
    "properties" : {
        "field1" : { "type" : "text" }
    }
}'

# typeが増えた
$ curl -X GET https://search-my-es-xxx.ap-northeast-1.es.amazonaws.com/my_index?pretty
{
  "my_index" : {
    "aliases" : { },
    "mappings" : {
      "my_type1" : {
        "properties" : {
          "field1" : {
            "type" : "text"
          }
        }
      }
    },
    "settings" : {
      "index" : {
        "creation_date" : "1508770567930",
        "number_of_shards" : "5",
        "number_of_replicas" : "1",
        "uuid" : "HkK-3dIlTNOw9rX_7tlHQw",
        "version" : {
          "created" : "5050299"
        },
        "provided_name" : "my_index"
      }
    }
  }
}

別のtypeに別のfield定義を追加

$ curl -XPUT https://search-my-es-xxx.ap-northeast-1.es.amazonaws.com/my_index/_mapping/my_type2 -d '
{
    "properties" : {
        "field2" : { "type" : "text" }
    }
}'

$ curl -X GET https://search-my-es-xxx.ap-northeast-1.es.amazonaws.com/my_index?pretty
{
  "my_index" : {
    "aliases" : { },
    "mappings" : {
      "my_type2" : {
        "properties" : {
          "field2" : {
            "type" : "text"
          }
        }
      },
      "my_type1" : {
        "properties" : {
          "field1" : {
            "type" : "text"
          }
        }
      }
    },
    "settings" : {
      "index" : {
        "creation_date" : "1508770567930",
        "number_of_shards" : "5",
        "number_of_replicas" : "1",
        "uuid" : "HkK-3dIlTNOw9rX_7tlHQw",
        "version" : {
          "created" : "5050299"
        },
        "provided_name" : "my_index"
      }
    }
  }
}

同じindexのtype違いで同名のfieldは同一の定義でないといけない

# my_type1で定義済みのfield1を別定義にしているのでこれはエラー(Mapper for [field1] conflicts with existing mapping)
$ curl -XPUT https://search-my-es-xxx.ap-northeast-1.es.amazonaws.com/my_index/_mapping/my_type2 -d '
{
    "properties" : {
        "field1" : {
            "type" : "text",
            "analyzer": "standard"
        }
    }
}'

# my_type1と同じ定義にする これはOK
$ curl -XPUT https://search-my-es-xxx.ap-northeast-1.es.amazonaws.com/my_index/_mapping/my_type2 -d '
{
    "properties" : {
        "field1" : {
            "type" : "text"
        }
    }
}'

$ curl -X GET https://search-my-es-xxx.ap-northeast-1.es.amazonaws.com/my_index?pretty
{
  "my_index" : {
    "aliases" : { },
    "mappings" : {
      "my_type2" : {
        "properties" : {
          "field1" : {
            "type" : "text"
          },
          "field2" : {
            "type" : "text"
          }
        }
      },
      "my_type1" : {
        "properties" : {
          "field1" : {
            "type" : "text"
          }
        }
      }
    },
    "settings" : {
      "index" : {
        "creation_date" : "1508770567930",
        "number_of_shards" : "5",
        "number_of_replicas" : "1",
        "uuid" : "HkK-3dIlTNOw9rX_7tlHQw",
        "version" : {
          "created" : "5050299"
        },
        "provided_name" : "my_index"
      }
    }
  }
}

document登録

• my_type1にfield2はないが自動で定義される

$ curl -XPUT https://search-my-es-xxx.ap-northeast-1.es.amazonaws.com/my_index/my_type1/1 -d '
{
    "field1": "aaa bbb",
    "field2": "aaa ccc"
}'

$ curl -XGET https://search-my-es-xxx.ap-northeast-1.es.amazonaws.com/my_index/_mapping/my_type1?pretty
{
  "my_index" : {
    "mappings" : {
      "my_type1" : {
        "dynamic" : "strict",
        "properties" : {
          "field1" : {
            "type" : "text"
          },
          "field2" : {
            "type" : "text",
            "fields" : {
              "keyword" : {
                "type" : "keyword",
                "ignore_above" : 256
              }
            }
          }
        }
      }
    }
  }
}

定義変更

• 自動mappingをしないようにする

$ curl -XPUT https://search-my-es-xxx.ap-northeast-1.es.amazonaws.com/my_index/_mapping/my_type1 -d '
{
    "dynamic" : "strict"
}'

# field3は未定のためエラー(mapping set to strict...)
$ curl -XPUT https://search-my-es-xxx.ap-northeast-1.es.amazonaws.com/my_index/my_type1/3 -d '
{
    "field1": "aaa bbb",
    "field2": "aaa ccc",
    "field3": "aaa ddd"
}'

結論

• indexは複数のtypeを持つ
• mappingはtypeごとに定義を持つ
• 同一indexのtype間で同名のpropertyは同一の定義のみ許可
• Documentは何らかのindex-typeに属する