Elasticsearch2とElasticsearch5で容量にどのような差があるのかパート1

Elasticsearch２と5で同じデータ量でサイズにどのくらい差があるのか、気になったので見てみる

手順

• dockerでElasticsearch２とElasticsearch5のイメージを作っておく
• 元になるデータを作っておく
• indexを作る
• bulk importしてElasticsearchにデータ突っ込む
• indexのサイズを見る

curl -X GET localhost:9200/photo_search/_stats/store?pretty=true

検証

10000件

• elasticsearch2

curl -X GET localhost:9200/photos/_stats/store?pretty=true
{
  "_shards" : {
    "total" : 10,
    "successful" : 5,
    "failed" : 0
  },
  "_all" : {
    "primaries" : {
      "store" : {
        "size_in_bytes" : 2394572,
        "throttle_time_in_millis" : 0
      }
    },
    "total" : {
      "store" : {
        "size_in_bytes" : 2394572,
        "throttle_time_in_millis" : 0
      }
    }
  },
  "indices" : {
    "photos" : {
      "primaries" : {
        "store" : {
          "size_in_bytes" : 2394572,
          "throttle_time_in_millis" : 0
        }
      },
      "total" : {
        "store" : {
          "size_in_bytes" : 2394572,
          "throttle_time_in_millis" : 0
        }
      }
    }
  }
}

• elasticsearch5

 curl -X GET localhost:9200/photos/_stats/store?pretty=true
{
  "_shards" : {
    "total" : 10,
    "successful" : 5,
    "failed" : 0
  },
  "_all" : {
    "primaries" : {
      "store" : {
        "size_in_bytes" : 2317554,
        "throttle_time_in_millis" : 0
      }
    },
    "total" : {
      "store" : {
        "size_in_bytes" : 2317554,
        "throttle_time_in_millis" : 0
      }
    }
  },
  "indices" : {
    "photos" : {
      "primaries" : {
        "store" : {
          "size_in_bytes" : 2317554,
          "throttle_time_in_millis" : 0
        }
      },
      "total" : {
        "store" : {
          "size_in_bytes" : 2317554,
          "throttle_time_in_millis" : 0
        }
      }
    }
  }
}

そこまで大きな差はない。むしろelasticsearch５の方がサイズが小さい？

20000件

• elasticsearch2

curl -X GET localhost:9200/photos/_stats/store?pretty=true
{
  "_shards" : {
    "total" : 10,
    "successful" : 5,
    "failed" : 0
  },
  "_all" : {
    "primaries" : {
      "store" : {
        "size_in_bytes" : 4248603,
        "throttle_time_in_millis" : 0
      }
    },
    "total" : {
      "store" : {
        "size_in_bytes" : 4248603,
        "throttle_time_in_millis" : 0
      }
    }
  },
  "indices" : {
    "photos" : {
      "primaries" : {
        "store" : {
          "size_in_bytes" : 4248603,
          "throttle_time_in_millis" : 0
        }
      },
      "total" : {
        "store" : {
          "size_in_bytes" : 4248603,
          "throttle_time_in_millis" : 0
        }
      }
    }
  }
}

• elasticsearch5

curl -X GET localhost:9200/photos/_stats/store?pretty=true
{
  "_shards" : {
    "total" : 10,
    "successful" : 5,
    "failed" : 0
  },
  "_all" : {
    "primaries" : {
      "store" : {
        "size_in_bytes" : 7414230,
        "throttle_time_in_millis" : 0
      }
    },
    "total" : {
      "store" : {
        "size_in_bytes" : 7414230,
        "throttle_time_in_millis" : 0
      }
    }
  },
  "indices" : {
    "photos" : {
      "primaries" : {
        "store" : {
          "size_in_bytes" : 7414230,
          "throttle_time_in_millis" : 0
        }
      },
      "total" : {
        "store" : {
          "size_in_bytes" : 7414230,
          "throttle_time_in_millis" : 0
        }
      }
    }
  }
}

40000件

• elasticsearch2

curl -X GET localhost:9200/photos/_stats/store?pretty=true
{
  "_shards" : {
    "total" : 10,
    "successful" : 5,
    "failed" : 0
  },
  "_all" : {
    "primaries" : {
      "store" : {
        "size_in_bytes" : 7401145,
        "throttle_time_in_millis" : 0
      }
    },
    "total" : {
      "store" : {
        "size_in_bytes" : 7401145,
        "throttle_time_in_millis" : 0
      }
    }
  },
  "indices" : {
    "photos" : {
      "primaries" : {
        "store" : {
          "size_in_bytes" : 7401145,
          "throttle_time_in_millis" : 0
        }
      },
      "total" : {
        "store" : {
          "size_in_bytes" : 7401145,
          "throttle_time_in_millis" : 0
        }
      }
    }
  }
}

• elasticsearch5

curl -X GET localhost:9200/photos/_stats/store?pretty=true
{
  "_shards" : {
    "total" : 10,
    "successful" : 5,
    "failed" : 0
  },
  "_all" : {
    "primaries" : {
      "store" : {
        "size_in_bytes" : 14973109,
        "throttle_time_in_millis" : 0
      }
    },
    "total" : {
      "store" : {
        "size_in_bytes" : 14973109,
        "throttle_time_in_millis" : 0
      }
    }
  },
  "indices" : {
    "photos" : {
      "primaries" : {
        "store" : {
          "size_in_bytes" : 14973109,
          "throttle_time_in_millis" : 0
        }
      },
      "total" : {
        "store" : {
          "size_in_bytes" : 14973109,
          "throttle_time_in_millis" : 0
        }
      }
    }
  }
}

同じmappingで件数だけ増やしていくと、サイズに差が出てくる。これにkeywordの設定など更に追加していくと違ってくるのだろう。引き続き検証した。