LoginSignup
maztak
@maztak (Takuya Matsuda)

Are you sure you want to delete the question?

If your question is resolved, you may close it.

Leaving a resolved question undeleted may help others!

We hope you find it useful!

開発環境と本番環境でSearchkickによる検索結果が違う

Q&A

ElasticsearchSearchkickbonsaiRubyonRails6.0

解決したいこと

開発環境と本番環境でSearchkickによる検索結果が違う。

RailsのElasticsearchのGem「Searchkick」を使って検索機能を実装しました。
しかしHerokuにデプロイした本番環境の検索結果と、ローカルサーバーでの検索結果が異なります。

具体的には、なぜかローカル環境だけ検索時のscoreが10倍されているようで、searchkickのクエリーとしてmin_score: 1としているフィルターをくぐり抜けてしまって関係ないものがヒットしてしまっているようです。

以下コードのmin_scoreの値を10倍したりして突き止めました。

home_controller.rb
 def products_query
      if params[:q].present?
        Product.search(params[:q], fields: [:keywords], body_options: {min_score: 1}, operator: "or", order: [{ id: :asc }])
      else
        Product.all.order(id: "desc")
      end
    end
product.rb
class Product < ApplicationRecord
    searchkick language: "japanese"

    def search_data
        {
            id: id,
            keywords: keywords
        }
    end
end

現在はproduction側が求める挙動になっているのでいいですが、後々のデプロイで問題になりそうなので原因を把握しておきたいです‥。もしお詳しい方がいたらご教授いただけると嬉しいですm(_ _)m

該当するソースコード

Railsコンソールでの検索実行結果

productionとローカルどちらも、boostが1と10のクエリを2回に分けて呼ばれているのが気になります。

production
> Product.search("加湿機", fields: [:keywords], body_options: {min_score: 1}, operator: "or", order: [{ id: :asc }], debug: true)

Elasticsearch Query
curl https://xxx:yyy@zzz.us-east-1.bonsaisearch.net:443/products_production/_search?pretty -H 'Content-Type: application/json' -d '{"query":{"bool":{"should":[{"dis_max":{"queries":[{"match":{"keywords.analyzed":{"query":"加湿機","boost":10,"operator":"or","cutoff_frequency":0.001,"analyzer":"searchkick_search"}}},{"match":{"keywords.analyzed":{"query":"加湿機","boost":1,"operator":"or","analyzer":"searchkick_search","fuzziness":1,"prefix_length":0,"max_expansions":3,"fuzzy_transpositions":true}}}]}}]}},"sort":[{"id":"asc"}],"timeout":"11s","_source":false,"size":10000,"min_score":1}'

Elasticsearch Results
{
  "took": 1,
  "timed_out": false,
  "_shards": {
    "total": 1,
    "successful": 1,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": {
      "value": 7,
      "relation": "eq"
    },
    "max_score": null,
    "hits": [
      {
        "_index": "products_production_20201213090523599",
        "_type": "_doc",
        "_id": "6",
        "_score": null,
        "sort": [
          6
        ]
      },
      {
        "_index": "products_production_20201213090523599",
        "_type": "_doc",
        "_id": "7",
        "_score": null,
        "sort": [
          7
        ]
      },
      {
        "_index": "products_production_20201213090523599",
        "_type": "_doc",
        "_id": "8",
        "_score": null,
        "sort": [
          8
        ]
      },
      {
        "_index": "products_production_20201213090523599",
        "_type": "_doc",
        "_id": "9",
        "_score": null,
        "sort": [
          9
        ]
      },
      {
        "_index": "products_production_20201213090523599",
        "_type": "_doc",
        "_id": "10",
        "_score": null,
        "sort": [
          10
        ]
      },
      {
        "_index": "products_production_20201213090523599",
        "_type": "_doc",
        "_id": "11",
        "_score": null,
        "sort": [
          11
        ]
      },
      {
        "_index": "products_production_20201213090523599",
        "_type": "_doc",
        "_id": "12",
        "_score": null,
        "sort": [
          12
        ]
      }
    ]
  }
}
localhost:3000
> Product.search("加湿機", fields: [:keywords], body_options: {min_score: 1}, operator: "or", order: [{ id: :asc }], debug: true)

Elasticsearch Query
curl http://localhost:9200/products_development/_search?pretty -H 'Content-Type: application/json' -d '{"query":{"bool":{"should":[{"dis_max":{"queries":[{"match":{"keywords.analyzed":{"query":"加湿機","boost":10,"operator":"or","analyzer":"searchkick_search"}}},{"match":{"keywords.analyzed":{"query":"加湿機","boost":1,"operator":"or","analyzer":"searchkick_search","fuzziness":1,"prefix_length":0,"max_expansions":3,"fuzzy_transpositions":true}}}]}}]}},"sort":[{"id":"asc"}],"timeout":"11s","_source":false,"size":10000,"min_score":1}'

Elasticsearch Results
{
  "took": 13,
  "timed_out": false,
  "_shards": {
    "total": 1,
    "successful": 1,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": {
      "value": 12,
      "relation": "eq"
    },
    "max_score": null,
    "hits": [
      {
        "_index": "products_development_20201213192742895",
        "_type": "_doc",
        "_id": "1",
        "_score": null,
        "sort": [
          1
        ]
      },
      // 中略
      {
        "_index": "products_development_20201213192742895",
        "_type": "_doc",
        "_id": "12",
        "_score": null,
        "sort": [
          12
        ]
      }
    ]
  }
}

kibana console (bonsai console)

production
{
  // "min_score": 1, 本来はこの行を入れてスコア1以上にフィルターします
  "query": {
    "multi_match": {
      "fields": [ "keywords.analyzed"],
      "query": "加湿機"
    }
  }
}

{
  "took": 1,
  "timed_out": false,
  "_shards": {
    "total": 1,
    "successful": 1,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": {
      "value": 12,
      "relation": "eq"
    },
    "max_score": 1.4967165,
    "hits": [
      {
        "_index": "products_production_20201213090523599",
        "_type": "_doc",
        "_id": "6",
        "_score": 1.4967165,
        "_source": {
          "id": 6,
          "keywords": "{加湿器,加湿機,気化式,ナノイー}"
        }
      },
      // 中略
      {
        "_index": "products_production_20201213090523599",
        "_type": "_doc",
        "_id": "3",
        "_score": 0.4183664,
        "_source": {
          "id": 3,
          "keywords": "{ドラム式乾燥機付き洗濯機,ドラム式洗濯乾燥機,ビッグドラム}"
        }
      }
    ]
  }
}
local:5601
GET /products_development/_search
{
  // "min_score": 1, 本来はこの行を入れてスコア1以上にフィルターします
  "query": {
    "multi_match": {
      "fields": [ "keywords.analyzed"],
      "query": "加湿機"
    }
  }
}

{
  "took" : 16,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 12,
      "relation" : "eq"
    },
    "max_score" : 1.3637168,
    "hits" : [
      {
        "_index" : "products_development_20201213192742895",
        "_type" : "_doc",
        "_id" : "6",
        "_score" : 1.3637168,
        "_source" : {
          "id" : 6,
          "keywords" : "{加湿器,加湿機,気化式,ナノイー}"
        }
      },
      // 中略
      {
        "_index" : "products_development_20201213192742895",
        "_type" : "_doc",
        "_id" : "3",
        "_score" : 0.34109813,
        "_source" : {
          "id" : 3,
          "keywords" : "{ドラム式乾燥機付き洗濯機,ドラム式洗濯乾燥機,ビッグドラム}"
        }
      }
    ]
  }
}

自分で試したこと

  • rake searchkick:reindex:allでのインデックスし直し
  • kibanaでDELETE /products_development_20201213192742895してからのインデックスし直し

環境

Gemfile.rb
source 'https://rubygems.org'
git_source(:github) { |repo| "https://github.com/#{repo}.git" }

ruby '2.7.1'

# Bundle edge Rails instead: gem 'rails', github: 'rails/rails'
gem 'rails', '~> 6.0.3', '>= 6.0.3.2'
# Use sqlite3 as the database for Active Record
group :development, :test do
  gem 'sqlite3', '~> 1.4'
end
group :production do
  gem 'pg', '0.20.0'
end

gem 'google-analytics-rails'
gem 'meta-tags'
gem 'elasticsearch', '7.2.0'
gem 'searchkick'

# 以下省略

思い当たる原因

  • ローカルとstagingのDBにproductionのデータをCSVにしてインポートした(前に上書きしようとしてproducts_pkeyがおかしくなったことがあった)
  • マイグレーションファイルでカラムを追加→削除した際にマイグレーションファイルだけ削除したがschema.rbの変更をそのままコミットしプッシュした（versionだけ書き換わったもの）
  • そもそもbody_options: {min_score: 1}はスコア1以上に絞るクエリーではない？
0

No Answers yet.

Your answer might help someone💌

LoginSign Up