Edited at

EC2インスタンスにelasticsearchをインストールする

More than 3 years have passed since last update.


Javaのインストール

$ sudo yum install java

$ java -version
java version "1.7.0_79"
OpenJDK Runtime Environment (amzn-2.5.5.1.59.amzn1-x86_64 u79-b14)
OpenJDK 64-Bit Server VM (build 24.79-b02, mixed mode)


elasticsearch本体のインストール

最新バージョンはこちらで確認

ここではElasticsearch 1.6.0をインストールする

$ wget https://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-1.6.0.tar.gz -O elasticsearch.tar.gz

$ tar -xf elasticsearch.tar.gz
$ sudo mv elasticsearch-* /usr/local/share/elasticsearch
$ sudo chown -R root:root /usr/local/share/elasticsearch/

$ vim elasticsearch/config/elasticsearch.yml

// 下記のコメントアウトを外す
http.port: 9200


起動スクリプトをインストール

$ curl -L http://github.com/elasticsearch/elasticsearch-servicewrapper/tarball/master | tar -xz

$ sudo mv *servicewrapper*/service /usr/local/share/elasticsearch/bin/

$ sudo /usr/local/share/elasticsearch/bin/service/elasticsearch install
Detected Linux:
Installing the Elasticsearch daemon..


起動

$ sudo service elasticsearch start

Starting Elasticsearch...
Waiting for Elasticsearch......................
WARNING: Elasticsearch may have failed to start.

エラー

$ sudo tail -f /var/log/messages

Jul 1 11:30:06 dev02 elasticsearch[31178]: Unable to write to the configured log file: /usr/local/share/elasticsearch/logs/service.log (No such file or directory)#012 Falling back to the default file in the current working directory: wrapper.log

ログの書き込み先がないと言われている

$ sudo mkdir -p /usr/local/share/elasticsearch/logs

$ sudo touch /usr/local/share/elasticsearch/logs/service.log

再び起動

$ sudo service elasticsearch start

Starting Elasticsearch...
Waiting for Elasticsearch......................
WARNING: Elasticsearch may have failed to start.

まだエラー

$ tail -f /usr/local/share/elasticsearch/logs/service.log


STATUS | wrapper | 2015/07/01 11:38:14 | Launching a JVM...
ERROR | wrapper | 2015/07/01 11:38:14 | JVM exited while loading the application.
INFO | jvm 5 | 2015/07/01 11:38:14 | OpenJDK 64-Bit Server VM warning: INFO: os::commit_memory(0x00000000bad30000, 986513408, 0) failed; error='Cannot allocate memory' (errno=12)
INFO | jvm 5 | 2015/07/01 11:38:14 | #
INFO | jvm 5 | 2015/07/01 11:38:14 | # There is insufficient memory for the Java Runtime Environment to continue.
INFO | jvm 5 | 2015/07/01 11:38:14 | # Native memory allocation (malloc) failed to allocate 986513408 bytes for committing reserved memory.
INFO | jvm 5 | 2015/07/01 11:38:14 | # An error report file with more information is saved as:
INFO | jvm 5 | 2015/07/01 11:38:14 | # /tmp/jvm-31983/hs_error.log
FATAL | wrapper | 2015/07/01 11:38:15 | There were 5 failed launches in a row, each lasting less than 300 seconds. Giving up.
FATAL | wrapper | 2015/07/01 11:38:15 | There may be a configuration problem: please check the logs.
STATUS | wrapper | 2015/07/01 11:38:15 | <-- Wrapper Stopped

Cannot allocate memory

しょぼいサーバだからか・・・

利用ヒープサイズを下げてみる

$ sudo vim /usr/local/share/elasticsearch/bin/service/elasticsearch.conf

#set.default.ES_HEAP_SIZE=1024
set.default.ES_HEAP_SIZE=512

再び起動

$ sudo service elasticsearch start

Starting Elasticsearch...
Waiting for Elasticsearch......
running: PID:32081

$ sudo lsof -i -nP | grep 9200

java 32083 root 96u IPv6 6352906 0t0 TCP *:9200 (LISTEN)

動いた

9200番ポートもLISTENになりました。

適当にテストデータを登録します。

$ curl -XPUT http://localhost:9200/mytest/test/1 -d '

{
"title" : "hogehoge",
"text" : "fugafuga"
}'

{"_index":"mytest","_type":"test","_id":"1","_version":2,"created":false}[

検索します

$ curl -XGET 'localhost:9200/mytest/test/1?pretty=true'

{
"_index" : "mytest",
"_type" : "test",
"_id" : "1",
"_version" : 2,
"found" : true,
"_source":
{
"title" : "hogehoge",
"text" : "fugafuga"
}
}

OK


kuromojiプラグインをインストール

日本語形態素解析のためのプラグイン

バージョン情報はこちら

elasticsearchが1.6.0なので、kuromoji-2.6.0をインストールする

$ sudo /usr/local/share/elasticsearch/bin/plugin --install elasticsearch/elasticsearch-analysis-kuromoji/2.6.0

$ sudo service elasticsearch restart
Stopping Elasticsearch...
Stopped Elasticsearch.
Starting Elasticsearch...
Waiting for Elasticsearch......
running: PID:32331


kuromojiの動作確認

$ curl -XPUT 'http://localhost:9200/kuromoji_test/' -d'

{
"index":{
"analysis":{
"tokenizer" : {
"kuromoji" : {
"type" : "kuromoji_tokenizer"
}
},
"analyzer" : {
"analyzer" : {
"type" : "custom",
"tokenizer" : "kuromoji"
}
}
}
}
}'

{"acknowledged":true}

kuromoji_testに文字列を投げる

$ curl -XPOST 'http://localhost:9200/kuromoji_test/_analyze?analyzer=analyzer&petty' -d '私はビールが大好きです'

{
"tokens":[
{
"token":"私",
"start_offset":0,
"end_offset":1,
"type":"word",
"position":1
},
{
"token":"は",
"start_offset":1,
"end_offset":2,
"type":"word",
"position":2
},{
"token":"ビール",
"start_offset":2,
"end_offset":5,
"type":"word",
"position":3
},{
"token":"が",
"start_offset":5,
"end_offset":6,
"type":"word",
"position":4
},{
"token":"大好き",
"start_offset":6,
"end_offset":9,
"type":"word",
"position":5
},{
"token":"です",
"start_offset":9,
"end_offset":11,
"type":"word",
"position":6
}
]
}

文字列が形態素解析されて返ってきました。

次に、日本語をインデックスに登録。

$ curl -XPUT http://localhost:9200/mytest/test/1 -d '

{
"title" : "テストだよ",
"text" : "イケてるサービス作ろうze!"
}'

{"_index":"mytest","_type":"test","_id":"1","_version":3,"created":false}

$ curl -XPUT http://localhost:9200/mytest/test/2 -d '
{
"title" : "テストだよ2",
"text" : "ビールうましうまし"
}'

{"_index":"mytest","_type":"test","_id":"2","_version":1,"created":true}

検索

$ curl -XGET http://localhost:9200/mytest/test/_search -d '

{
"query": {
"match":{"text":"イケてる"}
}
}'

{
"took":137,
"timed_out":false,
"_shards":{
"total":5,
"successful":5,
"failed":0
},
"hits":{
"total":1,
"max_score":0.16608897,
"hits":[{
"_index":"mytest",
"_type":"test",
"_id":"1",
"_score":0.16608897,
"_source": {
"title" : "テストだよ",
"text" : "イケてるサービス作ろうze!"
}
}]
}
}

無事日本語検索できました。