elasticsearch5 + kibana5 + logstash5 をdockerで動かす

参考

• Figure out what's up with 5.0 · Issue #98 · docker-library/elasticsearch
• dockerhub elasticsearch
• dockerhub kibana
• logstashでapacheのアクセスログをelasticsearchに送信し、kibanaでグラフ表示 - Qiita

docker-compose.yml

docker-compose.yml

es:
    image: elasticsearch:5
    ports:
      - "9200:9200"
      - "9300:9300"
    volumes:
      - ./es_config/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml
      - ./es_data/:/usr/share/elasticsearch/data/
    environment:
      - ES_JAVA_OPTS=-Xms512M -Xmx512M

ki:
    image: kibana:5
    ports:
      - "5601:5601"
    links:
        - es
    environment:
      - ELASTICSEARCH_URL=http://es:9200

• 5.0/config/elasticsearch.ymlを./es_config/elasticsearch.ymlに記述

./es_config/elasticsearch.yml

network.host: 0.0.0.0

# this value is required because we set "network.host"
# be sure to modify it appropriately for a production cluster deployment
discovery.zen.minimum_master_nodes: 1

起動

# sysctl -w vm.max_map_count=262144
docker-compose up

logstash (deb)

• LogstashからLogstash 5のdebインストールした

logstash.conf

input { file { path=> "/var/log/apache2/access.log" } }

filter {
  grok {
    match => { "message" => "%{COMBINEDAPACHELOG}" }
  }
  date {
    match => [ "timestamp" , "dd/MMM/yyyy:HH:mm:ss Z" ]
    locale => "en"
  }
  mutate {
    replace => { "type" => "apache_access" }
  }
}

output {
  elasticsearch { hosts => ["192.168.10.22:9200"] }
}

起動

sudo /usr/share/logstash/bin/logstash --path.settings=/etc/logstash/ -f logstash.conf

• logstash2系の場合、/opt/logstash/bin/logstash -f logstash.conf

• http://localhost:5601 にアクセスしログを受け取っていることを確認

logstash (docker)

logstash.conf

input { file { path=> "/var/log/apache2/access.log" } }

filter {
  grok {
    match => { "message" => "%{COMBINEDAPACHELOG}" }
  }
  date {
    match => [ "timestamp" , "dd/MMM/yyyy:HH:mm:ss Z" ]
    locale => "en"
  }
  mutate {
    replace => { "type" => "apache_access" }
  }
}

output {
  elasticsearch { hosts => ["192.168.10.22:9200"] }
  stdout { codec => rubydebug }
}

• root権限でlogstash起動

docker run  \
 -it --rm -v "$PWD":/config-dir \
 -v /var/log/apache2/:/var/log/apache2/:ro \
 logstash:5 \
 gosu root logstash -f /config-dir/logstash.conf