Help us understand the problem. What is going on with this article?

docker-composeでfluentd + elasticsearch + kibana4 環境をつくる

More than 3 years have passed since last update.
  • いろいろ探しましたが、集約した情報がなかったのでまとめました。
  • docker-compose 1.3ではlog-driver設定にfluentdが対応していないようです。 → 1.8で対応したので追記しました。

dockerおよびdocker-comoposeのインストール

  • docker/docker-composeのインストールは公式ページを参考にインストール
  • docker
  • docker-compose

設定ファイルなど

以下のファイルを作成する

docker-compose.yml
fluentd:
  build: ./fluentd
  links:
   - "elasticsearch"
# Fluentd logging driver対応の場合
  ports:
   - "22422:22422"
# Fluentd logging driver非対応の場合
  volumes:
   - /var/lib/docker/containers:/var/lib/docker/containers
   - /var/run:/var/run
   - /var/log/docker:/var/log/docker

elasticsearch:
  build: elasticsearch
  expose:
    - 9200

kibana:
  image: kibana
  links:
   - "elasticsearch"
  ports:
   - "5601:5601"

fluentd

Fluentd logging driver対応の場合

pluginsディレクトリが必須なので前もって作成する

mkdir -p fluentd/plugins

fluent-plugin-elasticsearchをインストールする

FROM fluent/fluentd

RUN ["gem", "install", "fluent-plugin-elasticsearch", "--no-rdoc", "--no-ri"]
fluentd/fluent.conf
<source>
  type forward
  port 24224
  bind 0.0.0.0
</source>
<match docker.**>
  type elasticsearch
  log_level debug
  host elasticsearch
  port 9200
  include_tag_key true
  logstash_format true
  flush_interval 5s
</match>

Fluentd logging driver非対応の場合

FROM kiyoto/fluentd:0.10.56-2.1.1
MAINTAINER kiyoto@treausure-data.com
RUN mkdir /etc/fluent
ADD fluent.conf /etc/fluent/
RUN ["apt-get", "update"]
RUN ["apt-get", "install", "--yes", "make", "libcurl4-gnutls-dev"]
RUN ["/usr/local/bin/gem", "install", "fluent-plugin-elasticsearch", "--no-rdoc", "--no-ri"]
RUN ["/usr/local/bin/gem", "install", "fluent-plugin-record-reformer", "--no-rdoc", "--no-ri"]
RUN ["/usr/local/bin/gem", "install", "fluent-plugin-docker-tag-resolver", "--no-rdoc", "--no-ri"]
ENTRYPOINT ["/usr/local/bin/fluentd", "-c", "/etc/fluent/fluent.conf"]
fluentd/fluent.conf
<source>
  type tail
  path /var/lib/docker/containers/*/*-json.log
  pos_file /var/log/fluentd-docker.pos
  time_format %Y-%m-%dT%H:%M:%S
  tag docker.log.*
  format json
</source>
<match docker.log.**>
  type docker_tag_resolver
</match>
<match docker.container.**>
  type record_reformer
  container_id ${tag_parts[4]}
  container_name ${tag_parts[3]}
  tag docker.all
</match>
<match docker.all>
  type elasticsearch
  log_level debug
  host elasticsearch
  port 9200
  include_tag_key true
  logstash_format true
  flush_interval 5s
</match>

elasticsearch

デフォルトではコンテナ外から接続できないのでnetwork.bind_hostを設定する

FROM elasticsearch

RUN echo "network.bind_host: 0.0.0.0" >> /usr/share/elasticsearch/config/elasticsearch.yml

起動方法

docker-compose.ymlのあるディレクトリで以下のコマンドを実行

docker-compose up -d

参考情報

Docker 1.8 の Fluentd logging driver でコンテナログをフォワードする

Why not register and get more from Qiita?
  1. We will deliver articles that match you
    By following users and tags, you can catch up information on technical fields that you are interested in as a whole
  2. you can read useful information later efficiently
    By "stocking" the articles you like, you can search right away
Comments
No comments
Sign up for free and join this conversation.
If you already have a Qiita account
Why do not you register as a user and use Qiita more conveniently?
You need to log in to use this function. Qiita can be used more conveniently after logging in.
You seem to be reading articles frequently this month. Qiita can be used more conveniently after logging in.
  1. We will deliver articles that match you
    By following users and tags, you can catch up information on technical fields that you are interested in as a whole
  2. you can read useful information later efficiently
    By "stocking" the articles you like, you can search right away
ユーザーは見つかりませんでした