More than 3 years have passed since last update.

Prometheus×Grafanaでfluentdのプロセス監視を行う【process-exporter】

Posted at 2020-05-26

Prometheusでプロセス監視をやりたい

Prometheusを使う上で主流なのは各アプリ毎の専用のexporterを使うことみたい。

https://prometheus.io/docs/instrumenting/exporters/#miscellaneous
Third-party exportersに載ってたprocess-exporterを使います。

動作確認したいからローカルで実行する

準備

以下を準備する


- config
    - grafana
        - grafana.env(雛型として用意してるだけ) 
    - process-exporter
        - config.yml
    - prometheus
        - prometheus.yml
    - fluentd-data
- docker-compose.yml

grafana/grafana.env

# [server]
# GF_SERVER_DOMAIN=localhost
# GF_SERVER_HTTP_PORT=3000
# GF_SERVER_PROTOCOL=http

process-exporter/config.yml

process_names:
    - name: "{{.Comm}}"
      comm:
      - fluentd

prometheus/prometheus.yml

global:
  scrape_interval:     15s
  evaluation_interval: 15s
  external_labels:
      monitor: 'codelab-monitor'
  
rule_files:
#   - "/var/app/prometheus/alert.rules"
  
scrape_configs:
  - job_name: 'prometheus'
    static_configs:
      - targets: 
        - 'localhost:9090'
    
  - job_name: 'node-exporter'
    static_configs:
      - targets:
        - '[host or ip address]:9100'

  - job_name: 'process-exporter'
    static_configs:
      - targets:
        - '[host or ip address]:9256'

**[host or ip address]**は、自分のマシンに割り当てられているlocalhost以外のip addressやhostnameを指定します。

docker-compose.yml

version: '3'
services:
  prometheus:
    image: prom/prometheus
    container_name: prometheus
    volumes:
      - ./config/prometheus/prometheus.yml:/etc/prometheus/prometheus.yml
    ports:
      - 9090:9090
      
  grafana:
    image: grafana/grafana
    container_name: grafana
    ports:
      - 3000:3000
    env_file:
      - ./config/grafana/grafana.env

  node-exporter:
    image: quay.io/prometheus/node-exporter
    container_name: node-exporter
    ports:
      - 9100:9100
    volumes:
      - /proc:/host/proc
      - /sys:/host/sys
      - /:/rootfs

  process-exporter:
    image: ncabatoff/process-exporter
    container_name: process-exporter
    ports:
        - 9256:9256
    volumes:
      - /proc:/host/proc
      - ./config/process-exporter:/config
    command: >
      --procfs /host/proc
      -config.path /config/config.yml

  fluentd:
    image: fluent/fluentd
    container_name: fluentd
    ports:
      - 24224:24224
    volumes: 
      - ./fluentd-data:/fluentd/log