Apache Kafka in Docker on Windows. (その２)

はじめに

Kafka Dockerとkafka-manager Dockerイメージを別々に起動してみる
Kafka Docker中でproducer、consumerプロセスを実行する

参考URL

https://zablo.net/blog/post/setup-apache-kafka-in-docker-on-windows
https://github.com/spotify/docker-kafka
https://hub.docker.com/r/mzagar/kafka-manager-docker/

バージョン確認、docker 起動

Apache Kafka in Docker on Windows.と同じ
https://qiita.com/0ashina0/items/e8b6a3ac4cf19805b15c

Kafka Docker

ダウンロード

URL：https://github.com/spotify/docker-kafka
「Clone or download」をクリックし、「Clone with HTTPS」の「Download ZIP」をクリックする

解凍

ダウンロードした「docker-kafka-master.zip」を解凍する。

docker起動

解凍したフォルダーdocker-kafka-master\kafkaで起動。（Dockerfileがあるフォルダー）
ADVERTISED_HOSTのIPは、ipconfigコマンドで確認(例：192.175.12.5)

docker run -p 2181:2181 -p 9092:9092 --env ADVERTISED_HOST=192.175.12.5 --env ADVERTISED_PORT=9092 spotify/kafka

起動イメージ

> docker run -p 2181:2181 -p 9092:9092 --env ADVERTISED_HOST=192.175.12.5 --env ADVERTISED_PORT=9092 spotify/kafka
/usr/lib/python2.7/dist-packages/supervisor/options.py:296: UserWarning: Supervisord is running as root and it is searching for its configuration file in default locations (including its current working directory); you probably want to specify a "-c" argument specifying an absolute path to a configuration file for improved security.
  'Supervisord is running as root and it is searching '
2018-10-21 02:45:09,390 CRIT Supervisor running as root (no user in config file)
2018-10-21 02:45:09,391 WARN Included extra file "/etc/supervisor/conf.d/zookeeper.conf" during parsing
2018-10-21 02:45:09,391 WARN Included extra file "/etc/supervisor/conf.d/kafka.conf" during parsing
2018-10-21 02:45:09,403 INFO RPC interface 'supervisor' initialized
2018-10-21 02:45:09,403 CRIT Server 'unix_http_server' running without any HTTP authentication checking
2018-10-21 02:45:09,403 INFO supervisord started with pid 1
2018-10-21 02:45:10,407 INFO spawned: 'zookeeper' with pid 8
2018-10-21 02:45:10,413 INFO spawned: 'kafka' with pid 9
2018-10-21 02:45:11,454 INFO success: zookeeper entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2018-10-21 02:45:11,454 INFO success: kafka entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)

kafka-manager Docker

ダウンロード

URL：https://github.com/mzagar/kafka-manager-docker

「Clone or download」をクリックし、「Clone with HTTPS」の「Download ZIP」をクリックする

解凍

ダウンロードした「docker-kafka-master.zip」を解凍する。

docker起動

解凍したフォルダーKafkaManagerで起動。（Dockerfileがあるフォルダー）
ZK_HOSTSのIPは、ipconfigコマンドで確認

docker run -it --rm  -p 9000:9000 -e ZK_HOSTS="192.175.12.5:2181" -e APPLICATION_SECRET=letmein sheepkiller/kafka-manager

Cluster、Topic作成

Apache Kafka in Docker on Windows.と同じ
https://qiita.com/0ashina0/items/e8b6a3ac4cf19805b15c

producer起動

> docker ps
CONTAINER ID        IMAGE                       COMMAND                  CREATED             STATUS              PORTS                                            NAMES
ce40c1bcd279        sheepkiller/kafka-manager   "./start-kafka-manag…"   2 hours ago         Up 2 hours          0.0.0.0:9000->9000/tcp                           distracted_wozniak
ae02d73f4bcd        spotify/kafka               "supervisord -n"         2 hours ago         Up 2 hours          0.0.0.0:2181->2181/tcp, 0.0.0.0:9092->9092/tcp   gallant_aryabhata
> docker exec -i -t ae02d73f4bcd  /bin/bash

root@ae02d73f4bcd:/# cd /opt/kafka_2.11-0.10.1.0/bin/
root@ae02d73f4bcd:/opt/kafka_2.11-0.10.1.0/bin# export KAFKA=192.175.12.5:9092
root@ae02d73f4bcd:/opt/kafka_2.11-0.10.1.0/bin# ./kafka-console-producer.sh --broker-list $KAFKA --topic test

consumer起動

> docker exec -i -t ae02d73f4bcd  /bin/bash

root@ae02d73f4bcd:/# cd /opt/kafka_2.11-0.10.1.0/bin/
root@ae02d73f4bcd:/opt/kafka_2.11-0.10.1.0/bin# export ZOOKEEPER=192.175.12.5:2181
root@ae02d73f4bcd:/opt/kafka_2.11-0.10.1.0/bin# ./kafka-console-consumer.sh --zookeeper $ZOOKEEPER --topic test

producerからメッセージ送信

root@ae02d73f4bcd:/opt/kafka_2.11-0.10.1.0/bin# ./kafka-console-producer.sh --broker-list $KAFKA --topic test
messageA

consumerでメッセージ確認

root@ae02d73f4bcd:/opt/kafka_2.11-0.10.1.0/bin# ./kafka-console-consumer.sh --zookeeper $ZOOKEEPER --topic test
Using the ConsoleConsumer with old consumer is deprecated and will be removed in a future major release. Consider using the new consumer by passing [bootstrap-server] instead of [zookeeper].
messageA