Qiita Teams that are logged in
You are not logged in to any team

Log in to Qiita Team
Community
OrganizationEventAdvent CalendarQiitadon (β)
Service
Qiita JobsQiita ZineQiita Blog
3
Help us understand the problem. What are the problem?

More than 1 year has passed since last update.

Apache Kafka in Docker on Windows. (その2)

はじめに

Kafka Dockerとkafka-manager Dockerイメージを別々に起動してみる
Kafka Docker中でproducer、consumerプロセスを実行する

参考URL

https://zablo.net/blog/post/setup-apache-kafka-in-docker-on-windows
https://github.com/spotify/docker-kafka
https://hub.docker.com/r/mzagar/kafka-manager-docker/

バージョン確認、docker 起動

Apache Kafka in Docker on Windows.と同じ
https://qiita.com/0ashina0/items/e8b6a3ac4cf19805b15c

Kafka Docker

ダウンロード

URL:https://github.com/spotify/docker-kafka
「Clone or download」をクリックし、「Clone with HTTPS」の「Download ZIP」をクリックする
image.png

解凍

ダウンロードした「docker-kafka-master.zip」を解凍する。

docker起動

解凍したフォルダーdocker-kafka-master\kafkaで起動。(Dockerfileがあるフォルダー)
ADVERTISED_HOSTのIPは、ipconfigコマンドで確認(例:192.175.12.5)

docker run -p 2181:2181 -p 9092:9092 --env ADVERTISED_HOST=192.175.12.5 --env ADVERTISED_PORT=9092 spotify/kafka

起動イメージ

> docker run -p 2181:2181 -p 9092:9092 --env ADVERTISED_HOST=192.175.12.5 --env ADVERTISED_PORT=9092 spotify/kafka
/usr/lib/python2.7/dist-packages/supervisor/options.py:296: UserWarning: Supervisord is running as root and it is searching for its configuration file in default locations (including its current working directory); you probably want to specify a "-c" argument specifying an absolute path to a configuration file for improved security.
  'Supervisord is running as root and it is searching '
2018-10-21 02:45:09,390 CRIT Supervisor running as root (no user in config file)
2018-10-21 02:45:09,391 WARN Included extra file "/etc/supervisor/conf.d/zookeeper.conf" during parsing
2018-10-21 02:45:09,391 WARN Included extra file "/etc/supervisor/conf.d/kafka.conf" during parsing
2018-10-21 02:45:09,403 INFO RPC interface 'supervisor' initialized
2018-10-21 02:45:09,403 CRIT Server 'unix_http_server' running without any HTTP authentication checking
2018-10-21 02:45:09,403 INFO supervisord started with pid 1
2018-10-21 02:45:10,407 INFO spawned: 'zookeeper' with pid 8
2018-10-21 02:45:10,413 INFO spawned: 'kafka' with pid 9
2018-10-21 02:45:11,454 INFO success: zookeeper entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2018-10-21 02:45:11,454 INFO success: kafka entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)

kafka-manager Docker

ダウンロード

URL:https://github.com/mzagar/kafka-manager-docker

「Clone or download」をクリックし、「Clone with HTTPS」の「Download ZIP」をクリックする
image.png

解凍

ダウンロードした「docker-kafka-master.zip」を解凍する。

docker起動

解凍したフォルダーKafkaManagerで起動。(Dockerfileがあるフォルダー)
ZK_HOSTSのIPは、ipconfigコマンドで確認

docker run -it --rm  -p 9000:9000 -e ZK_HOSTS="192.175.12.5:2181" -e APPLICATION_SECRET=letmein sheepkiller/kafka-manager

Cluster、Topic作成

Apache Kafka in Docker on Windows.と同じ
https://qiita.com/0ashina0/items/e8b6a3ac4cf19805b15c

producer起動

> docker ps
CONTAINER ID        IMAGE                       COMMAND                  CREATED             STATUS              PORTS                                            NAMES
ce40c1bcd279        sheepkiller/kafka-manager   "./start-kafka-manag…"   2 hours ago         Up 2 hours          0.0.0.0:9000->9000/tcp                           distracted_wozniak
ae02d73f4bcd        spotify/kafka               "supervisord -n"         2 hours ago         Up 2 hours          0.0.0.0:2181->2181/tcp, 0.0.0.0:9092->9092/tcp   gallant_aryabhata
> docker exec -i -t ae02d73f4bcd  /bin/bash
root@ae02d73f4bcd:/# cd /opt/kafka_2.11-0.10.1.0/bin/
root@ae02d73f4bcd:/opt/kafka_2.11-0.10.1.0/bin# export KAFKA=192.175.12.5:9092
root@ae02d73f4bcd:/opt/kafka_2.11-0.10.1.0/bin# ./kafka-console-producer.sh --broker-list $KAFKA --topic test

consumer起動

> docker exec -i -t ae02d73f4bcd  /bin/bash
root@ae02d73f4bcd:/# cd /opt/kafka_2.11-0.10.1.0/bin/
root@ae02d73f4bcd:/opt/kafka_2.11-0.10.1.0/bin# export ZOOKEEPER=192.175.12.5:2181
root@ae02d73f4bcd:/opt/kafka_2.11-0.10.1.0/bin# ./kafka-console-consumer.sh --zookeeper $ZOOKEEPER --topic test

producerからメッセージ送信

root@ae02d73f4bcd:/opt/kafka_2.11-0.10.1.0/bin# ./kafka-console-producer.sh --broker-list $KAFKA --topic test
messageA

consumerでメッセージ確認

root@ae02d73f4bcd:/opt/kafka_2.11-0.10.1.0/bin# ./kafka-console-consumer.sh --zookeeper $ZOOKEEPER --topic test
Using the ConsoleConsumer with old consumer is deprecated and will be removed in a future major release. Consider using the new consumer by passing [bootstrap-server] instead of [zookeeper].
messageA
Why not register and get more from Qiita?
  1. We will deliver articles that match you
    By following users and tags, you can catch up information on technical fields that you are interested in as a whole
  2. you can read useful information later efficiently
    By "stocking" the articles you like, you can search right away
3
Help us understand the problem. What are the problem?