Kafka 管理 メモ

Topics確認

背景：Kafkaクラスタ管理

cd /usr/hdp/current/kafka-broker/

# Access Kafka Cluster
bin/kafka-topics.sh  --zookeeper zzeng-hdp-1.field.hortonworks.com:2181,zzeng-hdp-2.field.hortonworks.com:2181,zzeng-hdp-3.field.hortonworks.com:2181 --describe

Topics 新規作成

bin/kafka-topics.sh --create --zookeeper  zzeng-hdp-1.field.hortonworks.com:2181,zzeng-hdp-2.field.hortonworks.com:2181,zzeng-hdp-3.field.hortonworks.com:2181 --replication-factor 3 --partitions 1 --topic zzeng01

Topics 一覧表示

bin/kafka-topics.sh --zookeeper  zzeng-hdp-1.field.hortonworks.com:2181,zzeng-hdp-2.field.hortonworks.com:2181,zzeng-hdp-3.field.hortonworks.com:2181 --list 

Topic内容を見る：Consumer

### Kafka 1.x
bin/kafka-console-consumer.sh --zookeeper zzeng-hdp-1.field.hortonworks.com:2181,zzeng-hdp-2.field.hortonworks.com:2181,zzeng-hdp-3.field.hortonworks.com:2181 --from-beginning --topic meetup_rsvp_avro

### Kafka 2.x
bin/kafka-console-consumer.sh --bootstrap-server zzeng-hdp-1.field.hortonworks.com:6667 --topic meetup_rsvp_avro