본문 바로가기

리눅스

[Kafka] 카프카 producer와 consumer 테스트

728x90

카프카 producer와 consumer 테스트

 

이미지 출처-https://www.tutorialspoint.com/apache_kafka/images/cluster_architecture.jpg

 

이미지 출처-https://www.researchgate.net/profile/Asad-Javed-9/publication/338516815/figure/fig2/AS:846484845125632@1578829122361/Example-of-Kafka-cluster-with-three-brokers.png

카프카 토픽 생성

topic list

$ kafka-topics.sh --list --bootstrap-server localhost:9092

topic create

- replication-factor : 복제본 개수(2)
- partitions : 파티션 개수(3)

$ kafka-topics.sh --create --bootstrap-server localhost:9092 --replication-factor 2 --partitions 3 --topic helloworld
Created topic helloworld.

topic list

$ kafka-topics.sh --list --bootstrap-server localhost:9092
helloworld

topic delete

$ kafka-topics.sh --delete --bootstrap-server localhost:9092 --topic helloworld

프로듀서(Producer), 컨슈머(Consumer) 실행하기

프로듀서 실행 / 메시지 Write

$ kafka-console-producer.sh --broker-list localhost:9092 --topic helloworld

컨슈머(Consumer) 실행 / 메시지 Read

$ kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic helloworld

 --from-beginning 옵션을 추가하면 토픽에 있는 모든 레코드를 확인 할수 있다.

$ kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic helloworld --from-beginning

컨슈머 그룹(Consumer Group) list

$ kafka-consumer-groups.sh --list --bootstrap-server localhost:9092
console-consumer-58810

컨슈머(Consumer) status and Offset 확인

$ kafka-consumer-groups.sh --describe --group console-consumer-58810 --bootstrap-server localhost:9092

GROUP                  TOPIC           PARTITION  CURRENT-OFFSET  LOG-END-OFFSET  LAG             CONSUMER-ID                                                            HOST            CLIENT-ID
console-consumer-58810 helloworld      0          -               1               -               consumer-console-consumer-58810-1-034dcce6-8cc4-46d9-a09d-dfcf32469460 /192.168.0.103   consumer-console-consumer-58810-1

컨슈머 그룹(Consumer Group) delete

$ kafka-consumer-groups.sh --delete --group console-consumer-58810 --bootstrap-server localhost:9092
Deletion of requested consumer groups ('console-consumer-58810') was successful.

 

Kafka 문서 : https://kafka.apache.org/documentation/#introduction

Kafka 시스템 구조 알아보기 : https://velog.io/@jwpark06/Kafka-%EC%8B%9C%EC%8A%A4%ED%85%9C-%EA%B5%AC%EC%A1%B0-%EC%95%8C%EC%95%84%EB%B3%B4%EA%B8%B0

 

728x90