Kafka Node Cluster Docker Compose

README.md

Referer links

• https://hub.docker.com/r/wurstmeister/kafka/
• https://raw.githubusercontent.com/wurstmeister/kafka-docker/master/docker-compose.yml

Details

• Should create kafka-ext network first by running docker network create kafka-ext before compose up
• Cannot access broker out of docker container, so program which need to access should running in Docker container
• For container which need broker, should join kafka-ext network, and connect by --broker-list kafka_1:9092,kafka_2:9092,kafka_3:9092

docker-compose.yaml

version: '3'

networks:
  kafka-ext:
    external: true

services:
  zookeeper:
    image: wurstmeister/zookeeper
    container_name: 'zookeeper'
  kafka_1:
    image: wurstmeister/kafka
    container_name: 'kafka_1'
    networks:
      - default
      - kafka-ext
    depends_on:
      - zookeeper
    environment:
      KAFKA_ADVERTISED_HOST_NAME: 'kafka_1'
      KAFKA_ADVERTISED_PORT: '9092'
      KAFKA_ZOOKEEPER_CONNECT: 'zookeeper:2181'
      KAFKA_CREATE_TOPICS: 'default-topic:1:3'
  kafka_2:
    image: wurstmeister/kafka
    container_name: 'kafka_2'
    networks:
      - default
      - kafka-ext
    depends_on:
      - zookeeper
    environment:
      KAFKA_ADVERTISED_HOST_NAME: 'kafka_2'
      KAFKA_ADVERTISED_PORT: '9092'
      KAFKA_ZOOKEEPER_CONNECT: 'zookeeper:2181'
      KAFKA_CREATE_TOPICS: 'default-topic:1:3'
  kafka_3:
    image: wurstmeister/kafka
    container_name: 'kafka_3'
    networks:
      - default
      - kafka-ext
    depends_on:
      - zookeeper
    environment:
      KAFKA_ADVERTISED_HOST_NAME: 'kafka_3'
      KAFKA_ADVERTISED_PORT: '9092'
      KAFKA_ZOOKEEPER_CONNECT: 'zookeeper:2181'
      KAFKA_CREATE_TOPICS: 'default-topic:1:3'