Running Multi Node Zookeeper Cluster on Docker

Introduction 

I am working on installing Apache Nifi on multi node cluster. For this I needed a Zookeeper multi node cluster. 

Zookeeper is a centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. 

Multi Node Cluster

We will run 3 node cluster on docker. We will use below docker-compose.yaml as below:

version: "3"
services:
  zk01:
    hostname: zk01
    container_name: zk01
    image: 'bitnami/zookeeper:3.7'
    ports:
      - '2181'
      - '2888'
      - '3888'
    environment:
      - ALLOW_ANONYMOUS_LOGIN=yes
      - ZOO_SERVER_ID=1
      - ZOO_SERVERS=0.0.0.0:2888:3888,zk02:2888:3888,zk03:2888:3888
    networks:
      - zk_net

  zk02:
    hostname: zk02
    container_name: zk02
    image: 'bitnami/zookeeper:3.7'
    ports:
      - '2181'
      - '2888'
      - '3888'
    environment:
      - ALLOW_ANONYMOUS_LOGIN=yes
      - ZOO_SERVER_ID=2
      - ZOO_SERVERS=zk01:2888:3888,0.0.0.0:2888:3888,zk03:2888:3888
    networks:
      - zk_net

networks:
  zk_net:
    driver: bridge

I am using bitnami-zookeeper image for it. You can use  latest version instead of 3.7 version if interested. 

This will launch a 3 node zookeeper cluster without publishing/exposing the nodes to the host machines. 

$  docker-compose -f docker-compose.yaml up

This will show the logs of all the 3 nodes as they start running.

After few minutes the log will be like below:

We can check the status by running below command:

$  docker ps

This will show all the 3 instances running.

You can now login to one of the instance/node using below command.

$ docker exec -it zk02 /bin/bash
I have no name!@zk02:/$

This will log you in to the zk02 node.

Testing the Cluster

We will now create some entry on zk02 and it will be reflected/replicated to all the other Zookeeper nodes immediately. We will start zookeeper cli for it.

$ zkCli.sh -server zk02:2181
/opt/bitnami/java/bin/java
Connecting to zk02:2181
2021-10-12 05:56:18,058 [myid:] - INFO  [main:Environment@98] - Client environment:zookeeper.version=3.7.0-e3704b390a6697bfdf4b0bef79e3da7a4f6bac4b, built on 2021-03-17 09:46 UTC
2021-10-12 05:56:18,064 [myid:] - INFO  [main:Environment@98] - Client environment:host.name=zk02
2021-10-12 05:56:18,065 [myid:] - INFO  [main:Environment@98] - Client environment:java.version=11.0.12
2021-10-12 05:56:18,069 [myid:] - INFO  [main:Environment@98] - Client environment:java.vendor=BellSoft

This will open below zk cli command prompt: 

2021-10-12 05:56:18,195 [myid:zk02:2181] - INFO  [main-SendThread(zk02:2181):ClientCnxn$SendThread@1005] - Socket connection established, initiating session, client: /172.28.0.2:43996, server: zk02/172.28.0.2:2181
2021-10-12 05:56:18,249 [myid:zk02:2181] - INFO  [main-SendThread(zk02:2181):ClientCnxn$SendThread@1438] - Session establishment complete on server zk02/172.28.0.2:2181, session id = 0x20010a8acf10000, negotiated timeout = 30000

WATCHER::

WatchedEvent state:SyncConnected type:None path:null
[zk: zk02:2181(CONNECTED) 0] 

Now run create : 

[zk: zk02:2181(CONNECTED) 0] create /hello world
Created /hello

We can query it as below

[zk: zk02:2181(CONNECTED) 1] get /hello
world

We can also check by logging into other nodes. We will follow the above steps to login and query on other node.

Clean up

We can delete the entry:

[zk: zk02:2181(CONNECTED) 2] delete /hello

Happy Coding !!