# Microservice Communication with Kafka

### Overview

Communication between Microservices can be categorized into two types, Synchronous and Asynchronous Communication. while REST and gRPC are Synchronous ways of communication, Kafka communications are asynchronous and do not wait for a response. Here we will see the local setup of Kafka in node.js and communications between services with Kafka queue.

### Installation

To install Apache Kafka, Java is the only prerequisite. Make Sure at least Java 11 is already installed in your system.

* Download the latest version of Apache Kafka from the official [site](https://kafka.apache.org/downloads) under Binary Downloads.
    
* Extract the contents (double click in the Finder) to a directory of your choice.
    
* Now you have to start two services **zookeeper** and **kafka** from downloaded binaries.
    
* Suppose the extracted directory is kafka\_2.13-3.60 (The name of the directory may be different based on your installed Kafka version), Go to the Root Directory where you have extracted Kafka and run the following command to start zookeeper.
    
    ```bash
    kafka_2.13-3.6.0/bin/zookeeper-server-start.sh kafka_2.13-3.6.0/config/zookeeper.properties
    ```
    
* ![](https://cdn.hashnode.com/res/hashnode/image/upload/v1698500802237/43a21ed5-5f69-436d-8762-4747d232528d.png align="center")
    
* Open another Terminal window and run the following command to start the local Kafka server.
    

```bash
kafka_2.13-3.6.0/bin/kafka-server-start.sh kafka_2.13-3.6.0/config/server.properties
```

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1698500911859/cb660f6d-d88c-46f4-8c72-198458c69444.png align="center")

* Kafka is Now Started on `localhost:9092` . Ensure both terminals are running always otherwise you will shut down the zookeeper or Kafka.
    
* you can also make a single executable script to run both commands at different terminals as per your operating system. (In macOS you can use something similar to this to start both as a single command).
    
* you can read more about downloaded Kafka binaries on the official [site](https://kafka.apache.org/intro).
    

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1698769299307/4978d017-538f-4086-a1f7-4420d497d9a1.png align="center")

### Setup in Node.js

We must install the Kafka client library for node.js to use our locally running Kafka server. for this tutorial, we will be installing [kafkaJS](https://kafka.js.org/) from [npm](https://www.npmjs.com/package/kafkajs).

To utilize Kafka for microservice communications, we need to write some boilerplate code for different roles such as producer, consumer, and admin in the Kafka client library such that the producer will produce messages in some topic, and then a consumer can consume that message. with the help of the admin client users can configure their Kafka clients.

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1698766783261/981b77a9-ce76-408d-a4e8-57ea9ef9a787.png align="center")

In the `admin.js` file, we need to provide the address of the running Kafka broker during initialization, In this case, our local Kafka instance is running at `localhost:9092.` we can also create a topic and use that topic in producer and consumer files for sending and receiving messages as per the above templates. For now, we are just consoling received messages.

Now our basic template is ready we just need the driver code to use our `sendMessage` function. for that, we have created `index.js` to act as both producer as well as consumer based on passed `argv` from the command line. In producer mode, it will continuously send messages using our `sendMessage` function. In consumer mode, it will subscribe to the topic and start listening to it using `initConsumer`.

Now open two terminals and enter the following command.

* npm start producer
    
* npm start consumer
    

Note: - you may also need to add the `start` command in the script section of the `package.json` file as per your folder structure to run the `index.js` file

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1698767838995/5e6b4a79-cde5-4023-b7ea-d038e68f9620.png align="center")

First, start the consumer using the above command and then start the producer and see the message passing between these two services in action as below.

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1698768302812/d24159e4-4112-474c-8442-fec30dc0522b.gif align="center")

Hope this helps to get some understanding of Kafka with node.js. Let me know if you have any other questions.

### Resources

1. [https://kafka.js.org/](https://kafka.js.org/)
    
2. [https://kafka.apache.org/](https://kafka.apache.org/intro)