• SpringBoot-Kafka Tutorial

Introduction

• Apache Kafka Introduction
• Messages & Event Introduction
• Kafka Topic Introduction
• Ordering of Events
• Kafka Broker Introduction
• Kafka Server Installation

Kafka CLI

• Topic
• Producer
• Consumer

Kafka Producer

• Producer Microservice
• Acknowledgement & Retries
• Idempotent Producer

Kafka Consumer

• Consumer Microservice
• Handle Deserializer Errors
• Dead Leter Topic
• Exceptions & Retries
• Consumers in Consumer Group
• Idempotent Consumers
• Kafka Transactions

Apache Kafka CLI: Producer

Apache Kafka is a distributed streaming platform that allows you to publish and subscribe to streams of data. In this tutorial, you will learn how to use the Apache Kafka command-line interface (CLI) to produce messages to Kafka topics.

Producer can publish kafka topics in two different ways:
1. 1. Produce messages to kafka topic without key
2. 2. Produce messages to kafka topic with key

1. Produce messages to kafka topic without key

To produce messages to Kafka topics without a key, we nee to use the kafka-console-producer.sh script that comes with the Kafka installation. This script allows you to send messages to a Kafka topic from the standard input (stdin). Before we run the script, we need to make sure to bring up the kafka cluster.

Let's, asume that the Kafka cluster running on localhost:9092. This can be changed to match your cluster configuration. To produce messages to a Kafka topic called product-created-events-topic, we need to run the following command in a terminal:

./kafka-console-producer.sh --bootstrap-server localhost:9092 --topic product-created-events-topic

1. --bootstrap-server: option specifies the address of the Kafka broker that the producer will connect.
2. --topic: option specifies the name of the topic that the producer will send messages to.

After we run the command, we will see a > prompt, indicating that the producer is ready to accept messages from stdin. we can type any message, and press enter to send it to the topic.

For example:
1. > Hello, world!
2. > This is a test message.
3. > I am learning how to use Kafka CLI.

2. Produce messages to Kafka topics with a key

Messages in Kafka topics are stored as a key-value pair. If you send a message without a key, then the key is null. However, sometimes you may want to send a message with a key, for example, to ensure that messages with the same key are stored in the same partition, and thus preserve the order of the messages.

To produce messages to Kafka topics with a key, you will use the same kafka-console-producer.sh script, but with some additional properties. The first property is parse.key=true, which tells the producer to expect each message to be in the format of key:value. The second property is key.separator=:, which tells the producer what character to use to separate the key and the value in each message.

To produce messages with a key to the same topic product-created-events-topic, you can run the following command in a terminal:
./kafka-console-producer.sh --bootstrap-server localhost:9092 --topic product-created-events-topic --property "parse.key=true" --property "key.separator=:"
After you run the command, you will see a > prompt, indicating that the producer is ready to accept messages from stdin. You can type any message you want, as long as it follows the key:value format, and press enter to send it to the topic. For example:

1. > productId:123
2. > productName:Kafka mug
3. > productPrice:9.99

Each line that you type will be sent as a separate message to the topic, with the key and the value separated by the : character. You can verify that the messages are received by the topic by using another terminal to run the kafka-console-consumer.sh script, with the same properties as the producer. For example:
./kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic product-created-events-topic --from-beginning --property "print.key=true" --property "key.separator=:"
The print.key=true property tells the consumer to print both the key and the value of each message, separated by the key.separator character. You should see the messages that you typed in the producer terminal, in the same order. For example:

1. > productId:123
2. > productName:Kafka mug
3. > productPrice:9.99

---