CentOS builds Kafka message middleware

Little scum · Posted on 12/12/2021 12:25:22 PM

Kafka is an open-source streaming platform developed by the Apache Software Foundation, written in Scala and Java. Kafka is a high-throughput distributed publish-subscribe messaging system that processes all the consumer's action flow data in the website. This action (web browsing, searching, and other user actions) is a key factor in many social functions on the modern web. This data is often resolved by processing logs and log aggregation due to throughput requirements. This is a viable solution for log data and offline analytics systems like Hadoop, but requiring the limitations of real-time processing. The purpose of Kafka is to unify online and offline message processing through Hadoop's parallel loading mechanism, and also to provide real-time messages through clustering.

Review:

Kafka operations for .NET/C# message queues [with source code]
https://www.itsvse.com/thread-9619-1-1.html

Kafka configures JMX remote monitoring
https://www.itsvse.com/thread-9968-1-1.html

First, I'm using a CentOS 8 system with Kafka version:kafka_2.13-3.0.0。

Download the unzip command as follows:

Login is visible.

Configure Kafka

Go to the /data/middleware/kafka_2.13-3.0.0/config folder and modify itserver.propertiesThe main amendments are as follows:

Login is visible.

Launch Zookeeper and Kafka

Start command

Login is visible.

Stop command

Login is visible.

Register Zookeeper and Kafka as a service

Go to the /etc/systemd/system folder

Create a new zookeeper.service service file with the following configuration:

Login is visible.

Start and set up the startup

Login is visible.

Create a new kafka.service service file and configure it as follows:

Login is visible.

Start and set up the startup

Login is visible.

Check the status of the Zookeeper and Kafka services as shown in the following image:

Kafka default port:9092, zookeeper default port:2181thereinto19988 It's a JMX port!