【Practical Combat】Kafka parameter tuning solves the problem of too large message body

Little scum · Posted on 12/13/2021 4:25:11 PM

When sending messages using Kafka, the producer max.request.size defaults to:1048576(1M) in size.

If the producer message body exceeds 1MB, the following error is reported:

The message is 18685640 bytes when serialized which is larger than 1048576, which is the value of the max.request.size configuration.

solution

For example, in the spring project, set the maximum request size in the project's configuration file20MBAs follows:

Login is visible.

After the setup is completed, restart the project, and you can see that the settings have been successful, as shown in the figure below:

When the message sent exceeds 1MB, a new error will be reported, as follows:

The request included a message larger than the max message size the server will accept.

solution

Log in to the Kafka server and modifyserver.propertiesfile, add the following configuration:

Login is visible.

Default: 1000012, close to 1M, the maximum size of messages that the server can receive.

Restart the Kafka service and the issue is resolved.

There are no other settings for consumersFor example:

Login is visible.

(End)

Little scum · Posted on 12/13/2021 4:31:31 PM

Note: This parameter does not work on the created topic after it is adjusted

Adjust the created topic parameters: bin/kafka-configs.sh --zookeeper localhost:2181 --alter --topic topicName --config max.message.bytes=62914560

Perfect solution

flying fish · Posted on 12/26/2021 5:16:30 PM

Learn to learn...

【Practical Combat】Kafka parameter tuning solves the problem of too large message body

Related Posts

Sections viewed