This article is a mirror article of machine translation, please click here to jump to the original article.

Architect_Programmer_Code Agriculture Network»Architect › Programming › Technical chat › Windows Kafka ERROR Failed to clean up log for __con ...
View: 15810|Reply: 2

Windows Kafka ERROR Failed to clean up log for __consumer_offsets

[Copy link]
Posted on 7/6/2021 1:30:04 PM | | |
Review:

Kafka configures JMX remote monitoring
https://www.itsvse.com/thread-9968-1-1.html

Detailed explanation of Kafka configuration parameters
https://www.itsvse.com/thread-9965-1-1.html

Kafka manually sets the offset offset
https://www.itsvse.com/thread-9641-1-1.html

Kafka operations for .NET/C# message queues [with source code]
https://www.itsvse.com/thread-9619-1-1.html


The Windows Kafka error is as follows:

[2021-07-06 09:05:54,809] ERROR Failed to clean up log for __consumer_offsets-42 in dir C:\tmp\kafka-logs due to IOException (kafka.server.LogDirFailureChannel)
java.nio.file.FileSystemException: C:\tmp\kafka-logs\__consumer_offsets-42\00000000000000000000.timeindex.cleaned -> C:\tmp\kafka-logs\__consumer_offsets-42\00000000000000000000.timeindex.swap:


[2021-07-06 09:06:06,460] ERROR [Log partition=__consumer_offsets-42, dir=C:\tmp\kafka-logs] Could not find offset index file corresponding to log file C:\tmp\kafka-logs\__consumer_offsets-42\ 00000000000000000000.log, recovering segment and rebuilding index files... (kafka.log.Log)

[2021-07-06 09:06:10,607] ERROR Error while creating ephemeral at /brokers/ids/0, node already exists and owner '72057597827809280' does not match current session '72057597827809313' (kafka.zk.KafkaZkClient$ CheckedEphemeral)
[2021-07-06 09:06:10,616] ERROR [KafkaServer id=0] Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)
org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode = NodeExists
        at org.apache.zookeeper.KeeperException.create(KeeperException.java:126)
        at kafka.zk.KafkaZkClient$CheckedEphemeral.getAfterNodeExists(KafkaZkClient.scala:1904)
        at kafka.zk.KafkaZkClient$CheckedEphemeral.create(KafkaZkClient.scala:1842)
        at kafka.zk.KafkaZkClient.checkedEphemeralCreate(KafkaZkClient.scala:1809)
        at kafka.zk.KafkaZkClient.registerBroker(KafkaZkClient.scala:96)
        at kafka.server.KafkaServer.startup(KafkaServer.scala:308)
        at kafka. Kafka$.main(Kafka.scala:109)
        at kafka. Kafka.main(Kafka.scala)


Temporary solution

Stop zookeeper and Kafka, delete all files under the C:\tmp\kafka-logs\__consumer_offsets-42\ folder.





Previous:Under Windows, see the program that Java.exe belongs to
Next:Installation and deployment of Linux Redis source code

Related Posts
 Landlord| Posted on 7/6/2021 1:34:36 PM |
The kafka process hangs up from time to time.

1、ERROR Failed to clean up log for __consumer_offsets-30 in dir /tmp/kafka-logs due to IOException (kafka.server.LogDirFailureChannel)

2. Linux will clean the files in the /tmp directory on a regular basis, and my kafka log file directory is placed in the /tmp/kafka-logs directory, resulting in it being cleaned up regularly, so kafka will make an error when trying to read or add logs.
grep log.dirs /opt/kafka_2.12-2.3.0/config/server.properties
/tmp/kafka-logs
3. Modify
log.dirs=/opt/kafka_2.12-2.3.0/kafka-logs/
4. Restart kafka
5. Add the kafka log directory to the cleanup whitelist
The clearing of the /tmp directory under centos7 is responsible for the service systemd, and its related configuration files are in the /usr/lib/tmpfiles.d directory, we modify the configuration file tmp.conf and add the kafka log directory to it.
#防止删除kafka日志文件
X /tmp/kafka-logs

The cleaning of the /tmp directory in centos6 is achieved through tmpwatch, which relies on the timing of cron, and the scheduling file is /etc/cron.daily/tmpwatch
#防止删除kafka日志文件
X /tmp/kafka-logs
 Landlord| Posted on 7/12/2021 10:57:17 AM |
The Kafka Windows system will hang up after running for a period of time
https://www.itsvse.com/thread-9984-1-1.html
Disclaimer:
All software, programming materials or articles published by Code Farmer Network are only for learning and research purposes; The above content shall not be used for commercial or illegal purposes, otherwise, users shall bear all consequences. The information on this site comes from the Internet, and copyright disputes have nothing to do with this site. You must completely delete the above content from your computer within 24 hours of downloading. If you like the program, please support genuine software, purchase registration, and get better genuine services. If there is any infringement, please contact us by email.
Mail To:help@itsvse.com