Why Apache Kafka is dropping ZooKeeper

Change is coming for users of Apache Kafka, the leading distributed event-streaming platform. Apache ZooKeeper, Kafka's metadata management tool, will soon be phased out in favor of internal technology.

Colin McCabe, a member of the Apache Kafka project management committee and an engineer at Confluent, which leverages Kafka, explained the reason for the change. ZooKeeper stores persistent cluster metadata and handles maintaining dynamic configurations and topics, as well as partitions within the topics, he said. But ZooKeeper adds an extra layer of management. Storing metadata internally within Kafka will make managing it easier and enable better guarantees around issues such as versioning, according to McCabe.

To read this article in full, please click here