SimpleKafka — Kafka-like Distributed Messaging System
Academic Project | August 2026
A Kafka-like distributed messaging system built from scratch using Java, Maven, Apache ZooKeeper, and distributed-systems concepts. The project implements brokers, persistent storage, client libraries, producer and consumer APIs, topic partitioning, replication, broker coordination, and a client-broker binary communication protocol.
Key Outcomes
01 Built core components including brokers, persistent storage, client libraries, and producer/consumer APIs.
01 Achieving 79 messages/sec with core components like brokers and client libraries
02 Created topic partitioning and replication processes that improved system reliability, managed by ZooKeeper, enhancing throughput by 13%
03 Added leader/follower mechanics for replicated topics.
04 Used Apache ZooKeeper for broker coordination and discovery.
05 Engineered persistent message storage utilizing sequential log segments, optimizing retrieval speeds by 5%
06 Implemented a client-broker binary protocol with request/response communication.
07 Added broker discovery functionality for producer and consumer clients.
Java
Maven
Apache ZooKeeper
Distributed Systems
Message Queues
Binary Protocols
Persistent Storage
Topic Partitioning
Replication
Producer-Consumer Architecture