Skip to main content


Showing posts from June 7, 2020

Kafka: Introduction to core concepts

Apache Kafka was developed by LinkedIn and donated to Apache. Apache Kafka is a distributed streaming platform that can handle high volume of data. Pull or Push? I initially misunderstood Kafka as a push based messaging system. However Kafka has chosen traditional pull approach. In Kafka, data is pushed to the broker by producers and pulled from the broker by the consumers. IMAGE1 Why Kafka? Kafka is a reliable messaging system which is fast and durable. We can list it's benifits as; Scalable - Kafka's partion model allows data to distributed across multipel servers, making it highly scalable.  Durable - Kafka's data is written to disk making it highly durable agaisnt server failures. Multiple producers - Kafka can handle multpile producers which publish to the same topic. Multiple consumers - Kafka is designed so that multipel consumers can read messages without interfering with each other. High performance - All these features allows high performace distributed messaging