Company
Date Published
Author
Olivia Greene, Ahmed Saef Zamzam, Kai Waehner, Prabha Manepalli, Weifan Liang
Word count
1282
Language
English
Hacker News points
None

Summary

Apache Kafka and Apache Mesos are two well-known projects that have undergone significant changes since the last blog post from Confluent in July 2015. The Microservices paradigm offers benefits like autonomous teams and agile development, but also creates challenges such as increased communication between distributed instances and the need for orchestration. Confluent's resources can help build a scalable microservice architecture using Kafka, its Streams API, and Mesos. Apache Mesos is a cluster manager that provides efficient resource isolation and sharing across distributed applications or frameworks, while DC/OS is a Mesosphere-backed framework on top of Apache Mesos. A practical example shows how to set up a Kafka and microservices infrastructure on DC/OS, leveraging the benefits of both systems. The Kafka Streams API allows for lightweight but scalable stream processing logic, which can be added to any JVM application or microservice. Mesosphere's DC/OS enables management of both Kafka infrastructure components and Kafka Streams microservices, providing automated provisioning, upgrading, elastic scaling, unified management, and monitoring.