Building a Simple, Pure-Rust, Async Apache Kafka Client

In the development of InfluxDB's IOx, a new asynchronous Kafka client called RSKafka was created to replace rust-rdkafka. The reasons for this change include complexity, bindings limitations, buffering/modularity issues, expertise and insights concerns, and feasibility considerations. RSKafka is fully-async and uses the Rust standard library for reading and writing integers in big-endian encoding, with a focus on modularity and flexibility. Testing involved serialization roundtrips, serialization snapshots, integration tests, unit tests, and fuzzing to ensure spec compliance and edge case coverage. The client provides features such as batching, custom aggregation algorithms, and extensions through traits, making it suitable for various use cases including IOx. RSKafka is deliberately minimal in its current form but open to community contributions and is available on GitHub.

No tracked trend matches for this post yet.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.