Company
Date Published
Author
Abhilash Mandaliya
Word count
1662
Language
English
Hacker News points
None

Summary

The Aerospike Kafka Inbound Connector allows streaming from Apache Kafka to Aerospike Database Enterprise Edition, enabling low-latency and high-throughput pipelines. The connector can be extended with a message transformer, which is a custom piece of code that transforms incoming Kafka messages into Aerospike records after performing operations or transformations on them. This feature enables users to perform complex operations on maps or lists in Kafka records before writing them as Aerospike records, develop custom code for functionality not natively supported by the connector, filter out sensitive data from records, create an Aerospike record with bins generated by tweaking Kafka message key/value, and more. The message transformer is thread-safe, easy to use, and flexible, allowing users to inject objects such as InboundMessageTransformerConfig and/or AerospikeReader. It can be deployed by copying the .jar file to the Aerospike Kafka Inbound Connector directory and configuring it in the `aerospike-kafka-inbound.yml` file.