Company
Date Published
Author
Simon Bumm
Word count
1269
Language
English
Hacker News points
None

Summary

Taktile, a low-code platform used by global credit and risk teams to design, build, and evaluate automated decision flows at scale, processes millions of daily tracking events, which need to be anonymized and loaded into its Snowflake Data Warehouse. To achieve this, Taktile chose dlt (data loading tool) as the instrumentation engine, running it on AWS Lambda due to its secure, scalable, and lightweight nature. dlt's ability to automatically schema detect, normalize unstructured data, and provision staging destinations makes it a powerful tool for event ingestion. By using AWS SAM, a lightweight Infrastructure-As-Code framework, Taktile can easily deploy and manage their resources. However, there are caveats to be aware of when setting up dlt on AWS Lambda, such as handling local files and database secrets. Despite these considerations, dlt and AWS Lambda form a great foundation for building a production-grade instrumentation engine.