August 2024 Summaries

Chalk recently won Best Technology at The GenAI Collective's Demo Night, where Elliot showcased a tool that merges structured data with large language model (LLM) analysis in a unified feature store. This was demonstrated by ingesting financial transactions from a traditional database and employing Gemini to analyze unstructured memo lines for retrieving merchant categories, cleaning memos, and extracting additional insights. The implementation of this feature pipeline involves using Python and the Chalk framework, which simplifies prompt engineering by handling structured data and caching responses to reduce API costs. The demo, which can be found on GitHub, highlighted Chalk's capability to facilitate the creation of feature pipelines, and the event was celebrated alongside other members of the ML/AI community in San Francisco.

In the blog post, the authors discuss the integration of PartiQL, a SQL dialect for DynamoDB, with Chalk, a machine learning platform that allows users to define features in Python. The main challenge addressed is PartiQL's limitation of not supporting the "AS" aliasing feature in SQL SELECT clauses, which is crucial for maintaining consistency and ease of use within Chalk's data extraction processes. The authors explore several solutions, including renaming features or creating metadata comments, but ultimately opt for enhancing PartiQL to support aliasing using abstract syntax trees (ASTs) and DuckDB's SQL parser. This approach allows for parsing, modifying, and executing PartiQL queries with aliasing, enabling Chalk to offer a seamless experience for querying DynamoDB data sources. The solution involves parsing queries, editing ASTs, and applying aliases to query results, thereby maintaining performance and user experience without altering existing infrastructure. The authors emphasize the importance of supporting diverse data sources for Chalk's feature engineering capabilities and invite interested individuals to explore career opportunities at Chalk.