Lance × DuckDB: SQL for Retrieval on the Multimodal Lakehouse Format

Lance is an open-source lakehouse format designed to manage multimodal AI data by addressing the fragmented nature of current retrieval systems, where embeddings, text, metadata, and analysis layers are often separately stored and processed. It spans the file, table, and catalog layers of the lakehouse stack to ensure data stability and scalability. The introduction of the Lance × DuckDB extension allows users to seamlessly perform retrieval and analytics tasks using SQL within DuckDB, a portable SQL query engine. This extension makes it possible to run vector, full-text, and hybrid searches directly on Lance datasets and supports operations such as joining, aggregating, and materializing results without leaving the DuckDB environment. By integrating with cloud services and object storage, the extension provides an efficient solution for retrieval-augmented generation (RAG) systems and multimodal data management, allowing teams to iterate quickly and manage data artifacts effectively without the need for repeated data copying.