November Feature Roundup
Blog post from LanceDB
Lance 0.19.2 introduces significant enhancements, including flexible data handling for inserts, an experimental storage type for managing large video columns, and full-text search capabilities that now include not-yet-indexed data. The release allows for more adaptable schema appending, enabling users to insert data with varying field order and nullability without disrupting ETL processes, thus facilitating schema evolution. The new balanced storage feature addresses the challenge of storing large unstructured data like videos by separating them into a "blob" storage class, optimizing performance for small columns even within datasets containing large data types. Additionally, full-text search now immediately incorporates newly added data, although index updates are still necessary for unmatched terms to optimize search results. LanceDB's subsequent release, version 0.13.0, ensures feature parity across its Python, Node, and Rust SDKs, incorporating all features from Lance 0.19.2 and streamlining the remote client implementation for consistent functionality.