Lance File 2.1 is Now Stable
Blog post from LanceDB
The Lance file format has released its stable 2.1 version, offering compression benefits without compromising random access performance, addressing a limitation from version 2.0. The update introduces cascading encoding, reduces IOPS when reading nested data, and includes features like distinguishing null structs from null values, along with optional repetition index caching. While the upgrade is not mandatory, it could benefit workflows that frequently scan smaller columns. Users are advised to upgrade to Lance version 0.38.0 or higher to ensure compatibility when working with 2.1 files, as older versions may not fully support the new format. Future developments, including version 2.2, aim to simplify migration processes, enhance compression, and improve support for struct packing and JSON encoding. Users are encouraged to join the community on Discord and GitHub to contribute and discuss further improvements.