Crafting Our Full-Text Search in SurrealDB: A Journey Beyond Lucene and Tantivy

SurrealDB's journey to develop a full-text search capability was driven by the need to create a user-friendly solution without compromising the integrity of its ACID SQL database. Initially considering Lucene-based solutions and the Rust-based Tantivy, SurrealDB faced challenges such as language inconsistency and index management in distributed environments. The absence of inherent ACID compliance in these technologies led SurrealDB to build its own search system to maintain atomicity, consistency, isolation, and durability, ensuring that transactions involving full-text searches are reliable and consistent. This development not only enhanced search functionality but also preserved the database's integrity, crucial for applications relying on data accuracy. Looking forward, SurrealDB plans to introduce vector search capabilities to further advance its database technology, demonstrating a commitment to innovation and user experience.