Vespa Product Updates, May 2019: Deploy Large Machine Learning Models, Multithreaded Disk Index Fusion, Ideal State Optimizations, and Feeding Improvements

Vespa, an open-source big data processing and serving engine developed primarily by Yahoo engineers, has released several updates in May 2019 aimed at enhancing its performance and usability. These updates include multithreaded disk index fusion, which allows content nodes to sustain higher feed rates by utilizing multiple threads, and improvements to cluster-internal communications for high throughput feeding operations by fully utilizing a 10 Gbps network. Additionally, optimizations to the ideal state calculations reduce the impact on read and write operations during state transitions. Vespa now also supports downloading machine learning models during deployment by referencing them via URL, which is ideal for applications where models are frequently updated externally. The Vespa team encourages community feedback and contributions to further evolve the platform.