January 2020 Summaries

In January 2020, Vespa announced several updates to its open-source big data processing and serving engine. New tensor functions have been added to enhance support for complex neural networks like BERT models, enabling more sophisticated data handling through operations like slicing and merging tensors. Vespa also introduced new sizing guides to optimize performance for applications with specific cost or performance needs and improved the performance of returning matched elements in large maps or arrays by making it 5 times faster. Additionally, a latency reduction was achieved for applications using boolean fields due to a new bitCount cache. A regression error from version 7.141 that could cause data loss or inconsistencies was identified and fixed in version 7.157.9, with users advised to upgrade to avoid potential issues. Developed primarily by Yahoo engineers and used in various Yahoo and Verizon Media products, Vespa encourages community feedback and contributions to continue its growth and evolution.

Organizations around the world, regardless of their industry or geographical location, typically progress through four levels of data maturity as they strive to leverage data for competitive advantage. The initial stage, Level 0: Latent, involves generating data during regular operations without its systematic use in decision-making. At Level 1: Analysis, data is analyzed and used by humans to inform decisions, often with the help of traditional data analytics tools. Level 2: Learning sees data being utilized to teach systems to make decisions automatically based on specified objectives, employing machine learning technologies. Finally, Level 3: Acting, involves real-time decision-making by systems that incorporate up-to-date data, often using advanced tools like Vespa for big data serving. Identifying and understanding an organization’s current data maturity level can help address challenges and guide the transition to more advanced stages, with future discussions exploring the drivers and benefits of achieving Level 3 maturity.