April 2025 Summaries

The blog post explores the development of a sophisticated retail AI assistant using Vespa and LangGraph to enhance customer interactions through agentic AI applications. It highlights the complexity of Retrieval Augmented Generation (RAG) systems, illustrating how basic systems perform LLM inference on search results, whereas advanced versions incorporate query analysis, chain-of-thought planning, and high-quality retrieval to deliver comprehensive answers. The implementation focuses on a retail chatbot assistant that transforms user queries into precise queries based on conversation history, leveraging Vespa's real-time update features to prevent recommending out-of-stock items, thus maintaining customer satisfaction and maximizing conversion. The architecture employs Vespa for backend retrieval, LangGraph for managing complex workflows, Streamlit for UI frontend, Tavily for web searches, and OpenAI’s GPT-4o-mini model for LLM processes. The system can autonomously address inquiries through its knowledge base, Vespa queries, or web searches, demonstrating the potential of combining LangGraph and Vespa in creating an intuitive AI workflow, ultimately offering a personalized and contextually aware virtual retail assistant experience.

Vespa, an advanced search platform, offers compelling features for those familiar with Solr, including tensor support for vector search, integration with embedding models, flexible ranking, and scalability through virtual buckets. Vespa's architecture allows for high performance, especially in complex queries, by trading off some initial write speed for faster query responses, which is advantageous when handling multiple vectors or semantic search. Unlike Solr, Vespa requires an application package for configuration, aligning with CI/CD practices, and supports real-time writes without the need for soft commits. Vespa's ranking capabilities are robust, allowing for complex functions, ML models, and hybrid search approaches, while its faceting is managed via grouping, similar to Solr's JSON Facet API. Vespa also excels in vector search, using tensors for semantic searches, enabling hybrid search by combining vector and lexical scores. Although Vespa lacks some dynamic APIs and tools found in Solr, it provides a strong foundation for scalable and efficient search, appealing to those interested in integrating AI-driven search capabilities.

As AI search requirements expand, companies must choose between vector databases, search platforms with built-in vector capabilities, and cloud databases with vector add-ons, with each option impacting search accuracy, scalability, and costs differently. While vector databases excel in similarity searches, they often lack comprehensive search features such as ranking and filtering, making traditional search platforms with native vector support a more balanced choice for performance, flexibility, and cost-efficiency. Search platforms like Vespa.ai integrate vector search with advanced ranking, filtering, and scalability, providing a more robust solution for AI-powered search by combining vector similarity with precise search capabilities. Vespa.ai's success is exemplified by Vinted's migration, which improved search performance and reduced costs. Ultimately, while vector databases have their niche uses, a hybrid search platform that incorporates both vector and traditional search elements offers the most effective solution for enterprise AI search needs.

Perplexity, a leading AI Search company, enhances its AI capabilities by leveraging Vespa.ai to build scalable and reliable search solutions. The effectiveness of AI models is heavily reliant on the quality of information retrieval, which is why Perplexity focuses on developing robust search functionalities. By using Vespa.ai, Perplexity indexes billions of web pages and user files, storing data in both textual and vector formats, along with structured metadata, to ensure comprehensive coverage and real-time updates. This enables the AI to analyze user queries, execute precise searches, and feed relevant information to large language models for accurate responses. The process demands high-speed operations across distributed systems, supporting many concurrent experiments and ensuring operational resilience. This partnership with Vespa.ai allows Perplexity to continuously improve its search engine's performance and problem-solving capabilities.

E-commerce is rapidly evolving, and traditional search systems are struggling to meet the demands of growing product catalogs and customer expectations for real-time, personalized experiences. The webinar "Unlock the Future of eCommerce" on May 8 will showcase how Vespa, an AI-powered, cloud-native platform, is transforming the sector by integrating search, recommendations, and real-time personalization to offer tailored shopping experiences. Companies like Vinted have successfully transitioned from Elasticsearch to Vespa, experiencing significant improvements in search latency, indexing time, and infrastructure costs, which in turn have driven business value. Originally developed for Yahoo!, Vespa now serves as an open-source platform and a managed cloud service, providing retailers with a unified, AI-native approach to delivering advanced customer experiences at scale. The webinar will feature insights from Vinted's journey and offer practical takeaways and architectural insights for retail leaders looking to innovate and move beyond the limitations of legacy search systems.

The blog post provides a guide on using Logstash to transfer data into Vespa by leveraging the Vespa Output plugin's detect_schema mode, which can automatically generate a Vespa application package from existing data. The tutorial outlines the process for both local Vespa instances and Vespa Cloud, detailing steps such as downloading and configuring Logstash, installing the necessary plugin, and deploying the application package. For local Vespa, it describes running Logstash to create and deploy the application package, while for Vespa Cloud, it involves additional steps like creating a tenant and application, generating mTLS certificates, and using the Vespa CLI for deployment. The post emphasizes the flexibility of Logstash for various data sources and offers resources for further exploration, including Vespa CLI commands and a free trial for deploying applications on Vespa Cloud.

Vespa has introduced a new capability for document enrichment using Large Language Models (LLMs) to enhance search applications by transforming raw text into structured data and adding contextual information, thereby improving search relevance. This process traditionally required dedicated natural language processing pipelines or third-party APIs, but with LLMs, tasks such as named entity extraction, categorization, keyword generation, anonymization, translation, and summarization can be accomplished without custom code. Vespa supports both local LLMs and external OpenAI-compatible APIs, and has integrated a new indexing expression called "generate" that uses LLMs during document ingestion to create enriched fields for search without additional latency. This approach differs from retrieval-augmented generation (RAG) as it does not rely on LLMs at query time but rather enriches documents during ingestion, allowing them to be more effectively indexed and searched. The performance and cost of document enrichment depend on the chosen LLM, with smaller models offering cost-effective solutions for less complex tasks. Vespa's document enrichment is scalable and can be extended with custom components for specific applications, offering a practical and powerful method for large-scale document enrichment in search applications.

In the Vespa Newsletter for April 2025, several updates and enhancements to Vespa's AI-driven search infrastructure are highlighted, including a significant boost in lexical search query performance, the introduction of Pyvespa Relevance Evaluator, and a global-phase rank-score-drop-limit feature. The newsletter discusses the integration of compact tensor representation for improved performance and showcases new applications like video search and retrieval with TwelveLabs, as well as an agentic Streamlit chatbot. It also promotes Vespa Voice, a new podcast, and highlights upcoming AI-related events featuring Vespa team members. The updates aim to enhance Vespa's capabilities in handling large-scale data retrieval and processing, blending classic information retrieval techniques with modern AI methods to optimize search performance and relevance.