February 2024 Summaries

Large Language Models (LLMs) and Generative AI are revolutionizing business operations, as demonstrated by Walmart's innovative use of these technologies to enhance e-commerce search functions through semantic caching. Rohit Chatter, Walmart's Chief Software Architect, emphasizes that semantic caching allows for more nuanced understanding of customer queries, significantly improving the efficiency of search results by handling variations in phrasing and reducing zero search result queries. Despite challenges related to latency and cost, Walmart is strategically combining traditional and semantic caching methods to optimize performance and reduce costs. Chatter envisions a future where generative AI, augmented reality, and virtual reality converge to offer unprecedented shopping experiences. Walmart's commitment to integrating these advanced technologies reflects its role in redefining e-commerce and enhancing customer interactions, marking a significant step forward in the seamless blending of technology and retail.

Rohit Chatter, Chief Software Architect at Walmart Tech Global, participated in a fireside chat with the LLMs in the Prod community, discussing Walmart's transition to Generative AI and semantic caching in its retail operations. The conversation covered Walmart's shift from traditional NLP to Generative AI models, such as BERT-based Mini LM V6, to improve e-commerce search by handling complex and contextually relevant product groupings and enhancing product recommendations. The company fine-tunes models like MiniLMv2 and T0 with customer engagement data to enhance search relevance for ambiguous queries, using techniques like Approximate Nearest Neighbour search for relevance matching. Walmart also implements semantic caching to cluster queries based on conceptual similarity, achieving about a 50% cache hit rate. Challenges include reducing search latency and plans for future developments in personalization, voice, and visual search. The session highlighted Walmart's strategies for improving customer experience and long-term ROI from Generative AI implementations.