November 2023 Summaries

This blog post demonstrates how to build an open source chatbot using LangChain and Milvus in under 5 minutes. The process involves creating a retrieval augmented generation (RAG) stack with LangChain, which allows for answering questions about custom data while reducing hallucinations. The text is grounded on factual, custom data such as product documentation to ensure accuracy. The source code for the live chatbot is available on GitHub. The blog post also explains how to use Milvus, a high-performance vector database optimized for fast storage, indexing, and searching of embeddings or vectors. OpenAI's language models like GPT series are used in this process. Overall, the RAG retrieval and question-answering chatbot on custom documents is shown to be efficient and cost-effective as it allows free calls to data almost all the time for retrieval, evaluation, and development iterations, with only a paid call to OpenAI once for the final chat generation step.

SmartNews, a leading news app, faced the challenge of optimizing ad recommendations for its highly engaged user base. The company turned to Milvus after researching solutions that could handle high-throughput and low-latency queries. Milvus's vector similarity search capabilities were instrumental in optimizing SmartNews's dynamic ad vector recall. Adopting Milvus led to more relevant ads, increasing click-through rates and driving up ad revenue. The company has upgraded its Milvus to 2.2.4 and is looking forward to leveraging new features for building even more real-time and reliable systems.

The Open Source Advent is a project that aims to introduce participants to open-source software. For 25 days in December, one open-source project will be featured on social media along with a tutorial for quick start-up. Participants can earn points by starring the project's GitHub repo, creating repos using the project, and making posts tagging the company page. Extra points are awarded for writing a PR that gets merged or writing a blog about their experience. The top three scorers will receive swag packs from Zilliz and partners, as well as shoutouts on social media. Participants can join the Open Source Advent Discord Channel to submit their entries between December 26th, 2023, and January 2nd, 2024. Winners will be announced on January 8th, 2024.

Milvus is an open-source vector database designed for building AI applications using unstructured data embeddings. It provides four SDKs, including Java, Python, React, and Ruby. The text outlines the steps to install and start a Milvus server, connect to it, create a collection with schema and index, insert data into the collection, and query the collection. Additionally, it mentions using LangChain and Milvus for building chatbots in an upcoming blog post. Resources are provided to get started with Milvus and Zilliz.

Credal, an enterprise AI platform, aims to make Generative AI integration safer and more accessible for businesses. Their solution focuses on seamlessly integrating data from various sources while ensuring privacy and security. At the core of their offering is Milvus, an open-source vector database that enables efficient search, filtering, and data curation capabilities. Credal's architecture prioritizes high-quality data interpretations and effective communication with GenAI models. The platform offers observability and governance tools for administrators and IT teams, including features like PII redaction, audit logging, and data access controls. Milvus's scalability and robustness make it a game-changer for Credal, enabling them to deliver "Useful AI, made safe" to businesses worldwide.

Zilliz Cloud is now available on Microsoft Azure, expanding its presence across major cloud platforms including AWS and Google Cloud. This integration allows Azure-centric developers and enterprises to leverage the unique capabilities of Zilliz Cloud for vector database workloads. The move also signifies seamless access to Azure's cutting-edge AI services such as Semantic Kernel. Future enhancements include expansion into new Azure regions, availability on the Azure Marketplace, and continuous integration efforts to ensure data security and optimize application performance.

Zilliz Cloud has released the beta version of Milvus 2.3, introducing new features to enhance data management and querying processes for developers. The update includes Cosine similarity integration, Range Search feature, Upsert functionality, raw vector returns, JSON_CONTAINS filter, entity count, and more. Additionally, Zilliz Cloud has introduced enhanced enterprise features such as improved Role-Based Access Control (RBAC), expanded geographical options with the general availability of AWS EU Frankfurt region, and Self-Service Account and Organization Deletion feature. Furthermore, Zilliz Cloud is now available on Microsoft Azure in the azure-east-us region, completing its availability across major cloud platforms including AWS and Google Cloud. The company invites feedback from developers to shape the future of their vector database technology.

Zilliz has introduced Upsert, Kafka Connector, and Airbyte integration to enhance data flow efficiency in its vector database. Upsert simplifies the update process by inserting or updating data based on atomicity. The Kafka Connector enables real-time streaming of vector data from Confluent/Kafka into Milvus or Zilliz vector databases, enhancing capabilities for Generative AI and e-commerce recommendations. Airbyte Integration streamlines data transfer and processing in LLMs and vector databases, improving search functionality. These enhancements aim to improve search performance and streamline the entire data pipeline, making it more efficient and developer-friendly.

Milvus, the vector database system, has released versions 2.3.2 and 2.3.3 with significant improvements aimed at enhancing performance and user experience. The latest updates include support for array data types, complex delete expressions, integration of TiKV for metadata storage, FP16 vector type, and vector index MMAP. Other enhancements include a rolling upgrade experience, performance optimization, upgraded CDC (Change Data Capture), bulk insert of binlog data with partition keys, and the return of binary metric types such as SUBSTRUCTURE and SUPERSTRUCTURE. The developer community's contributions have been instrumental in shaping these updates, and feedback is welcome for future enhancements.

LangChain, an open-source library for LLM orchestration, recently added the "Self Query" retriever. This feature allows users to query vector databases like Milvus using LangChain. The implementation of this self-query retriever is covered in lines 189 to 233 of the base.py file in the self-query folder. The only class method for the self-query base class is from_llm, which has eight specified parameters and one allowing keyword arguments (kwargs). Four required parameters are llm, vectorstore, document_contents, and metadata_field_info. Other optional parameters include structured_query_translator, chain_kwargs, enable_limit, and use_original_query. The self-query retriever implementation involves parsing the self-query parameters, creating an LLM chain, and returning a self-query retriever. This feature enables users to build simple retrieval augmented generation (RAG) applications using an LLM, vector database, and prompts to interface with the LLM.

AWS re:Invent, one of the largest global cloud computing events, will take place in Las Vegas from November 27 to December 1, 2023. Zilliz invites attendees to visit their booth (#1339) and meet the team behind Milvus, a vector database solution. The event offers opportunities for innovative solution demos, problem-solving expertise, collaboration, community engagement, and swag. Attendees can also book private demos or meetings with Zilliz's experts to discuss specific projects or use cases. Additionally, users are invited to join the team for dinner to share project updates and feedback. Resources on vector databases are recommended for those interested in learning more before the event.

In this tutorial, we learn how to ground our Retriever- Augmenter-Generator (RAG) results using LlamaIndex and citations. We start by setting up the necessary libraries and environment variables for our chatbot. Next, we define the parameters of our RAG chatbot, including the embedding model, vector database, and data abstractions. Finally, we implement citations via LlamaIndex's CitationQueryEngine module to ensure grounded results. This tutorial uses Zilliz Cloud as a fully managed and optimized version of Milvus for persisting data across multiple projects.

OpenAI's built-in Retrieval feature in its Assistants API has some limitations, such as scalability constraints and lack of customization. These issues can be addressed by integrating a custom retriever powered by a vector database like Milvus or Zilliz Cloud. This approach allows developers to optimize and configure the retrieval process according to their specific needs, improving overall efficiency.

Milvus, an open-source vector database, has introduced a new feature called Range Search to enhance its similarity search capabilities. This feature allows developers to specify a distance range for relevant vectors in their searches, addressing limitations of traditional KNN searches in recommendation systems where results can be either too similar or too diverse. The technical architecture and usage guide for Range Search are outlined, along with details on when to use it over Top-K search. The feature is not limited to recommendation engines but has broader applications in areas like content matching, anomaly detection, and NLP search tasks. It is now available for public preview on Zilliz Cloud.

HackNC 2021, an annual hackathon event hosted by the University of North Carolina at Chapel Hill, saw over 1,300 registrations and 650 participating hackers. Zilliz, a data observability company, was represented during the event with a workshop and keynote speech. The winning project, "wellSpent," is an expense tracking app that provides users with a dynamic pie chart of their expenses, transaction lists, and various financial planning tools. Congratulations to the team behind wellSpent for their victory in the Best Use of Zilliz category.

CalHacks, a hackathon event held in San Francisco from October 27-29, featured over 1000 students from around the world participating in various projects. With $137,650 in prize money and sponsor awards, several innovative projects were awarded for their use of Milvus, an open-source vector database. The winning project, Second Search, utilized Milvus to search lecture videos by embedding video caption text into vectors and returning relevant sections based on user queries. Other notable projects included Jarvis, which described visual scenes to visually impaired users, an AI 911 agent that assessed emergency situations, and Mental Maps, a chatbot for mental well-being tracking.

Confluent, a data streaming platform, has announced the availability of its Kafka Connector for open-source Milvus and Zilliz Cloud. This collaboration enables seamless real-time vector data streaming from Confluent to Milvus or Zilliz vector databases, significantly enhancing real-time Generative AI powered by large language models (LLMs) like OpenAI's GPT-4. The integration of Zilliz and Confluent allows for continuous flow of real-time Confluent vector streams converted from unstructured data to be ported to Milvus/Zilliz, empowering developers to build applications for various use cases such as real-time semantic search, image/video/audio similarity search and retrieval augmented generation. The integration opens up possibilities for various sectors and applications, including enhancing Generative AI with a real-time knowledge base and optimizing personalized recommendations for e-commerce platforms.

Alexandr Guzhva, an expert in performance optimization, joined Zilliz to outcompete its competitors and fully utilize his expertise. With over 15 years of experience in finance and two years at Meta, he has contributed significantly to the FAISS library and written more than 2 million lines of code. Zilliz's focus on advanced similarity search methods and integration with NVIDIA Raft attracted him to the company. His goal is to improve Zilliz products and contribute to Milvus OSS, potentially applying his knowledge of ANNS for time series prediction in the future.

Troop, a tech company revolutionizing shareholder activism and engagement, leverages machine learning and AI technologies to enable investors to participate in corporate governance. Using the Milvus vector database, Troop built a solution that empowers individuals for collective financial activism in major corporations. The integration of Milvus enabled scalability, efficient handling of massive datasets, separation of storage and compute, rapid scaling of nodes, data partitioning, and improved semantic search capabilities. This infrastructure supports retrieval augmented generation (RAG) to process large volumes of unstructured data and build intelligent shareholder voting recommendation engines.