September 2023 Summaries

Milvus 2.3 introduces GPU support, unlocking a 10x increase in throughput and significant reductions in latency. This strategic innovation is aimed at enhancing vector searching capabilities, particularly with the rise of Large Language Models (LLMs) like GPT-3. The integration of Milvus and NVIDIA GPUs allows for efficient searching through massive datasets and expands the AI landscape. To get started with the Milvus GPU version, users need to install CUDA drivers, configure Milvus GPU settings, build Milvus locally, and run it in standalone mode or using a provided docker-compose file.

LangChain, known for orchestrating interactions with large language models (LLMs), has introduced self-querying capabilities. This tutorial demonstrates how to perform self-querying on Milvus, the world's most popular vector database. The process involves setting up LangChain and Milvus, obtaining necessary data, informing the model about expected data format, and finally, performing self-querying. Self-querying allows an LLM to query itself using the underlying vector store, creating a simple retrieval augmented generation (RAG) app in the CVP framework.

Unstructured data, which makes up 80% of global data, is becoming increasingly prevalent. Vector embeddings are numerical representations used to work with unstructured data such as text, images, audio, and videos. They can be extracted from trained machine-learning models and have high dimensionality to store complex data. Vector embeddings are the de facto way to work with unstructured data, allowing for comparisons between data points. When generating embedding vectors, factors like vector size, training data quality, and quantity should be considered. Vector embeddings can be used to debug training data by detecting errors through clustering, finding samples not present in the training data, identifying hallucinations, and fixing errors in retrieval augmented generation (RAG). Additionally, they can be indexed, stored, and queried using vector databases like Milvus or Zilliz Cloud. The power of vector embeddings is evident from their wide range of use cases, making them a valuable tool for working with unstructured data in machine learning applications.

Zilliz Cloud has integrated with Confluent Cloud, allowing users of both platforms to access real-time data streams across their entire business for building AI applications. The integration enables the ingestion, parsing, and processing of real-time data into Zilliz Cloud using Confluent's Kafka producer and consumer APIs. This collaboration opens new avenues for leveraging Generative Artificial Intelligence (GenAI) in real-time scenarios, such as personalized responses and content generation platforms. The integration also enhances traditional AI use cases like recommender systems and anomaly detection. With easy access to data streams from across their entire business, Zilliz users can now create a real-time knowledge base, build governed, secured, and trusted AI applications, and experiment, scale, and innovate faster.

Milvus 2.3 has been released with significant updates, including the transformative upgrade of Knowhere 2.0. Key features of Knowhere 2.0 include support for GPU indexes, Cosine similarity, ScaNN index, ARM architecture, range search, optimized filter queries, code structure and compilation enhancements, MMap support, and retrieval of original vectors. These improvements aim to elevate Milvus's performance and user experience in vector databases.

This blog compares two vector databases, Weaviate and Zilliz/Milvus. While both are designed to manage vector data, they serve different needs. Weaviate is a strong choice for developers seeking quick and straightforward implementation, while Zilliz/Milvus excels in handling large-scale, high-performance, low-latency applications. The benchmark results show that Zilliz Cloud outperforms Weaviate Cloud in terms of queries per second (QPS), queries per dollar (QP$), and latency. Furthermore, a feature comparison reveals differences in scalability, functionality, and purpose-built features between the two vector databases.

In the first part of the Chat Towards Data Science blog series, we guide you through building a chatbot using your dataset as the knowledge backbone. We employ web scraping techniques to collect data for our knowledge base and store it in Zilliz Cloud, a fully managed vector database service built on Milvus. The tutorial covers creating a chatbot for the Towards Data Science publication, demonstrating how to prompt the user for a query, vectorize the query, and query the vector database. However, we discovered that while the results are semantically similar, they are not exactly what we desire. In the next part of this blog series, we will explore using LlamaIndex to route queries and see if we can achieve better results.

This guide explores the use of Pgvector, an extension of PostgreSQL that allows developers to store and query vector data. It covers setting up Pgvector, integrating it with PostgreSQL, using it for similarity searches, understanding its indexes and limitations, and comparing it with dedicated vector databases like Milvus and Zilliz. The article also discusses the advantages of using dedicated vector databases over traditional relational databases and provides benchmarking results to help developers choose the best solution for their projects.

Meta AI has released its open-source large language model (LLM), Llama 2, which is available for free use in commercial applications. It comes in three sizes and supports context lengths of up to 4096 tokens. Llama Chat, the fine-tuned model of Llama 2, has been trained on over 1 million human annotations and is specifically tailored for conversational AI scenarios. The performance of Llama 2 in answering questions was compared with that of ChatGPT, showing that both models excel at answering questions based on real-world knowledge. However, Llama 2 faces challenges maintaining answer quality when confronted with complex text formatting. Llama 2 stands out by not requiring high-end GPUs and can operate smoothly on desktop-level GPUs, especially after undergoing low-bit quantization.

Milvus 2.3, the latest version of the pioneering vector database, offers numerous enhancements and new features that make it an excellent choice for users looking to build applications ranging from recommendation systems and chatbots to artificial general intelligence (AGI) and retrieval augmented generation (RAG). The updated version balances performance, cost, and scalability while providing multiple deployment options. It also empowers developers with simplicity by enhancing its API and supporting data integration with other products. Furthermore, Milvus 2.3 ensures stability and second-level availability through improved system reliability features. Future updates will introduce additional cutting-edge features to enhance the user experience further.

This blog compares Elastic Cloud and Zilliz Cloud, two vector database cloud services. It delves into benchmarks to offer a performance perspective and performs an in-depth feature analysis of both platforms. The results show that Zilliz outperforms Elastic Cloud in terms of QPS, queries per dollar (QP$), and latency. Additionally, the blog highlights the features of each platform, such as scalability, multi-tenancy, data isolation, API support, and user interface/administrative console. It also provides a migration tutorial for moving from Elasticsearch to Zilliz Cloud.