March 2023 Summaries

Weaviate is a cloud native search engine that supports Prometheus metrics publishing, Kubernetes liveness and readiness checks, environment variable configuration, and simplified deployment via Helm charts. To integrate Weaviate with an existing observability stack, one can use Grafana agent or Datadog agent to scrape these metrics. Key metrics to monitor include heap usage, batch latency, object latency, and query latency and rate.

OpenAI's ChatGPT has taken the world by storm with its ability to generate human-like responses based on a vast general knowledge base. Large language models (LLMs) like GPT-4 are voracious readers, having read everything from Wikipedia to internet content. They build statistical models of language by measuring which words are often used together across different types of texts and learning higher level concepts. However, current limitations prevent them from fully understanding deeper meanings of language or solving mathematical problems. Efforts are being made to improve these models' comprehension capabilities.

In this blog post, an alternative encoding technique called Tile encoder is presented as a distribution-based encoder for compressing vectors in memory. Unlike KMeans, which requires fitting data to centroids and calculating distances to all centroids, the Tile encoder leverages the known underlying distribution of the data beforehand. Results show that recall results are very similar when using KMeans vs Tile encoder, with significant improvements in fitting and encoding times for the latter. The Tile encoder also allows for quick compressing without downtime, making it a promising alternative to KMeans for vector compression tasks.

Weaviate has introduced vector compression algorithms in its latest version v1.18, aiming to offer similar performance at a fraction of memory requirements and cost. The main goal is to balance recall performance and memory management. Product Quantization (PQ) is the chosen compression algorithm for vectors. Experiments on datasets like Sift1M, Gist, and DeepImage96 show that PQ can significantly reduce memory usage while maintaining acceptable recall rates and latency. Weaviate's HNSW+PQ feature allows HNSW to work directly with compressed vectors, improving memory efficiency without sacrificing performance.

Weaviate 1.18 has been released with new features, performance improvements, and fixes. Key updates include faster filtering using Roaring Bitmaps, improved HNSW-PQ for better performance at lower RAM requirements, enhancements to BM25 and Hybrid Search, replication features like tunable consistency and repair-on-read, a Cursor API for systematic object retrieval, and an Azure Storage backup module. These updates aim to improve the scalability, speed, and reliability of Weaviate as a vector database.