March 2026 Summaries

Zilliz Cloud introduces Functions and Inference Services, designed to automate the embedding and reranking processes in AI search pipelines, which traditionally require manual embedding generation and separate reranking services. Now available in Public Preview for third-party models and Private Preview for Zilliz Hosted Models, these services allow users to insert raw text and conduct searches with natural language, while Zilliz Cloud manages embedding generation, vector storage, and result reranking. Functions are categorized into Pre-search Functions, which convert text into searchable representations, and Post-search Functions, which refine and reorder results. The Inference Services power these model-based Functions by calling models from third-party providers or utilizing Zilliz's GPU infrastructure for Hosted Models, ensuring data privacy and reducing latency. This integration simplifies the search workflow by eliminating the need for client-side embedding code and guaranteeing model consistency, ultimately enhancing search quality and operational efficiency for scalable applications.

In a comprehensive evaluation of embedding models for Retrieval-Augmented Generation (RAG) in 2026, ten models were tested across scenarios often overlooked by public benchmarks, such as cross-modal retrieval, cross-lingual retrieval, key information retrieval, and dimension compression. The study highlights Gemini Embedding 2 as the most versatile model, excelling in cross-lingual tasks and long-document retrieval but lacking in dimension compression. Qwen3-VL-2B, an open-source model, outperformed closed-source APIs in cross-modal tasks due to its smaller modality gap, while Voyage Multimodal 3.5 and Jina Embeddings v4 were noted for effective dimension compression. The CCKM benchmark introduced in the study aims to fill the gaps left by traditional metrics like MTEB by assessing models across multiple modalities and retrieval challenges. As the field rapidly evolves, the article emphasizes the importance of building custom evaluation pipelines tailored to specific data types and application needs to ensure optimal model selection.

In a comprehensive evaluation, Turbopuffer and Zilliz Cloud, two serverless vector databases, were compared for their performance and cost efficiency in multi-tenant AI applications. The study revealed significant differences in areas like search accuracy, query latency, and cost management. Turbopuffer, despite its attractive initial pricing model, struggled with search accuracy under multi-tenant filtering, resulting in lower recall rates compared to Zilliz Cloud, which maintained a consistent 0.99+ recall. Moreover, Turbopuffer exhibited higher cold query latencies across all tenant sizes and faced write ingestion stalls under load, while Zilliz Cloud demonstrated more stable performance with lower latencies and no interruptions. The billing structure of Turbopuffer, which charges based on the total namespace size rather than the actual data queried, led to unexpectedly high costs, especially at scale, whereas Zilliz Cloud offered free writes and queries, making it more cost-effective in the long run. Additionally, Turbopuffer's rate limiting posed challenges under high concurrency, affecting the scalability of applications reliant on it. These findings suggest that while Turbopuffer might appear cost-effective initially, Zilliz Cloud provides superior performance and scalability for production-scale multi-tenant vector search applications.

In evaluating Turbopuffer and Zilliz Cloud for enterprise readiness in multi-tenant vector searches, the focus extends beyond performance and cost to key compliance and operational criteria. Turbopuffer shows significant gaps in search correctness, delete consistency, and enterprise readiness, particularly under narrow filtering conditions where recall drops to 0.54, posing a compliance risk for industries with strict data quality requirements. It also struggles with unpredictable delete latency, stale metadata, and post-delete inconsistencies that could affect GDPR compliance. Unlike Zilliz Cloud, Turbopuffer lacks essential security certifications, private link support, and robust operational tooling such as real-time dashboards and automated alerts. Zilliz Cloud offers comprehensive features, including dedicated deployment options, extensive monitoring capabilities, and a range of security and compliance certifications, making it more suitable for enterprise environments that demand high data protection and disaster recovery standards.

At NVIDIA's GTC 2026, Zilliz and its open-source vector database, Milvus, were highlighted as central to the emerging infrastructure for unstructured data, which is becoming increasingly important for AI systems. Originally designed to handle large-scale similarity searches with GPU acceleration, Milvus has evolved to address the broader infrastructure challenges of integrating unstructured data into AI workflows, such as making data searchable in a meaningful way while managing costs and complexity. With the introduction of Milvus 2.6 and its AI Lakebase architecture, Zilliz aims to streamline data retrieval and storage by integrating vector retrieval directly with enterprise data lakes, thus addressing issues like data silos and iteration costs in AI systems. This evolution promises to enhance both the scalability and continuous improvement of AI infrastructure, making the handling of unstructured data more efficient and opening possibilities for future developments in AI applications.

Zilliz Cloud has announced the availability of Customer-Managed Encryption Keys (CMEK), allowing users to manage their own encryption keys through their cloud provider’s Key Management Service (KMS), thereby enhancing data security and compliance for enterprises in regulated industries. This feature addresses the limitations of platform-managed encryption by giving customers full control over their encryption keys, facilitating instant access revocation, and ensuring clear segregation of duties in alignment with various compliance frameworks such as GDPR and HIPAA. Especially pertinent for vector databases, which handle sensitive data like customer records and proprietary documents, CMEK mitigates security risks by ensuring that encryption keys remain within the customer's control, while Zilliz Cloud never possesses the master key. The implementation is designed to be straightforward, with minimal setup time and no downtime during key rotations. CMEK is currently available for dedicated clusters on AWS, with plans to expand to other regions and cloud providers, and it supports seamless integration within existing security monitoring infrastructures.

Context engineering is an essential process in developing robust production systems for language models, focusing on assembling, processing, and managing the contextual information these models require to generate accurate and efficient responses. It integrates techniques such as prompt engineering, retrieval-augmented generation (RAG), and multi-agent systems to handle the challenges of real-world data usage, including latency and cost issues. The process consists of foundational components, such as data retrieval, processing, and orchestration, and complex systems that combine these elements into complete applications. Key aspects include context retrieval and generation, context processing, and context management, with an emphasis on optimizing long-sequence processing through architectural innovations and optimization techniques. Additionally, self-refinement and adaptation strategies are employed to enhance model outputs through cyclical feedback mechanisms and meta-learning, while multimodal and relational context integration expands the model's capabilities to handle diverse data types and complex reasoning tasks.

Zilliz has introduced the Zilliz Cloud Plugin for Claude Code, aiming to integrate its cloud services seamlessly into AI developers' workflows. This plugin allows developers to manage various tasks such as cluster management, vector searches, and data operations directly from a terminal session using natural language commands. By leveraging Claude Code, developers can execute commands in plain English, streamlining the interaction with Zilliz Cloud's capabilities, including vector search, RBAC, and observability, without leaving their work environment. The platform is built on Milvus, a popular open-source vector database, and offers significant scalability, reliability, and security features suitable for AI workloads, supporting deployment across AWS, GCP, and Azure with minimal operational overhead. This initiative underscores a shift from traditional UI-based interactions to terminal-based workflows, emphasizing integration with AI coding agents to enhance developer productivity.

The AWS outage in the Middle East highlighted the vulnerability of cloud regions to physical damage and the critical need for robust cross-region disaster recovery strategies, particularly for vector databases that serve as long-term memory for AI applications. These databases, unlike stateless services, pose significant recovery challenges due to their stateful nature, with index rebuilds taking extensive time and effort. Zilliz Cloud has addressed this issue by becoming the first vector database to offer native cross-region disaster recovery, featuring automated failover, real-time replication, and a global endpoint that requires no application changes during region transitions. This solution ensures continuity and reliability, even in the event of regional failures, by allowing seamless failover and data salvage capabilities. As AI technologies become more integral to daily operations, the ability to maintain AI functionality during outages becomes increasingly crucial, making solutions like Zilliz Cloud's disaster recovery capabilities essential for minimizing downtime and preserving data integrity.

Claude Code, an AI tool for handling complex coding tasks, can now be operated remotely via mobile devices using three different methods: Anthropic's official Remote Control, the community-built Happy Coder app, and a hacker setup involving SSH, Tailscale, and tmux. Remote Control offers a simple, subscription-based solution with minimal setup, while Happy Coder provides a free, open-source alternative with features like push notifications and voice input, supporting both Claude Code and Codex. The SSH method, although requiring more technical know-how, allows for free, full control without reliance on third-party applications. These methods enable users to approve actions and manage coding tasks on the go, addressing the challenge of being tied to a desk while Claude Code runs long processes. Additionally, tools like Typeless, memsearch, and cc-tmux-worktree-orchestration enhance the mobile coding experience by improving input efficiency, maintaining session context, and enabling parallel task management. The evolving landscape of AI coding tools reflects a broader trend of decoupling from traditional desk setups, making mobile workflows increasingly viable.

In an analysis of serverless vector databases, James Luan highlights the challenges and unexpected costs associated with using Turbopuffer for a code assistant product, revealing how the optimistic pricing calculator significantly underestimated actual expenses due to billing structures that penalize large datasets and uneven tenant sizes. The blog post explores the inherent performance limitations of Turbopuffer's architecture, such as the reliance on S3 storage, which leads to cold start latency issues and scalability constraints due to the SPFresh index. Turbopuffer's design as a search index rather than a full-fledged database results in recall and consistency challenges, particularly under filtering and multi-tenant scenarios. Luan contrasts this with Zilliz Cloud's approach, which utilizes an NVMe caching strategy and filter-aware indexing to maintain high performance and predictability, emphasizing the importance of evaluating infrastructure choices with realistic production-scale testing to avoid unforeseen operational and financial consequences.