May 2025 Summaries

Vectorize has achieved SOC 2 Type II compliance, reflecting its commitment to data security and system reliability by adhering to the industry-standard framework established by the AICPA. This compliance demonstrates Vectorize's ongoing dedication to the five trust principles: security, availability, processing integrity, confidentiality, and privacy, verified through months of independent third-party auditing. By ensuring secure data handling from ingestion to retrieval, Vectorize addresses key concerns for regulated companies and enterprises, thus facilitating adoption and procurement. The company partnered with Prescient Security to audit and validate its security measures, which include implementing internal policies, automating infrastructure, conducting vulnerability scans, and enforcing access controls. As Vectorize continues to grow, it remains dedicated to maintaining and improving its security practices, and invites current and potential customers to discuss their security needs and obtain the SOC 2 Type II report as part of its commitment to trust and transparency.

Automatic Metadata Extraction is a new feature in Vectorize that significantly enhances the handling of unstructured documents in Retrieval Augmented Generation (RAG) pipelines by automatically extracting structured information. This feature uses the Iris model to analyze documents and apply predefined schemas, thereby improving retrieval capabilities, providing enhanced context for language models, and organizing documents more effectively. It supports two types of metadata: document metadata, which provides high-level information like title and author, and section metadata, which offers detailed data like part numbers and technical specifications. The feature is particularly beneficial in sectors such as financial services, manufacturing, and healthcare, where it aids in classifying documents and extracting specific data points. With a visual schema editor, users can easily create or generate schemas without needing to write JSON. By integrating extracted metadata into text chunks, the system improves retrieval quality and ensures consistent information availability. This advancement allows organizations to gain deeper insights and provide more precise information to users, enhancing the value derived from their document collections.

Accurate AI agent performance hinges not just on model quality but on the effectiveness of data retrieval systems that supply the model with pertinent information. While developers often rely on general-purpose semantic search endpoints expecting models to autonomously gather necessary data, this approach can yield inconsistent outcomes due to insufficient context retrieval. The solution lies in building structured retrieval systems that differentiate between operational data and unstructured knowledge, utilizing semantic search as a foundation for retrieval augmented generation (RAG) systems. By enriching vector data with structured metadata, particularly in complex domains like legal documents, agents can more precisely access relevant data, improving context accuracy and response quality. Tools like Vectorize streamline this process by allowing developers to define document schemas and metadata, thus enhancing semantic search results and enabling efficient, scalable AI projects.