June 2026 Summaries

Merlin is introduced as an agentic intelligence layer for Encord, aiming to enhance AI data infrastructure by integrating with existing tools to streamline data management processes. This innovative layer, launching in beta, is designed to facilitate the entire data lifecycle, from setting up labeling schemas and review stages to optimizing model performance by identifying and resolving data issues. Merlin allows users to manage their AI data infrastructure through conversational interfaces, reducing manual efforts traditionally needed for preparing raw data for training. Early access to Merlin is being offered to select customers via MCP, with plans to expand its integration to platforms like Slack and other agentic coding environments. This marks a significant step in Encord's strategy to incorporate agentic intelligence into its platform, inviting interested parties to participate in shaping its future development.

Encord has integrated NVIDIA's Cosmos Reason 2 and Cosmos Embed models into its platform, enhancing its capabilities for processing and analyzing video data. Cosmos Reason 2 automates the pre-labeling of physical AI video by generating natural-language descriptions of actions, objects, and scene contexts, which assists annotators by providing a starting point for refinement, thereby streamlining the labeling process. Cosmos Embed, on the other hand, enables video datasets to be searchable by behavior rather than just scenes, using action-aware embeddings that facilitate natural-language queries to identify specific behaviors within video clips. This integration allows teams across various fields, such as robotics, autonomous vehicles, and industrial inspection, to efficiently generate structured, searchable training data by focusing on relevant actions and scenarios, ultimately improving model training and reducing the time spent on manual labeling.

Security video annotation involves labeling surveillance footage to train computer vision models for detecting people, vehicles, objects, and events relevant to safety and security, which is more challenging than general video annotation due to the continuous scale, real-time requirements, adversarial edge cases, and privacy concerns. This process is crucial for AI security use cases like intrusion detection, retail loss prevention, crowd monitoring, and traffic analytics, requiring specific annotation types such as bounding boxes, polygons, and segmentation masks. The workflow includes defining objectives, curating footage, setting up annotation, applying AI-assisted labeling, reviewing, and iterating with model feedback, while overcoming specific challenges like low light, occlusion, motion blur, re-identification, and class imbalance. Compliance with regulations like GDPR and biometric data laws is crucial, necessitating tools with access controls, audit trails, and encryption to ensure data security and privacy. Encord provides a video-first AI data platform designed for handling long and complex security footage, offering features like automated tracking, multimodal support, customizable QA workflows, and compliance with enterprise security standards.

Labeling data for machine learning is a complex, ongoing process that requires a structured workflow to ensure consistency and accuracy, which directly impact model performance. Rather than being a one-time task, data labeling involves defining an ontology, curating datasets, deciding on a labeling model, utilizing AI-assisted pre-labeling, and establishing quality assurance (QA) and inter-annotator agreement processes. It is crucial to version labels and use active learning to continuously improve model performance. Teams often encounter issues such as ontology drift, annotator inconsistency, and inadequate QA capacity as they scale, highlighting the importance of treating data labeling as an engineering problem with defined stages and checkpoints. Choosing a suitable platform that supports the entire labeling workflow, rather than just individual tasks, can help avoid common pitfalls and enhance the efficiency and quality of the labeling process.