May 2026 Summaries

MCP Security and API Security, while sharing some surface similarities, are fundamentally different due to the unpredictable and natural language-based nature of MCP interactions, which traditional API security measures cannot adequately address. In conventional API security, predictable and structured client interactions are protected through established measures like OAuth, rate limiting, and endpoint verification, but these do not suffice for MCP, where agentic AI systems use natural language prompts and make autonomous decisions at runtime. This results in vulnerabilities like prompt injection, tool poisoning, and rug pulls, which arise from mutable tool semantics and unpredictable call graphs that cannot be foreseen or controlled through traditional security protocols. Unlike APIs with static authorization and fixed tool semantics, MCP requires a dynamic approach that considers agent intent and action-level granularity, necessitating a dedicated control plane designed specifically for MCP systems. Credal emerges as a solution by offering MCP-native observability and real-time threat detection, focusing on monitoring agent behavior and understanding the nuances of tool semantics and prompt-based interactions to prevent and analyze attacks effectively.

AI agent security is a pressing concern for organizations evaluating platforms, with most existing guidance focusing on abstract principles rather than actionable capabilities. A significant number of companies have experienced security incidents involving AI agents, highlighting a gap between monitoring these agents and controlling them effectively. Key capabilities to demand from platforms include per-agent identity for clear attribution, source-system permission mirroring to ensure agents only access what their users can, and action-level policy enforcement to specify action parameters. Additionally, protection against tool drift, a unified audit log for comprehensive oversight, continuous discovery of known and shadow agents, and rigorous production controls are essential for secure deployment. Without these features, organizations risk mismanagement and security breaches, underscoring the importance of demanding platforms that can demonstrate these capabilities convincingly during evaluations.

An initiative was undertaken to evaluate the quality of agent runs by combining human and AI assessments, specifically a council of LLMs, to identify when agents used tools effectively and completed tasks without unnecessary iterations or errors. Initial findings showed a 60% agreement between human evaluators and the AI council, which was insufficient to replace human judgment but highlighted systematic discrepancies rather than random errors. The disagreements were often due to rubric ambiguities, severity-threshold mismatches, or known failure modes. By identifying these issues, explicit rules were created to guide an AI adjudicator in resolving disagreements, resulting in improved alignment between council and human evaluations, especially in areas like tool policy. The process demonstrated that while LLMs can provide valuable insights, they require calibration against human judgments to ensure reliability. This calibrated dataset will be used to develop a multi-head encoder to evaluate and predict tool-call quality, aiming to enhance agent performance monitoring.