February 2026 Summaries

Arcade Evals is a system designed to evaluate the effectiveness of tool definitions for Machine Learning Models (LLMs) in simulated environments, similar to how language instruction uses role-playing to prepare students for real-world interactions. By creating role-play scenarios where the LLM acts as a student, Arcade Evals allows developers to test the clarity and usability of tool definitions without executing real API calls. This method ensures that tools can be selected and populated with the correct arguments by the model, providing feedback on whether the tool descriptions are intuitive and aiding in iterative improvements. The evaluation framework uses rubrics to score tool performance, aiming to improve agent reliability by ensuring tools work well across diverse model capabilities. While it does not validate the actual execution of tools, this approach allows for safer and more cost-effective testing of tool schemas and definitions before deploying them to production environments.

Arcade.dev Evals is a framework designed to test whether large language models (LLMs) can correctly select and use MCP tools based on well-defined tool definitions, focusing on their practical application. The text highlights the importance of crafting high-quality tool definitions, emphasizing that they should not be treated like function signatures but more like detailed menu items that guide LLMs in selecting the right tool and formatting inputs correctly. Proper tool definitions, which include clear names, concise descriptions, and specific parameter formatting, significantly enhance the performance of LLMs by reducing ambiguity and token consumption during retries. The text provides examples of vague versus descriptive tool definitions and demonstrates how descriptive versions perform better in tests. Arcade Evals is built into the Arcade CLI and offers a method to evaluate MCP tools' effectiveness across multiple models without executing tools, ensuring that LLMs can accurately match and fill in tool parameters.

Paul Asjes, DevX Lead at ElevenLabs, is pioneering a shift in developer education by focusing on educating agents, which are increasingly responsible for code generation. He identifies a challenge where agents, trained on static datasets, can generate outdated code, adversely affecting the developer's experience. Asjes addresses this with a three-part strategy: teaching agents to reason about products, providing deterministic MCP tools for function execution, and offering MCP apps for user-friendly interfaces. His approach highlights that agents require concise, well-curated instructions rather than exhaustive documentation to perform effectively. ElevenLabs has developed "agent skills," standardized by companies like Microsoft and OpenAI, to guide agents in specific tasks while avoiding errors. Asjes differentiates between skills, which teach reasoning, and MCP tools, which ensure correct execution. Furthermore, ElevenLabs' MCP apps extend the agent's capabilities to non-technical users through intuitive interfaces. Despite these advancements, challenges persist in discovery, versioning, and standardization among agent skills, prompting Asjes to envision a future where managing agents becomes a central role in software development, potentially transforming the landscape of developer experience.

Arcade.dev offers a robust security framework for AI agent tool execution through a system called Contextual Access, which integrates into its MCP runtime. This framework addresses common enterprise challenges in deploying AI agents by replacing traditional service accounts and credentials with a three-layer security model: policy enforcement, scoped tool access, and contextual access. The first two layers ensure agents operate within the permissions of a user's existing identity and access only relevant tools, while the third layer—Contextual Access—allows for the injection of custom security logic at critical moments in the execution pipeline via webhooks. This approach provides enterprises with the ability to enforce complex security policies through three distinct hook points: Access, Pre-Execution, and Post-Execution, each of which can validate, modify, or block tool interactions in real time. The system enhances AI agent security by allowing organizations to implement tailored security measures without introducing new credentials or approval cycles, offering flexibility through webhooks that support multiple configurations and failure modes. Contextual Access is designed to be both sophisticated and adaptable, accommodating enterprise-specific security and compliance needs while ensuring AI agent deployments remain secure and manageable.

WebMCP, initiated by Alex Nahas at Amazon and now being developed as a W3C standard, aims to revolutionize how AI agents interact with websites by providing a structured alternative to the traditional visual and semantic methods. Through WebMCP, websites can expose specific JavaScript functions as tools that AI agents can directly access, enhancing efficiency and reliability in agent interactions. This new approach, supported by teams from Google and Microsoft, is designed to operate entirely client-side, allowing websites to act as "servers" for agents, and thereby simplifying the interaction model by focusing on explicit tool calls rather than broad DOM access. Although it does not fully eliminate security risks, WebMCP reduces the attack surface by allowing websites to specify which functions are available to agents. As the specification evolves from community incubation to formal draft, with experimental implementations already in Chrome, it signals a shift in web development practices by encouraging site owners to create agent-facing layers to improve user experiences.

Emerging from the rapidly evolving landscape of software design, Agent Patterns present a significant shift in how tools are created for AI agents, marking a departure from traditional integration models. Unlike past frameworks where middleware orchestrated actions, Agent Patterns allow AI agents to autonomously select and interact with tools, necessitating new design paradigms. This development emphasizes the need for production-grade tools that are agent-usable, taking into account factors such as agent experience, security, error management, and tool composition. With over 8000 tools developed, Arcade.dev has documented 54 patterns across 10 categories, offering a comprehensive framework to guide developers in creating tools that are compatible with AI agents. These patterns address the challenges of integrating AI agents with various systems, ensuring security, and facilitating error-guided recovery, ultimately aiming to enhance the ecosystem's ability to build more capable and efficient AI tools.

OpenClaw, also known as Moltbot or ClawdBot, quickly gained popularity as a personal AI agent harness, but its rapid adoption was marred by significant security concerns. Peter Steinberger, inspired by the potential of AI after leaving PSPDFKit, developed OpenClaw, which enables communication with multiple users across various channels via a gateway-connected computer. Despite its exciting potential to shape personal AI assistants' future, OpenClaw's launch highlighted vulnerabilities, including exposed servers and cryptocurrency theft risks, due to its full system access and browser control capabilities. Addressing these security issues involves running OpenClaw on separate systems with throwaway accounts, although recent development efforts focus on enhancing security through user and agent tool policies, sandboxing, and leveraging external runtimes like Arcade.dev, which isolates credentials from the harness. While improvements have been made, OpenClaw remains a tool for technically savvy early adopters, with recommendations to operate it in secure environments and maintain caution regarding the accounts used within its controlled browser.