April 2026 Summaries

A benchmark study evaluated 13 different models for automated code review across 50 pull requests from five major open-source projects to determine the most cost-effective model for identifying real bugs. The analysis revealed that the most expensive models were not necessarily the best performers, with models like MiniMax M2.7 and Kimi K2.5 offering high value at a fraction of the cost. GPT-5.2 emerged as the best overall model, delivering top-tier quality at $1.25 per pull request with a 60.5% F1 score. The study highlighted that model architecture and training have a more significant impact on quality than cost, with open-source models like Kimi K2.5 and GLM-5.1 providing competitive performance against more expensive options. The benchmark, which is open source, guides the selection of models for Droid's code review, and ongoing evaluations aim to refine methodologies and explore cost-effective review strategies such as multi-pass reviews with cheaper models.

Automated QA is a skill designed to enhance software testing by simulating real user interactions within applications and providing structured reports directly to pull requests. This tool operates by executing user-like actions such as filling out forms and calling endpoints, then posting a comprehensive QA report with visual evidence to GitHub, allowing reviewers to quickly assess pass/fail statuses and explore detailed logs. It features a streamlined approach where each pull request receives a single, updated QA comment, eliminating duplicate threads and ensuring fresh results with every commit. The process can be configured as an optional CI check, triggered on demand, enabling teams to focus on significant changes without disrupting workflows. Developers can also conduct local QA sessions to test branches before pushing, offering a fast feedback loop for broken flows. Setup involves running a command in a Droid session, which tailors the QA skill to the specific project by analyzing the codebase and collecting contextual information. This integration into the development loop ensures that testing becomes a seamless and reliable part of the code review process, providing clear and actionable insights without additional dashboards. Automated QA is included in all Factory plans, and comprehensive documentation is available for further details.

Droid Computers introduce a new approach to managing cloud agents by providing persistent machines for orchestrating Droids, either through Factory's managed cloud service or by utilizing personal devices. This persistence allows Droids to maintain a consistent development environment with intact configurations, credentials, and memory, enhancing their ability to perform complex tasks and self-improve over time. Unlike ephemeral sandboxes, which are suitable for short-term tasks, Droid Computers support ongoing workflows and asynchronous operations, enabling users to manage Droids remotely via mobile devices and initiate tasks that can run overnight. The system supports two setup options: Factory-managed cloud computers and Bring Your Own Machine (BYOM), which allows users to register existing devices, offering flexibility and integration with pre-configured environments. While BYOM is free for all users, managed cloud computers are accessible under specific Factory subscription plans, underscoring a shift in the industry towards persistent environments for more powerful and capable cloud agents.

Factory has raised $150 million in a Series C funding round led by Khosla Ventures, achieving a valuation of $1.5 billion, with participation from several prominent investors including Sequoia Capital and Blackstone. The company, founded three years ago with the mission to bring autonomy to software engineering, has developed Droids, which are used by developers at major enterprises such as Nvidia and Adobe. These Droids have evolved from aiding software development to becoming autonomous systems capable of handling complex workflows, significantly advancing Factory's platform. Recent innovations include the introduction of Missions for coordinating multi-step workflows and Factory Desktop for integrating Droids directly into users' systems. Factory plans to use the new capital to enhance its platform, focusing on aspects like model routing optimization and enterprise-grade governance, with the aim of revolutionizing software development through agent-native processes.

Missions is a system designed to address the limitations of single-agent contexts in complex projects by breaking down large tasks into focused units that are managed by specialized agents, each with narrowly defined goals and explicit validation processes. This architecture is based on the idea that agents perform best when their context is relevant and coherent, avoiding distractions that can dilute performance. The system involves an orchestrator that plans and decomposes tasks, workers that implement features with pre-defined tests, and validators that ensure correctness and completeness without bias. A real-world application of this system involved creating a Slack clone, demonstrating a structured implementation-validation process across milestones, which ensured high code coverage and reliability. Missions aim to enhance software development by leveraging improvements in model reasoning and execution, making it increasingly feasible for teams to undertake more ambitious projects efficiently, with Missions available for deployment in Droid sessions.

The Factory desktop app has been released, offering a native interface for managing multiple Droids across various software business tasks on macOS and Windows. It supports multi-agent sessions, allowing users to run multiple Droids simultaneously, each with its own context and history. The app provides persistent Droid Computers that resume with all settings and data intact, with options for cloud-managed machines and BYO hardware integration, including local models for secure, on-premise use. Users can control applications like VS Code, browsers, and spreadsheets through the app, enabling tasks like testing user flows, updating files, and generating reports. The app also features AI-native visualizations, a full plugin ecosystem, and mobile access, with enterprise teams reportedly increasing productivity and expanding usage beyond traditional technical roles. Available on all Factory plans, it inherits the CLI's security model, with local command execution and encrypted data handling.

Legacy-Bench is a new benchmark designed to evaluate AI agents' capabilities in handling legacy software engineering tasks across six language families, including COBOL, Fortran, and Java 7, which are foundational yet increasingly challenging due to retiring engineers and complex business rules embedded within the code. It provides a comprehensive set of tasks that involve fixing bugs, implementing new functionalities, and migrating code, reflecting real-world applications in critical infrastructure. The benchmark reveals significant performance variations among AI models, with agents excelling in bug fixing due to visible errors in languages like Java 7 but struggling with COBOL due to its silent errors and complex format precision requirements. The results indicate a steep learning curve in reading and writing new legacy code, with migration success heavily dependent on the target language. No single model consistently outperforms across all tasks, highlighting diverse strengths and weaknesses, and illustrating the need for systematic verification and iteration in legacy environments. As AI models improve their legacy language training and self-verification capabilities, the performance gap between legacy and modern benchmarks is expected to narrow, offering insights for those modernizing legacy systems.