How we brought AI to the Bitrise Build Hub

Bitrise has developed a sandboxed AI agent that enhances its AI strategy by operating on virtual machines within the Bitrise Build Hub, offering greater access to customer code and developer tools compared to its previous AI Assistant. This new agent is designed to run autonomously without requiring engineer intervention and uses pre-defined tool usage lists instead of interactive approvals. The AI agent's framework is provider-agnostic, allowing it to use different LLM providers based on configuration, and it supports prompt caching to prevent performance degradation as context grows. The team encountered a prompt caching issue that skewed success rates during testing, which they addressed by introducing a cache-busting flag. Additionally, session persistence has been implemented to maintain context across queries, and a basic context summarization mechanism is in place to manage the context window limit. The blog post also highlights the introduction of features to monitor agent runtime, token usage, and integrate logging, and mentions the open-sourcing of a simplified version of the core logic for further exploration and development. The final post in the series will discuss scaling AI feature development across Bitrise's R&D organization.