Inside Sentry’s Hackweek: An excuse to break things

Sentry's Hackweek provided the opportunity for employees to innovate creatively, with a team devising a project called Gremlins, which aimed to uncover bugs in SaaS applications through AI-powered web fuzzing agents. These agents, dubbed Gremlins, were designed to simulate unpredictable user interactions across different application components, using Sentry's tools to capture errors and profiling data. The team developed custom agents that utilized ARIA for creating simplified page representations and custom tools for interacting with web elements, finding the process of building these agents both challenging and insightful. They compared their homemade solution with the open-source browser-use tool, noting the strengths and weaknesses of each in terms of performance and resource usage. Ultimately, the project emphasized the potential of Large Language Models (LLMs) in software testing and the importance of balancing autonomy with security considerations, culminating in a playful and engaging Hackweek experience.