We Turned Our WireShark Wizard Into a Markdown File (and Four Other Lessons From Building an AI Agent)

Rocky AI, a new AI agent developed by Checkly, has been designed to integrate into their SaaS product to streamline the process of triaging failed Playwright Check tests. Over approximately six to eight months, Checkly developed this tool, initially facing challenges such as managing large data sets and guiding the AI to focus on relevant information. The team realized that a focus on data wrangling was crucial, as Playwright trace files and network PCAP files can be extensive, thus requiring pre-parsing and filtering to make them manageable for large language models (LLMs). Rocky AI's ability to perform root cause analysis involves codifying human skills, such as those used in ICMP and PCAP analysis, into structured formats that the AI can interpret. The team also discovered that while AI model updates have improved performance significantly, implementing a multi-model approach presents difficulties in maintaining quality control. Additionally, they found that while chat interfaces are popular, they are not always the most efficient way of interacting with AI; instead, automating the initial analysis and delivering results to users directly can be more effective. Rocky AI's development continues, with plans to expand its capabilities and features further.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.