Karpathy's Autoresearch found a 3-year-old bug in our query engine (and improved performance by 11%)

During a team offsite in Lisbon, an AI agent was utilized to analyze slow queries in ClickHouse, revealing a three-year-old bug related to improper use of primary keys with timestamp filters. This bug led to inefficient query performance, but the AI-driven analysis identified a fix that reduced granule scans by 62% and improved query speed significantly. The approach was inspired by Andrej Karpathy's concept of "autoresearch," which involves iteratively testing changes to improve system performance without the biases inherent in human coding practices. The team structured their analysis into campaigns with specific optimization goals and utilized a small coding agent, pi, to automate the process. This led to the discovery that the toTimeZone() function was preventing effective partition pruning. By modifying the query to allow the ClickHouse planner to see a bare timestamp, significant performance improvements were achieved. Going forward, the team plans to automate this process further by fetching slow queries from logs and running them through a pipeline that uses PostHog Code to implement and test improvements, aiming to make such optimizations a routine, automated task.

No tracked trend matches for this post yet.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.