Debugging Python memory issues in production with memray and AI

A backend service faced frequent restarts due to out-of-memory (OOM) issues triggered by Kubernetes, as certain queries under real production traffic caused memory spikes. The problem was traced using memray, a memory profiler for Python that can attach to live processes and track memory allocations, and Claude Code, an AI tool that efficiently analyzes large codebases. Memray identified that oversized query results were causing memory overloads, while Claude Code quickly pinpointed the issue within the codebase, revealing that the service was fetching and serializing excessively large datasets without limits. The solution involved implementing limits on the number of records returned by queries and capping the response payload size, which stabilized memory usage. The experience highlighted the importance of profiling in production environments, recording comprehensive data to understand allocation trends, applying query limits, and leveraging AI tools to expedite problem-solving and apply fixes efficiently.