June 2026 Summaries

Continuous profiling has been enhanced with new tools that improve data accessibility and analysis without disrupting workflow, particularly through the Model Context Protocol (MCP), which allows users to query production profiles using plain English. The update introduces the "sandwich" tool, which replaces the flat table view with a hierarchical tree structure that mirrors a flame graph, thereby preserving call stack hierarchies and allowing for more focused analysis. Alongside this, a new tool called "query_profile_flamegraph" now provides a genuine flame graph, enabling users to zoom into specific paths within the data. The MCP server, which works with clients like Polar Signals Cloud, ensures that model queries match the cloud UI by using shared templates, providing consistency and reliability in data analysis. Additionally, the continuous-profiling skill has been streamlined to make tool selection more intuitive, reducing the instruction size by 40% while maintaining effectiveness through rigorous evaluation tests.

During a Polar Signals hackathon, the author tackled creating a CUDA kernel to optimize string decompression, a task that served as an introduction to GPU programming and performance optimization. The project focused on implementing a kernel for decompressing FSST-encoded strings, a necessity for Vortex, Polar Signals' new file format, which aims for high-throughput decompression and query execution on GPUs. The initial kernel performance lagged behind CPU-based decompression, prompting the use of GPU profiling to identify and address memory-related bottlenecks. Key optimizations included memory load/store improvements, such as aligning loads and utilizing shared memory, though some attempts like reducing bank conflicts proved less effective. Eventually, optimizing memory stores yielded a significant performance boost, surpassing the CPU implementation. Further refinements included the split kernel optimization from the GSST paper, which enhanced execution efficiency by balancing workload across threads decompressing variable-length strings. Despite challenges, the project resulted in substantial improvements and highlighted the importance of understanding GPU architecture for achieving high performance. The final FSST CUDA kernel implementation achieved noteworthy throughput, and the experience underscored the potential of GPU profiling for future optimizations.

The text delves into the complexities of profiling JavaScript code executed by the V8 engine, highlighting the challenges faced when unwinding stacks and associating them with human-readable function names and line numbers. It explains the process V8 uses to parse JavaScript into bytecode for the Ignition interpreter and the subsequent compilation to machine code by one of three compilers: Sparkplug, Maglev, or TurboFan. Unlike native code, V8-generated code doesn't rely on .eh_frame data for stack unwinding due to its dynamic nature, instead utilizing frame pointers present in Node.js stacks. The article further discusses methods to detect V8 frames, distinguish them from native ones, and symbolize them by recovering function names and source line numbers. It highlights the use of debug symbols and manual version matching to discover necessary offsets, emphasizing the intricate work involved in maintaining a production-grade profiler for V8, supported by collaborations with the OTel Profiling community and the use of AI tools for accuracy checks.

The CUDA Profiling Tools Interface (CUPTI) has expanded its capabilities by incorporating Program Counter (PC) sampling, which allows developers to analyze CUDA program performance at the instruction level, identifying stall reasons and optimizing code efficiency. This feature, traditionally used in developer tools like NVidia NSight, can now be applied in production settings thanks to a low-overhead continuous profiler that minimizes performance impacts. PC sampling utilizes dedicated hardware to record the state of each GPU warp at configurable intervals, capturing PC offsets and stall reasons without timestamps or call stacks. The implementation involves a dynamic algorithm that periodically enables and disables PC sampling to maintain efficient data collection, while a shim library interfaces with the CUPTI to manage and transmit data to a backend for analysis. The data, collected in PC/stall-reason pairs, is processed and symbolized on the backend to provide detailed insights into GPU stalls, enhancing the utility of continuous profiling tools like Polar Signals. This advancement allows users to maintain a comprehensive production-level profiling environment, capturing valuable instruction-level GPU insights for performance optimization.