Agentic kernels in production

An agentic kernel-development framework is presented to bridge the gap between isolated kernel benchmarks and production inference optimization by profiling full model workloads, identifying model-level and per-kernel opportunities, generating implementations, and validating correctness and end-to-end performance within serving engines. Its model-level layer targets changes such as operation fusion, redundant-work removal, and reduced intermediate materialization, while its kernel-level layer explores implementation variants for critical kernels; successful patches and lessons from failures are retained in a knowledge base for future runs. Applied to Qwen-Image and FLUX.2 diffusion models on B300 GPUs using SGLang, the system achieved reported end-to-end latency reductions of 42.3% and 15.2%, respectively, through changes including pre-packed FP8 scales, fused QKV projections, normalization and quantization fusion, cached classifier-free guidance modulation, fused QK normalization with RoPE, fused SwiGLU quantization, and gated residual normalization. The framework is intended to be model- and serving-engine-agnostic, and early LLM experiments on vLLM reportedly improved throughput by up to 5.5% on MiniMax M3 and GLM-5.2, supporting the goal of generating kernels specialized for particular hardware, tensor shapes, and serving workloads.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.