August 2026 Summaries

Sail Research has open-sourced HTDYM, a performance-modeling tool designed to estimate the cost efficiency of deploying LLMs across different accelerators, model architectures, workloads, and sharding strategies. Rather than relying solely on benchmarks, which can be unavailable, time-consuming, or distorted by immature inference-engine implementations, HTDYM uses per-operation roofline-style estimates for compute, memory, and network communication, then searches feasible parallelism, placement, and MoE dispatch configurations. Its pricing engine lowers a model into an operation graph, resolves communication collectives through a shortest-path search, incorporates hardware specifications and configurable performance deratings and overlap assumptions, and ranks configurations by tokens per dollar relative to a minimum viable H100 baseline. Sail reports that the system identified TPU v6e as a favorable option for a prefill-heavy Gemma 4 31B workload, with an estimated threefold input-token cost reduction versus H200 hardware and real-world tuned performance reaching 87.5% of its prefill estimate. The project currently focuses on large, steady-state throughput workloads and does not fully model factors such as continuous batching, KV-cache transfer, speculative decoding, MoE routing imbalance, fixed overheads, or detailed hardware scheduling, so its estimates are less reliable for small-batch, latency-sensitive serving.