Achieving Great Dynamic Sampling with Refinery

Refinery, Honeycomb's tail-based dynamic sampling proxy, is designed to retain interesting traffic while discarding repetitive, less significant data, but its advanced functionality can initially seem counterintuitive to users. Honeycomb's Customer Architect team often fields queries from users struggling to achieve desired sample rates with Refinery, prompting detailed guidance on effective sampling and troubleshooting. Users are advised to ensure diagnostic configurations are correctly set, including using fields like RefineryTelemetry and Debugging options, and to leverage Usage Mode for enhanced visualization of sample rate data. The blog post also emphasizes the importance of tuning the EMADynamicSampler, particularly the AdjustmentInterval, to maintain stability in sample rates and suggests using root. prefixes to limit cardinality. For users seeking greater predictability, the RulesBasedSampler is recommended, offering deterministic tail-based sampling with specific rules for managing traffic. The blog concludes by highlighting the importance of periodically revisiting and refining sampling rules to align with evolving software and business needs, inviting new users to explore Honeycomb's services.