How We Cut a Query From 49 Seconds to 2 Seconds in OpenObserve â A 25Ã Win From Two Config Changes
Blog post from OpenObserve
OpenObserve achieved a dramatic improvement in query performance by implementing two key configuration changes: increasing the ZO_COMPACT_MAX_FILE_SIZE from 1GB to 10GB and enabling a footer cache. These adjustments reduced the number of S3 requests required from over 10,000 to approximately 615, thereby significantly decreasing query latency from 49 seconds to just 2 seconds, a roughly 25-fold enhancement. The increase in file size reduced the number of files and consequently the number of network round trips, while the footer cache facilitated the quicker access of metadata, reducing the reliance on S3 storage for each query. This optimization illustrates the importance of minimizing S3 requests and highlights that larger file sizes can lead to better performance by decreasing scheduling overhead and improving metadata reuse. These changes not only improve cold cache queries but also provide a consistent and reliable user experience, turning what were once minute-long queries into ones that return results almost instantaneously.
No tracked trend matches for this post yet.
Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.