Designing a faster data model to personalize browsing in real time

The Hotels Network (THN) collaborates with Tinybird to enhance real-time personalization for hotel booking websites by segmenting visitors and processing data through API endpoints. THN faced challenges in optimizing performance due to high read and write demands, making over 120 requests per second. Efforts to improve involved simplifying API endpoints, adjusting data partitioning strategies, and reducing query complexity by minimizing the number of data sources. While some strategies like partitioning by user key initially caused disk throttling, transitioning to a quarterly partition with a time-to-live (TTL) setting helped manage data accumulation. Further optimizations included reducing index granularity and altering join methods, leading to a significant reduction in endpoint response time from 340 ms to 170 ms. Despite these improvements, ongoing performance enhancements are pursued, such as experimenting with compact parts and profiling tools to identify additional areas for refinement, underscoring the importance of understanding how ClickHouse® merges parts for optimal performance.