Company
Date Published
Author
Premkumar Thangamani
Word count
1731
Language
English
Hacker News points
None

Summary

To optimize pagination for distributed data while maintaining ordering, an application-level sharding layout can be used to avoid hotspots on range-based indexes. By creating multiple shards and using the PRIMARY KEY of the table without additional indexes, it's possible to retrieve only the rows needed from each shard and perform an efficient k-way merge on the already sorted streams using the Merge Append operator. This approach allows for paginated results while maintaining the DESC ordering of the event_ts column, making it a suitable solution for getting paginated results from a table with range-sharded data.