August 2024 Summaries

A database throttler is a service or component that regulates incoming requests to prevent system overload. It focuses on throttling non-time critical operations such as ETLs, data imports, and mass purges of data. The choice of metrics for determining the health of a database varies depending on factors like scope of queries, hot spots or cold spots in affected data, and state of the page cache. Replication lag is a popular metric used to push back against long running jobs. Other common metrics include threads_running, transaction queue latency, queue delay, load average, and pool usage. A throttler should be able to push back based on a combination of these metrics rather than just one. The granularity of the metrics collected by the throttler can significantly affect its performance and efficiency.

In big databases, IOPS (Input/Output Operations Per Second) and throughput can become performance bottlenecks. Sharding is a technique that helps scale out IOPS and throughput beyond the limitations of a single server by distributing data across multiple servers. This reduces costs for large-scale workloads as it allows each instance to use more affordable EBS volumes. For example, in a sharded database, IO and throughput requirements are distributed across many primaries, enabling efficient handling of I/O intensive workloads like databases without needing to pay an EBS premium.

PlanetScale Insights has introduced a new index tracking feature to help developers monitor which indexes their queries are using and how that usage is changing over time. This feature provides a graph showing the percentage of queries using each index, as well as cumulative usage data. It also allows users to search for queries based on specific indexes or find unindexed queries. The implementation includes extracting query index usage information from MySQL and aggregating it into Insights every 15 seconds.

Data migrations are an essential part of managing databases, often required for version upgrades, sharding, or moving to a new platform. However, they can be challenging and risky, with potential downtime, data loss, and compatibility issues. PlanetScale has developed a process that enables zero-downtime migrations at petabyte scale. This involves taking consistent non-locking snapshots of the data, replicating changes throughout the migration process, verifying the new system, cutting over application traffic to the new system, and decommissioning the old system when ready. PlanetScale's approach ensures that the entire migration can be done without any downtime or impact on live production systems.