March 2025 Summaries

PlanetScale's vector search and storage support is now Generally Available (GA), offering improved query performance, memory efficiency, and robustness compared to its open beta period. Vectors are treated as a first-class data type in MySQL, allowing for normal relational operations such as joins, where, and select statements with vector indexes. The feature also supports advanced use cases like Euclidean distance ranking, product quantization, and larger-than-RAM indexes using the SPANN (Space-Partitioned Approximate Nearest Neighbors) index design. To get started, users can enable vectors in their database branches by following the provided examples and documentation, which includes a YouTube video for learning more about vector embeddings.

The new SQL evaluation engine in Vitess is significantly faster than the previous implementation, with performance improvements ranging from 20x to 50x compared to the original AST-based dynamic implementation. The new virtual machine design, which uses a callback-based approach and takes advantage of Go's closures feature, enables efficient execution of statically typed SQL expressions without requiring runtime type switching. This results in faster compilation times, easier maintenance, and improved performance. The benchmarks show that the new virtual machine outperforms both the original AST interpreter and the C++ implementation in MySQL, with some queries taking up to 20x less time to execute.

At PlanetScale, they have deployed millions of Amazon Elastic Block Store (EBS) volumes across the world and have a unique viewpoint into the failure rate and mechanisms of EBS. They see failures like this on a daily basis, with a frequent enough rate that they've built systems to monitor EBS volumes directly to minimize impact. The true rate of failure is constant, variable, and all by design due to the lack of performance guarantees when volumes are not operating to their specifications. To handle these failures, PlanetScale has developed mitigations such as monitoring metrics closely and performing zero-downtime reparents in seconds to another node in the cluster. They've also built a shared-nothing architecture that uses local storage instead of network-attached storage like EBS, allowing other shards and nodes in a database to continue operating without problem.

Non-volatile storage devices are a cornerstone of modern computer systems, allowing data to be stored even when the computer is powered off. Tape storage has been used since the 1950s, but it still exists today for certain applications due to its low cost and long shelf life. Hard disk drives (HDDs) were introduced in the 1970s and offer faster access times than tape storage, but are limited by mechanical components that can slow down data retrieval. Solid state drives (SSDs), on the other hand, store data electronically without mechanical components, offering much faster access times and lower latency. However, SSDs also have their own limitations, such as the need for careful data organization to minimize garbage collection and optimize performance. The shift from tape, disk, and solid-state storage has led to a new phenomenon: separating storage from compute, which can introduce significant latency deterioration due to network round trips. To overcome this issue, replication and direct-attached storage with automatic capacity management are being explored as solutions. Recently, PlanetScale announced Metal, an industry-leading solution that combines Vitess+MySQL cluster setup with direct-attached NVMe SSD drives for a full-fledged database platform with extremely durable data, scalable performance, and no artificial IOPS limits.

PlanetScale has announced the general availability of its new "Metal" class of nodes, which offer improved performance and cost savings for hosting relational databases on AWS and GCP. Metal instances are powered by locally-attached NVMe SSD drives, providing unlimited I/O on every cluster type, with some customers reporting a 65% drop in p99 query latency and 53% cost savings compared to Amazon Aurora. The new feature has been available in production for three months, serving over 5 trillion queries across 5 petabytes of storage. Customers such as Block, Intercom, Depot, and others have reported significant improvements in performance and cost efficiency on Metal.

PlanetScale Metal is a scalable cloud OLTP database platform that offers effectively unlimited IOPS, making it ideal for high-I/O databases. It uses fast, local NVMe drives to achieve performance and price without compromising durability or scalability. Unlike network-attached storage products like Amazon EBS and Google Persistent Disk, Metal's directly-attached NVMe drives offer lower and more consistent read and write latency, higher throughput, and are generally less expensive. PlanetScale Metal is designed for critical workloads where microseconds matter, making it suitable for databases with constant, wide-ranging, random reads, massive write throughput, low tolerance for high latency, and other demanding use cases. With its powerful hardware and optimized architecture, Metal delivers performance at a cost that's often comparable to or even lower than equivalent configurations using Amazon Aurora or PlanetScale.

The PlanetScale team successfully migrated their Query Insights database from a traditional MySQL setup to Metal, a high-performance, scalable database service. The migration involved upgrading one of the eight shards of the database to Metal, which resulted in significant performance improvements, with query latency decreasing substantially across all measured percentiles. The upgrade was achieved without changes to the application architecture or sharding configuration, and has provided substantial benefits including lower average backlog in Kafka consumers and additional capacity for handling increasing message volume.