November 2025 Summaries

Semantic caching is introduced as a technique to address high costs and latency issues when scaling AI workloads, particularly in applications relying on large language models (LLMs). By storing the meaning of user queries as vector embeddings, semantic caching allows for faster and cheaper responses by delivering cached results for semantically similar queries instead of repeatedly querying the LLM. This approach not only reduces the number of LLM calls, thereby decreasing costs, but also enhances response times by leveraging a low-latency database like ScyllaDB. ScyllaDB, with its built-in caching layer and vector search capabilities, is highlighted as an ideal platform for implementing semantic caching, offering high availability and strong performance metrics essential for real-time AI applications. The text outlines a basic workflow for implementing semantic caching, emphasizing the importance of maintaining cache accuracy through periodic invalidation to ensure up-to-date responses.

Task Manager is a tool designed to manage and monitor ScyllaDB's background operations, essential for maintaining data consistency, durability, and performance in distributed environments. It organizes operations like compaction and repair into hierarchical task trees, allowing administrators to track their progress and manage them effectively. Task Manager offers a REST API and command-line interface, enabling users to list, monitor, and control tasks, supporting both local node-specific and global cluster-wide operations. It provides detailed insights into each task's status, progress, and lifecycle, and enables the transformation of synchronous maintenance tasks into asynchronous ones by returning task IDs for ongoing operations. This system enhances visibility, consistency, and control over ScyllaDB's maintenance activities, facilitating stable and efficient database management.

DynamoDB and ScyllaDB, both distributed NoSQL databases, share a common origin from the Dynamo paper and offer features like multi-zone and multi-region deployments, but they diverge significantly in their tenancy models. DynamoDB operates as a multi-tenant database, allowing for lower infrastructure costs and shared burst capacity among tenants, but it faces challenges like the Noisy Neighbor issue and broader impact during outages, as seen in a recent incident. In contrast, ScyllaDB employs a single-tenant architecture, providing each user with dedicated resources such as VMs, VPCs, and endpoints, which enhances isolation and limits the impact of failures to individual users. While this model offers greater resilience and workload isolation, it requires careful resource scaling and depends on EC2 for node allocation. Despite these differences, ScyllaDB optimizes performance per node to offer competitive pricing compared to DynamoDB, emphasizing stronger tenant isolation and minimizing the blast radius of potential issues.

The article explores the architectural differences and performance implications of using Memcached as a caching system compared to ScyllaDB, a persistent database with caching capabilities. The author, alongside Memcached maintainer Alan Kasindorf, conducted benchmarks to assess how each system uses memory, handles various data models, and manages disk I/O. Memcached is highlighted for its efficiency with in-memory caching and pipelined requests, allowing higher throughput and scalability, especially with smaller payloads, while ScyllaDB supports complex data models and persistent storage, albeit with higher memory overhead. The tests revealed that Memcached's Extstore can store more items on disk, albeit with manual tuning, whereas ScyllaDB offers robust data persistence and high-performance disk I/O operations. The choice between the two depends on specific workload requirements, such as data model complexity and the need for data persistence, as each system comes with unique strengths and trade-offs that impact performance testing and operational efficiency.

ScyllaDB's 2025.3 release introduces a significant enhancement in its backup process by integrating native backup capabilities, leveraging Seastar's CPU and I/O scheduling to achieve up to 11 times faster backup speeds. This new approach eliminates the reliance on an external process, which previously caused resource contention and slower backup speeds when using ScyllaDB Manager and the rclone agent. By managing backups internally, ScyllaDB can prioritize resources more effectively, ensuring that user queries maintain low latency even during backup operations. Currently available for AWS with plans to expand to other platforms like GCP and Azure, this native backup system delivers a notable performance improvement, as demonstrated by a test where the native backup achieved a throughput of approximately 900 MB/s compared to 80 MB/s with rclone. The focus now shifts to optimizing the restore process, which presents complex challenges such as handling different cluster topologies and replication factors. Future plans include using object storage for both backup and restore, as well as for data tiering, allowing ScyllaDB to access data directly from object storage.