August 2023 Summaries

Integrating ScyllaDB with Presto and Metabase offers a comprehensive open-source data analytics solution that combines ScyllaDB’s high-throughput and low-latency NoSQL capabilities with Presto’s distributed SQL query engine, enabling efficient data processing and real-time insights. This integration facilitates seamless querying and analysis of large datasets by merging ScyllaDB’s speed with Presto’s processing power, while Metabase provides an intuitive user interface for data exploration and visualization without requiring technical expertise. By setting up this architecture using Docker, users can quickly initiate a data analysis environment that supports dynamic data structures and powerful visualization, making it easier to break down data silos and drive data-driven decision-making. The guide includes hands-on exercises with a sample database schema to demonstrate how to set up and utilize the system for creating interactive dashboards and visualizations, ultimately highlighting how businesses can leverage this integration to gain actionable insights and maintain a competitive edge in a data-centric landscape.

ScyllaDB Summit 2024 is a free, virtual event inviting database professionals to share their experiences and insights, particularly those centered around performance challenges and innovations. The call for speakers is open, encouraging participants to discuss topics such as distributed data challenges, ScyllaDB usage, best practices for high-throughput applications, and technologies integrated with or extending ScyllaDB. The event will feature short, 15-20 minute speaking slots, offering a platform for both experienced and first-time speakers to present to a global audience without the need for travel. All sessions will be pre-recorded, allowing speakers to engage with attendees during the live event, which is scheduled for February 14-15, 2024. The deadline for speaker submissions is October 20, and the summit aims to foster an interactive and collaborative environment for sharing authentic technical experiences.

ScyllaDB Enterprise 2023.1 introduces significant updates, including the implementation of Raft-based strongly consistent schema management, which enhances the reliability and speed of schema changes by serializing operations and reducing schema collisions. This release, based on ScyllaDB Open Source 5.2, also features distributed aggregations for faster query execution, particularly benefiting operations like SELECT COUNT(*). The update resolves over 100 issues, improving stability and performance. Additionally, support for the Alternator TTL, secondary indexing on collections, and automated tombstone garbage collection have been promoted from experimental to production-ready, offering users enhanced data management capabilities. These improvements position ScyllaDB as a robust alternative to Amazon DynamoDB, with flexible deployment options and lower operational costs. Users can now experience safer and more efficient database operations, especially in distributed environments, and the release notes provide further details on these enhancements.

Pinhome Engineering, Indonesia's largest property transaction platform, successfully enhanced its recommendation engine by migrating from MongoDB and PostgreSQL to ScyllaDB, resulting in significant improvements in cost efficiency and performance. The switch was driven by the need to handle increased web traffic and improve the accuracy and reliability of recommendations amid rapid user base growth. By adopting ScyllaDB, the company achieved a 60% reduction in costs, a 97% decrease in latency, and 90% storage savings. The new architecture allowed for faster processing of user data, leading to a more stable recommendation engine and better user experience. Additionally, the migration reduced code complexity, improved API performance, and resolved previous issues related to performance degradation and storage limitations. The strategic approach included optimizing algorithms, employing efficient data structures, and ensuring fault tolerance to process large data volumes effectively, thereby enhancing user engagement and satisfaction.

ScyllaDB offers several advanced features that may be overlooked but are beneficial for optimizing database performance and resource management. Workload Prioritization, available in ScyllaDB Enterprise, enables users to allocate resources efficiently across different types of workloads, reducing costs and improving performance. Heat-Weighted Load Balancing (HWLB) ensures consistent latencies by intelligently routing traffic during cache warm-ups, while Per Shard Concurrency Limit helps manage latency during traffic spikes by setting thresholds for concurrent requests. The Per Partition Rate Limit feature, introduced in ScyllaDB 5.1, allows for rate limiting on a per-table basis to handle specific high-load scenarios like spam attacks. Bypass Cache, an extension to the CQL protocol, prevents cache eviction during large data scans, maintaining low latencies and optimizing cache usage. These capabilities demonstrate ScyllaDB's adaptability in handling diverse workloads effectively.

Visually.io has transitioned from MongoDB to ScyllaDB for its main production real-time database, leveraging Kubernetes and spot instances to manage intense workloads at a quarter of the usual cloud compute cost. ScyllaDB, an open-source NoSQL database compatible with Apache Cassandra and DynamoDB, offers advantages such as zero downtime failovers, horizontal and vertical scaling, and efficient data sharding, which ensure high availability and low latency even when subjected to frequent failovers inherent to spot instances. The platform's architecture avoids common issues associated with Cassandra and enables seamless integration into Kubernetes environments through its Helm chart, operator, and manager, which automate deployment and management tasks. Monitoring is facilitated by Prometheus and Grafana, providing insights into metrics and performance, while Kubernetes ConfigMaps enable the creation of dashboards for detailed database monitoring. Despite the occasional latency spikes due to the volatile nature of spot instances, the cost savings and operational efficiency make ScyllaDB a compelling choice for scalable database management.

Machine learning feature stores serve as central databases that contain real-time and historical features, facilitating data engineers and scientists in discovering, monitoring, and analyzing features for predictive models. These stores typically include both online and offline databases, with online databases providing real-time feature data for production models and offline databases storing historical features for batch processing. ScyllaDB is highlighted as a high-performance, low-latency NoSQL database ideal for supporting feature store architectures due to its ability to handle large volumes of read and write operations, making it suitable for mission-critical workloads. ScyllaDB's compatibility with Cassandra and DynamoDB allows for seamless migration, and its integration with feature store tools like Feast further enhances its utility. By consolidating databases, ScyllaDB can streamline infrastructure maintenance, offering a single solution for both online and offline feature storage needs, thus addressing the performance requirements essential for efficient model development.