October 2022 Summaries

ScyllaDB, in collaboration with Pythian, is offering a free online High Performance NoSQL Systems Masterclass, which aims to provide insights into choosing and optimizing NoSQL databases based on their scalability, reliability, and performance. The event will cover the evolution from the SQL vs. NoSQL debate to current considerations of selecting the right NoSQL database for specific use cases. It will delve into wide column NoSQL databases, using ScyllaDB and Apache Cassandra as examples, highlighting their architecture, best practices, and the benefits of distributed, leaderless clustering models. The sessions will be led by experts including Peter Corless, Allan Mason, and Felipe Cardeneti Mendes, and will conclude with a Q&A session to reinforce learning. Participants will also undergo a test to assess their understanding, with opportunities for retesting if needed. This Masterclass is part of a series aimed at enhancing professional skills in distributed data systems and performance engineering.

P99 CONF 2022, hosted by ScyllaDB, concluded successfully, bringing together a global community of engineers focused on performance and the P99 metric. The event featured a diverse lineup of speakers who discussed various topics, including game development insights by Charity Majors, toolmaking by Bryan Cantrill, and database workloads by Avi Kivity. Attendees had the opportunity to engage in live chats and follow-up discussions in the Speakers Lounge, fostering deeper interactions with experts. The conference also highlighted the importance of questioning traditional metrics and encouraged continued dialogue on performance strategies post-event. Attendees and those interested can access all talks and materials on demand, and the community is invited to contribute to ongoing discussions and technical content. Additionally, a High Performance NoSQL Masterclass is upcoming, promising further exploration of scalable and efficient database solutions.

P99 CONF 2022 offered a deep dive into the challenges and solutions surrounding low-latency performance in computing, bringing together thousands of engineers to share their experiences and insights. Day 1 featured over 30 engineering talks, with keynotes from notable figures like Gil Tene, who discussed the evolving approach to P99 latency measurement, Liz Rice, who demonstrated the use of eBPF for high-performance networking in Kubernetes, and Armin Ronacher, who shared strategies for optimizing event processing pipelines at Sentry.io. Dor Laor from ScyllaDB provided insights into overcoming latency issues in databases by addressing speed mismatches, task quota violations, and isolation imperfections. The event highlighted the importance of learning from "misery metrics" to improve system performance, with sessions on topics such as asynchronous programming, database query optimization, and dynamic packet manipulation using eBPF. Day 2 promised to expand on these themes with discussions on emerging technologies like WebAssembly and chaos engineering, featuring keynotes from industry leaders and further specialized track sessions.

ScyllaDB collaborated with the University of Warsaw to develop a new Go driver, aiming to outperform both the existing GoCQL and ScyllaDB Rust drivers in terms of performance. Taking advantage of Go 1.18's generics, the new driver was designed to follow ScyllaDB's shared-nothing architecture, minimizing the use of mutexes and focusing on efficient asynchronous communication using Go channels and atomics. This architectural choice, along with optimized memory management and request coalescing, significantly reduced the impact of the garbage collector and improved performance, enabling the driver to execute operations up to four times faster than its predecessor and twice as fast as the Rust driver. The driver demonstrated its capabilities by handling high concurrency with reduced latency and increased throughput, with plans for further optimization as it continues to be developed on GitHub.

Cobli's blog post chronicles their transition from Apache Cassandra to ScyllaDB, detailing their strategic approach to this significant database migration. Initially guided by ScyllaDB's documented method, which included dual writing and validation phases, Cobli opted for a more aggressive "stop-the-world" approach due to time and resource constraints. This alternative method involved halting services temporarily to migrate data in time blocks, ensuring data consistency by leveraging the immutable nature of their time-series data. Despite the potential for downtime and other risks such as low post-migration visibility, Cobli's approach prioritized predictability in migration time and operational security. The decision was influenced by the need to avoid extensive reconfigurations and redeployments associated with ScyllaDB's approach, particularly given their numerous client systems and streaming data processing jobs. The post concludes with the anticipation of further insights and learnings from the ongoing migration process.

P99 CONF 2022, an open-source industry conference hosted by ScyllaDB, focused on the intricacies of low-latency data systems and brought together experts to discuss a variety of topics related to SQL, NoSQL, and event streaming. The conference featured sessions on diverse subjects such as query performance optimization, testing Kubernetes storage with Sherlock, and implementing WebAssembly for user-defined functions to maintain low latency. Presentations also covered techniques like using Apache Arrow Flight SQL for database throughput, leveraging OpenTelemetry for measuring latency in event-driven architectures, and employing innovative strategies for performance tuning in distributed databases like TiDB and ScyllaDB. Experts shared insights into optimizing complex data ingestion pipelines, managing the challenges of measuring latency, and adopting new protocols like Apache Iceberg to enhance data lake architectures. The event emphasized understanding and overcoming latency issues, with sessions exploring topics such as the advantages of using C# as a system language and RageDB’s experience with the Seastar framework for fast graph database development.