January 2023 Summaries

The ScyllaDB Summit, held virtually on February 15 and 16, 2023, offered a comprehensive exploration of ScyllaDB's innovations and strategies for data-intensive applications. Attendees had access to over 25 sessions covering a range of topics, including ScyllaDB's serverless transition, its use in real-world applications by companies like Discord, Epic Games, and ShareChat, and its integration with various technologies such as Pulsar, Flink, and Ansible. The summit emphasized interactive engagement with speakers, providing opportunities for participants to connect with peers globally and discuss ScyllaDB's engineering advancements, including its compatibility with Amazon DynamoDB, its move to serverless architecture, and its low-latency capabilities. Additionally, sessions highlighted user strategies for optimizing ScyllaDB, such as Discord's migration from Cassandra, and provided insights into the development of next-generation drivers and performance optimization techniques. The event concluded with opportunities for attendees to have their questions answered, participate in contests, and witness the announcement of the ScyllaDB Innovation Award winners.

In a hands-on lab, the integration of ScyllaDB's Change Data Capture (CDC) with Apache Kafka is explored, highlighting the use of the ScyllaDB CDC source connector to push real-time row-level changes from ScyllaDB tables to a Kafka server. This connector, a Debezium component compatible with Kafka Connect, reads CDC logs to produce Kafka messages for each database operation like INSERT, UPDATE, or DELETE, tracking changes and ensuring fault tolerance through retry mechanisms and offset tracking. The lab involves setting up a three-node ScyllaDB cluster with CDC-enabled tables and configuring a Kafka server using the Confluent platform to visualize and manage these changes as Kafka messages. Despite some limitations, such as the omission of complex data types, this integration facilitates the creation of scalable, real-time data pipelines by converting database changes into Kafka streams that can be consumed by other applications or systems, showcasing the synergy between ScyllaDB and Kafka for modern, event-driven data architectures.

In late November, Guy Shtub attended the OEB learning technologies conference in Berlin, where he delivered a talk and gained insights into the latest trends in educational technology. He highlighted the stark contrast between Berlin's snowy climate and his sunny home in Israel and appreciated the lively atmosphere of the Christmas markets. The conference, held in the Ku’damm area, featured numerous companies showcasing innovations like AI-generated videos, gamification solutions, and advanced LMS platforms. Guy found particular interest in gamification ideas for ScyllaDB University and noted the impressive Zoom addon called Class, which enhances live teaching experiences. Despite some limitations with his current LMS, LearnDash, he observed the potential of integrating Learning Tools Interoperability (LTI) in the future. Discussions with colleagues and talks by experts provided valuable strategies for creating engaging training material and enhancing learner engagement through personalized communication and experiential learning. Guy expressed a desire for a more balanced organization of conference sessions to ensure relevance for all attendees and concluded that the event was a worthwhile experience, recommending it to others in the field.

ScyllaDB Summit 2023 is an annual online conference focusing on the NoSQL database ScyllaDB, featuring a diverse lineup of keynote speakers and sessions led by technical leaders, open-source users, and commercial customers. Highlights include presentations from ScyllaDB Co-Founders Dor Laor and Avi Kivity, as well as talks by industry experts from companies like Discord, Strava, Hulu, and Epic Games, who share their experiences and innovations using ScyllaDB in various applications. Topics covered range from serverless architecture transformations and database migrations to new developments in ScyllaDB, offering attendees insights into cutting-edge database trends and practical use cases. The event promises extensive on-demand content, aiming to engage both returning attendees and newcomers interested in the evolving landscape of cloud computing and database management.

ScyllaDB Enterprise 2022.2 introduces a range of updates and enhancements for its NoSQL database, including over 100 stability and performance fixes. This version aligns more closely with the open-source ScyllaDB project, mirroring features available in ScyllaDB Open Source 5.1. Notable updates include the production-ready Time To Live (TTL) feature for the DynamoDB-compatible API, allowing configurable expiration delays, and per-partition rate limits for managing read and write operations to avoid performance issues. The new Load and Stream feature facilitates easier data restoration and rebalancing across clusters, regardless of topology changes since the last backup. Performance improvements have been made by eliminating exceptions in the read and write paths, while a new CQL extension, PRUNE MATERIALIZED VIEW, helps maintain consistency in materialized views by removing "ghost rows." Existing ScyllaDB Enterprise customers are encouraged to upgrade in coordination with the support team, while ScyllaDB Cloud users will receive automatic upgrades. A 30-day trial is available for potential new users, with recommendations to coordinate with ScyllaDB experts for optimal results.

ScyllaDB University was created to address the need for comprehensive, scalable, and accessible training for users of ScyllaDB's technical products, which was initially challenging due to limited resources for face-to-face training. The initiative began before the COVID-19 pandemic, but demand increased significantly during it, prompting the development of an online, self-paced training program that is free for users and cost-effective for the company. The platform, built using WordPress and the Learndash LMS, offers a flexible and robust solution that aligns with ScyllaDB's open-source culture, featuring multiple courses, hands-on labs, and quizzes. It has evolved to include ScyllaDB University LIVE, a quarterly online event with tracks for beginners and advanced users, fostering interaction and engagement through live presentations and chat features. The success of ScyllaDB University has been reflected in positive feedback from users and its role in enhancing the company's understanding of user engagement, as well as improving search visibility for broader database industry terms. The platform continues to develop with ongoing improvements aimed at increasing community engagement and maintaining current, relevant content.

Choosing a high-performance database for an application requires careful consideration of several factors due to the vast number of options available, with nearly 400 systems ranked by DB-Engines.com as of December 2022. The text highlights the "Cambrian explosion" of database technologies, noting the importance of not confusing popularity with suitability for specific use cases. It outlines five critical factors to consider: software architecture, hardware utilization, interoperability, RASP (Reliability, Availability, Scalability, Serviceability, and Performance), and deployment flexibility. Users must assess aspects such as data structures, query languages, hardware efficiency, integration with development environments, and deployment options to ensure the selected database meets their performance, scalability, and operational needs. The article emphasizes the necessity of evaluating the database’s ability to handle workloads, its suitability for specific transaction and data models, and the importance of avoiding vendor lock-ins.

Bryan Cantrill's talk at the ScyllaDB Summit 2022 reflects on past technological predictions he made with fellow infrastructure technologists at Sun Microsystems, analyzing which predictions were accurate and which were not. His insights emphasize that predictions often reveal more about the present mindset than future events. Notably, while they foresaw developments like the iPhone and the decline of Dennard scaling, they missed major trends such as cloud computing and open source's transformative potential. Cantrill now focuses on how existing technologies can evolve, highlighting the growing ubiquity of compute, the democratizing potential of open FPGAs and HDLs, the maturity of open source EDA tools like KiCad, and the rise of open source firmware. He also emphasizes Rust's revolutionary impact on system software, particularly for embedded systems. Cantrill believes these technologies have reached an inflection point where they will become more widespread and influential due to their open nature, suggesting a future of hardware-software co-design.