October 2022 Summaries

YugabyteDB has successfully completed its Service Organization Controls (SOC) 2 Type 2 and SOC 3 attestations, covering its distributed SQL database, YugabyteDB Anywhere, and YugabyteDB Managed, demonstrating its commitment to security and data protection. This achievement provides enterprises with confidence in their providers' processes and controls, particularly for business-critical transactional applications requiring strong consistency, high availability, and security. The SOC 2 Type 2 audit evaluates an organization's security program over a specified period, confirming that Yugabyte effectively followed documented security systems and controls without exception, earning an unmodified opinion from the independent accounting firm. This attestation is a testament to Yugabyte's dedication to defining, implementing, and following a comprehensive security program, giving enterprises peace of mind regarding their data's safety.

Saul Van Beurden, Head of Technology at Wells Fargo, discussed the bank's modernization, innovation, and digital transformation initiatives with Yugabyte CEO Bill Cook during a fireside chat. The conversation covered key areas such as building skills for future technologies, ensuring secure and stable plans, driving business speed, scalability, and full cloud migration to both public and private clouds. Van Beurden emphasized the importance of knowing one's point of departure well, articulating an ambition and vision, breaking down strategic pillars or initiatives, getting resources and priorities, promoting or hiring talent onto the team, and baking compliance and security requirements into solutions. He also highlighted the need for a "stacked" approach to technology, distinguishing new applications from existing workloads, and ensuring upskilling is adopted at scale through the entire organization.

The difference between client-side and server-side latencies arises from how SQL statements are processed by PostgreSQL and YugabyteDB databases, with the extended query protocol allowing for more granular measurement of processing steps. The pg_stat_statements view primarily measures execution step latency, excluding parsing and planning steps, which can also take time. Understanding the application or client side timing requires knowledge of what exactly is timed, with the simple query protocol providing a straightforward measure but the extended query protocol including additional steps in the client-side observed latency. Server-side logging using the pg_min_duration_statement parameter provides a full picture of statement processing latency when the extended query protocol is used.

YugabyteDB Managed is a fully managed offering of the YugabyteDB database, delivering power and ease of use in one product. It's a distributed SQL database that can be created and connected to in minutes with zero operational overhead. Developers love it for its Postgres compatibility, linear scalability, and resilience, but want to focus on building modern cloud-native applications rather than managing infrastructure and DBA tasks. YugabyteDB Managed provides instant productivity, security by design, world-class operations, freedom to deploy globally, and elastic pricing. It abstracts away underlying cloud complexities, giving users the freedom to build a cloud-agnostic operational model and manage a database across different cloud providers.

YugabyteDB databases can export data to Amazon S3 using Change Data Capture (CDC) and open table formats like Apache Iceberg, achieving low-latency data ingestion while avoiding costly rewrites. Log-based CDC is recommended for low latency export due to its ability to capture all change types and process changes without competing with other workloads. File formats such as CSV, JSON, Parquet, and ORC are popular choices for building file-based data lakes on object stores like Amazon S3. Transactional data lake technologies like Apache Iceberg add metadata layers to provide ACID transactions and schema evolution. A CDC pipeline using Debezium and Kafka supports low-latency export, consisting of two stages: exporting CDC events from YugabyteDB to Kafka and writing records to Apache Iceberg tables in Amazon S3. The pipeline provides separation of concerns between the two stages, allowing each component to be paused or restarted independently. Two modes are available when writing data to Amazon S3: Trace Inserts, Updates, and Deletes, and Replay Inserts, Updates, and Deletes, with the latter providing atomic transactions and primary key-level ordering of transactions sufficient for most OLAP databases.

As a Microsoft partner, Yugabyte is now part of the Microsoft Intelligent Data Platform ecosystem, offering native integration between YugabyteDB and Azure Synapse Analytics to simplify data connection and analysis. Businesses are embracing modern database solutions like YugabyteDB to store and deliver strongly consistent data, but face challenges in integrating transactional data with analytics platforms. The new integration enables a 1-click experience to stream data changes from YugabyteDB into Azure Synapse Analytics, eliminating complexity and automating CDC integration. This native integration accelerates time to value, democratizes data-driven decisions, and eliminates data silos, allowing businesses to bring together data from transactional systems for enhanced analytics.

Wells Fargo is transitioning to a distributed SQL platform to address the need for resiliency, scalability, and manageability in their database estate. They are looking for an ACID-compliant relational platform that can support the majority of their workloads, with a focus on near linear horizontal scalability. The company wants to ensure that the new platform is not only technically sound but also has a strong community of users and developers who can build awareness and adoption around it. Wells Fargo aims to use distributed SQL as a general-purpose platform for relational computing, with a commitment to evolving and maturing it over time.

Transparency is crucial for scaling a business as it builds trust, fosters long-lasting relationships, drives outcomes through impact, ownership, and technical excellence, and creates a culture of openness, security, and mutual respect with stakeholders. This transparency allows companies to navigate growth challenges effectively, maintain a steady pace while creating a strong globally distributed team, and ultimately achieve success by focusing on people-first strategies that elevate product quality, customer service, and business scale.

YugabyteDB Anywhere provides an API to capture table-level size metrics, which can be used to track the growth of tables and databases over time. To do this, users need to create required database objects such as a database, sequence, and table to store the data. They also need to configure Python modules and environment variables with the necessary credentials and IP addresses. Once these are set up, users can capture metrics by connecting to the YugabyteDB platform API and storing the data in a persistent table using PostgreSQL. The data is then validated to ensure it has been successfully captured, and the process can be scheduled to run daily.