January 2022 Summaries

Cloud migration has become a significant trend, driven by various factors such as cost savings, elasticity, and enhanced security. A study by Starburst and Red Hat revealed that the majority of data is transitioning from on-premises to cloud environments, with a predicted increase from 56% to 62% within a year. The flexibility to operate across multiple cloud platforms and the ability to scale resources efficiently without physical constraints are key advantages attracting businesses. Additionally, cloud environments empower end-users by simplifying access to hardware and tools, ultimately boosting productivity and reducing analysis time. Security is another critical factor, as cloud providers continually update their platforms to offer robust data protection. Starburst Enterprise aids in this transition by acting as a single point of access, allowing seamless data migration and enabling compliance with global regulations while maintaining efficient data handling and accessibility.

Starburst Galaxy is an advanced data integration platform designed to enhance innovation by providing a robust embedded data analytics engine for accessing data across multiple platforms, sources, and clouds. Developed from the open-source Trino engine, Starburst Galaxy can quickly provision new clusters within popular cloud providers and regions, ensuring optimal performance and eliminating egress costs. The platform supports multi-cloud environments, allowing customers to build scalable, revenue-generating platforms without worrying about infrastructure challenges. With features like adjustable cost-performance ratios, unlimited cluster creation, and an idle-shutdown mechanism to reduce costs, Starburst Galaxy enables companies to focus on core business functions while efficiently managing their data ecosystems. By freeing users from infrastructure concerns, it empowers them to harness cross-cloud analytics and generate new business value.

Digital transformation is reshaping business models by integrating digital capabilities with existing processes and infrastructure, emphasizing the importance of leveraging current investments rather than starting from scratch. As companies face growing demands for data-driven insights, the focus should be on providing business users with access to aggregated data products in a self-service manner, rather than merely granting access to databases. This can be achieved by adopting a Data Mesh strategy, which allows for agile connection and fast analytics across existing data sources without the need for disruptive migrations. The approach emphasizes the importance of maintaining architectural flexibility, enabling back-end design optimizations without front-end disruptions, and ultimately shifting the focus from data infrastructure to business insights and outcomes. As digital capabilities continue to advance with technologies like 5G and AI, businesses must prepare for ongoing change while ensuring transformation programs are driven by business solutions.

Starburst Galaxy is a fully-managed cloud service designed to simplify access and management of data queries, built upon the open-source Massively Parallel Processing (MPP) SQL query engine Trino. It enables users to connect to data lakes and start querying within minutes, eliminating the need for complex hardware provisioning or software configurations. Starburst Galaxy offers a seamless, intuitive experience with native security features and cost-control mechanisms such as idle shutdowns, allowing users to manage clusters efficiently while ensuring only authorized access to data. The platform provides automatic upgrades and patches, ensuring users have immediate access to the latest features and performance enhancements, supported by a global team of Trino experts.

In the article, the author discusses the challenges of using Tableau's cross-database joins for large-scale data queries, highlighting that while the feature allows for cross-database querying, it can be inefficient and costly when dealing with significant data volumes. The author explores an alternative approach using Starburst, a tool designed for federated queries, which significantly improves performance by minimizing data movement and reducing query execution time. Through a practical example involving TPC-H sample data, the author demonstrates how Starburst efficiently handles federated queries by dynamically filtering and joining data from various sources, resulting in faster visualization rendering in Tableau. This approach is presented as a more effective solution for handling large datasets, emphasizing the flexibility and scalability of Starburst, especially with its SaaS offering, Starburst Galaxy, which provides easy deployment and resource management options for users.

Data Mesh is a decentralized approach to enterprise data management that emphasizes federated computational governance, as outlined in Zhamak Dehghani's O’Reilly book "Data Mesh: Delivering Data-Driven Value at Scale." This governance model focuses on automating data usability, security, compliance, and quality through a federated system where individual domains manage their data products while adhering to global standards. Systems thinking is employed to view the mesh as an interconnected ecosystem, promoting a balance between domain autonomy and global interoperability. The governance model is supported by a federated team that includes domain representatives and platform stakeholders, with guidelines and incentives to ensure effective collaboration and policy adherence. Computational governance, enabled by automation, integrates governance functions seamlessly into data products, thereby enhancing the efficiency and effectiveness of data management within the mesh framework.

Salesforce's Customer 360, a powerful SaaS application, provides businesses with a comprehensive view of their customers by integrating various data points across sales, service, and marketing. However, traditional approaches to analytics, which often involve complex ETL processes and data replication, can hinder the speed and efficiency of gaining insights. Starburst offers a solution with its enterprise-grade platform that allows for real-time analytics without the need for extensive data movement. With over 40 connectors, including those for Salesforce and other enterprise systems like SAP and data lakes, Starburst enables data professionals to use standard ANSI SQL for querying and federating data from multiple sources. This approach not only leverages Salesforce's capabilities but also integrates seamlessly with data visualization tools like Tableau, facilitating live data connections and reducing technical debt. As a result, businesses can achieve faster insights and enhance their decision-making processes, regardless of where their data resides.

Data Mesh is a paradigm shift from centralized data management to a decentralized system, emphasizing federated computational governance as a core principle to balance domain autonomy with global standards. This governance model delineates responsibilities between domain-specific and centralized IT functions, ensuring that data governance aspects like security, compliance, availability, quality, and provenance are effectively managed. In this setup, domains have autonomy over data quality and provenance, while centralized IT handles broader governance aspects like inter-domain standards and authentication. The model encourages collaboration through a cross-domain governance council, fostering shared responsibility and agility in adapting to compliance and standard changes. Starburst supports this model by providing tools for secure and efficient data management without excessive replication, facilitating governance through integration with various data catalogs and lineage products.