June 2022 Summaries

The role of Chief Data Officer (CDO) has evolved significantly over the past decade, from managing data to driving innovation and transformation within organizations. However, CDOs face high expectations and short job tenures due to challenges such as failed data migrations, inability to support new data workflows, and difficulties in maintaining data governance. To navigate these complexities, third-wave CDOs require access to a best-of-breed data observability solution that provides operational intelligence across all layers of their data infrastructure. This will enable them to monitor the state of their data landscape, harmonize diverse data types, and ensure data reliability and accuracy for mission-critical operations.

Data engineers are crucial to modern businesses, creating systems that gather, handle, and transform unprocessed data into valuable information for data scientists and business analysts. Their role is essential in ensuring reliable data infrastructure and accessibility, which has become increasingly important as companies undergo digital transformation and generate more complex data at higher frequencies. Data engineers need skills such as stream processing, data lake technologies, data transformation frameworks, and orchestration of pipeline technologies to build pipelines that fill data lakes and real-time data streams. Their work contributes significantly to company growth by enabling effective use of data for business decisions and improving the ROI of data investments.

Tristan Spaulding, Acceldata's Head of Product, and Paige Bartley, Senior Research Analyst at 451 Research, discussed the increasing demand for data observability solutions in a session titled "Prevention vs Cure: Modern Data Management Calls for Data Observability." They assessed the rise of data observability as enterprises migrate their data operations to the cloud. The conversation covered the complexity of modern data environments and the challenges faced by data teams, including increased pain points and responsibilities. Data observability emerged as an essential tool for maintaining effective data operational management in light of these complexities.

Acceldata has integrated its Data Observability platform with Snowflake, a leading data ecosystem used by many global brands. During the recent Snowflake Summit, Acceldata introduced the Acceldata Data Observability Cloud for Snowflake to help users maximize their return on investment in Snowflake. The solution provides comprehensive visibility and insight into performance, quality, and cost of data environments. Attendees at the summit showed great interest in how Acceldata's multidimensional approach can improve data reliability and pipeline health.

The Chief Data Officer (CDO) plays a crucial role in modern businesses, requiring vision, strategy, execution, and real-time problem solving to optimize data infrastructure and deliver business benefits. Acceldata's VP of Global Sales and Field Operations, Dave Yehle, emphasizes the importance of integrating data with business and strategic decisions for better decision-making and improved outcomes. Little Caesars' VP of Global Data and Analytics, Drew Smith, shares how this approach helps in making menu decisions, improving operations, and forecasting restaurant traffic. The key traits a CDO should possess include responsibility, accountability, and the ability to connect data with strategy for faster decision-making with confidence.

Apache Kafka is widely used in modern data stacks due to its ability to process real-time messages and large volumes of data daily. Originally developed by LinkedIn engineers, Kafka has matured over the years and now powers many enterprises like Uber, Walmart, Twitter, and Tencent. However, most companies still own a variety of small Kafka clusters managed using different tools and best practices, leading to higher management overhead, lack of economies of scale, data blind spots, and hindered innovation. Acceldata provides a turnkey solution that grants enterprises ongoing visibility into Kafka and their data pipelines, helping prevent potential problems and enabling cost optimization.

Snowflake is a cloud-based data warehousing platform that offers scalable computing resources and seamless data pipelines, but its pay-per-usage pricing model can lead to unexpectedly high costs if not managed properly. Acceldata's Data Observability Cloud provides advanced features for monitoring and optimizing Snowflake usage, including cost tracking across all services, anomaly detection, granular resource cost analysis, storage repository insights, timeout investigation, admin-level insights, capacity planning, and query performance metrics. By using Acceldata, users can gain more control over their Snowflake environment, predict costs, improve performance, and optimize the value they get out of their annual contract in a matter of minutes.

PubMatic, a leading US advertising technology company, has implemented Acceldata's Pulse product to improve its data infrastructure and reduce costs. The company serves 200 billion ad impressions daily and processes over 2 Petabytes of new data. Its large-scale environment includes thousands of nodes and multiple open-source Hadoop big data stack tools. However, frequent outages, performance bottlenecks, and high MTTR metrics were causing operational issues. By using Acceldata Pulse, PubMatic has improved its data system performance, reduced 'cost per ad impression', decreased OEM support costs by $10 million, optimized HDFS storage block footprint by 30%, consolidated Kafka clusters, and saved millions of dollars in unnecessary software licenses.