September 2022 Summaries

The individual began their engineering career driven by the desire to solve real-world problems and build things from scratch. They have worked with several companies including IBM Global Business Services, Roamware, GE Healthcare, Oracle, 247.ai, and Hortonworks before joining Acceldata as a Principal Engineer. The person appreciates Acceldata's company culture for its transparency, open communication, problem-focused approach, and creative freedom. They lead a team of ten people with diverse skill sets in engineering and analytics, emphasizing the importance of hiring individuals with the right attitude and skills, as well as trust-based teamwork. The person shares Acceldata's mission to create the most valuable data observability solution for enterprises. Their significant contributions include building the engineering infrastructure and making wise judgments early on. They believe in leveraging resources suited to specific situations and would have considered a career in teaching Math and Physics if not in their current field. A fun fact about them is that they can survive on eating just Biryani for life, and their favorite place is Budapest. Their personal mantra keeps changing with time, and they enjoy watching Christmas movies. In the past few months, they have been learning about interior design while renovating their home.

Snowflake is a popular cloud data warehouse that has experienced significant growth in recent years, boasting over 6,800 enterprise customers and $1.2 billion in annual revenue. The company's success can be attributed to its low start-up costs, constant innovation, and manageability features. A survey conducted by Acceldata reveals that cost is the primary concern for Snowflake users, followed closely by data quality and performance. Most respondents have been using Snowflake for 1-3 years and have annual contracts with a contract size typically between $250,000 - $1 million. The majority of these users also employ other data platforms alongside Snowflake. As organizations grow, so does their Snowflake environment, with almost half of respondents having more than 51 Snowflake users in their organization.

Apache Kafka has become the standard for real-time streaming data, with companies increasingly adopting it to build and operate great data products. However, managing Kafka clusters can be difficult, time-consuming, and expensive. To address this issue, many enterprises are turning to data observability platforms that provide automated visibility and control over their environments. Companies have two main options for using Kafka: Kafka-as-a-service or fully-managed Kafka. While the latter reduces operational hassle, it does not eliminate the need for continuous monitoring and management of costs. A data observability platform like Acceldata can help companies optimize their Kafka environments by providing ongoing visibility, alerts, and recommendations to prevent cost overruns and improve overall performance.

Raghu Mitra, co-founder and Director of Engineering at Acceldata, started his career as a software engineer in 2005. He quickly realized he enjoyed problem-solving and saw the world of data had a huge problem: projects were almost impossible to complete. This revelation led him to co-found Acceldata, an enterprise data observability leader. Raghu's typical day as a co-founder involves checking messages in the morning, attending standard meetings, ensuring team members are unblocked, working on code, and coordinating with international teams. He believes that being part of Acceldata is worth it because he can make a difference and solve the massive problem data enterprises face.

Organizations are increasingly leveraging Hadoop data observability to optimize their data transformation programs, simplify cloud migration, and improve overall efficiency. The Acceldata Data Observability platform helps data teams maximize ROI from their data stack by providing visibility into complex environments. As Cloudera ends support for legacy Hadoop platforms, organizations have multiple options including migrating to CDP, modernizing on alternative platforms, or self-support models. However, a fourth option exists with Acceldata's Data Observability platform and support services, offering lower risk, cost, and talent requirements while maintaining flexibility in current and future big data environments.

The Rubik's Cube and managing DataOps share several similarities. Both involve complex logic problems that require strategies and best practices to solve effectively. The Rubik's Cube has millions of starting positions, while modern data infrastructures are multi-layered systems with petabytes or exabytes of data. Solving the Rubik's Cube requires algorithms and sequences of moves, much like managing DataOps necessitates a well-thought-out strategy. Both fields have thriving expert communities that require specialized knowledge and best practices to excel in. Additionally, variants of the Rubik's Cube can scale incredibly in complexity, similar to how data operations teams and infrastructures can vary widely in size. Lastly, both are solvable with the aid of best practices and algorithms, emphasizing the importance of implementing strategies and tools for efficient management.