July 2022 Summaries

In a Q&A format, Po Hong, a Lead Solutions Architect at Onehouse, shares insights about his career journey and motivations for joining the company, highlighting his extensive experience in big data and cloud computing at AWS, where he played a pivotal role in migrating clients to modern data architectures. He emphasizes his admiration for Apache Hudi's open-source approach and its transformative impact on data lake architectures by incorporating transactional capabilities. Po values the dynamic and impactful nature of working at a startup like Onehouse, which fosters professional growth and innovation. He particularly resonates with the company's customer-centric culture, which mirrors principles he admired at Amazon. As Onehouse continues to grow, he looks forward to expanding its product offerings across major cloud platforms and achieving milestones that demonstrate both product maturity and customer value.

Apache Hudi is an open-source lakehouse technology that is gaining traction in the big data community for its ability to enhance data lakes with transactions, concurrency, upserts, and advanced storage performance optimizations. While it is well-integrated with AWS services like EMR, Redshift, and Glue, Hudi is also emerging as a viable alternative for building data lakes on Microsoft Azure. It seamlessly integrates with Azure services such as Synapse Analytics, HDInsight, and ADLS Gen2, offering flexibility to use open file formats like Parquet with popular query engines such as Apache Spark, Flink, and Hive. The article provides a detailed guide on setting up Hudi within Azure Synapse Analytics, highlighting its capabilities in handling upserts, merges, time travel queries, and efficient incremental data pipelines, along with advanced concurrency controls for data deletion. Despite limited documentation, the guide aims to raise awareness of Hudi's potential on Azure, suggesting it as a robust choice alongside Delta Lake from Databricks for developing scalable data platforms.