September 2024 Summaries

Hex has increased execution speeds up to 10x by migrating from pandas Dataframes to a DuckDB-based architecture that directly queries Arrow data stored remotely in S3, instead of materializing dataframes into local memory. This new architecture uses lazy loading for dataframes and has seen improvements to project runtimes in the ballpark of 5-10x speedups, with some internal projects going from 30+ second runtimes to just a handful of seconds. The performance gains are most pronounced in projects that primarily use SQL and no-code cells, while projects that include a lot of Python references will see less dramatic improvements.

First-class security practices are essential for data tools to protect against legal risks, financial penalties, and loss of customer trust, which can be detrimental to a company. Hex emphasizes the significance of robust security protocols and documentation, having formed year-long partnerships with enterprises due to their meticulous attention to security details. A comprehensive checklist is recommended when evaluating the security of any data analytics or BI tool, focusing on storage and encryption, data handling, security policies, access control, compliance with regulations like HIPAA and GDPR, incident response, and vendor reputation. The checklist includes considerations such as encryption standards, data retention policies, risk assessments, user access controls, regulatory compliance, incident management, and the reliability of vendor security practices. Hex underscores the importance of security not only as a feature but as a fundamental necessity for safeguarding valuable company data, advocating for transparency and adherence to industry standards to ensure trust and reliability.

GameChanger, an app by Dick's Sporting Goods, needed a clear lifetime value (LTV) metric at the user level to improve marketing, finance, and product strategies. The data team used Hex notebook for data preparation and exploration, then focused on LTV components: "lifetime" (time-to-churn) and "value" (monthly revenue per user). They modeled LTV for both paid and free users, incorporating advertising potential and likelihood of subscription. The insights were portrayed in their BI tool to answer questions about churn trends and average LTV. The product, marketing, and finance teams used the LTV metric to improve forecasts and strategies.

Data analysts often face challenges when using different tools for data analysis, leading to inefficiencies and hindered insights. Hex, a multi-modal analytics workspace, addresses this issue by allowing users to work with their preferred languages or tools within the same platform. This approach enables teams to adopt hybrid workflows that are more efficient and versatile, ultimately speeding up critical data projects. By supporting SQL, Python, and no-code cells, Hex empowers analysts to choose the most suitable method for each task without being limited by tool constraints.