February 2025 Summaries

Hex, a data analysis application, was experiencing performance issues due to rapid growth and the complexity of its features. The company turned to virtualization to optimize performance, leveraging React Virtuoso for efficiently rendering large lists. However, this approach proved challenging with nested content, where scrolling was jittery. To address this issue, Hex developed a custom-built VirtualizedElement component that renders only visible elements, using react-intersection-observer to control mounting and unmounting of cells based on their proximity to the viewport. This solution resulted in significant performance improvements across the board, including reduced initial render time, interaction lag, page load, render lag, and memory usage. The custom-built VirtualizedElement component is now a great fit for Hex notebooks, offering direct performance wins and deepened understanding of product needs.

Measuring the impact of most data work is challenging, as it's difficult to quantify improvements in data quality or infrastructure investments. Data leaders often try to measure ROI through baroque spreadsheets, but this approach can be tedious and underwhelming. Instead, the best way to tell the story of a data team's value is for other people to tell it, focusing on what stakeholders will say - do they love what your team is doing and would they recommend it to others? If not, maybe the data team isn't providing enough value. Data teams must internalize that their true value lies in supporting the success of other teams, rather than feeling like a "back office" organization. To drive ROI, data leaders need to convince stakeholders to give a damn about their work and inspire action, recognizing that the "ROI" of insights will round to zero unless they have an impact.

The concept of "insights" in data analysis has been widely discussed, but it's argued that agency is a more important dimension when it comes to the role of data teams. Agency refers to the ability to influence decision-making and drive impact, rather than just providing accurate information. The author suggests that data teams should focus on both accuracy and effectiveness, and that there are four quadrants: High-Impact Partners (high accuracy and high agency), Low-ROI-Trivia Team (high accuracy but low agency), "At Least No One Listens to Them" (low accuracy and low-agency), and Drunk Drivers (low accuracy and high agency). The author believes that many data teams think they're either in the first two quadrants or don't know what to do about it. To achieve agency, data teams need to focus on presentation, socialization, and persuasion, as well as speed and pragmatism. The ability to build trusted relationships and influence among stakeholders is also crucial, but this requires human delivery mechanisms such as charisma and accountability. Ultimately, the author believes that more data people are needed who can figure out how to impact the business with their insights.