April 2017 Summaries

The 2017 NFL Draft highlighted the ongoing debate among General Managers (GMs) about the value of star running backs like Leonard Fournette and Christian McCaffrey, particularly in the context of the traditional adage that "you have to run to set up the pass." Despite the prominence of successful teams like the Atlanta Falcons, Dallas Cowboys, and New England Patriots, which demonstrated strong running games complemented by high-powered passing offenses, data analysis using Mixpanel suggests that the quality of a running game does not significantly enhance the effectiveness of the passing game. The analysis revealed that while passes following "good" runs gain slightly more yardage on average, the success of a passing play is largely independent of the preceding run's quality. The study also indicated that the NFL is increasingly becoming a pass-heavy league, implying that GMs should focus on drafting talent that aligns with this trend, rather than relying heavily on running backs to drive offensive success.

In a conversation between a Wall Street bank's Chief Digital Officer (CDO) and analytics expert Tom Davenport, the importance of having a clear data strategy was highlighted, as many enterprises collect vast amounts of data but lack clarity on its operationalization. Davenport emphasizes that a data strategy should go beyond mere collection to include purpose and actionable use, advocating for a balanced approach between offense and defense in data management. He believes that enterprises often default to defensive tactics such as cybersecurity due to fear of mishaps, which limits experimentation and innovation. Instead, Davenport suggests a mixed approach where analytics is used offensively to drive revenue, create new products, and enhance customer relationships. He proposes restructuring teams to be more decentralized, promoting better collaboration between technology and business units, and embedding analytics into business processes for true operational analytics. The ultimate goal is for companies to define a comprehensive data strategy that aligns with business objectives, ensuring they are truly data-driven rather than merely collecting data without purpose.

The concept of an "aha moment" in product development is often seen as a mythical turning point that leads to user retention, but it lacks rigor and may not effectively drive product improvement. Mixpanel's journey with their machine learning products, Predict and Signal, illustrates this challenge. Predict aimed to identify valuable users based on their actions but faced criticism for not explaining why certain users were more likely to convert. This insight led to the development of Signal, which focuses on understanding user engagement through actionable insights rather than just identifying a single pivotal moment. Signal analyzes user behavior patterns to provide deeper insights into how certain actions correlate with user retention, emphasizing the importance of frequency and timing. By moving beyond the simplistic "aha moment" and focusing on a lifecycle perspective, Signal helps product managers refine their products based on data-driven insights, enabling them to make informed decisions quickly. This approach underscores the importance of aligning machine learning tools with the fundamental question of how to improve a product, offering a more nuanced understanding of user engagement.

Mixpanel's Signal is a tool designed to help product teams identify user behaviors that correlate with higher retention and engagement by automating the process of data analysis, allowing users to validate assumptions quickly. Inspired by examples like Facebook's "seven friends in 10 days" for user retention, Signal enables users to discover specific actions that impact conversion rates, reducing guesswork and freeing data scientists to focus on more complex issues. By analyzing the statistical relationship between user actions and retention metrics, Signal provides actionable insights with Key Findings that explain the significance of each result, eliminating false positives and highlighting hidden drivers of retention. This tool is accessible to Mixpanel users under the Engagement tab, where they can explore correlations and optimize user actions based on data-driven insights.