Hypothesis Testing explained in 4 parts

The article explores the complexities of hypothesis testing, highlighting the often inconsistent blend of p-value and significance testing with traditional hypothesis testing in educational resources. It introduces ten key concepts incrementally, using visualizations and intuitive explanations to clarify them, including the distinction between standard deviation and standard error, and the conditions under which a hypothesis can be "accepted" or "failed to reject." The article delves into the role of alpha, beta, type I and II errors, and the critical value with the null hypothesis, as well as the concept of power and minimum detectable effects (MDE) in testing. The Neyman-Pearson framework is emphasized throughout, focusing on the practical applications of hypothesis testing and the fundamental tradeoff between mistake and discovery. The discussion covers the relationships between sample size, power, and MDE, offering practical recommendations for using power effectively in experiments and addressing common pitfalls like peeking and multiple comparisons.