Beyond Power Calculations Assessing Type S (Sign) and Type M (Magnitude) Errors

摘要: Statistical power analysis provides the conventional approach to assess error rates when designing a research study. However, is flawed in that narrow emphasis on statistical significance placed as primary focus of study design. In noisy, small-sample settings, statistically significant results can often be misleading. To help researchers address this problem context their own studies, we recommend design calculations which (a) probability an estimate being wrong direction (Type S [sign] error) and (b) factor by magnitude effect might overestimated M [magnitude] or exaggeration ratio) are estimated. We illustrate with examples from recent published discuss largest challenge calculation: coming up reasonable estimates plausible sizes based external information.