P -values in genomics: Apparent precision masks high uncertainty

摘要: Scientists often interpret P-values as measures of the relative strength statistical findings. This is common practice in large-scale genomic studies where are used to choose which numerous hypothesis test results should be pursued subsequent research. In this study, we examine P-value variability assess degree certainty provide. We develop prediction intervals for a replication study given observed an initial study. The depend on value P and ratio sample sizes between studies, but not underlying effect size or size. valid most large-sample tests any context, can presence single multiple tests. While highly variable, future explicitly predicted based from important predictor provide handy calculator implementing these apply them Alzheimer's disease recent findings Cross-Disorder Group Psychiatric Genomics Consortium. suggests that overinterpretation very significant, factor contributing unexpectedly high incidence non-replication. Formal also realistic interpretations comparisons associated with different estimated sizes.