Assessing Effects of Unreplicated Perturbations: No Simple Solutions

摘要: We address the task of determining effects, on mean population density or other parameters, an unreplicated perturbation, such as arises in environmental assessments and some ecosystem-level experiments. Our context is Before-After-Con- trol-Impact-Pairs design (BACIP): several dates Before After samples are collected simultaneously at both Impact site a nearby "Control." One approach to test whether Impact-Control difference has changed from perturbation. If conventional used, checks its as- sumptions important messy part analysis, since BACIP data do not necessarily satisfy them. It been suggested that these needed for ran- domization tests, because they insensitive assumptions can be adjusted allow others. A major aim this paper refute suggestion: there no panacea difficult technical problems analysis compare randomization t with standard modified (Welch- Satterthwaite-Aspin) test, which allows unequal variances. conclude less likely yield valid inferences than Welch it requires identical distributions small sample sizes either equal variances larger ones. The formal requirement Normality crucial test. Both parametric tests require time location effects additive differences different independent. These should tested; if seriously wrong, alternative analyses needed. This will often long series data. Finally, assessing importance P value hypothesis rarely useful estimate size effect. Especially effect varies conditions, flexible estimation methods approximate answers preferable formally exact values.