Posts tagged "Hypothesis Testing"

3 items found

Use Exact Tests for Nested Experimental Designs

Hierarchical resampling provides exact statistical tests for nested experimental designs by combining bootstrap resampling within experimental units with permutation testing at the randomization level. This approach maintains Type I error control while using all available information, unlike traditional methods that either pool inappropriately or discard useful data.

Analyzing nested experimental designs: A user-friendly resampling method to determine experimental significance

RU Kulkarni, CL Wang, CR Bertozzi
PLoS Computational Biology (2022)

While hierarchical experimental designs are near-ubiquitous in neuroscience and biomedical research, researchers often do not take the structure of their datasets into account while performing statistical hypothesis tests. We present Hierarch, a Python package for analyzing nested experimental designs. Using a combination of permutation resampling and bootstrap aggregation, Hierarch can be used to perform hypothesis tests that maintain nominal Type I error rates and generate confidence intervals that maintain the nominal coverage probability without making distributional assumptions about the dataset of interest.