Hierarchical resampling provides exact statistical tests for nested experimental designs by combining bootstrap resampling within experimental units with permutation testing at the randomization level. This approach maintains Type I error control while using all available information, unlike traditional methods that either pool inappropriately or discard useful data.
A guide to planning scientific projects and asking the right research questions during your PhD, based on insights from my piece for New Science on strategic research planning. Check out the full article here.
RU Kulkarni, CL Wang, CR Bertozzi
PLoS Computational Biology (2022)
While hierarchical experimental designs are near-ubiquitous in
neuroscience and biomedical research, researchers often do not
take the structure of their datasets into account while performing
statistical hypothesis tests. We present Hierarch, a Python package for
analyzing nested experimental designs. Using a combination of permutation
resampling and bootstrap aggregation, Hierarch can be used to
perform hypothesis tests that maintain nominal Type I error rates
and generate confidence intervals that maintain the nominal
coverage probability without making distributional assumptions
about the dataset of interest.