Posts tagged "Hypothesis Testing"

4 items found

An Empirical Bayes Approach to Churn Estimation

Update, August 2026: I no longer stand behind this approach. The model below measures how surprising a user’s silence is against their own history, and in production that turned out to flag the users least likely to leave. I’ve written up what went wrong and what replaced it in Churn Risk Is a Conditional Rate. The original post is left as written.

Analyzing nested experimental designs: A user-friendly resampling method to determine experimental significance

RU Kulkarni, CL Wang, CR Bertozzi
PLoS Computational Biology (2022)

While hierarchical experimental designs are near-ubiquitous in neuroscience and biomedical research, researchers often do not take the structure of their datasets into account while performing statistical hypothesis tests. We present Hierarch, a Python package for analyzing nested experimental designs. Using a combination of permutation resampling and bootstrap aggregation, Hierarch can be used to perform hypothesis tests that maintain nominal Type I error rates and generate confidence intervals that maintain the nominal coverage probability without making distributional assumptions about the dataset of interest.