Date of Award


Document Type

Honors Thesis (Colby Access Only)


Colby College. Mathematics and Statistics Dept.


Liam O'Brien


Hypothesis testing is frequently utilized in a wide range of disciplines as researchers attempt to draw inferences from data. Although most hypothesis tests theoretically require certain assumptions for their accuracy, these assumptions are often not known or simply ignored. The robustness of a test is defined as the ability of the test to withstand assumption violations with respect to its Type I error probability. This study seeks to empirically investigate the robustness of several two-sample hypothesis tests. Through Monte Carlo simulations, exact Type I error rates are calculated for ive different tests under a wide range of simulation setups. Two of the five tests are parametric: Student’s t-test and Welch’s t-test. The remaining three tests are distribution free: the Mann-Whitney U test, a bootstrap-based test, and a permutation test. Results demonstrate that Welch’s t-test, the bootstrap test, and the permutation test perform reasonably well under a variety of assumption violations.


Full-text download restricted to Colby College campus only.


statistics, hypothesis testing, null hypothesis, alternative hypothesis

Multimedia URL