Post Hoc Tests – Pairwise Comparisons with corrections


The function pairwise.t.test() allows for performing multiple t-tests on a multitude of groups using all possible pairwise comparisons. This comes handy as a follow up to one-way ANOVA, two-way ANOVA, … In addition, these pairwise t-tests assume normality of distribution and equality of variances which you needed anyway to perform ANOVA.

This function is rather simple: the syntax is pairwise.t.test(data, groups, p.adjust.method= ) where data is the vector containing your dependent variable and groups is the vector containing the grouping factor. In addition, pairwise.t.test() allows you to apply the method of your choice when it comes to correcting/adjusting p-values to control type I errors. Simply add the name or abbreviation for the method to the argument p.adjust.method=. The methods are none (no correction), bonferroni, holm, hochberg, hommel, BH and BY. Choosing the right correction for your analysis depends a lot on your experimental design… and certainly a bit in you wish to find lower p-values at the cost of making type I errors…

Let’s the example from one-way ANOVA where the dataframe and ANOVA where coded the following way:

size<-c(25,22,28,24,26,24,22,21,23,25,26,30,25,24,21,27,28,23,25,24,20,22,24,23,22,24,20,19,21,22)
location<-c(rep("ForestA",10), rep("ForestB",10), rep("ForestC",10))
my.dataframe<-data.frame(size,location)
results<-aov(size~location, data=my.dataframe)
summary(results)

Now let’s look at 3 cases of post hoc pairwise t-tests, the first one without correction, the second one with bonferroni correction and the last one with hochberg correction:

 
pairwise.t.test(size, location, p.adjust.method="none")
pairwise.t.test(size, location, p.adjust.method="bonferroni")
pairwise.t.test(size, location, p.adjust.method="hochberg")

Skjermbilde 2016-08-08 13.01.54

The three outputs are presented above. Each one displays the same type of table with pairwise comparisons, the last line shows the adjustment method. The results are very different in these 3 cases. Using no correction or hochberg correction, 2 p-values drop under 0.05 (yellow and blue boxes), whereas only one p-value is less than 0.05 when using bonferroni correction. This difference may be decisive for interpreting the outcome of your experiment and may change a lot the conclusions of your research. Therefore, it is important that you weigh the pros and cons when deciding which adjustment method to apply to your dataset.