East Carolina University

Department of Psychology

**Multiple Comparisons Among
Proportions**

My friend, Laura Anderlozzi, has conducted a 2 x 6 test and obtained significant results. The editor of the journal to which she submitted the manuscript asked her to make pairwise comparisons. What is she to do now (message Professor Karl on Facebook and ask for help). There are several options.

**What Comparisons are to Be Made?**

There are six columns, A, B, C, D, E, and F, and two rows,
1 and 2. Suppose we wanted to consider only columns A and B. We
compute for column A the proportion of scores in row 1 (call this *p _{1}*), and for
column B we compute the proportion of scores in row 1 (call this

**Run Multiple 2 x 2 Pearson Chi-Square Analyses**

Here you would have to run 15 contingency table analyses, rows x (AB, AC, AD, AE, AF, BC, BD, BE, BF, CD, CE, CF, DE, DF, EF), each comparing the proportion in one column with that in another column. Aside from this being rather tedious, one may be concerned about familywise error rate when conducting so many comparisons. You could use the Bonferroni or the Sidak inequality to cap familywise error rate.

**Find an Online Calculator for Comparing Two Proportions**

There are lots of such calculators out there. Make each of the (in this case 15) comparisons. Possibly less tedious than the multiple 2 x 2 Chi-Square analyses, but the familywise error concern remains.

**Use My SPSS
Macro for Comparing Two Proportions**

This should be less tedious than the methods above. You enter one line of summary data (proportions and sample sizes) for each comparison and then run the syntax file. The output is confidence intervals. If an interval does not contain zero, the comparison is significant.

**Google "Marascuilo Procedure"**

You will get lots of hits. Doing this by hand is tedious. I was unable to find an online calculator to do it.

**Use the
Multiple Comparison Test for Proportions in a 2xc Crosstabulation in SAS Macro**

If you are comfortable with SAS, this seems like a good
solution. The approach taken is very similar to that which Tukey
developed for
making pairwise comparisons among means (HSD). When interpreting the
output, remember that the differences between group proportions (Diff) is after
the arcsin transformation that is applied prior to analysis. These will
not be the same as the differences between the untransformed group proportions.
Also note that the output may indicate that a difference between proportions is
significant even when their difference is less than that of a difference already
declared not significant. For example, for ordered means A, B, C, D, E, F,
the output may indicate that the difference between B and D is significant even
though the difference between A and D is not. In such cases, I declare the
difference between B and D not to be significant. APA-style reference:
Elliott, A. C., & Reisch, J. S. (2006, March). *Implementing a multiple
comparison test for proportions in a 2xc crosstabulation in SASŪ*. Paper
presented at the SAS Users Group International Conference, San Francisco, CA.
Retrieved from http://www2.sas.com/proceedings/sugi31/204-31.pdf

**Small Expected Frequencies**

If you have a column with small total counts, comparisons involving that column will lack power. My usual advice for dealing with this is to delete any such column from the analysis or combine it with another column (if that makes sense).

Contact Information for the Webmaster,

Dr. Karl L. Wuensch

This page most recently revised on the 4^{th} of March, 2015.