Publication

Hidden multiplicity in multiway ANOVA: Prevalence, consequences, and remedies. Prevalence and remedies

Cramer, A. O. J., van Ravenzwaaij, D., Matzke, D., Steingroever, H., Wetzels, R., Grasman, R. P. P. P., Waldorp, L. J. & Wagenmakers, E-J., Apr-2016, In : Psychonomic Bulletin & Review. 23, 2, p. 640-647 8 p.

Research output: Contribution to journalArticleAcademicpeer-review

Copy link to clipboard

Documents

DOI

  • Angélique O.J. Cramer
  • Don van Ravenzwaaij
  • Dora Matzke
  • Helen Steingroever
  • Ruud Wetzels
  • Raoul P.P.P. Grasman
  • Lourens J. Waldorp
  • Eric-Jan Wagenmakers
Many psychologists do not realize that exploratory use of the popular multiway analysis of variance harbors a multiple-comparison problem. In the case of two factors, three separate null hypotheses are subject to test (i.e., two main effects and one interaction). Consequently, the probability of at least one Type I error (if all null hypotheses are true) is 14 % rather than 5 %, if the three tests are independent. We explain the multiple-comparison problem and demonstrate that researchers almost never correct for it. To mitigate the problem, we describe four remedies: the omnibus F test, control of the familywise error rate, control of the false discovery rate, and preregistration of the hypotheses.
Original languageEnglish
Pages (from-to)640-647
Number of pages8
JournalPsychonomic Bulletin & Review
Volume23
Issue number2
Early online date15-Sep-2015
Publication statusPublished - Apr-2016

    Keywords

  • FALSE DISCOVERY RATE, BONFERRONI PROCEDURE, REGISTERED-REPORTS, STATISTICAL POWER, DESIGN, TESTS

View graph of relations

Download statistics

No data available

ID: 27952633