Publication

Dealing with Distributional Assumptions in Preregistered Research

Williams, M. N. & Albers, C. J., 12-Sep-2019, In : Meta-Psychology. 3, 15 p.

Research output: Contribution to journalArticleAcademicpeer-review

APA

Williams, M. N., & Albers, C. J. (2019). Dealing with Distributional Assumptions in Preregistered Research. Meta-Psychology, 3. https://doi.org/10.15626/MP.2018.1592

Author

Williams, Matt N. ; Albers, Casper J. / Dealing with Distributional Assumptions in Preregistered Research. In: Meta-Psychology. 2019 ; Vol. 3.

Harvard

Williams, MN & Albers, CJ 2019, 'Dealing with Distributional Assumptions in Preregistered Research', Meta-Psychology, vol. 3. https://doi.org/10.15626/MP.2018.1592

Standard

Dealing with Distributional Assumptions in Preregistered Research. / Williams, Matt N.; Albers, Casper J.

In: Meta-Psychology, Vol. 3, 12.09.2019.

Research output: Contribution to journalArticleAcademicpeer-review

Vancouver

Williams MN, Albers CJ. Dealing with Distributional Assumptions in Preregistered Research. Meta-Psychology. 2019 Sep 12;3. https://doi.org/10.15626/MP.2018.1592


BibTeX

@article{c8e22f2cced8494ab9464b9ba7282501,
title = "Dealing with Distributional Assumptions in Preregistered Research",
abstract = "Virtually any inferential statistical analysis relies on distributional assumptions of some kind. The violation of distributional assumptions can result in consequences ranging from small changes to error rates through to substantially biased estimates and parameters fundamentally losing their intended interpretations. Conventionally, researchers have conducted assumption checks after collecting data, and then changed the primary analysis technique if violations of distributional assumptions are observed. An approach to dealing with distributional assumptions that requires decisions to be made contingent on observed data is problematic, however, in preregisteredresearch, where researchers attempt to specify all important analysis decisions prior to collecting data. Limited methodological advice is currently available regarding how to deal with the prospect of distributional assumption violations in preregistered research. In this article, we examine several strategies that researchers could use in preregistrations to reduce the potential impact of distributional assumption violations. We suggest that pre-emptively selecting analysis methods that are as robust as possible to assumption violations, performing planned robustness analyses, and/or supplementing preregistered confirmatory analyses with exploratory checks of distributional assumptions may all be useful strategies. On the other hand, we suggest that prespecifying “decision trees” for selecting data analysis methods based on the distributional characteristics of the data may not be practical in most situations.",
author = "Williams, {Matt N.} and Albers, {Casper J.}",
year = "2019",
month = "9",
day = "12",
doi = "10.15626/MP.2018.1592",
language = "English",
volume = "3",
journal = "Meta-Psychology",
publisher = "University of Groningen, Linnaeus University",

}

RIS

TY - JOUR

T1 - Dealing with Distributional Assumptions in Preregistered Research

AU - Williams, Matt N.

AU - Albers, Casper J.

PY - 2019/9/12

Y1 - 2019/9/12

N2 - Virtually any inferential statistical analysis relies on distributional assumptions of some kind. The violation of distributional assumptions can result in consequences ranging from small changes to error rates through to substantially biased estimates and parameters fundamentally losing their intended interpretations. Conventionally, researchers have conducted assumption checks after collecting data, and then changed the primary analysis technique if violations of distributional assumptions are observed. An approach to dealing with distributional assumptions that requires decisions to be made contingent on observed data is problematic, however, in preregisteredresearch, where researchers attempt to specify all important analysis decisions prior to collecting data. Limited methodological advice is currently available regarding how to deal with the prospect of distributional assumption violations in preregistered research. In this article, we examine several strategies that researchers could use in preregistrations to reduce the potential impact of distributional assumption violations. We suggest that pre-emptively selecting analysis methods that are as robust as possible to assumption violations, performing planned robustness analyses, and/or supplementing preregistered confirmatory analyses with exploratory checks of distributional assumptions may all be useful strategies. On the other hand, we suggest that prespecifying “decision trees” for selecting data analysis methods based on the distributional characteristics of the data may not be practical in most situations.

AB - Virtually any inferential statistical analysis relies on distributional assumptions of some kind. The violation of distributional assumptions can result in consequences ranging from small changes to error rates through to substantially biased estimates and parameters fundamentally losing their intended interpretations. Conventionally, researchers have conducted assumption checks after collecting data, and then changed the primary analysis technique if violations of distributional assumptions are observed. An approach to dealing with distributional assumptions that requires decisions to be made contingent on observed data is problematic, however, in preregisteredresearch, where researchers attempt to specify all important analysis decisions prior to collecting data. Limited methodological advice is currently available regarding how to deal with the prospect of distributional assumption violations in preregistered research. In this article, we examine several strategies that researchers could use in preregistrations to reduce the potential impact of distributional assumption violations. We suggest that pre-emptively selecting analysis methods that are as robust as possible to assumption violations, performing planned robustness analyses, and/or supplementing preregistered confirmatory analyses with exploratory checks of distributional assumptions may all be useful strategies. On the other hand, we suggest that prespecifying “decision trees” for selecting data analysis methods based on the distributional characteristics of the data may not be practical in most situations.

U2 - 10.15626/MP.2018.1592

DO - 10.15626/MP.2018.1592

M3 - Article

VL - 3

JO - Meta-Psychology

JF - Meta-Psychology

ER -

ID: 96364674