The Underappreciated Effects of Unreliability on Multiple Regression and Mediation

Abstract

There is an increasing trend for researchers in the social sciences to draw causal conclusions from correlational data. Even researchers who use relatively causally neutral language in describing their findings, imply causation by including diagrams with arrows. Moreover, they typically make recommendations for intervention or other applications in their discussion sections, that would make no sense without an implicit assumption that the findings really do indicate causal pathways. The present manuscript commences with the generous assumption that regression-based procedures extract causation out of correlational data, with an exploration of the surprising effects of unreliability on causal conclusions. After discussing the pros and cons of correcting for unreliability, the generous assumption is questioned too. The conclusion is that researchers should be more cautious in interpreting findings based on correlational research paradigms.