As Gelman and Loken (2014) note, “it can seem entirely appropriate to look at the data and construct reasonable rules for data exclusion, coding, and analysis that can lead to statistical significance” (p. Though some forms of data-dredging are lamentably common, it is important to note that often such problems arise from a lack of awareness rather than malfeasance. ‘false positives’) and is thus unreliable. Such an analysis can often generate statistically significant results in absence of a true effect (i.e. While many different choices might be defensible, a canonical case of p-hacking would involve trying out multiple different options and reporting the result which yields the lowest p-value (particularly when alternative choices generate values that do not yield a significant result). how to handle outliers, whether to combine groups, including/excluding covariates) which will produce a statistically significant p-value. In contrast, p-hacking occurs when an initial analysis produces results which are close to being statistically significant, then, in absence of a study protocol, researchers can make analytic choices (e.g. Ideally, these choices are guided by the principles of best practice and prespecified in a publicly available protocol. For example, in nearly any analysis of data there are several “researcher degrees of freedom”- i.e., choices that must be made in the process of analysis.

fishing, p-hacking), but each essentially involves probing the data in unplanned ways, finding and reporting an “attractive” result, without accurately conveying the course of analysis. Data-dredging bias is a general category which includes a number of misuses of statistical inference (e.g.

Data dredging/p-hacking