globalen

Data mining

Data mining, also referred to as data dredging, data fishing, data fitting or data snooping, is the misuse of quantitative data analysis to report patterns that can be presented as statistically significant when, in fact, there is no real underlying phenomenon.

P-hacking is a form of data mining. This is usually done by performing large numbers of statistical tests of many different hypotheses on the data, and only reporting the results that can be considered significant in terms of p-value.

Data mining is typically found in medical studies, in fields such as epidemiology or psychology for example, where large datasets are used. But it also used in other scientific disciplines, in particular in finance. In recent years, many prominent academics have warned about the risk of data mining in financial research.