Data dredging - Revision history

Karl Jones at 19:49, 14 November 2016

2016-11-14T19:49:05Z

Karl Jones: /* See also */

2016-11-14T19:47:51Z

‎See also

Karl Jones: /* See also */

2016-11-14T19:47:29Z

‎See also

Karl Jones: /* See also */

2016-11-14T19:47:16Z

‎See also

Karl Jones: /* External links */

2016-11-14T19:45:40Z

‎External links

Karl Jones: Created page with "'''Data dredging''' (also '''data fishing''', '''data snooping''', and p-hacking) is the use of data mining to uncover patterns in data that can be presented as statistica..."

2016-11-14T19:45:07Z

Created page with "'''Data dredging''' (also '''data fishing''', '''data snooping''', and p-hacking) is the use of data mining to uncover patterns in data that can be presented as statistica..."

New page

'''Data dredging''' (also '''data fishing''', '''data snooping''', and p-hacking) is the use of [[data mining]] to uncover patterns in data that can be presented as statistically significant, without first devising a specific hypothesis as to the underlying causality.

== Description ==

The process of data mining involves automatically testing huge numbers of hypotheses about a single data set by exhaustively searching for combinations of variables that might show a correlation. Conventional tests of statistical significance are based on the probability that an observation arose by chance, and necessarily accept some risk of mistaken test results, called the significance. When large numbers of tests are performed, some produce false results, hence 5% of randomly chosen hypotheses turn out to be significant at the 5% level, 1% turn out to be significant at the 1% significance level, and so on, by chance alone. When enough hypotheses are tested, it is virtually certain that some falsely appear statistically significant, since almost every data set with any degree of randomness is likely to contain some spurious correlations. If they are not cautious, researchers using data mining techniques can be easily misled by these results.

The multiple comparisons hazard is common in data dredging. Moreover, subgroups are sometimes explored without alerting the reader to the number of questions at issue, which can lead to misinformed conclusions.

== See also ==

* [[Data mining]]

== External links ==

* [https://en.wikipedia.org/wiki/Data_dredging Data dredging] @ Wikipedia

@@ Line 1: / Line 1: @@
-'''Data dredging''' (also '''data fishing''', '''data snooping''', and p-hacking) is the use of [[data mining]] to uncover patterns in data that can be presented as statistically significant, without first devising a specific hypothesis as to the underlying causality.
+'''Data dredging''' (also '''data fishing''', '''data snooping''', and p-hacking) is the use of [[data mining]] to uncover patterns in data that can be presented as [[Statistical significance|statistically significant]], without first devising a specific hypothesis as to the underlying causality.
 == Description ==
@@ Line 16: / Line 16: @@
 * [[Look-elsewhere effect]]
 * [[Misuse of statistics]]
 * [[Overfitting]]
 * [[Pareidolia]]

@@ Line 20: / Line 20: @@
 * [[Post hoc analysis]]
 * [[Predictive analytics]]
 == External links ==

@@ Line 14: / Line 14: @@
 * [[Data mining]]
 * [[Lincoln–Kennedy coincidences urban legend]]
-* [[Look-elsewhere effect]]* [[Misuse of statistics]]
+* [[Look-elsewhere effect]]
 * [[Overfitting]]
 * [[Pareidolia]]

← Older revision		Revision as of 19:45, 14 November 2016
Line 14:		Line 14:

	* [https://en.wikipedia.org/wiki/Data_dredging Data dredging] @ Wikipedia		* [https://en.wikipedia.org/wiki/Data_dredging Data dredging] @ Wikipedia
		+
		+	[[Category:Computer science]]
		+	[[Category:Data]]
		+	[[Category:Information]]
		+	[[Category:Metadata]]
		+	[[Category:Privacy]]