How Much Science Is Fake?

Abstract

‘Fake’ science is either fabricated - where quantitative elements are invented - or falsified - where results are dishonestly engineered from real data. Previous metascientific research estimates that ~2% of scientists are guilty of faking at least one paper. In opposition, this paper argues (1) these estimates are naive, and are irreparably contaminated with procedural and social desirability biases; (2) the proportion of scientists faking papers is a poor frame for understanding the problem of scientific mendacity, and less important than the proportion of fake scientific output; (3) there is limited evidence for the trustworthiness of this output, but it all suggests the proportion of fake papers is substantially higher; (4) work that could more accurately define this figure does not exist yet, due primarily to wilful ignorance on the parts of scientists, scientific institutions, and funding bodies; (5) even low estimates of the scale of this problem represent an existential threat to the scientific enterprise.

Keywords: Lorem, ipsum, dolor

James Heathers

Cipher Skin Ltd.

null

Cipher Skin Ltd. #400 1331 17th St Denver CO

No funding or COI to declare.

[email protected]null

* * *null

null

* * *null

How Much Science Is Fake?

Within science, ‘fabrication’ is inventing reported data, summary data, or statistical outcomes, ‘falsification’ is manipulating any part of the research process sufficiently to actively misrepresent real research. Together with plagiarism, they represent the acts which constitute serious scientific misconduct.

Both can be difficult to define. Falsification in particular begins somewhere on a continuum of questionable research practices which includes a constellation of ways data may be misrepresented: managing outliers, post-hoc subgroup analysis, outcome switching, p-hacking, etc. Fabrication is less slippery - either data is invented or it is not - but may also cross over with data imputation, typographical errors and clerical mistakes, and other forms of negligence or sloppiness (such as the piecemeal cleaning and reconstruction of biological images, which has both benign and nefarious components). The perpetual problem is that the author’s intent is difficult to strongly ascertain outside of severe cases - if a whole career’s worth of data is fabricated, an author can be easily judged to be mendacious; but in isolated examples, a pattern of behaviour cannot be established.

(Fanelli 2009)null

A straw poll of colleagues involved in metascientific work in and around the veracity of data produces a figure which is substantially higher. Common estimates are 1-5% of all papers contain fabricated data, and 2-10% contain falsified results. This is only congruent with the figure presented above if rare individual scientists are serial and not occasional fabricators.

So, which figure is correct?

“... we must recognize that 99.9999 percent of reports are

accurate and truthful, often in rapidly advancing frontiers where data are hard to collect.

There is no evidence that the small number of cases that have surfaced require a fundamental

change in procedures that have produced so much good science.” (Koshland 1987)null

(John et al. 2012)null

(Tijdink et al. 2014; Necker 2014)null

(George 2016)null

(George and Buyse 2015)null

(Citron and Ginsparg 2015)null

(Bik et al. 2016)null

(Krumpal 2014)null

https://publicationethics.org/files/paper-mills-cope-stm-research-report.pdfnull

Lorem ipsum dolor sit amet, consectetuer adipiscing elit, sed diam nonummy nibh euismod tincidunt ut laoreet dolore magna aliquam erat volutpat (Lorem, 20XX). Duis autem vel eum iriure dolor in hendrerit in vulputate velit esse molestie consequat, vel illum dolore eu feugiat nulla facilisis at vero eros et accumsan.

Method

Participantsnull

Lorem ipsum dolor sit amet, consectetuer adipiscing elit, sed diam nonummy nibh euismod tincidunt ut laoreet dolore magna aliquam erat volutpat. Duis autem vel eum iriure dolor in hendrerit in vulputate velit esse molestie consequat, vel illum dolore eu feugiat nulla facilisis at vero eros et accumsan.

Assessments and Measures

Ut blandit malesuada quam, ac varius tortor gravida eget. Vestibulum id ligula leo, ut accumsan mi. Sed tristique euismod convallis. Nulla facilisi. Etiam vestibulum est id orci interdum vitae porta enim blandit. Cras sit amet arcu dolor, at venenatis erat. Vestibulum accumsan placerat mauris. Morbi nec nibh nibh. Duis ultricies posuere nunc. Morbi at tellus quis magna vestibulum eleifend.

Heading 3 is the beginning of a paragraph ending with a period. Maecenas ullamcorper bibendum consequat. Pellentesque ultrices, eros eu tincidunt pretium, magna leo volutpat libero, non bibendum diam nunc eget urna. Vivamus eu tortor et dui aliquam vestibulum at vel augue. Vivamus elit dui, porttitor eget egestas at, rhoncus in justo. Curabitur tristique, elit ac venenatis volutpat, eros mauris iaculis diam, vitae rhoncus erat metus vitae eros.

First Heading 4 level in the section. Nulla congue egestas ante, id ultricies orci dignissim commodo. Fusce placerat, libero eu pharetra pulvinar, lorem dui pulvinar nisi, et semper orci orci vitae magna. Nullam sodales, felis id feugiat scelerisque, tortor nulla interdum mauris, ac porttitor odio dolor eget eros.

0Second Heading 4 level in the section. Duis sit amet ipsum pretium erat accumsan iaculis vitae eget risus. Donec ut dui in lorem volutpat fermentum bibendum pulvinar libero. Nunc imperdiet eros et mi posuere pellentesque. Donec tincidunt ipsum eget nisl ullamcorper eu placerat libero ullamcorper. Maecenas id luctus ligula. Cras condimentum eleifend nibh sit amet iaculis. Suspendisse placerat sollicitudin mi, vel ornare augue hendrerit ac. Nulla sed suscipit sapien. Cras pellentesque orci lectus, eu consequat enim.

0First Heading 5 level in the section. Nulla congue egestas ante, id ultricies orci dignissim commodo. Fusce placerat, libero eu pharetra pulvinar, lorem dui pulvinar nisi, et semper orci orci vitae magna. Nullam sodales, felis id feugiat scelerisque, tortor nulla interdum mauris, ac porttitor odio dolor eget eros.

Second Heading 5 level in the section.0Duis sit amet ipsum pretium erat accumsan iaculis vitae eget risus. Donec ut dui in lorem volutpat fermentum bibendum pulvinar libero. Nunc imperdiet eros et mi posuere pellentesque. Donec tincidunt ipsum eget nisl ullamcorper eu placerat libero ullamcorper. Maecenas id luctus ligula. Cras condimentum eleifend nibh sit amet iaculis. Suspendisse placerat sollicitudin mi, vel ornare augue hendrerit ac. Nulla sed suscipit sapien. Cras pellentesque orci lectus, eu consequat enim.

Results

Maecenas id luctus ligula. Cras condimentum eleifend nibh sit amet iaculis. Suspendisse placerat sollicitudin mi, vel ornare augue hendrerit ac. Nulla sed suscipit sapien. Cras pellentesque orci lectus, eu consequat enim.

Outcome 1

Lorem ipsum dolor sit amet, consectetuer adipiscing elit, sed diam nonummy nibh euismod tincidunt ut laoreet dolore magna aliquam erat volutpat.

Outcome 2

Lorem ipsum dolor sit amet, consectetuer adipiscing elit, sed diam nonummy nibh euismod tincidunt ut laoreet dolore magna aliquam erat volutpat.

Discussion

null

* * *null

References

Lastname, C. (2008). Title of the source without caps except Proper Nouns or: First word after colon. The Journal or Publication Italicized and Capped, Vol#(Issue#), Page numbers.
Lastname, O. (2010). Online journal using DOI or digital object identifier. Main Online Journal Name, Vol#(Issue#), 159-192. doi: 10.1000/182
Lastname, W. (2009). If there is no DOI use the URL of the main website referenced. Article Without DOI Reference, Vol#(Issue#), 166-212. Retrieved from0http://www.example.com