Is Industrial Research the New Gold Standard?
Considered for so long to be the poor relation to academia, industry researchers may well represent the best of science.
Scientific research is sound. The scientific method, based on careful experimentation and observation to test hypotheses, is robust. Its record – the scientific literature – is accurate and dependable, buttressed by the expert evaluation provided by peer review. We can confidently build on (the vast majority of) what’s published to learn ever more about how the world around us works. Right?
Wrong. We are realizing that there are huge cracks in the reliability of science and the scientific record. Just a year ago, a study co-written by a researcher at Amgen (1) reported that 47 of 53 “landmark” publications in cancer research, papers from distinguished researchers published in prominent journals, could not be replicated. The previous year, researchers at another drug company reported that “In almost two-thirds of the projects, there were inconsistencies between published data and in-house data” (2).
The scientific enterprise must face up to these damning statistics. It’s not as though the problem is a one off. For example, a 1995 article entitled “Why Most Published Research Findings Are False” (3) attracted limited interest and generated no concrete counter-measures. In fact, The Reproducibility Initiative, described in last month’s issue by Elizabeth Iorns, might be the first attempt to address the issue.
This lack of replicability has many causes but incompetence and outright fraud are not prominent among them, despite the occasional, sensational case. That’s a mixed blessing. On the one hand, it’s comforting that the vast majority of researchers are both competent and honest; on the other, it means that there are deep issues to be addressed, among which the cherry-picking of data, poor controls, inadequate number of repeats, non-publishable negative results, and bad experimental habits loom large. The reward system in academia exacerbates all of these problems.
This leads me to wonder if more reliable research comes out of industry, particularly analytical labs. Industry scientists have less emphasis on publishing, less pressure to generate a ‘flashy’ result and, given the demands of providing products or services, they are more focused on reproducibility. On page 44 of this issue, Lloyd Snyder nails the case for publication by industry researchers. To his arguments, might we add that industry scientists can teach academia a thing or two about the design and reproducibility of research?
A comparison of the reproducibility would certainly be interesting.
Richard Gallagher
Editorial Director
- C. G. Begley & L. M. Ellis, Nature 483, 531, (2012).
- F. Prinz et al., Nature Reviews Drug Discovery 10, 712 (2011).
- J.P. Ioannidis, PLoS Med. Aug 2 (8):e124 (2005) ; Epub Aug 30 (2005).
Richard Gallagher is no stranger to quality, style or credibility. With Science, Nature and The Scientist all under his editorial belt, Richard teamed up with two good friends to form Texere Publishing, a new company with a great deal of know-how. Richard's also no stranger to contention: "You've constantly got to have an eye out for an editorial subject that will really stir the pot. We're aiming to be always relevant, but never predictable. About The Analytical Scientist, he says, Our vision is to capture commitment and success in analytical science in very particular way: by telling stories. Getting it right is an enormous, exciting challenge. Like so many professionals in the analytical sciences, we'll be thinking it, dreaming it and living it every day.
