Rush at Your Peril

I have now been working in the field of cancer biomarker discovery for over 30 years. The investment over the last decade or so has been high – as has the excitement – but, in my opinion, the actual yield of new cancer biomarkers has been very poor.

There are many reasons for this, but I believe there is a major problem that needs to be tackled: false discovery. All too often, a group will believe that they have discovered a new biomarker, and they will hurry to publish in a prestigious journal. And all too often, it becomes apparent that the biomarker is not performing as expected, and that the results are unreliable and can’t be reproduced. In short, the discovery is a false one.

I have previously published papers highlighting the fact that a great number of studies describing novel biomarkers actually suffer from false discovery (1). Another issue is that a research group may indeed discover a biomarker showing a statistical difference between, for example, a non-cancer group and a cancer group, but the differences are not sufficient for the biomarker to be used in the clinic – the performance is too poor. Again, we end up with published papers that contain information on a biomarker that is not useful; it can never make it to the clinic. If you were to delve into the literature and examine 100 or even 1,000 published biomarkers, you would likely find that half of them are actually false discoveries, and that the other half are so poor that they don’t provide clinically useful information...

Some questions to ask yourself when setting up this type of study: were your samples properly collected, processed and stored? Have you selected your control groups appropriately? Are you looking at men, women, or both? What age are the controls versus the diseased group? What methods are you using to interpret your data? These are common areas for error to creep in – and the responsibility for making sure that every parameter is controlled and accounted for in any project lies with the investigator. It’s also important for all of us to remain vigilant against rushed and inadequate research. I personally publish a lot of papers essentially saying “this paper is false, and this one, and this one.”

Unfortunately, even though the information on how to avoid false discovery is out there, many researchers do not appear to be heeding it. But if you rush a study, even though you might get published in a big journal, your poor methods mean that whatever you publish will not enter patient care – it will fall into the cracks and make no real impact.

My recommendation? When you read that first report on the glorious discovery of a new biomarker, you should also wait and see if subsequent validation reports, especially by other groups, corroborate the results. With so many biomarkers proving too good to be true, I would advise a healthy level of skepticism!