Toying with Athletes

The stated mission of the World Anti-Doping Agency (WADA) is “to lead a collaborative worldwide campaign for doping-free sport”.

This is (arguably) a laudable goal but a vociferous group of analytical scientists is far from satisfied with efforts to achieve it, questioning the effectiveness of anti-doping programs, the injustices created in their implementation, and the non-collaborative approach that WADA has taken. 

Here, four of the critics respond to a series of questions in what we hope will be the start of an open discussion on the need for anti-doping control, what its goals should be, and how these goals can best be accomplished. WADA has been offered the right to reply to criticisms and to contribute more widely to the debate, and we invite you to have your say below.

What are your major concerns over anti-doping approaches as they stand today?

It would be easy to imagine that the anti-doping system works perfectly well: cheating athletes are caught and punished, and sports are fairer as a result, right? This expert group does not think so.

It is statistically challenged
Klass Faber: “Current practice is not state-of-the-art; sub-optimal methods are being used for data analysis.  Instrumentation is not the problem. What is being done with (or to) the measurement results, that’s the problem. The criteria for a positive result, that is, the decision limits, are not statistically underpinned. There is not a single test for which the risk of false-positives and false-negatives is known. That worries me a lot.”

It’s a closed system
Peter Kootstra: “The whole system is closed, which is maintained under the auspices of independent accreditation bodies. The results I have seen and the discussions I have held with scientists from doping laboratories do not make me happy. There is a lot of confirmation bias (the tendency to favor information that confirms your beliefs), under the guise that ‘every athlete is guilty, we just can’t prove it’.”

There is no right to a fair trial
Bob Blackledge: “I am neither pro nor con as far as the need for monitoring of possible sports doping by competitive athletes goes. But individuals pulled over by the police for traffic violations, ticketed for illegal parking, or charged with driving under the influence of alcohol or drugs (all actual crimes) have far more rights to a fair trial before a jury of their peers than an athlete charged with doping.”

It is non-uniform 
Doewe de Boer: “Worldwide, results must fulfil certain minimum criteria. However, some laboratories perform much better than those minimum standards and, therefore, create non-uniformity. The effect is a kind of regional bias in anti-doping control.”

The Panel

Douwe de Boer

•  Analytical biochemist
• Independent anti-doping consultant and expert witness in legal sports cases
• Active in the field of anti-doping analysis since 1986
• PhD in Pharmacy (analysis of anabolic androgens in urine samples), from an International Olympic Committee (IOC)-accredited lab
• Technical and scientific director of the IOC- and WADA-accredited Portuguese anti-doping lab in Lisbon (1998-2004)

Klaas Faber 

• Chemometrician specializing in the use of optimal methods for data analysis and validation of final results
• PhD in chemometrics and post-doc with Bruce Kowalski, one of the founders of the field (1994-1996)
• Forensic and food research institutes advisor
• Private chemometrics consultant since 2002
• Independent anti-doping expert since 2006

Peter Kootstra 

• Analytical chemist specializing in laboratory accreditation (for example, ISO/IEC 17025)
• Broad knowledge of analytical methods used in anti-doping laboratories
• Independent consultant on laboratory quality
• Fifteen years experience in residue analyses (veterinary drugs); 25 years worldwide experience in laboratory accreditation
• Expert witness in doping case

Bob Blackledge 

• Analytical chemist specializing in forensic field
• Retired in 2006 after over 35 years in forensic science. Still active as a consultant
• 28 years experience in forensic laboratories of the US Army and Navy, working on analysis of suspected drug items
• Experienced in laboratory and individual certification
• Expert witness in over 200 court cases

What are your views on the effects/perceived effects of doping substances?

Unlike pharmaceuticals, the physiological effects of which are heavily researched and supported by rigorous trials, substances used in doping are often poorly understood. Even within this group of experts there is a wide range of opinions on the impact of doping substances. 

There is a lack of evidence
De Boer: “Evidence-based studies in respect to sport performance-enhancing effects and health effects are often missing. They may be under- or overestimated, but because of a lack of adequate studies, this can’t be confirmed or denied.”

There is a lot of propaganda
Faber: “Of course most compounds do have a performance-enhancing effect - but so does beet juice. The health risks often appear to be exaggerated. Think, for example, of the notorious epogen (EPO) or of growth hormone: Bernat López has strongly questioned the dangers of both (2, 3). Just recently, WADA issued a warning for GW1516, announced as the ‘new EPO’, yet there appears to be no evidence of its detrimental effects (4). Is emphasizing dangers that are not corroborated by scientific evidence an act of good faith, erring on the side of caution, or is it propaganda?”

The placebo effect is a factor
Kootstra: “There are only a few compounds that will have an effect, depending on the sport. Anabolic steroids will have some effect in sports where power is needed. Beta-blockers help reduce tremors when you need a steady hand. Most of the effects are psychological – merely a placebo.”

The ratchet effect is a concern
De Boer: “Unfortunately, anti-doping control does not solve the problem of doping, but rather shifts it to the abuse of sometimes more dangerous substances. Pharmacological substances still under investigation, not yet approved for clinical use, or even disapproved for clinical use, are becoming of interest.”

Do you understand what ends up on the prohibited list, and why?

Given that the latest doping drugs are designed to be undetectable, the historical (and very long) list of prohibited substance must be constantly updated and evaluated. But this group of experts believe that significant question marks hang over the choices made, and those that make them.

Legacy and politics play a big role
De Boer: “The list of prohibited substances and methods originated in the 1960s, without clear objective reasoning. Only afterwards was the reasoning objectified, with criteria written to extend the list. In the early days, the list was evaluated and extended by the International Olympic Committee (IOC) mainly based on scientific grounds, but WADA’s current list is also influenced by political arguments. Politicians have more influence on WADA than they ever had within the IOC, because they finance 50 percent of WADA’s budget. For that reason, a relatively low threshold for a cannabinoid was maintained on the prohibited list longer than could be justified scientifically. Only very recently, was its low threshold upgraded from 15 ng to 150 ng/ml of urine.” 

There are contradictions
Faber: “It is not always clear why a substance is banned (understatement!). Conversely, it is not clear at all why pain medication is allowed. How can you have clean sport if pain medication is allowed? It is certainly performance enhancing. What’s wrong with telling athletes to stay at home, regain their natural strength, and then compete? Also, pain medication leads to many addicts and sport casualties, notably in American football (5).”

Are the standards for anti-doping laboratories appropriate?

All anti-doping laboratories must be certified by both ISO and WADA. But the existence of standards does not always guarantee reproducibility, if standard operating procedures (SOPs) are not followed (6).

It is impossible to say
Kootstra: “Difficult to address since most required data are not available. Doping laboratories are inspected by employers of other doping laboratories…”

There are clear violations
Blackledge: “I am only quite familiar with one case, Floyd Landis and the 2006 Tour de France. Clearly, there were procedural violations of WADA rules, ISO requirements, and the lab’s SOP. A supervisor at a WADA-certified laboratory in Paris had checked an instrumental data page and then used a rubber stamp “ASSURANCE QUALITÉ - LNDD”. And yet lower on the page a technician had heavily marked over in black ink an original number (unknown) and changed it to ‘5’. What explanation is there for the fact that the lab found exogenous testosterone at a rate three times greater than other WADA-certified labs?” 

There are insufficient guarantees of quality
De Boer: “The rules are defined in the International Standards for Laboratories (ISL) and are based on all relevant ISO requirements and regulations. National accreditation bodies verify if local anti-doping laboratories are following the ISL, but cannot guarantee uniform quality. The analytical scientists themselves inside anti-doping laboratories play a very important role in assuring the required scientific level.”

Analytical science is not infallible, indeed margins of error must be tolerated in all scientific data. For athletes, false positive tests for doping are a huge concern (while the ‘victims’ of false negative results are unlikely to complain).  Our panel holds that athletes who challenge a test result are treated unrighteously.

Lack of access to documentation is unethical
Faber: “They do not, for a fact. Anti-doping regulation is such that the laboratory result is assumed to be reliable. One can only appeal on procedural grounds, that is, was the test properly carried out? Moreover, the defence does not receive sufficient documentation to challenge a test result; that has also been arranged in anti-doping rules. How convenient!”

‘Strict liability’ severely limits defense
De Boer: “As long as the principle of strict liability is applied to athletes (meaning that they are responsible regardless of whether they were aware or not), ‘fair’ has a very special meaning.  This is especially so if one measurement and, in theory, even one molecule, is enough to sanction an athlete. Anti-doping authorities reveal as little detail about their analytical strategies as is legally required because of potential abuse of those details by dishonest athletes. Unfortunately, the inability to access all relevant details also severely limits the ability of honest athletes to defend themselves. The anti-doping authorities hide themselves behind regulations, which they themselves set up and write.”

Duplicates are not handled appropriately 
Kootstra: “The B-sample analysis is a duplicate of the A-sample – same laboratory, same method, same instrumentation, different analyst. It almost seems like there could be no other result. Validation reports, QA control, instrument parameters – none of these can be inspected by the athlete or their representative. No discussion is possible.”

You would fare better to be on a criminal charge
Blackledge: “The table below compares the criminal justice system (of the US) with that of doping hearings:”

What should be done to improve anti-doping control?

A number of improvements are suggested by the commentators, including altering the thresholds at which substances are considered to be cheating. A consensus exists that more openness in the system, in the form of real engagement with anti-doping stakeholders, may be the key to a brighter future. 

First, clarify the WADA mission
Blackledge: “Before changes can occur at the laboratory and analyst level, there must be changes at WADA. This can begin with the mission statement: WADA and its accredited laboratories must only be concerned with monitoring those substances that can have a positive effect on athletic performance. For example, can it be clearly shown that marijuana use positively affects athletic performance? Also, there must be established cut-off levels for every banned substance. Below these levels, any indications of their presence should be considered neither capable of improving performance nor an indication of an athlete’s intention to cheat. For example, traces of clenbuterol were found in a urine sample from the cyclist, Alberto Contador. Whether these traces were or were not the result of eating meat obtained from Spain should not be the question. The question should be whether clenbuterol, at that level, could have enhanced his athletic performance.”

Open up a bit
Faber: “The inhabitants of doping control laboratories can do better by opening the window to the outside world. I would argue that the best science is not being delivered. More importantly, about 50 percent of the convictions are questionable because thresholds are not being applied. The use of party drugs outside competition is expressly allowed in anti-doping rules; it is the negligible trace found in-competition that leads to a conviction. Drink a few alcoholic beverages on Friday and played a match on Saturday? Not a problem, because there’s a threshold for alcohol. Smoked cannabis on Wednesday, and played a match the following weekend? Positive test and a sanction!”

Get athletes, analytical scientists involved
De Boer: “In theory, athletes are stakeholders of WADA and therefore have influence on the development of regulations. In practice, they are not well organized or well represented. Analytical scientists within anti-doping laboratories also limit themselves to regulations and claim that it is not their task or responsibility to sanction. However, if analytical progress is pushing identification limits lower and lower, the science will play an increasingly important role. Analytical scientists are also stakeholders in WADA and can exert their influence and provide proper advice – while doing so, they should keep the dilemma of fair chance in their minds. This is a social responsibility, rather than their analytical responsibility (which, I would argue, they are already fulfilling adequately).”

Analytical scientists need to say “no”
Faber: “The laboratory personnel should not assist in nonsense convictions. These labs have become production lines: ‘you ask, we deliver’. The exonerating part of the proof currently being omitted, on a global scale, is simply mind-boggling.”

What do you think?
Do the issues raised make you sceptical about anti-doping effort, or do you feel that WADA is on the right track? If you work in another field, how do the criteria compare to doping tests? If you work in a WADA-certified lab, what’s the inside view?

Please comment below.