Forensic Science Under Fire

“FBI admits flaws in hair analysis over decades” – Washington Post (1), “FBI admits pseudoscientific hair analysis used in hundreds of cases” – Smithsonian (2), “Thirty years in jail for a single hair: the FBI’s ‘mass disaster’ of false conviction” – The Guardian (3).

The headlines in popular media are damning to say the least. A report published on the Innocence Project (4) kick started the press frenzy – but what does it mean for the reputation of ‘analytical’ procedures? Jose Almirall, a professor in the Department of Chemistry and Biochemistry and Director of the International Forensic Research Institute at Florida International University, US, explores the issue.

How important is this case?

The Innocence Project report describes the errors in the reporting and testimony in around 90 percent of hair analysis and comparison cases reviewed. It is important to note that all these cases were examined prior to 2000 – the year that mitochondrial DNA analysis on hair evidence was included in the analysis protocol. It is also important to note that the FBI laboratory self reported the errors, even if this was not done as soon as it could have been. A large number of cases and individuals were impacted and these revelations will have a huge social and economic cost for years to come.

What exactly has gone wrong?

The errors were due to the overstatement of the significance of a match, when hairs were found to match in a hair comparison. Forensic scientists conduct the physical, chemical and sometimes biological analysis of evidence and then interpret the significance of the evidence as it relates to implicating an individual to a crime by associating the suspect to another person, to an event or to a location. The significance of the evidence should not be understated nor overstated, and forensic scientists balance the need to be precise in the language they use to describe the significance with the available support from the collective body of knowledge in the relevant scientific discipline. Until 2012, consensus standards focused on the interpretation of hair analysis and comparisons did not exist. It appears that some of the statements made by the examiners in their reports and in their testimony were not supported by the available scientific literature.

Could this happen again?

The National Academy of Sciences report of 2009 (5) shed a bright light on the need to address many of the weaknesses within the forensic sciences, and so today, it is much less likely that such a systematic misinterpretation of evidence can take place. But (and it’s a big but) there are still some disciplines for which there are no consensus guidelines or standards that address the significance of the evidence. The risk still remains to overstate (or understate) the value of the evidence for those laboratories not adhering to recommended standards or that conduct work in an area where there are no standards.

How about the role of analytical science and scientists?

Forensic scientists in the areas of biochemistry and molecular biology (DNA analysis) standardized both the analytical protocols and the interpretation of the significance of a matching DNA profile very early in the development of DNA analysis – and it was the FBI laboratory that took the lead in this effort. The relevant analytical community has developed scientifically rigorous guidelines and standards that are now widely used to promote precise language (including statistical descriptors) to communicate the significance of a DNA match for a particular case. DNA evidence is very reliable as a result and this high-quality evidence is also responsible for the exoneration of many wrongly convicted for other reasons (some of which do not have to do with science in the courtroom; for example, false eyewitness testimony, defense incompetence or prosecutorial misconduct).

What lessons should we learn?

The forensic science community should strive to develop interpretation standards that are supported by the scientific literature and, where the research to support interpretation guidelines does not yet exist, society should place a value on funding this research and encourage the community to develop consensus standards based on the research.

What do we do now?

One important (and ambitious) effort that was initiated this year is the creation of the Organization of Scientific Area Committees (OSAC), spearheaded by the National Institute of Standards and Technology (NIST). The aim of this effort is to bring together the top experts in diverse forensic disciplines to: identify existing standards in both the analytical sciences but also the interpretation of scientific evidence in order to raise awareness of the existing standards by creating a national registry develop a strategy to create new standards in those forensic disciplines that need them.

It will then be expected that operational laboratories be required to use standard methods of analysis when such methods are warranted so that appropriate analyses are executed and the correct interpretation of the evidence is properly communicated to the court. We are making progress, but there is still a lot of work to be done.