Quis Custodiet Ipsos Custodes

Back in the 1960s, civilian research scientists at Gulf General Atomic (GGA) studied the feasibility of using compositional comparisons of questioned (crime scene) and known (suspect) bullets and bullet fragments to determine putative source. The scientists were on contract from the US Atomic Energy Commission (AEC, now the Department of Energy, DoE), which was seeking commercial applications for its considerable capital (nuclear reactors) and intellectual resources. The research actually ascended to high profile forensic significance during the JFK assassination investigation after one of the original researchers on the GGA team testified before the US Senate Subcommittee on Assassinations to the results of the JFK analyses.

The US Federal Bureau of Investigation (FBI) Laboratory seized the opportunity to expand its forensic services by adopting the methodology used by the GGA scientists for comparative bullet lead analysis (CBLA). The original GGA team used three analytes (antimony, copper and arsenic) in efforts to characterize bullet lead, and concluded – in a 1970 contract report to the AEC – that three analytes were insufficient to uniquely characterize lead sources (known as “pots” in lead smelting parlance).

It took almost two decades of forensic application using the three original analytes before the FBI Laboratory added silver, bismuth, tin, and eventually cadmium. The FBI Laboratory then confidently, but misguidedly, intuitively concluded that six (and eventually seven) analytes, coupled with the precision of sophisticated instrumental analysis of neutron activation analysis (NAA) and/or optical emission spectroscopy (OES) to parts per million (ppm) levels, surely must be sufficient to “fingerprint” a molten source of lead. For nearly forty years, FBI laboratory examiners testified that crime scene bullets and defendant (suspect) bullets “originated from the same source of lead produced on the same day.” Eventually examiners became emboldened to claim (and testify) source attribution as the “same box of bullets.”

The curiosity of forensic metallurgist William Tobin and his subsequent research team was piqued in the late 1990s because the underlying premises required for efficacy of forensic CBLA practice contradicted well-established metallurgical principles. Three underlying premises – or assumptions – are logically required to validate opinions offered in criminal trials by CBLA examiners: i) 50-60 mg samples used for forensic analyses are representative of their sources, ii) the “sources” were compositionally uniform (homogeneous), and iii) that each of the “sources” was unique.

In 1998, out of scientific interest and an attempt to reconcile the apparent contradiction to long-established metallurgical principles and phenomena, the forensic metallurgy team began to research the scientific and legal underpinnings of the practice, including secondary lead smelting and refining, the complete domain literature, and transcripts of testimonies from judicial proceedings. The team was shocked at what they found.

Although there was extensive peer-reviewed literature on the analytical chemistry methodology of neutron activation analysis (NAA) and inductively coupled plasma atomic (and later optical) emission spectroscopy (ICP/OES) for generating precise compositional data for bullet lead, there was not a single comprehensive or meaningful publication establishing or corroborating the validity of the three required underlying assumptions for the practice. Worse still, researchers in a subsequent study of retail product distribution in the USA found shocking geographic concentrations of same-composition bullets in local and regional areas, dealing a fatal blow to forensic and probative utility of the practice.

If everyone owned bullets of the same composition in a geographical locale of a shooting homicide, a finding of “analytically indistinguishable” bullets in the possession of a suspect compared to those recovered from a victim should never be presented to courts as incriminating evidence. In such a hypothetical scenario, there is no probative value of a claimed “match” between bullets recovered from a crime scene and those recovered from a suspect. Therefore, unless comprehensive and meaningful studies have been conducted and presented as to rarity or prevalence of analytically indistinguishable bullets in a regional population of bullets at the time of a shooting crime, such evidence should not be used to determine guilt or innocence in judicial proceedings.

Based primarily on flaws demonstrated by the metallurgical research, consequent publications, and court testimonies of forensic metallurgical and statistical research team members, a consequent study by the National Research Council (NRC) of the US National Academy of Sciences (NAS), publications by the NRC statisticians, and reporting by the media, the following objectionable aspects of the forensic practice were exposed:

• The forensic practice was developed almost exclusively by forensic examiners and analytical chemists. There was no evidence of vital cross-discipline input from metallurgists or statisticians;
  No comprehensive or meaningful study existed of the assumption that each pot (‘source’) of lead was unique in composition;
• No comprehensive or meaningful study existed of the assumption that each forensic sample was representative of its original source;
• No comprehensive or meaningful study existed that each source was compositionally homogeneous;
• No study existed as to whether or not bullet composition “matches” had any probative value for criminal trials, largely because there existed no study of retail product distribution in the USA;
• There had never been any true (mainstream) scientific oversight during development of the forensic practice;
• Papers promoting the practice had been “peer-reviewed” only by the very insular community of CBLA practitioners themselves and/or only by chemists relating to the analytical chemistry methodology, which was never under challenge.  The actual underlying assumption for the practice, that “same composition = same source” (the legal “task at hand”) was never tested.

As it turns out, the FBI practitioners had established themselves as “the world’s CBLA experts” based on nothing more than speculative and intuited assumptions that were eventually shown to be false, perpetuating probably the most sophisticated “junk science” that ever deluded jurisprudence in North America.