Science in the Courts!

The Human Factor

William Thompson, Committee Chair of the OSAC Human Factors Committee, explains why the ‘human element’ is an important factor in forensic science testimony.

I study the underlying psychology or psychological dynamics of human decision making. For a long time, I’ve been interested in how experts – and especially forensic science experts – evaluate evidence and reach conclusions.

The human factor comes into forensics at two levels. In part, it’s the psychology of the expert; how they make decisions (and sometimes make mistakes). The second part is the psychology of communicating scientific findings – particularly, to a jury or to lawyers who may not have any expertise – in a way that allows them to understand and draw appropriate conclusions.

Setting standards

I work with the OSAC (Organization of Scientific Area Committees for Forensic Science), an organization designed to help foster and create standards and practice guidelines in forensic science. After a 2009 report from the US National Academy of Sciences identified the human element in forensics as needing additional study, I was invited as a lawyer and a psychologist to take part in the OSAC’s human factors sub-committee.

The Human Factors Committee is not empowered to create standards on its own, but we are expected to provide advice to other units within OSAC, comprising forensic scientists, industry professionals, and researchers. We also provide advice directly to the Forensic Science Standards Board, which ultimately reviews and approves those standards.

The three areas that we’re most interested in are: i) how to minimize bias and interpretation – particularly contextual bias and the possibility that scientific conclusions are influenced by irrelevant investigative facts; ii) effective communication; and iii) assessment and testing of individual examiner performance.

We have to learn a great deal about a diverse range of forensic science disciplines; on the one hand, we’re dealing with DNA analysis and analytical chemistry, and on the other, we’re looking at pattern matching tasks, debris analysis, and forensic pathology. However, they all involve humans, and so they are all susceptible to human error.

Justice is blind

The Human Factors Committee has been identifying possible sources of bias, and how we minimize those effects. We need to make sure that the decisions made by experts are as good as they can be. Surprisingly, despite forensic science having been around in one form or another for over 100 years, there has been little discussion of the proper basis for a forensic scientist’s opinion. When it comes to a particular analytical task, there’s a surprising amount of debate over what is relevant. To avoid bias, we are constantly pressing forensic scientists to identify what kinds of evidence are relevant (and irrelevant) to particular tasks. Is it just the physical evidence in front of them, or is it necessary to take context into account? Should the fingerprint examiner, for example, be drawing scientific conclusions solely from the fingerprints, or should they be influenced by other investigative facts in the case?

Forensic scientist Michael Taylor and his colleagues did a series of studies (1) looking at whether experts who examine blood stain patterns are influenced by something other than the physical characteristics of the patterns themselves.  They created spatter patterns for the study so that their actual origin would be known, and experts were asked to see how accurately they could characterize these patterns. They were given some contextual information in the form of police reports – and this information was found to influence the conclusion. If the police report mentioned somebody had heard a gunshot, the pattern was more likely to be interpreted as high velocity splatter, whereas if the report mentioned that someone had been heard coughing, the same pattern was more likely to be interpreted as expiration. Studies have documented similar effects in latent print analysis, forensic anthropology, document examination, crime scene analysis, and even DNA analysis.  The fact that the analytical conclusions of the forensic scientist can be unconsciously influenced by contextual factors that seem irrelevant to the scientific task is an important concern.

Based on those findings, we have considered different ways of sequencing the workflow to ‘blind’ forensic investigators to unnecessary information; for example, the blood splatter pattern expert doesn’t necessarily need to read the police report until after they have interpreted the pattern. Ultimately, it may be impossible to eliminate all bias from our justice system, given the ubiquity and variety of bias we’re confronted with. (There are even studies showing that judges become harsher in their sentencing right before lunch!) Nevertheless, we want to help forensic scientists gather and present the best possible evidence. 

Present (and correct)

The big debate in forensic science right now, and a good example of the issues the OSAC committee tackles, is how best to present source conclusions. Source conclusions involve an examiner comparing two items and trying to reach a conclusion about whether the items come from the same or a different source. Were these two shell casings fired by the same gun? Were these two fingerprints from the same person?

How should a forensic scientist state their conclusions to a lay jury if they wish them to be understood and interpreted appropriately? In days past, many forensic experts would simply make an identification; they’d say, “I’ve examined these two prints and I’ve determined that they came from the same shoe.” There’s been a lot of criticism of that approach, because it implies that decisions can be reached with certainty, when we know that they can’t.

With that in mind, how can you convey those findings in a way that makes the uncertainty more transparent or understandable to the lay audience? Do you give the conclusion and include data – for example, true and false positive rates? Or do you explain that the conclusion is merely ‘likely’ and try to compute some probability? There is currently a lot of debate about this.

I think we need to avoid giving categorical conclusions – it seems wrong to me to state that things are certain when they are not. To state: “I think there’s a 99 percent chance these two items come from the same source,” is also problematic, because it’s difficult to reach conclusions of that type based on scientific evidence alone.

Research has found that people tend to misunderstand the probabilities they are given. They often mistake ‘random match probability’ as ‘source probability’; in other words, an expert might state the probability of a match coming from a random person – and laypeople often think of it as probability it didn’t come from the accused. In some instances, that can be a serious error.

For a long time, it was thought that people would give very little weight to likelihood ratios because they wouldn’t understand them, but we’ve debunked that through our studies (2)(3).

The question of how best to communicate findings is a difficult one and I’m not sure that I know the right answer – the more research I do, the less certain I am! What is certain is that we need testing and validation of different approaches.

Before the court

A lot of human factor issues also become legal issues – so within OSAC, we have a lot of interaction with the Legal Resource Committee. Ascertaining the proper basis on which a forensic scientist should form his or her opinion, or what we want them to try to take into account (or not) are ultimately legal concerns.

We have had cases in the United States where forensic pathologists have taken everything into account – not just forensic findings, but all the background information, the police reports and so on. They have testified on that basis, and the cases have been overturned on appeal by courts – they felt that the experts have encroached on the jury’s role by considering issues beyond the scientific. So, the legal questions are very much intertwined with psychological and scientific issues.

I am a lawyer and I used to practice law, so I have been involved in cases both as a lawyer and as an expert witness. I have to say, I like being a lawyer more than being an expert. It’s more fun to ask the questions than to have to answer them! To scientists testifying, I would say the following: it can be quite grueling, but it can be a very positive experience. And it’s really beneficial for the legal system and for society in general if good scientists are willing to come to court and explain what they know. One of the big problems in criminal justice is the difficulty of finding experts who are willing to give their opinion, so I would encourage anybody who is interested to give it a try. Once you are on the stand, be very clear, speak to the lawyers in advance, be very firm about what you can and cannot say… And stick to your guns. Don’t allow lawyers to talk you into anything.

It’s a complex field and my OSAC role is a challenging one, but it continues to fascinate me. It’s absolutely necessary that we have lawyers who are scientifically literate and capable of understanding the nuances of forensic science evidence.  I have enjoyed working as a lawyer, but I feel at this point in my life and career my time is better devoted to working with forensic scientists to help them get it right, rather than trying to challenge what they’re doing in court.

William C Thompson is Professor Emeritus of Criminology, Law, and Society and Psychology and Social Behavior and Law at the University of California Irvine, Committee Chair of the Human Factors Committee of OSAC, USA.

Stand and Deliver

Kenyon Evans-Nguyen discusses the unique and challenging experience of presenting your science in court

I’ve been on the stand many times, back when I was a practitioner – and it can be terrifying! It’s a unique experience. Some practitioners are almost like survivors of trauma; they’ve been up on the stand and had to endure these stressful experiences where the object of the opposing attorney is to tear down everything that they’re saying – and not necessarily based on science.

When your paper gets peer reviewed, you may face harsh criticism, but their arguments have to be scientific; their goal is not to invalidate your science, but to make sure it is rigorous. When the FDA examines your new drug, they go through everything with a fine-tooth comb, but their goal is to make sure the science is good. The goal of a defense attorney can be to cast doubt; if they get you to speak in terms that are inaccessible to the jury (referred to as ‘muddying the water’), then they’ve won. If the science, or your defense of it,  confuses the jury, your science goes down the drain.

I train students on how to take the stand, and I warn them beforehand that it’s going to be weird and uncomfortable, and it’s probably going to make them angry. It’s part of the interview process in forensics – they want to know how good your science is, but then they will intentionally mess with you. They may give you a tour, and then at the end of the interview a couple of hours later, they’ll test you on obscure details. They’ll ask you bizarre questions you wouldn’t be asked in a typical interview and stare at you to see how you respond, deliberately throwing you off. They have to establish that you will be able to think on your feet and keep yourself together when things get adversarial on the stand.

It’s not a fun class to teach, because you have to be harsh to students you care about – but it’s for their own good in the end. Most will ‘crack’ a few times along the way, but they understand why it’s important and they usually get there in the end. They need to understand that the opposing counsel will adapt their approach to the personality of the person they are questioning: if you’re timid, they try to bully you into conceding; if you’re brash or bold, they try to make you angry so you blow your top and say things you didn’t mean or want to say.

As a scientist, testifying in court means teaching science to the jury – while being subjected to the world’s worst heckler. It’s intense. But if you can keep your cool and present the facts, you can really make a difference to a case, and that is what makes it truly worthwhile.

Where There’s a Will…

While providing scientific testimony for a high profile case, Harold McNair had several grueling encounters on the stand…

I’ve been involved in several cases that have been important, but a major one was the will of Howard Hughes.

I entered the scene when I was teaching an undergrad class at Virginia Tech. A man walked in, packing a weapon, with a bag handcuffed to his wrist. He said, “Are you Professor McNair? Can we talk alone?” If somebody has a gun, I tend to do as they say...

“I’m a federal marshal and I have the Howard Hughes will right here,” he said. “It’s worth $400 million. Do you have a secure place to keep this?”

I said, “Yes, but why would I want to?”

“I have a request from the FBI on behalf of the Hughes family. They would like you to do an analysis of the ink and see if it could be the ink that Hughes regularly used.”

It turned out to be a very interesting case. An attendant in a gas station close to Las Vegas claims that early one morning he saw Howard Hughes in pajamas wandering around, looking for things related to nuclear bombs. (He was paranoid about those things.) The gas station attendant, Melvin Earl Dummar, claimed Hughes befriended him and that Dummar took him back to his hotel.

When Howard Hughes died, he left behind maybe half a million dollars – probably $4b today. People thought he would leave most of the money to Stanford University and the Hughes Medical Center – maybe a few friends and family. However, this handwritten will (dated before Hughes’ death) left $156m to the Mormon church, and $156m to Dummar. It became known as the Mormon will.

Of course, the whole story is bogus. The will had been forged at a much later date. Howard Hughes would never have been wandering around in his pajamas in the desert. He had a chute in his hotel room, and he would climb into a special temperature-controlled “coffin” to go down to his car in the basement.

So it was my job to analyze the ink and clear up any doubt. Howard Hughes typically bought blue Paper Mate pens, maybe 1,000 for the year. Paper Mate originally bought the ink from DuPont – but just after the will had been written, Paper Mate decided to make its own. Using HPLC, I discovered the Paper Mate ink had two pigments plus two impurities – impurities that did not appear until about 1971 – and that this was the ink in which the will was written. The ink on the document said 1968 – and so it could not be valid. The judge and jury didn’t accept HPLC because it was too novel a method (I was one of the pioneers that introduced it), so they told me to analyze again by thin layer chromatography. I returned to Virginia Tech, performed the analysis, and the results were the same. The Mormon will had been written in 1971 or later.

When I went to testify, the plaintiff was extremely aggressive.

He said, “You wear glasses. You have a physical handicap – you don’t see very well – and you didn’t see very well on this case. You made a big mistake, didn’t you?”

He continued, “Tell the audience how much money you make consulting and fabricating these lies.”

I said, “I don’t receive anything. I had to get permission from the governor of Virginia even to come here. The FBI asked me to come and testify.”

“Why would you do that?” he scoffed.

I said, “Because I am, and have been the FBI’s expert on inks and dyes for many years.” We won 
the case.

It was the first time I’d been to trial, the first time I’d had someone eat me alive verbally – I really enjoyed the challenge of it!

When the federal marshal escorted me back to the airport (there’d been a threat along the lines of “shooting the guy from Holland because he was questioning the validity of the will”), I was walking across the street and the plaintiff lawyer called out to me, “Sorry, Dr McNair,  just trying to win the case – no hard feelings!”