In 2009, the National Academies of Science published a massive report on forensics. For many Americans, forensics is possibly the most familiar of all the sciences. It's the one we welcome into our living rooms every night, along with TV crime dramas and murder mysteries. But the report's conclusions might surprise you.
For one thing, it's hard to even generalize about the state of forensic science in the United States, because everything from standard practices to accreditation varies widely by sub-discipline, law-enforcement agency, and whether the law enforcement is happening at a local, state, or federal level. Worse, it's not at all clear that some of those sub-disciplines have a sound, scientific basis. For instance, DNA analysis tends to be pretty well-supported by evidence, while fingerprint analysis remains an art, more dependent on the person looking at the fingerprint than on hard laws of anatomy. Of course, the report also found that there simply hasn't been enough research done to determine how scientific most disciplines of forensic science are to begin with. Meanwhile, the Supreme Court has allowed trial judges to certify forensic techniques as reliable even though we don't know whether they they are or not — and those decisions have been made in a haphazard, inconsistent way from one judge to the next.
Given how much our legal system relies on this stuff, we should all be feeling more than a little uncomfortable right about now. The state of forensic science, combined with its importance, virtually guarantees that there are innocent people behind bars (or worse) and criminals on the loose.
Tonight, on PBS, NOVA will premier a documentary on the flaws of forensics and how they might be solved. I liked the show and I think it's definitely worth watching. That said, I think NOVA took an angle on this information that made the show less useful (and less important) than it might have been.
The NOVA documentary, "Forensics on Trial", is engaging and fun to watch. The basic structure of the show presents three real-world cases where forensics failed us, explains what went wrong, and introduces us to cutting-edge technologies that might be able to make crime scene investigations more reliable in the future.
For instance, a partial fingerprint on a plastic bag briefly made Brandon Mayfield — an American lawyer and convert to Islam — suspect number one in the 2004 Madrid train bombings. The fingerprint wasn't an exact match (although the FBI claimed at the time that it was) and, in fact, there were 20 people with fingerprints similar enough to the one found in Madrid that the FBI opened investigations on them. Mayfield was held without charge for more than two weeks, until Spanish authorities convinced the FBI that they had other suspects who made more sense as potential bombers, and whose fingerprints more closely matched the one that was found.
So what do we do when one fingerprint could reasonably belong to enough different people that the suspects could play a game of softball against each other while we figure it out?
NOVA suggests improving fingerprinting technology, using a high-tech film that can capture all the lines and ridges and whorls at a much-more-detailed level.
Would the people who do fingerprint analysis like to have more-detailed prints to work from? Probably. But that wouldn't solve the much-more basic problem highlighted in the National Academies report. Techniques like fingerprint analysis — which are based on humans or computers attempting to find and match patterns — are incredibly prone to bias. Not the kind of bias where the analyst has it in for the suspect personally, but the kind that happens when the analyst is thinking about what she's heard about the case from other people; when the analyst knows that other analysts have already called the prints a match; or when the analyst is under pressure to quickly find the dangerous terrorist. Here's a description of a study on this topic, as related in the National Academies report:
Recent research provided additional evidence of this sort of bias through an experiment in which experienced fingerprint examiners were asked to analyze fingerprints that, unknown to them, they had analyzed previously in their careers. For half the examinations, contextual biasing was introduced. For example, the instructions accompanying the latent prints included information such as the “suspect confessed to the crime” or the “suspect was in police custody at the time of the crime.” In 6 of the 24 examinations that included contextual manipulation, the examiners reached conclusions that were consistent with the biasing information and different from the results they had reached when examining the same prints in their daily work.
Without also reading the National Academies report, it's easy to come away from the NOVA documentary thinking that all we need to do is lay some improved technology on top of the existing forensic science system. In the notes I took while watching the screener, I wrote, "Is the problem here a lack of adequate technology, a mishandling of technology (bad methods/training/application), or a flawed foundation to begin with?"
NOVA doesn't really provide a clear answer. Meanwhile, the reply from the National Academies report appears to be, "Yes. All of that."
The NOVA documentary did a good job of alerting me to a problem. But it could have done a better job of explaining the true nature of the problem. More importantly, it left a big empty hole in terms of what comes next. If this isn't something that's just going to be solved by incremental improvements in technology (and it seems like that's the case), well then what? Has this report had any impact on the courts since 2009? Have the forensic sciences begun to improve standardization of methods, training, and accreditation? Did Congress appropriate any funds for research that would help us understand whether some of these sub-disciplines can be trusted at all?
And if not, why not?
Maybe this is a job for Frontline. But, from my perspective, leaving those issues less-than-well-addressed knocks "Forensics on Trial" down from a "must-see" to an "interesting way to spend an hour".
Hey, at least it's better than CSI.
• "Forensics on Trial" premiers tonight. Check your local listings.
• Read the full National Academies report on the failures of forensic science.
• Read a 2010 report on how these findings might affect the practice of law.
Image: Courtesy of ©Providence Pictures
