Some day, you may be the defendant in a criminal trial that turns on whether the software in a forensic device reached a reliable conclusion about a DNA test or other piece of evidence. Wouldn't you like to have your own experts check the source code on that device?
Today's forensic evidence is generated by black boxes whose owners — police labs — are often contractually prohibited from reverse-engineering them or subjecting them to other tests.
It's a modern ducking-stool. The Enlightenment taught us that even the most rigorous of scientists can deceive themselves about the outcomes of experiments, and so their conclusions must be subjected to adversarial peer-review (where your friends point out your mistakes and your enemies pillory you for making them). Dieselgate taught us that software can produce inanimate object that lie when they're subjected to regulatory scrutiny.
Without transparency, we put every accused criminal at the mercy of a justice system that runs on robotic witchfinders' accusations instead of evidence.
Snip from “Convicted by Code,” by Rebecca Wexler at Slate:
Short-circuiting defendants’ ability to cross-examine forensic evidence is not only unjust—it paves the way for bad science. Experts have described cross-examination as “the greatest legal engine ever invented for the discovery of truth.” But recent revelations exposed an epidemic of bad science undermining criminal justice. Studies have disputed the scientific validity of pattern matching in bite marks, arson, hair and fiber, shaken baby syndrome diagnoses, ballistics, dog-scent lineups, blood spatter evidence, and fingerprint matching. Massachusetts is struggling to handle the fallout from a crime laboratory technician’s forgery of results that tainted evidence in tens of thousands of criminal cases. And the Innocence Project reports that bad forensic science contributed to the wrongful convictions of 47 percent of exonerees. The National Academy of Sciences has blamed the crisis in part on a lack of peer review in forensic disciplines.
Nor is software immune. Coding errors have been found to alter DNA likelihood ratios by a factor of 10, causing prosecutors in Australia to replace 24 expert witness statements in criminal cases. When defense experts identified a bug in breathalyzer software, the Minnesota Supreme Court barred the affected test from evidence in all future trials. Three of the state’s highest justices argued to admit evidence of additional alleged code defects so that defendants could challenge the credibility of future tests.
Cross-examination can help to protect against error—and even fraud—in forensic science and tech. But for that “legal engine” to work, defendants need to know the bases of state claims. Indeed, when federal district Judge Jed S. Rakoff of Manhattan resigned in protest from President Obama’s commission on forensic sciences, he warned that if defendants lack access to information for cross-examination, forensic testimony is “nothing more than trial by ambush.”
Convicted by Code [Rebecca Wexler/Slate]
