BAE developed the London Counter Fraud Hub, which uses machine learning systems to detect benefit fraud; after trials in the boroughs of Camden, Ealing, Croydon and Islington, the system has been approved for regular use, despite an admitted 20% failure rate.
The councils estimate that BAE’s algorithm will be instrumental in terminating 40,000 allegedly fraudulent benefits claims in its first year of deployment, meaning that 8,000 households will be incorrectly accused of fraud and will have to undergo a lengthy, bureaucratic process to prevent loss of benefits, which could cost them their homes and the money they rely on for food and other necessities. These 8,000 households are already strained and cannot afford professional advice or help while they fight for their rightful benefits.
Ealing Council defended the move, saying that the system will not terminate accounts, rather it will single out accounts for human review, which could lead to termination. Longstanding experience with automated fraud-detection systems has shown that the judgments of these systems impart a veneer of empirical legitimacy that turns the presumption of innocence on its head.
The Counter Fraud Hub is the latest in a series of brutal, automated benefits administration system, following on from the catastrophic Universal Credit scheme, which unjustly destroyed thousands of peoples’ lives, costing them shelter, food, education and employment.
Joanna Redden, co-director of Cardiff University’s Data Justice Lab said: “When automating a system like this, when you know some people are going to be wrongly identified as committing fraud, and that many will have few means or resources there are serious concerns that need to be addressed.
“I would urge the councils who are considering automating this process not to do so, particularly given what we know about how this kind of system can go wrong.”
Thousands face incorrect benefit cuts from automated fraud detector [Roland Manthorpe/Sky]
(via ./)