British Police Crime-Prediction Tool Produced Unreliable Results
▼ Summary
– The Think Family Database, launched in 2016 by Bristol City Council and Avon and Somerset Police, stores sensitive data on nearly half a million people, including police reports, mental health records, and free school meal status.
– Machine-learning models were built on the database to assign risk scores to adults and children, aiming to create a “picture of threat, harm, and risk” in the region.
– Avon and Somerset Police created at least 23 predictive analytics models, including algorithms to predict burglary risk, failure to appear in court, and domestic abuse victimization.
– Public awareness was minimal; John Pegram, a local accountability group leader, only learned about the Offender Management App in 2023 and was later confirmed to be on it without knowing his risk score.
– At least two risk-scoring models were quietly abandoned after staff deemed them untrustworthy, and independent reviews warned of a lack of transparency and potential harm to public trust.
The Think Family Database quietly tracks nearly half a million residents in Bristol, England, yet for years, most of them had no idea it existed. Launched in 2016 by Bristol City Council and the regional Avon and Somerset Police, this repository hoards an array of sensitive data: police intelligence reports, housing status, mental health records, teenage pregnancies, parenting course enrollments, and free school meal eligibility. On top of this trove, officials built machine-learning models to assign risk scores to thousands of adults and children, aiming to create what they termed a “picture of threat, harm, and risk” across the region. At a 2022 event focused on combating child exploitation, one police data scientist described the method bluntly: “I essentially dump all that data in a big bucket and stir it with a data-science spatula, and we come out with a lovely risk score for everybody.”
This risk-scoring system was just one piece of Avon and Somerset Police’s broader predictive analytics program. The force developed at least 23 separate models, including algorithms to predict burglary risks, court no-shows, disappearances, and domestic abuse victimization. A senior officer even mentioned creating a “league table” of the area’s most dangerous criminals, likely referencing the Offender Management App, which was designed to hold data on roughly 300,000 people in the region.
Public awareness of these tools has been murky. John Pegram, who leads a local police accountability group in Bristol, says he only learned about the Offender Management App in 2023, years after its creation. When he did, he suspected he might be listed. “I think I knew I was on the app,” Pegram recalls. In early 2024, he filed a request to understand how police were using his data, but officials refused to disclose details. Months later, after hiring solicitors, police confirmed his inclusion but offered no further explanation. Like many across Bristol, the UK, and increasingly worldwide, Pegram remains in the dark about whether an algorithm scored him, what that score might be, or how it could shape his interactions with authorities.
Through a collaboration with the nonprofit newsroom Liberty Investigates, the Bristol Cable, and Lighthouse Reports, WIRED obtained hundreds of pages from public records requests, painting the most detailed picture yet of Avon and Somerset’s regional experiment with data collection and predictive analytics. (Liberty, Liberty Investigates’ parent organization, was briefly involved in a potential legal challenge and still supports Pegram’s litigation.)
The investigation reveals that at least two of these risk-scoring models were quietly abandoned after Bristol City Council staff concluded they could no longer trust the results. Previously unreported documents show government inspectors and independent reviewers flagged a striking lack of transparency around some program elements, warning that the systems could erode public trust. Police data disclosed to WIRED, including more than 36,000 model performance scores, appears in some cases to demonstrate “genuinely poor predictive performance,” according to an independent analyst who reviewed the data.
(Source: Wired)
