A police officer was accused of using AI to fabricate evidence in Derbyshire this month.1
Fabricated evidence is nothing new, but the scale of the problem will only get worse as the use of AI grows.
There are several distinct issues that could affect our legal system that arise from the use of AI.
The most obvious is false evidence being created and used knowingly and fraudulently. Such evidence could mislead investigators, lawyers, or juries. AI tools are dramatically reducing the cost of effort required to produce convincing falsehoods.
Following this is false evidence being used unknowingly. Perhaps an investigator that hasn’t been adequately trained in the use of AI-assisted tools, or a juror that asks ChatGPT for legal guidance without understanding that the AI model is unreliable in matters of fact.
As questions may arise about the legitimacy of evidence, so too will questions be asked of genuine evidence. Doubts will be cast on evidence that would exonerate the innocent or pin the guilty.
The ‘Liar’s Dividend’ is already a known phenomenon:2 when accused of something, the accused refutes the evidence, claiming it as “fake news”. Even where evidence is genuine, the existence of convincing AI-generated alternatives can undermine confidence in what is being presented.
The problem is exacerbated by the lack of robust and reliable AI detection tools. Many products claim to be able to identify AI-produced text, images, or audio. Their accuracy, however, is inconsistent at best. False positives will lead to genuine evidence being dismissed, and false negatives allow fake evidence to bypass scrutiny.
Where the standard of proof is high, such as in legal proceedings, AI cannot be relied upon to solve the problems caused by AI.
The UK judiciary has published guidance for the use of AI.3 Their commentary also highlights how AI models are largely trained on data from the internet, which has a significant historical and US bias in terms of legal content. Training data can also reflect historic biases and social attitudes, including biases against minority groups.
AI models are also black boxes. The training corpus and process behind many AI models is not publicly available. Any safeguards or instructions programmed into the model are unknown. In theory, a model-developer could could influence outputs in a way that favours its own interests.
Courts have adapted to new technologies before; photography and digital communications, for example. AI presents similar challenges, but at a greater scale and more significant speed. The rate at which AI tooling is evolving will put significant pressure on the abilities of juries, lawyers, and judges to distinguish authentic evidence from fabrication. Maintaining that skill may become one of the legal system’s biggest challenges in the years ahead.