Reliability & accuracy
What are AI hallucinations in legal work, and how do you avoid them?
An AI hallucination is a confident, fluent output that is simply wrong — an invented case, a misquoted section, or a citation to authority that does not exist. In legal work the danger is that hallucinations look exactly like good work. You avoid them by using AI that cites a verifiable source for every claim, is built to decline rather than guess, and by checking each citation against the primary source before you rely on it.
Why hallucinations happen
Language models predict plausible text, not verified truth. When a model has no grounding source for a question, it fills the gap with specifics that read convincingly but were never real — and legal citations, with their predictable format, are especially easy to fabricate.
The problem is not that the model is careless; it is that producing a confident answer is exactly what it is built to do, even when it should say it does not know.
Why they are dangerous in law
A fabricated case looks identical to a real one on the page. Lawyers in several jurisdictions have filed AI-invented citations in real proceedings and been sanctioned for it. The output carried all the authority of good research and none of the substance.
In law the cost lands twice: on the client whose matter relied on it, and on the practitioner whose name is on the document.
How to avoid them
Use legal AI that returns a source on every answer, prefer tools built to refuse rather than guess when no authority exists, and verify every citation against the primary source before relying on it.
Treat any un-sourced AI prose as a starting draft, never as authority. The discipline is simple: no source, no reliance.
Frequently asked questions
Have lawyers really filed AI-invented cases in court?
Yes — lawyers in several jurisdictions have been caught filing citations to cases that an AI fabricated. In each instance the failure was not using AI, but filing its output without verifying the authorities.
Can hallucinations be eliminated completely?
Not entirely, but the risk drops sharply when the tool grounds every answer in a citation, is built to decline rather than guess, and a human verifies the source. Grounding plus verification is the practical answer.
How do I tell if an AI answer is hallucinated?
Open the cited source. If there is no source, or the source does not say what the AI claims, treat the answer as unreliable until you can verify it against primary authority.
See how Quillio handles this in practice
AI built for Australian and New Zealand law — a citation on every answer, client content stored in Australia, and a free trial so you can test it on your own files.