The Calibration Problem
The briefs generated by current top-tier AI tools are not bad. That is the problem.
When AI-generated legal writing is obviously bad — stilted, incoherent, legally naive — it gets caught. Supervising attorneys read it, find the problems, fix them or discard the output. The quality-control mechanisms work because the failures are salient.
We are now in a more dangerous regime. The best AI brief drafting outputs are fluent, legally literate, and structurally competent. They read like a competent associate wrote them. Which means they will receive less scrutiny than they deserve — and the errors they contain are precisely the kind that don't announce themselves.
What AI Briefs Get Wrong That Looks Right
I have reviewed approximately forty AI-assisted briefs in the past year. The errors I see most often are not citation hallucinations — those are, by now, the most scrutinized failure mode. The errors that concern me more are structural and argumentative.
First: AI briefs systematically underweight the counterarguments. A human brief writer who has lived with a case develops an intuition for where the other side will hit hardest. AI tools, trained on winning arguments, produce argument sections that are confident and comprehensive in forward direction and thin on the hardest counterarguments.
Second: AI briefs lack what I call doctrinal calibration — the sense of how important a line of cases actually is in a given circuit, given the actual panel composition and the institutional history of how that doctrine has been applied in practice. A technically accurate statement of the law can still be strategically wrong, and AI tools do not yet have the local knowledge to catch this.
Third: the AI does not know what it does not know about your case. The AI has read the documents you gave it. It has not felt the deposition where the opposing expert made a statement that undermines their theory. It has not been in the room when your client told you the thing that changed your entire understanding of the case. That context lives in the attorney's head, not in the documents.
A Proposal
I am not arguing that AI should not be used in brief writing. I use it myself, primarily for research synthesis, structure outlining, and drafting sections that are low-stakes and well-precedented.
What I am arguing is this: the quality bar for supervised review of AI brief drafts should be set higher than we are currently setting it, precisely because the outputs are now good enough to be misleading. Treat AI-generated argument sections the way you treat a second-year associate's first draft on an issue they have never worked on — not as a baseline to edit, but as a hypothesis to interrogate.
The courts that are currently sanctioning lawyers for hallucinated citations will, within two years, be adjudicating sanctions for strategically deficient AI briefs. The profession needs to get ahead of this.