Contract Review at Scale: Building Legal Document Intelligence

Legal document review is one of those tasks where AI confidence matters as much as AI capability. Get a clause wrong, and a client might sign away rights they meant to retain.

That constraint shaped every technical decision we made when building our Legal Document Intelligence system.

The Problem With Naive RAG

The first instinct most people have is to chunk the document, embed it, and run similarity search against a playbook. We tried this. The recall was acceptable. The precision wasn't.

The issue: legal meaning is highly context-dependent. A termination clause buried in section 14.3 might directly modify a payment term in section 4.1. Naive chunking severs those relationships.

What We Built

We use a hierarchical parsing approach. First, we extract the document's structural outline using Unstructured and LlamaParse. Then we build a semantic graph of clause relationships before we do any extraction.

The risk-flagging step compares extracted clauses against a configurable playbook stored in a vector database. But it's not just similarity — we run a secondary classification pass that rates deviation severity (low / medium / high) with a mandatory rationale.

Every AI decision is logged to an append-only audit table. Every clause extraction includes the source text, the confidence score, and the model's reasoning. That audit trail isn't optional — it's what makes the system legally defensible.

Results

The system reviewed a 120-page investment agreement in 18 minutes, flagging 11 clauses for attention. Our client's legal team validated all 11 as genuine issues. Zero false negatives on a blind test set of 40 contracts.

That's not magic. That's careful prompt engineering layered on top of a well-designed data pipeline.