LexiQ — Document Extraction Engine
A fine-tuned layout-aware model that pulls structured fields out of contracts and invoices — and flags what it is unsure about.
96.8%
Field accuracy
~8s
Per document
12
Field types
The build
Every business that handles paperwork at volume has people retyping fields into a spreadsheet. LexiQ is the machine-learning path out of that: an OCR and layout pipeline, a model fine-tuned on annotated documents, and — the part that decides whether anyone actually trusts it — a per-field confidence score that routes uncertain extractions to a human instead of guessing silently.
This project was delivered under our AI & Model Training service. Models trained on your data, not a generic demo.
- Scope
- Dataset, training, evaluation, inference API, review UI
- Duration
- 7 weeks
- Stack
- PyTorch · LayoutLMv3 · Tesseract / PaddleOCR · FastAPI · PostgreSQL · React
What made this hard
Handing a contract to a general-purpose language model and asking for the fields gets you an answer that is right most of the time and confidently wrong the rest — with no way to tell which is which. For a finance or legal team, an extraction that is 90% right and silent about the other 10% is unusable, because someone still has to read every document to find the errors. The problem is not accuracy on its own. It is knowing where the accuracy ran out.
How we built it
- 1
Treat layout as signal
A total on an invoice is identified as much by where it sits as by what it says. We used a layout-aware architecture that takes token positions and the page image alongside the text, rather than flattening everything to a string and throwing the geometry away.
- 2
Build the dataset deliberately
We annotated a corpus spanning the document variants that matter — different vendors, scan quality, single and multi-page, tables that break across pages. The unglamorous labelling work is what set the ceiling on accuracy; the model choice moved it far less.
- 3
Score every field, not every document
The model emits a calibrated confidence per extracted field. Above threshold it is written straight through; below it, that single field is queued for review. A reviewer corrects one value instead of re-reading the whole contract.
- 4
Close the loop
Every human correction is stored as new labelled data. Periodic retraining uses exactly the cases the model got wrong, so the review queue shrinks over time instead of staying constant.
How the pieces fit
Upload → OCR with layout retention → LayoutLMv3 inference → structured JSON with a confidence per field → threshold split: high-confidence fields written through, low-confidence fields into a review queue → React review UI showing the field highlighted on the page image → corrections stored as training data for the next retrain cycle. Served behind a FastAPI endpoint, versioned so a model rollback is a config change.
Decisions we had to defend
Fine-tune rather than prompt a general model
A large general model needs no training data and gives you no calibrated confidence, no guarantee of a stable output schema and a per-document cost that scales badly. A fine-tuned layout model runs on our own hardware, returns consistent JSON, and tells us when it is unsure.
Human in the loop by design
We deliberately did not chase full automation. Routing the uncertain 3% to a person is what makes the confident 97% trustworthy — and it is the argument that gets a finance team to actually adopt the thing.
Evaluate on a held-out set, and say so
Accuracy quoted on training data is marketing. Ours is measured on documents the model never saw, reported per field type rather than as one flattering average, because a model that nails dates and fumbles totals is not 96% useful.
Where it ended up
96.8% field-level accuracy on a held-out set across twelve field types, at roughly eight seconds per document end to end. More useful than the headline number: the confidence routing means the error rate on auto-accepted fields is far lower, because the doubtful ones never get auto-accepted. The deliverable includes the training notebooks and the evaluation breakdown, so the numbers can be checked rather than taken on faith.