RAG legal assistant for a law firm

Associates spent an average of 15 minutes per document search, several times a day, across a base of 200,000 documents (case law, contracts, internal memos, doctrine).

The firm is bound by professional secrecy: zero leakage possible, data hosted in France, complete traceability of queries.

Early experiments with ChatGPT had shown unacceptable hallucinations (invented case law) — a serious RAG was indispensable.

Retrieval quality had to exceed 90% on business queries — the firm refused any unsourced answer.

Multi-stage RAG: semantic chunking, embeddings (Cohere multilingual v3), pgvector store, hybrid search (BM25 + dense), cross-encoder reranker, mandatory citations.

Model: Mistral Large self-hosted on Scaleway VPS (total confidentiality, zero data retention). Fallback to a smaller model for simple queries (cost savings).

Continuous evaluation pipeline: 500 reference questions, metrics (recall@5, citation accuracy, hallucination rate, p95 latency). Langfuse dashboard to analyse every query.

Next.js interface with conversation thread, clickable citations, draft mode (no model call) for ultra-sensitive documents.

Full GDPR audit: DPIA, lawful basis, minimisation, right to erasure, anonymised query logging.

Average research time: 15 min → 1 min 45 (-88%). 100% associate adoption within 3 weeks.

Measured hallucination rate: 1.7% (vs ~30% for non-RAG ChatGPT on the same corpus).

Citation accuracy: 96.3% — every answer is sourced, verifiable, and a colleague can click through to the original document.

Marginal cost: €0.12 on average per query (vs €12 of associate time saved). ROI within 5 months.