RAG without regret: a pragmatic checklist

Retrieval augments the model; it doesn’t replace judgment. Most RAG pain comes from treating the vector store as a magic bucket instead of a pipeline with versioning, access control, and measurable quality.

Ingestion is a product

Every document change should be traceable. When legal asks “what did the assistant see last Tuesday?” you need an answer. Design ingestion with job ids, source metadata, and rollback paths.

Chunking is a hypothesis

Test chunk sizes and overlap against real questions — not arbitrary defaults. Measure retrieval precision@k on a labeled set that reflects production traffic, not only offline toy queries.

• Separate paths for static vs frequently updated content
• Explicit handling for tables and lists
• Human review hooks for low-confidence retrieval

When in doubt, ship a smaller surface with higher trust. You can always widen the corpus; it’s painful to regain trust after a high-profile wrong answer.