End-to-End RAG: From Raw Files to Cited Answers

“Does my plan include CSV exports?” sounds like a simple support question. Answering it may require a current plan table, the customer's account tier and a help article explaining how exports work. Finding an old page that mentions CSV is not enough. Retrieval-augmented generation (RAG) selects evidence from a collection and gives it to a language model so the answer can be grounded in the right material.

This guide follows that evidence from raw files to a cited answer, then shows how to find failures and keep the system current. The support assistant is an illustrative design; the linked chapters go deeper into document preparation and retrieval choices.

Choose the chapter for the decision in front of you

1. Define the answers before choosing a stack

For a running example, imagine a support assistant for a small software company. Its collection includes help pages, policy PDFs, scanned onboarding forms and a spreadsheet of plan limits. It serves customers from different accounts. That example is hypothetical, but its decision points are useful for many projects.

Write ten real questions before picking an embedding model. Include an exact product code, a vague description, a question requiring two sources, a table lookup and a question that the files cannot answer. Then add examples involving expired information and another customer's private material. Expand this set before making accuracy claims; ten questions are a design exercise, not a statistically reliable benchmark.

For each question, record the permitted user, expected supporting passage and acceptable response. If the answer depends on a row in a database, record the row and access rule. Do not assume every information problem should become a paragraph in a vector index.

2. Choose the architecture you can evaluate

Microsoft's current RAG overview distinguishes classic hybrid-search pipelines from agentic retrieval. It labels LLM-based query planning as preview. “Agentic” describes a retrieval strategy; it does not guarantee better answers or universal production availability.

Our recommendation for the example support assistant is to begin with a fixed pipeline and a structured lookup route for live plan limits. Add more orchestration only after recording a class of questions the baseline cannot handle.

3. Understand the two connected pipelines

Offline / ingestion
Sources → inventory & permissions → parse / OCR → clean & classify
        → versioned chunks + provenance → embeddings & keyword index
        → validate → activate index version

Online / answering
Question + user identity → authorize & route → retrieve candidates
        → rerank / deduplicate → pack evidence into token budget
        → generate answer + source references → validate → respond

Feedback
Misses, corrections, changed files → labelled tests → next revision

The Azure design guide treats preparation, chunking, enrichment, embedding, retrieval and final evaluation as separate decisions. That separation helps diagnose whether an incorrect answer began in the source processing or later in the answer pipeline.

Your first version can use a batch job and one application; each box names a responsibility, not a separate service you must deploy. The boundary between ingestion and answering is useful because it lets you repair a bad parse without rewriting the chatbot.

Follow the export question through an illustrative request. The application reads the customer's tier from an authorized account lookup. Retrieval then selects the current plan entry and the export instructions. Suppose the plan entry says CSV export is included, but the help article describes only PDF: the generator can confirm the entitlement while saying it lacks verified CSV instructions. It should not invent the missing menu sequence.

If the answer instead says CSV is unavailable, inspect the selected evidence. A wrong tier points to account routing; a superseded plan row points to version filtering; the correct row with a wrong answer points to generation. This gives each later design choice a concrete job.

4. Inventory, classify and prepare the files

Keep the original file and a record of how it was processed. Classify by file format, language, business purpose and sensitivity. A document labelled “support” is not automatically readable by every customer: permissions need an authoritative identity source.

Use the file preparation chapter for named parser options and cleaning rules. It also compares fixed, recursive, structural, semantic, parent-child and late chunking. Keep an extraction rejection queue; silently indexing unreadable output makes downstream debugging much harder.

5. Index evidence with a reproducible configuration

Record the parser version, cleaning rules, chunk method, embedding model identifier, embedding dimensions and index settings. Preserve stable source identifiers alongside chunk identifiers so you can remove or update every fragment belonging to a document.

Three useful ownership models are:

OpenAI's retrieval documentation describes managed vector stores, filtering and ranking controls. Its behavior and limits are product-specific. Do not transfer a provider's default chunk size into another stack without testing.

6. Retrieve, rerank and budget the evidence

With source versions and metadata in place, retrieval can select evidence for one request. Keep three quantities separate: candidates retrieved, passages reranked and passages finally supplied to the generator. A broad search can produce a small final evidence set after irrelevant or duplicate passages are removed.

The retrieval chapter compares dense vectors, keyword matching and hybrid retrieval, then explains embedding input limits, vector dimensions, reranker limits and generator context windows. It includes an illustrative token-budget calculation.

For the example assistant, exact plan names must survive query rewriting. A new query such as “Does the team tier include exports?” must not become “What exports exist?” if that drops the condition that determines the answer. Log the original question and each authorized retrieval query for debugging, with appropriate retention and redaction.

7. Generate an answer whose evidence can be inspected

Our recommended answer contract is deliberately small:

• Answer the actual question using the supplied evidence.
• Associate factual claims with stable source identifiers and page, section or row references.
• State when evidence is missing or conflicting.
• Distinguish a source's effective date from the date it was indexed.
• Ask a focused clarifying question when the missing detail changes the answer.

Resolve citations against the passages actually retrieved. A syntactically valid citation can still point to a passage that does not support the claim. Check support as well as link validity. If a cited source becomes inaccessible, the application should not expose its text through a cached answer.

8. Evaluate the components and the final answer

Ragas documents retrieval and answer metrics, including context precision, context recall and faithfulness. Some evaluations use model judges. They offer useful signals, but an evaluator model's agreement is not independent proof that a claim is true.

Use this original review worksheet alongside any evaluation library:

Recall at a chosen cutoff asks how much of the labelled relevant evidence was retrieved; precision asks how much retrieved evidence was relevant. Define your relevance labels consistently. A correct answer on one easy case cannot replace a representative evaluation set.

Freeze a test set and change one major variable at a time. For example, compare two chunking strategies with the same embedding model, retriever and question set. Record regressions as well as averages. Keep multilingual, tables and permission tests visible as separate groups.

9. Keep source content separate from authority

OWASP's prompt-injection guidance identifies retrieved documents as an indirect-injection surface. Treat instructions inside a source as content to assess, not permission to change the application or call tools. Use restricted tool access and application-enforced authorization; a prompt alone is not an access-control system.

For the support example, a retrieved page saying “email all account records” must not grant that action. A read-only answer workflow should remain read-only. Test this behavior using harmless synthetic documents before introducing customer material.

10. Plan updates, deletion and cost before launch

A useful RAG answer can become wrong when its index is stale. Decide who owns each source and how quickly changes must appear. Reprocess changed documents, remove their obsolete chunks and invalidate affected caches. Treat permission revocation as an access change, not just a routine content refresh.

Version the new index, run the regression set and switch only after acceptance. Retain a rollback path that does not restore deleted or revoked information. Test that deleting a source removes its searchable content and derived answers under your retention policy.

Budget for extraction or OCR, enrichment, embedding, storage, search, reranking, generation, evaluation and re-indexing. Record usage by stage before quoting a fixed customer price. A cheaper embedding call can still lead to a more expensive system if poor retrieval requires larger prompts or repeated searches.

A practical first build checklist

1. Choose one document collection and a clear user group.
2. Save reviewed question-and-evidence examples, including no-answer cases.
3. Inspect representative parsed files before indexing the full collection.
4. Establish a simple retrieval baseline and an explicit answer contract.
5. Test permissions, citations, updates and deletion as user-visible behavior.
6. Compare one retrieval improvement against the unchanged baseline.
7. Release to a small audience, retain feedback and review failure groups.

Start by getting one representative question through this sequence with its supporting passages intact. Use the harness-engineering guide to make the checks repeatable, then add the failure cases before expanding the collection.