Lesson 5.3: Grounding, citations and long documents

Learn to tie answers to supplied text with checkable citations, test what a prompt does when the answer is missing, and handle very long documents.

By the end of this lesson you will be able to write instructions that tie every claim in an answer to the text you supplied, design a citation format that a person or a program can check, test how a prompt behaves when the answer is not in the documents, split a long document into overlapping chunks and work out how many you need, and measure how well grounded a set of answers really is, with an honest margin of error. You should have read Lessons 2.2, 2.4 and 5.1, since this lesson takes grounding, verification and testing further. It concerns the prompting side only: how to retrieve the right passages automatically is covered in the AI Engineering track, and here we assume the passages are already in front of the model. No programming is required.

Giving a model your documents and telling it to answer from them is the most effective step you can take against invented facts. It is not a guarantee. A model can quietly mix its own knowledge with yours, miss a relevant passage in a long document, cite a source that does not say what it is credited with, or answer a question the documents do not address. Systems that answer questions about policies, contracts, regulations and reports live or die on handling these four problems, and each can be improved with careful prompts and measured with simple tests.

What grounded means

An answer is grounded when every claim in it is supported by the text you supplied. It is not enough that the claims are true in the world. A statement can be correct and ungrounded, because the model recalled it from training and the documents say nothing about it. For a task such as answering from a company policy, that is still a failure: the reader believes the policy says so, and next quarter the policy changes and the memory does not.

There are two modes of working. Mode · What the model may use · Typical instruction · Good forClosed-book · Only its own knowledge · None · Brainstorming, general explanations Open-book, strict · Only the supplied text · "Use only the text below" · Policy and contract answers, compliance, audit Open-book, flexible · Supplied text first, general knowledge flagged · "Prefer the text. Mark anything not from it as [outside]" · Research, drafting with background Pick the mode deliberately and say so. The flexible mode, with an explicit marker for outside knowledge, is a good compromise when pure strictness would leave the answer unhelpful, since it keeps the two sources visible.

Predict first: you ask an assistant "Does the policy allow an exception for senior customers?" The policy text you pasted says nothing about senior customers. The instruction says "Answer using the policy." What will a typical model do, and how could you find out for certain? Decide before reading on.

Many models will produce an answer anyway, perhaps hedged, perhaps stated as fact, built from how such policies usually work. To find out for certain you cannot rely on a single attempt. You need to test deliberately with questions whose answers are absent, which is the subject of a later section.

Designing citations

A citation ties a claim to its place in the source. Done well, it turns verification from a search into a glance. Four design choices decide whether it works.

Label the sources. Give each document, or each chunk of a long one, an identifier the model can use: [S1], [S2], or a section number such as [Policy 4.2]. Put the label with the text, not only in your prompt's description of it.

text Source S1: Leave policy, version dated 1 April 2025, section 4. """ 4.2 Employees may carry forward a maximum of ten days of earned leave to the following calendar year. """

Source S2: HR circular dated 15 August 2026. """ With effect from 1 October 2026, the carry-forward limit is reduced to seven days. """

Require a quotation, not just a label. A bare reference such as "(S1)" is easy to get wrong and hard to check. Ask for the exact words: [S2: "the carry-forward limit is reduced to seven days"]. A quotation can be found with a text search in seconds, and a made-up one fails the search immediately.

Quote first, then answer. Models produce more faithful answers when they must gather the evidence before writing the conclusion. A pattern that works well is a two-part reply: first list the relevant quotations with their labels, then write the answer using only those. The evidence step makes the model commit to what the text says before it interprets it.

Specify what to do with conflicts and gaps. "If the sources disagree, show both quotations and state which is newer. If no source addresses the question, reply 'Not covered in the provided documents' and stop."

text Answer the question using only the sources above.

Step 1: List every quotation from the sources that bears on the question, each with its source label. If there are none, write "No relevant text" and stop. Step 2: Using only those quotations, answer in at most three sentences, with the source label after each claim. Rules: if sources conflict, state which is newer and why you used it. Do not use knowledge from outside the sources.

Question: What is the current carry-forward limit?

The reply to this should name seven days, citing S2, and note that S2 supersedes S1's ten. A reader can verify both quotations in seconds. Note how much of the prompt is about behaviour at the edges. The central instruction is a single line, while the rest governs conflicts, gaps and format.

Checking citations automatically

If you build tools, a quotation can be checked by a program. It takes the quoted string and searches the source text for it. A quote that is found verbatim is a strong sign the model read the passage. A quote not found is flagged as a failed citation, and the answer can be rejected or retried. This catches fabricated and altered quotations cheaply. It cannot tell whether the quotation actually supports the claim, which still needs a person or a careful second check, so automatic checking narrows the problem and does not close it. For everyday users, the equivalent is pressing Ctrl+F.

Testing what happens when the answer is missing

Ask a grounded system only questions it can answer, and it will look excellent. The real test is the question it cannot answer from the documents. A trustworthy prompt says so. A flawed one invents.

Build this into your test set from Lesson 5.1. Include a fifth to a quarter of cases whose answers are not in the documents, and mark the correct response as "not covered". Also include partially answerable questions, where the text answers one half, and conflicting cases, where two sources disagree. Then measure, separately, how often the system (a) answers correctly when the text covers it, (b) says "not covered" when it does not and (c) invents an answer when it should have declined.

The last number matters most. A system that is right 95 per cent of the time on answerable questions and invents answers to a third of the unanswerable ones is dangerous, because in real use people ask unanswerable questions all the time.

The pattern in the chart is typical: the first instruction helps a lot, and each further one helps less. It also shows that no version reaches zero, which is why checking remains part of the process.

Long documents

A document too long for the context window cannot simply be pasted. Even one that fits may be handled badly, since models can use the start and end of a long input more reliably than the middle, as Lesson 1.2 noted. Four techniques help.

Put the question where it will be seen

For a long document, place your instruction and the question both before and after the text, or at least at the end. A common layout is: the task and rules at the top, the document in fences, then the question repeated at the bottom with the key rule restated. It costs a few words, and often improves recall of the relevant passage.

Test with a planted fact

You can measure how well a tool finds information in a long document with a simple experiment. Insert a distinctive sentence, such as "The reference code for this contract is ZX-4471", at five different positions in the document: near the start, at 25 per cent, in the middle, at 75 per cent, and near the end. For each, in a fresh chat, ask "What is the reference code?" Count how many the tool finds. If it finds all five, you can have more confidence in that length. If it misses the middle ones, you have found a limit, and you should shorten the input or split the document. Repeat for the actual length you work with, since results differ by length and by tool.

Chunk with overlap

When the document is longer than the window allows, split it into chunks and process them in turn. Cut at natural boundaries, such as sections, not mid-sentence. Let consecutive chunks overlap by about 10 per cent, so that a passage straddling a boundary appears whole in at least one chunk. Keep the section heading with each chunk and label each with its place in the document, such as "Chunk 7 of 15, Section 4".

How many chunks do you need? With a document of 40,000 words, chunks of 3,000 words and a 10 per cent overlap, each chunk adds 2,700 new words, so you need 40,000 ÷ 2,700 = 14.8, which rounds up to 15 chunks. Each is about 4,000 tokens, so you process about 60,000 tokens in all, a little more than the 53,000 in the document itself because of the overlap.

Processing chunks one at a time raises a design question: what do you do with the results? Two common patterns serve different jobs. For finding specific facts, run the same question against every chunk, ask each to quote relevant passages or say "none", and then merge and check the quotations. For summarising, summarise each chunk and then combine the summaries, accepting that detail is lost at each stage, as Lesson 4.2 warned. In both patterns, keep the labels, so that every claim in the final answer can be traced back to a chunk and a quotation.

Many documents

With several documents, put the name, date and version at the top of each, and tell the model how to treat them: "These are three versions of the policy. The most recent supersedes earlier ones." Limit how many you include, since irrelevant documents add noise and cost. And tell the model that not everything it receives will be relevant: "Some of the text below may not bear on the question. Ignore what does not." Where a system picks passages automatically, this last line matters, because retrieval is imperfect and the model will otherwise try to use whatever it is given.

Measuring groundedness

To know how well grounded your answers are, check a sample. Take 20 to 40 answers, split each into its separate claims, and for every claim decide whether a quotation in the supplied text supports it: supported, partly supported or unsupported. The share supported is your groundedness. You can have a model do a first pass, asking it to find the supporting quotation for each claim or say "none", and then verify its verdicts on a sample yourself, using the calibration approach from Lesson 5.1.

Suppose you check 40 claims and find 36 supported. Groundedness is 36 ÷ 40 = 90 per cent. The margin of error for 40 claims is 1.96 × √(0.9 × 0.1 ÷ 40) = 0.093, about 9 points, so the plausible range runs from about 81 to 99 per cent. If your target is 95 per cent, 40 claims cannot tell you whether you meet it. You would need several hundred.

Worked examples

Example 1: a policy question service. A bank's operations team builds a prompt that answers staff questions from three policy documents. They label each document, require a quotation for every claim, add the quote-first structure and a "Not covered" reply. On a test set of 30 questions, 6 of which are unanswerable, the first version invents answers to 4 of the 6. After adding the "Not covered" rule and a worked example of declining, it invents on 1. They add that one case to the test set, and note it as a past failure.

Example 2: a long contract. A lawyer must find every clause on termination in a 90-page contract of about 38,000 words. She splits it by section into chunks with the heading attached, asks each chunk for "every sentence about termination, quoted, or 'none'", and merges the results. She then checks that each quotation appears verbatim in the contract. Two do not. Both turn out to be paraphrases, and she corrects them. A single prompt on the whole document had returned a tidy summary that missed a clause buried in a schedule.

Example 3: a planted fact. A team tests a tool on a 30,000-word report by planting a reference code in five places. The tool finds it at the start, the 25 per cent point, the 75 per cent point and the end, and misses it in the middle. They conclude that for documents of this length the middle is unreliable, and they restructure the workflow to split reports in half and ask each half separately, and to repeat the question at the end.

Common mistakes

Assuming "use the text" is enough. Add quotations, a conflict rule and a "not covered" reply. Citing with bare labels. Require the exact words, so checking is a search. Testing only answerable questions. Include unanswerable, partial and conflicting cases. Pasting a very long document and trusting the middle. Test with planted facts, and split if needed. Chunks cut mid-sentence, with no headings. Cut at boundaries, overlap by about 10 per cent and keep the heading. Mixing versions without saying which wins. Name, date and rank your sources. Believing a small sample. A groundedness of 90 per cent on 40 claims could be anywhere from 81 to 99. Letting a model grade itself without checking. Calibrate it against your own judgement.

Key takeaways

An answer is grounded when every claim is supported by the supplied text. A correct claim from the model's memory is still ungrounded for a policy question. Choose a mode deliberately: closed-book, strict open-book or flexible open-book with outside knowledge marked. Good citations use source labels, exact quotations and a quote-first structure, with rules for conflicts and for missing answers. Quotations can be checked by a text search or a program. Test what happens when the answer is absent, with unanswerable, partial and conflicting cases. The rate of invented answers matters most. For long documents, restate the question near the end, test with planted facts, and chunk with about 10 per cent overlap at natural boundaries. A 40,000-word document needs about 15 chunks of 3,000 words. Measure groundedness on a sample of claims, and report it with its margin: 36 of 40 supported is 90 per cent, plausibly 81 to 99.

Check yourself

1. Why ask for exact quotations and not just source labels? A quotation can be found with a text search, so a wrong or invented citation is caught at once. A bare label is hard to check. 2. Why include unanswerable questions in a test set? To see whether the prompt declines when the text is silent, which is where invented answers appear. 3. A 24,000-word document is split into chunks of 2,000 words with a 10 per cent overlap. How many chunks? 24,000 ÷ (2,000 × 0.9) = 13.3, so 14 chunks. 4. What is the purpose of a planted fact? To measure where in a long document a tool reliably finds information, such as whether the middle is missed. 5. You check 50 claims and 46 are supported. What is the groundedness and approximate margin of error? 46 ÷ 50 = 92 per cent. 1.96 × √(0.92 × 0.08 ÷ 50) = about 7.5 points.

Try this

You need an assistant and about sixty minutes.

1. Choose two or three short documents you may share, such as policies or reports. Label each with a name, date and ID. 2. Write 20 questions: 12 the documents answer, 4 they do not, 2 where the documents disagree and 2 where they answer only half. 3. Write a grounded prompt with the quote-first structure, a conflict rule and a "Not covered" reply. Run all 20 questions. 4. Score each answer: correct and cited, correct but uncited, wrong, invented where it should have declined. Find the share invented on the unanswerable ones. 5. Remove the "Not covered" rule and the quotation requirement and rerun. Compare the rates, as in the pilot chart. 6. Plant a distinctive fact at five positions in a long document, and test whether the tool finds it each time. Use the chunk simulator to plan how you would split the document. 7. Check 30 claims across your answers for support, and use the groundedness simulator to state the plausible range. Item · Version with rules · Version without rulesCorrect and cited · · Invented answers where text was silent · · Planted fact found (out of 5) · · Groundedness (claims supported) · · A good result is a grounded prompt with a test set that includes the cases that matter, a measured rate of invented answers and a planted-fact result for your document length. In the final lesson of this module we look at how to get output that other software can use, and how to write instructions for tools and agents.

Tags: Prompt Engineering Grounding Citations Long documents Advanced