Making a model cite its sources

A music teacher records a 30 minute private lesson. By that evening, the student should have a practice plan for the week: what to work on, in what order, for how many minutes, and a clear definition of done for each item.

This is a good task for a language model. It is also a task where a plausible wrong answer is worse than no answer at all. If the model decides the teacher said 80 beats per minute when the teacher said 72, the student practices at the wrong tempo for a week and arrives at the next lesson having reinforced the exact thing they were supposed to fix.

Nobody catches it. The teacher does not have the recording open. The student was not paying close attention. The document is fluent, specific, confident, and wrong.

I built this system for Virtunity, an audition-preparation product I build for band students. I am a band director; the lessons in the eval set are my own. These are notes on the architecture, the failure modes I actually observed, and what each one forced me to change.

The model can propose facts. It is never allowed to establish them.

Every claim presented as coming from the teacher has to cite verbatim evidence from the transcript, and deterministic code, not another model, decides whether that evidence supports it.

Filled = model call · Outlined = deterministic code · Repair runs only when validation fails

Why the obvious version does not work

The obvious version is one call. Transcript in, practice plan out, good prompt, a structured output schema so the shape is guaranteed.

The shape being guaranteed is exactly the problem. A JSON schema will happily promise you that tempoBpm is an integer. It will not tell you the teacher never said a tempo. Schema validation gives you well-formed output, and well-formed output is what makes fabrication dangerous, because it sails through every check you have and lands in front of a student looking like a fact.

What I needed was not a shape contract. It was an evidence contract.

The architecture

Six stages, two model calls in the happy path. The tags matter more than the numbers: half of this pipeline is ordinary code that cannot be talked out of its answer.

1. 01
   ExtractModel call

   Temperature 0, structured output. Pulls assignments out of the transcript. Each one must carry a verbatim evidenceQuote.

2. 02
   ValidateDeterministic code

   Every quote must appear verbatim in the transcript. Every number must appear inside the quote that is supposed to support it.

3. 03
   RepairModel callonly on failure

   Replay the rejected output, then send the failed checks and the fragments they name. Ask for a targeted fix, not a rewrite.

4. 04
   DraftModel call

   Temperature 0.1, structured output, cheaper model. Presentation only, over a payload that has already been verified.

5. 05
   ValidateDeterministic code

   Different rules: coverage, exclusions, arithmetic, provenance.

6. 06
   RepairModel callonly on failure

   Same targeted mechanism as stage 3, then a stronger-model retry before failing closed.

Stage 1 is a strong model doing careful reading. Stage 4 is a cheaper model doing presentation. Splitting them is where most of the cost savings came from, and it also clarified the design: extraction is a retrieval problem, drafting is a writing problem, and they do not need the same model.

The unit that makes all of this work is the evidenceQuote. Every assignment the extractor produces must carry one, and it must be a verbatim substring of the transcript.

What the validator actually checks

This is the part that matters, so here it is concretely. None of it is a model. It is a few hundred lines of TypeScript that either passes or produces a list of specific failed checks.

On extraction

• Every evidenceQuote, on both assignments and exclusions, must appear verbatim in the transcript after whitespace normalization. Not semantically present. Present.
• Every tempoBpm, reps, and durationMinutes presented as teacher-stated must appear inside its verified evidenceQuote. The model-authored task text cannot establish a number. Matching normalizes the ways people actually say numbers out loud: digits and number words through twenty, “a minute” and “an hour,” ranges like “five or ten minutes,” and minute forms written adjacent to the unit.
• No duplicate assignment IDs.
• focusTags must come from a fixed vocabulary of fourteen. Anything outside it is an error, not a new tag.
• Assignment count has a target and a hard ceiling. Over target is a warning; over ceiling is an error.
• A quote under four words, or one with no obligation token in it, raises a weak_evidencewarning. “Play slowly” is technically in the transcript, and it is not evidence that an assignment was given.

On the drafted plan

• Every step’s sourceExtractionIDs must reference an assignment that actually exists. The model cannot cite something it made up.
• Every extracted assignment must be cited by at least one step. This is the coverage check, and it catches the opposite failure from fabrication: the teacher assigned six things and the plan quietly includes four.
• estimatedMinutes must equal the sum of minutesPlanned across steps. Models are bad at addition and good at sounding certain about it.
• dailyMinutesTarget must equal the value the application passed in. The model does not get to renegotiate the target.
• The minutes sum must land within 10% of that target.
• Step count target and ceiling, no duplicate step IDs, at most six focus tags.

A guarantee I tightened while writing this

That second bullet did not say what it says now until I audited the implementation against this field note.

What I found: numeric validation was testing the number against a support string built from both the transcript-verified evidenceQuote and the model-authored taskfield. The quote is checked verbatim against the transcript. The task field is not. It is the model’s own restatement. So a model-written task could, in principle, support its own number even when the cited quote did not mention it. The drafting instructions made it explicit, telling the model a duration counted as teacher-stated if either field carried the number.

The benchmark had been catching invented numbers in practice, so the failure mode was covered. The invariant was simply weaker than I believed it was.

Before changing anything, I replayed the stricter rule over every stored extraction artifact I had, 94 files across the model comparison and dialect runs. Of eleven numeric values present, nine were already supported by the verified quote, zero were supported only by task text, and two were supported by neither. Those two were the same invented durationMinutes=10 on Lesson 12, in pre-repair attempt files from two separate runs. The loophole was real in the code and unexercised in the data.

So I closed it. Numeric grounding now scores against the verified evidenceQuote alone, in production and in the offline harness, and the drafting instruction was corrected to match. Then I reran the full scenario suite live.

• All six ground-truth checks still pass, including the reed break-in schedule, which is the most number-dense assignment in the set.
• Both exclusion scenarios are unchanged, still passing.
• No legitimate teacher-stated number was rejected. Final extraction and plan validation came back with zero errors and zero warnings on all four scenarios.
• Repairs went down, not up. The standard scenario previously needed one repair and now needs none, which dropped it from three model calls to two.
• The one repair that did fire, on the synthetic 60 minute transcript, was the legitimate catch: the same invented ten minutes, rejected with the new message.

The repair count falling was the part I did not predict, and the cause is worth naming. The old instruction told the model a duration could be teacher-stated on the strength of its own task text, while the check increasingly disagreed in spirit. Once instruction and check said the same thing, the model stopped claiming durations it could not support, and the round trip that used to correct it disappeared. When your prompt and your validator disagree, you pay for it in repairs.

Grounded and filler

Two problems pull against each other. A plan built only from a transcript is often short of the student’s daily practice target, because a teacher does not narrate a full practice session out loud. But if you tell a model to fill the remaining time, it will invent repertoire.

So every step is tagged grounded or filler, and the two are constrained differently.

A grounded step must carry both source IDs and source evidence, and each quote it lists must be exactly the evidenceQuote of the assignment it cited. Not a paraphrase, not a different line from the transcript. This closes a specific hole I will come back to.

A filler step must carry no source IDs and no evidence at all, and its visible text has to match a generic pattern (warm-up, long tones, scales, review) and must not match a specific-material pattern (lesson, measure, piece, etude, concerto, sonata, march, overture, numbered exercises). A filler step also cannot claim a teacher-stated duration.

The effect is that filler is prevented from presenting itself as assigned repertoire, under the validator’s specific-material rules. It can say “long tones for five minutes.” It cannot say “the Rose etude for five minutes,” because “etude” is on the specific-material list and the step is rejected. Padding is allowed. Padding that impersonates an assignment is not. The guarantee is exactly as wide as that vocabulary, which is a list I extend when a new evasion shows up rather than something I can claim is complete.

Duration provenance works the same way. Each step’s durationSource is teacher_stated, suggested, or none, and a step claiming teacher_statedmust have that number present in the cited assignment’s transcript-verified evidenceQuote. A suggestion and an instruction look identical in a rendered document, so they are kept distinguishable all the way to the interface.

Exclusions, which turned out to be the hard part

Teachers say things like “don’t worry about the scales this week, just the etude.” This is a normal, casual, mid-sentence instruction, and it was the thing the system got wrong most often and most embarrassingly. A plan that assigns the exact thing the teacher just waved off is worse than a mediocre plan. It tells the teacher the software was not listening.

Exclusions became a first-class part of the extraction schema: explicitlyExcludedAssignments, each with a name, aliases, a reason, and its own verbatim evidence quote. Then two checks:

• At extraction, if any assignment’s text matches an exclusion term, that is an error. The comment in the code reads exclusion wins.
• At plan validation, every visible surface is swept: the lesson summary, the key observations, and each step’s title, do-text, and done-when. If an excluded term appears in text a student will read, the plan is rejected.

Two eval scenarios lock this down. One is the casual mid-lesson exclusion. The other checks that when a teacher assigns something and then changes their mind later in the same lesson, the later instruction wins. Both pass.

Repair, not retry

When validation fails, the naive move is to retry the whole call and hope for better sampling. That is expensive and it does not converge, because nothing told the model what was wrong.

The repair call is the same conversation with two turns appended. The rejected output is replayed back as an assistant turn, so the complete object is in context and the model is looking at its own work. The new user turn carries only the failed checks and the specific fragments those checks named, with the instruction to return the complete object in the original schema and change only what the failed checks require.

The narrowing is a focusing device, not a context restriction. The model has everything it had the first time, plus its own answer; the repair turn tells it exactly where to look.

Repair is bounded at one attempt per stage, and the two stages fail differently:

• Extraction gets one repair. If validation fails again, the request fails closed. There is no path that returns an unverified extraction.
• Drafting gets one repair, and if that still fails, one clean re-attempt at the stronger extraction model instead of the cheap drafting model, with the original messages and no repair framing. If that also fails validation, the request fails closed.

Failing closed matters more than it sounds. The alternative is a plan that reaches a student carrying claims that nothing checked.

What a repair costs depends on which stage needed it, and the asymmetry is instructive. The drafting stage never sees the transcript, since it works from the compact verified extraction, so its repairs run around 1,500 to 3,000 input tokens. An extraction repair re-sends the transcript and is not cheap; on the 60 minute synthetic it cost about 11,900 input tokens. That gap is a good reason to spend the careful model on extraction and get it right the first time.

The failure modes I actually observed

Four, in the order I found them.

assignment_2 has unsupported durationMinutes=10;
the number is absent from its evidenceQuote and task

grounded step step_1 sourceEvidence is not the cited
assignment evidenceQuote: "Play slowly."

The eval harness

The benchmark runs the production prompts, schemas, temperatures, and output limits copied unchanged from the deployed cloud function, against frozen transcripts of real lessons, scrubbed before use with a substitution map kept out of version control.

Scoring is against hand-written ground truth, not model judgment. I wrote down what I actually said in those lessons, as six checkable statements, including things like lesson 12 number 3: read the correct notes, B to G rather than B to A, and the reed break-in schedule of one minute per reed on day one, two on day two, five on day three. The harness checks whether each configuration recovered them, and separately verifies evidence quotes against the transcript, extraction source IDs, step counts, minute totals, and unsupported tempo and repetition values.

The scenario set is small and deliberately adversarial: a standard 30 minute lesson at target duration, two exclusion scenarios, and a synthetic 60 minute transcript that tests whether the current token budgets handle a longer lesson without truncating. The budgets are linear in scheduled lesson length by construction; what the synthetic transcript checks is whether that is actually enough, on one case.

Current verdicts: the standard scenario scores 6/6 on ground truth, exclusions pass, no excluded material leaks into the plan, and minutes land exactly on target. Both exclusion scenarios pass. The 60 minute synthetic finishes on stop rather than a length cutoff.

What the harness does not do is judge whether a plan is good teaching. It checks that the plan is faithful to the lesson. Whether the sequencing is pedagogically sound is still a human review, and I think that is the correct division of labor.

Results

76% cheaper, with 74% lower aggregate API latency than the version it replaced, and stricter validation. Aggregate API latency is the sum of the vendor calls, not end-to-end wall clock; the stages run in sequence, so it is the dominant term, but it is not the same measurement.

That combination surprised me, and the reason is worth stating: the expensive part was never the validation. It was asking one large model to do careful reading and polished writing in the same call. Splitting the work let the reading happen once, at temperature 0, in a compact schema, and let a cheaper model handle presentation over a payload that was already verified.

The middle column is the version where I had the right idea and the wrong decomposition. I am including it because the naive split made things worse before it made them better.

What I would tell someone building something similar

• Pick a unit of evidence and make it mechanically checkable. A verbatim substring is a crude primitive, and that is exactly why it works. No interpretation, no second model to trust, no threshold to tune. Semantic similarity would have been more flexible, but it would have weakened the guarantee I wanted: I was not asking whether the evidence meant roughly the same thing. I wanted to know whether the cited words were actually there.
• Do not use a model to check a model when arithmetic will do. Sums, set membership, coverage, substring presence, and vocabulary constraints cover most of what actually goes wrong. They are free, instant, and they do not have opinions.
• Separate retrieval from presentation. They want different models, different temperatures, and different validation. Merging them is why the first version was both expensive and unverifiable.
• Tag provenance and carry it to the interface. Grounded versus filler, and teacher-stated versus suggested, are two bits that let the product be honest about what it knows. Both were added because a fluent document had erased a distinction that mattered.
• Repair beats blind retry. Give the model the exact failed checks and the fragments they concern instead of resampling the whole operation. Drafting repairs are cheap because that stage never sees the transcript; extraction repairs are not, which is another reason to optimize for getting extraction right the first time.
• Write ground truth from the source, before you look at any output. Every temptation to grade generously disappears when the answer key predates the attempt.

Honest status

Let the model do language. Do not let it do facts. Make the boundary something a computer can check.