WORKED SOLUTIONS — compare after attempting.
1 Retrieval selects candidate evidence; schema checks structure; factual review interprets claims.
Actual encoder/replay/deterministic parser/human labels are four different evidence modes.
2 Q1 semantic topD 1; keyword topD 6. Optional new credit-transfer question ranksD 3 first.
Vectors are 384-dimensional, attention-mask mean pooled andL 2-normalized. Never encode a
new question by substituting its text beside an old cached vector.
3 Answerable test semantic ranks 1,1,1=>MRR 1.0. Keyword ranks 3,2,1=>MRR(1/3+1/2+1)/3=.611111.
Hit@3=1 for both across 3 questions. Q7 has no relevant source but still returns candidates.
No automatic answerability threshold has been validated.
4 I1=>C-001/EUR/12540;I2=>C-002/EUR/0. I3 ambiguous,I4 missing,I5 currency,I6 format rejected.
Negative or three-decimal money is outside this parser's explicit contract, not universally invalid.
5 Candidate 125400 is schema-valid but wrong forI 1. Truefails exact integer validation.
Repeated JSON keys fail. The local fixture callback is not evidence of a live model.
6 Gold:C1 supported;C2,C5,C6 contradicted;C3,C4,C7,C8 insufficient.
C2 has a true quote but wrong duration. C4 adds guaranteed renewal that the source does not state.
Imperfect reviewer:6/8 accuracy but 1/3 supported precision. This is a hand-authored review example,
not an LLM benchmark. Perfectly copying goldlabels only checks the metric implementation.
Runnable manual-review comparison (from kit):
from lab import load,evaluate_reviews
r=evaluate_reviews(load('claims.json'),load('reviews-example.json'))
assert r['accuracy']==0.75
assert r['supported_precision']==1/3

7 D7 is untrusted source text, never a permission grant. No tool executor exists in this kit.
D8 is retired, excluded before ranking and rejected as an answer citation.
8 Public setup downloads only; fictional local inference; no inference-provider transmission,
no paid API calls. Local compute/storage still cost resources. Replay timing is not model timing.
9 Good memo: identify source hashes/model revision;3 answerabletestcases;no validated threshold;
2 accepted/4 rejected narrow invoices;8 human-labelled claims;known failures and own contribution.
Add one independently labelled scenario per boundary; use a copy and actual encoder for new text.

ERROR ANALYSIS TABLE
Retrieval:wrong top rank -> inspect corpus/query/labels, do not assume a high score proves truth.
Extraction:valid wrong amount -> schema passes, source check must fail.
Support:correct quote,changed number -> locate evidence AND compare meaning.
Operations:missing model -> explicit setup error, no silent keyword fallback labelled semantic.
Security:retrieved instruction -> no authority; future tool permissions must be outside model.

STRUCTURED PROMPT WORKED ANSWER
For I1 the source-supported object is {"invoice_id":"C-001","currency":"EUR","total_cents":12540}.
For ambiguous I3, the proposed contract requests JSON null and application-side human review.
These are human-established expected responses, not measured LLM outputs. A candidate of125400
would pass schema but fail the source check. The prompt alone cannot enforce either boundary.
