Objective & prerequisites
Create an honest evaluation record for a mock assistant contract. Prerequisites: basic Python expressions and files; use a disposable local practice folder and fictional data. This shared workshop complements the existing data/AI tracks and does not create a second progress system.
Understand the concept
Separate schema validity, evidence support, retrieval coverage and useful task performance. A schema-valid answer can still misstate its cited source. Use known-answer cases, unsupported questions, unknown citations and adversarial source text. Record which checks are automated and which require human interpretation. Costs, latency and provider behavior cannot be inferred from a local mock.
Worked example
structured_answer accepts a declared supported answer withS1 and an empty abstention. It rejects unknown citations and malformed abstention. It checks citation membership, not semantic truth. A misleading answer citingS1 could still pass this structural validator.
python3 foundation_labs.py FoundationTests.test_ai_contractTry it yourself
Write a four-case evaluation table with expected behavior, observed structural result and a separate factual-support review. Add one schema-valid but factually wrong answer and explain the gap. Record your prediction before running the supplied reference, then compare the observed result and explain any difference.
Worked solution
The wrong-but-cited answer demonstrates why membership alone is insufficient. A reviewer must compare the claim to source content or use an appropriately tested factual-support evaluator. No real LLM quality, token cost or production latency was measured.
Knowledge check
Does a valid citation ID prove a claim is supported by the cited passage? Try answering without rereading the example.
Check and explain
No. Membership establishes an allowed source reference, not entailment or accuracy.
Mini-project, mistakes & handover
Produce a short README with the question, exact local command, input meaning, checks and actual results. Include one failure case and its explanation. Never silently turn an exception into a passing result. Mark simulated behavior explicitly, credit the supplied teaching code and identify your own contribution. These activities are unsaved self-practice, not certification.
Authoritative reference
01-prompt-injection/ — consulted for the underlying interface/concept. The lesson and fictional examples are original teaching material. Python reference tested with 3.12; designed for 3.10+. Git lab requires Git 2.28+.
Try the activity before reading its reference. Keep your sources, commands and assumptions with your work. Companion activities are unsaved self-practice and do not award a certificate or change lesson progress.