Read this example after your own attempt.
This is guided educational work, not a real client engagement, causal study, accredited assessment or operational approval. Extract the practice pack before opening this page to keep its evidence links together.
Case brief · 24-ticket source · Dictionary · Checked outputs · Your plan template · Review rubric
Decision, scope and evidence.
Audience: operations manager, with service-lead review. Decision: continue a limited pilot and seek stronger evidence before a wider rollout decision. Question: how did recorded response timeliness differ between the two creation cohorts, overall and by channel?
Population: the 24 supplied fictional tickets only, one ticket per row. Week 1 is [1 June 2026 00:00, 8 June 00:00) UTC; Week 2 is [8 June 00:00, 15 June 00:00) UTC. Snapshot is 22 June 00:00 UTC. All tickets are mature for 72 hours. Keep all rows; no unsupported exclusions. This is deliberately constructed teaching data, not a representative operational sample.
Limits: no randomized assignment, routing exposure, staff hours, cost or reopen history. Chat entry and rota changed concurrently. No causal, productivity or company-wide effectiveness claim.
Define the denominator before the chart.
- Four-hour timeliness: first human response elapsed hours in [0,4], divided by valid observed responses in the eligible cohort. Show eligible count and observed coverage. Investigate missing/invalid responses; do not silently count as late.
- 72-hour first resolution: first resolution elapsed hours in [0,72], divided by all mature eligible tickets. Keep mature unresolved tickets in the denominator. First resolution does not guarantee no reopening.
- Optional satisfaction: mean of observed valid ratings, with respondent/eligible coverage. Blanks stay missing; describe respondents only.
- Zero and missing: a zero numerator is known zero successes when observations exist. A zero denominator yields an unavailable rate. Document invalid durations separately.
A rising headline needs context.
| Measure | Week 1 | Week 2 |
|---|---|---|
| Eligible / observed responses | 12 / 12 | 12 / 12 |
| Timely responses | 8/12 · 66.67% | 10/12 · 83.33% |
| Email timeliness | 4/8 · 50% | 2/4 · 50% |
| Chat timeliness | 4/4 · 100% | 8/8 · 100% |
| Chat share | 4/12 · 33.33% | 8/12 · 66.67% |
| 72-hour first resolution | 9/12 · 75% | 9/12 · 75% |
| Rating mean / response coverage | 4.0/5 · 8/12 | 4.0/5 · 8/12 |
The headline changes by 16.67 percentage points, or a 25% relative rate increase. Within-channel rates stay unchanged. Using Week 1 weights (2/3 email, 1/3 chat) gives 66.67% in both periods. This is a descriptive mix calculation, not proof of a positive or zero routing effect.
One plan, in a reproducible order.
- Confirm decision, scope, exercise target and field interpretation.
- Retain source; check 24 unique IDs, UTC parsing, event ordering, snapshot/maturity and categories. Document six missing resolutions and eight missing ratings without deleting their tickets.
- Assign creation cohorts, derive elapsed hours and reconcile 12+12 tickets with response coverage.
- Calculate overall and channel counts/rates, channel weights, guardrail and respondent-only satisfaction. Reconcile segments and report exact fractions.
- Review evidence limits, write the handover and request exposure, staffing and longer consistent event history before considering an attributable evaluation.
Acceptance: traceable source/version; reconciled counts and weighted rates; correct units and missing-data treatment; clear alternative explanations and no unsupported causal claim. Proposed review roles and operational tolerances need actual stakeholder agreement. No sign-off is invented.
Turn a vague request into a decision brief
Worked brief: Audience: operations manager, with the service lead reviewing definitions. Decision: whether to continue the limited pilot and seek stronger evaluation before wider rollout. Question: how did recorded first-response timeliness differ between tickets created on 1–7 and 8–14 June, overall and by channel? Scope: 24 fictional tickets in the supplied general/billing extract, observed to 22 June 2026 00:00 UTC; no customer-level conclusions. Deliverable: a one-page evidence note, cohort/channel table, metric contract and reproducible analysis plan. Limitation: concurrent channel-entry and staffing changes, small teaching data and no assignment/control record prevent an attributable routing-effect estimate. A proposal is to request exposure and staffing records; it is not confirmation that those data exist. Approval of expansion remains a management decision.
Five useful questions: Which queues were exposed (defines treatment population)? What is the approved response target (defines numerator)? Does the timestamp exclude bots (defines measurement)? What changed in staffing (identifies alternatives)? What evidence would justify expansion or stopping (defines decision requirements)? “Prove routing worked” is the leading phrase. An all-queue cost-effectiveness decision is unsupported without exposure, capacity, cost and stronger outcome evidence.
Define scope, grain and fair comparison windows
Scope: one ticket per row; key ticket_id. Week 1 is [2026-06-01 00:00, 2026-06-08 00:00) UTC and Week 2 is [2026-06-08 00:00, 2026-06-15 00:00) UTC. Snapshot is 2026-06-22 00:00 UTC. Retain all 24 supplied rows: 12 per cohort. Include general/billing and email/chat only as represented; no unsupported exclusions. Out of scope: causal routing effect, unique-customer outcomes and productivity/cost return.
The 7 June arrival belongs to Week 1 despite its resolution date. The hypothetical 14 June 23:00 arrival has seven days and one hour of follow-up, so it is mature for 72 hours. A ticket arriving two hours before a live snapshot is immature for that outcome and must be excluded from the mature-outcome denominator with a reported count, not marked a failure. Filtering to resolved tickets removes legitimate mature non-resolutions, making a success rate conditional on already succeeding. These hypothetical boundary tickets are reasoning examples, not extra rows to append to the source.
Specify metrics, denominators and guardrails
Primary: eligible tickets are the specified creation cohort; observed denominator requires valid first-human-response timestamps. Numerator is observed elapsed duration ≤4 hours and ≥0. Report coverage; investigate missing/invalid responses rather than silently treating them as failures. Observed counts are 12/12 per week. Rates are 8/12 = 66.6667% and 10/12 = 83.3333%; difference = 16.6667 percentage points; relative change = 25%.
Segments: Week 1 email 4/8 and chat 4/4; Week 2 email 2/4 and chat 8/8. Counts reconcile 8+4=12 and 4+8=12. With fixed weights 2/3 email and 1/3 chat, both rates are (2/3 × 0.5) + (1/3 × 1) = 2/3. Do not average the channel rates without weights.
Guardrail: mature eligible tickets with first resolution recorded within 72 hours divided by all mature eligible tickets, including unresolved: 9/12 = 75% each. Satisfaction: sum of valid ratings / observed ratings = 32/8 = 4.0 each; coverage 8/12. A blank rating is not zero and the result describes respondents only. A zero denominator yields unavailable, with count zero, not 0%.
Request the evidence and plan quality checks
Inventory: cohort → created_at_utc; timeliness → creation/first response; guardrail → creation/first resolution plus snapshot; mix → channel; satisfaction context → optional rating and coverage. Routing exposure, staffing hours and longer baseline are unavailable requests, not imputed columns.
Checks/actions: unique nonblank key → investigate duplicates; parsed UTC dates → quarantine malformed rows; creation in window → count exclusions; nonnegative response/resolution durations → investigate invalid events; events ≤ snapshot and mature eligibility → separate unobserved/immature outcomes; categories in dictionary and ratings 1–5 → flag unexpected values. For every action keep source rows and record the reason.
Six blank resolutions are mature tickets with no first resolution recorded; retain them in the guardrail denominator. Eight blank ratings are missing observations; retain tickets for other metrics and report respondent coverage. No row deletion follows automatically. Request pseudonymous ticket events, channel/queue, relevant exposure and staffing aggregates via approved access, not names or conversations by default. A bounded descriptive cohort/channel comparison can proceed; an attributable routing-effect or productivity estimate cannot.
Write and review a defensible analysis plan
Plan: confirm scope/definitions; validate source keys/timestamps/categories; assign creation cohorts and mature outcomes; compute observed timeliness, channel rates/weights and resolution guardrail; reconcile, review and write the evidence note. Acceptance requires 24 rows → 12+12, observed response coverage 12/12 each, channel counts/timely counts reconciling, correct percentage-point labels and explicit uncertainty.
Worked handover: The supplied fictional extract shows recorded four-hour first-response timeliness rising from 8/12 (66.67%) to 10/12 (83.33%), a 16.67 percentage-point difference. This does not establish that routing caused an improvement. Email timeliness is 50% in each cohort and chat timeliness is 100% in each, while chat's share rises from 4/12 to 8/12. The 72-hour first-resolution guardrail is 9/12 (75%) in both weeks. All tickets have recorded first responses and sufficient follow-up, but the sample is small and deliberately constructed. Concurrent channel-entry and rota changes, absent exposure records and the lack of a control prevent an attributable effect estimate. I propose reviewing the pilot scope with the service lead and requesting ticket-level routing exposure, staffing coverage and a longer consistently recorded comparison period before an expansion decision. Confirm the service target and rollout criteria with the decision owner; this learning plan supplies no actual operational approval.
The fixed-weight result is 66.67% in each period. It answers an arithmetic composition question; it does not observe what the same tickets would have experienced under another routing process. See the separate worked plan for the complete metric contract, inventory and ten-point rubric.
Your independent contribution.
Explain the fixed-weight calculation in your own words and propose one additional source with its analytical purpose and failure action. Save your own editable plan and calculations. Label this fictional guided work honestly if used in a portfolio.