Specify metrics, denominators and guardrails
Learn the reasoning. Try the task. Compare your answer.
Prepared for DataCareerHub · Curriculum version 2026-10-15.1
By the end of this lesson
- Write a numerator, denominator and missing-data rule for each measure.
- Distinguish percentage points from relative percentage change.
- Use segment checks and guardrails to challenge a headline.
Harbor Helpdesk · UTC · Snapshot 22 June 2026 00:00 · Prepared fictional extract · No Python or SQL required.
Understand the method
A metric name is not a definition. “Fast response” can mean an average delay, the share meeting a target, an automated acknowledgement or the first human reply. A metric contract records the exact calculation, population, units, observation rule and unavailable state. It gives reviewers a way to reproduce the result and spot a denominator that silently changed.
Primary measure: recorded four-hour timeliness
The fictional exercise policy counts a first human response as timely when elapsed calendar hours from creation are between zero and four, inclusive. These are elapsed hours, not business hours; no holiday or staffed-hours calendar is supplied. Divide timely tickets by tickets with valid observed first-response timestamps in the eligible creation cohort. Also report eligible count, observed count and coverage. In this prepared extract every ticket has a valid response, so the observed and eligible denominators both equal 12 per week.
In a real extract, a missing response timestamp can mean no response, missing instrumentation or an event not yet observed. This case does not give you authority to convert an unexplained missing timestamp into a late response. Investigate, report coverage and label bounds or an agreed policy. A negative duration is invalid, not especially fast. A zero duration is a potentially valid value requiring a check against automated acknowledgements; it is not the same as a blank.
A small arithmetic preview
Week 1 has eight timely tickets out of twelve: 66.67%. Week 2 has ten out of twelve: 83.33%. The absolute difference is 16.67 percentage points. The relative change is (83.33% − 66.67%) ÷ 66.67% = 25%, using unrounded fractions. Neither phrase means response times were “25% faster”; the measure is a share meeting a threshold, not a duration.
Now inspect channel composition. Email is four timely out of eight in Week 1 and two out of four in Week 2: 50% in both. Chat is four out of four and eight out of eight: 100% in both. The share of tickets entering chat rises from one third to two thirds. Weight both periods at the Week 1 composition—two thirds email and one third chat—and the illustrative standardized rate is 66.67% in both periods. The headline improvement is compatible with a changed mix while these within-channel rates stay unchanged. This decomposition is descriptive; it does not identify the cause of individual outcomes or prove routing had no effect.
A guardrail and a secondary measure
Use the share of mature eligible tickets with recorded first resolution within 72 calendar hours as a service guardrail. The numerator is nine in each cohort and the denominator is twelve: 75% in both. Keep mature unresolved tickets in the denominator. First resolution is not evidence that a ticket never reopened or a customer was satisfied; those fields are unavailable.
Satisfaction is optional context. Eight ratings are present in each week, with a mean of 4.0/5 among respondents. Coverage is eight out of twelve, or 66.67%. Blank ratings are missing, not zero. Respondents may differ from nonrespondents, and the file has no sampling protocol. Do not present that mean as satisfaction for every customer.
Use counts alongside rates, state units and avoid rounding before comparison. If there are no eligible observations, return an unavailable rate. Guardrails do not automatically approve rollout: management must agree meaningful tolerances and assess evidence quality before a live decision.
Practical exercise
Complete metric_contract_template.txt for response timeliness, 72-hour resolution and satisfaction. State denominator, numerator or aggregation, units, boundary, missingness and one misuse to avoid.
- Verify 8/12, 10/12 and their difference by hand.
- Explain the 25% relative change without calling response times faster.
- Reconcile email/chat counts to each week's total.
- Calculate both rates using Week 1 channel weights.
- Describe what a zero-denominator result should display.
These checks test definitions; they do not authorize a causal claim.
Open the worked solution
Primary: eligible tickets are the specified creation cohort; observed denominator requires valid first-human-response timestamps. Numerator is observed elapsed duration ≤4 hours and ≥0. Report coverage; investigate missing/invalid responses rather than silently treating them as failures. Observed counts are 12/12 per week. Rates are 8/12 = 66.6667% and 10/12 = 83.3333%; difference = 16.6667 percentage points; relative change = 25%.
Segments: Week 1 email 4/8 and chat 4/4; Week 2 email 2/4 and chat 8/8. Counts reconcile 8+4=12 and 4+8=12. With fixed weights 2/3 email and 1/3 chat, both rates are (2/3 × 0.5) + (1/3 × 1) = 2/3. Do not average the channel rates without weights.
Guardrail: mature eligible tickets with first resolution recorded within 72 hours divided by all mature eligible tickets, including unresolved: 9/12 = 75% each. Satisfaction: sum of valid ratings / observed ratings = 32/8 = 4.0 each; coverage 8/12. A blank rating is not zero and the result describes respondents only. A zero denominator yields unavailable, with count zero, not 0%.
One more question.
How these learning materials are prepared and maintained · Report a correction: support@datacareerhub.io
Read and practice without an account. Sign in with an existing account to save lesson completion.
Next lesson →Harbor Helpdesk is a fictional teaching case. Descriptive comparisons do not establish routing causality. Portfolio practice is not a claim of client work or an accredited qualification. Explore practical lessons, worked examples and projects. Follow a published path, practise the method and build work you can explain.