Write and review a defensible analysis plan
Learn the reasoning. Try the task. Compare your answer.
Prepared for DataCareerHub · Curriculum version 2026-10-15.1
By the end of this lesson
- Sequence methods, validation and deliverables in one plan.
- Predefine interpretations and limitations without choosing a desired outcome.
- Review the plan using a practical rubric and an independent extension.
Harbor Helpdesk · UTC · Snapshot 22 June 2026 00:00 · Prepared fictional extract · No Python or SQL required.
Understand the method
An analysis plan is a compact agreement about how evidence will be produced and used. It connects the decision brief, scope, metric contract and data checks from the earlier lessons. It should be detailed enough that another analyst can repeat the work, but short enough that a stakeholder can review the important choices. You are planning the analysis, not presenting a collection of attractive chart ideas.
Use a seven-part structure
- Audience and decision, with a neutral question.
- Population, grain, comparison periods and snapshot.
- Source inventory, provenance and missing dependencies.
- Metric definitions, denominators and unavailable states.
- Methods and checks, in execution order.
- Deliverables, review responsibilities and acceptance conditions.
- Limitations, unresolved decisions and next evidence.
Keep the source immutable. Build a checked working copy, classify creation cohorts, derive response/resolution elapsed hours, and record eligible/observed counts. Reconcile 24 rows to 12 per cohort before computing rates. Then produce overall and channel tables, examine channel weights, review the 72-hour guardrail and include rating coverage only as optional context. The order matters: a polished narrative does not repair a broken denominator.
Choose methods that match evidence
For this teaching extract, counts, proportions and an explicit channel-mix comparison are sufficient. Report exact fractions with rounded display values. Do not introduce a machine-learning model or a significance test merely to make a small descriptive task appear sophisticated. The data were deliberately constructed and were not randomly sampled; a p-value from them would not establish routing efficacy. A causal question needs an appropriate evaluation design and richer evidence, not just a different chart.
Predefine how you will interpret possible patterns. If the overall rate rises but within-channel rates do not, inspect composition before recommending an operational explanation. If the primary measure improves while the resolution guardrail worsens, surface the conflict. If coverage differs, investigate measurement before comparing performance. These are analytical review rules, not automatic management approval thresholds. Obtain actual business tolerances rather than inventing them.
Define deliverables and acceptance
Deliver a one-page plan, metric contract, quality checklist and a short evidence preview with source fractions. A reviewer should be able to trace every number to the ticket rows and every proposed data request to a limitation. Suggested learning acceptance conditions are: correct cohort assignment, reconciled counts, explicit units, correct missing/zero handling, a channel check and no unsupported causal claim. They assess your analytical work; they are not Harbor's approved rollout criteria.
Propose a review sequence: stakeholder confirms the decision/scope; data owner confirms field definitions and access; analyst reproduces calculations; an independent reviewer challenges the interpretation. These are role assignments to agree in a real project. The fictional exercise has no actual sign-off or automatic assessment. Keep version, date and changes so later scope revisions are visible.
Practice independent judgement
Use the worked plan after drafting your own. Explain where you chose the same method and where a different justified approach would be valid. A strong submission can recommend continuing a limited evaluation while collecting exposure and staffing records. It should not claim the intervention is ineffective or effective from this file alone. Finish with a short handover: what was answered, what remains unknown and what evidence would resolve the next decision.
For an independent extension, imagine the chat mix had stayed at Week 1 weights. Show the standardized arithmetic and label it an illustrative composition check. Do not call it an observed alternative history or a causal counterfactual. That distinction is the final test of whether the plan protects its reader from overinterpreting a useful calculation.
Practical exercise
Complete analysis_plan_template.txt and the metric contract, then write a 150–200 word stakeholder handover. Include five execution steps, four acceptance checks and three unresolved dependencies. Use review_rubric.txt before opening the worked plan.
Independent extension: use fixed Week 1 channel weights to compare the periods, then write two sentences explaining why that result is descriptive rather than causal. Save the source, your own plan and the checks locally. Label the case fictional and the solution guided if you include it in a portfolio; do not represent it as client work.
Open the worked solution
Plan: confirm scope/definitions; validate source keys/timestamps/categories; assign creation cohorts and mature outcomes; compute observed timeliness, channel rates/weights and resolution guardrail; reconcile, review and write the evidence note. Acceptance requires 24 rows → 12+12, observed response coverage 12/12 each, channel counts/timely counts reconciling, correct percentage-point labels and explicit uncertainty.
Worked handover: The supplied fictional extract shows recorded four-hour first-response timeliness rising from 8/12 (66.67%) to 10/12 (83.33%), a 16.67 percentage-point difference. This does not establish that routing caused an improvement. Email timeliness is 50% in each cohort and chat timeliness is 100% in each, while chat's share rises from 4/12 to 8/12. The 72-hour first-resolution guardrail is 9/12 (75%) in both weeks. All tickets have recorded first responses and sufficient follow-up, but the sample is small and deliberately constructed. Concurrent channel-entry and rota changes, absent exposure records and the lack of a control prevent an attributable effect estimate. I propose reviewing the pilot scope with the service lead and requesting ticket-level routing exposure, staffing coverage and a longer consistently recorded comparison period before an expansion decision. Confirm the service target and rollout criteria with the decision owner; this learning plan supplies no actual operational approval.
The fixed-weight result is 66.67% in each period. It answers an arithmetic composition question; it does not observe what the same tickets would have experienced under another routing process. See the separate worked plan for the complete metric contract, inventory and ten-point rubric.
One more question.
How these learning materials are prepared and maintained · Report a correction: support@datacareerhub.io
Read and practice without an account. Sign in with an existing account to save lesson completion.
Continue to Module 02 →Harbor Helpdesk is a fictional teaching case. Descriptive comparisons do not establish routing causality. Portfolio practice is not a claim of client work or an accredited qualification. Explore practical lessons, worked examples and projects. Follow a published path, practise the method and build work you can explain.