Objective & prerequisites
Define the population, analysis unit, event and observation window before calculating. Prerequisites: percentages, basic tables, Python lists/dictionaries and the earlier analysis foundations. Download and extract the full kit, then open a terminal in product-analytics-kit. Python 3.10+ and the standard library are sufficient. Spreadsheet users can inspect the CSVs and reproduce the small controls manually. All people, events and experiment outcomes are invented teaching data. No live tracking, public experiment, payment or registration is activated. These free companion lessons use the existing studio without changing paid access or course progress.
python3 analysis.py
python3 -m unittest discover -s tests -vUnderstand the concept
A product metric is a rule for turning evidence into a decision-relevant quantity. Begin with a question: where do new learners stop between starting a course and attempting a practice exercise? Define the unit as a stable fictional user_id rather than an event row or a browser visit. Specify the eligible population, identity policy, internal-user exclusion, starting action, conversion action, time zone, ordered sequence and observation cutoff. A rate requires both numerator and denominator. Record whether a time boundary is inclusive, whether later attempts can re-enter and whether a repeated event counts again. A tracking plan describes how evidence is collected; a metric contract describes how that evidence is interpreted. Neither automatically proves that the event reflects meaningful learning.
Worked example
The product snapshot contains 13 signup records, including one internal test user. Excluding that account leaves 12 eligible users. The event watermark is 2026-01-26 00:00 UTC, exclusive: the teaching source is assumed complete before that instant. The funnel is the earliest start, then a later lesson, then a later exercise within 72 hours of that start; only one attempt per user counts. Retention separately uses practice_active, not merely opening a page. These are declared lab rules, not claims about DataCareerHub’s real users or Amplitude’s default settings.
from analysis import read_csv, product_data
people, events, audit = product_data(read_csv("users.csv"), read_csv("events.csv"))
assert len(people) == 12
assert audit["internal_users_excluded"] == 1Try it yourself
Complete metric-contract-template.txt for the three-step funnel and day-7 retention. Write one ambiguous definition such as “active this week,” then rewrite it to state the action, unit, exact window and eligibility rule. Explain why course completion and exercise attempt are not interchangeable. State your denominator, time window and expected result before running the reference. Use a copy for changes and keep the original evidence intact.
Worked solution
The funnel denominator is eligible users with a first start in the observed data; in this fixture all 12 qualify and all 72-hour windows are complete. Retention uses the original signup cohort, not only users who converted. Day 7 means the elapsed interval [signup+7 days, signup+8 days), and an entire cohort row is withheld until all its members have that full interval. The watermark is a completeness assumption documented by the case; it is not inferred from the latest event timestamp.
Knowledge check
If 12 users generate 30 lesson events, is the user conversion denominator 30?
Check and explain
No. The denominator follows the declared user population. Repeated actions may describe intensity, but they do not create additional people.
Evidence, mistakes & handover
Keep a metric definition and an auditable count beside every percentage. Distinguish a measured zero, missing telemetry and an incomplete observation window. Record exact commands, actual outputs, assumptions and one failed case. Do not describe an observational association or a manufactured experiment fixture as proof of real business impact. The kit includes separate exercises, worked solutions, a metric contract, an experiment plan template, a decision memo template, tests and a visual worked report. Credit the supplied starting point and explain your own contribution.
Authoritative reference
— reference for the underlying concept; vendor defaults can differ from this lesson’s explicit definitions. Lesson prose, case data and exercises are original. No analytics vendor installation, live user telemetry or production experimentation is claimed.
Try the activity before reading its reference. Keep your sources, commands and assumptions with your work. Companion activities are unsaved self-practice and do not award a certificate or change lesson progress.