Objective & prerequisites
Distinguish exact-day return, rolling return and incomplete data. Prerequisites: percentages, basic tables, Python lists/dictionaries and the earlier analysis foundations. Download and extract the full kit, then open a terminal in product-analytics-kit. Python 3.10+ and the standard library are sufficient. Spreadsheet users can inspect the CSVs and reproduce the small controls manually. All people, events and experiment outcomes are invented teaching data. No live tracking, public experiment, payment or registration is activated. These free companion lessons use the existing studio without changing paid access or course progress.
python3 analysis.py
python3 -m unittest discover -s tests -vUnderstand the concept
Exact-day retention asks whether a user performed a specified return action during one particular interval after signup. Here day 1 is [24 h,48 h) and day 7 is [168 h,192 h); these are elapsed UTC intervals, not local calendar days. Returning twice in an interval still counts once. A later return at day 8 does not count for exact day 7. Rolling or “on or after” retention is a different metric with its own bounded observation policy. An empty cell can mean not yet observable, and must not be displayed as 0%. This implementation withholds an entire cohort row unless every member’s relevant interval is complete, preserving the fixed cohort denominator. Other systems may use mature subsets, but they must label the changing denominator explicitly.
Worked example
At the exclusive January 26 watermark, day-7 intervals for January 19 signups have just begun and are incomplete. That cohort’s count and rate are null. The first two mature cohorts return 2/4 and 1/4. Pooled mature day-7 retention is (2+1)/(4+4) = 37.50%; dividing by all 12 users would incorrectly report 25%. U04 returns on day 8 and does not count. U01’s two day 1 events count as one returning user. Overall day 1 is 6/12 = 50%.
from analysis import read_csv, product_data, retention
p,e,_=product_data(read_csv("users.csv"),read_csv("events.csv"))
rows=retention(p,e,7)
mature=[r for r in rows if r["mature"]]
assert sum(r["returned"] for r in mature)/sum(r["size"] for r in mature)==0.375
assert rows[-1]["rate"] is NoneTry it yourself
Reproduce pooled mature day-7 retention from counts, not an unlabelled average of percentages. Then change the latest cohort size in a paper example to 8 and show why an unweighted mean can disagree with a pooled rate. Explain why moving the watermark forward is not enough unless the event source is also complete to that new time. State your denominator, time window and expected result before running the reference. Use a copy for changes and keep the original evidence intact.
Worked solution
Filter to mature rows, sum returned and size, then divide:3/8. With an 8-user cohort returning 2 and a 4-user cohort returning 2, pooled return is 4/12 =33.33%, while the unweighted mean of 25% and 50% is 37.50%. Do not backfill missing event history with zeros or simply assert a later watermark. Advancing the observation boundary requires the actual additional source data and its completeness assurance.
Knowledge check
Is a null retention rate evidence that no one returned?
Check and explain
No. In this case null marks incomplete observation. A mature zero requires a fully observed interval with no qualifying return.
Evidence, mistakes & handover
Keep a metric definition and an auditable count beside every percentage. Distinguish a measured zero, missing telemetry and an incomplete observation window. Record exact commands, actual outputs, assumptions and one failed case. Do not describe an observational association or a manufactured experiment fixture as proof of real business impact. The kit includes separate exercises, worked solutions, a metric contract, an experiment plan template, a decision memo template, tests and a visual worked report. Credit the supplied starting point and explain your own contribution.
Authoritative reference
— reference for the underlying concept; vendor defaults can differ from this lesson’s explicit definitions. Lesson prose, case data and exercises are original. No analytics vendor installation, live user telemetry or production experimentation is claimed.
Try the activity before reading its reference. Keep your sources, commands and assumptions with your work. Companion activities are unsaved self-practice and do not award a certificate or change lesson progress.