WORKED ANSWERS — compare after attempting exercises.
1 See metric-contract-worked.txt. Unit=user, not event; define strict order and observation.
2 45-1 exact retry-1 internal event-2 after-watermark events=41. Unknown users or
conflicting event IDs fail. A delivery retry is not the same as a different repeated action.
3 Completers U01,U02,U04,U05,U09; intermediate U03,U06,U10; start-only U07,U08,U11,U12.
Moving U06 to exactly firststart+72h yields12,8,6; final6/12=50%,laststep6/8=75%.
Runnable exercise solution (from kit directory):
import copy
from datetime import timedelta
from analysis import read_csv,product_data,funnel
p,e,_=product_data(read_csv('users.csv'),read_csv('events.csv'))
changed=copy.deepcopy(e)
start=next(x['at'] for x in changed if x['user_id']=='U06' and x['event_name']=='start')
next(x for x in changed if x['user_id']=='U06' and x['event_name']=='exercise')['at']=start+timedelta(hours=72)
assert funnel(p,changed)['counts']==[12,8,6]

4 Jan 5:n4,day 1=3/4,day 7=2/4;Jan 12:n4,2/4,1/4;Jan 19:n4,1/4,N/A.
Channels are evenly alternating in each cohort; channel imbalance is not observed here.
Neither acquisition cohorts nor future-behavior groups are randomized treatments.
5 Mature day 7 is 3/8=37.5%, not 3/12=25%. Exact day 7 excludes day 8. Null is not zero.
Paper unequal-size example:2/8 and 2/4 => pooled4/12=33.33%,simple mean37.5%.
6 ITT 20%,28%; exposed-only 22.22%,35% conditions on post-assignment behavior and changes
the population. Missing outcome rows/observed0 stop the report; never silently zero-fill.
7 Difference .28-.20=.08=8 percentage points; .08/.20=.40=40% relative.
SE=sqrt(.2*.8/100+.28*.72/100)=.060133; margin≈.117859.
95% normal interval[-.037859,.197859]; includes zero and meaningful harm/benefit.
No superiority or equivalence is established. Outcome counts are manufactured.
Sample planning uses:
ceil((z(.975)*sqrt(2*pbar*(1-pbar))+z(.8)*sqrt(p0*(1-p0)+p1*(1-p1)))^2/(p1-p0)^2)
p0=.20,p1=.28,pbar=.24 gives 447 per arm. Approximation assumes independent binary
units and fixed horizon; it does not guarantee realized significance or operational power.
SRM uses2*BinomialCDF(min(counts),n,.5),capped1;130/70 flags at.001.
SRM passing is not proof of correct assignment. A planned fixed100/100 randomized design
has structurally balanced counts; diagnostic illustrates independent Bernoulli50:50 assignment, not a calibrated test of
this fixed-count randomizer. A real diagnostic must match its assignment design.
Guardrail5/100 versus 7/100:+2 points observed. Sparse counts fail this kit's normal interval.
Use a separately verified suitable method and practical harm margin before safety claims.
8 See decision-memo-worked.txt. Fix wrong denominator, wrong lift units, unsupported launch.

MISTAKE CLINIC
A High exercise event count proves high user conversion: false; validate ordered unique users.
B Latest cohort has0% day 7: false; unobservable window is N/A.
C Bigger cohorts always change retention: not necessarily; use consistent numerator/denominator.
D p(SRM)=1 proves perfect experiment: false; it is only one diagnostic.
E Repeatedly inspect and stop when95% interval excludes0: violates fixed-horizon interpretation.
F Missing tracking means no action: false unless independent complete-observation rule establishes it.
