DataCareerHub
Explore learning
← Advanced causal inferenceFree companion · 60 minute suggested activity

Elm 2: evaluate assignment without dropping non-compliers

Advanced experiment lab · Preserve the assignment estimand and enumerate a small exact randomization test under an explicit design.

Learn → Try → ExplainReviewed 2026-11-01
01

The declared design

Original fictional reminder study: complete random assignment chooses exactly 3 of 6 units. Assignment precedes outcome; assume a well-defined reminder policy, no interference and complete same-time follow-up. Points are measured 0–10. The fixture is hypothetical, not an authenticated experiment log. We target the average effect of assignment to the reminder policy, not the effect of actually receiving a session.

02

Fixture and task

Compute difference in mean points by original assignment, keeping all units in those groups. Compare receipt groups only descriptively. Then enumerate all fixed-size assignments under the sharp null of no assignment effect for ANY unit.

unit | assigned | received | points
u1   | 1        | 1        | 8
u2   | 1        | 1        | 7
u3   | 1        | 0        | 6
u4   | 0        | 1        | 4
u5   | 0        | 0        | 5
u6   | 0        | 0        | 6
03

Your attempt

Predict the two contrasts. Explain why removing u3 or moving u4 into the assigned group would change the analysis question and discard the original randomization comparison. What would missing outcomes require beyond simply dropping rows?

Worked solution

Assigned outcomes 8,7,6 average 7; control 4,5,6 average 5. The intention-to-treat estimate is 2 points under the stated design. Receipt groups 8,7,4 versus 6,5,6 differ by 2/3 point; that is a different descriptive comparison. Treatment receipt may be related to other causes of outcome; no effect-of-receipt estimator is implemented.

05

Exact enumeration contract

Under the sharp null, hold outcomes fixed and enumerate all choose(6,3)=20 equally likely assignments under COMPLETE fixed-size randomization. Compute each mean difference and count absolute statistics at least as large as the observed absolute difference, including ties. A blocked or cluster design would require its own allowed assignments. This is not arbitrary shuffling of observational exposure labels.

Randomization solution

Exactly 4 of 20 assignments have absolute difference at least 2; the specified two-sided sharp-null p-value is 0.2. Equal outcome values still belong to distinct units. A p-value is not probability the null is true, an effect-size estimate or a prediction of replication. This result neither proves no effect nor establishes that the policy works.

07

Failure and missingness review

Reject duplicate unit IDs, invalid assignments, out-of-range/non-finite outcomes and missing outcomes in this complete-outcome teaching fixture. A real study needs a documented missing-outcome analysis and sensitivity plan. Changing receipt alone must not change the assignment estimate or enumeration result; if everyone has identical receipt, that descriptive contrast is undefined while assignment analysis can remain defined.

08

Interview defense and rubric

Score 0–2 for estimand, original assignment retained, design-compatible enumeration, sharp-null interpretation and missingness/compliance limits. The offline supporter pack provides validated inputs, runnable enumeration and mutation exercises. Do not claim a real randomized trial, production uplift or certification.

09

Authoritative reference

Read the official reference ↗

Miguel A Hernán and James M Robins, Causal Inference: What If. Review the distinction between random assignment and observed treatment. The exact toy enumeration is original. Next: Elm 3.

Make the reasoning yours.

Try the activity before reading its reference. Keep your sources, commands and assumptions with your work. Companion activities are unsaved self-practice and do not award a certificate or change lesson progress.

Resume guidance and examples are for learning and preparation only. They do not guarantee interviews, a job offer or employment.

These original companion activities are free. Advanced offline packs are separate supporter extras; companion activities do not change saved course progress.

Learner dashboard ↗ · Ask a question ↗