DataCareerHub
Explore learning
← Advanced forecastingFree companion · 50 minute suggested activity

Cedar 3: audit forecast uncertainty instead of promising certainty

Advanced forecasting lab · Freeze an empirical error band, measure held-out coverage and distinguish an interval from a guarantee.

Learn → Try → ExplainReviewed 2026-11-01
01

A limited uncertainty exercise

You have eight fictional calibration forecast errors and four later held-out cases. These are one-step forecasts assumed issued before outcomes; no real forecasting model is fitted here. Build an empirical absolute-error band, not a formal conformal method or a guaranteed coverage claim.

02

Calibration contract

Calibration signed errors are-4,-3,-2,-1,+1,+2,+3,+4. Sort absolute errors and choose nearest-rank empirical quantile 0.75: rank ceil(0.75*n), one-based. Freeze this radius before inspecting test outcomes. A chosen calibration quantile is not a promise that 75% of future values will fall inside the resulting band.

03

Your attempt

Calculate the radius. Each held-out point forecast is 20; actuals are 18,23,25,15. Construct [max(0,forecast-radius),forecast+radius], inclusive of both endpoints. Count covered cases and calculate average width. Then state at least two limits of this evidence.

Worked solution

Sorted absolute errors are 1,1,2,2,3,3,4,4. Rank 6 gives radius 3. Each test band is[17,23], width 6. Actual 18 and 23 are inside;25 and 15 are outside. Observed coverage is 2/4=50%, not a verified long-run 50% rate or a 75% future guarantee.

05

Do not repair the test into validation

If you enlarge the band after reading these four test outcomes, document that it has become a design/selection set. A redesigned method needs new final evidence. Report width alongside coverage: a very wide band may cover outcomes but have little decision value. Tiny synthetic samples cannot establish calibration.

06

Boundary and zero exercises

Forecast 1 with radius 3 gives[0,4], width 4; the lower bound is clipped because demand is nonnegative. A point exactly at either endpoint counts as covered. A zero-error calibration sample gives a zero-width band, not evidence that future outcomes are certain. Missing or non-finite observations should fail validation.

07

Horizon and regime review

One-step residuals cannot automatically justify an eight-step interval. Investigate horizon, changing scale, dependence, structural breaks and label availability. A prediction interval concerns a future observation; it is not a confidence interval for an estimated mean. No uncertainty method is made trustworthy merely by drawing a shaded band.

08

Review and explain

Score 0–2 for calibration separation, quantile convention, endpoint/clipping, coverage/width and limits. Explain why our empirical radius exercise should not be marketed as a guaranteed service level. The offline supporter project includes a reproducible implementation and mutation prompts.

09

Authoritative reference

Read the official reference ↗

3/prediction-intervals.html Rob J Hyndman and George Athanasopoulos, Forecasting: Principles and Practice, 3rd ed: prediction intervals. Our deliberately limited empirical-band exercise is original and does not reproduce their method or example. Next: Cedar 4.

Make the reasoning yours.

Try the activity before reading its reference. Keep your sources, commands and assumptions with your work. Companion activities are unsaved self-practice and do not award a certificate or change lesson progress.

Resume guidance and examples are for learning and preparation only. They do not guarantee interviews, a job offer or employment.

These original companion activities are free. Advanced offline packs are separate supporter extras; companion activities do not change saved course progress.

Learner dashboard ↗ · Ask a question ↗