DataCareerHub
Explore learning
← Advanced forecastingFree companion · 60 minute suggested activity

Cedar 2: compare baselines without leaking the future

Advanced forecasting lab · Build expanding-window evaluations and keep model selection separate from final evidence.

Learn → Try → ExplainReviewed 2026-11-01
01

Baselines before complexity

Compare last-observation naive with period-four seasonal naive. Last-observation naive repeats the most recent value at every horizon. Seasonal naive repeats the last observed four-quarter cycle. A more complex model must earn its place against an agreed baseline; these fixtures do not establish which method is best for real data.

02

Your chronological split

Development contains periods 1–16. Origins 8–14 inclusive mean that 8 through 14 quarters are already observed. Forecast horizons 1 and 2 at each origin, always passing only the observed history to the forecasting function. This yields 14 forecast pairs, all development targets. Select minimum pooled MAE; break exact ties alphabetically by method name. Freeze the method before final evaluation.

03

A two-step example

At origin 8 the observed series ends 10,20,30,40,12,22,32,42; the next two actuals are 14,24. Predict with both methods and compute their errors before opening the answer. These actuals are used for scoring, not to form the forecast.

Worked solution

Naive predicts 42,42; errors are-28,-18, giving MAE 23 for these two pairs. Seasonal naive predicts 12,22; errors+2,+2 give MAE 2. Do not use 14 or 24 to adjust the forecasts before scoring. Across the full Cedar development tournament, naive MAE 120/7 versus seasonal MAE 2; seasonal wins the stated fixture comparison.

05

Final rolling evaluation

Freeze the method, then evaluate origins 16–22 inclusive at horizons 1/2. At later origins earlier test observations have become available and may enter history; that is permitted in a predeclared rolling evaluation. You must not retune the selected method using test losses. Report horizon 1 and horizon 2 separately and note that overlapping forecast pairs are not independent observations.

06

Future-mutation test

Change only observations after an origin. Its predictions must stay unchanged. A later origin may legitimately change after an altered observation has entered its history. Change only final periods 17–24: development selection must remain unchanged. Random row splits or preprocessing fitted on the complete time series violate this chronology.

Rolling versus fixed-origin solution

On this construction the frozen seasonal method has rolling MAE 2 at both horizons. A single eight-quarter forecast issued at origin 16 repeats 16,26,36,46 twice and has MAE 3 against the next two cycles. Fixed-origin and rolling evaluation answer different questions; they do not provide interchangeable performance claims.

08

Interview defense and rubric

Score 0–2 for history slicing, correct seasonal period, split discipline, horizon metrics and mutation reasoning. Explain why refitting or extending history under a frozen rule is different from choosing a new rule after test results. Do not call 14 overlapping pairs 14 independent quarters.

09

Authoritative reference

Read the official reference ↗

3/tscv.html Rob J Hyndman and George Athanasopoulos, Forecasting: Principles and Practice, 3rd ed: time-series cross-validation. Read the chronological evaluation principle; the Cedar fixture/code are original. Next: Cedar 3.

Make the reasoning yours.

Try the activity before reading its reference. Keep your sources, commands and assumptions with your work. Companion activities are unsaved self-practice and do not award a certificate or change lesson progress.

Resume guidance and examples are for learning and preparation only. They do not guarantee interviews, a job offer or employment.

These original companion activities are free. Advanced offline packs are separate supporter extras; companion activities do not change saved course progress.

Learner dashboard ↗ · Ask a question ↗