Compare a simple model with a baseline
Learn the reasoning. Try the task. Compare your answer.
Prepared for DataCareerHub · Curriculum version 2026-10-15.1
By the end of this lesson
- Apply compare a simple model with a baseline to the fictional case.
- Record the rule, evidence and validation result.
- Explain a limitation and a defensible next step.
Fictional learning case · Local practice files · No personal data.
Understand the method
Compare a simple model with a baseline
A baseline shows what can be achieved without the proposed complexity. For duration regression, a training mean is a simple fixed predictor; a median is another useful baseline for absolute-error loss. For classification, a training majority-class predictor exposes misleading accuracy in an imbalanced sample. Compare predictions on exactly the same held-out records and include error units and denominators.
Work through the case
The pilot linear model has mean absolute error 1.407 hours versus 5.862 for the training-mean baseline on 20 test rows. The depth-one classifier and majority baseline both predict no late parcels at the default threshold, obtaining 65% accuracy while missing all seven late parcels. That apparently reasonable accuracy conceals zero recall.
A method you can reproduce
MAE is the mean of absolute actual-minus-predicted differences. For classification record true positives, false positives, false negatives and true negatives. Precision is TP/(TP+FP) and is unavailable when there are no positive predictions; recall is TP/(TP+FN). The reference returns null rather than inventing zero precision. Record whether the model improves the relevant measure, not just whether it exceeds an arbitrary accuracy target.
Check your reasoning
An improved pilot MAE does not prove the model will generalise to new operating conditions or improve decisions. The poor classifier is useful evidence; do not hide it or tune repeatedly on its test to manufacture success.
Calculate absolute error
For observed durations 10, 14 and 18 hours with predictions 12, 12 and 15, the absolute errors are 2, 2 and 3 hours. MAE is 7/3 = 2.333 hours. Compute the baseline on the same records and target definition. Do not average a model score on twenty parcels with a baseline score on a different ten-parcel subset.
Further reading: NumPy least squares, Preventing leakage, Cross-validation. These official references support the methods; the case and exercises are authored for DataCareerHub.
Practical exercise
Reproduce the pilot comparison and write a short error note. Calculate majority-baseline accuracy and recall from seven late and thirteen non-late test parcels.
Write your answer in learner_worksheet.txt and retain the evidence file. Attempt this before opening the solution.
Open the worked solution
The always-not-late baseline gets 13/20 = 65% accuracy and 0/7 = 0 recall. Precision is unavailable because it makes no positive predictions. The linear model improves MAE on this synthetic pilot; the stump provides no classification advantage at the fixed threshold. Report both findings.
Independent extension: change one relevant input in a separate copy. Predict the result first, rerun the checks and explain any difference. Keep the supplied source unchanged.
One more question.
How these learning materials are prepared and maintained · Report a correction: support@datacareerhub.io
Read and practice without an account. Sign in with an existing account to save lesson completion.
Review the module checklist →Review the complete course →This is a fictional learning case. Results do not establish real-world performance or causal effects. Portfolio practice is not a claim of client work or an accredited qualification. Explore practical lessons, worked examples and projects. Follow a published path, practise the method and build work you can explain.