Training and test data
Learn the reasoning. Try the task. Compare your answer.
Prepared for DataCareerHub · Curriculum version 2026-10-15.1
By the end of this lesson
- Apply training and test data to the fictional case.
- Record the rule, evidence and validation result.
- Explain a limitation and a defensible next step.
Fictional learning case · Local practice files · No personal data.
Understand the method
Training and test data
Training fits the method; a test estimates performance on unseen examples after decisions are fixed. A random split is sometimes appropriate for independent, identically distributed cases, but it can mix future observations into training. A chronological split better matches prediction for later arrivals when time matters. Repeated entities may also require grouped separation.
Work through the case
The pilot uses its first 40 dispatches for training and last 20 for a one-off teaching test. Fix the features, preprocessing and model before scoring the 20 rows. The main 180-row dataset belongs to later modules and has separate training, validation and test partitions. Pilot results must not be reported as the final main experiment.
A method you can reproduce
Check sorted dispatch dates, unique parcel IDs and disjoint partitions. Fit all preprocessing on the first 40 pilot rows. Use those fitted parameters unchanged on the final 20. If you change the approach after seeing pilot test results, treat it as exploratory and seek a fresh holdout for a final claim. No split makes the small generated dataset representative of real logistics.
Check your reasoning
An outcome can be visible in a downloadable file without being legitimate training input. The workflow discipline is to avoid consulting held-out outcomes during design. A production study would often restrict access to them or use an independent evaluation owner.
Further reading: NumPy least squares, Preventing leakage, Cross-validation. These official references support the methods; the case and exercises are authored for DataCareerHub.
Practical exercise
Write down the pilot split before running code. Verify no parcel ID overlaps. Explain why the main test set must not be used to choose preprocessing or a threshold.
Write your answer in learner_worksheet.txt and retain the evidence file. Attempt this before opening the solution.
Open the worked solution
Training has 40 pilot rows and test has 20. Their IDs are disjoint and dates ordered. Fit medians, scaling and categories only on the training rows. The main test is reserved for Module 5; using it to choose a threshold would contaminate its role as a final estimate.
Independent extension: change one relevant input in a separate copy. Predict the result first, rerun the checks and explain any difference. Keep the supplied source unchanged.
One more question.
How these learning materials are prepared and maintained · Report a correction: support@datacareerhub.io
Read and practice without an account. Sign in with an existing account to save lesson completion.
Next lesson →This is a fictional learning case. Results do not establish real-world performance or causal effects. Portfolio practice is not a claim of client work or an accredited qualification. Explore practical lessons, worked examples and projects. Follow a published path, practise the method and build work you can explain.