Your first prediction experiment

1. Features and targets

Features and targets

Features are inputs available when a prediction is made; the target is the outcome to be predicted. Identifiers can memorise entities rather than generalise. Date fields can encode drift or be useful for splitting without belonging in the model. Start with a feature allowlist and explicitly exclude fields that reveal the target or occur after prediction.

Work through the case

The separate pilot_parcels.csv contains 60 synthetic parcels. Use distance_km, load_ratio and zone as predictors. actual_hours is the regression target; late is its thresholded classification target. Both are outcomes, never inputs for the other task in this experiment. post_delivery_rating is especially tempting because it correlates with delay, but it arrives after delivery.

A method you can reproduce

Build X from the three allowed fields and y from the chosen target. Record missing numeric values rather than deleting those rows without measuring coverage. Preprocessing parameters must be fitted on training rows only. The reference uses training medians for missing values and training mean/standard deviation for scaling; category levels are also learned on training rows. New categories receive an explicit unknown-category count.

Check your reasoning

Excluding the target column by name is insufficient if a proxy still reveals it. A feature audit needs field meaning and timing. The reference code’s explicit FEATURES list makes the selection inspectable but cannot replace business review in a real project.

Further reading: NumPy least squares, Preventing leakage, Cross-validation. These official references support the methods; the case and exercises are authored for DataCareerHub.

Exercise

List allowed and excluded pilot fields. Explain why dispatch_date is used for ordering but not included as a predictor. Identify two leakage paths.

Write your answer in learner_worksheet.txt and retain the evidence file. Attempt this before opening the solution.

2. Training and test data

Training and test data

Training fits the method; a test estimates performance on unseen examples after decisions are fixed. A random split is sometimes appropriate for independent, identically distributed cases, but it can mix future observations into training. A chronological split better matches prediction for later arrivals when time matters. Repeated entities may also require grouped separation.

Work through the case

The pilot uses its first 40 dispatches for training and last 20 for a one-off teaching test. Fix the features, preprocessing and model before scoring the 20 rows. The main 180-row dataset belongs to later modules and has separate training, validation and test partitions. Pilot results must not be reported as the final main experiment.

A method you can reproduce

Check sorted dispatch dates, unique parcel IDs and disjoint partitions. Fit all preprocessing on the first 40 pilot rows. Use those fitted parameters unchanged on the final 20. If you change the approach after seeing pilot test results, treat it as exploratory and seek a fresh holdout for a final claim. No split makes the small generated dataset representative of real logistics.

Check your reasoning

An outcome can be visible in a downloadable file without being legitimate training input. The workflow discipline is to avoid consulting held-out outcomes during design. A production study would often restrict access to them or use an independent evaluation owner.

Further reading: NumPy least squares, Preventing leakage, Cross-validation. These official references support the methods; the case and exercises are authored for DataCareerHub.

Exercise

Write down the pilot split before running code. Verify no parcel ID overlaps. Explain why the main test set must not be used to choose preprocessing or a threshold.

Write your answer in learner_worksheet.txt and retain the evidence file. Attempt this before opening the solution.

3. Compare a simple model with a baseline

Compare a simple model with a baseline

A baseline shows what can be achieved without the proposed complexity. For duration regression, a training mean is a simple fixed predictor; a median is another useful baseline for absolute-error loss. For classification, a training majority-class predictor exposes misleading accuracy in an imbalanced sample. Compare predictions on exactly the same held-out records and include error units and denominators.

Work through the case

The pilot linear model has mean absolute error 1.407 hours versus 5.862 for the training-mean baseline on 20 test rows. The depth-one classifier and majority baseline both predict no late parcels at the default threshold, obtaining 65% accuracy while missing all seven late parcels. That apparently reasonable accuracy conceals zero recall.

A method you can reproduce

MAE is the mean of absolute actual-minus-predicted differences. For classification record true positives, false positives, false negatives and true negatives. Precision is TP/(TP+FP) and is unavailable when there are no positive predictions; recall is TP/(TP+FN). The reference returns null rather than inventing zero precision. Record whether the model improves the relevant measure, not just whether it exceeds an arbitrary accuracy target.

Check your reasoning

An improved pilot MAE does not prove the model will generalise to new operating conditions or improve decisions. The poor classifier is useful evidence; do not hide it or tune repeatedly on its test to manufacture success.

Calculate absolute error

For observed durations 10, 14 and 18 hours with predictions 12, 12 and 15, the absolute errors are 2, 2 and 3 hours. MAE is 7/3 = 2.333 hours. Compute the baseline on the same records and target definition. Do not average a model score on twenty parcels with a baseline score on a different ten-parcel subset.

Further reading: NumPy least squares, Preventing leakage, Cross-validation. These official references support the methods; the case and exercises are authored for DataCareerHub.

Exercise

Reproduce the pilot comparison and write a short error note. Calculate majority-baseline accuracy and recall from seven late and thirteen non-late test parcels.

Write your answer in learner_worksheet.txt and retain the evidence file. Attempt this before opening the solution.