Explore distributions without deleting inconvenient values
Learn the reasoning. Try the task. Compare your answer.
Prepared for DataCareerHub · Curriculum version 2026-10-15.4
By the end of this lesson
- Calculate and interpret mean, median and quartiles.
- Use a stated IQR convention to flag unusual values.
- Report a sensitivity analysis without rewriting the baseline.
April–May teaching extract · Order values in EUR · Delivery blanks remain missing · Use a local spreadsheet; Python is optional.
Understand the method
A single average compresses a distribution. Before calling the average order “typical,” examine where most values lie, how wide the range is and whether a few large records influence the mean. Use the 40 verified completed orders across both months. Do not include returned or cancelled orders in one statistic and exclude them from another. Keep the same population while changing the summary technique.
Mean and median answer different questions
The arithmetic mean divides total completed-order value by the number of completed orders. It is useful for relating order count to the total: total value equals count multiplied by mean. The median describes the middle of the sorted values. With 40 records, take the mean of the 20th and 21st values. It is less sensitive to an extreme amount, but it does not preserve the additive link to total sales. Report the measure appropriate to your question rather than selecting whichever supports a preferred story.
Here the completed-order total is €6,320. Dividing by 40 gives a mean of €158. The two central values are both €110, giving a median of €110. This difference invites an inspection of the distribution. It does not establish that any record is wrong. Plot a frequency table or histogram with clearly stated EUR boundaries and counts; keep the €1,000 record visible rather than clipping the axis without explanation.
Specify a quartile convention
Software packages can calculate sample quartiles differently. This module uses inclusive linear interpolation: sort n values, place percentile p at zero-based position (n−1) × p, and interpolate between its neighbouring values. Excel PERCENTILE.INC follows this convention. The supplied worked script implements it directly. Name the convention in your notes so a small difference from a different box-plot implementation is not mistaken for a source error.
For this fixture Q1 is €100 and Q3 is €180. The interquartile range is €80. A common exploratory upper fence is Q3 + 1.5 × IQR, giving €300; the lower fence is −€20. Values outside these fences merit investigation. A020 at €1,000 is the only flagged order. The fence is an exploratory screen, not a rule authorising deletion. In particular, it does not replace the dictionary’s non-negative-value validity rule.
Investigate before choosing a treatment
The verified reference confirms A020’s €1,000 amount. It is a legitimate large order within the exercise. Retain it in the baseline. Investigate its contribution, show its source ID and explain how it affects summaries. If a different source contract proved a recording mistake, correction would require that evidence. A z-score, IQR fence or visual surprise alone cannot supply the true value.
Make sensitivity transparent
As a separate scenario, remove only A020 and recompute. The remaining 39 completed orders total €5,320, with mean €136.41. Label this “excluding one verified high-value order,” preserve the original €158 baseline and explain that the population changed. This is a robustness exploration, not an improved estimate or a corrected sales total. Showing both results helps a reader see concentration without concealing real activity.
Practical exercise
Filter the verified data to completed orders. Compute count, total, mean, median, Q1, Q3, IQR and both fences using inclusive interpolation. Build a frequency table using [0,100), [100,150), [150,200), [200,300) and [300,1100] EUR bins; the last interval includes its upper boundary. Identify flagged orders by ID and check their values against the reference.
Deliverable: a distribution summary and a labelled baseline/exclusion comparison. Explain why the verified €1,000 order remains in the baseline even though it exceeds the fence. You may use a spreadsheet or the optional worked Python file after attempting the task.
Open the worked solution
| Measure | Completed population |
|---|---|
| Count / total | 40 / €6,320 |
| Mean / median | €158 / €110 |
| Q1 / Q3 (inclusive) | €100 / €180 |
| IQR / lower fence / upper fence | €80 / −€20 / €300 |
| Flagged record | A020, €1,000, verified |
| Without A020 | 39 orders, €5,320, mean €136.41 |
The frequency counts are 7, 14, 18, 0 and 1 respectively. They sum to 40. Count values equal to €100 in the second bin, €200 in the fourth and €1,000 in the fifth. The 20th and 21st sorted completed values are both €110. Inclusive quartiles give positions 9.75 and 29.25 in zero-based indexing; the neighbouring values at those positions are €100 and €180 respectively.
Calculate IQR as 180 − 100 = 80 and the upper fence as 180 + 1.5 × 80 = 300. The €1,000 order lies above the fence and is supported by the reference, so a defensible answer retains it and discusses concentration. A baseline mean of €158 and an exclusion mean of €136.41 describe different populations. Do not substitute the lower scenario into the original monthly totals.
One more question.
Method references: NIST exploratory data analysis and box plots and IQR fences. The case and exercises are authored for DataCareerHub; the quartile convention is stated in the dictionary.
How these learning materials are prepared and maintained · Report a correction: support@datacareerhub.io
Read and practice without an account. Sign in with an existing account to save lesson completion.
Next lesson →Fictional training data. Portfolio practice is not a claim of client work or an accredited qualification. Explore practical lessons, worked examples and projects. Follow a published path, practise the method and build work you can explain.