Define the extract and build a useful data profile
Learn the reasoning. Try the task. Compare your answer.
Prepared for DataCareerHub · Curriculum version 2026-10-15.4
By the end of this lesson
- Define the population, grain, period and units before exploring.
- Profile rows, distinct keys, categories and missing fields.
- Separate extract coverage from real-world completeness.
April–May teaching extract · Order values in EUR · Delivery blanks remain missing · Use a local spreadsheet; Python is optional.
Understand the method
Exploratory data analysis begins with a question and a deliberate look at the data. For this module, Northstar Shop asks whether May’s larger completed-order sales justify expanding a sales initiative. Before comparing numbers, establish what each row means, which records belong in the comparison and what the extract cannot tell you. A polished chart cannot repair a population that changes between calculations.
A new period in the same fictional business
Modules 1–3 used February and March. Module 4 introduces a separate April–May 2026 teaching extract. It does not revise earlier results or describe real customers. The verified file contains 44 unique orders, 22 in each month. Each month has 20 completed, one returned and one cancelled order. The raw file contains 46 records because two identifiers occur twice. One repetition is identical; the other has conflicting channel values. A row count and an order count therefore answer different questions.
Write the analytical contract
The business measure is the recorded value of completed orders, in EUR. These are order values, not audited revenue, cash collected, profit or customer lifetime value. For the headline calculation, include completed orders dated from 1 April through 31 May. Use exclusive upper boundaries of 1 May and 1 June for monthly filters. Keep returned and cancelled rows available for a separate status analysis. The business-key grain is one verified order per order_id.
The supplied dictionary defines delivery_days as a recorded elapsed duration for a completed order. Blank means no measurement was supplied. It does not mean zero days. There is no customer identifier, advertising spend, product margin, transaction timestamp or experiment assignment. You cannot calculate customer conversion, profitability, daily operating speed or marketing causality from these fields. Listing these absences early prevents attractive but unsupported metrics later.
Build a compact profile
Start with total records, distinct identifiers, duplicate groups, date parsing, category frequencies, numeric ranges and blank counts. Profile the untouched raw file first. Count blank order values separately from negative values and count invalid delivery text separately from missing delivery measurements. A single “bad rows” total hides different actions and can double-count a record with several issues. Preserve record identifiers beside every exception.
After preparation, repeat the profile on the verified population and reconcile row counts. Record both observed date bounds and the intended reporting window. The last order is dated the 22nd in each supplied month; that does not prove the business had no orders afterwards. For the exercise we treat the fixture as the complete supplied extract, not independent proof of complete company activity. The stated comparison is descriptive of these records.
Make the profile useful
A useful profile connects a finding to a consequence. “A duplicate key exists” leads to checking whether it is an export repetition or a conflicting business record. “There is a blank duration” leads to reporting observed coverage before comparing averages. Keep the raw extract unchanged and maintain your decisions separately. This makes your exploration reproducible rather than a sequence of undocumented edits.
Practical exercise
Download orders_raw.csv, orders_verified.csv and the setup/data dictionary. Without editing the raw file, record the reporting population, business key, currency, row count and distinct order count. List at least four checks you will perform before calculating sales. Then profile the verified file by month and status.
Deliverable: a one-page profile containing raw records, distinct identifiers, duplicate groups, verified orders, completed orders and a paragraph stating the evidence boundary. Explain why the latest observed date cannot establish coverage of all real-world activity.
Open the worked solution
The raw extract has 46 rows and 44 distinct order IDs. A004 and Y007 each appear twice. The verified extract has 44 rows and 44 IDs. April and May each contain 22 orders: 20 completed, one returned and one cancelled. The analytical sales population therefore contains 40 completed orders, not 44 or 46 records.
| Check | Expected observation | Consequence |
|---|---|---|
| Grain | One verified order per order_id | Resolve repeated keys before totals |
| Value units | EUR, already in major currency units | Do not divide by 100 as in Module 3 |
| Monthly completed denominator | 20 orders per month | Use the same population for sales and AOV |
| Source coverage | Supplied April–May fixture only | Avoid claiming complete company trading coverage |
A sound scope note is: “Compare recorded completed-order values in the supplied April–May 2026 extract. Preserve excluded statuses for status counts. Treat delivery blanks as unavailable measurements. This fictional sample cannot identify customer conversion, profit or a causal effect of the initiative.” Use the dictionary to check parsing rules, inspect categorical values, identify negative or blank amounts and review duplicate records. A source profile describes what arrived; it does not authorize repairs.
One more question.
Method references: NIST exploratory data analysis and box plots and IQR fences. The case and exercises are authored for DataCareerHub; the quartile convention is stated in the dictionary.
How these learning materials are prepared and maintained · Report a correction: support@datacareerhub.io
Read and practice without an account. Sign in with an existing account to save lesson completion.
Next lesson →Fictional training data. Portfolio practice is not a claim of client work or an accredited qualification. Explore practical lessons, worked examples and projects. Follow a published path, practise the method and build work you can explain.