File formats and encodings
Learn the reasoning. Try the task. Compare your answer.
Prepared for DataCareerHub · Curriculum version 2026-10-15.1
By the end of this lesson
- Apply file formats and encodings to the fictional case.
- Record the rule, evidence and validation result.
- Explain a limitation and a defensible next step.
Fictional learning case · Local practice files · No personal data.
Understand the method
File formats and encodings
A file extension describes a container, not the meaning or quality of its contents. CSV stores rows and delimiters as text; it has no dependable field types. JSON can nest objects and arrays, so one object does not always become one analytical row. Excel can contain several sheets, formulas and formatted blanks. Select the specific sheet and header rather than treating every visible cell as data. Encoding translates bytes into characters. UTF-8 is common; a byte-order mark can attach an invisible character to the first header if decoded incorrectly.
Work through the case
Cedar Supply supplies orders_raw.csv, products.json and a separate monthly planning table. The order CSV uses UTF-8 with a byte-order mark and comma delimiters. It contains 15 data records; the header is not a record. Unit prices use a decimal point and EUR. The product file contains three objects with sku and category. Planning is a different business process; its values are targets, not recorded sales.
A method you can reproduce
Copy the source to a working folder. Record filename, owner, export time, encoding, delimiter and SHA-256. Open CSV with an explicit encoding and newline handling; inspect the parsed headers and a few records before converting types. In a spreadsheet, import identifier fields as text to preserve leading zeros. Keep identifiers such as L001 unchanged. If characters display incorrectly, investigate the declared encoding rather than replacing unfamiliar characters. A hash identifies the bytes you received; it does not certify that those bytes describe reality.
with open('orders_raw.csv', encoding='utf-8-sig', newline='') as f:
rows = list(csv.DictReader(f))
print(len(rows), list(rows[0]))Check your reasoning
A successful open is not evidence of correct parsing. Splitting each line on a comma breaks quoted fields containing commas. Let a CSV parser handle quoting and record boundaries; do not guess the delimiter from a single unusual row.
Further reading: pandas merge validation, pandas melt, Python CSV parsing. These official references support the methods; the case and exercises are authored for DataCareerHub.
Practical exercise
Inventory all three supplied sources. Count data records and inspect the first header for a byte-order mark. Explain why products.json and the planning CSV cannot simply be appended to order rows.
Write your answer in learner_worksheet.txt and retain the evidence file. Attempt this before opening the solution.
Open the worked solution
The order source has 15 data records and eight columns. Decode using utf-8-sig to remove the optional leading mark. products.json has three lookup entries; plan_wide.csv has three category rows and two monthly values per category. These have different grains and schemas. Hash each source before editing a copy; never replace the original export.
Independent extension: change one relevant input in a separate copy. Predict the result first, rerun the checks and explain any difference. Keep the supplied source unchanged.
One more question.
How these learning materials are prepared and maintained · Report a correction: support@datacareerhub.io
Read and practice without an account. Sign in with an existing account to save lesson completion.
Next lesson →This is a fictional learning case. Results do not establish real-world performance or causal effects. Portfolio practice is not a claim of client work or an accredited qualification. Explore practical lessons, worked examples and projects. Follow a published path, practise the method and build work you can explain.