Executive Learning · Free learning

Data Cleaning

The unglamorous work that determines whether anything downstream can be trusted.

In this module

0 of 6 activities complete

Reading

0% read

The five common defects

Missing values, duplicates, inconsistent formats, outliers and definition mismatches account for the overwhelming majority of data quality incidents.

Each has a defensible treatment, and each treatment is a decision that must be documented. Silently dropping rows is the most common way an analysis becomes misleading.

Cleaning is a policy, not a task

Mature organisations encode cleaning rules once, in the pipeline, with tests. Immature organisations clean in spreadsheets, differently, every time.

Real-world case studies

Case study · Operations

A manufacturer recovers a failed predictive maintenance programme

Situation
A manufacturer had installed vibration and temperature sensors across two plants but its failure prediction pilot performed no better than a maintenance calendar.
Challenge
Sensor feeds contained duplicated readings, unit mismatches between plants and long unlabelled outage periods, so the model was learning the instrumentation rather than the machinery.
How data was used
Readings were deduplicated, harmonised to common units, aligned to a single clock, and outage periods were reconciled against maintenance work orders.
How AI was applied
An anomaly detection model was retrained on the cleaned series, with alerts tuned to give maintenance teams a usable lead time.
Business outcome
Unplanned downtime fell against the prior year, and maintenance moved from fixed intervals to condition-based scheduling.

Lesson for the learner

When a model underperforms, suspect the data pipeline before the algorithm. Cleaning is usually the highest-return work available.

Data Cleaning, executive briefing

17:10 · Video briefing

Key takeaways

Missing values, duplicates, format drift, outliers and definition mismatch cause most incidents
Every cleaning decision must be recorded, because it changes the answer
Cleaning belongs in the pipeline with automated tests, not in a spreadsheet

Write three cleaning rules

Turn implicit judgement into explicit policy.

Take one dataset from Module 2
Write the rule for missing values, duplicates and outliers
Decide who signs off a change to those rules

Module 4 quiz

1 questions · 70% to pass

01An analyst deletes all rows with a missing revenue value before reporting. What is the primary risk?Applied
The report will render slowlyThe deletion may be systematic and bias the resultThere is no risk, this is best practiceThe file size increases

Answer every question to submit.

Your notes

Lesson discussion

Ask questions, share how you are applying this in your organisation, and learn from other executives.

Loading discussion…