Data Quality and Data Governance
Data Quality and Governance is an intermediate-level course of 5 lessons (about 1 h 22 min), with 15 exercises and 20 quiz questions. Prerequisites: Intermediate SQL (joins, subqueries). Exercises run in the browser, with nothing to install.
- Data engineering
- Intermediate
- 5 lessons
- 1 h 22 min
- 15 exercises
- Exercises run in the browser
The course in brief
This course teaches you how to make data reliable and well controlled. You will measure quality across its dimensions, write data contracts, use a catalog and lineage to analyze the impact of a change, apply GDPR principles in a data platform, then build an improvement plan with its indicators.
What you will be able to do
- Define and compute quality indicators across the dimensions of completeness, validity, accuracy, consistency, uniqueness, and freshness
- Write a data contract and distinguish compatible changes from breaking changes
- Document a dataset in a catalog and carry out an impact analysis with lineage
- Identify personal and sensitive data and apply minimization, pseudonymization, and retention periods
- Handle a data incident and build a prioritized improvement plan
Syllabus: 5 lessons
- Name and define the main dimensions of data quality
- Turn a business requirement into a measurable quality rule
- Compute quality indicators on a dataset
- 02
Data Contracts
- Explain what a data contract is and what it contains
- Distinguish a compatible change from a breaking change
- Write the contract for a dataset
- 03
Data Catalog and Lineage
- Explain the role of a data catalog and of metadata
- Distinguish the roles of owner, data steward, and technical team
- Use lineage to analyze the impact of a change
- 04
GDPR and Personal Data in Data Platforms
- Identify the personal and sensitive data in a dataset
- Apply GDPR principles to a data platform (purpose, minimization, retention)
- Distinguish pseudonymization from anonymization
GDPRPersonal dataPseudonymization and anonymizationRetention period
- 05
Building a Quality Improvement Plan
- Prioritize quality problems by impact and frequency
- Perform a root cause analysis of a data incident
- Build an action plan with tracking indicators
PrioritizationRoot cause analysisIncident managementContinuous improvement
Final project
Audit and governance plan for customer data
A fictional retailer selling online and in stores has a customers table fed by the website, the checkouts, and the loyalty program. Carry out the quality and compliance audit of this data and propose a governance plan that can be applied within six months.
Deliverables
- A table of quality rules (dimension, formula, threshold) and the measurements obtained on a supplied extract
- The data contract for the customers table, with service level and classification
- The catalog entry, the glossary entries (customer, active customer), and a diagram of the lineage down to the reports
- A classification of the columns with respect to the GDPR, with proposed retention periods and minimization measures
- A prioritized improvement plan (impact × frequency score) with owners, deadlines, and indicators
- The write-up of a simulated incident with its root cause analysis
Assessment criteria
- Each quality rule is measurable, linked to a dimension, and justified by a use
- The contract clearly distinguishes the commitments and flags breaking changes
- Personal and sensitive data are correctly identified, without confusing pseudonymization and anonymization
- The prioritization is argued and the actions fix the causes at the source
- The deliverable invents no quantified legal obligation and refers legal trade-offs to the DPO
Frequently asked questions
What are the dimensions of data quality?
The most common are completeness, validity, accuracy, consistency, uniqueness, and freshness. Each one translates into measurable rules with a threshold suited to the use.
What is a data contract?
It is a written, versioned agreement between the producer of a dataset and its consumers, describing the schema, the meaning of the columns, the quality rules, the freshness, and the owner.
Is pseudonymized data subject to the GDPR?
Yes. Pseudonymized data remains personal data, because the person can be re-identified. Only truly anonymized data falls outside the scope of the GDPR.
Do I need to know how to code to follow this course?
Reading simple SQL queries helps, but the exercises are case studies and questions: measuring, classifying, writing a contract or an action plan.
This course, told for your job
Say what you want to be able to do: “Data Quality and Governance” takes its place in your path, with examples from your field. The concepts stay those of the course.