Lesson 8 — Tools, Techniques & Correction Modes
Tools support DQ management; they do not define business quality by themselves.
Six tool families
- Data Profiling — discovers content, nulls, distributions, frequencies, ranges, patterns, relationships, anomalies.
- Business Rule Templates / Engines — capture/automate testable rules.
- Parsing / Formatting — decomposes or recognizes components.
- Transformation / Standardization — maps existing values/components into a target approved form.
- Data Enrichment — adds new context or data.
- Incident Management — logs, assigns, escalates, tracks, and preserves issue history.
Profiling is evidence, not judgment
If profiling shows 15% of MIDDLE_NAME is null, that is a factual condition. It becomes a DQ defect only if an applicable business requirement says those values must be present.
Profiling can reveal what is happening. Root-cause analysis explains why.
Parse vs Standardize vs Enrich
Dr. Jane M. Smith→ identify title/given/middle/family = parse.Tennessee,Tenn.,TN→TN= standardize/transform.- add latitude/longitude from trusted source = enrich.
Enrichment can improve usefulness, but “more data” does not automatically mean “more accurate data.”
Three correction modes
Fully automated
Deterministic rule commits correction without human review.
Manually-directed
Automation proposes or applies high-confidence corrections while ambiguous/sensitive cases go to a human/Steward review point.
Manual
A human performs correction through a controlled interface with edits, authorization, and audit.
Manual correction does not mean uncontrolled direct SQL. Direct production patches are explicitly risky because they can bypass application rules, testing, change control, and audit.
Long-term exam rule
If the stem asks for the best long-term response, look for root-cause prevention before choosing repeated cleansing—unless the facts explicitly make controlled ongoing correction the rational operating design.
Source: pp. 451–460.