Confusion Cluster — Reference, Master & Data Quality
7. Reference Data vs Master Data
Plain language
Reference Data controls the choices used to classify/describe things. Master Data identifies the persistent business things themselves.
Formal distinction
- Reference Data: controlled permissible/classifying values, lists, mappings, crosswalks, taxonomies and versions.
- Master Data: persistent core business entities reused across processes, often requiring identity resolution, identifiers, relationships and survivorship.
Similarities
Both are shared enterprise data and both require governance/stewardship/quality.
Differences
Classification/value-domain control vs entity identity/context.
Deciding clue
“What allowed value describes this?” → Reference.
“Which Customer/Product/Location is this?” → Master.
Common trap
Shared data is not automatically Master Data.
Mini scenario
Active / Inactive / Pending → Reference Data.
Customer 84721 and its enterprise identity → Master Data.
Counterexample
A controlled list of country codes may be business-critical and used everywhere, but it remains Reference Data; importance/reuse does not turn a code set into a master entity.
Retrieval check
❓ Two systems use different country codes and need a governed translation/crosswalk. RDM or MDM?
✅ Answer: Reference Data Management.
🧠 Why: the problem is controlled permissible values/mappings, not reconciling persistent entity identity.
📖 Source: Chapter 10 pp. 332–340. See Ch10 Deep Card A.
8. Profiling vs Initial Assessment vs Monitoring
Plain language
Profiling is a way to look at the data; Initial Assessment is the baseline management judgment; Monitoring is repeated ongoing measurement.
Formal distinction
- Profiling: technique that reveals data patterns/statistics.
- Initial Assessment: baseline management activity that judges condition/priorities.
- Monitoring: recurring measurement/trends/alerts against rules/thresholds.
Similarities
The same profiling technology can contribute to both assessment and monitoring.
Differences
Technique vs lifecycle/management activity.
Deciding clue
PROFILE sees → ASSESS judges baseline → MONITOR watches over time.
Common trap
Naming the tool rather than the management activity the stem is asking about.
Mini scenario
First exploration of null distribution on a newly scoped Customer dataset → profiling supporting initial assessment.
Hourly invalid-record rate with threshold alerts → monitoring.
Counterexample
A profiling job scheduled every hour can be part of monitoring; the fact that profiling technology is used does not make “profiling” and “monitoring” synonyms.
Retrieval check
❓ An hourly rule score is trended and triggers an alert when it crosses threshold. Profiling, Initial Assessment or Monitoring?
✅ Answer: Monitoring.
🧠 Why: the decisive clue is recurring measurement and alerting over time; profiling may be a supporting technique.
📖 Source: Chapter 13 pp. 443–447, 451. See Ch13 Deep Card E.
9. Prevention vs Correction vs Root-Cause Remediation
Plain language
Prevention stops new defects. Correction fixes data that is already wrong. Root-cause remediation changes the underlying cause so the problem stops recurring.
Formal distinction
- Prevention: controls/process design that prevents future defect creation.
- Correction: repairs existing defective records.
- Root-cause remediation: removes underlying business/technical/process factors causing recurrence.
Similarities
All are part of Data Quality improvement and can occur in one improvement program.
Differences
Future defect vs existing defect vs underlying cause.
Deciding clue
Before → Prevention. After → Correction. Why it keeps happening → Root-cause remediation.
Common trap
Repeated cleansing feels productive but can leave the defect-producing process unchanged.
Mini scenario
- add input validation → Prevention;
- repair 50,000 bad records → Correction;
- redesign the source workflow creating those bad records → Root-cause remediation.
Counterexample
A one-time correction can be entirely appropriate when the causal process was already fixed; “correction” is not automatically weak if recurrence has been prevented.
Retrieval check
❓ The team cleans the same invalid addresses every month but does not change the entry process. What is missing?
✅ Answer: Root-cause remediation (and likely prevention).
🧠 Why: correction repairs existing data, but the recurring production mechanism remains.
📖 Source: Chapter 13 pp. 428, 444–445, 457–459. See Ch13 Deep Card F.
10. Validity vs Accuracy
Plain language
Validity asks whether a value is allowed by the rules. Accuracy asks whether it is true in the real world.
Formal distinction
- Validity: conformance to domain, type, range, format, precision or allowed values.
- Accuracy: correspondence with reality.
Similarities
A defect can violate both; both are Data Quality dimensions used to judge fitness.
Differences
Rule conformance vs truth/reality.
Deciding clue
Allowed? → Validity. True? → Accuracy.
Common trap
Using “accuracy” as a generic synonym for any bad data.
Mini scenario
An email address has valid syntax but belongs to the wrong customer → valid form, inaccurate value.
AGE = 247 violates the approved range → validity problem.
Counterexample
A value can be both invalid and inaccurate, but when the question asks for the best dimension, identify the explicit failure mechanism in the stem.
Retrieval check
❓ A date matches YYYY-MM-DD but records the wrong customer’s actual birth date. Which dimension is most directly violated?
✅ Answer: Accuracy.
🧠 Why: the value can conform perfectly to format/range rules yet fail to represent reality for that customer.
📖 Source: Chapter 13 pp. 430–432. See Ch13 Deep Card A.
← Architecture / Modeling / Metadata · Confusion Book Home →