Scenarios 11–15
11 — History plus current stream
Situation: dashboard needs full historical aggregates plus last-seconds events.
Primary: completeness-vs-latency architecture.
Best: batch for history + speed for current + serving for combined view.
Weaker: speed alone loses history; batch alone loses current events.
Changed: current events not needed until next day → remove speed; batch can dominate.
Source: pp. 479–480.
12 — Biased source filter
Situation: model trained only on mobile-app users will be applied to all customers.
Primary: population/sample bias.
Supporting: DQ, Governance, Ethics.
Best: assess representativeness; broaden or constrain target population.
Weaker: model tuning cannot create evidence for excluded customers.
Changed: model used only for mobile-app customers → population mismatch may disappear.
Source: pp. 486, 489, 498.
13 — Anonymous until combined
Situation: two anonymous sources together reveal enough location/behavior to identify people.
Primary: privacy through recombination.
Supporting: Security, Governance, Ethics.
Best: govern combination/output; apply access, masking/encryption/provisioning controls as needed.
Weaker: original anonymity does not guarantee combined anonymity.
Changed: combined output remains safely aggregated with no person/small group inferable → specific re-identification risk reduced.
Source: pp. 486, 498–499.
14 — Training star, test failure
Situation: 99% training accuracy, 61% untouched test accuracy.
Primary: over-fitting/generalization failure.
Best: revisit complexity/features/representativeness; use validation/cross-validation and unseen-data evidence.
Weaker: deploy because training score is high.
Changed: representative test performance is also strong → over-fitting no longer primary; feasibility/ethics/operations become next.
Source: pp. 488–490, 494–495.
15 — Test set reused weekly
Situation: team checks same test after every feature change and keeps changes when score improves.
Primary: loss of independent test evidence.
Best: validation/cross-validation for iterative selection; untouched test for final assessment.
Weaker: calling it “test” does not preserve independence.
Changed: reserve a new untouched holdout → new set can provide independent test evidence.
Source: pp. 489–490, 494–495.