Integrated Capstone — From Opportunity to Production
Situation
A retailer wants store-level stockout prediction using POS history, supplier feeds, weather, logistics events, and real-time shelf sensors. Supplier schemas change; weather coverage varies by region; customer-related POS data can become identifiable when joined to location; sensors stream continuously. The prototype performs well on training data. The business acts on replenishment only every 30 minutes. Leaders want an executive dashboard and automatic reorder recommendations.
Seven-part resolution
1. Primary problem
Design a governed, source-aware, quality-aware Data Science lifecycle that turns varied Big Data into feasible production prediction/prescription.
2. Leading Knowledge Area
Big Data and Data Science.
3. Supporting Knowledge Areas
Metadata Management; Data Quality; Integration; Data Security; Data Governance; Data Architecture; Reference & Master Data; DW/BI.
4. Roles
Business sponsor; Data Scientist; Analytic Design Lead; Platform/Ingestion Architects; Metadata/DQ specialists; Data Stewards; security/privacy specialist; operations consumer.
5. Best activity/control sequence
- define measurable stockout decision and 30-minute actionable latency;
- evaluate/select sources for meaning, coverage, reliability, bias, privacy, cost;
- capture source/schema/lineage Metadata at ingest;
- profile DQ and track schema changes;
- align store, product, time, weather region, supplier and reference context;
- preserve separate training/validation/test evidence and evaluate generalization;
- assess recombination/privacy risk in location + customer information;
- choose architecture to meet 30-minute, not imaginary millisecond, requirement;
- communicate predictions/assumptions clearly to executives and operations;
- add prescriptive reorder recommendations only when action logic is supported;
- deploy with monitoring for model performance, value, source changes, DQ, cost, adoption, and new questions.
6. Tempting weaker response
“Build the fastest streaming platform and train on every available field.” It ignores value/latency fit, source trust, Metadata, DQ, privacy, bias, over-fitting, and operational governance.
7. Changed fact
If the use case becomes safety-critical sub-second action, architecture should shift toward a much stronger speed/operational-analytics requirement. The governance, DQ, Metadata, and model-quality obligations do not disappear.
Source: synthesis of pp. 473–501.