Skip to content

← Lab Environment

Environment Architecture & Reproducibility

Authoritative formatted source: Google Doc · PDF

Environment principles

  • Local-first and understandable: avoid cloud complexity until a chapter truly benefits from it.
  • Reproducible: a clean workstation or future-you should be able to rebuild from documented installs, SQL and Python requirements.
  • Synthetic only: no real sensitive or employer data.
  • Source-preserving: raw/source files remain unchanged; transformations create staged/curated outputs.
  • Script-first for learning-critical database state: important objects should be creatable from SQL rather than only GUI clicks.
  • Evidence-aware: outputs, screenshots and decisions have known locations.
  • Destroyable: the lab can be reset without risking personal files or production systems.
Software Role Initial status
PostgreSQL 18 Primary relational lab DBMS Required
DBeaver Community Database navigator / SQL workbench Required
Python 3.x + venv Scripting environment Required
pandas DataFrame profiling/transformation Required after Python foundations
VS Code + Python extension Code/project workspace Required
Git Version control/evidence Required early
PowerShell Windows command-line literacy/setup Built in; learn basics
diagrams.net ERD/architecture/lineage visuals Recommended
DuckDB Later file analytics Optional until triggered
Metabase Later BI/dashboard lab Optional until Chapter 11
OpenMetadata + Docker Later metadata/catalog/quality lab Optional advanced; do not install initially

Suggested local repository structure

meridian-cdmp-lab/
  README.md
  .gitignore
  requirements.txt
  docs/
  data/
    raw/
    reference/
    generated/
  sql/
    00_setup/
    01_core/
    02_security/
    03_integration/
    04_master_reference/
    05_warehouse/
    06_quality/
  python/
    generators/
    profiling/
    integration/
    quality/
  diagrams/
  evidence/
  backups/

PostgreSQL database and schema plan

Use one local training database named meridian_lab. The schema names below are a Meridian pedagogical design, not a DAMA-prescribed architecture.

Schema Meridian purpose Typical contents
raw Landing/source-shaped database inputs when SQL loading is useful CRM/POS source-shaped extracts
staging Clean/standardize/prepare before trusted integration Parsed/normalized fields, source keys
core Operational canonical relational model for the lab Customer, Product, Order, OrderItem, Supplier, Location
master Trusted/mastered shared entities created in later labs Customer master, mastered supplier/product views/tables
reference Controlled code/value sets Country/state, order status, return reason, product category
warehouse Analytical structures Dimensions, facts, analytical views
metadata Lab-maintained technical/business context stored relationally Definitions, source/target mapping, lineage records
quality Rules, profiling results and issue observations DQ rules, test results, issue history
audit Selected training evidence/control logs Load/run records, validation outcomes

Database roles for security labs

Role Purpose
meridian_admin Lab owner for setup only; do not use as everyday analyst.
meridian_loader Loads approved data into designated schemas.
meridian_analyst Reads approved analytical/curated data; should not automatically access all sensitive columns.
meridian_steward Reads domain data and maintains selected reference/master stewardship artifacts where the lab requires.
meridian_bi Reads warehouse views needed for dashboards.
meridian_restricted_test Deliberately constrained account used to prove denied access.

Rebuild sequence

  1. Install software from official sources and record versions.
  2. Clone/copy the lab repository.
  3. Create a fresh Python virtual environment and install declared packages.
  4. Create meridian_lab and required lab roles using controlled setup steps.
  5. Run schema/DDL scripts in order.
  6. Generate or copy synthetic source files.
  7. Load data using documented SQL/Python.
  8. Run validation checks and compare expected counts/constraints.
  9. Only then begin a chapter lab.

Safety and rollback

  • Never point lab scripts at a work or production database.
  • Use an unmistakable lab database name and local connection profile.
  • Before destructive UPDATE/DELETE: SELECT the target first, use a transaction when practical, and record the expected row count.
  • Never store passwords in source code or Git.
  • Keep a known-good rebuild path so break-it labs are safe.
  • Preserve raw source files; create new outputs instead of overwriting evidence-producing inputs.

Source: Meridian Lab Environment & Setup Blueprint, Sections 1–7.