Section 08

Same Shape, Different Scale

The system shape stays the same even when the infrastructure changes.

The core idea of this playbook is that the data system shape stays the same even when infrastructure changes.

The layers remain:

text

Source → Bronze → Silver → Gold → Output

Only the execution environment changes.

Layer mapping across deployments

Layer	DuckDB	Railway + Postgres	AWS + Postgres	Snowflake
Source	Local CSV / Excel	App / API / files	S3 / API / database	Cloud stage / external source
Bronze	Local DuckDB tables	Raw schema	Raw schema or S3 landing	External stage / raw tables
Silver	SQL in DuckDB	SQL jobs	ECS / Lambda / dbt jobs	dbt models
Gold	DuckDB reporting tables	Postgres reporting tables	Curated reporting schema	Data marts
Output	Excel / CSV	API / dashboard	BI / API / Excel	BI / governed analytics

Migration pattern

text

DuckDB version:
CSV → Python → DuckDB → Excel

Railway version:
CSV/API → Worker → Postgres → App/API

AWS version:
Source → S3 → ECS/Lambda → Postgres → BI/API

Snowflake version:
Source → Stage → Snowpipe/dbt → Data Mart → BI

Driver Compliance Across Systems

The same Driver Compliance pipeline runs on three engines. Only the ingestion command changes — the Bronze → Silver → Gold logic is identical.

DuckDB version

sql

SELECT * FROM read_csv_auto('claims.csv');

Postgres version

sql

COPY bronze.raw_claims FROM 'claims.csv' CSV HEADER;

Snowflake version

sql

COPY INTO bronze.raw_claims FROM @stage/claims.csv;

Logic remains the same
Only ingestion method changes
Bronze → Silver → Gold remains identical