Section 24
Architecture Diagrams
Reference architectures for each deployment.
Source
API / DB / files
Ingestion
raw landing
Bronze
raw_*
Silver
_clean
Gold
_summary
Outputs
BI / API / files
Local
text
CSV / Excel → Python → DuckDB → Excel ExportRailway
text
CSV / API → Railway Worker → Postgres → App / DashboardAWS
text
Source → S3 → ECS/Lambda → Postgres → BI / APISnowflake
text
Source → Stage → Snowpipe / dbt → Data Mart → BILocal — detailed
text
┌─────────────┐ ┌──────────────┐ ┌────────────────────────┐
│ data/raw/ │ ─▶ │ Python jobs │ ─▶ │ warehouse.duckdb │
│ *.csv/json │ │ (pipelines/) │ │ bronze / silver / gold │
└─────────────┘ └──────────────┘ └──────────┬─────────────┘
│
┌───────▼────────┐
│ exports/*.xlsx │
│ Streamlit app │
└────────────────┘AWS — detailed
text
┌──────────┐ ┌──────────┐ ┌────────────┐ ┌─────────────┐
│ Sources │──▶│ S3 │──▶│ ECS / λ │──▶│ Postgres │
│ APIs/DBs │ │ bronze/ │ │ transform │ │ bronze/ │
└──────────┘ └──────────┘ └────────────┘ │ silver/ │
│ gold/ │
└──────┬──────┘
│
┌────────────────────┼────────────────────┐
▼ ▼ ▼
Metabase FastAPI Reverse-ETLSnowflake — detailed
text
┌──────────┐ ┌──────────┐ ┌─────────────┐ ┌──────┐ ┌──────┐
│ Sources │──▶│ Stage │──▶│ Warehouse │──▶│ dbt │──▶│ BI │
│ APIs/DBs │ │ (S3) │ │ bronze/ │ │ │ │ │
└──────────┘ └──────────┘ │ silver/gold │ └──────┘ └──────┘
└─────────────┘