Section 17

Snowflake / Enterprise System

Warehouse-first architecture with governance and separation of compute.

Layout

text
Snowflake account
├── DATABASE: ANALYTICS
│   ├── SCHEMA: BRONZE   (raw, owned by ingest role)
│   ├── SCHEMA: SILVER   (transform role)
│   ├── SCHEMA: GOLD     (read-only to BI)
│   └── SCHEMA: AUDIT
└── WAREHOUSES
    ├── WH_INGEST_XS
    ├── WH_TRANSFORM_M
    └── WH_BI_S  (auto-suspend 60s)

Ingest with Snowpipe

snowpipe.sql
CREATE OR REPLACE STAGE bronze_stage
  URL='s3://co-data/bronze/claims/'
  STORAGE_INTEGRATION = s3_int
  FILE_FORMAT = (TYPE=PARQUET);

CREATE OR REPLACE PIPE bronze.pipe_raw_claims
  AUTO_INGEST = TRUE AS
COPY INTO bronze.raw_claims
FROM @bronze_stage
FILE_FORMAT = (TYPE=PARQUET)
ON_ERROR = 'CONTINUE';

Transform with dbt

dbt_project.yml
models:
  analytics:
    bronze:
      +materialized: incremental
      +schema: bronze
    silver:
      +materialized: incremental
      +schema: silver
    gold:
      +materialized: table
      +schema: gold

Governance

  • Roles: INGEST, TRANSFORM, ANALYST, BI.
  • Row-access policies on PII columns.
  • Object tagging for cost allocation.
  • Resource monitors per warehouse.