Data Systems
Playbook · v1.0
Intro
  • 01Overview
  • 02How to Read This
  • 03Why This Matters
  • 04Design Philosophy
Start Here
  • 05Start Here
  • 06Decision Framework
  • 07End-to-End Example
  • 08Same Shape, Different Scale
  • 09Operations Stack
Foundations
  • 10Core Principles
  • 11System Design Flow
Getting Started
  • 12Build Your First Data System
  • 13When to Use Each System
Deployments
  • 14Local Data System (DuckDB)
  • 15Cloud Data System (Postgres / AWS)
  • 16Railway Low-Cost System
  • 17Snowflake / Enterprise System
Standards
  • 18Naming Conventions
  • 19Schema Templates
  • 20Validation & Data Quality
Operations
  • 21How to Run the System
  • 22Orchestration & Scheduling
  • 23Runbooks
  • 24Architecture Diagrams
  • 25Copy-Paste Templates
Workflow
  • 26Project Kickoff Checklist
  • 27New Project Template
Built for engineers · ship early
⌘K
Section 22

Orchestration & Scheduling

Schedule pipelines reliably.

  • Use cron for simple pipelines
  • Use Airflow/Dagster for complex pipelines
bash
0 2 * * * python main.py
Previous
How to Run the System
Next
Runbooks