Skip to main content

Module 8.1: Orchestration Fundamentals

What an orchestrator actually stores and why the order matters: runs bound to an immutable data interval, the task-instance lifecycle, the dependency edges that turn one failure into a blast radius, sensors that wait and time out, and the retry-versus-SLA-versus-silent-success split that decides what deserves a page. All read with SQL over a simulated scheduler metadata catalog.

0/3

Module 8.2: Idempotency, the Core Property

The spine of the level: the three idempotent-write implementations every 2025-2026 source converges on, each graded by literally running your script twice. Partition replacement driven by an injected run parameter instead of the wall clock, a keyed merge that survives in-batch duplicates whose timestamps tie and a batch that redelivers what you already applied, and an append-only landing zone whose idempotency contract is scoped to the projection built on top of it.

0/3

Module 8.3: Incremental Loads, Late Data & Backfills

The half of incremental loading the shipped SQL lessons stop short of: quantifying the rows a strict watermark silently dropped before you fix anything, healing them with a bounded rolling window, running a manifest-driven backfill that also rebuilds the downstream aggregate it invalidated, and composing the whole daily pipeline into one script you can defend out loud.

0/4

Module 8.4: The Transformation Workflow

The analytics-engineering workflow behind every dbt line on a job description: models as SELECTs built in ref order, data tests and unit tests as queries that pass at zero rows, lineage as a queryable graph, and a dev-to-prod capstone that survives a rerun.

0/3