Track

dbt, from engine to metaprogramming

There are two tutorials, meant to be read in order. The first explains how dbt Core works internally. The second teaches you to program it with Jinja. Advanced Jinja is much easier once you know when and where dbt renders it.

dbt Core 1.9 · dbt-postgres PostgreSQL warehouse Airflow orchestration Level: intermediate → expert

Learning path

Step 1

dbt Core Architecture & Internals

The packages, the life of a run, parsing, the DAG, compilation, execution and artifacts.

Step 2

Advanced dbt Jinja

The rendering model, macros, compile-time queries, dispatch, overrides, materializations, tests and the graph object.

The running example

Both tutorials use the dwlab project as their running example. It loads AdventureWorks2022 from SQL Server into PostgreSQL staging tables. dbt then builds int_* views, a Type 1 dimsalesterritory, an incremental Type 2 dimemployee and an rpt_load_summary table. Airflow runs every step and passes a run_id to dbt through --vars.

Because the examples come from a real project, you can compile them, run them and break them on purpose to see what happens.