Track

Data Warehousing

Lab:Ubuntu 24.04 VM on AWS+Docker · docker-compose+SQL Server 2022 on Linux (container)+Airflow 2.7 (containers)The whole track runs on one Ubuntu Linux VM deployed in AWS: SQL Server 2022 and Airflow both run in Docker containers.

Build a data warehouse the way it is built at work: from an operational source system, through staging, dimensions and facts, a data quality gate and reporting views, with every step scheduled, logged and re-runnable. Everything runs on a real lab, and every number on these pages came from that lab: one Ubuntu Linux VM in AWS running SQL Server 2022 and Apache Airflow in Docker containers.

14 chapters ~4 hours SQL Server 2022 · T-SQL · Apache Airflow 2.7 · Docker Level: beginner → advanced

Tutorials

Tutorial · End to end

End-to-End Data Warehouse on Linux + Docker: SQL Server 2022 + Airflow

The SBEcoLTDDW warehouse from the LinkedIn Learning course Implementing a Data Warehouse with SQL Server 2022, extended into a full pipeline: AdventureWorks2022 → staging → SCD2 dimensions → incremental facts → columnstore and indexed views → DQ gate → reporting, orchestrated by Airflow.

How to read the diagrams

Colour / markMeaning
AFTealApache Airflow: orchestration only
DWBlueSQL Server 2022 and the SBEcoLTDDW warehouse
SRCOrangeSource systems and files the warehouse reads from
—Grey arrowsData flow
Dashed teal arrowsAirflow runs a SQL file on SQL Server
Numbered circlesMatch the numbered steps under each diagram

Use ⤢ Full screen on any diagram to see it at full size. On a phone, scroll the diagram sideways.