Applying AI in Data Engineering - the Agentic Data Engineering
How to apply Agentic AI principles and processes to ease the frictions of the current state of data engineering
Data Engineering
Pipelines, streaming, orchestration, data quality and the craft of moving data reliably at scale.
Idempotent Data Pipelines: Making Re-runs Boring
Retries and backfills are inevitable. Design pipelines so that running them twice gives exactly the same result as running them once.