Design and build reliable, monitored pipelines that move data from source systems into warehouses and marts, ready for analytics.
What this covers
- Ingesting raw exports and source data into a repeatable pipeline
- Validating data before it is loaded: schema, null and business-rule checks
- Modeling clean data into a star-schema warehouse and SQL marts
- ELT with dbt, with automated tests that run on every build
- Dashboards or a read-only API on top, so stakeholders can use the data without writing SQL
Tools used across this work: Python, SQL, dbt, DuckDB and Streamlit.
How I approach it
Bad data should be caught at the gate, not discovered three reports later. Raw data passes through validation checks before it reaches the warehouse. Every model is covered by automated tests, so a broken pipeline fails loudly instead of shipping wrong numbers to a dashboard. Transformation logic is written once and re-run on every new data drop, so reporting stays consistent instead of drifting between manual edits.
Work that shows it
- Retail ETL Warehouse: a validated data-quality gate in front of a star-schema warehouse, with a read-only analytics API.
- Olist E-Commerce Analytics Engineering: a dbt star schema on DuckDB over 99,441 real orders, with 34 automated tests and a Streamlit dashboard.
- Retail Analytics Warehouse: a dbt and DuckDB ELT pipeline with 35 automated data-quality tests.
- Marketing Analytics Data Pipeline: raw campaign exports turned into clean, query-ready tables.
- Marketing Attribution Warehouse: a warehouse and pipeline supporting multi-touch attribution.
Choosing the right approach
Whether a pipeline should transform data before or after loading depends on masking needs, compute and who owns the logic. I explain the trade-offs in ETL vs ELT: how to choose between them, and cover rerun safety in idempotent data pipelines.
Work with me
If you have data scattered across exports or systems and need it in one trustworthy place, get in touch with a short description of your sources, where the data should end up, and what reporting you need.