Two years ago I started writing a blog post to present this concept I came up with as part of my consulting, presentations and training. It’s really nothing fancy – it’s just a simple architecture to create an Azure based Data Warehouse in a cheap and traditional way. Using a SQL Database, Data Factory (or Integration Pipelines in Synapse Analytics) and Power BI. I never accomplished to finish the blog post, so I basically only shared a link to a slides from a presentation and a link to download the code.
I have afterwards presented the solution at many conferences and it’s also a major part of my “Azure BI in practice” training. This also means I have continuously refined it and now I finally found some time to update the GitHub repo.
But not only update the code, also update the readme file to give an introduction to the various components and concepts, so YOU can get the full overview and get up and running quick and easy! The code is of course totally free to use and modify so it can work best in your solution. I’m very happy to receive feedback if you made some adjustments that could benefit others.
As of now, the code consists of two parts:
- Integration Pipeline (Data Factory/Synapse) Templates
- SQL Database Project
The goal is to make a complete Data Warehouse solution that you can run for around $100 per month. Estimated based on a 3 hour nightly full load batch processing – handling 10’s of millions of rows in multiple fact tables. Need to update more often or handle more data? Then it’s “just” a matter of scaling the SQL Database to a bigger tier which is built into the solution. You can even benchmark the performance of different pricing tiers to see, what’s best in your case.
Sounds interesting? Then go to the GitHub repo to read the full description and download the code!
