Skip to content

The dataflow evolution: From ETL to ISPS

For numerous years, the foundation of our Data Warehouse architectures has revolved around the traditional ETL dataflow process, consisting of the following stages:

  • Extract
  • Transform
  • Load

In this process, data is initially extracted from diverse sources in its raw format, followed by transformations, typically within a SQL database, before finally loading the data into a data mart or a semantic data model.

However, a few years ago, we introduced a new layer in this architecture to better accommodate semi-structured and unstructured data. This innovation coincided with the emergence of data lakes, which prompted a shift to ELT. I’ll argue that we still need the last “Load” step, resulting in a more fitting term, ELTL.

Simultaneously, Microsoft overhauled many of their Data Platform architectures, introducing entirely new labels for the process:

  • Ingest
  • Store
  • Prepare
  • Serve

This prompts a contemplation: should we now refer to it as an ISPS dataflow, or should we maintain the traditional terminology that we have relied upon for years?

Leave a Reply

Your email address will not be published. Required fields are marked *