Data Factory Solution Templates
Data Factory has brought a quicker path to solutions by introducing standard pipeline templates that can be created through a simple GUI wizard. This is an exceptional feature to help streamline time to value from ingestion to decision making.
Current Template List:
Bulk Copy from Database
Copy Data from Amazon S3 to Azure Data Lake Store
Copy Data from Google BigQuery to Azure Data Lake Store
Copy Data from HDFS to Azure Data Lake Store
Copy Data from Netezzato Azure Data Lake Store
Copy Data from OnPrem SQLServer to SQL Data Warehouse
Copy Data from Oracle to SQL Data Warehouse
Copy Multiple Files from Containers between File Stores
Delta Copy from Database
ETL with Azure Databricks
Schedule Azure-SSIS Integration Runtime to Execute SSIS Packages
Transform Data using on-demand HDInsight
Access the #Microsoft documentation here: https://docs.microsoft.com/en-us/azure/data-factory/solution-templates-introduction
Quick Overview of how to use Delta Copy from Database
It is easy to get started using a template. You can access the template menu from the main page in Data Factory by clicking the Create Pipeline from Template button as seen below.
2. Select Template Type
The current available templates that are listed above are seen below with the activity icon in front and the source and sink dataset logos following. Choose the Delta Copy from Database template.
3. Create Linked Services and Datasets.
The template wizard will walk you through filling in the activities with the necessary linked services and datasets needed for the template, and for the Delta copy template, the output can be seen below.
This Delta methodology is one way to manage delta copies, by identifying a source modify timestamp column and capturing the latest value in a sql table.
Walk through the full details here: https://docs.microsoft.com/en-us/azure/data-factory/solution-template-delta-copy-with-control-table
Though this is an enabler for new ETL and ELT developers it should not be looked at as an Ingestion, ETL, or ELT architecture. In some cases having a stand alone pipeline per unit of work is ideal. However, with the introduction of ELT practices, a unit of work becomes more about the source system type and not necessarily the source system object.