Append and Merge to combine data from multiple data source in Power Platform dataflows

Let us continue with our previous post where we loaded the data from on-prem SQL DB to CDS using dataflows.

Load data from SQL On-Premise to CDS (Common Data Service) using Power Platform dataflows

Now suppose we have another table (or any other data source) having the contact details, which we would like to append/merge along with our previous data source.

For simplicity, here we have created a copy of the existing table and added a new record with different values.

Append and Merge to combine data from multiple data source in Power Platform dataflows

Edit the existing data flow created earlier.

Click on Get data and select the SQL Server database.

*The other data sources that are available.

Specify the connection details to connect to the database.

Select the new table that we had created.

Next click on Append queries, select Append queries as new, to create a new query.

https://c0.pubmine.com/sf/0.0.3/html/safeframe.htmlREPORT THIS AD

Specify the primary table and table to append.

The new Query by default named Append will have the data from both the tables combined i.e. appended, which can then be used to load data to the existing contact entity or load to a new entity.

Specify the mapping in the data flow that completes the configuration of the dataflow.

That was about Append queries, now let us take another simple example to see Merge queries in action.

Again let us create one more table that has additional detail (e.g. Mobile Number) for the contact records. Just for simplicity, we are creating a table else it could be any of the supported data sources.

Let us edit the dataflow created earlier, and select Get data to get the data from the SQL Server database.

Here we have selected the new table Contact Details.

This time select Merge queries.

Specify the left table and right table for merge and the join type.

Select the join keys column.

Here we have opted for Inner Join.

The result includes the 1 matching row.

Expand the MyContactDetails column to populate the MobileNumber data.

https://c0.pubmine.com/sf/0.0.3/html/safeframe.htmlREPORT THIS AD

Now with the data transformed, click on Next and define the mapping using the new merged query.

After a successful refresh, we can see the mobile phone field updated.

Let us update the value again for the mobile number field and run the refresh.

We can see the value updated for the CDS contact entity.

If the tables are already related using foreign key relationship for e.g. Contact and Contact Details, the Get data will load the related table, without any need of merge queries.