Microsoft Fabric: Moving Dataflows Gen 2 to different workspaces

Comments 0

Share to social media

Dataflows Gen 2 are not supported by source control and deployment pipelines yet.

When we need to promote a dataflow from Dev to Test and Prod, we need to move the dataflow manually.

However, the dataflows don’t have a Save As feature. It’s possible to edit the JSON from the queries, but it would need to be one by one, so it’s not the best option.

Let’s see the steps needed to move a dataflow from one workspace to another. For a matter of example, let’s consider the dataflow access a lakehouse on the Dev environment. When copied to the Test environment, the new copy needs to be linked to a new dataflow in the Test environment.

Check the steps to copy a Dataflow Gen 2 to another workspace:

  • Open the lakehouse on the Test workspace and copy its URL. You will need it later. The URL contains the Workspace Id (after groups) and the lakehouse Id (after lakehouses)

  • Export the dataflow Gen 2 as a template

A file with a PQT extension will be downloaded

  • On the Test workspace, create a new Dataflow Gen 2
  • Click on the link in the middle of the window to Import the template – the PQT file

Repeat the steps below for each query:

  • Select the Navigation1 step

  • Change the workspace Id on this step using the workspace Id from the captured lakehouse URL

  • Select the Navigation 2 step
  • Change the lakehouse Id using the Id captured from the Test lakehouse URL

  • Adjust the dataflow name and publish

Summary

The process is a bit complex, or at least, more complex than we would expect for a simple promotion from one environment to another.

However, we hope to only need this process during a short period, while we wait the dataflow support in the source control and deployment pipelines.

Load comments

About the author

Dennes Torres

See Profile

Dennes Torres is a Data Platform MVP and Software Architect living in Malta who loves SQL Server and software development and has more than 20 years of experience. Dennes can improve Data Platform Architectures and transform data in knowledge. He moved to Malta after more than 10 years leading devSQL PASS Chapter in Rio de Janeiro and now is a member of the leadership team of MMDPUG PASS Chapter in Malta organizing meetings, events, and webcasts about SQL Server. He is an MCT, MCSE in Data Platforms and BI, with more titles in software development. You can get in touch on his blog https://dennestorres.com or at his work https://dtowersoftware.com