{"id":105089,"date":"2025-01-15T19:44:51","date_gmt":"2025-01-15T19:44:51","guid":{"rendered":"https:\/\/www.red-gate.com\/simple-talk\/?p=105089"},"modified":"2025-01-15T19:57:41","modified_gmt":"2025-01-15T19:57:41","slug":"fabric-copy-job-whats-new-in-it","status":"publish","type":"post","link":"https:\/\/www.red-gate.com\/simple-talk\/blogs\/fabric-copy-job-whats-new-in-it\/","title":{"rendered":"Fabric Copy Job: What&#8217;s new in it"},"content":{"rendered":"<p>At first sight, the new Copy Job may seem only one more redundancy: Why do we need this?<\/p>\n<p>We have Data Pipelines for the Copy Activity and also a Copy Assistant to help us configure it. What&#8217;s new about the Copy Job?<\/p>\n<p><strong>Spoiler:<\/strong> It&#8217;s much more than a new UI<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" width=\"922\" height=\"224\" class=\"wp-image-105090\" src=\"https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/2025\/01\/word-image-105089-1.png\" srcset=\"https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/2025\/01\/word-image-105089-1.png 922w, https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/2025\/01\/word-image-105089-1-300x73.png 300w, https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/2025\/01\/word-image-105089-1-768x187.png 768w\" sizes=\"auto, (max-width: 922px) 100vw, 922px\" \/><\/p>\n<h2>The new Copy Job Object and UI<\/h2>\n<p>The Copy Job is exactly what the name says: a &#8220;dedicated data pipeline&#8221; intended only to execute a copy activity. We schedule it exactly as a data pipeline.<\/p>\n<p>The new UI is cute, you finish the configuration with the job already scheduled for every 15 minutes by default.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" width=\"1280\" height=\"384\" class=\"wp-image-105091\" src=\"https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/2025\/01\/a-screenshot-of-a-computer-description-automatica-5.png\" alt=\"A screenshot of a computer\n\nDescription automatically generated\" srcset=\"https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/2025\/01\/a-screenshot-of-a-computer-description-automatica-5.png 1280w, https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/2025\/01\/a-screenshot-of-a-computer-description-automatica-5-300x90.png 300w, https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/2025\/01\/a-screenshot-of-a-computer-description-automatica-5-1024x307.png 1024w, https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/2025\/01\/a-screenshot-of-a-computer-description-automatica-5-768x230.png 768w\" sizes=\"auto, (max-width: 1280px) 100vw, 1280px\" \/><\/p>\n<p>These are only the small details.<\/p>\n<h2>Incremental Load with the Copy Job<\/h2>\n<p>In my opinion, this is the great advantage of this new feature: It builds an incremental load for you.<\/p>\n<p>The configuration for incremental load requires an incremental column. A date\/time column which will allow the job to identify changed records.<\/p>\n<p>The source model is the responsible for updating the date\/time column every time a record is updated. After that, the job manages all the work.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" width=\"1280\" height=\"390\" class=\"wp-image-105092\" src=\"https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/2025\/01\/a-screenshot-of-a-computer-description-automatica-6.png\" alt=\"A screenshot of a computer\n\nDescription automatically generated\" srcset=\"https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/2025\/01\/a-screenshot-of-a-computer-description-automatica-6.png 1280w, https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/2025\/01\/a-screenshot-of-a-computer-description-automatica-6-300x91.png 300w, https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/2025\/01\/a-screenshot-of-a-computer-description-automatica-6-1024x312.png 1024w, https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/2025\/01\/a-screenshot-of-a-computer-description-automatica-6-768x234.png 768w\" sizes=\"auto, (max-width: 1280px) 100vw, 1280px\" \/><\/p>\n<p>The job will make a first execution with a full load, and continue with incremental load every 15 minutes, by default.<\/p>\n<h2>Copy Job and Other Options<\/h2>\n<p>How this new feature compare with the other options?<\/p>\n<p>We have the <a href=\"https:\/\/www.youtube.com\/watch?v=-MTaVYewtY8\" target=\"_self\" rel=\"noopener\">CDC Ingestion from Azure SQL<\/a> and <a href=\"https:\/\/www.red-gate.com\/simple-talk\/databases\/sql-server\/bi-sql-server\/fabric-and-azure-sql-in-a-few-clicks-mirroring-a-database\/\" target=\"_self\" rel=\"noopener\">Mirroring from Azure SQL<\/a>, how the new job compares with them?<\/p>\n<p>Let&#8217;s enumerate some differences:<\/p>\n<ul>\n<li>It is not real time, it works by schedule<\/li>\n<li>It requires a special field in the source for the incremental work. CDC doesn&#8217;t<\/li>\n<li>The Copy Job is not limited to Azure SQL, the two previous options are<\/li>\n<\/ul>\n<p>You can also take a look on <a href=\"https:\/\/www.youtube.com\/watch?v=_QYf8lRNfCU&amp;list=PLNbt9tnNIlQ5TB-itSbSdYd55-2F1iuMK\">Fabric Monday 58: Copy Job and Incremental Load<\/a><\/p>\n<h2>Summary<\/h2>\n<p>Everyday Microsoft makes it easier to put data into Fabric<\/p>\n","protected":false},"excerpt":{"rendered":"<p>At first sight, the new Copy Job may seem only one more redundancy: Why do we need this? We have Data Pipelines for the Copy Activity and also a Copy Assistant to help us configure it. What&#8217;s new about the Copy Job? Spoiler: It&#8217;s much more than a new UI The new Copy Job Object&#8230;&hellip;<\/p>\n","protected":false},"author":50808,"featured_media":105097,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[2,159164],"tags":[159242,145486,159243,158997],"coauthors":[6810],"class_list":["post-105089","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-blogs","category-microsoft-fabric","tag-copy-job","tag-data-factory","tag-incremental-load","tag-microsoft-fabric"],"acf":[],"_links":{"self":[{"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/posts\/105089","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/users\/50808"}],"replies":[{"embeddable":true,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/comments?post=105089"}],"version-history":[{"count":2,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/posts\/105089\/revisions"}],"predecessor-version":[{"id":105095,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/posts\/105089\/revisions\/105095"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/media\/105097"}],"wp:attachment":[{"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/media?parent=105089"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/categories?post=105089"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/tags?post=105089"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/coauthors?post=105089"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}