{"id":72281,"date":"2017-08-18T16:25:58","date_gmt":"2017-08-18T16:25:58","guid":{"rendered":"https:\/\/www.red-gate.com\/simple-talk\/?p=72281"},"modified":"2021-02-23T20:27:14","modified_gmt":"2021-02-23T20:27:14","slug":"azure-resource-manager-arm-templates","status":"publish","type":"post","link":"https:\/\/www.red-gate.com\/simple-talk\/cloud\/infrastructure-as-a-service\/azure-resource-manager-arm-templates\/","title":{"rendered":"Azure Resource Manager (ARM) Templates"},"content":{"rendered":"<h2>What is ARM and what are ARM Templates?<\/h2>\n<p>Azure is managed using an API: Originally it was managed using the Azure Service Management API or ASM which control deployments of what is termed \u201cClassic\u201d. This was replaced by the Azure Resource Manager or ARM API. The resources that the ARM API manages are objects in Azure such as network cards, virtual machines, hosted databases.<\/p>\n<p>The main benefits of the ARM API are that you can deploy several resources together in a single unit and that the deployments are idempotent, in that the user declares the type of resource, what name to use and which properties it should have; the ARM API will then either create a new object that matches those details or change an existing object which has the same name and type to have the same properties.<\/p>\n<p>ARM Templates are a way to declare the objects you want, the types, names and properties in a JSON file which can be checked into source control and managed like any other code file. ARM Templates are what really gives us the ability to roll out Azure \u201cInfrastructure as code\u201d.<\/p>\n<h2>What can ARM templates do<\/h2>\n<p>An ARM template can either contain the contents of an entire resource group or it can contain one or more resources from a resource group. When a template is deployed, you have the option of either using \u2018complete\u2019 or \u2018incremental\u2019 mode.<\/p>\n<p>The \u2018complete\u2019 mode deletes any objects that do not appear in the template and the resource group you are deploying to. In this scenario, what you get is the ability to know that whenever you deploy you will be in exactly the same state.<\/p>\n<p>The \u2018incremental\u2019 deployment uses the template to add additional resources to an existing resource group. The benefit of this is that you don\u2019t lose any infrastructure that is missing from the template but the downside is that you will have to clear up any old resources some other way.<\/p>\n<p>The ideal deployment is \u2018complete\u2019 but it does mean that you need to have a good automated deployment pipeline with at least one test environment where you can validate that the template doesn\u2019t rip the heart out of your beautiful production environment.<\/p>\n<h2>What don\u2019t they do<\/h2>\n<p>The ARM API deploys resources to Azure, but doesn\u2019t deploy code onto those resources. For example you can use ARM to deploy a virtual machine with SQL Server already installed but you can\u2019t use ARM to deploy a database from an SSDT DacPac.<\/p>\n<p>To save time when designing solutions, it is important to understand that ARM API is used simply for resources and we need to use some other technology such as DSC or PowerShell to manage the deployments onto the infrastructure once it is deployed.<\/p>\n<h2>How are they tested<\/h2>\n<h3>Functional testing<\/h3>\n<p>The actual ARM templates are JSON, which is essentially a block of text that is designed to be read by a machine rather than being simple to read for a human. Formatting does help but it is still a single block of text that doesn\u2019t have an actual testing framework, so testing really comes down to performing a deployment and seeing what it creates. If used as part of a wider suite of tests, then the test process should be:<\/p>\n<ul>\n<li>Deploy to a test environment, possibly a dev\/test subscription in Azure<\/li>\n<li>Deploy code and application tests<\/li>\n<li>Execute tests<\/li>\n<li>Report results<\/li>\n<\/ul>\n<p>If all the tests return \u2018success\u2019 then the template is, by definition, valid.<\/p>\n<h3>Basic testing<\/h3>\n<p>Apart from the functional testing, it is possible to validate whether a template is actually deployable to the resource group you want to deploy. If you use the PowerShell function \u201cTest-AzureRmResourceGroupDeployment\u201d, it will take your template, parse it and check that it is syntactically correct as well as validating that you meet requirements such as not hitting quote limits before you deploy.<\/p>\n<p>This is really useful, because the ARM template is not compiled. This prevents you from using a \u201cbuild\u201d step in the build process that you could otherwise use to make sure that the ARM template is even deployable. Furthermore, it wouldn\u2019t be ideal in the \u201ccomplete\u201d type of deployments if you deleted various resources and then failed to create all the new resources that you wanted because you had hit your quota limits in Azure.<\/p>\n<h2>ARM template execution<\/h2>\n<p>There are two important concepts to understand when using ARM templates. The first is that the ARM REST API is the part that actually does the \u201cheavy lifting\u201d. It is this that provides the idempotency of the whole process. The REST API is well documented, updated regularly and available on either the Microsoft docs site or on github if you felt like contributing:<\/p>\n<p>Azure REST API Reference &#8211; <a href=\"https:\/\/docs.microsoft.com\/en-us\/rest\/api\/\">https:\/\/docs.microsoft.com\/en-us\/rest\/api\/<\/a><\/p>\n<p>Azure\/azure-rest-api-specs &#8211; <a href=\"https:\/\/github.com\/Azure\/azure-rest-api-specs\">https:\/\/github.com\/Azure\/azure-rest-api-specs<\/a><\/p>\n<p>There is one set of REST APIs called \u201cResource Management\u201d which is where you send an ARM template. The REST API takes the template and:<\/p>\n<ul>\n<li>Parses the JSON<\/li>\n<li>Fills in any parameters that are passed in<\/li>\n<li>Executes any ARM template functions<\/li>\n<li>Calls the REST API of whatever type of resource that needs to be created to create it<\/li>\n<\/ul>\n<p>If we look at this simple example of an ARM template which will deploy a single storage account:<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" width=\"967\" height=\"858\" class=\"wp-image-72282\" src=\"https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/2017\/08\/word-image-97.png\" \/><\/p>\n<p>The process that the REST API goes through is to read the parameter \u201c<strong>storageAccountType<\/strong>\u201d from the parameters that are also passed in at the same time as the deployment. Because the \u201c<strong>storageAccountType<\/strong>\u201d parameter also specifies what the \u201c<strong>allowedValues<\/strong>\u201d are the parameter is validated against the list of <strong>allowedValues<\/strong>. If the parameter passed in has a typo or other mistake then the deployment is cancelled at this point. If you omit the \u201c<strong>allowedValues<\/strong>\u201d then this check will not happen.<\/p>\n<p>The REST API then creates the variable <strong>diagStorageAccountName<\/strong> and the value of the variable is the result of the ARM template functions <strong>concat<\/strong> which concatenates the string \u2018diags\u2019 with the output of <strong>uniqueString(resourcegroup().id<\/strong>) which will create a unique string for that resource group. The function <strong>uniqueString<\/strong> is passed the id of the resource group to use as a base string for <strong>uniqueString<\/strong> so it will be unique per resource group. If you need a second string to be unique you would need to pass another string to hash or omit the base string. The function <strong>uniqueString<\/strong> is best used with a base string so subsequent deployments do not create new objects each time.<\/p>\n<p>The REST API then uses the resources section to call the resource specific APIs. There is only one resource to create in this example and it is of type \u201cMicrosoft.Storage\/storageAccounts\u201d. The type in the ARM template maps directly to another of the REST <em>APIs<\/em> types which are documented:<\/p>\n<ul>\n<li><a href=\"https:\/\/docs.microsoft.com\/en-us\/rest\/api\/storagerp\/storageaccounts#StorageAccounts_Create\">https:\/\/docs.microsoft.com\/en-us\/rest\/api\/storagerp\/storageaccounts#StorageAccounts_Create<\/a><\/li>\n<\/ul>\n<p><img loading=\"lazy\" decoding=\"async\" width=\"1227\" height=\"145\" class=\"wp-image-72283\" src=\"https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/2017\/08\/word-image-98.png\" \/><\/p>\n<p>The <strong>apiVersion<\/strong> tells the resource manager which version of the API to call, the properties you can set and also the behaviour of the API can and does change between versions so it is important you get the correct API version. The API version is specified for each resource so you can deploy two things of the same type (storage account for example) but deploy them using different versions of the REST API.<\/p>\n<p>The location is determined from the function <strong>resourceGroup<\/strong>() which returns a list of properties, one of which is location.<\/p>\n<p>The value that is set for \u201c<strong>sku<\/strong>\u201d is resolved from the parameter \u201c<strong>storageAccountName<\/strong>\u201d. Any value in the JSON template that is surrounded by [ and ] is evaluated as code. The full list of template functions that can be used is documented <a href=\"https:\/\/docs.microsoft.com\/en-us\/azure\/azure-resource-manager\/resource-group-template-functions\">https:\/\/docs.microsoft.com\/en-us\/azure\/azure-resource-manager\/resource-group-template-functions<\/a><\/p>\n<p>The resource manager then creates the JSON which is going to be sent to the <strong>storageAccount<\/strong> REST API which will look like:<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" width=\"475\" height=\"259\" class=\"wp-image-72284\" src=\"https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/2017\/08\/word-image-99.png\" \/><\/p>\n<p>This JSON is then sent to the following URI as a PUT request:<\/p>\n<p>\/subscriptions\/{subscriptionId}\/resourceGroups\/{resourceGroupName}\/Microsoft.Storage\/storageAccounts\/diags{someUniqueStringCreatedAtRuntime}?api-version=2016-01-01<\/p>\n<p>The <strong>storageAccounts<\/strong> REST API is then responsible for checking whether there is a storage account with the name we have. If there is not then it creates a new one and makes sure it has the same properties as the ones we pass in. If it already exists then the storage REST API will just ensure the properties are set correctly.<\/p>\n<p>When the <strong>storageAccount<\/strong> PUT request happens, if there is something it cannot do such as create an account with the same name as an existing one in another resource group or subscription (maybe even created a different person or organisation) then it will fail. There are also some changes that it finds impossible such as the compute API changing the base image a virtual machine was created for. You will sometimes need to delete resources and start again for some changes.<\/p>\n<p>When the PUT request finishes, the API returns a JSON document which contains the definition of the object that it just created so some things that can\u2019t be known at build time are available. This is useful for cases where you create something like a storage account which creates the access keys when it is created. You could, in your ARM template, reference the key and use that in other objects that need a storage account name and key.<\/p>\n<p>The properties are returned whether or not the object already existed, so in that way you know that whether it is the first time that the resource was created, or the hundredth time it was checked and already existed, you can still deploy downstream dependent resources.<\/p>\n<h2>What else can we do in ARM templates?<\/h2>\n<h3>Defining dependencies<\/h3>\n<p>In some cases, you need to deploy resources in a specific order: For instance, in order to create a virtual machine, you will need to deploy a NIC and for a NIC you might want to deploy a public ip address. If you try to create the NIC and reference a public ip address that doesn\u2019t exist, then the deployment will fail. The answer to this is that dependencies, as well as each resource, can have a \u201c<strong>dependsOn<\/strong>\u201d section that lists all the dependencies that need to be created before the resource can be created. Dependencies are really useful, but should be used with caution because they limit the number of simultaneous things the resource manager can do. The more dependencies you have and the longer the dependency chains, then the longer that deployments will take.<\/p>\n<p>It is also possible to define dependencies by nesting resources, and in that case child dependencies are created after the parent resource is created.<\/p>\n<p>Finally there is a third way to create a dependency which is to use the \u201c<strong>referenc<\/strong>e\u201d template function. What this does is to get the properties of another resource, the same as in the output to the PUT request above. This enables you to do things such as to use an account key from a storage account which may not be known at build time. If you use a reference to access another resource in the template then an implicit dependency is created for you.<\/p>\n<p>The dependencies are documented by Microsoft:<\/p>\n<p>https:\/\/docs.microsoft.com\/en-us\/azure\/azure-resource-manager\/resource-group-define-dependencies<\/p>\n<h3>Copy Sections<\/h3>\n<p>When we deploy resources, we may want just one, but more likely we will want a number of similar resources such as virtual machines to be created. To help with this, there is the \u201ccopy\u201d section which specified how many copies of a resource we want. This is great, because it means that, if we want 100 virtual machines, we don\u2019t have to define 100 resources in the ARM template. The way it works is that you add a copy block to your resource such as:<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" width=\"870\" height=\"454\" class=\"wp-image-72285\" src=\"https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/2017\/08\/word-image-100.png\" \/><\/p>\n<p>What this says is that we should have 3 storage accounts created. There are two things to note here: the first is that, because we are saying \u2018make 3 copies of this\u2019, we need to make the name unique amongst the three objects. We do this so that, instead of just using the account name variable, we append the <strong>copyIndex<\/strong> which is the id of the copy operation. We do this because, if we pass 1 into the <strong>copyIndex<\/strong> function, it starts at 1 rather than 0 so we get<strong> accountName1<\/strong>, <strong>accountName2<\/strong> and <strong>accountName3<\/strong>.<\/p>\n<p>The second thing is that the copy block has a name. This is really useful because we can use it in a <strong>dependsOn<\/strong> block and the dependent resources will not be created until all three of the storage accounts have been created. This is useful in such cases as creating virtual machines where you could get the first machine taking longer to deploy than the last. If you didn\u2019t have the ability to reference by copy name you would need to add a <strong>dependsOn<\/strong> to all the resources individually which would get messy quickly.<\/p>\n<h3>Conditionals<\/h3>\n<p>Conditionals are a recent addition to ARM templates. They specify whether or not to deploy a specific resource. This can be useful when handling different environments where, in development, you might only want one virtual machine without a load balancer but in production you might want five machines with a load balancer. To use the same template you could pass the number in as a parameter but to choose whether or not to deploy the load balancer you can return <strong>false<\/strong> in the <strong>condition<\/strong> property.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" width=\"844\" height=\"479\" class=\"wp-image-72286\" src=\"https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/2017\/08\/word-image-101.png\" \/><\/p>\n<p>In this example we have a \u201c<strong>condition<\/strong>\u201d property that checks whether a parameter is set to \u201cyes\u201d or not.<\/p>\n<p>Conditionals should definitely be used with caution and I would suggest having at least one environment before production where the entire template is deployed. To be fair, they fulfil a need that was missing and which previously involved passing whole blobs of JSON in as parameters.<\/p>\n<h2>How do you get started?<\/h2>\n<p>There are a couple of approaches, the easiest of which is to use the quick start templates created by Microsoft which give examples of how to use most resources. The quick start templates are available here:<\/p>\n<p><a href=\"https:\/\/azure.microsoft.com\/en-gb\/resources\/templates\/\">https:\/\/azure.microsoft.com\/en-gb\/resources\/templates\/<\/a><\/p>\n<p>The second approach is to deploy some resources: In the azure portal, there is a button called \u201cAutomation Script\u201d which will generate the ARM template to deploy the entire resource group. This always creates a script for the entire resource group even if you click the \u201cAutomation Script\u201d on one particular resource.<\/p>\n<p>Not every type of resource can be generated today, although it seems as if Microsoft keep increasing the scope of the resources that they generate ARM templates for. It might be that you just need to wait a few weeks.<\/p>\n<p>The automation script doesn\u2019t, in my opinion, generate the ideal templates. The names are odd and even though they put names behind parameters they use the name of the resource in the parameter names so it is hard to read and edit the scripts. I would say that they are best used to show how a resource is configured and just take the parts from the script that you need to re-use and manually create the other parts of the template or use a company-shared template to create the ARM template.<\/p>\n<p>The automation also generates the scripts that can be used by different clients such as PowerShell, Bash and C#. This gives you the code you need to send your ARM template to the ARM REST API for processing. Again, these are quite verbose and you don\u2019t need everything. For example, if you are using PowerShell, then all you really need to do is to create the resource group itself if it doesn\u2019t exist, and then call \u201c<strong>New-AzureRmResourceGroupDeployment<\/strong>\u201d from the <strong>AzureRM<\/strong> module.<\/p>\n<h2>Documentation<\/h2>\n<p>The ARM templates capabilities themselves as opposed to the actual resources are documented:<\/p>\n<p><a href=\"https:\/\/docs.microsoft.com\/en-us\/azure\/azure-resource-manager\/\">https:\/\/docs.microsoft.com\/en-us\/azure\/azure-resource-manager\/<\/a><\/p>\n<p>The resources and their properties are documented:<\/p>\n<p><a href=\"https:\/\/docs.microsoft.com\/en-us\/azure\/templates\/\">https:\/\/docs.microsoft.com\/en-us\/azure\/templates\/<\/a><\/p>\n<p>This only documents the latest version of an API. For the exact details of what can and cannot be used in an ARM template for a resource the JSON schemas are available:<\/p>\n<p><a href=\"https:\/\/github.com\/Azure\/azure-resource-manager-schemas\">https:\/\/github.com\/Azure\/azure-resource-manager-schemas<\/a><\/p>\n<p>Although these are not as easy to follow as the documentation page, they do give the exact details of what can and cannot be deployed.<\/p>\n<p>Sometimes the REST <em>APIs<\/em> are updated before the documentation so, if you are using something that is bleeding edge, then you can be lucky if you look at the documentation for the REST API, see what properties you can set and then just put them in the ARM template.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>If you need a way of deploying  infrastructure-as-code to Azure, then Azure Resource Manager (ARM) Templates are the obvious way of doing it simply and repeatedly.  They define the objects you want, their types, names and properties in a JSON file which can be understood by the ARM API. Ed Elliott takes the mystery out of a simple means of  specifying your Azure environment, whether it is a VM with blockchain software, SQL Server  or a Web App on Linux with PostgreSQL<br \/>\n&hellip;<\/p>\n","protected":false},"author":59464,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[137091,45],"tags":[],"coauthors":[11314],"class_list":["post-72281","post","type-post","status-publish","format-standard","hentry","category-azure","category-infrastructure-as-a-service"],"acf":[],"_links":{"self":[{"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/posts\/72281","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/users\/59464"}],"replies":[{"embeddable":true,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/comments?post=72281"}],"version-history":[{"count":4,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/posts\/72281\/revisions"}],"predecessor-version":[{"id":72775,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/posts\/72281\/revisions\/72775"}],"wp:attachment":[{"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/media?parent=72281"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/categories?post=72281"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/tags?post=72281"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/coauthors?post=72281"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}