2187-just_azure.svg

By now you should start to see the advantages of deploying your application to a Platform-as-a-Service environment like Cloud Services. You just take the code, upload it and the rest (like setting up the virtual machines) is done for you. But depending on the complexity of your project this might not be enough. You might need to customize the Web/Worker Role Instance before it starts. For example you may need to configure IIS, install an IIS extension using the Web Platform Installer, install a monitoring agent, join a domain, etc.

In a traditional environment you would simply start a Remote Desktop connection or execute some Remote PowerShell scripts to customize your environment, but that won’t work here. Remember that we’re working with a “disposable infrastructure”. Machines can be added or removed at any time, we might need to delete a deployment (which also removes the instances within that deployment) and then redeploy it. Or if something goes wrong and a few machines become unhealthy the Fabric Controller might even need to deploy the application to other machines. Oh, and remember that the changes made to our instances are not even persisted.

Luckily the Cloud Services platform allows you to control the lifetime of an instance: you’re able to do something when the instance starts, when it’s running and even when before the instance is being shut down. If you’re doing work on an instance (maybe reconfiguring it or updating something) you can even communicate with the load balancer to let it know that the instance is busy.

Tapping into the lifetime of the instances means that our customizations, whatever they might be, are included in the Service Package and are part of the deployment, allowing us to make sure that every instance is configured in exactly the same way.

Start, Run, Stop

When creating Web and Worker Role projects you might have noticed a file being added to your project: WebRole.cs or WorkerRole.cs. These classes inherit from the RoleEntryPoint class and runs in a dedicated process (WaIISHost.exe and WaWorkerHost.exe). During the lifecycle of the instance the Fabric Controller will call 3 different methods on the RoleEntryPoint:

  • OnStart: the OnStart method will be called when your instance starts and will allow you to run some custom code to configure anything you would need before doing some real work. This could go from configure IIS, to configuring the Azure Diagnostics Monitoring (for Azure Diagnostics 1.0) or even subscribing to external events such as a configuration change. An example of this can be found in the previous article of this series under “Updating the configuration settings” or in the sample design patterns for cloud applications (External Configuration Storage Pattern).
    While this method is being called the status of the machine will be “Busy”. For Web Roles this means that the load balancer will not be sending requests to this instance.
  • Run: the Run method will be where the real work is happing, especially for a Worker Role. This is where you’ll be processing messages from a queue, execute some code that needs to run every X minutes/hours/days/, start an external process, etc. Make sure that this method never returns, since this will cause the Instance to recycle. You would typically have an infinite loop running here to make sure the method never returns.

    In the case of a Web Role you don’t need any code in this method. The actual application that you’ll be running is the web application which has been deployed to IIS.

    While this method is being called the status of the machine will be “Ready”, allowing the load balancer to send requests to this instance.

  • OnStop: In case of a graceful shutdown, this is the method that will give you a last chance to finish something you need to do on this specific instance. This is where you might want to stop your worker from processing additional messages from your queue, ship some data to durable storage (like Blob Storage), or transfer the remaining log files. But keep in mind that you only get 5 minutes to do so! After the five minutes if the OnStop code hasn’t returned the platform will continue with shutting the instance down.
    While this method is being called the status of the machine will be “Busy”.

Figure 1 shows an example of a Worker Role which logs every call to OnStart/Run/OnStop to table storage (using the Azure SDK version 2.5).

Figure 1: Writing every event (OnStart/Run/OnStop) to Table Storage

After running this code in the emulator we’ll be able to follow the lifecycle of the Role Instance as you can see in Figure 2.

2213-CloudService-4-Figure2-620.png

Figure 2: Tracking the lifecycle in Table Storage

Startup Tasks

The OnStart seems like the perfect way to configure your Instance, and in many scenarios it is. But there are times where you’ll need to run a script or a process and doing this from within your code might not be the easiest way to do it.

Before the OnStart method is called you’re able to run “Startup Tasks”, which can be scripts or processes you can execute to configure your Instance. Depending on what your task will be doing you’ll have different execution contexts and types to choose from.

The execution context defines if your task runs with administrator privileges or not:

  • Limited: runs with the same privileges as the WaWorkerHost.exe/WaIISHost.exe process.
  • Elevated: run with administrator privileges

Anything you would need to do to configure IIS or Windows will typically require you to use an Elevated execution context.

There are also different Startup Task types available allowing you to choose when they should be executed and how they impact the way the Instances starts and stops:

  • Simple: the system will wait for the task to exit before the other tasks are executed. Only when all Simple Startup Tasks have been executed will the system continue by calling the OnStart and Run methods. Keep in mind that your Instance’s status will be Busy while these tasks are executed, so make sure you only run tasks that you need to be complete before the role starts.
  • Background: the system will not wait for these tasks to exit. This could for example be useful if you’re running some monitoring agent on your roles and you want it to be active while the application is running. Anything that you don’t need to have finished before your application can start you could run as a Background Startup Task.
  • Foreground: this is similar to the Background Startup Tasks. The system will not wait for these tasks to complete and will continue with the OnStart/Run methods. The major difference is the behavior of the instance when the WaIISHost.exe/WaWorkerHost.exe process stops. This would typically cause the instance to recycle (restart), but if you have a Foreground task running that will not be the case. Your instance will keep running as long as the Foreground task is running, even if the WaIISHost.exe/WaWorkerHost.exe stops because of a problem.
    Note: this will not block a graceful shutdown in case of a deployment upgrade, after scaling down, when shutting down the machine, etc.

There is one special requirement when it comes to running scripts as Startup Tasks. You’ll need to save the script without encoding in the ANSI format. Using Visual Studio this can do this using the Advanced Save Options (Figure 3).

2213-CloudService-4-Figure3-620.png

Figure 3: Saving a Startup Task without encoding.

Besides the execution context and the type you can also use environment variables in your Startup Tasks. These can be values you define in the Service Definition file or XPath queries against the Web/Worker Role Schema. These values can be used to find the path to a local resource in which you could store some log files or to verify if you’re running in the emulator or not.

Figure 4 shows the definition of a Startup Task with a custom environment variable I’m using to define the filename of the log file and the XPath queries to find the path to a local resource and some additional information about my instance.

Figure 4: Sample Startup Task definition

Note how I’m referencing the Startup Task to execute in the commandLine attribute. The current working directory for a Startup Task is the bin folder. The easiest way to add a new Startup Task to your role is by right-clicking the role in Visual Studio and choose Add > New Item as indicated in Figure 5. After adding the Startup Task the file will not be available in the bin folder when your role is deployed, but you’ll find it in the bin’s parent folder. That’s why you’ll need to prefix the Startup Task with “..\” when referencing it in the commandLine attribute.

2213-CloudService-4-Figure5-620.png

Figure 5: Adding a new Startup Task

The example in figure 6 shows a pretty complete Startup Task: everything is logged to a local resource, the script is only executed once, an external PowerShell script is executed (for which the output is also logged) and we’ve added some error handling to the script.

Figure 6: Sample Startup Task

Before using startup tasks in your services you might want to take a look at the following whitepaper on MSDN: Best Practices for Startup Tasks.

Lifecycle Overview

Figure 7 shows a high level overview of the Role Lifecycle (a detailed overview can be found on Kevin Williamson’s blog). The gray blocks in the figure only apply to Web Roles.

2213-CloudService-4-Figure7.png

Figure 7: Lifecycle

One word of caution when configuring IIS through startup tasks is the possible race condition with the IIS Configurator (which will configure IIS properly). You might notice issues with your Startup Tasks because IIS might not have been configured yet or the IIS Configurator is still busy. There are different approaches to solve this, but the easiest way to do so is by running your IIS customizations from the OnStart method, where you’ll be sure that IIS Configurator has already finished.

By using the ServerManager class and setting the execution context of your Role to Elevated you can do pretty much anything with IIS from within the OnStart method. Part 2 of this series shows a short example in Figure 10 where we changed the Idle Timeout of the application pools to 2 hours.

Custom Entry Points

Throughout this series we’ve talked about WebRole.cs and WorkerRole.cs files and the OnStart/Run/OnStop methods that come from the base RoleEntryPoint class. The packaging process will create file (__entrypoint.txt) in the package that points to the assembly which contains the entry point. This is the default behavior.

2213-CloudService-4-Figure8-620.png

Figure 8: Default Role Entry Point

But the Service Definition file also allows you to customize this by specifying a custom entry point:

  • NetFxEntryPoint: allows you to point to an assembly which contains the entry point
  • ProgramEntryPoint: allows you to start an external process as an entry point

Figure 9 shows an example of starting a Node.js server as a custom entry point. The advantage of starting this process as an entry point as opposed to using startup tasks or “Process.Start” on the Run method of your Worker Role is that you get monitoring and recovery for free. The process being started as an entry point is basically your Worker Role process, so if this process stops your instance will recycle.

Figure 9: Node.js entry point

The way infrastructure should be

Even though the Web and Worker Role instances are not managed by us, we still have lots of flexibility to configure the machines the way we want it. Since nothing is persisted we’re forced to script everything, we can’t rely on changes made through Remote Desktop. And this is a good thing! Gone are the days that only a few people in the team know how to properly deploy and configure the application. Deployment is handled by the Azure platform and the custom configuration can be found in Startup Task or the OnStart method.

If we scale out or if the Fabric Controller needs to deploy new instances to replace unhealthy instances the new instances will be configured in exactly the same way as the instances that are already running. There’s no room for error if we script everything.

And as always, the samples we covered in this article are available on GitHub: https://github.com/justazure/JustAzure.CloudServices , or you can download it from the link below.

Our instances are now ready, so in the next part of this series we’ll cover the deployment, debugging and monitoring aspect of Cloud Services. Don’t miss it!