A technical deep-dive by Clive Tong
Now C#5 has arrived .NET Reflector’s technical lead Clive explains how async code works, and why you need it.
Clive’s been leading the effort to make sure .NET Reflector can accurately decompile asynchronous code so that you can start investigating C#5 code right away.
Just understanding how async actually works at the front end is no mean feat, and when you bear in mind that we’re juggling up to 3 different compilers, it’s been a fascinating challenge.
Before we dive in, it’s worth grounding ourselves first. Let’s start with a quick look at what async is designed for, and then we’ll take a deeper dive into the nuts and bolts, and how they apply to us.
Async is designed to solve a problem that every developer has run into when writing a GUI application – GUI locking & freezing. Most windowing libraries avoid the need to take locks by having all of the GUI code run on a single thread, with this thread using some kind of mailbox to prevent asynchronous message arrival.
We obviously want to prevent asynchronous message arrival because, when the string of a TextBox is being updated to “fred”, we don’t want to get a message telling us to update it to “joe” while we are still in the middle of processing the change to “fred”. The key to this enforcement of linear processing is that the GUI is only going to respond to user interaction if the single GUI-owned thread is available to do processing of incoming events. If we need to do a blocking operation, like hitting a web service, and we do it on the GUI-owned thread, then the GUI is going to feel dead, with screen updates and actions not having any effect until the blocking operation finishes.
This is unacceptable, particularly on touch controlled devices, where the feedback has to be quick to be effective. Visiting some sites on my iPhone’s web browser can be painful, as I often touch to cause some kind of action, but then find that Safari is still busy processing the page. Frustratingly, this leaves me in a state where I don’t know if Safari is going to do something when it gets less busy, or whether my earlier interaction has just been ignored. WinRT is going to be designed for touch from day one, and its designers have apparently paid careful attention to existing touch Operating Systems, and learned some valuable lessons. As a result, WinRT requires all operations that are going to take more than 50 milliseconds to offer an asynchronous version (and, in lots of cases, only an asynchronous version), as a means of keeping the user experience of Metro applications slick and intuitive. The question then becomes how to make it easy for developers to use this new functionality.
In the past we’ve been able to defer work to a later point in time by capturing the context as a closure, passing this closure to some object that is responsible for doing the wait, and then having that closure called when the code should continue running. This continuation-passing style makes the code very complicated to read, and makes it very hard to follow the control flow of the code.
Let’s take a look at an example of this using a very simplistic example. Suppose we want to read two strings from the Console, and write their total length in characters to the Console. If at any point a blank line is entered, then we don’t want to read any more data, but we still want to print the total number of characters:
int totalCharacters = 0;
string input = Console.ReadLine();
if (!string.IsNullOrEmpty(input))
{
Console.WriteLine("First string: {0}", input);
totalCharacters += input.Length;
input = Console.ReadLine();
if (!string.IsNullOrEmpty(input))
{
Console.WriteLine("Second string: {0}", input);
totalCharacters += input.Length;
}
}
Console.WriteLine(totalCharacters);
The trouble with this code is that it is blocking the main thread. That doesn’t matter in this case, but it would if it were an operation on the GUI. The thread also sits inside Console.ReadLine, consuming the resources that a thread requires (such as 1mb of stack space), without doing any useful work. If this was some kind of (web) server application, then this would be wasteful and would inhibit scalability. Ideally, we’d really like to record the state of the program, but then pass the thread back to the runtime, only grabbing it later when we have more work to do.
Let’s say we were provided with some kind of asynchronous version of Console.ReadLine, such as a function of the following form, which takes responsibility for getting data from the console and then calling an action function with that data when it is available on some available thread:
static void ReadLineAndThenDo(Action<string>action)
{
ThreadPool.QueueUserWorkItem(
delegate
{
string line = Console.ReadLine();
action(line);
});
}
In that situation, it would be possible for the outer function to release the thread it is running on, knowing that the action function is going to be called on a suitable thread when the result is available.
There are two ways that you might implement the new program. You might use the continuation-passing style we briefly mentioned earlier, using lambda expressions to capture the state of the local variables, which we can use for later parts of the computation:
static void Attempt2()
{
int totalCharacters= 0;
ReadLineAndThenDo(
input =>
{
if (!string.IsNullOrEmpty(input))
{
Console.WriteLine("First string: {0}",input);
totalCharacters += input.Length;
ReadLineAndThenDo(
input2 =>
{
if (!string.IsNullOrEmpty(input2))
{
Console.WriteLine(“Second string: {0}”,input2);
totalCharacters += input2.Length;
}
Console.WriteLine(totalCharacters);
});
}
});
}
Here, the change has been localised to our function, but what a change it is! Every time we want to do an asynchronous call, we have had to pass in the rest of the code as a lambda expression (input => …) or (input2 => …). The indentation gets deeper and deeper, and things become very complicated if we have a loop construct of any kind in the code. The logic of the code can become obfuscated in order to fit into the programming language.
However, if we take another look at the very first version of our string reader, we can see that there are distinct sections of code, with a new section starting each time we pass control to a ReadLine. For example, we start in state one, which does the initial setup of the local variables, and we then do a ReadLine and assume we enter state two when this has finished. As we enter state two, we know that there is new input available, and we can process this until we get to the next ReadLine, which starts state three:
/*State 0 */
inttotalCharacters = 0;
string input = Console.ReadLine();
/* State 1 */
if (!string.IsNullOrEmpty(input))
{
Console.WriteLine("First string: {0}",input);
totalCharacters += input.Length;
input = Console.ReadLine();
/* State 2 */
if (!string.IsNullOrEmpty(input))
{
Console.WriteLine("Second string: {0}",input);
totalCharacters += input.Length;
}
}
/* State 3 */
Console.WriteLine(totalCharacters);
This feels very much like what is known as a state machine in Computer Science parlance, which is the second way we can implement this program. We’ve seen such implementations used before, in the compilation of iterator blocks by the C# compiler, so there is already existing compiler technology for implementing such transformations.
We can implement the state machine using a new compiler generated class, Attempt3Machine. This will have a state variable, m_State, recording the state of the machine as one of its fields. We’ll lift the local variables of the method into the fields of the state machine (necessary because they live between the calls into our state machine), and we’ll implement the machine in a way that uses a single method to transition it from its current state to the next state. We call this method MoveNext, in the spirit of iterator blocks implementation.
We’ll now need to change our asynchronous version of ReadLineto interact with our state machine. For the moment, the ReadLinewill read a line from the Console, set this into the m_Input field which corresponds to the local variable input in the original code, and will then force the machine to make a transition:
static void ReadLineAndThenDo2(Attempt3MachinestateMachine)
{
ThreadPool.QueueUserWorkItem(
delegate
{
string line = Console.ReadLine();
stateMachine.m_Input= line;
stateMachine.MoveNext();
});
}
This then leaves us with a state machine with the following implementation:
class Attempt3Machine
{
intm_State = 0;
private int m_TotalCharacters= 0;
public string m_Input;
public void MoveNext()
{
switch (m_State)
{
case 0:
m_State= 1;
ReadLineAndThenDo2(this);
return;
case 1:
if (!string.IsNullOrEmpty(m_Input))
{
Console.WriteLine("First string: {0}",m_Input);
m_TotalCharacters+= m_Input.Length;
m_State= 2;
ReadLineAndThenDo2(this);
return;
}
goto case 3;
case 2:
if (!string.IsNullOrEmpty(m_Input))
{
Console.WriteLine("Second string: {0}",m_Input);
m_TotalCharacters+= m_Input.Length;
}
goto case 3;
case 3:
Console.WriteLine(m_TotalCharacters);
return;
}
}
}
And with a method that uses this of the form:
static void Attempt3()
{
new Attempt3Machine().MoveNext();
}
So where has this got us? Well, the translation between the code above and something that started out like the following is fairly mechanical:
int totalCharacters = 0;
string input = await Console.ReadLine();
if (!string.IsNullOrEmpty(input))
{
Console.WriteLine("First string: {0}", input);
totalCharacters += input.Length;
input = await Console.ReadLine();
if (!string.IsNullOrEmpty(input))
{
Console.WriteLine("Second string: {0}", input);
totalCharacters += input.Length;
}
}
Console.WriteLine(totalCharacters);
The position of the Await expressions tells us where we need to have new states for the state machine, and we simply need to push the code in-between into the relevant state transition blocks.
By using the Await in the above code, we have kept the code in a state where it is easy to follow the control flow, even though the method stops running at the point where the await is called.
There are, of course, several loose ends to tidy up. First, it looks easy in the above code to split the code into the transitions on a line by line basis, but in reality you can put multiple Await expressions in a single statement:
int x = await f() + await g();
This means we may need a new field location to store the result of the first Await while we are waiting for the second Await to complete. We could use several locations, or just have a single location that stores the intermediate value typed as an object, and then cast as necessary when wanting to use this value. The current semantics of await only allow a single outstanding call at any time, so we need to wait for f() to finish beforeg() is launched.
If we have an async function, then the caller needs to have something that they can use to get the value when it is finally calculated. .NET 4 included the rather brilliant Task<>abstraction, which reflects a computation that may still be running. A Task<int> object, for example, represents a calculation that may eventually return an integer. Using its Result property, a client can fish out the result value, blocking until it is available, and find out details of any exceptions that were thrown. Tasks support a number of very powerful operators, such as a ContinueWith (which allows you to schedule the execution of one task when some other task completes), and a means of waiting for one or many tasks to finish. Tasks also support a model of synchronous cancellation, with the user function being expected to regularly check for a demand for cancellation, and throwing an exception if this is required. As a result, any method that uses await must either return void or Task<…>.
Threading always adds complication to any design, and our current example doesn’t really have a good threading model. Some operations need to be called on certain threads, so our code would really need to transition back onto one of these threads before it is called, probably by using the Post operation of the synchronization context that was active when we launched the asynchronous call. .NET also has a notion of an execution context which may need to flow across to any worker threads that are launched. We may need to capture these values when we launch a call, and then get back to the correct state before we restart in the next transition.
Exceptions also need to be handled and stored away in the Task object associated with the current method, so that callers can later get the correct details of what went wrong. If the method finishes successfully, we’re also going to need to store the result away in the task, or at least mark the task as finished if there is no return value. This will allow callers to wait on our method finishing.
There are also several missing pieces in the story of the communication between the state machine (when it fires off the asynchronous work) and the object that is going to do the work. The example above hacked this, by having the asynchronous method we were calling (DoReadLineAndThenDo) know lots of detail about the caller, such as the field in which to write the return value. This code is obviously too specific, and is purely for demonstration purposes. We get around this by adding indirection, having the asynchronous ReadLinereturn a Task, and then hook an OnCompletion action to the returned task which causes the state machine to move to the next state.
Asynchronous code is clearly incredibly useful, and remarkably cunning in its implementation, which means that we’ve had to work twice as hard to unravel it at the other end. Hopefully you can see how quickly async code can start to become convoluted, and how backtracking the various calls and state transitions in a real application poses a challenge.
Async is implemented using a compiler transform to implement code as a state machine, though this requires a few support classes in the runtime library, and that a set of methods need to be implemented on types that support asynchronous calls following the Await pattern. No changes are required to the CLR, though some support has been added inside mscorlib, and extra methods have been added to the Task<…> type to support the calls of the Awaiter pattern.
Async really seems to have hit the sweet spot for a technology that allows you to fairly transparently run non-blocking code on a single thread, and user code looks very much like it would for a straightforward blocking implementation. In fact, you can almost write it that way first, and then change it into non-blocking code by just changing the method’s return type to Type<…>, marking it with async, and then using await and asynchronous versions of methods that you call.
Naturally, it isn’t quite that simple in practice. For example, having a function return too early might not interact well with constructs such as locks and exception handlers, and potentially offers the chance of re-entrancy in cases that were previously safe. Nevertheless, async is not as opaque as perhaps it first seems, and you could learn a huge amount by using .NET Reflector to start investigating C#5 as soon as possible.
