A colleague of mine once asked about cancellation tokens in AWS. This question got me thinking about this problem and got me curious on whether there is any support. Turns out it is an interesting topic with lots of pitfalls.
If you don’t know about cancellations tokens, they are used in C# are used to signal that a task or operation should be cancelled, and the desired outcome is that code stops any further execution. You create a cancellation token by using a CancellationTokenSource object, which manages cancellation tokens via its CancellationTokenSource.Token property. This token can then get passed around to any number of threads, tasks, or long-running operations that should receive a cancellation notification. Alternatively, the Cancel method can be used to cancel the request manually.
C# has supported cancellation tokens since .NET Framework 4, and the technology has evolved much since those early days. Cancellation tokens allow a long-running operation to stop what it is doing and gracefully die. Interestingly, programming in the AWS cloud introduces a new set of challenges if you want your code to support cancellation tokens.
In ASP.NET, cancellation tokens work by letting the web server notify the API controller that the client has cancelled the request. You simply add a CancellationToken parameter into the action method, and this parameter will be automatically bound to the HttpContext.RequestAborted token for the request.
For better or worse, ASP.NET does a lot of heavy lifting for you, so it may be surprising to find cancellation token support doesn’t really work in AWS. In this article, I will tackle this problem head on and show you all how it can be done.
The focus will be entirely on .NET Core 6.0 running on AWS. Because cancellation tokens are an advanced topic, I will assume a level of comfort with C#, the .NET SDK, and the AWS CDK (Cloud Development Kit). I highly recommend cloning the source code off GitHub so you can play with the solution on your own.
The code sample is too large to build it step-by-step so I will provide a quick getting started guide so you can deploy everything to AWS. I chose to go with the AWS CDK because it is flexible and powerful. The CDK code is also minimal.
Getting Started with the AWS CDK
The solution provided in the download has three projects. There are two apps with two different approaches to tackle cancellation tokens. The third is for the CDK to automatically deploy everything to AWS. Each app is a lambda function, one that implements ASP.NET and one that is just a simple lambda function.
This is what the folder structure looks like:
Figure 1. Folder structure
The ASP.NET and Function lambda functions can be pushed to the Elastic Container Registry (ECR) via docker images. You do a docker build on the Dockerfile to create the image and then push it to ECR.
The Dockerfile is a text file that contains instructions for building the docker image. The build is done in stages. For example, the build stage uses the full .NET SDK, while the final image uses the minimal base necessary to run the lambda function. The Dockerfile below builds the ASP.NET lambda function.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 |
FROM public.ecr.aws/lambda/dotnet:6-arm64 as base FROM mcr.microsoft.com/dotnet/sdk:6.0 as build WORKDIR /src COPY Aws.CancellationTokens.AspNet.csproj /src RUN dotnet restore COPY . /src RUN dotnet publish -c Release --runtime linux-arm64 --self-contained false -o /src/publish FROM base AS final WORKDIR /var/task COPY --from=build /src/publish . CMD ["Aws.CancellationTokens.AspNet::Aws.CancellationTokens.AspNet.LambdaEntryPoint::FunctionHandlerAsync"] |
To work with docker and ECR images, go to the AWS console and click on Elastic Container Registry. Then, click on “Create repository”, and set the Visibility to Private.
The name of the repository needs to be the same as the one declared in the CDK. I recommend net-aws-cancellation-tokens-aspnet, and net-aws-cancellation-tokens-function. You can change the name but be sure to change it in the CDK too.
If you click on the repository, there is a “View push commands” button that give you step-by-step instructions on how to push a docker image into ECR.
For example:
|
1 2 3 4 5 6 7 |
> aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin ACCOUNT_ID.dkr.ecr.us-east-1.amazonaws.com > docker build -t net-aws-cancellation-tokens-aspnet . > docker tag net-aws-cancellation-tokens-aspnet:latest ACCOUNT_ID.dkr.ecr.us-east-1.amazonaws.com/net-aws-cancellation-tokens-aspnet:latest > docker push ACCOUNT_ID.dkr.ecr.us-east-1.amazonaws.com/net-aws-cancellation-tokens-aspnet:latest |
Once the images are in the ECR repositories, you can use Docker Desktop to validate the images. The images should also show up in ECR.
Next, turn to the CDK and deploy the solution. Open the Aws.CancellationTokens.Cdk\Program.cs file and set the ACCOUNT_ID to your own AWS account id.
Look in the root folder for a cdk.json file and change directory into that folder. Then use the cdk command to deploy the CDK.
|
1 2 3 4 5 6 7 |
> dotnet build > cdk boostrap aws://ACCOUNT_ID/us-east-1 > cdk synth > cdk deploy |
You should have two endpoints to play with at the end of the deploy process.
Both API endpoints will return a Gateway Timeout (504) to simulate the use of cancellation tokens via Task.Delay but you can change the delay, so it works correctly.
Inspect the CDK
Open the CancellationTokensStack.cs file in the CDK project folder. The Stack declares two lambda functions and hooks them up to the AWS gateway so they can respond to HTTP requests.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 |
// ASP.NET lambda function var repositoryAspNet = Repository.FromRepositoryName( this, "ecr-repository-aspnet", "net-aws-cancellation-tokens-aspnet"); var handlerAspNet = new DockerImageFunction( this, "lambda-aspnet", new DockerImageFunctionProps { Architecture = Architecture.ARM_64, Timeout = Duration.Seconds(15), // timeout duration MemorySize = 128, LogRetention = RetentionDays.THREE_DAYS, Code = DockerImageCode.FromEcr(repositoryAspNet) }); var apiAspNet = new RestApi( this, "rest-api-aspnet", new RestApiProps { RestApiName = "net-aws-cancellation-tokens-aspnet", Description = "Support cancellation tokens with ASP.NET" }); var restIntegrationAspNet = new LambdaIntegration(handlerAspNet); apiAspNet.Root.AddProxy(new ProxyResourceOptions { AnyMethod = true, DefaultIntegration = restIntegrationAspNet }); // Simple lambda function var repositoryFunction = Repository.FromRepositoryName( this, "ecr-repository-function", "net-aws-cancellation-tokens-function"); var handlerFunction = new DockerImageFunction( this, "lambda-function", new DockerImageFunctionProps { Architecture = Architecture.ARM_64, Timeout = Duration.Seconds(15), // timeout duration MemorySize = 128, LogRetention = RetentionDays.THREE_DAYS, Code = DockerImageCode.FromEcr(repositoryFunction) }); var apiFunction = new RestApi( this, "rest-api-function", new RestApiProps { RestApiName = "net-aws-cancellation-tokens-function", Description = "Support cancellation tokens with a Lambda function" }); var restIntegrationFunction = new LambdaIntegration(handlerFunction); apiFunction.Root.AddMethod("POST", restIntegrationFunction); |
This CDK declares the lambda functions, integrates them with the AWS gateway, and uses ECR docker images. As you can see, there is nothing special about the CDK to work with cancellation tokens. This infrastructure codes simply sets a timeout duration which kills the lambda function ungracefully. All the actual work happens in the app itself, which I will investigate next.
Cancellation Tokens in the AWS Serverless Cloud
One of the many assumptions ASP.NET makes is the Kestrel web server. This is the backend API that can receive cancellation notifications to gracefully kill the request.
In the serverless cloud, a lambda function is simply an event that fires from the AWS gateway. This event has no ties to a web server, so any cancellation notifications do not make it across to the HTTP context object. This means a cancellation notification does not affect the HttpContext.RequestAborted object with the cancellation token.
Luckily, lambda functions do come with a timeout setting, the max is 15 minutes, so you are able to exploit this fact and gracefully cancel the request when it gets too close to the timeout.
Next, I will explore how to cancel the request via a timeout for both a simple lambda function and a lambda function that runs ASP.NET.
Cancellation Tokens in a Simple Lambda Function
A straightforward lambda function has the ILambdaContext object with the ILambdaContext.RemainingTime property. The cancellation token can be initialized using this information with a time limit.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 |
public class Function { private static readonly TimeSpan GracefulStopTimeLimit = TimeSpan.FromSeconds(3); public async Task<APIGatewayProxyResponse> FunctionHandler( APIGatewayProxyRequest request, ILambdaContext context) { var cts = new CancellationTokenSource( context.RemainingTime > GracefulStopTimeLimit ? context.RemainingTime.Subtract(GracefulStopTimeLimit) : GracefulStopTimeLimit); try { await Task.Delay(TimeSpan.FromSeconds(15), cts.Token); return new APIGatewayProxyResponse { Body = JsonSerializer.Serialize( new Casing(request.Body.ToLower(), request.Body.ToUpper())), StatusCode = 200 }; } catch (TaskCanceledException e) { return new APIGatewayProxyResponse { Body = e.Message, StatusCode = 504 }; } } } |
The GracefulStopTimeLimit is a TimeSpan that can be set to 3 seconds. If the request times out via the cancellation token, the code throws a TaskCanceledException which can be handled gracefully. Once you have the cancellation token, it is possible to pass this down throughout the lambda function. Be sure to put the try/catch in the outermost layer so it can handle timeouts gracefully.
In local development the lambda function does not run directly on a web server but uses the lambda-test-tool-6.0. This test tool does not pass in a timeout setting, which is the reason for the check and going with the GracefulStopTimeLimit when the setting is missing.
Cancellation Tokens in an ASP.NET Lambda Function
ASP.NET can run on top of a lambda function via the APIGatewayProxyFunction entry point. This spins up ASP.NET during a cold start, and hooks everything up for you which includes controllers, middleware, and the Startup class. Additionally, ASP.NET supports local development via a local entry point which runs Kestrel.
In ASP.NET, the cancellation token automatically binds itself to a parameter in the controller’s action method. There is a built-in binder that does this for you. To support cancellation tokens in AWS, this binder must be replaced with one that relies on the timeout duration.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 |
public class TimeoutModelBinderProvider : IModelBinderProvider { public IModelBinder? GetBinder(ModelBinderProviderContext context) { if (context.Metadata.ModelType != typeof(CancellationToken)) { return null; } var config = context.Services.GetRequiredService<IHttpContextAccessor>(); return new TimeoutModelBinder(config); } private class TimeoutModelBinder : CancellationTokenModelBinder, IModelBinder { private static readonly TimeSpan GracefulStopTimeLimit = TimeSpan.FromSeconds(3); private readonly IHttpContextAccessor _httpContextAccessor; public TimeoutModelBinder(IHttpContextAccessor httpContextAccessor) { _httpContextAccessor = httpContextAccessor; } public new async Task BindModelAsync(ModelBindingContext bindingContext) { var httpContext = _httpContextAccessor.HttpContext!; var lambdaContext = httpContext.Items["LambdaContext"] as ILambdaContext; await base.BindModelAsync(bindingContext); if (bindingContext.Result.Model is CancellationToken cancellationToken && lambdaContext is not null && lambdaContext.RemainingTime > GracefulStopTimeLimit) { var remainingTimeSource = new CancellationTokenSource(); remainingTimeSource.CancelAfter(lambdaContext.RemainingTime.Subtract(GracefulStopTimeLimit)); var newTokenSource = CancellationTokenSource.CreateLinkedTokenSource( remainingTimeSource.Token, cancellationToken); var model = (object) newTokenSource.Token; bindingContext.ValidationState.Clear(); bindingContext.ValidationState.Add(model, new ValidationStateEntry { SuppressValidation = true }); bindingContext.Result = ModelBindingResult.Success(model); } } } } |
To replace the built-in cancellation token binder, simply remove the CancellationTokenModelBinderProvider and add the one that uses the timeout. Here, the technique is to use an extension method to make this easier to use.
|
1 2 3 4 5 6 |
public static MvcOptions ConfigureGracefulTimeout(this MvcOptions options) { options.ModelBinderProviders.RemoveType<CancellationTokenModelBinderProvider>(); options.ModelBinderProviders.Insert(0, new TimeoutModelBinderProvider()); return options; } |
This makes the cancellation token available via the same mechanism in the controller. The nice thing about this approach is that you don’t have to change anything in your existing code. If the action method already assumes a valid cancellation token parameter, it can keep doing just that. This is ideal because the cancellation token you get in local development works the same way too.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 |
[Route("api/[controller]")] public class ValuesController : ControllerBase { [HttpGet] public async Task<IActionResult> Get(CancellationToken cancellationToken) // cancellation token parameter { try { await Task.Delay(TimeSpan.FromSeconds(15), cancellationToken); return Ok(new[] {"value1", "value2"}); } catch (TaskCanceledException e) { return StatusCode(504, e.Message); } } } |
In local development ASP.NET runs on a web server like Kestrel. This means that cancellation token support works out of the box, so the timeout replacement can be skipped for local development, an environment variable check can do the trick.
|
1 2 3 4 5 |
if (Environment.GetEnvironmentVariable("ASPNETCORE_ENVIRONMENT") is null) { services.AddHttpContextAccessor(); services.Configure<MvcOptions>(options => options.ConfigureGracefulTimeout()); } |
Note the timeout model binder relies on the HTTP context accessor so it can find the ILambdaContext with the timeout setting.
Cancellation Tokens in AWS Fargate
Cancellation token support in Fargate really depends on the ENTRYPOINT in the Dockerfile. If the ASP.NET app spins up via dotnet run then it runs on Kestrel, which is a web server with support for cancellation tokens out of the box. The reason I did not include a Fargate app is because the code looks like any other ASP.NET application you have already written in .NET Core.
If you are still having problems with cancellation token support, I would check for anything that proxies the HTTP request over to the load balancer. The cancellation notification must be able to make it across the request pipeline, before it ever reaches Kestrel.
Conclusion
C# in general has excellent support for cancellation tokens. Support can go sideways when the assumptions made are no longer valid. Hopefully, the techniques discussed so far will get your cancellation tokens working in AWS.
Load comments