C# HttpClient Wrapper for Asynchronous REST resources

May 08, 2020

In a previous post I showed how to use the asynchronous REST pattern to work around the problem of R Plumber only being able to serve one request at a time. Now I’m going outline a HTTP client wrapper that abstracts the process of issuing the POST create resource request, polling the status of the resource and returning and deserializing the result. The client of the wrapper can then make a request for an asynchronous resource and await it synchronously (or not) in their client code.

Currently the latest practise in an ASP.NET Core site is to use services.AddHttpClient() in the Startup.cs file to define a named HttpClient that can be generated by IHttpClientFactory in dependent services:

 
// in Startup.cs

//...
using Microsoft.Extensions.DependencyInjection;
using System.Net.Http;


public void ConfigureServices(IServiceCollection services)
{

	string analyticsApiUrl = "https://my-R-analytics-api/"

	services.AddHttpClient("AnalyticsAPI", client =>
	{
		client.BaseAddress = new Uri(analyticsApiUrl);
		client.DefaultRequestHeaders.Add("Accept", "application/json");
	})
	.ConfigurePrimaryHttpMessageHandler(() =>
	{
		return new HttpClientHandler()
		{
			AllowAutoRedirect = true
		};
	});;

	services.AddTransient<IAnalyticsHttpWrapper, AnalyticsHttpWrapper >();

	//...
}

And then a standard HTTP Request Wrapper class with one generic async GET request method might look something like this:

public interface IAnalyticsHttpWrapper
{
	Task<T> Get<T>(string path);
}

public class AnalyticsHttpWrapper : IAnalyticsHttpWrapper
{
	private readonly IHttpClientFactory _httpClientFactory;
	private readonly string _httpClientName;
	private readonly HttpClient _httpClient;

	public AnalyticsHttpAccess(IHttpClientFactory httpClientFactory)
	{
		_httpClientFactory = httpClientFactory;
		_httpClientName = "AnalyticsAPI";
		_httpClient = _httpClientFactory.CreateClient(_httpClientName);
	}

	public async Task<T> Get<T>(string path)
	{
		var response = await _httpClient.GetStringAsync(path);
		return JsonConvert.DeserializeObject<T>(response);
	}
}

(An example I adapted from this stackoverflow answer)

We could quite happily consume this wrapper from a client class like so:

public class MyClientClass
{
	private readonly IAnalyticsHttpWrapper _analyticsHttpWrapper;

	public MyClientClass(IAnalyticsHttpWrapper analyticsHttpWrapper)
	{
		_analyticsHttpWrapper = analyticsHttpWrapper;
	}

	public async Task<MyAnalyticsResults> GetAnalyticsResults(int analysisId) 
	{
		return await _analyticsHttpWrapper.Get<MyAnalyticsResults>($"/analysis/{analysisId}/result");
	}
}

All well and good. But when we move our API to an asynchronous request-response pattern, async Task<T> Get<T>(string path) won’t work as implemented. How can we change our AnalyticsHttpWrapper class so that it works with the new pattern, while minimizing the changes needed to our client classes, such as MyClientClass?

If our API dependency is following the asynchronous request-response pattern, which we set up our R Plumber API to do in the previous post, it is actually quite easy to do.

Our AnalyticsHttpWrapper class becomes:

public class AnalyticsHttpWrapper : IAnalyticsHttpWrapper
{
	private readonly IHttpClientFactory _httpClientFactory;
	private readonly string _httpClientName;
	private readonly HttpClient _httpClient;
	private readonly int _defaultPollingInterval;
	private readonly int _defaultPollingTimeout;

	public AnalyticsHttpWrapper(IHttpClientFactory httpClientFactory)
	{
		_httpClientFactory = httpClientFactory;
		_httpClientName = "AnalyticsAPI";
		_httpClient = _httpClientFactory.CreateClient(_httpClientName);
		_defaultPollingInterval = 2000; // 2 seconds
		_defaultPollingTimeout = 1000*60*45; // 45 mins
	}

	public async Task<T> GetAsynchronous<T>(string path)
	{
		var httpResponse = await PollUntilResourceCreated(path);

		await CheckResourceCreatedResponseOk(httpResponse);

		var jsonContent = await httpResponse.Content.ReadAsStringAsync();
		var responseContent = JsonConvert.DeserializeObject<T>(jsonContent);
		
		return responseContent;
	}

	private async Task<HttpResponseMessage> PollUntilResourceCreated(string path)
	{
		var createResourceResponse = await _httpClient.PostAsync(path, null);
		if (createResourceResponse.StatusCode != System.Net.HttpStatusCode.Accepted)
		{
			var content = await createResourceResponse.Content.ReadAsStringAsync();
			throw new AnalyticsHttpException(
				$"Create resource did not respond with (202): {(int)createResourceResponse.StatusCode}", 
				content);
		}

		var tokenSource = new CancellationTokenSource();
		var cancellationToken = tokenSource.Token;

		var timeoutTask = Task.Delay(_defaultPollingTimeout, cancellationToken);
		var statusLocation = createResourceResponse.Headers.Location;

		HttpResponseMessage httpResponse;
		
		do
		{
			TimeOutCheck(timeoutTask, statusLocation);
			httpResponse = await _httpClient.GetAsync(statusLocation); // should auto redirect to resource location when resource creation completed
			await Task.Delay(_defaultPollingInterval);
		}
		while (httpResponse.StatusCode == System.Net.HttpStatusCode.Accepted);
		
		tokenSource.Cancel(); // Cancel timeout timer
		
		return httpResponse;
	}


	private void TimeOutCheck(Task timeoutTask, Uri location)
	{
		if (timeoutTask.IsCompleted)
		{
			 throw new AnalyticsHttpException(
				@$"Resource queued at {location.ToString()} did not finish after {TimeSpan.FromMilliseconds(_defaultPollingTimeout).TotalMinutes}; aborting");
		}
	}

	private async Task CheckResourceCreatedResponseOk(HttpResponseMessage httpResponse)
	{
		if (httpResponse.StatusCode == System.Net.HttpStatusCode.OK)
		{
			return;
		}
		else
		{
			var content = await httpResponse.Content.ReadAsStringAsync();
			throw new AnalyticsHttpException(
				$"Create resource accepted, but retrieving results responded: {(int)httpResponse.StatusCode}", 
				content);
		}
	}
}

public class AnalyticsHttpException : Exception
{
	public string HttpResponseContent { get; }

	public AnalyticsHttpException(string message, string httpResponseContent) : base(message)
	{
		HttpResponseContent = httpResponseContent;
	}
	public AnalyticsHttpException(string message) : base(message)
	{
	}

}

Our Task<T> Get<T>(string path) becomes Task<T> GetAsynchronous<T>(string path). In PollUntilResourceCreated, we first create the resource and expect a 202 response, with the status location in the location header. Then we keep making GET requests to the this location every 2 seconds, until we get a different response from 202 Accepted. We check the response is a success, and deserialize the object as normal and return it to the client.

A few things to note here: I’ve designed the API so I know what HTTP responses to expect in each scenario, so this implementation works fairly reliably. Other asynchronous APIs may do things differently, for instance if the API can create the resource quickly, it might immediately return a 201 Created response with a location to this resource, rather than needing to be polled.

Because we setup our HttpClient to automatically follow redirects, we can simply allow httpResponse = await _httpClient.GetAsync(statusLocation) in our do-while to keep executing, safe in the knowledge that when the status of the resource is completed, a redirect will be returned and our HTTP response content should contain the JSON results we expect.

There is also a default timeout of 45 minutes setup by using Task.Delay to start a task will complete when the timeout is reached. This is to stop the polling potentially continuing indefinitely if the resource on the server gets stuck in creation for whatever reason.

Finally, the naming of the public method GetAsynchronous is perhaps misleading, as it implies we are doing a GET request whereas in reality we are doing a POST request with several GETs until a redirect. We can’t really apply REST semantics now, so maybe something like CreateResourceAndRetrieve<T>(string path) is more appropriate. Regardless of what we call it, we just have to change our client classes to use this new implementation, and they will work as before.

Full code here