Return timedOut, but don't cancel task, let exception come and log exception

[aalia]

I am using Retry and timeout policy for my http client.
It normally takes more time for an exception to come and then if we retry it for say 3 times, it will take even more time. We want to reduce this time. So, what we want is, to set a timeout, even if exception haven't occurred, it should timeout. But, because we need the information for exception, we want to continue the call in a background thread may be, so that we can log the exception details.

Following is the code that I have done till now, we are using .NET core, so adding in Startup.cs:

 services.AddHttpClient<IClient,
                            Client>("Client", c => { c.DefaultRequestHeaders.Add("Accept", "application/json"); })
 
            .SetHandlerLifetime(Timeout.InfiniteTimeSpan)
            .AddPolicyHandler(GetFallbackPolicy())
            .AddPolicyHandler(GetRetryPolicy())
            .AddPolicyHandler(GetTimeOutPolicy());
			
			private IAsyncPolicy<HttpResponseMessage> GetRetryPolicy()
        {
            return HttpPolicyExtensions
                    .HandleTransientHttpError() 
                    .OrInner<WebException>(exception => new[] {WebExceptionStatus.ConnectionClosed,
                  WebExceptionStatus.Timeout,
                  WebExceptionStatus.RequestCanceled }.
            Contains(exception.Status))                   
                    .OrResult(msg => msg.StatusCode == System.Net.HttpStatusCode.NotFound)
                    .WaitAndRetryAsync(3, retryAttempt => TimeSpan.FromSeconds(Math.Pow(500, retryAttempt)));
        }

        private IAsyncPolicy<HttpResponseMessage> GetFallbackPolicy()
        {            
            return HttpPolicyExtensions
                    .HandleTransientHttpError().Or<Exception>()
                    .FallbackAsync<HttpResponseMessage>(fallbackAction: (context) =>
                    {
                        return new Task<HttpResponseMessage>(() => new HttpResponseMessage() { StatusCode = HttpStatusCode.InternalServerError });
                    }, onFallbackAsync: (exception) =>
                    {                        
                        throw exception.Exception;
                    });
        }

        private IAsyncPolicy<HttpResponseMessage> GetTimeOutPolicy()
        {
            return Policy.TimeoutAsync(30, TimeoutStrategy.Optimistic).AsAsyncPolicy<HttpResponseMessage>();           
        }

We don't want to cancel the task, it should go on and log exception, and it should return timedout, whne timed out. Please help me how to go about it?

[reisenberger]

See the documentation on TimeoutPolicy with TimeoutStrategy.Pessimistic. In this mode, the timed-out action is not cancelled: it is passed as the task property to the onTimeoutAsync delegate, and you can attach a continuation to capture what eventually happens.

The parallel calls will of course consume more resource - the new calls (retries) will run in parallel with the continuing old (walked-away-from) calls. If that's an issue, a compromise could be to have this mode as an option, which you can switch on in production to gather more diagnostic information, when relevant.

[aalia]

Thank you so much @reisenberger for valuable reply. It is helping me a lot.
I updated my code for timeout policy as Pessimistic:

private IAsyncPolicy<HttpResponseMessage> GetTimeOutPolicy()
    {
        return Policy.TimeoutAsync(2, TimeoutStrategy.Pessimistic, async (context, timespan, task) =>
        {
           task.ContinueWith(t =>
            { 
                    if (t.IsFaulted)
                {
                    _logger.LogInformation("faulted");
                    _logger.LogInformation($"{context.PolicyKey} at {context.OperationKey}: execution timed out after {timespan.TotalSeconds} seconds, eventually terminated with: {t.Exception}.");

                }
                else if (t.IsCanceled)
                {
                    _logger.LogInformation("Canceled");                  
                    _logger.LogInformation($"{context.PolicyKey} at {context.OperationKey}: execution timed out after {timespan.TotalSeconds} seconds, task cancelled.");
                    }
                else
                {
                    _logger.LogInformation("task completed");
                    }
            });
        }).AsAsyncPolicy<HttpResponseMessage>();           
    }

But, the task is getting cancelled. So, the second condition (t.IsCanceled) is true. Am I doing it differently? it should not honour cancellation, it should be faulted and log the actual exception.

[reisenberger]

@aalia You're right. In principle, TimeoutStrategy.Pessimistic can achieve what you asked (not cancel and capture the outcome of the walked-away-from task), but there is an additional requirement: the delegate executed through the policy must also ignore any CancellationToken passed to it.

Explanation: In TimeoutStrategy.Pessimistic the implementation does still pass the policy's timing-out CancellationToken to the executed code. This was an intentional design choice, for two reasons: (i) it is still always preferable, resource-wise, to cancel if possible; (ii) it allows TimeoutStrategy.Pessimistic to be used on underlying code which mixes elements which do and don't honour cancellation.

So the behaviour you are seeking is not possible with off-the-shelf timeout policy configured via the existing HttpClientFactory methods.

---

To achieve what you asked, you would have to create code to not pass the CancellationToken into lower levels of the http call. You could do any one of:

(1) Code a simple DelegatingHandler overriding SendAsync() to not pass on the supplied CancellationToken, and insert that into the HttpClient configuration on HttpClientFactory, inside the TimeoutPolicy (list it after the TimeoutPolicy), using the .AddHttpMessageHandler<THandler>() overload

or

(2) Code your own custom handler similar to PolicyHttpMessageHandler, but make the underlying http execution intentionally ignore the passed CancellationToken - something like this:

  response = await policy.ExecuteAsync((c, ct) => SendCoreAsync(request, c, CancellationToken.None), context, cancellationToken).ConfigureAwait(false);

or

(3) Code your own timeout policy as a custom proactive Policy which does not pass the policy's timing-out CancellationToken to the executed code

---

Note that any of (1) (2) or (3) will also lose you other uses of cancellation through HttpClient, such as:

• cancelling all pending requests via CancelPendingRequests() and Dispose()
• the overall timeout provided by HttpClient.Timeout, which also operates by CancellationToken

---

To re-iterate downsides/cautions about doing this in production: Given this approach effectively chooses to suspend any local timeout, you risk the app tieing up resources waiting for responses which never come - this could become a resource issue which affects your application performance over time. If the called system is under your control, it may be more effective to instrument the called system to determine why it isn't responding in a timely fashion.

[reisenberger]

Closing due to no further questions at this time. If you have further questions, please post again (or start a new issue), and we will gladly pick up the conversation.

[lor1mp]

Hi @reisenberger -- just following up on this. This statement:

In TimeoutStrategy.Pessimistic the implementation does still pass the policy's timing-out CancellationToken to the executed code. This was an intentional design choice, for two reasons: (i) it is still always preferable, resource-wise, to cancel if possible; (ii) it allows TimeoutStrategy.Pessimistic to be used on underlying code which mixes elements which do and don't honour cancellation.

Is true in theory... but doesn't seem to be in practice, we've observed the downstream cancellation token not being cancelled. I believe I've root caused it down to the fact that the exceptions generated by the cancellation cause the CancellationTokenSource combinedSource to exit the using scope, and seemingly if the source of the token gets disposed, the token is never signaled.

This isn't apparent in unit tests, because the UT SystemClock actually cancels the downstream token as part of it's behaviour in a timeout.

I have a branch with a test, and fix for this will post the PR.

PR: #1697