RetryTask using Task Parallel Library

Have you ever had to implement some form of retry functionality? I have, a couple of times in fact. Common scenarios would be to retry web service requests in the event of CommunicationException, or retry some time consuming method invocation. The parameters for these retry logic would be to wait a specified amount of time and to retry a specified number of times.

A very common retry logic I’ve seen to implement the retry logic is using a while loop to enforce the maximum number of retries, and a Thread.Sleep to perform the wait time. This is an example below.

public class Calculate
{
    private readonly Calculator _calculator;

    public Calculate(Calculator calculator)
    {
        _calculator = calculator;
    }

    public int CalculateWithRetry()
    {
        const int maxRetryAttemps = 2;
        int retryAttempts = 0;
        while (retryAttempts <= maxRetryAttemps)
        {
            try
            {
                return _calculator.Calculate();
            }
            catch (InvalidOperationException)
            {
                retryAttempts++;
                Thread.Sleep(TimeSpan.FromSeconds(2));
            }
        }
        return 0;
    }
}

There is absolutely nothing wrong with doing that in my opinion, it’s straightforward and the simplest implementation I can think of.

I thought to myself though, if there is another way to write a retry task. I explored different opinions, and found one option to be both interesting and intriguing. I decided to try and implement the retry logic using the Task Parallelism Library (TPL) in .NET Framework 4.5.

I created a generic RetryTask, that is able to take in an Action as well as a Func<T> as retry-able actions, first being a void and other returning a result. The RetryTask also takes in a generic TException, which triggers the retry when that exception is being thrown. For instance, you can create a RetryTask with a CommunicationException, so in the event of timeouts, the web service request can be retried.

public class RetryTask : IRetryTask
{
    private readonly ILogger _logger;
    private int _retryWaitIntervalSeconds;
    private int _retryAttempts;

    public RetryTask() : this(new ConsoleLogger())
    {}

    public RetryTask(ILogger logger)
    {
        _logger = logger;
    }

    public bool Execute<TException>(Action action, int retryWaitIntervalSeconds, int retryAttempts)
        where TException : Exception
    {
        _retryWaitIntervalSeconds = retryWaitIntervalSeconds;
        _retryAttempts = retryAttempts;
        int result;
        Func<int> actionFuncWrapper = () =>
            {
                action();
                return 0;
            };
        return StartTaskWithRetryRecursive<int, TException>(actionFuncWrapper, out result);
    }

    public bool Execute<TResult, TException>(Func<TResult> action, int retryWaitIntervalSeconds, int retryAttempts, out TResult result)
        where TException : Exception
    {
        _retryWaitIntervalSeconds = retryWaitIntervalSeconds;
        _retryAttempts = retryAttempts;
        return StartTaskWithRetryRecursive<TResult, TException>(action, out result);
    }

    private bool StartTaskWithRetryRecursive<TResult, TException>(Func<TResult> action, out TResult result, int retryAttemptNumber = 1)
        where TException : Exception
    {
        result = default(TResult);

        var taskCompletionSource = new TaskCompletionSource<bool>();

        Task<TResult> task = StartNewTask<TResult, TException>(action, taskCompletionSource);

        WaitForTaskCompletion(taskCompletionSource);
        if (taskCompletionSource.Task.Result) 
        {
            result = task.Result;
            return true;
        }

        StartDelay(retryAttemptNumber);
        if (retryAttemptNumber <= _retryAttempts)
            return StartTaskWithRetryRecursive<TResult, TException>(action, out result, retryAttemptNumber + 1);

        return false;
    }

    private void WaitForTaskCompletion(TaskCompletionSource<bool> taskCompletionSource) 
    {
        try 
        {
            taskCompletionSource.Task.Wait();
        } 
        catch (AggregateException ex) 
        {
            _logger.Error("unexpected error encountered whilst invoking action, {0}", ex.ToString());
            throw;
        }
    }

    private void StartDelay(int attemptNumber) 
    {
        _logger.Info("attempting retry number {0} after delay of {1} seconds...", attemptNumber, _retryWaitIntervalSeconds);
        Task.Delay(TimeSpan.FromSeconds(_retryWaitIntervalSeconds)).Wait();
    }

    private Task<TResult> StartNewTask<TResult, TException>(Func<TResult> action, TaskCompletionSource<bool> taskCompletionSource)
        where TException : Exception
    {
        Func<object, TResult> taskFunc = obj => action();  
        var startTask = new Task<TResult>(taskFunc, taskCompletionSource);

        // creates two mutually exclusive continuation tasks, one that will only run if startTask is successful, and another that will run if startTaskFails.
        startTask.ContinueWith<TResult>(ContinueWithOnCompletionTask, taskCompletionSource, TaskContinuationOptions.OnlyOnRanToCompletion);
        startTask.ContinueWith<TResult>(ContinueWithOnFaultedTask<TResult, TException>, taskCompletionSource, TaskContinuationOptions.OnlyOnFaulted);

        startTask.Start();
        return startTask;
    }

    private TResult ContinueWithOnFaultedTask<TResult, TException>(Task<TResult> antecedantTask, object taskCompletionSource) 
        where TException : Exception
    {
        if (antecedantTask.Exception != null)
        {
            if (antecedantTask.Exception.InnerException.GetType() == typeof(TException)) 
            {
                _logger.Warning("{0} exception occurred: {1}", typeof(TException).Name, antecedantTask.Exception.InnerException.ToString());
                ((TaskCompletionSource<bool>)taskCompletionSource).SetResult(false); // signal that a TException error occurred and we should retry
            } 
            else
            {
                ((TaskCompletionSource<bool>)taskCompletionSource).SetException(antecedantTask.Exception);
            }
        }
        return default(TResult);
    }

    private TResult ContinueWithOnCompletionTask<TResult>(Task<TResult> antecedantTask, object taskCompletionSource) 
    {
        ((TaskCompletionSource<bool>)taskCompletionSource).SetResult(true); // signals all went well
        return antecedantTask.Result;
    }
}

Comparatively, the RetryTask is far more complex and also harder to read for someone who’s unfamiliar with the TPL. For myself, this is a more of a learning exercise; Also I’m not a big fan of using while loops and Thread.Sleep in production code, and using TPL is an alternative.

Some of the TPL functions used in the RetryTask are:

• TaskCompletionSource
• Task.ContinueWith
• TaskContinuationOptions (See OnlyRanToCompletion and OnlyOnFaulted)
• Task.Delay

The source code can be found in EdLib.

Let me know how you would implement a Retry implementation Image may be NSFW.
Clik here to view.

Image may be NSFW.
Clik here to view.