[API Proposal]: Making "Process asynchronous tasks as they complete" easy by using IAsyncEnumerable

[Vijay-Nirmal]

EDITED on 1/23/2024 by @stephentoub:

namespace System.Threading.Tasks;

public class Task
{
+   public static IAsyncEnumerable<Task> WhenEach(params Task[] tasks);
+   public static IAsyncEnumerable<Task> WhenEach(params ReadOnlySpan<Task> tasks); // params when possible
+   public static IAsyncEnumerable<Task> WhenEach(IEnumerable<Task> tasks);

+   public static IAsyncEnumerable<Task<TResult>> WhenEach(params Task<TResult>[] tasks); // params for now; move it to ReadOnlySpan overload when that syntax is possible
+   public static IAsyncEnumerable<Task<TResult>> WhenEach(params ReadOnlySpan<Task<TResult>> tasks); // params when possible
+   public static IAsyncEnumerable<Task<TResult>> WhenEach(IEnumerable<Task<TResult>> tasks);
}

---

Background and motivation

Currently, if we need to "Process asynchronous tasks as they complete" then we need to write lots of unnecessary codes and its not straight forward, something like below.

// Using currently available APIs
List<Task<int>> downloadTasks = downloadTasksQuery.ToList();
while (downloadTasks.Any())
{
    Task<int> finishedTask = await Task.WhenAny(downloadTasks);
    downloadTasks.Remove(finishedTask);
    Process(await finishedTask);
}

API Proposal

namespace System.Threading.Tasks
{
    public class Task : IAsyncResult, IDisposable
    {
        public static IAsyncEnumerable<Task> WhenEach(params Task[] tasks);  // Please change the name, if needed
        public static IAsyncEnumerable<Task> WhenEach(IEnumerable<Task> tasks);
        public static IAsyncEnumerable<Task<TResult>> WhenEach(params Task<TResult>[] tasks); 
        public static IAsyncEnumerable<Task<TResult>> WhenEach(IEnumerable<Task<TResult>> tasks);
    }
}

API Usage

// Using newly created APIs
await foreach (var finishedTask in Task.WhenEach(downloadTasksQuery))
{
    Process(await finishedTask);
}

Alternative Designs

No response

Risks

No response

Updates

(Others can edit this section and add more info)

• WhenEach name suggested by @theodorzoulias, [API Proposal]: Making "Process asynchronous tasks as they complete" easy by using IAsyncEnumerable  #61959 (comment)

Tagging subscribers to this area: @dotnet/area-system-threading-tasks
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Background and motivation

Currently, if we need to "Process asynchronous tasks as they complete" then we need to write lots of unnecessary codes and its not straight forward, something like below.

// Using currently available APIs
List<Task<int>> downloadTasks = downloadTasksQuery.ToList();
while (downloadTasks.Any())
{
    Task<int> finishedTask = await Task.WhenAny(downloadTasks);
    downloadTasks.Remove(finishedTask);
    Process(await finishedTask);
}

API Proposal

namespace System.Threading.Tasks
{
    public class Task : IAsyncResult, IDisposable
    {
        public static IAsyncEnumerable<Task> WhenAnyAsEnumerable(params Task[] tasks);  // Please change the name, this is not a good name for this method :)
        public static IAsyncEnumerable<Task> WhenAnyAsEnumerable(IEnumerable<Task> tasks);
        public static IAsyncEnumerable<Task<TResult>> WhenAnyAsEnumerable(params Task<TResult>[] tasks); 
        public static IAsyncEnumerable<Task<TResult>> WhenAnyAsEnumerable(IEnumerable<Task<TResult>> tasks);
    }
}

API Usage

// Using new available APIs
await foreach (var finishedTask in Task.WhenAnyAsEnumerable(downloadTasksQuery))
{
    Process(await finishedTask);
}

Alternative Designs

No response

Risks

No response

Author:	Vijay-Nirmal
Assignees:	-
Labels:	api-suggestion, area-System.Threading.Tasks, untriaged
Milestone:	-

[svick]

This method already exists in AsyncEx, where it's called OrderByCompletion. It returns a (non-async) collection of wrapper tasks, but I suspect that's because it's older than IAsyncEnumerable<T> and using IAsyncEnumerable<T> returning the original tasks is the better approach today.

[theodorzoulias]

@Vijay-Nirmal a name that might be more suitable for the Task.WhenAnyAsEnumerable method is Task.WhenEach. :-)

[sakno]

You can achieve the same behavior with TaskCompletionPipe that has native support of IAsyncEnumerable<T> as well as channel-like methods WaitToReadAsync and TryRead.