Add combinatorial support #4260
Description
Summary
Add combinatorial support.
Background and Motivation
Reduce gap with xUnit and NUnit to help user migrations and improve testing capabilities.
Proposed Feature
Introduce the following attributes:
[CombinatorialValues]
[CombinatorialValues] (or [CombinatorialDataRows] to match [DataRow]) allowing a list of values or no values (let's use params object?[]? values).
When some values are provided they will be used for the given parameter. When no arguments are provided and the type is bool or enum we would generate all values of the type.
Note this is called Values for NUnit and CombinatorialValues for xUnit.
[CombinatorialMemberData]
[CombinatorialMemberData] (or CombinatorialDynamicValues] to match [DynamicData]) allowing to provide a member returning IEnumerable<T> where T matches the type of the parameter (shall we allow IEnumerable<object>?).
Let's avoid the mistake of DynamicData by not asking the user to define the kind of member there.
Shall we support only properties and methods like DynamicData or allow support fields?
Other sources
We could consider more options but starting with those would be already a great start. The member would give the flexibility to build ranges, random etc...
Alternative Designs
We could consider having the 2 attributes without the Combinatorial prefix to consider they are only providing data and instead have [CombinatorialTestMethod] or a marker attribute like [Combinatorial].
This would allow to generate scenarios like:
[TestMethod]
public void TestSomething(
[Values(1, 2, 3)] int i,
[Values("A", "B", "C")] string s)
{
}which is equivalent to
[TestMethod]
[DataRow(1, "A")]
[DataRow(2, "B")]
[DataRow(3, "C")]
public void TestSomething(int i, string s)
{
}It would also help with generating other kind of permutations.