Public types missing from reference assemblies don't work with ParamsSource

[svick]

Consider the following benchmark:

using System;
using System.Collections.Generic;
using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Configs;
using BenchmarkDotNet.Environments;
using BenchmarkDotNet.Jobs;
using BenchmarkDotNet.Running;
using BenchmarkDotNet.Toolchains.CsProj;

class Program
{
    static void Main() => BenchmarkRunner.Run<Benchmark>();
}

public class MainConfig : ManualConfig
{
    public MainConfig()
    {
         Add(Job.Default.With(Platform.X64).With(CsProjCoreToolchain.NetCoreApp30));
         Options = ConfigOptions.KeepBenchmarkFiles;
    }
}

[Config(typeof(MainConfig))]
public class Benchmark
{
    public static IEnumerable<StringComparer> Comparers =>
        new[] { StringComparer.Ordinal };

    [ParamsSource(nameof(Comparers))]
    public StringComparer Comparer;

    [Benchmark]
    public void B() { }
}

When I attempt to run it, I get the following error:

Job-YBSJDZ.notcs(125,41): error CS0234: The type or namespace name 'OrdinalComparer' does not exist in the namespace 'System' (are you missing an assembly reference?) [C:\code\tmp\hwapp\bin\Release\netcoreapp3.0\Job-YBSJDZ\BenchmarkDotNet.Autogenerated.csproj]

The relevant generated code is:

instance.Comparer = (System.OrdinalComparer)System.Linq.Enumerable.ToArray(Benchmark.Comparers)[0];;

The problem here is that for some reason (dotnet/corert@83bcbe0), the actual type of StringComparer.Ordinal, OrdinalComparer is public, but it does not exist in reference assemblies. This means that the C# compiler won't see it, but the BDN code generator will consider it as the correct type to generate:

BenchmarkDotNet/src/BenchmarkDotNet/Extensions/ReflectionExtensions.cs

Line 40 in 0a4550f

while (!(type.IsPublic || type.IsNestedPublic) && type.BaseType != null)

[adamsitnik]

Hello @svick

Thanks for an excellent bug report!

I don't have a good solution as of today, but you can use the workaround I have used few days ago in the performance repo: https://github.com/dotnet/performance/blob/master/src/benchmarks/micro/corefx/System.Runtime/Perf.StringComparer.cs

[Params(
    StringComparison.Ordinal, StringComparison.OrdinalIgnoreCase,
    StringComparison.InvariantCulture, StringComparison.InvariantCultureIgnoreCase)]
public StringComparison Comparison { get; set; }

private StringComparer _comparer;

[GlobalSetup]
public void Setup() => _comparer = GetStringComparer();

private StringComparer GetStringComparer()
{
    switch (Comparison)
    {
        case StringComparison.CurrentCulture:
            return StringComparer.CurrentCulture;
        case StringComparison.CurrentCultureIgnoreCase:
            return StringComparer.CurrentCultureIgnoreCase;
        case StringComparison.InvariantCulture:
            return StringComparer.InvariantCulture;
        case StringComparison.InvariantCultureIgnoreCase:
            return StringComparer.InvariantCultureIgnoreCase;
        case StringComparison.Ordinal:
            return StringComparer.Ordinal;
        case StringComparison.OrdinalIgnoreCase:
            return StringComparer.OrdinalIgnoreCase;
        default:
            throw new NotSupportedException($"{Comparison} is not supported");
    }
}

[adamsitnik]

FWIW I've tested and it's still a thing for .NET 5

[mohitthakur220]

I am also trying to test performance of a console application using Dotnet Benchmark
it's giving a error related to missing assembly reference. even System.Data.Entity reference is given in solution. and Entity Framework reference is also given.

[timcassell]

@mohitthakur220 Can you share your benchmark project? Or a minimal repro?

[mohitthakur220]

Only benchmark project I can share but it has lot of dependencies with other projects in the same solution.
is there any way to have a call?

[timcassell]

@mohitthakur220 Just put it in a git repository, share the link, and explain how you are running the benchmarks.