Skip to content

Add compression benchmarks that capture recent zlib regressions #4900

@ericstj

Description

@ericstj

Highly compressible data

dotnet/runtime#118457 (comment) includes a comparison and sample benchmark that includes a large number of iterations of highly-compressible data. This regressed in 9.0 and was missed. We should add a benchmark to cover that.

    [Benchmark]
    public void CompressionTest()
    {
        var testData = _testData[TestFile];
        testData.CompressedStream.Position = 0;

        using (var z = new DeflateStream(testData.CompressedStream, CompressionMode.Compress, true))
        {
            for (int i = 0; i < testData.Iterations; i++)
            {
                z.Write(testData.Data);
            }
        }
    }

https://github.com/dotnet/performance/blob/main/src/benchmarks/micro/libraries/System.IO.Compression/TestData/alice29.txt should fit the bill, so simply adding some iterations to

public void Compress()
{
CompressedFile.CompressedDataStream.Position = 0; // all benchmarks invocation reuse the same stream, we set Postion to 0 to start at the beginning
var compressor = CreateStream(CompressedFile.CompressedDataStream, level);
compressor.Write(CompressedFile.UncompressedData, 0, CompressedFile.UncompressedData.Length);
}
might be enough to capture.

Heap fragmentation

dotnet/runtime#117949 addresses a case where removal of our custom allocator regressed performance on windows x64.

We were able to reproduce that with a benchmark that @stephentoub wrote:

using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Running;
using System.IO.Compression;

BenchmarkSwitcher.FromAssembly(typeof(Bench).Assembly).Run(args);

public class Bench
{
    private Barrier _barrier = new Barrier(Environment.ProcessorCount - 1);

    [Benchmark]
    public void Test()
    {
        Task.WaitAll(Enumerable
            .Range(0, _barrier.ParticipantCount)
            .Select(_ => Task.Run(() =>
            {
                _barrier.SignalAndWait();
                for (int length = 1; length < 1000; length++)
                {

                    byte[] buffer = new byte[length];
                    Random.Shared.NextBytes(buffer);
                    using (var z = new ZLibStream(new MemoryStream(), CompressionMode.Compress))
                    {
                        for (int i = 0; i < 100; i++)
                        {
                            z.Write(buffer);
                        }
                    }
                }
            })).ToArray());
    }
}
Method Runtime Mean Error StdDev Ratio RatioSD
Test .NET 8.0 153.2 ms 3.05 ms 8.44 ms 1.00 0.08
Test .NET 9.0 289.5 ms 5.64 ms 8.61 ms 1.90 0.12

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions