Skip to content

Tests: Add benchmark suite for easily measuring improvements #2931

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Jul 25, 2025

Conversation

NickCraver
Copy link
Collaborator

This is a benchmark suite addition to easily add and evaluate changes for future PRs, starting with #2928 so we can easily demonstrate wins and test an array of cases.

Example initial run:

Method Runtime s host port Mean Error StdDev Op/s Gen0 Allocated
ParseInt64 .NET 8.0 -1 ? ? 7.617 ns 0.0464 ns 0.0434 ns 131,291,739.3 - -
ParseInt32 .NET 8.0 -1 ? ? 7.883 ns 0.0679 ns 0.0635 ns 126,853,845.4 - -
ParseInt64 .NET Framework 4.8.1 -1 ? ? 50.202 ns 0.1326 ns 0.1240 ns 19,919,676.6 - -
ParseInt32 .NET Framework 4.8.1 -1 ? ? 50.142 ns 0.4437 ns 0.3933 ns 19,943,168.3 - -
ParseInt64 .NET 8.0 0 ? ? 7.370 ns 0.0465 ns 0.0388 ns 135,693,273.4 - -
ParseInt32 .NET 8.0 0 ? ? 7.625 ns 0.0531 ns 0.0496 ns 131,153,072.4 - -
ParseInt64 .NET Framework 4.8.1 0 ? ? 48.089 ns 0.2482 ns 0.2322 ns 20,794,796.8 - -
ParseInt32 .NET Framework 4.8.1 0 ? ? 46.931 ns 0.1704 ns 0.1423 ns 21,307,968.3 - -
ParseInt64 .NET 8.0 123442 ? ? 11.067 ns 0.0604 ns 0.0565 ns 90,361,521.4 - -
ParseInt32 .NET 8.0 123442 ? ? 11.163 ns 0.1182 ns 0.1106 ns 89,584,171.6 - -
ParseInt64 .NET Framework 4.8.1 123442 ? ? 60.421 ns 0.2843 ns 0.2660 ns 16,550,495.4 - -
ParseInt32 .NET Framework 4.8.1 123442 ? ? 59.490 ns 0.4267 ns 0.3782 ns 16,809,663.5 - -
ParseInt64 .NET 8.0 64 ? ? 8.279 ns 0.0350 ns 0.0310 ns 120,791,230.5 - -
ParseInt32 .NET 8.0 64 ? ? 8.285 ns 0.0381 ns 0.0318 ns 120,701,002.1 - -
ParseInt64 .NET Framework 4.8.1 64 ? ? 51.321 ns 0.4413 ns 0.4128 ns 19,485,217.5 - -
ParseInt32 .NET Framework 4.8.1 64 ? ? 49.911 ns 0.1307 ns 0.1222 ns 20,035,756.7 - -
ParseEndPoint .NET 8.0 ? host.com -1 NA NA NA NA NA NA
ParseEndPoint .NET Framework 4.8.1 ? host.com -1 NA NA NA NA NA NA
ParseEndPoint .NET 8.0 ? host.com 0 10.885 ns 0.0784 ns 0.0655 ns 91,873,383.2 0.0038 32 B
ParseEndPoint .NET Framework 4.8.1 ? host.com 0 34.381 ns 0.1880 ns 0.1667 ns 29,085,410.1 0.0140 88 B
ParseEndPoint .NET 8.0 ? host.com 65345 11.156 ns 0.2687 ns 0.3587 ns 89,639,864.8 0.0038 32 B
ParseEndPoint .NET Framework 4.8.1 ? host.com 65345 34.433 ns 0.2386 ns 0.2115 ns 29,041,723.3 0.0140 88 B

@mgravell
Copy link
Collaborator

I'll push a couple more things to your PR for double and format, then run before/after

@mgravell
Copy link
Collaborator

FYI:

Classic .NET toolchain is supported only for Windows, benchmark 'blahwhateverz' will not be executed

I'll see what I can do...

mgravell added 3 commits July 25, 2025 15:36
- add double parse
- add double format
@mgravell mgravell merged commit b995ebc into main Jul 25, 2025
3 of 5 checks passed
@mgravell mgravell deleted the craver/benchmarks branch July 25, 2025 14:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants