Automatic perf stats in baselines #57730
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
typescript-bot
added
Author: Team
For Uncommitted Bug
jakebailey
reviewed
Mar 12, 2024
|
What happens if a test uses |
jakebailey
approved these changes
Mar 15, 2024
This was referenced Mar 15, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This adds automatic baselines similar to what you get from
--extendedDiagnosticswhenever the harness detects an "interesting" amount of something happening. To reduce baseline noise from future changes, these values are rounded to metric-specific intervals, so e.g. if we go from 513 to 517 symbols, that's not going to register a baseline diff (it will print back as "500").This should let us easily create "don't blow stuff up" tests when things like #57710 happen.
I want to make this opt-in/opt-out on a per-test base but the harness code is not well-suited to threading that info through and I want to do a follow-up refactor that cleans up the baseliner to have access to the per-test options.