Coverlet MTP extension #1788
Open
Coverlet MTP extension #1788
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Bertk
added
feature
Collaborator
Author
|
The code coverage dropped and shall be fixed before merge. main branch: line 90% branch 87% |
Bertk
added
the
* NO MERGE *
Bertk
force-pushed
the
coverlet-MTP
branch
2 times, most recently
from
592ff6a to
010a409
Compare
- Updated target frameworks to net472 in coverlet.core and coverlet.msbuild.tasks projects - Adjusted CoverletToolsPath for multi-targeting support in buildMultiTargeting props and targets - Created unit tests for Coverlet.MTP command line options validation - Added documentation for Coverlet.MTP integration
- Added CoverletExtensionCollector to handle test session lifecycle for coverage collection. - Introduced CoverletExtensionCommandLineProvider for command line options. - Created CoverletExtensionConfiguration to manage configuration settings. - Developed CoverletLoggerAdapter for logging integration with Microsoft Testing Platform. - Implemented CoverletExtensionEnvironmentVariableProvider for environment variable management. - Added CoverletExtensionProvider to register the Coverlet extension with the testing platform. - Created TestingPlatformBuilderHook to facilitate extension registration. - Updated project files to include necessary dependencies and configurations for Coverlet. - Added support for multiple target frameworks (net8.0 and net9.0). - Included build and packaging configurations for Coverlet.MTP. - Implemented command line options for coverage report formats and exclusions. - Established logging mechanisms for better traceability during coverage collection.
…ng and add coverage cleanup messages
…ationString for consistency and lowercase output
… consistency across environments
There was a problem hiding this comment.
Pull request overview
This pull request introduces Microsoft Testing Platform (MTP) support to coverlet through a new coverlet.MTP package, enabling code coverage collection for projects using the modern Microsoft Testing Platform instead of VSTest.
Key Changes:
- New
coverlet.MTPpackage with full MTP extension implementation - Core infrastructure updates to support managed instrumentation restore control
- Comprehensive test projects for validation and unit testing
- CI/CD workflow additions including GitHub Actions
- Dependency upgrades to support .NET 8/9 and MTP v2.x
- Documentation for MTP integration
Reviewed changes
Copilot reviewed 100 out of 105 changed files in this pull request and generated 10 comments.
Show a summary per file
| File | Description |
|---|---|
| src/coverlet.MTP/* | New MTP extension package with command-line providers, collectors, and configuration |
| src/coverlet.core/Helpers/InstrumentationHelper.cs | Added DisableManagedInstrumentationRestore parameter to control backup/restore behavior |
| src/coverlet.core/Coverage.cs | Added parameter support for instrumentation restore control |
| src/coverlet.collector/* | Extended collector settings with DisableManagedInstrumentationRestore flag |
| test/coverlet.MTP.unit.tests/* | Unit tests for MTP extension components |
| test/coverlet.MTP.validation.tests/* | Integration validation tests with sample projects |
| test/coverlet.tests.utils/TestUtils.cs | Added GetBuildConfigurationString() helper for cross-platform path handling |
| Directory.Packages.props | Upgraded dependencies including Microsoft.Testing.Platform v2.0.2, xunit v3.2.1 |
| .github/workflows/dotnet.yml | New GitHub Actions workflow for cross-platform CI/CD |
| Documentation/Coverlet.MTP.Integration.md | Comprehensive MTP integration guide with examples |
| global.json | Updated SDK version to 9.0.308 |
| eng/.yml, eng/.sh | Updated build and test scripts for MTP projects |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Updated target frameworks to net472 in coverlet.core and coverlet.msbuild.tasks projects - Adjusted CoverletToolsPath for multi-targeting support in buildMultiTargeting props and targets - Created unit tests for Coverlet.MTP command line options validation - Added documentation for Coverlet.MTP integration
- Added CoverletExtensionCollector to handle test session lifecycle for coverage collection. - Introduced CoverletExtensionCommandLineProvider for command line options. - Created CoverletExtensionConfiguration to manage configuration settings. - Developed CoverletLoggerAdapter for logging integration with Microsoft Testing Platform. - Implemented CoverletExtensionEnvironmentVariableProvider for environment variable management. - Added CoverletExtensionProvider to register the Coverlet extension with the testing platform. - Created TestingPlatformBuilderHook to facilitate extension registration. - Updated project files to include necessary dependencies and configurations for Coverlet. - Added support for multiple target frameworks (net8.0 and net9.0). - Included build and packaging configurations for Coverlet.MTP. - Implemented command line options for coverage report formats and exclusions. - Established logging mechanisms for better traceability during coverage collection.
…ng and add coverage cleanup messages
…ment for consistency across environments" This reverts commit 06a684c.
- Expanded MTP integration docs and added usage ToDo - Enabled package generation in coverlet.MTP.csproj - Added strong name signing for validation tests - Updated validation test project: new dependencies, utility refs, and test project copying - Updated InternalsVisibleTo for new test assemblies
Introduce a comprehensive validation test suite for the Coverlet Microsoft Testing Platform (MTP) extension, including new integration and CLI option tests. Add isolated test infrastructure with sample projects and custom MSBuild props/targets to ensure tests run in a pure MTP environment. Update packaging logic in coverlet.MTP to improve dependency handling and NuGet layout, and adjust build system to better support MTP scenarios. Also includes minor bug fixes and ensures correct local package versioning in tests.
- Enhance test output with detailed diagnostics and error context
- Use robust assertions with informative failure messages
- Create temp test projects under artifacts/tmp for isolation
- Add retries and file attribute handling to test dir cleanup
- Check for test executable and coverlet.MTP.dll before running
- Refactor HelpCommandTests for consistent path handling
- Fix sample class naming mismatch in test project
- Add condition to MSBuild import for props file robustness
- Update NuGet config and project file handling for clarity
Renamed the constant sutName to SutName in HelpCommandTests to follow .NET PascalCase naming conventions for constants. Updated all references to use the new name for consistency.
Switch all test projects to xunit.v3.mtp-v2 for Microsoft Testing Platform (MTP) integration. Bump Microsoft.Testing.Platform to 2.0.2 and manage Moq version via property. Make Coverlet MTP extension opt-in by default and register as TestingPlatformExtension. Minor formatting and encoding adjustments included.
- Rename --coverage to --coverlet-coverage and update all related option names for consistency and clarity (e.g., --formats, --exclude, --include). - Introduce CoverletOptionNames static class to centralize option name constants. - Update command-line providers to use new option names and improve descriptions. - Add validation for output formats; deprecate/comment out output path option. - Ensure help output always shows options; only collect coverage if --coverlet-coverage is set. - Update tests to use new flags and add help output verification. - Remove redundant code and improve comments for maintainability. - Update csproj to clean up package references and add local NuGet restore source.
Refactor CoverletExtensionCollector to use ITestHostProcessLifetimeHandler, moving instrumentation and report generation to new lifecycle methods. Modernize configuration handling and command-line parsing. Move InternalsVisibleTo attributes to AssemblyInfo.cs. Update tests for new APIs and improved diagnostics. Clean up obsolete code and improve maintainability.
- Allow multiple --formats options (ArgumentArity.OneOrMore) instead of comma-separated values - Enforce single, supported format per --formats argument - Update tests to use repeated --formats options - Expand and clarify process exit code to error message mapping
- Add BasicNoCoverage_CollectsDataForCoveredLines test to verify test execution without coverage collection. - Enhance Assert.True error messages to include process ErrorText for better debugging. - Update generated test project to reference TrxReport and explicitly include Tests.cs. - Refactor BranchTests to use explicit string type for result variables. - Minor code style and message consistency improvements.
Add detailed diagnostic logging and debugger attach support via environment variables. Enhance test module path resolution and configuration logging. Refactor DI container creation and introduce MtpProcessExitHandler for in/out-of-process execution. Add default exclude filter, improve report output handling, and expand command line parsing. Integrate Microsoft.Extensions.Configuration and update test project output path for consistency.
Introduce InstrumentationDiagnostics.cs to provide centralized, async diagnostic logging for code instrumentation and coverage collection. Logs instrumented modules, excluded modules, coverage result generation, and errors using ILogger, improving transparency and troubleshooting for Coverlet users. No changes to existing code; this is a new utility class.
Refactor to introduce CoverletCommandLineOptionDefinitions as the single source of truth for all command-line options, eliminating duplication and inconsistencies. Remove CoverletCommandLineOptionsProvider and update CoverletExtensionCommandLineProvider to use the centralized definitions and validation. Rename and clean up option constants in CoverletOptionNames. Update CoverageConfiguration and related logging for new source mapping file handling. Adjust tests and namespaces for consistency with new option names and structure.
- Refactor InstrumentationHelper backup/restore logic: - Skip and log if a module or symbol file is already in the backup list instead of throwing. - Check for backup existence before restore; log warnings or verbose messages as needed. - Add IsCurrentlyRunningAssembly to avoid restoring the running assembly. - Log the number of modules to restore and skip missing or running assemblies. - Update CoverletMTPCommandLineTests to match the current set of supported options. - Update HelpCommandTests to reflect new, clearer option descriptions.
Refactor build pipeline to use dotnet test for coverlet.MTP.unit.tests with MSBuild-based code coverage collection and opencover output. Increase minimum required line coverage from 70% to 90%. Update coverlet.MTP.unit.tests.csproj to include necessary imports and package references for coverage integration. Add comments for alternative coverage approaches and modernize test execution paths.
- Add support for reading Coverlet settings from appsettings.json or any Microsoft.Extensions.Configuration source. - Introduce CoverletMTPSettings, CoverletMTPConstants, and CoverletMTPSettingsParser for strongly-typed configuration and parsing. - Add CoverletConfigurationExtensions for easy registration with MTP. - Implement CoverletTestSessionHandler to use parsed configuration. - Rename all command line options from coverage-* to coverlet-* for consistency. - Enable and test the SourceMappingFile option. - Update and expand tests to cover configuration parsing, defaults, and new option names. - Update build/test pipeline and project files to support new config and cross-platform test execution.
- Add detailed Coverlet.MTP integration documentation and usage guide - Refactor CoverletExtensionCollector for clarity, logging, and env var handling - Overhaul test infra: generate SUT + test projects in solution, robust cleanup, and real-world structure - Improve coverage validation: assert on actual data, not just file presence - Add/configure .mcp.json for NuGet MCP server - Minor fixes: exit code handling, error messages, and CoverletTestSessionHandler interface compliance
Explicitly use ./ and .\ prefixes for test executables in build.yml scripts to ensure correct execution on all platforms. This improves reliability of unit test runs by avoiding path resolution issues on different operating systems.
Youssef1313
reviewed
Dec 24, 2025
Youssef1313
reviewed
Dec 24, 2025
Youssef1313
reviewed
Dec 24, 2025
Youssef1313
reviewed
Dec 24, 2025
Youssef1313
reviewed
Dec 24, 2025
Youssef1313
reviewed
Dec 24, 2025
Youssef1313
reviewed
Dec 24, 2025
- Add comprehensive CoverageConfigurationTests.cs for all config options using Moq - Fix devcontainer.json syntax and ensure GitHub Actions extension is included - Enable Dependabot for GitHub Actions and NuGet with custom schedule - Improve Coverlet.MTP.Integration.md: typo fix, clarify usage, update sample project and coverage command - Simplify build.yml by unifying test execution to cross-platform dotnet exec - Make CoverletDataCollector version dynamic from assembly - Remove default CoverletMTPEnabled property and make props import conditional - Set IsTestingPlatformApplication to false in csproj and tidy formatting
Youssef1313
reviewed
Dec 24, 2025
Youssef1313
reviewed
Dec 25, 2025
| return defaultFormats; | ||
| } | ||
|
|
||
| public string GetOutputDirectory() |
There was a problem hiding this comment.
Where is this method called? I haven't cloned the repo with your branch yet but I'm not able to see where it's called from the GitHub browser diff (except for LogConfigurationSummary which is logging-only).
Youssef1313
reviewed
Dec 25, 2025
| _ => $"coverage.{format}" | ||
| }; | ||
|
|
||
| string fullPath = fileName; |
There was a problem hiding this comment.
This is not correct.
This should use Path.Combine(serviceProvider.GetConfiguration().GetTestResultDirectory(), fileName)
Collaborator
Author
There was a problem hiding this comment.
Thanks. I missed to fix this line.
Youssef1313
reviewed
Dec 25, 2025
| /// Data collector for Coverlet code coverage. | ||
| /// Compatible with Microsoft.Testing.Platform V2.0.2 | ||
| /// </summary> | ||
| internal sealed class CoverletDataCollector : IDataProducer, ITestSessionLifetimeHandler |
There was a problem hiding this comment.
Is this extension registered? I don't see where the usage is.
- Eliminate --coverlet-output and --coverlet-source-mapping-file options from Coverlet.MTP integration, code, and docs. - Coverage output directory now determined by Microsoft Testing Platform's test result directory. - Update documentation and help output to reflect new behavior and remove obsolete references. - Remove CoverletDataCollector class; rely on CoverletExtensionCollector and MTP extension model. - Update build script and test projects for new output handling and configuration. - Improve documentation examples for merging and reporting coverage. - Refine table formatting and minor test project settings.
Introduce CoverletExtensionCollectorTests to comprehensively verify construction, option parsing, environment variable handling, and integration with Microsoft Testing Platform extension interfaces. Tests cover normal and edge cases, ensuring correct behavior without modifying production code.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This pull request introduces several infrastructure and configuration improvements to the project, focusing on modernizing dependencies, enhancing CI/CD workflows, and enforcing code quality standards.
Key update: New coverlet.MTP package for Microsoft Testing Platform support
Continuous Integration and Automation:
.github/workflows/dotnet.ymlGitHub Actions workflow to automate building, testing (with coverage), and reporting for .NET projects across Ubuntu, Windows, and macOS. This workflow includes steps for restoring dependencies, building, running tests with coverage, generating reports, and publishing test results and coverage summaries to PRs..github/dependabot.ymlto enable automated dependency updates with Dependabot, scheduled weekly.Dependency and Package Management:
Directory.Packages.props, includingMicrosoft.CodeAnalysis(to 4.13.0),Microsoft.NET.Test.Sdk(to 18.0.1),xunit.v3(to 3.2.1), and others. Added new packages for Microsoft Testing Platform, banned API analyzers, and updated several system and test libraries to their latest versions.CoverletVersionto 8.0.0 inDirectory.Build.propsand improved test logger and output configuration for better compatibility with new test platforms and CI environments.Solution Structure:
coverlet.MTP,coverlet.MTP.validation.tests,coverlet.MTP.unit.tests) to the solution file, including their build and folder organization.Most important changes:
1. Solution and API Governance
2. Documentation
Documentation/Coverlet.MTP.Integration.md).