Skip to content

Commit ae21a60

Browse files
BertkBertk
committed
Refactor MTP integration, docs, and test infra overhaul
- Add detailed Coverlet.MTP integration documentation and usage guide - Refactor CoverletExtensionCollector for clarity, logging, and env var handling - Overhaul test infra: generate SUT + test projects in solution, robust cleanup, and real-world structure - Improve coverage validation: assert on actual data, not just file presence - Add/configure .mcp.json for NuGet MCP server - Minor fixes: exit code handling, error messages, and CoverletTestSessionHandler interface compliance
1 parent e4898ab commit ae21a60

File tree

5 files changed

+1266
-769
lines changed

5 files changed

+1266
-769
lines changed

‎.mcp.json‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
{
2+
"servers": {
3+
"nuget": {
4+
"type": "stdio",
5+
"command": "dnx",
6+
"args": [
7+
"NuGet.Mcp.Server",
8+
"--prerelease",
9+
"--yes"
10+
]
11+
}
12+
}
13+
}

‎Documentation/Coverlet.MTP.Integration.md‎

Lines changed: 249 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -7,9 +7,256 @@ More information is available here:
77
- [Microsoft.Testing.Platform overview](https://learn.microsoft.com/en-us/dotnet/core/testing/microsoft-testing-platform-intro?tabs=dotnetcli)
88
- [Microsoft.Testing.Platform extensibility](https://learn.microsoft.com/en-us/dotnet/core/testing/microsoft-testing-platform-architecture-extensions)
99

10-
coverlet.MTP uses MTP interface and implement coverlet.collector functionality.
10+
coverlet.MTP implements coverlet.collector functionality for Microsoft.Testing.Platform.
1111

12-
December 2025: [Microsoft coverage can be used for xunit](https://xunit.net/docs/getting-started/v3/code-coverage-with-mtp) as well.
12+
## Supported Runtime Versions
1313

14+
Since version `8.0.0`:
15+
16+
- .NET Core >= 8.0
17+
18+
## Quick Start
19+
20+
### Installation
21+
22+
Add the `coverlet.MTP` package to your test project:
23+
24+
```bash
25+
dotnet add package coverlet.MTP
26+
```
1427

1528
ToDo: Usage details
29+
30+
A sample project file looks like:
31+
```xml
32+
<Project Sdk="Microsoft.NET.Sdk">
33+
<PropertyGroup>
34+
<TargetFramework>net8.0</TargetFramework>
35+
<IsPackable>false</IsPackable>
36+
<IsTestProject>true</IsTestProject>
37+
<!-- Enable Microsoft Testing Platform -->
38+
<UseMicrosoftTestingPlatformRunner>true</UseMicrosoftTestingPlatformRunner>
39+
<TestingPlatformDotnetTestSupport>true</TestingPlatformDotnetTestSupport>
40+
<OutputType>Exe</OutputType>
41+
</PropertyGroup>
42+
<ItemGroup>
43+
<!-- Use xunit.v3.mtp-v2 for MTP v2.x compatibility -->
44+
<PackageReference Include="xunit.v3.mtp-v2" Version="3.2.1" />
45+
<PackageReference Include="Microsoft.Testing.Platform" Version="2.0.2" />
46+
<PackageReference Include="coverlet.MTP" Version="8.0.0" />
47+
</ItemGroup>
48+
</Project>
49+
```
50+
51+
### Basic Usage
52+
53+
To collect code coverage, run your test executable with the `--coverlet` flag:
54+
55+
```bash
56+
dotnet exec <test-assembly.dll> --coverlet
57+
```
58+
59+
Or using `dotnet test` with MTP enabled projects:
60+
61+
```bash
62+
dotnet test --coverlet
63+
```
64+
65+
After the test run, a `coverage.json` file containing the results will be generated in the current directory.
66+
67+
## Command Line Options
68+
69+
The `coverlet.MTP` extension provides the following command line options. To see all available options, run:
70+
71+
```bash
72+
dotnet exec <test-assembly.dll> --help
73+
```
74+
75+
### Coverage Options
76+
77+
| Option | Description |
78+
|:-------|:------------|
79+
| `--coverlet` | Enable code coverage data collection. |
80+
| `--coverlet-output-format <format>` | Output format(s) for coverage report. Supported formats: `json`, `lcov`, `opencover`, `cobertura`, `teamcity`. Can be specified multiple times. |
81+
| `--coverlet-output <path>` | Output path for coverage files. |
82+
| `--coverlet-include <filter>` | Include assemblies matching filters (e.g., `[Assembly]Type`). Can be specified multiple times. |
83+
| `--coverlet-include-directory <path>` | Include additional directories for instrumentation. Can be specified multiple times. |
84+
| `--coverlet-exclude <filter>` | Exclude assemblies matching filters (e.g., `[Assembly]Type`). Can be specified multiple times. |
85+
| `--coverlet-exclude-by-file <pattern>` | Exclude source files matching glob patterns. Can be specified multiple times. |
86+
| `--coverlet-exclude-by-attribute <attribute>` | Exclude methods/classes decorated with attributes. Can be specified multiple times. |
87+
| `--coverlet-include-test-assembly` | Include test assembly in coverage. |
88+
| `--coverlet-single-hit` | Limit the number of hits to one for each location. |
89+
| `--coverlet-skip-auto-props` | Skip auto-implemented properties. |
90+
| `--coverlet-does-not-return-attribute <attribute>` | Attributes that mark methods as not returning. Can be specified multiple times. |
91+
| `--coverlet-exclude-assemblies-without-sources <value>` | Exclude assemblies without source code. Values: `MissingAll`, `MissingAny`, `None`. |
92+
| `--coverlet-source-mapping-file <path>` | Path to a SourceRootsMappings file. |
93+
94+
### Examples
95+
96+
**Generate coverage in JSON format (default):**
97+
98+
```bash
99+
dotnet exec TestProject.dll --coverlet
100+
```
101+
102+
**Generate coverage in Cobertura format:**
103+
104+
```bash
105+
dotnet exec TestProject.dll --coverlet --coverlet-output-format cobertura
106+
```
107+
108+
**Generate coverage in multiple formats:**
109+
110+
```bash
111+
dotnet exec TestProject.dll --coverlet --coverlet-output-format json --coverlet-output-format cobertura --coverlet-output-format lcov
112+
```
113+
114+
**Specify output directory:**
115+
116+
```bash
117+
dotnet exec TestProject.dll --coverlet --coverlet-output ./coverage/
118+
```
119+
120+
**Include only specific assemblies:**
121+
122+
```bash
123+
dotnet exec TestProject.dll --coverlet --coverlet-include "[MyApp.]"
124+
```
125+
126+
**Exclude test assemblies and specific namespaces:**
127+
128+
```bash
129+
dotnet exec TestProject.dll --coverlet --coverlet-exclude "[.Tests]" --coverlet-exclude "[]MyApp.Generated."
130+
```
131+
132+
**Exclude by attribute:**
133+
134+
```bash
135+
dotnet exec TestProject.dll --coverlet --coverlet-exclude-by-attribute "Obsolete" --coverlet-exclude-by-attribute "GeneratedCode"
136+
```
137+
138+
## Coverage Output
139+
140+
Coverlet can generate coverage results in multiple formats:
141+
142+
- `json` (default) - Coverlet's native JSON format
143+
- `lcov` - LCOV format
144+
- `opencover` - OpenCover XML format
145+
- `cobertura` - Cobertura XML format
146+
- `teamcity` - TeamCity service messages
147+
148+
By default, coverage files are written to the current working directory. Use `--coverlet-output` to specify a different location.
149+
150+
## Filter Expressions
151+
152+
Filter expressions allow fine-grained control over what gets included or excluded from coverage.
153+
154+
**Syntax:** `[Assembly-Filter]Type-Filter`
155+
156+
**Wildcards:**
157+
158+
- `*` matches zero or more characters
159+
- `?` makes the prefixed character optional
160+
161+
**Examples:**
162+
163+
- `[*]*` - All types in all assemblies
164+
- `[coverlet.*]Coverlet.Core.Coverage` - Specific class in matching assemblies
165+
- `[*]Coverlet.Core.Instrumentation.*` - All types in a namespace
166+
- `[coverlet.*.tests?]*` - Assemblies ending with `.test` or `.tests`
167+
168+
Both `--coverlet-include` and `--coverlet-exclude` can be used together, but `--coverlet-exclude` takes precedence.
169+
170+
## Excluding From Coverage
171+
172+
### Using Attributes
173+
174+
Apply the `ExcludeFromCodeCoverage` attribute from `System.Diagnostics.CodeAnalysis` to exclude methods or classes:
175+
176+
```csharp
177+
[ExcludeFromCodeCoverage] public class NotCovered { // This class will be excluded from coverage }
178+
```
179+
180+
Additional attributes can be specified using `--coverlet-exclude-by-attribute`.
181+
182+
### Using Source File Patterns
183+
184+
Exclude source files using glob patterns with `--coverlet-exclude-by-file`:
185+
186+
```bash
187+
dotnet exec TestProject.dll --coverlet --coverlet-exclude-by-file "**/Generated/*.cs"
188+
```
189+
190+
## How It Works
191+
192+
The `coverlet.MTP` extension integrates with the Microsoft Testing Platform using the extensibility model:
193+
194+
1. **Test Host Controller Extension**: Implements `ITestHostProcessLifetimeHandler` to instrument assemblies before tests run and collect coverage after tests complete.
195+
196+
2. **Before Tests Run**:
197+
- Locates the test assembly and referenced assemblies with PDBs
198+
- Instruments assemblies by inserting code to record sequence point hits
199+
200+
3. **After Tests Run**:
201+
- Restores original non-instrumented assemblies
202+
- Reads recorded hit information
203+
- Generates coverage report in the specified format(s)
204+
205+
## Comparison with coverlet.collector (VSTest)
206+
207+
| Feature | coverlet.MTP | coverlet.collector |
208+
|:--------|:-------------|:-------------------|
209+
| Test Platform | Microsoft Testing Platform | VSTest |
210+
| Configuration | Command line options | runsettings file |
211+
| Output Location | Configurable via `--coverlet-output` | TestResults folder |
212+
| Default Format | JSON | Cobertura |
213+
214+
## Known Limitations
215+
216+
- Threshold validation is not yet supported (planned for future releases)
217+
- Report merging is not yet supported (use external tools like `dotnet-coverage` or `reportgenerator`)
218+
219+
> [!TIP]
220+
> **Merging coverage files from multiple test runs:**
221+
>
222+
> Use `dotnet-coverage` tool:
223+
> ```bash
224+
> dotnet-coverage merge coverage/**/coverage.cobertura.xml -f cobertura -o coverage/merged.xml
225+
> ```
226+
>
227+
> Or use `reportgenerator`:
228+
> ```bash
229+
> reportgenerator -reports:"**/*.cobertura.xml" -targetdir:"coverage/report" -reporttypes:"HtmlInline_AzurePipelines_Dark;Cobertura"
230+
> ```
231+
232+
## Troubleshooting
233+
234+
### Enable Diagnostic Output
235+
236+
Use the MTP diagnostic options to get detailed logs:
237+
238+
```bash
239+
dotnet exec TestProject.dll --coverlet --diagnostic --diagnostic-verbosity trace --diagnostic-output-directory ./logs
240+
```
241+
242+
### Debug Coverlet Extension
243+
244+
Set the environment variable to attach a debugger:
245+
246+
```bash
247+
set COVERLET_MTP_DEBUG=1 dotnet exec TestProject.dll --coverlet
248+
```
249+
250+
## Requirements
251+
252+
- .NET 8.0 SDK or newer
253+
- Microsoft.Testing.Platform 2.0.0 or newer
254+
- Test framework with MTP support (e.g., xUnit v3 (xunit.v3.mtp-v2), MSTest v3, NUnit with MTP adapter)
255+
256+
## Related Documentation
257+
258+
- [VSTest Integration](VSTestIntegration.md) - For VSTest-based projects using `coverlet.collector`
259+
- [MSBuild Integration](MSBuildIntegration.md) - For MSBuild-based coverage collection
260+
- [Global Tool](GlobalTool.md) - For standalone coverage collection
261+
- [Known Issues](KnownIssues.md) - Common issues and workarounds
262+

0 commit comments

Comments
 (0)