-
Notifications
You must be signed in to change notification settings - Fork 684
Rename test env controlling and project test capabilities variables #8518
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
7 commits
Select commit
Hold shift + click to select a range
36a36e2
Rename test env controlling and project test capabilities variables
RussKie a9ebfd3
fixup! Rename test env controlling and project test capabilities vari…
RussKie 1cf2f17
fixup! Rename test env controlling and project test capabilities vari…
RussKie 874a295
fixup! Rename test env controlling and project test capabilities vari…
RussKie 7bd832f
Update eng/Testing.targets
RussKie 6b5a19e
fixup! Update eng/Testing.targets
RussKie 832bf9c
fixup! Update eng/Testing.targets
RussKie File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,3 +1,96 @@ | ||
<Project> | ||
|
||
<!-- | ||
Environment variables: | ||
- IsGitHubActionsRunner: indicates whether tests are currently run on GitHub Actions; computed, overridable. Locally this can be set by "/p:GITHUB_ACTIONS=true". | ||
- IsAzdoCIRunner: indicates whether tests are currently run on Azure DevOps; computed, overridable. Locally this can be set by "/p:SYSTEM_TEAMPROJECT=foo". | ||
- IsAzdoHelixRunner: indicates whether tests are currently run on Helix; computed, overridable. Locally this can be set by "/p:PrepareForHelix=true". | ||
|
||
Project capabilities: | ||
- IsTestProject: indicates whether the project is a test project; default is false; computed, overridable. | ||
- IncludeTestUtilities: indicates whether the test project must not include the TestUtilities project reference; default is false; overridable. | ||
|
||
Project requirements: | ||
- RunOnGithubActions: indicates whether tests should run on GitHub Actions (either Windows or Linux); computed. | ||
- RunOnGithubActionsWindows: indicates whether tests should run on Windows in GitHub Actions; default is true; overridable. | ||
- RunOnGithubActionsLinux: indicates whether tests should run on Linux in GitHub Actions; default is true; overridable. | ||
- RunOnAzdoCI: indicates whether tests should run on Azure DevOps (either Windows or Linux); always false, if RunOnAzdoHelix=true; computed. | ||
- RunOnAzdoCIWindows: indicates whether tests should run on Windows in Azure DevOps; default is true; overridable. | ||
- RunOnAzdoCILinux: indicates whether tests should run on Linux in Azure DevOps; default is true; overridable. | ||
- RunOnAzdoHelix: indicates whether tests should run on Helix (either Windows or Linux); computed. | ||
- RunOnAzdoHelixWindows: indicates whether tests should run on Windows in Helix; default is true; overridable. | ||
- RunOnAzdoHelixLinux: indicates whether tests should run on Linux in Helix; default is true; overridable. | ||
--> | ||
|
||
<PropertyGroup> | ||
<!-- See https://docs.github.com/actions/writing-workflows/choosing-what-your-workflow-does/store-information-in-variables#default-environment-variables --> | ||
<IsGitHubActionsRunner Condition=" '$(GITHUB_ACTIONS)' == 'true' ">true</IsGitHubActionsRunner> | ||
|
||
<!-- See https://learn.microsoft.com/azure/devops/pipelines/build/variables#system-variables --> | ||
<IsAzdoCIRunner Condition=" '$(SYSTEM_TEAMPROJECT)' != '' ">true</IsAzdoCIRunner> | ||
|
||
<IsAzdoHelixRunner Condition=" '$(PrepareForHelix)' == 'true' ">true</IsAzdoHelixRunner> | ||
</PropertyGroup> | ||
|
||
<PropertyGroup> | ||
<RunOnGithubActions>false</RunOnGithubActions> | ||
<RunOnGithubActions Condition=" '$(RunOnGithubActionsWindows)' == 'true' or '$(RunOnGithubActionsLinux)' == 'true' ">true</RunOnGithubActions> | ||
|
||
<RunOnAzdoHelix>false</RunOnAzdoHelix> | ||
<RunOnAzdoHelix Condition=" '$(RunOnAzdoHelixWindows)' == 'true' or '$(RunOnAzdoHelixLinux)' == 'true' ">true</RunOnAzdoHelix> | ||
|
||
<RunOnAzdoCI>false</RunOnAzdoCI> | ||
<RunOnAzdoCI Condition=" '$(RunOnAzdoCIWindows)' == 'true' or '$(RunOnAzdoCILinux)' == 'true' ">true</RunOnAzdoCI> | ||
<!-- If a test is run on Helix, then we don't run the test on AzDO --> | ||
<RunOnAzdoCI Condition=" '$(RunOnAzdoHelix)' == 'true' ">false</RunOnAzdoCI> | ||
</PropertyGroup> | ||
|
||
<PropertyGroup Condition=" '$(SkipTests)' == '' and '$(IsTestProject)' == 'true' "> | ||
<!-- Skip tests by default unless explicitly set to false --> | ||
<SkipTests>true</SkipTests> | ||
|
||
<!-- Only run tests if the build is running on GitHub Actions --> | ||
<SkipTests Condition=" '$(IsGitHubActionsRunner)' == 'true' and '$(RunOnGithubActions)' == 'true' ">false</SkipTests> | ||
|
||
<!-- Only run tests if the build is running on Helix infra --> | ||
<SkipTests Condition=" '$(IsAzdoHelixRunner)' == 'true' and '$(RunOnAzdoHelix)' == 'true' ">false</SkipTests> | ||
|
||
<!-- Otherwise, run tests on AzDO CI agents --> | ||
<SkipTests Condition=" '$(IsAzdoCIRunner)' == 'true' and '$(RunOnAzdoCI)' == 'true' ">false</SkipTests> | ||
</PropertyGroup> | ||
|
||
<ItemGroup Condition=" '$(IsTestProject)' == 'true' and '$(IncludeTestUtilities)' == 'true' "> | ||
<ProjectReference Include="$(RepoRoot)tests\Aspire.TestUtilities\Aspire.TestUtilities.csproj" /> | ||
</ItemGroup> | ||
|
||
<!-- | ||
The following target is used to announce the test capabilities of the project. | ||
--> | ||
<Target Name="_AnnounceProjectTestCapabilities" BeforeTargets="RunTests" Condition="'$(IsTestProject)' == 'true'"> | ||
<PropertyGroup> | ||
<_IsGitHubActionsRunner>$(IsGitHubActionsRunner)</_IsGitHubActionsRunner> | ||
<_IsGitHubActionsRunner Condition="'$(_IsGitHubActionsRunner)' == ''">false</_IsGitHubActionsRunner> | ||
<_IsAzdoCIRunner>$(IsAzdoCIRunner)</_IsAzdoCIRunner> | ||
<_IsAzdoCIRunner Condition="'$(_IsAzdoCIRunner)' == ''">false</_IsAzdoCIRunner> | ||
<_IsAzdoHelixRunner>$(IsAzdoHelixRunner)</_IsAzdoHelixRunner> | ||
<_IsAzdoHelixRunner Condition="'$(_IsAzdoHelixRunner)' == ''">false</_IsAzdoHelixRunner> | ||
</PropertyGroup> | ||
<ItemGroup> | ||
<_Runner Include=" - GitHub Actions: $(_IsGitHubActionsRunner)" /> | ||
<_Runner Include=" - Azure DevOps: $(_IsAzdoCIRunner)" /> | ||
<_Runner Include=" - Helix: $(_IsAzdoHelixRunner)" /> | ||
<_Requirement Include=" - GitHub Actions: $(RunOnGithubActions) (Windows: $(RunOnGithubActionsWindows) / Linux: $(RunOnGithubActionsLinux))" /> | ||
<_Requirement Include=" - Azure DevOps: $(RunOnAzdoCI) (Windows: $(RunOnAzdoCIWindows) / Linux: $(RunOnAzdoCILinux))" /> | ||
<_Requirement Include=" - Helix: $(RunOnAzdoHelix) (Windows: $(RunOnAzdoHelixWindows) / Linux: $(RunOnAzdoHelixLinux))" /> | ||
</ItemGroup> | ||
|
||
<PropertyGroup> | ||
<_NewLine>%0D%0A</_NewLine> | ||
<_RunnerList>@(_Runner, '%0D%0A')</_RunnerList> | ||
<_RequirementList>@(_Requirement, '$(_NewLine)')</_RequirementList> | ||
</PropertyGroup> | ||
|
||
<Message Text="Project: $(MSBuildProjectName)$(_NewLine)Runner Context:$(_NewLine)$(_RunnerList)$(_NewLine)Runner support:$(_NewLine)$(_RequirementList)" /> | ||
</Target> | ||
|
||
</Project> |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this logic correct?
Shouldn't
RunOnGithubActionsWindows
be considered only when on Windows, andRunOnGithubActionsLinux
be considered only on Linux, instead of ORing them together without any OS checks?Currently, it seems like Playground is run on GitHub Actions on Windows (VSTest just ignores when no tests are run, but with my PR for MTP I'm getting zero tests ran for Playground)
Could you please double check @RussKie? Am I missing something here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, the logic is correct.
These variables denote the "capabilities":
aspire/eng/Testing.targets
Lines 13 to 22 in ac098f7
...and used to generate runsheets, which then executed on individual build agents.
You can see how it is being used in https://github.com/dotnet/aspire/pull/8618/files#diff-ce47bb41900ee3082587fce1184e86392a2c25a05882281792f04e8e8abc51f4.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
After a third look and two cups of tea later, I see the issue now. Thank you for pointing it out.
SkipTests
should be OS-aware, so that we don't run a test on Windows, if the test project declared it couldn't be run on Windows. This is getting fixed in #8687.