xUnit v3 upgrade by aaronpowell · Pull Request #960 · CommunityToolkit/Aspire

aaronpowell · 2025-11-11T05:04:05Z

Upgrading the tests to use xUnit v3, which is what is used by aspire now too.

Close #961

Co-authored-by: aaronpowell <434140+aaronpowell@users.noreply.github.com>

…ET 10 Co-authored-by: aaronpowell <434140+aaronpowell@users.noreply.github.com>

Co-authored-by: aaronpowell <434140+aaronpowell@users.noreply.github.com>

… but the MassTransit.ActiveMQ package isn't updated yet Tracked: MassTransit/MassTransit#6133

…ss ambiguous referencing

…able

I feel dirty doing that...

Youssef1313 · 2025-11-16T22:11:32Z

.github/workflows/tests.yaml

      - name: Run tests
        run: >-
-          dotnet test ${{ github.workspace }}/${{ env.TEST_PROJECT_PATH }}
+          dotnet run ${{ github.workspace }}/${{ env.TEST_PROJECT_PATH }}


You need to:

Remove --logger "console;verbosity=normal".

Replace --logger trx with --report-trx (and ensure Microsoft.Testing.Extensions.TrxReport is referenced)

Replace all the blame-related args with --crashdump, --hangdump, and --hangdump-timeout 7m

--results-directory is changing the meaning a little bit with this PR. Previously it would be relative to current working directory ${{ github.workspace }} while with this PR it will be relative to the test executable. So either pass the full path, or keep using dotnet test which will handle the path transformation for you.

--collect "XPlat Code Coverage" needs to be replaced with --coverage (and ensure Microsoft.Testing.Extensions.CodeCoverage is referenced)

Youssef1313 · 2025-11-16T22:11:51Z

.github/workflows/tests.yaml

+          --report-trx
+          --report-trx-filename "${{ matrix.name }}-${{ matrix.os }}.trx"
+          --ignore-exit-code 8
+          -- RunConfiguration.CollectSourceInformation=true


This shouldn't be needed.

Youssef1313 · 2025-11-16T22:12:31Z

Directory.Packages.props

-    <PackageVersion Include="xunit.runner.visualstudio" Version="2.8.2" />
-    <PackageVersion Include="xunit.extensibility.execution" Version="2.9.3" />
-    <PackageVersion Include="Microsoft.DotNet.XUnitExtensions" Version="9.0.0-beta.24568.1" />
+    <PackageVersion Include="xunit.v3" Version="3.2.0" />


nit: there is now xunit.v3.mtp-v2 package.

Youssef1313 · 2025-11-16T22:12:48Z

Directory.Packages.props

+    <PackageVersion Include="xunit.v3" Version="3.2.0" />
+    <PackageVersion Include="xunit.v3.assert" Version="3.2.0" />
+    <PackageVersion Include="xunit.v3.extensibility.core" Version="3.2.0" />
+    <PackageVersion Include="xunit.runner.visualstudio" Version="3.1.5" />


nit: no longer needed if you don't care about VSTest support.

Suggested change

<PackageVersion Include="xunit.runner.visualstudio" Version="3.1.5" />

Youssef1313 · 2025-11-16T22:13:33Z

tests/.runsettings

  <RunConfiguration>
    <!-- Filter out failing (wrong framework, platform, runtime or activeissue) tests -->
-    <TestCaseFilter>category!=unsupported-platform</TestCaseFilter>
+    <TestCaseFilter>category!=failing</TestCaseFilter>


runsettings isn't supported with MTP.

Youssef1313 · 2025-11-16T22:14:49Z

tests/Directory.Build.props

+         https://learn.microsoft.com/dotnet/core/testing/microsoft-testing-platform-exit-codes -->
+    <TestRunnerAdditionalArguments>$(TestRunnerAdditionalArguments) --ignore-exit-code 8</TestRunnerAdditionalArguments>
+
+    <!-- <UseMicrosoftTestingPlatformRunner>true</UseMicrosoftTestingPlatformRunner> -->


Youssef1313

You need to also update global.json to specify the test runner as MTP, and probably ensure it requires .NET 10.

It seems there are two different paths for resolving the reference expression that _should_ return the same, but in the tests they don't seem to, so we'll force eval

Evangelink · 2025-11-26T15:00:08Z

@aaronpowell please feel free ping @Youssef1313 or I so we can move forward here.

…/Aspire into xunit-3-upgrade

…ng the app host

Evangelink · 2025-12-11T12:50:29Z

examples/powershell/CommunityToolkit.Aspire.PowerShell.AppHost/Program.cs

 // outputs "the sum of 2 and 3 is 5"
 var script2 = ps.AddScript("script2", """
-    & ./scripts/script.ps1 @args
+    & ./Scripts/script.ps1 @args


I don't think that Powershell is case sensitive (cc @nohwnd)

This is up to the file system not up to powershell. If this needs to run on linux and the filesystem is case sensitive then ./scripts and ./Scripts are two different paths. On windows we typically don't use case sensitive file systems.

https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_case-sensitivity?view=powershell-7.5

Evangelink · 2025-12-11T12:51:35Z

tests/CommunityToolkit.Aspire.Hosting.McpInspector.Tests/AppHostTests.cs


 // Forcing linux only due to: https://github.com/modelcontextprotocol/inspector/issues/893
-[RequiresLinux]
+[RequiresWindows]


How was this working before?

I have no idea why I made that change, I'm going to revert it 🤣

Evangelink

@aaronpowell the latest errors are weird, it's complaining about a file that doesn't exist when I pull this branch.

I have also noticed the following missing changes:

In global.json, add:

"test": {
    "runner": "Microsoft.Testing.Platform"
  }

In Directory.Packages.props and tests/Directory.Build.props remove:

coverlet.collector -> not compatible with MTP, instead use MS coverage
Microsoft.NET.Test.Sdk -> VSTest testhost part, no longer needed as you are moving to MTP
xunit.runner.visualstudio -> VSTest adapter for xUnit, also no longer needed.

I'd suggest to be careful with the --ignore-exit-code 8 set on all test projects because you are allowing all projects not to run tests without failure. I don't know if you have some other guard in place but if you don't then you would potentially no longer run any test without noticing.

…/Aspire into xunit-3-upgrade

The PowerShell Runspace doesn't when the app host does, which I think is the underlying cause as to why the tests hang. This now monitors the script resources so that when they complete we force a dispose on them (thus freeing any process they may have used) and once all are completed the Runspace is disposed, releasing it and **I think** that should allow the test host to terminate

This reverts commit 1d1b730.

Xunit 3 upgrade hang fix

Copilot AI and others added 19 commits November 6, 2025 05:00

Initial plan

4e9036e

Add .NET 10 (preview) as a TFM for multi-targeting builds

7236d38

Co-authored-by: aaronpowell <434140+aaronpowell@users.noreply.github.com>

Change default TFM to .NET 10 and update GitHub Actions to install .N…

303a76f

…ET 10 Co-authored-by: aaronpowell <434140+aaronpowell@users.noreply.github.com>

Update devcontainer to install .NET 10 as preview

c6b19fc

Co-authored-by: aaronpowell <434140+aaronpowell@users.noreply.github.com>

Add .NET 10 to setup-runtimes-caching action

1e4e20d

Co-authored-by: aaronpowell <434140+aaronpowell@users.noreply.github.com>

Fixing some versioning

ac50650

Hard coding an update to use Apache.NMS.AMQP@2.4.0 as 2.2.0 has a CVE…

9441794

… but the MassTransit.ActiveMQ package isn't updated yet Tracked: MassTransit/MassTransit#6133

Fixing some projects that specified their own TFM and trying to addre…

ef89379

…ss ambiguous referencing

Using an improved approach approach to adding system.linq.asyncenumer…

e66de71

…able

Updating how dev certs are setup

51e57ba

Turns out you can't upgrade around the CVE, so we now suppress it

f3ff1ff

I feel dirty doing that...

Update to xunit v3

df16921

Disabling JSON schema tests as we don't generate those files

1ac2ad3

Some adjustments to not be so aspire configured

5840c2e

Oh, some of that was used

dcfbd73

Adding filter for failing tests to be skipped

102aaab

using dotnet <assembly> to run tests rather than dotnet test

733964b

forgot I made that change

f10b449

updating a few more deps

425a39e

aaronpowell self-assigned this Nov 11, 2025

aaronpowell added the awaiting response Waiting for the author of the issue to provide more information or answer a question label Nov 11, 2025

Base automatically changed from copilot/add-dotnet-10-tfm-support to main November 15, 2025 20:52

Youssef1313 reviewed Nov 16, 2025

View reviewed changes

Merge branch 'main' into xunit-3-upgrade

58b4b33

aaronpowell added 2 commits November 25, 2025 12:14

Removing reference to runsettings

239aca3

Performing an earlier eval of the connection string

0ab87ce

It seems there are two different paths for resolving the reference expression that _should_ return the same, but in the tests they don't seem to, so we'll force eval

aaronpowell added 6 commits November 27, 2025 10:39

Merge branch 'main' into xunit-3-upgrade

70133ad

Merge branch 'main' into xunit-3-upgrade

f925398

Merge branch 'xunit-3-upgrade' of https://github.com/CommunityToolkit…

d7d256d

…/Aspire into xunit-3-upgrade

Adjusting msbuild props

8273771

Linux file systems are case sensitive - why has this ever worked?

fb3743d

The path when the scripts run during tests is different to just runni…

91576a2

…ng the app host

Evangelink reviewed Dec 11, 2025

View reviewed changes

Allowing mcp tests to run on all OS's

3a0368b

Evangelink reviewed Dec 18, 2025

View reviewed changes

aaronpowell and others added 13 commits December 19, 2025 10:10

Merge branch 'main' into xunit-3-upgrade

51d39e6

Fixing copiler error

0b37f60

Updated traits

c3fab49

Merge branch 'main' into xunit-3-upgrade

7974f67

Merge branch 'main' into xunit-3-upgrade

ad34cdb

Merge branch 'main' into xunit-3-upgrade

a099502

Fixing compiler errors

4f896d3

Merge branch 'xunit-3-upgrade' of https://github.com/CommunityToolkit…

f1d373a

…/Aspire into xunit-3-upgrade

Fixing a few failing test suites

6107c68

Revert "Ensuring PowerShell terminates properly"

b93dc7c

This reverts commit 1d1b730.

hook applicationstopping on host lifecycle to shutdown RS pool

7ed360d

Merge pull request #1113 from oising/xunit-3-upgrade

15e7108

Xunit 3 upgrade hang fix

github-actions bot added the Stale label Jan 25, 2026

github-actions bot closed this Jan 28, 2026

aaronpowell reopened this Feb 2, 2026

aaronpowell removed the Stale label Feb 2, 2026

Conversation

aaronpowell commented Nov 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Youssef1313 left a comment

Choose a reason for hiding this comment

Uh oh!

Evangelink commented Nov 26, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Evangelink left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

aaronpowell commented Nov 11, 2025 •

edited

Loading