-
-
Notifications
You must be signed in to change notification settings - Fork 608
Description
Swift Testing is the biggest thing from Apple for testing since Xcode 4 integrated test results in the editor. Nimble's expectations should be supported from Swift Testing.
Unfortunately, there are a couple hurdles preventing us from doing that.
First is actually registering failures with Swift Testing. We can get a basic integration using the Issue.record
static method. However, if we had the ability to directly create and record issues, then we could improve our integration dramatically. I filed swiftlang/swift-testing#474 to record on their end providing that API.
Second, and much more of a blocker, is actually detecting whether the test runner is Swift Testing or XCTest. This is necessary to correctly report errors to either Swift Testing or XCTest.
In the lab I had with the Testing engineers yesterday, they mentioned that I could use Test.current != nil
to detect positively if Swift Testing is being used. However, Test.current
is implemented as a task local value, which means that we can't actually guarantee that Test.current != nil
means that we're using the Swift Testing test runner. For example, if you use Task.detached
from a test, and then check Test.current
inside that detached task, you will get back nil.
Another approach might be to utilize our XCTest test tracking, and if the current test case is nil, then fall back to report to Swift Testing.
A third approach might be to try reporting to both? We'll likely have to adapt NimbleXCTestHandler
to gracefully fail if there isn't an XCTestCase to report to, and similarly engineer the future NimbleSwiftTestingHandler
to gracefully fail.