Skip to content

[DNM] Enable exit test value capturing #1165

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 0 additions & 15 deletions Package.swift
Original file line number Diff line number Diff line change
Expand Up @@ -95,13 +95,6 @@ let package = Package(
return result
}(),

traits: [
.trait(
name: "ExperimentalExitTestValueCapture",
description: "Enable experimental support for capturing values in exit tests"
),
],

dependencies: [
.package(url: "https://github.com/swiftlang/swift-syntax.git", from: "602.0.0-latest"),
],
Expand Down Expand Up @@ -340,14 +333,6 @@ extension Array where Element == PackageDescription.SwiftSetting {
.define("SWT_NO_LIBDISPATCH", .whenEmbedded()),
]

// Unconditionally enable 'ExperimentalExitTestValueCapture' when building
// for development.
if buildingForDevelopment {
result += [
.define("ExperimentalExitTestValueCapture")
]
}

return result
}

Expand Down
66 changes: 51 additions & 15 deletions Sources/Testing/Testing.docc/exit-testing.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,21 +67,7 @@ The parent process doesn't call the body of the exit test. Instead, the child
process treats the body of the exit test as its `main()` function and calls it
directly.

- Note: Because the body acts as the `main()` function of a new process, it
can't capture any state originating in the parent process or from its lexical
context. For example, the following exit test will fail to compile because it
captures a variable declared outside the exit test itself:

```swift
@Test func `Customer won't eat food unless it's nutritious`() async {
let isNutritious = false
await #expect(processExitsWith: .failure) {
var food = ...
food.isNutritious = isNutritious // ❌ ERROR: trying to capture state here
Customer.current.eat(food)
}
}
```
<!-- TODO: discuss @MainActor isolation or lack thereof -->

If the body returns before the child process exits, the process exits as if
`main()` returned normally. If the body throws an error, Swift handles it as if
Expand All @@ -106,6 +92,56 @@ status of the child process against the expected exit condition you passed. If
they match, the exit test passes; otherwise, it fails and the testing library
records an issue.

### Capture state from the parent process

To pass information from the parent process to the child process, you specify
the Swift values you want to pass in a [capture list](https://docs.swift.org/swift-book/documentation/the-swift-programming-language/closures/#Capturing-Values)
on the exit test's body:

```swift
@Test(arguments: Food.allJunkFood)
func `Customer won't eat food unless it's nutritious`(_ food: Food) async {
await #expect(processExitsWith: .failure) { [food] in
Customer.current.eat(food)
}
}
```

If a captured value is an argument to the current function or is `self`, its
type is inferred at compile time. Otherwise, explicitly specify the type of the
value using the `as` operator:

```swift
@Test func `Customer won't eat food unless it's nutritious`() async {
var food = ...
food.isNutritious = false
await #expect(processExitsWith: .failure) { [self, food = food as Food] in
self.prepare(food)
Customer.current.eat(food)
}
}
```

Every value you capture in an exit test must conform to [`Sendable`](https://developer.apple.com/documentation/swift/sendable)
and [`Codable`](https://developer.apple.com/documentation/swift/codable). Each
value is encoded by the parent process using [`encode(to:)`](https://developer.apple.com/documentation/swift/encodable/encode(to:))
and is decoded by the child process [`init(from:)`](https://developer.apple.com/documentation/swift/decodable/init(from:))
before being passed to the exit test body.

If a captured value's type does not conform to both `Sendable` and `Codable`, or
if the value is not explicitly specified in the exit test body's capture list,
the compiler emits an error:

```swift
@Test func `Customer won't eat food unless it's nutritious`() async {
var food = ...
food.isNutritious = false
await #expect(processExitsWith: .failure) {
Customer.current.eat(food) // ❌ ERROR: implicitly capturing 'food'
}
}
```

### Gather output from the child process

The ``expect(processExitsWith:observing:_:sourceLocation:performing:)`` and
Expand Down
25 changes: 0 additions & 25 deletions Sources/TestingMacros/ConditionMacro.swift
Original file line number Diff line number Diff line change
Expand Up @@ -630,15 +630,6 @@ extension ExitTestConditionMacro {
) -> Bool {
var diagnostics = [DiagnosticMessage]()

if let closureExpr = bodyArgumentExpr.as(ClosureExprSyntax.self),
let captureClause = closureExpr.signature?.capture,
!captureClause.items.isEmpty {
// Disallow capture lists if the experimental feature is not enabled.
if !ExitTestExpectMacro.isValueCapturingEnabled {
diagnostics.append(.captureClauseUnsupported(captureClause, in: closureExpr, inExitTest: macro))
}
}

// Disallow exit tests in generic types and functions as they cannot be
// correctly expanded due to the use of a nested type with static members.
for lexicalContext in context.lexicalContext {
Expand All @@ -664,22 +655,6 @@ extension ExitTestConditionMacro {
}
}

extension ExitTestExpectMacro {
/// Whether or not experimental value capturing via explicit capture lists is
/// enabled.
///
/// This member is declared on ``ExitTestExpectMacro`` but also applies to
/// ``ExitTestRequireMacro``.
@TaskLocal
static var isValueCapturingEnabled: Bool = {
#if ExperimentalExitTestValueCapture
return true
#else
return false
#endif
}()
}

/// A type describing the expansion of the `#expect(processExitsWith:)` macro.
///
/// This type checks for nested invocations of `#expect()` and `#require()` and
Expand Down
44 changes: 0 additions & 44 deletions Sources/TestingMacros/Support/DiagnosticMessage.swift
Original file line number Diff line number Diff line change
Expand Up @@ -845,50 +845,6 @@ extension DiagnosticMessage {
)
}

/// Create a diagnostic message stating that a capture clause cannot be used
/// in an exit test.
///
/// - Parameters:
/// - captureClause: The invalid capture clause.
/// - closure: The closure containing `captureClause`.
/// - exitTestMacro: The containing exit test macro invocation.
///
/// - Returns: A diagnostic message.
static func captureClauseUnsupported(_ captureClause: ClosureCaptureClauseSyntax, in closure: ClosureExprSyntax, inExitTest exitTestMacro: some FreestandingMacroExpansionSyntax) -> Self {
let changes: [FixIt.Change]
if let signature = closure.signature,
Array(signature.with(\.capture, nil).tokens(viewMode: .sourceAccurate)).count == 1 {
// The only remaining token in the signature is `in`, so remove the whole
// signature tree instead of just the capture clause.
changes = [
.replaceTrailingTrivia(token: closure.leftBrace, newTrivia: ""),
.replace(
oldNode: Syntax(signature),
newNode: Syntax("" as ExprSyntax)
)
]
} else {
changes = [
.replace(
oldNode: Syntax(captureClause),
newNode: Syntax("" as ExprSyntax)
)
]
}

return Self(
syntax: Syntax(captureClause),
message: "Cannot specify a capture clause in closure passed to \(_macroName(exitTestMacro))",
severity: .error,
fixIts: [
FixIt(
message: MacroExpansionFixItMessage("Remove '\(captureClause.trimmed)'"),
changes: changes
),
]
)
}

/// Create a diagnostic message stating that an expression macro is not
/// supported in a generic context.
///
Expand Down
33 changes: 5 additions & 28 deletions Tests/TestingMacrosTests/ConditionMacroTests.swift
Original file line number Diff line number Diff line change
Expand Up @@ -452,7 +452,6 @@ struct ConditionMacroTests {
}
}

#if ExperimentalExitTestValueCapture
@Test("#expect(processExitsWith:) produces a diagnostic for a bad capture",
arguments: [
"#expectExitTest(processExitsWith: x) { [weak a] in }":
Expand All @@ -470,34 +469,12 @@ struct ConditionMacroTests {
]
)
func exitTestCaptureDiagnostics(input: String, expectedMessage: String) throws {
try ExitTestExpectMacro.$isValueCapturingEnabled.withValue(true) {
let (_, diagnostics) = try parse(input)

#expect(diagnostics.count > 0)
for diagnostic in diagnostics {
#expect(diagnostic.diagMessage.severity == .error)
#expect(diagnostic.message == expectedMessage)
}
}
}
#endif
let (_, diagnostics) = try parse(input)

@Test(
"Capture list on an exit test produces a diagnostic",
arguments: [
"#expectExitTest(processExitsWith: x) { [a] in }":
"Cannot specify a capture clause in closure passed to '#expectExitTest(processExitsWith:_:)'"
]
)
func exitTestCaptureListProducesDiagnostic(input: String, expectedMessage: String) throws {
try ExitTestExpectMacro.$isValueCapturingEnabled.withValue(false) {
let (_, diagnostics) = try parse(input)

#expect(diagnostics.count > 0)
for diagnostic in diagnostics {
#expect(diagnostic.diagMessage.severity == .error)
#expect(diagnostic.message == expectedMessage)
}
#expect(diagnostics.count > 0)
for diagnostic in diagnostics {
#expect(diagnostic.diagMessage.severity == .error)
#expect(diagnostic.message == expectedMessage)
}
}

Expand Down
2 changes: 0 additions & 2 deletions Tests/TestingTests/ExitTestTests.swift
Original file line number Diff line number Diff line change
Expand Up @@ -381,7 +381,6 @@ private import _TestingInternals
}
}

#if ExperimentalExitTestValueCapture
@Test("Capture list")
func captureList() async {
let i = 123
Expand Down Expand Up @@ -559,7 +558,6 @@ private import _TestingInternals
}
}
#endif
#endif
}

// MARK: - Fixtures
Expand Down