|
| 1 | +# Range-based confirmations |
| 2 | + |
| 3 | +* Proposal: [SWT-NNNN](NNNN-ranged-confirmations.md) |
| 4 | +* Authors: [Jonathan Grynspan](https://github.com/grynspan) |
| 5 | +* Status: **Awaiting review** |
| 6 | +* Implementation: [swiftlang/swift-testing#598](https://github.com/swiftlang/swift-testing/pull/598), [swiftlang/swift-testing#689](https://github.com/swiftlang/swift-testing/pull689) |
| 7 | +* Review: TBD |
| 8 | + |
| 9 | +## Introduction |
| 10 | + |
| 11 | +Swift Testing includes [an interface](https://swiftpackageindex.com/swiftlang/swift-testing/main/documentation/testing/confirmation(_:expectedcount:isolation:sourcelocation:_:)) |
| 12 | +for checking that some asynchronous event occurs a given number of times |
| 13 | +(typically exactly once or never at all.) This proposal enhances that interface |
| 14 | +to allow arbitrary ranges of event counts so that a test can be written against |
| 15 | +code that may not always fire said event the exact same number of times. |
| 16 | + |
| 17 | +## Motivation |
| 18 | + |
| 19 | +Some tests rely on fixtures or external state that is not perfectly |
| 20 | +deterministic. For example, consider a test that checks that clicking the mouse |
| 21 | +button will generate a `.mouseClicked` event. Such a test might use the |
| 22 | +`confirmation()` interface: |
| 23 | + |
| 24 | +```swift |
| 25 | +await confirmation(expectedCount: 1) { mouseClicked in |
| 26 | + var eventLoop = EventLoop() |
| 27 | + eventLoop.eventHandler = { event in |
| 28 | + if event == .mouseClicked { |
| 29 | + mouseClicked() |
| 30 | + } |
| 31 | + } |
| 32 | + await eventLoop.simulate(.mouseClicked) |
| 33 | +} |
| 34 | +``` |
| 35 | + |
| 36 | +But what happens if the user _actually_ clicks a mouse button while this test is |
| 37 | +running? That might trigger a _second_ `.mouseClicked` event, and then the test |
| 38 | +will fail spuriously. |
| 39 | + |
| 40 | +## Proposed solution |
| 41 | + |
| 42 | +If the test author could instead indicate to Swift Testing that their test will |
| 43 | +generate _one or more_ events, they could avoid spurious failures: |
| 44 | + |
| 45 | +```swift |
| 46 | +await confirmation(expectedCount: 1...) { mouseClicked in |
| 47 | + ... |
| 48 | +} |
| 49 | +``` |
| 50 | + |
| 51 | +With this proposal, we add an overload of `confirmation()` that takes any range |
| 52 | +expression instead of a single integer value (which is still accepted via the |
| 53 | +existing overload.) |
| 54 | + |
| 55 | +## Detailed design |
| 56 | + |
| 57 | +A new overload of `confirmation()` is added: |
| 58 | + |
| 59 | +```swift |
| 60 | +/// Confirm that some event occurs during the invocation of a function. |
| 61 | +/// |
| 62 | +/// - Parameters: |
| 63 | +/// - comment: An optional comment to apply to any issues generated by this |
| 64 | +/// function. |
| 65 | +/// - expectedCount: A range of integers indicating the number of times the |
| 66 | +/// expected event should occur when `body` is invoked. |
| 67 | +/// - isolation: The actor to which `body` is isolated, if any. |
| 68 | +/// - sourceLocation: The source location to which any recorded issues should |
| 69 | +/// be attributed. |
| 70 | +/// - body: The function to invoke. |
| 71 | +/// |
| 72 | +/// - Returns: Whatever is returned by `body`. |
| 73 | +/// |
| 74 | +/// - Throws: Whatever is thrown by `body`. |
| 75 | +/// |
| 76 | +/// Use confirmations to check that an event occurs while a test is running in |
| 77 | +/// complex scenarios where `#expect()` and `#require()` are insufficient. For |
| 78 | +/// example, a confirmation may be useful when an expected event occurs: |
| 79 | +/// |
| 80 | +/// - In a context that cannot be awaited by the calling function such as an |
| 81 | +/// event handler or delegate callback; |
| 82 | +/// - More than once, or never; or |
| 83 | +/// - As a callback that is invoked as part of a larger operation. |
| 84 | +/// |
| 85 | +/// To use a confirmation, pass a closure containing the work to be performed. |
| 86 | +/// The testing library will then pass an instance of ``Confirmation`` to the |
| 87 | +/// closure. Every time the event in question occurs, the closure should call |
| 88 | +/// the confirmation: |
| 89 | +/// |
| 90 | +/// ```swift |
| 91 | +/// let minBuns = 5 |
| 92 | +/// let maxBuns = 10 |
| 93 | +/// await confirmation( |
| 94 | +/// "Baked between \(minBuns) and \(maxBuns) buns", |
| 95 | +/// expectedCount: minBuns ... maxBuns |
| 96 | +/// ) { bunBaked in |
| 97 | +/// foodTruck.eventHandler = { event in |
| 98 | +/// if event == .baked(.cinnamonBun) { |
| 99 | +/// bunBaked() |
| 100 | +/// } |
| 101 | +/// } |
| 102 | +/// await foodTruck.bakeTray(of: .cinnamonBun) |
| 103 | +/// } |
| 104 | +/// ``` |
| 105 | +/// |
| 106 | +/// When the closure returns, the testing library checks if the confirmation's |
| 107 | +/// preconditions have been met, and records an issue if they have not. |
| 108 | +/// |
| 109 | +/// If an exact count is expected, use |
| 110 | +/// ``confirmation(_:expectedCount:isolation:sourceLocation:_:)`` instead. |
| 111 | +public func confirmation<R>( |
| 112 | + _ comment: Comment? = nil, |
| 113 | + expectedCount: some RangeExpression<Int> & Sendable, |
| 114 | + isolation: isolated (any Actor)? = #isolation, |
| 115 | + sourceLocation: SourceLocation = #_sourceLocation, |
| 116 | + _ body: (Confirmation) async throws -> sending R |
| 117 | +) async rethrows -> R |
| 118 | +``` |
| 119 | + |
| 120 | +### Ranges without lower bounds |
| 121 | + |
| 122 | +Certain types of range, specifically [`PartialRangeUpTo`](https://developer.apple.com/documentation/swift/partialrangeupto) |
| 123 | +and [`PartialRangeThrough`](https://developer.apple.com/documentation/swift/partialrangethrough), |
| 124 | +are valid when used with this new interface, but may have surprising behavior |
| 125 | +because they implicitly include `0`. If a test author writes `...10`, do they |
| 126 | +mean "zero to ten" or "one to ten"? The programmatic meaning is the former, but |
| 127 | +some test authors might mean the latter. If an event does not occur, a test |
| 128 | +using `confirmation()` and this `expectedCount` value would pass when the test |
| 129 | +author meant for it to fail. |
| 130 | + |
| 131 | +Swift Testing will attempt to detect these ambiguous uses of `...n` and `..<n` |
| 132 | +expressions and diagnose them, recommending that test authors explicitly write |
| 133 | +`0` or `1` as a lower bound. |
| 134 | + |
| 135 | +### Unbounded ranges |
| 136 | + |
| 137 | +Finally, an overload is added that takes an "unbounded range" (which is not |
| 138 | +technically a range at all, but… a closure?) This overload is marked unavailable |
| 139 | +because an unbounded range is effectively useless for testing: |
| 140 | + |
| 141 | +```swift |
| 142 | +@available(*, unavailable, message: "Unbounded range '...' has no effect when used with a confirmation.") |
| 143 | +public func confirmation<R>( |
| 144 | + _ comment: Comment? = nil, |
| 145 | + expectedCount: UnboundedRange, |
| 146 | + isolation: isolated (any Actor)? = #isolation, |
| 147 | + sourceLocation: SourceLocation = #_sourceLocation, |
| 148 | + _ body: (Confirmation) async throws -> R |
| 149 | +) async rethrows -> R { |
| 150 | + fatalError("Unsupported") |
| 151 | +} |
| 152 | +``` |
| 153 | + |
| 154 | +## Source compatibility |
| 155 | + |
| 156 | +This change is additive. Existing tests are unaffected. |
| 157 | + |
| 158 | +Code that refers to `confirmation(_:expectedCount:isolation:sourceLocation:_:)` |
| 159 | +by symbol name may need to add a contextual type to disambiguate the two |
| 160 | +overloads at compile time. |
| 161 | + |
| 162 | +## Integration with supporting tools |
| 163 | + |
| 164 | +The type of the associated value `expected` for the `Issue.Kind` case |
| 165 | +`confirmationMiscounted(actual:expected:)` will change from `Int` to |
| 166 | +`any RangeExpression & Sendable`[^1]. Tools that implement event handlers and |
| 167 | +distinguish between `Issue.Kind` cases are advised not to assume the type of |
| 168 | +this value is `Int`. |
| 169 | + |
| 170 | +## Alternatives considered |
| 171 | + |
| 172 | +- Doing nothing. We have identified real-world use cases for this interface |
| 173 | + including in Swift Testing’s own test target. |
| 174 | +- Allowing the use of any value as the `expectedCount` argument so long as it |
| 175 | + conforms to a protocol `ExpectedCount` (we'd have range types and `Int` |
| 176 | + conform by default.) It was unclear what this sort of flexibility would let |
| 177 | + us do, and posed challenges for encoding and decoding events and issues when |
| 178 | + using the JSON event stream interface. |
| 179 | + |
| 180 | +## Acknowledgments |
| 181 | + |
| 182 | +If significant changes or improvements suggested by members of the community |
| 183 | +were incorporated into the proposal as it developed, take a moment here to thank |
| 184 | +them for their contributions. This is a collaborative process, and everyone's |
| 185 | +input should receive recognition! |
| 186 | + |
| 187 | +Generally, you should not acknowledge anyone who is listed as a co-author. |
| 188 | + |
| 189 | +[^1]: In the future, this type will change to |
| 190 | + `any RangeExpression<Int> & Sendable`. Compiler support is required |
| 191 | + ([96960993](rdar://96960993)). |
0 commit comments