Document scope of applicability for `TestWatcher` implementations

[wazzeps]

Referring to #2560

Steps to reproduce

	@Disabled
	@ParameterizedTest
	@ValueSource(strings = {"a", "b"})
	void testParam(String s) {
		System.out.println(s);
	}

With TestWatcher implementation

	@Override
	public void testDisabled(ExtensionContext context, Optional<String> reason) {
		System.out.println("DISABLED");
	}

Expected: 2 outputs "DISABLED"
Actual: no output, testDisabled method wasn't called

Context

• junit-jupiter-api:5.9.2
• junit-jupiter-engine:5.9.2
• Gradle
• IntelliJ IDEA

[sbrannen]

I have confirmed that disabled events are not supplied to a TestWatcher for @TestTemplate methods (e.g., @RepeatedTest and @ParameterizedTest, and I have changed the title of this issue to reflect that.

The following can be used to reproduce the issue.

class WatcherTests {

	@RegisterExtension
	Extension watcher = new TestWatcher() {

		@Override
		public void testSuccessful(ExtensionContext context) {
			System.err.println("SUCCESSFUL: " + context.getRequiredTestMethod().getName());
		}

		@Override
		public void testDisabled(ExtensionContext context, Optional<String> reason) {
			System.err.println("DISABLED  : " + context.getRequiredTestMethod().getName());
		}
	};

	@Disabled
	@Test
	void test() {
	}

	@Disabled
	@RepeatedTest(2)
	void repeatedTest() {
	}

	@Disabled
	@ParameterizedTest
	@ValueSource(strings = { "a", "b" })
	void parameterizedTest(String s) {
	}

}

Since the documentation states the following, I am labeling this as a bug.

Please note that this API is currently only used to report the results of @test methods and @TestTemplate methods (e.g., @RepeatedTest and @ParameterizedTest).

Thanks for raising the issue, @wazzeps.

[sbrannen]

Oops. I retract that.

Everything works fine if the TestWatcher is registered at the class level.

For example, changing the declaration to static as follows causes the disabled events to be reported as expected.

	@RegisterExtension
	static Extension watcher = new TestWatcher() { // ...

I am therefore repurposing this issue to improve the documentation for TestWatcher to make users of the API aware of the different semantics when a TestWatcher is not registered at the class level.