k8s-ci-robot merged approved PR with `needs-ok-to-test` label

[janetkuo]

What happened: This PR was merged after /lgtm and /approve: kubernetes-sigs/agent-sandbox#288, even though it hasn't passed tests (still with needs-ok-to-test label)

What you expected to happen: I'd expect the PR not merged until ok-to-test label exists and tests have passed

[janetkuo]

/sig testing

[Pnkcaht]

Solved @janetkuo :)

[cblecker]

@janetkuo Going to answer your question here:
ok-to-test only applies to tests Prow is aware of, and uses branch protection rules to prevent merges if things aren't passing. Looking at the tests on that PR, they appear to be just GitHub actions tests.

Options to address:

1. Convert all your tests into prow jobs, so that prow is fully aware of the jobs and can enforce proper branch protection
2. Manually make prow aware that you're using GitHub actions adding the additional required checks to the tide branch protection config to ensure that they need to pass before merging. For example:

   test-infra/config/prow/config.yaml

   Lines 622 to 628 in 9c058c9

   	kustomize:
   	required_status_checks:
   	contexts:
   	- Lint
   	- Test Linux
   	- Test MacOS
   	- Test Windows

Option 1 would take more work, but preserves the existing test workflows like the rest of the Kubernetes project and handles things automatically. Option 2 is easier, but you need to manually maintain that list to ensure that the tests you care about are respected.

[janetkuo]

@cblecker thanks for the detailed explanation! I didn't configure GitHub actions or any of those tests workflows in the agent-sandbox repo. They seem to come from GitHub Copilot code review, triggered by the PR author.

I suspect that technically anyone could bypass prow tests in other Kubernetes projects via Copilot code review.