fix: add noop fallback to smoke-copilot workflows on non-PR triggers by Copilot · Pull Request #4872 · github/gh-aw-firewall

Copilot · 2026-06-13T14:25:42Z

Smoke copilot workflows run on schedule and workflow_dispatch with no PR context, but their agent prompts unconditionally instructed add_comment on "the current pull request" — causing the safe-outputs layer to report missing_data: pull request number on every scheduled run.

Changes

.github/workflows/smoke-copilot-byok-aoai-apikey.md (primary): restructures the Output section to gate add_comment/add_labels on a PR trigger, and fall back to noop with a PASS/FAIL summary on schedule/workflow_dispatch
smoke-copilot-byok-aoai-entra.md, smoke-copilot-byok.md, smoke-copilot.md, smoke-copilot-pat.md: same fix applied consistently across all copilot smoke variants
scripts/ci/smoke-copilot-workflow.test.ts: new test enforcing the noop fallback pattern is present in all five workflows (mirrors the existing smoke-codex-workflow.test.ts)

The pattern applied matches the already-fixed smoke-codex.md:

**If triggered by a pull request**, call `add_comment` to post a brief comment...

If all tests pass on a pull request trigger:
- Use the `add_labels` safe-output tool to add the label `smoke-copilot-byok-aoai-apikey` to the pull request

**If triggered by workflow_dispatch or schedule** (no PR context), call `noop` with a concise PASS/FAIL summary instead. Do NOT attempt to add pull request comments or labels when there is no pull request.

Update Output sections in all smoke-copilot* workflow prompts to handle schedule/workflow_dispatch triggers gracefully. When the workflow is not triggered by a pull request, the agent now calls `noop` instead of attempting to add a comment/label to a non-existent PR. Fixes the "missing required data: pull request number" error reported when smoke-copilot-byok-aoai-apikey runs on a scheduled trigger. Also adds a test (smoke-copilot-workflow.test.ts) that enforces the noop fallback pattern across all five copilot smoke workflows.

github-actions · 2026-06-13T15:42:10Z

✅ Coverage Check Passed

Overall Coverage

Metric	Base	PR	Delta
Lines	96.60%	96.64%	📈 +0.04%
Statements	96.47%	96.51%	📈 +0.04%
Functions	98.78%	98.78%	➡️ +0.00%
Branches	91.22%	91.25%	📈 +0.03%

📁 Per-file Coverage Changes (1 files)

File	Lines (Before → After)	Statements (Before → After)
`src/config-writer.ts`	89.9% → 91.1% (+1.19%)	89.9% → 91.1% (+1.19%)

Coverage comparison generated by scripts/ci/compare-coverage.ts

Copilot

Pull request overview

This PR fixes scheduled and manually-dispatched Copilot smoke workflows that previously tried to comment/label “the current pull request” even when no PR context exists, causing safe-outputs to fail with missing PR metadata on non-PR triggers.

Changes:

Updated all five smoke-copilot*.md workflow prompts to gate add_comment/add_labels behind PR triggers and use noop with a PASS/FAIL summary for schedule/workflow_dispatch.
Added a Jest test to enforce the noop-fallback output pattern across all Copilot smoke workflow variants.

Show a summary per file

File	Description
scripts/ci/smoke-copilot-workflow.test.ts	Adds a regression test ensuring Copilot smoke workflow prompts include PR-gated outputs plus a noop fallback for non-PR triggers.
.github/workflows/smoke-copilot.md	Updates Output instructions to avoid PR-only safe-outputs on schedule/workflow_dispatch by using `noop`.
.github/workflows/smoke-copilot-pat.md	Same Output gating + noop fallback pattern for the PAT variant.
.github/workflows/smoke-copilot-byok.md	Same Output gating + noop fallback pattern for the direct BYOK variant.
.github/workflows/smoke-copilot-byok-aoai-entra.md	Same Output gating + noop fallback pattern for the Entra-authenticated AOAI variant.
.github/workflows/smoke-copilot-byok-aoai-apikey.md	Same Output gating + noop fallback pattern for the AOAI API-key variant.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Files reviewed: 6/6 changed files
Comments generated: 0

github-actions · 2026-06-13T15:46:16Z

Smoke Test: Copilot BYOK (Direct) Mode ✅ PASS

Running in direct BYOK mode (COPILOT_PROVIDER_API_KEY) via api-proxy → api.githubcopilot.com

✅ GitHub MCP connectivity (retrieved 2 merged PRs)
✅ GitHub.com HTTP connectivity
✅ File write/read test
✅ BYOK inference working (direct mode via api-proxy sidecar)

Status: PASS

🔑 BYOK report filed by Smoke Copilot BYOK

github-actions · 2026-06-13T15:46:21Z

Smoke test report

feat(api-proxy): implement OTLP fan-out to multiple endpoints #4845 feat(api-proxy): implement OTLP fan-out to multiple endpoints
fix(docker-host): pass through loopback TCP DOCKER_HOST for ARC/DinD orchestration #4832 fix(docker-host): pass through loopback TCP DOCKER_HOST for ARC/DinD orchestration
GitHub page title: ✅
Smoke file write/read: ✅
Discussion comment: ✅
Build (npm ci && npm run build): ✅
Overall: PASS

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

registry.npmjs.org

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "registry.npmjs.org"

See Network Configuration for more information.

🔮 The oracle has spoken through Smoke Codex

github-actions · 2026-06-13T15:46:40Z

🔬 Smoke Test Results

Test	Result
GitHub MCP connectivity	✅
GitHub.com HTTP check	❌ (template vars unsubstituted)
File write/read	❌ (template vars unsubstituted)

Overall: FAIL — pre-step outputs (SMOKE_HTTP_CODE, SMOKE_FILE_PATH, SMOKE_FILE_CONTENT) were not substituted into the prompt, so 2/3 tests could not be verified.

cc @lpcox @Copilot

📰 BREAKING: Report filed by Smoke Copilot

github-actions · 2026-06-13T15:46:43Z

Smoke Test Results — Auth mode: PAT (COPILOT_GITHUB_TOKEN)

Test	Result
GitHub MCP	✅ (PR list verified)
GitHub.com HTTP	❓ (pre-step data not substituted)
File write/read	❓ (pre-step data not substituted)

Overall: FAIL — pre-computed test data unavailable (template variables not resolved)

@lpcox @Copilot

🔑 PAT report filed by Smoke Copilot PAT

github-actions · 2026-06-13T15:48:01Z

🏗️ Build Test Suite Results

Ecosystem	Project	Build/Install	Tests	Status
Bun	elysia	✅	1/1 passed	✅ PASS
Bun	hono	✅	1/1 passed	✅ PASS
C++	fmt	✅	N/A	✅ PASS
C++	json	✅	N/A	✅ PASS
Deno	oak	N/A	1/1 passed	✅ PASS
Deno	std	N/A	1/1 passed	✅ PASS
.NET	hello-world	✅	N/A	✅ PASS
.NET	json-parse	✅	N/A	✅ PASS
Go	color	✅	1/1 passed	✅ PASS
Go	env	✅	1/1 passed	✅ PASS
Go	uuid	✅	1/1 passed	✅ PASS
Java	gson	✅	1/1 passed	✅ PASS
Java	caffeine	✅	1/1 passed	✅ PASS
Node.js	clsx	✅	All passed	✅ PASS
Node.js	execa	✅	All passed	✅ PASS
Node.js	p-limit	✅	All passed	✅ PASS
Rust	fd	✅	1/1 passed	✅ PASS
Rust	zoxide	✅	1/1 passed	✅ PASS

Overall: 8/8 ecosystems passed — ✅ PASS

Generated by Build Test Suite for issue #4872 · ◷

github-actions · 2026-06-13T15:48:37Z

Smoke Test: GitHub Actions Services Connectivity

Check	Result
Redis PING (`host.docker.internal:6379`)	❌ No response
PostgreSQL pg_isready (`host.docker.internal:5432`)	❌ No response
PostgreSQL SELECT 1	❌ No response

host.docker.internal resolves to 172.17.0.1 but ports 6379 and 5432 are unreachable — service containers appear not to be running in this workflow environment.

Overall: FAIL

🔌 Service connectivity validated by Smoke Services

Initial plan

03b7d1d

Copilot AI assigned Copilot and lpcox Jun 13, 2026

Copilot started work on behalf of lpcox June 13, 2026 14:26 View session

Copilot AI changed the title ~~[WIP] Fix missing data report in Smoke Copilot BYOK AOAI workflow~~ fix: add noop fallback to smoke-copilot workflows on non-PR triggers Jun 13, 2026

Copilot finished work on behalf of lpcox June 13, 2026 14:31

Copilot AI requested a review from lpcox June 13, 2026 14:31

lpcox marked this pull request as ready for review June 13, 2026 15:40

Copilot AI review requested due to automatic review settings June 13, 2026 15:40

Copilot started reviewing on behalf of lpcox June 13, 2026 15:41 View session

Copilot AI reviewed Jun 13, 2026

View reviewed changes

Copilot AI had a problem deploying to aoai-model June 13, 2026 15:42 Failure

github-actions Bot added smoke-copilot-byok smoke-codex labels Jun 13, 2026

github-actions Bot mentioned this pull request Jun 13, 2026

[aw] Contribution Check failed #4820

Closed

github-actions Bot added the build-test label Jun 13, 2026

Copilot AI temporarily deployed to aoai-model June 13, 2026 15:51 Inactive

Copilot AI temporarily deployed to aoai-model June 13, 2026 15:52 Inactive

lpcox merged commit a760c15 into main Jun 13, 2026
76 of 80 checks passed

lpcox deleted the copilot/aw-fix-missing-data-issue branch June 13, 2026 16:09

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix: add noop fallback to smoke-copilot workflows on non-PR triggers#4872

fix: add noop fallback to smoke-copilot workflows on non-PR triggers#4872
lpcox merged 2 commits into
mainfrom
copilot/aw-fix-missing-data-issue

Copilot AI commented Jun 13, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Jun 13, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

github-actions Bot commented Jun 13, 2026

Uh oh!

github-actions Bot commented Jun 13, 2026

Uh oh!

github-actions Bot commented Jun 13, 2026

Uh oh!

github-actions Bot commented Jun 13, 2026

Uh oh!

github-actions Bot commented Jun 13, 2026

Uh oh!

github-actions Bot commented Jun 13, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

Copilot AI commented Jun 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

Uh oh!

github-actions Bot commented Jun 13, 2026

✅ Coverage Check Passed

Overall Coverage

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Copilot's findings

Uh oh!

github-actions Bot commented Jun 13, 2026

Smoke Test: Copilot BYOK (Direct) Mode ✅ PASS

Uh oh!

github-actions Bot commented Jun 13, 2026

Uh oh!

github-actions Bot commented Jun 13, 2026

🔬 Smoke Test Results

Uh oh!

github-actions Bot commented Jun 13, 2026

Uh oh!

github-actions Bot commented Jun 13, 2026

🏗️ Build Test Suite Results

Uh oh!

github-actions Bot commented Jun 13, 2026

Smoke Test: GitHub Actions Services Connectivity

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Copilot AI commented Jun 13, 2026 •

edited

Loading