-
Notifications
You must be signed in to change notification settings - Fork 421
Commit 3e83f99
authored
Improved ci prompt testing suite (#270)
* CI: streamline Unity licensing (ULF/EBL); drop cache mounts & EBL-in-container; NL suite: clarify T-E/T-J, anchor positions, EOF brace targeting, SHA preconditions
* CI: support both ULF + EBL; validate ULF before -manualLicenseFile; robust readiness wait; use game-ci @v2 actions
* CI: activate EBL via container using UNITY_IMAGE; fix readiness regex grouping
* CI: minimal patch — guard manualLicenseFile by ulf.ok, expand error patterns, keep return-license @v2 for linter
* CI: harden ULF staging (printf+chmod); pass ULF_OK via env; use manual_args array for -manualLicenseFile
* CI: assert EBL activation writes entitlement to host mount; fail fast if missing
* CI: use heredoc in wait step to avoid nested-quote issues; remove redundant EBL artifact copy; drop job-level if and unused UNITY_VERSION
* CI: harden wait step (container status check, broader ready patterns, longer timeout); make license return non-blocking
* CI: wait step — confirm bridge readiness via status JSON (unity_port) + host socket probe
* CI: YAML-safe readiness fallback (grep/sed unity_port + bash TCP probe); workflow_dispatch trigger + ASCII step names
* CI: refine license error pattern to ignore benign LicensingClient channel startup; only match true activation/return failures
* Improve Unity bridge wait logic in CI workflow
- Increase timeout from 600s to 900s for Unity startup
- Add 'bound' to readiness pattern to catch more bridge signals
- Refine error detection to focus only on license failures
- Remove non-license error patterns that could cause false failures
- Improve error reporting with descriptive messages
- Fix regex escaping for unity port parsing
- Fix case sensitivity in sed commands
* Add comprehensive Unity workflow improvements
- Add project warm-up step to pre-import Library before bridge startup
- Expand license mounts to capture full Unity config and local-share directories
- Update bridge container to use expanded directory mounts instead of narrow license paths
- Provide ULF licenses in both legacy and standard local-share paths
- Improve EBL activation to capture complete Unity authentication context
- Update verification logic to check full config directories for entitlements
These changes eliminate cold import delays during bridge startup and provide
Unity with all necessary authentication data, reducing edge cases and improving
overall workflow reliability.
* Refine Unity workflow licensing and permissions
- Make EBL verification conditional on ULF presence to allow ULF-only runs
- Remove read-only mounts from warm-up container for Unity user directories
- Align secrets gate with actual licensing requirements (remove UNITY_SERIAL only)
- Keep return-license action at v2 (latest available version)
These changes prevent workflow failures when EBL has issues but ULF is valid,
allow Unity to write preferences during warm-up, and ensure secrets detection
matches the actual licensing logic used by the workflow steps.
* fix workflow YAML parse
* Normalize NL/T JUnit names and robust summary
* Fix Python import syntax in workflow debug step
* Improve prompt clarity for XML test fragment format
- Add detailed XML format requirements with exact specifications
- Emphasize NO prologue, epilogue, code fences, or extra characters
- Add specific instructions for T-D and T-J tests to write fragments immediately
- Include exact XML template and TESTID requirements
- Should fix T-D and T-J test failures in CI by ensuring proper fragment format
* Fix problematic regex substitution in test name canonicalization
- Replace unsafe regex substitution that could create malformed names
- New approach: preserve correctly formatted names, extract titles safely
- Prevents edge cases where double processing could corrupt test names
- Uses proper em dash (—) separator consistently
- More robust handling of various input formats
* CI: NL/T hardening — enforce filename-derived IDs, robust backfill, single-testcase guard; tighten prompt emissions; disallow Bash
* fix: keep file ID when canonicalizing test names
* CI: move Unity Pro license return to teardown after stopping Unity; keep placeholder at original site
* CI: remove revert helper & baseline snapshot; stop creating scripts dir; prompt: standardize T-B validation to level=standard
* CI: remove mini workflow and obsolete NL prompts; redact email in all Unity log dumps
* NL/T prompt: enforce allowed ops, require per-test fragment emission (incl. failures), add T-F..T-J XML templates
* NL suite: enforce strict NL-4 emission; remove brittle relabeling; keep canonicalization + backfill
* NL/T: minimize transcript; tighten NL-4 console reads; add final errors scan in T-J
* ci: add local validate-nlt-coverage helper
* CI: add staged report fragment promotion step (reports/_staging -> reports/) to support multi-edit reporting
* CI: add staged report fragment promotion step (reports/_staging -> reports/) to support multi-edit reporting
* CI: minor polish and guardrails; keep staged reports promotion and placeholder detection
* read_console: default count=50; normalize types str->list; tolerate legacy payload shapes
* read_console: harden response parsing for legacy shapes (data as list, tuple entries)
* Docs: refresh CI workflow and prompts (remove mini suite refs; per-test emissions, staging, guard)
* CI: move T coverage check after staged promotion; accept _staging as present; dedupe promotion step
* CI: make T retry conditional on explicit coverage probe (not failure()); respect _staging in probe1 parent e219080 commit 3e83f99Copy full SHA for 3e83f99
File tree
Expand file treeCollapse file tree
19 files changed
+1480
-979
lines changedFilter options
- .claude
- prompts
- .github/workflows
- UnityMcpBridge
- Editor/Tools
- UnityMcpServer~/src/tools
- scripts
- tests
Expand file treeCollapse file tree
19 files changed
+1480
-979
lines changedCollapse file: .claude/prompts/nl-unity-claude-tests-mini.md
.claude/prompts/nl-unity-claude-tests-mini.md
Copy file name to clipboardExpand all lines: .claude/prompts/nl-unity-claude-tests-mini.md-45Lines changed: 0 additions & 45 deletions
This file was deleted.
Collapse file: .claude/prompts/nl-unity-suite-full.md
.claude/prompts/nl-unity-suite-full.md
Copy file name to clipboardExpand all lines: .claude/prompts/nl-unity-suite-full.md-234Lines changed: 0 additions & 234 deletions
This file was deleted.
0 commit comments