Skip to content

fix: improve optional skill review quality - #57

Merged
martinfrancois merged 1 commit into
mainfrom
fix/skill-review-quality-wording
Jun 2, 2026
Merged

fix: improve optional skill review quality#57
martinfrancois merged 1 commit into
mainfrom
fix/skill-review-quality-wording

Conversation

@martinfrancois

Copy link
Copy Markdown
Owner

Summary

  • Problem: Tessl quality review had dropped to 90% because SKILL.md was compact but not concrete enough in the main workflow.
  • Why it matters: the skill should keep the recent lift while also passing Tessl's quality review at 100%.
  • What changed: tightened workflow wording, reduced a repeated hard-stops.md reference, and added small inline Java examples for guard/read cleanup and nullable value + parser + lazy fallback flow.
  • What did not change: no active eval tasks, criteria, weights, runtime references, package metadata, or CI/release settings changed.

Change Type

  • Skill behavior
  • Evals or scoring
  • Documentation
  • CI, release, or dependency automation
  • Repository metadata or contribution process
  • Other maintenance

Linked Issue

  • Fixes #
  • Related #

User-Visible Behavior

Users should see the same Java Optional guidance with a more direct, executable main workflow. Tessl skill review now reports 100%.

Bug Fix Details

  • Root cause: the prior compact SKILL.md had enough guidance for eval behavior, but Tessl review marked conciseness/actionability/progressive disclosure down because key instructions were too descriptive and examples were mostly in references.
  • Test, eval, or guardrail added: no new eval was added; the existing headline suite was rerun with Sonnet 4.6.
  • If no test or eval was added, why not: this is a runtime wording cleanup, and existing headline evals already cover the behavior/lift risk.

Validation

Checks most contributors can run:

  • python3 scripts/validate_skill.py skills/java-optionals
  • python3 scripts/validate_eval_criteria.py evals evals-reference
  • python3 -m py_compile scripts/validate_skill.py scripts/validate_eval_criteria.py
  • bash -n scripts/check_publish_dry_run.sh
  • tessl plugin lint .
  • markdownlint, if Markdown changed
  • Manual rendered-doc or example review, if docs or examples changed

Tessl-authenticated checks:

  • bash scripts/check_publish_dry_run.sh .
  • tessl plugin publish --dry-run --bump patch .
  • tessl skill review --threshold 90 skills/java-optionals/SKILL.md, if skill text or references changed
  • tessl eval run --variant with-context --variant without-context ., if skill behavior,
    evals, or benchmark claims changed

bash scripts/check_publish_dry_run.sh ., tessl skill review, and hosted Tessl evals require
Tessl authentication. Hosted evals also require a linked Tessl project. If you can't run one of
them, leave it unchecked and explain why in the details.

Details:

python3 scripts/validate_skill.py skills/java-optionals
Skill is valid.

python3 scripts/validate_eval_criteria.py evals evals-reference
Validated 52 scenario(s): 4 natural, 48 explicit.

python3 -m py_compile scripts/validate_skill.py scripts/validate_eval_criteria.py
bash -n scripts/check_publish_dry_run.sh

tessl plugin lint .
Plugin martinfrancois/java-optionals@0.1.26 is valid.

tessl skill review --threshold 100 skills/java-optionals/SKILL.md
Review Score: 100%

tessl eval run --agent claude:claude-sonnet-4-6 --variant with-context --variant without-context --label "quality wording final sonnet 4.6 76ee485" .
Run id: 019e86bb-314f-7542-8826-75a8cd90ae02
Baseline: 159/360
With context: 360/360
Lift: 2.26x

Human Verification

Reviewed the SKILL.md diff to confirm it only changes runtime skill wording and does not touch active eval content or scoring. The final Sonnet 4.6 run kept every with-context headline scenario at 100%.

Review Checklist

  • Docs updated, or N/A
  • Evals updated, or N/A
  • Scenario directories include task.md, criteria.json, and capability.txt, or N/A
  • Scenario invocation style is classified as natural or explicit, or N/A
  • Natural activation prompts don't explicitly invoke the skill, or N/A
  • Explicit invocation prompts are labeled as explicit, or N/A
  • Headline criteria include compile/artifact checks, or N/A
  • Headline criteria include behavior correctness checks, or N/A
  • Runtime references contain no eval answer keys, scenario inventory, hosted run IDs, or fixed
    score claims
  • Java baseline compatibility has been considered, or N/A
  • OptionalInt, OptionalLong, and OptionalDouble guidance has been considered, or N/A
  • Optional-producing stream terminals and collectors are covered, or N/A
  • Java 26 Javadocs were checked for Optional-family coverage, or N/A
  • Valid README package-runner instructions were preserved, or N/A
  • Tessl package commands match the verified plugin package format
  • Full/reference eval reporting is not hidden or cherry-picked
  • Tessl checks were run, or unavailability is documented
  • PR title or squash title uses Conventional Commits
  • Redaction checked: no Tessl tokens, GitHub tokens, package manager tokens, private repository
    links, private eval artifacts, private registry/workspace links, local host paths, or
    proprietary Java source

AI Assistance (if used)

  • AI-assisted PR
  • I confirm I understand and reviewed the change
AI prompts / session logs (optional)
The maintainer asked to raise Tessl skill review quality to 100% without hurting headline eval lift. Codex tightened SKILL.md and verified the result with local validation, Tessl skill review, and a hosted Sonnet 4.6 headline eval.

@martinfrancois
martinfrancois merged commit 8854855 into main Jun 2, 2026
8 checks passed
@martinfrancois
martinfrancois deleted the fix/skill-review-quality-wording branch June 2, 2026 05:18
martinfrancois pushed a commit that referenced this pull request Jun 2, 2026
🤖 I have created a release *beep* *boop*
---


##
[0.1.27](v0.1.26...v0.1.27)
(2026-06-02)


### Bug Fixes

* improve optional skill review quality
([#57](#57))
([8854855](8854855))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant