vyos-documentation.git/.github/workflows/ai-validation.yml, branch T9032-circinus

T8960: ai-validation — mint vyos-bot token via get-token@production

2026-06-03T00:57:16+00:00

Swap the cross-repo-checkout token from the dedicated vyos-docs-reviewer App (VYOS_APP_ID/_PRIVATE_KEY) to the shared vyos-bot App via the fleet get-token composite, scoped contents:read. Skip-check now gates on APP_CLIENT_ID + APP_PRIVATE_KEY + ANTHROPIC_API_KEY (org-level vyos-bot creds); reword the stale token comment. PR-comment posting still uses the default GITHUB_TOKEN. Byte-identical to the paired reference copy in VyOS-Networks/vyos-docs-opus-reviewer (scripts/ai-validation.yml). 🤖 Generated by [robots](https://vyos.io) (cherry picked from commit 459109f13adf026d012c4e803f4a86a8f84e6ce3)

ci(ai-validation): allow Pass 2 review on external-contributor PRs

2026-05-29T09:20:41+00:00

The upstream `anthropics/claude-code-action` performs a write-permission check on `github.actor` before our skip logic runs. On `pull_request_target` the actor is the PR author; external contributors resolve to `read` and the action exits 1 with `Actor does not have write permissions to the repository`. Net effect: AI validation has been failing on every external-contributor PR (LiudmylaNad, teslazonda, scottlaird in the last 4 weeks) while succeeding on maintainer PRs. Failure reproduced on run 26541079685 (PR #2061). Fix: set `allowed_non_write_users: '*'` on the Pass 2 step. The action bypasses the actor check when this input is set and `github_token` is provided (already the case). The action also auto-scrubs Anthropic / cloud / GHA secrets from subprocess envs when this input is set. Safe in THIS workflow because the existing defense-in-depth bounds what Pass 2 can do with untrusted PR content: - `allowedTools` restricted to inline-comment + read-only surfaces - `github_token` is the PR-scoped default (not the broader VYOS_APP_ID) - prompt marks PR content as untrusted via `` - workspace-wipe removes `CLAUDE.md` / `.claude/` before Pass 2 - prepare bundles MD via `git show HEAD:` (blob, not `cp`) Full rationale inlined as a comment block above the new input. 🤖 Generated by [robots](https://vyos.io) (cherry picked from commit 1fa39bd7ac64898f7a21922bc0f195d115d3cdeb)

ci(ai-validation): skip prepare on Mergify-authored PRs

2026-05-14T06:45:06+00:00

Lifts the existing Mergify-author short-circuit (today inside validate's `secrets-check` step) to a job-level `if:` on `prepare`, so the whole pipeline skips for backport/queue PRs. Why now: every Mergify backport whose merge ref shares no shallow ancestor with the (advanced) base branch fails the prepare step at git diff "$BASE...HEAD" --name-only ... fatal: FETCH_HEAD...HEAD: no merge base (because base is `git fetch --no-tags --depth=1` and the merge ref is `fetch-depth: 2`). Proximate symptom: run 25842928620 on PR #2042 (sagitta backport of #2023). AI Validation isn't a required check so the queue isn't blocked, but every Mergify backport is left with a red "prepare" check that adds noise to PR review. The validate-level skip in commit 0e8a2956 was correct for the "claude-code-action rejects bot-initiated runs" failure mode but fires too late — prepare has already run and crashed before validate's `if: needs.prepare.outputs.has_md_changes == 'true'` even evaluates. Implementation: single job-level `if:` on prepare. validate's `needs: [prepare]` cascades the skip naturally (skipped needs make the dependent's expression-based `if:` evaluate against empty outputs). The in-step author check in validate stays as defense-in-depth. 🤖 Generated by [robots](https://vyos.io) (cherry picked from commit 7d94d6116be1a4776b7317cb5190a83dd065e571)

ci(ai-validation): skip validation on Mergify-authored PRs

2026-05-13T22:22:47+00:00

The upstream anthropics/claude-code-action rejects bot-initiated workflow runs with: Workflow initiated by non-human actor: mergify (type: Bot). Add bot to allowed_bots list or use '*' to allow all bots. This leaves a failing required check on every Mergify backport PR (#2025 today is the proximate symptom — circinus backport of #2016 is blocked from auto-merging by this). Per org policy, Mergify-authored PRs skip the bot-review flow entirely: the underlying change was already reviewed on the source PR. Re-running validation on the backport would also post duplicate findings. Fix: extend the existing `secrets-check` step (kept the id for backwards compat with the cascade of `if: steps.secrets-check.outputs.skip != 'true'` gates) to also short-circuit when `github.event.pull_request.user.login` equals `mergify[bot]`. Add a `skip_reason` output so the notify-on-PR step can distinguish bot-skip (silent) from missing-secrets (post a notice comment). Implementation notes: - PR author is bound through env: PR_AUTHOR (same pattern as secret bindings) so the template-expansion happens before bash sees the value as an already-quoted env variable. - Uses `printf '...%s\n' "$PR_AUTHOR"` for the notice line so the shell — not the YAML template engine — interpolates the author into the workflow log output. - Renamed step name to "Decide whether to skip validation" to reflect the broader scope; step id stays `secrets-check`. 🤖 Generated by [robots](https://vyos.io) (cherry picked from commit d75de0c24b971a2d87e817ac3cb8e09cb816aac0)

ci(ai-validation): stop retargeting origin to fork URL

2026-05-12T06:22:29+00:00

The previous CR-driven fix (7a7a8002) re-pointed origin to the PR's HEAD fork on the assumption that claude-code-action calls `git fetch origin `. That assumption is wrong for fork PRs: the action detects a fork PR and fetches `pull//head` instead, and those refs only exist on the base repo. Retargeting origin to the fork therefore breaks the fork-PR fetch path with: PR # is from a fork, fetching via refs/pull//head... fatal: couldn't find remote ref pull//head Observed on run 25689321499 (PR #1891 from natali-rs1985/vyos-documentation). Leave origin pointed at vyos/vyos-documentation (the base repo) — that's where actions/checkout left it after the merge-ref checkout, and where `pull//head` resolves. Replace the step with an explanatory comment. 🤖 Generated by [robots](https://vyos.io) (cherry picked from commit 68f87deca228ad2b6b07c4eb64f201c0838e494f)

ci(ai-validation): pin validate to prepare merge_sha + fix 2 stray escapes

2026-05-11T09:51:11+00:00

CR/Copilot pass on the merge-ref + retarget-origin revision raised: 1. (CR on #1991, line 278, Major) Re-resolving refs/pull//merge in validate is racy. GitHub may advance the merge ref between prepare and validate (rapid pushes), so Pass 1 (artifact from prepare) and Pass 2 (validate workspace) can see different revisions. concurrency.cancel-in-progress narrows the window but does not make the ref immutable. Fix: prepare outputs its post-checkout `git rev-parse HEAD` as `merge_sha`, validate checks out THAT exact sha. Both jobs now operate on the same revision regardless of subsequent pushes. 2. (Copilot on #1991, two findings on line 294) The merge-ref + retarget commit introduced two NEW stray heredoc escapes: `PR"'sclaude-code-action'"s`. Same class of error as the previous-fix-cycle: shell heredoc escape sequences survived verbatim into YAML. Fix: replace with plain `PRs` / `claude-code-actions` in the inline comment block and step name. 3. (Copilot on #1990/#1991, "PR title/description says HEAD repo + head.sha") Metadata cleanup tracked separately — PR titles + bodies updated to describe the merge-ref + retarget-origin approach (no workflow file change for that part). Mirrored byte-identically across the 3 open wave-5 PRs (#1990 rolling, #1991 circinus, #1993 sagitta follow-up). Canonical sync once these merge.

ci(ai-validation): merge-ref checkout + retarget origin to fork + wipe CLAUDE.md/.claude (CR findings)

2026-05-11T09:32:34+00:00

Two new CR findings on PR #1990 (both raised on the same review cycle): 1. (CR, line 296, Major, Heavy lift) Tree drift between Pass 1 and Pass 2. The prior fix in this PR used `head.sha` (origin=fork) so the action's `git fetch origin ` resolves — but that left validate's workspace tree at PR HEAD while prepare bundled `diff-md.patch` + `_changed_md/` from `refs/pull//merge`. For PRs where base also contributes content to a touched file, Pass 2 workspace-Read could disagree with Pass 1's view. Fix: switch the validate checkout back to `refs/pull//merge` (workspace tree now matches the prepare bundle) and address the action's `git fetch origin ` requirement separately by `git remote set-url origin ` after the checkout. The workspace tree is unchanged by the remote retarget; only the fetch destination is updated. For public forks (the only kind targeting public vyos/vyos-documentation) the unauthenticated fetch succeeds. 2. (CR, line 316, Major) `anthropics/claude-code-action@v1` invokes the Claude Code CLI, which auto-discovers and loads `CLAUDE.md` and `.claude/**` (memory, skills, hooks, MCP, plugins) from the workspace at session startup. A malicious fork could pre-place a `CLAUDE.md` with prompt-injection content, or a `.claude/skill.md` marked as auto-load. Documented at: - code.claude.com/docs/en/claude-directory.md - code.claude.com/docs/en/memory Fix: add `CLAUDE.md` and `.claude` to the wipe list so Claude's auto-discovery starts from a clean slate (workflow-controlled state only). `--bare` would also disable this but would lose the `mcp__github_inline_comment` MCP server the action provides, so wiping reserved paths is the better fit. Mirrored across rolling (#1990), circinus (#1991), and the sagitta follow-up (#1993). Canonical sync once these merge.

ci(ai-validation): address 3 follow-up CR/Copilot findings on validate-checkout step

2026-05-11T09:22:47+00:00

After the first fix bundle landed on the three wave-5 PRs, three more findings surfaced: 1. (CR on #1991, line 312) `rm -f` silently no-ops on directories. A fork could commit `changed-md.txt/`, `diff-md.patch/`, or `pass1-findings.json/` AS DIRECTORIES (not files), and the wipe step would skip them — the subsequent artifact-download writes into the real-file path would then collide with the fork-placed directory, producing unpredictable behaviour. Fix: use `rm -rf` for ALL reserved paths uniformly. The distinction was cosmetic at best; making it uniform closes the gap. 2. (Copilot on #1991, line 311) `.venv/` is also fork-attackable. `astral-sh/setup-uv` with `activate-environment: true` creates `${{ github.workspace }}/.venv` and prepends its `bin/` to PATH. If a fork pre-populates `.venv/bin/`, those binaries land in PATH for subsequent steps (Pass 1 runs `vyos-doc-review`, Pass 2 runs `claude-code-action` whose internals may shell out). Fix: add `.venv` to the wipe list. 3. (Copilot on #1992, line 298) The inline comment block contains a literal `PR's` — a stray shell heredoc escape sequence that survived the commit verbatim. Confusing in YAML. Fix: replace with plain `PRs`. Mirrored byte-identically across rolling/circinus/sagitta + canonical follow-up after this wave merges.

ci(ai-validation): address 4 Copilot findings on validate-checkout step

2026-05-11T09:14:18+00:00

CR pass on #1990/#1991/#1992 raised 4 findings on the new validate- side actions/checkout step. All valid; folding into one bundle: 1. Fork PR head-ref not found in base repo (#1990, #1991, #1992 each raised this). The previous form checked out refs/pull//merge from origin (vyos/vyos-documentation), leaving origin pointed at the base repo. claude-code-action then runs `git fetch origin ` where is e.g. `contributor/branch` — which does NOT exist in the base repo for fork PRs, so the fetch fails the same way "no .git" failed before. Fix: check out the PR HEAD repo (the fork for fork PRs) at the head sha. origin now resolves to the head repo where the head ref exists, so the action's fetch succeeds. github.token is the default GITHUB_TOKEN which can read public forks (the only kind of fork that can target vyos/vyos-documentation, since that repo is public). 2. Untrusted PR content at workspace root can pre-create reserved paths (#1991). The PRs tree at workspace root could in principle contain `.reference-db/extracted/...`, `pass1-findings.json`, `_changed_md/poisoned.md`, etc., which subsequent steps would treat as workflow-generated. The fail- closed gate only checks `[ ! -d .reference-db/extracted ]` so a PR-placed empty `.reference-db/extracted/` would bypass it. Fix: add a "Wipe reserved workspace paths" step right after the checkout that `rm -rf`s the workflow-owned paths (.reference-db, .vyos-1x, reviewer, reviewer-src, _changed_md) and `rm -f`s the workflow-owned files (pass1-findings.json, diff-md.patch, changed-*.txt) — every one of these is recreated by the trusted producer step immediately following. 3. Checkout runs unconditionally even on skip=true paths (#1991). The checkout only needs to run when validate will actually use it (skip != true → Pass 2 runs). Fix: gate the checkout (and the wipe step) on `if: steps.secrets-check.outputs.skip != true`. Move them AFTER the secrets-check step in step order so the conditional is meaningful. 4. "NO credentials" wording misleading (#1992). actions/checkout uses the GITHUB_TOKEN to download the tree — it is not an unauthenticated fetch. `persist-credentials: false` only suppresses writing the token into the resulting .git/config. Fix: rename the step to "Checkout PR HEAD (no persisted credentials)" and update the inline comment block to be explicit that the token is used for the download but not persisted. Mirrored byte-identically across rolling/circinus/sagitta + canonical follow-up after this wave merges.

ci(ai-validation): checkout PR merge ref in validate (claude-code-action needs git workspace)

2026-05-11T08:40:27+00:00

Smoke verify on PR #1977 (run 25659408343 after github_token fix landed) reached Pass 2 with valid auth but failed at action setup: fatal: could not open `docs/quick-start.md` for reading: No such file or directory fatal: not a git repository (or any of the parent directories): .git Action failed with error: Command failed: git fetch origin --depth=20 yuriy/docs-smoke-test-quickstart The action expects to run inside a git checkout of the PR's repo so it can `git fetch ` and read changed files from the working directory. Our split-job design checks out vyos-1x, reviewer-src, and the reviewer-config sparse-checkout into named subdirs but leaves the workspace root empty (we only have _changed_md/ via artifact download). Add `actions/checkout` of `refs/pull//merge` as the FIRST step in validate. Subsequent steps that create named subdirs (_changed_md/, .vyos-1x/, reviewer-src/, reviewer/, .reference-db/) coexist with the docs/scripts/.github content checked out here — no path collisions. Trust model unchanged: * persist-credentials: false (no token in fork-content .git/config) * No shell step executes fork code (no pip/npm/make against the workspace; Pass 1 runs the trusted reviewer CLI; Pass 2 only uses allowlisted Read/Glob/Grep/mcp__github_inline_comment + Bash restricted to `gh pr comment|diff|view`) * The prepare job continues to provide the canonical bundled changed-md set via the pr-input artifact; validate now ALSO has the full PR tree available for cross-reference reads Mirrored byte-identically across rolling/circinus/sagitta + canonical follow-up.