vyos-documentation.git/scripts, branch feature/T9082-codeql-onboard

docs: Cloudflare Workers hosting pipeline (apex, content workers, CI, previews) (#2140)

2026-07-10T14:14:13+00:00

* docs-infra: scaffold Cloudflare workers workspace (versions.json v2, matrix, toolchain) 🤖 Generated by [robots](https://vyos.io) * docs-infra: record full Phase-0 plan decision in workers/PLAN.md 🤖 Generated by [robots](https://vyos.io) * docs-infra: shared content worker — asset serving, cache classes, X-Docs-Build, canary no-store 🤖 Generated by [robots](https://vyos.io) * docs-infra: run worker script before assets; test fetch entrypoint 🤖 Generated by [robots](https://vyos.io) * docs-infra: apex manifest loader + dispatch map + runtime binding guard (TDD) 🤖 Generated by [robots](https://vyos.io) * docs-infra: apex redirects (aliases, PDF, trailing-slash) + special paths (TDD) 🤖 Generated by [robots](https://vyos.io) * docs-infra: PDF redirect honors pdf:null and preserves query 🤖 Generated by [robots](https://vyos.io) * docs-infra: apex UA gate — allowlist-wins, log-only AI crawlers, empty block list at launch 🤖 Generated by [robots](https://vyos.io) * docs-infra: apex router (pipeline §3.2), themed 404/503, /kb seam, env configs + congruence test 🤖 Generated by [robots](https://vyos.io) * docs-infra: add missing-User-Agent regression test for apex UA gate 🤖 Generated by [robots](https://vyos.io) * docs-infra: R2-streaming preview worker — MIME map, noindex, no-store (TDD) 🤖 Generated by [robots](https://vyos.io) * docs-infra: preview 404 no-store + fetch handler tests 🤖 Generated by [robots](https://vyos.io) * docs-infra: bootstrap script — binding-target workers must exist before apex deploys 🤖 Generated by [robots](https://vyos.io) * docs-infra: apex run_worker_first, lockfile for npm ci, PDF Location from manifest 🤖 Generated by [robots](https://vyos.io) * docs-infra: derive html_baseurl from DOCS_VERSION_SLUG with RTD fallback (canonical gate prereq) 🤖 Generated by [robots](https://vyos.io) * docs-infra: version picker + status banner + language scaffold (vanilla JS, TDD pure core) 🤖 Generated by [robots](https://vyos.io) * docs-infra: picker preserves query+hash across switch; valid breadcrumb markup 🤖 Generated by [robots](https://vyos.io) * docs-infra: Pagefind search wrapper with runtime base-path + preview prefix handling (TDD) 🤖 Generated by [robots](https://vyos.io) * docs-infra: pagefind wrapper — asset-failure notice + UI stylesheet load 🤖 Generated by [robots](https://vyos.io) * docs-infra: gate Pagefind searchbox to CF builds (RTD keeps stock search until cutover) 🤖 Generated by [robots](https://vyos.io) * docs-infra: deploy sanity gates — limits, critical pages, count-delta, canonical (TDD) 🤖 Generated by [robots](https://vyos.io) * docs-infra: hermetic gate tests via fixture versions.json 🤖 Generated by [robots](https://vyos.io) * docs-infra: docs-build workflow — candidate/smoke/promote two-stage deploy + registry + rollback Two-stage CF Workers pipeline: build in pinned container, assemble artifact, sanity gates, deploy candidate, scoped pre-traffic smoke via canary apex, promote (rollback-id capture, hostname purge, registry upload), post-promote probe + auto-rollback. DOCS_CF_LIVE repo variable gates every docs.vyos.io production interaction pre-cutover. scripts/docs_gates/smoke.py adds one authorized check beyond the spec: the version's index.html probe asserts the #vyos-search mount div is present in the response body, guarding CI silently forgetting DOCS_VERSION_SLUG (which would otherwise ship stock RTD search without the Pagefind gate noticing). 🤖 Generated by [robots](https://vyos.io) * docs-infra: build docs image in-workflow with buildx cache (v4.1 — digest pin dropped) Plan v4.1 amendment: the ghcr.io digest-pinned image does not exist (workflow would hard-fail at the first docker step on every push). Replace the BUILD_IMAGE env placeholder with an in-workflow docker build from docker/Dockerfile via docker/setup-buildx-action@v3 + docker/build-push-action@v6 (context: docker/, load: true, tags: docs-build:local, GHA cache from/to). The checked-out commit is the pin; buildx GHA cache keeps repeat builds cheap. Sphinx-build step swaps to docs-build:local; inner script unchanged. 🤖 Generated by [robots](https://vyos.io) * docs-infra: apex/preview deploy workflow — canary auto, production behind environment approval * docs-infra: apex-deploy concurrency guard (per-ref, cancel-in-progress) 🤖 Generated by [robots](https://vyos.io) * docs-infra: fork-safe PR preview pipeline — approval record, R2 prefixes, label consumption, cleanup * docs-infra: nightly preview sweep — pipefail + per-prefix failure isolation 🤖 Generated by [robots](https://vyos.io) * docs-infra: nightly canary QA — per-entry sweep + URL-parity corpus vs RTD 🤖 Generated by [robots](https://vyos.io) * docs-infra: parity sweep scoped to CF-built versions; transport-error resilience 🤖 Generated by [robots](https://vyos.io) * docs-infra: one-off bootstrap workflow (binding targets — runs once on this push) 🤖 Generated by [robots](https://vyos.io) * docs-infra: remove one-off bootstrap workflow (bootstrap complete) 🤖 Generated by [robots](https://vyos.io) * docs-infra: one-off canary apex + preview deploy (route targets for Task 3.6 step 2c) 🤖 Generated by [robots](https://vyos.io) * docs-infra: remove one-off canary deploy workflow (targets live) 🤖 Generated by [robots](https://vyos.io) * docs-infra: address Phase-0 CodeRabbit findings (canonical gate, error caching, registry pointer, validation) 🤖 Generated by [robots](https://vyos.io) * docs-infra: strengthen manifest tests (full dispatch iteration, mutation-free validate) 🤖 Generated by [robots](https://vyos.io) * docs-infra: address GitHub CodeRabbit review (pointer-after-probe, fail-closed sweeps, block-precedence UA gate, preview hardening) 🤖 Generated by [robots](https://vyos.io) * docs-infra: adversarial review fixes — error no-store, probe retry, PR-list membership, preview dotted-segment 🤖 Generated by [robots](https://vyos.io) * docs-infra: serve oversized legacy PDF from R2 via apex (spec §5 fallback) The 1.3 PDF (29.2 MiB) exceeds the 25 MiB static-asset cap and is absent from the legacy content Worker's build, so /_/downloads/en/1.3/pdf/ (and the picker's PDF link) 301'd into a dead-end 404 post-cutover. Add the R2 object fallback spec §5 already documented but never implemented: a DOCS_PDFS R2 bucket binding on the apex Worker, a manifest pdf_r2_key field (1.3 only), and a router step ahead of version dispatch that streams the object with its own cache class (canary/error still force no-store). 🤖 Generated by [robots](https://vyos.io) * docs-infra: PDF R2 fallback honors Range + If-None-Match, preserves ETag 🤖 Generated by [robots](https://vyos.io)

ci(doc-linter): anchor .. code-block:: detection + drop debug print

2026-05-14T04:45:25+00:00

Addresses Copilot review on PR #2023: 1. .. code-block:: tracking was triggered by a plain substring check, which matched mid-line occurrences too. In MD prose like ``.. code-block::`` (docs/documentation.md:222) this set in_rst_codeblock=True spuriously and could suppress line-length checks downstream. Replace with a leading-whitespace- anchored regex and gate on file_ext in ('.rst', '.txt') or an open {eval-rst} MyST fence so the directive opener is only recognized where it can actually occur. 2. print('start') in main() was leftover debug noise — remove it.

ci(doc-linter): fix \b regression in compressed-IPv6 regex — replace bare removal with word-boundary prefix

2026-05-13T21:55:52+00:00

Agent-Logs-Url: https://github.com/vyos/vyos-documentation/sessions/cdefcaf2-e89e-4090-b39a-15b385b774df Co-authored-by: andamasov <12631358+andamasov@users.noreply.github.com>

ci(doc-linter): lint added + renamed files, not only modified

2026-05-13T21:49:50+00:00

Previous workflow: env: FILES_MODIFIED: ${{ steps.file_changes.outputs.files_modified }} run: python scripts/doc-linter.py "$FILES_MODIFIED" `trilom/file-changes-action`'s `files_modified` output is modifications-only. A PR adding a new `.md`/`.rst` doc page passed `files_added`, never `files_modified`, so a brand-new page with long lines or real public IPs slipped past the linter entirely. Workflow: also pass `files_added` and `files_renamed` as separate positional args. Each output is a JSON array (action v1.2.4) and is passed via env to avoid shell-quoting issues. Linter: `main()` now accepts one OR multiple positional argv entries, each a JSON array of paths. Arrays are merged and deduplicated before linting. Single-arg invocations remain backward-compatible. Switched from `ast.literal_eval` to `json.loads` — the action's outputs are JSON, and `json.loads` is the right tool (and dodges literal_eval-via-`eval`-substring linter warnings). Test coverage: - Two JSON arrays merge -> single linter run on union. - Empty-string argv entry skipped (no `files_renamed` in many PRs). - Malformed JSON -> falls back to walking DOCS_ROOT. - No argv -> walks DOCS_ROOT. - Single-arg invocation -> backward-compat preserved. Tracked as item 7 of the rolling-side cleanup backlog from PR #2014 / #2019 / #2020 reviews. 🤖 Generated by [robots](https://vyos.io)

ci(doc-linter): exclude docs/_rst_legacy/ and docs/_build/ from lint scope

2026-05-13T21:48:16+00:00

`is_docs_path()` returned True for any path under `docs/`, including the archived RST shadows under `docs/_rst_legacy/` and the build output under `docs/_build/`. Sphinx excludes both from the build (per `docs/conf.py`'s exclude_patterns) and AGENTS marks `_rst_legacy` as reference-only. The linter shouldn't process either. Add a `DOCS_EXCLUDED_SUBDIRS = ('_build', '_rst_legacy')` constant. After confirming a path is under `docs/`, walk each excluded subtree and reject the path if it's contained. Also unify the auto-discover walk fallback to call `is_docs_path()` for the filter — previously it had its own hand-rolled `"_build" not in path` check that didn't handle `_rst_legacy` at all and would have walked the entire legacy archive. Prune `dirs[:]` in-place at each walk level so we don't descend into the excluded subtrees in the first place — optimization on top of correctness. Reverted the `_dirs` -> `dirs` rename here because we now mutate it. Test coverage: 10 hand-coded `is_docs_path()` cases — all pass: - `docs/configuration/foo.md` -> True - `docs/_rst_legacy/foo.rst` -> False (was True) - `docs/_rst_legacy/subdir/rst-foo.rst` -> False (was True) - `docs/_build/html/index.html` -> False (was True) - `docs/_include/foo.txt` -> True (live snippets stay in scope) - `docs` -> True - `AGENTS.md`, `README.md`, `.github/copilot-instructions.md`, `scripts/doc-linter.py` -> False (already correct) Tracked as item 4 of the rolling-side cleanup backlog from PR #2014 / #2019 / #2020 reviews. 🤖 Generated by [robots](https://vyos.io)

ci(doc-linter): distinguish prose-bearing directive fences from code blocks

2026-05-13T21:47:20+00:00

The line-length skip was test_line_length = not (in_md_fence or in_rst_codeblock) `in_md_fence` was True for every MyST/Markdown fence regardless of content type. That includes admonition directives like `:::{note}`, `:::{warning}`, `:::{tip}` whose content is normal prose, not preformatted code. Long lines in admonitions were silently skipped, contradicting the documented 80-char rule which exempts code blocks only. Track an `is_code` property on each fence-stack entry. A fence is code-bearing when: - info string is empty (plain ``` per CommonMark), OR - info string doesn't start with `{` (bare language tag like `python`, `bash`, `yaml`), OR - info string is `{}` and `` is in the CODE_BEARING_DIRECTIVES set (`code-block`, `code`, `sourcecode`, `cfgcmd`, `opcmd`, `cmdinclude`, `cmdincludemd`, `literalinclude`, `parsed-literal`, `raw`, `command-output`, `eval-rst`). Anything else is prose-bearing (`{note}`, `{warning}`, `{tip}`, `{deprecated}`, `{seealso}`, …) and its content gets line-length checked. `in_md_code_fence` checks the topmost stack entry — the innermost fence wins, so a `{note}` containing an inner `{code-block}` lints the outer prose lines and skips the inner code-block body. The classic `is_suppression_marker()` call still uses `in_md_fence` because suppression markers are about "any fence depth" not "code-bearing depth". `{eval-rst}` is kept in CODE_BEARING_DIRECTIVES to preserve current behavior — its body is RST and any line-length on nested `.. code-block::` is handled by the separate RST tracker. Tightening eval-rst is a separate change if wanted. Test coverage: - `_fence_is_code` classifier: 16 cases (code-like vs prose-like) all pass. - Integration: long line in `{note}` flagged ✓; long line in ```python``` not flagged ✓; long line in `{cfgcmd}` not flagged ✓; nested `{note}` > ```text``` — inner skipped ✓; nested `{note}` > prose — flagged ✓. Sweep over current `docs/` tree: 28 new warnings surface across the existing pages (long prose inside admonition directives that the previous logic had been silently hiding). CI on PR scope is changed files only, so the new findings appear only when contributors touch those pages — they won't break this PR or future infra PRs. Tracked as item 5 of the rolling-side cleanup backlog from PR #2014 / #2019 / #2020 reviews. 🤖 Generated by [robots](https://vyos.io)

ci(doc-linter): fix RST code-block exit on short dedented lines

2026-05-13T21:44:52+00:00

The dedent check inside `handle_file_action()` was if in_rst_codeblock: if len(line) > rst_codeblock_indent and not line[rst_codeblock_indent].isspace(): in_rst_codeblock = False This worked only when the next line was at least `rst_codeblock_indent + 1` chars long — the indexing `line[rst_codeblock_indent]` requires that. A short dedented line (e.g., a single character at column 0 under a directive indented at column 4) failed the length guard and `in_rst_codeblock` stayed True. The block remained open longer than it should, suppressing line-length checks on subsequent prose until either EOF or the next `.. code-block::` reset the state. Replace with a leading-whitespace-count check: on any non-blank line, exit the block when leading-ws is <= the directive's column. Blank lines don't reset the block context. Test: a 3-line file with `.. code-block:: text` directive at col 0, one body line, a single `a` at col 0, then a 113-char line at col 0. With the old logic the long line is still treated as inside the code block and not flagged. With the new logic the single-`a` dedent exits the block and the long line is flagged as expected. Tracked as items 6 and 12 of the rolling-side cleanup backlog from PR #2014 / #2019 / #2020 reviews. 🤖 Generated by [robots](https://vyos.io)

ci(doc-linter): drop \s prefix from compressed-IPv6 regex branch

2026-05-13T21:44:19+00:00

The leading-compression group in `IPV6GROUPS` was r'(?:\s' + IPV6SEG + r':){1,7}:' The `\s` required whitespace before each hextet in the repeated group. In practice this meant compressed forms with leading hextets — `2001:db8::`, `64:ff9b::`, `fe80::1` — only matched when preceded by whitespace inside the line. The linter calls `lint_ipv6(line.strip())`, so at start-of-stripped-line there's no whitespace, and the address fell through to no match. Real-world impact: a documentation page mentioning `2001:4860:4860::8888` (Google DNS) or `64:ff9b::1` (NAT64 well-known prefix) at the start of a line silently passed the IPv6 documentation-address check. None of the other groups in `IPV6GROUPS` use a `\s` prefix. This one was inconsistent. Drop the `\s` so the branch matches compressed forms directly, like its peers. Verified with 6 hand-coded cases (RFC 3849 doc range, Google DNS, NAT64 prefix, mid-line and start-of-line positions). All pass. Tracked as item 3 of the rolling-side cleanup backlog from PR #2014 / #2019 / #2020 reviews. 🤖 Generated by [robots](https://vyos.io)

ci(doc-linter): check every IP on a line, not just the first

2026-05-13T21:43:41+00:00

`lint_ipv4()` and `lint_ipv6()` used `re.search`, which returns only the first match. A line like Set DNS forwarder 192.0.2.1 then fall back to 8.8.8.8 flagged nothing because `192.0.2.1` (RFC 5737 documentation range) is allowed and the search stopped there. The real public IP `8.8.8.8` slipped through despite being exactly the case the linter was meant to catch. Switch both functions to `re.finditer` and walk every match: return on the first disallowed address; only return None when all matches on the line are allowed (private / multicast / non-global). Also fix the casing of "private space" in both error messages — was "private Space" with a stray capital. Verified with 7 hand-coded cases (allowed + public mixes, boundary cases, IPv6 RFC 3849 / Google DNS). All pass. Tracked as items 1, 2, and 10 of the rolling-side cleanup backlog from PR #2014 / #2019 / #2020 reviews. 🤖 Generated by [robots](https://vyos.io)

ci(doc-linter): delete lint_AS placeholder

2026-05-13T21:43:01+00:00

`lint_AS()` and its `NUMBER` regex were a placeholder for a future AS-number documentation-range check (RFC 5398). `lint_AS()` was never called from anywhere — it'd merely `pass` on `re.search` hit. Pure dead code that made it look like AS-number linting existed when it didn't. Delete: - the `NUMBER` regex constant - the `lint_AS()` function If/when AS-number linting is actually desired, implement it properly: hook into the lint loop in `handle_file_action()`, return the standard `(message, line, severity)` tuple on violations, and define the allowed AS ranges from `/^AGENTS.md/` (currently 64496–64511 and 65536–65551). Tracked as item 8 of the rolling-side cleanup backlog from PR #2014 / #2019 / #2020 reviews. 🤖 Generated by [robots](https://vyos.io)