summaryrefslogtreecommitdiff
path: root/scripts/doc-linter.py
AgeCommit message (Collapse)Author
2026-05-14ci(doc-linter): anchor .. code-block:: detection + drop debug printYuriy Andamasov
Addresses Copilot review on PR #2023: 1. .. code-block:: tracking was triggered by a plain substring check, which matched mid-line occurrences too. In MD prose like ``.. code-block::`` (docs/documentation.md:222) this set in_rst_codeblock=True spuriously and could suppress line-length checks downstream. Replace with a leading-whitespace- anchored regex and gate on file_ext in ('.rst', '.txt') or an open {eval-rst} MyST fence so the directive opener is only recognized where it can actually occur. 2. print('start') in main() was leftover debug noise — remove it. (cherry picked from commit e87278ef35660a6257b55f4585274a52d3124583)
2026-05-14ci(doc-linter): fix \b regression in compressed-IPv6 regex — replace bare ↵copilot-swe-agent[bot]
removal with word-boundary prefix Agent-Logs-Url: https://github.com/vyos/vyos-documentation/sessions/cdefcaf2-e89e-4090-b39a-15b385b774df Co-authored-by: andamasov <12631358+andamasov@users.noreply.github.com> (cherry picked from commit be8090a3a09adb950557c2887c9a27c017ffd31d)
2026-05-14ci(doc-linter): lint added + renamed files, not only modifiedYuriy Andamasov
Previous workflow: env: FILES_MODIFIED: ${{ steps.file_changes.outputs.files_modified }} run: python scripts/doc-linter.py "$FILES_MODIFIED" `trilom/file-changes-action`'s `files_modified` output is modifications-only. A PR adding a new `.md`/`.rst` doc page passed `files_added`, never `files_modified`, so a brand-new page with long lines or real public IPs slipped past the linter entirely. Workflow: also pass `files_added` and `files_renamed` as separate positional args. Each output is a JSON array (action v1.2.4) and is passed via env to avoid shell-quoting issues. Linter: `main()` now accepts one OR multiple positional argv entries, each a JSON array of paths. Arrays are merged and deduplicated before linting. Single-arg invocations remain backward-compatible. Switched from `ast.literal_eval` to `json.loads` — the action's outputs are JSON, and `json.loads` is the right tool (and dodges literal_eval-via-`eval`-substring linter warnings). Test coverage: - Two JSON arrays merge -> single linter run on union. - Empty-string argv entry skipped (no `files_renamed` in many PRs). - Malformed JSON -> falls back to walking DOCS_ROOT. - No argv -> walks DOCS_ROOT. - Single-arg invocation -> backward-compat preserved. Tracked as item 7 of the rolling-side cleanup backlog from PR #2014 / #2019 / #2020 reviews. 🤖 Generated by [robots](https://vyos.io) (cherry picked from commit 1ef5684729646ca3a24aff83ab8edd0aa57914c7)
2026-05-14ci(doc-linter): exclude docs/_rst_legacy/ and docs/_build/ from lint scopeYuriy Andamasov
`is_docs_path()` returned True for any path under `docs/`, including the archived RST shadows under `docs/_rst_legacy/` and the build output under `docs/_build/`. Sphinx excludes both from the build (per `docs/conf.py`'s exclude_patterns) and AGENTS marks `_rst_legacy` as reference-only. The linter shouldn't process either. Add a `DOCS_EXCLUDED_SUBDIRS = ('_build', '_rst_legacy')` constant. After confirming a path is under `docs/`, walk each excluded subtree and reject the path if it's contained. Also unify the auto-discover walk fallback to call `is_docs_path()` for the filter — previously it had its own hand-rolled `"_build" not in path` check that didn't handle `_rst_legacy` at all and would have walked the entire legacy archive. Prune `dirs[:]` in-place at each walk level so we don't descend into the excluded subtrees in the first place — optimization on top of correctness. Reverted the `_dirs` -> `dirs` rename here because we now mutate it. Test coverage: 10 hand-coded `is_docs_path()` cases — all pass: - `docs/configuration/foo.md` -> True - `docs/_rst_legacy/foo.rst` -> False (was True) - `docs/_rst_legacy/subdir/rst-foo.rst` -> False (was True) - `docs/_build/html/index.html` -> False (was True) - `docs/_include/foo.txt` -> True (live snippets stay in scope) - `docs` -> True - `AGENTS.md`, `README.md`, `.github/copilot-instructions.md`, `scripts/doc-linter.py` -> False (already correct) Tracked as item 4 of the rolling-side cleanup backlog from PR #2014 / #2019 / #2020 reviews. 🤖 Generated by [robots](https://vyos.io) (cherry picked from commit 379ed4757b62a7c1df965a59bc4f1fd0cef2d3e8)
2026-05-14ci(doc-linter): distinguish prose-bearing directive fences from code blocksYuriy Andamasov
The line-length skip was test_line_length = not (in_md_fence or in_rst_codeblock) `in_md_fence` was True for every MyST/Markdown fence regardless of content type. That includes admonition directives like `:::{note}`, `:::{warning}`, `:::{tip}` whose content is normal prose, not preformatted code. Long lines in admonitions were silently skipped, contradicting the documented 80-char rule which exempts code blocks only. Track an `is_code` property on each fence-stack entry. A fence is code-bearing when: - info string is empty (plain ``` per CommonMark), OR - info string doesn't start with `{` (bare language tag like `python`, `bash`, `yaml`), OR - info string is `{<directive>}` and `<directive>` is in the CODE_BEARING_DIRECTIVES set (`code-block`, `code`, `sourcecode`, `cfgcmd`, `opcmd`, `cmdinclude`, `cmdincludemd`, `literalinclude`, `parsed-literal`, `raw`, `command-output`, `eval-rst`). Anything else is prose-bearing (`{note}`, `{warning}`, `{tip}`, `{deprecated}`, `{seealso}`, …) and its content gets line-length checked. `in_md_code_fence` checks the topmost stack entry — the innermost fence wins, so a `{note}` containing an inner `{code-block}` lints the outer prose lines and skips the inner code-block body. The classic `is_suppression_marker()` call still uses `in_md_fence` because suppression markers are about "any fence depth" not "code-bearing depth". `{eval-rst}` is kept in CODE_BEARING_DIRECTIVES to preserve current behavior — its body is RST and any line-length on nested `.. code-block::` is handled by the separate RST tracker. Tightening eval-rst is a separate change if wanted. Test coverage: - `_fence_is_code` classifier: 16 cases (code-like vs prose-like) all pass. - Integration: long line in `{note}` flagged ✓; long line in ```python``` not flagged ✓; long line in `{cfgcmd}` not flagged ✓; nested `{note}` > ```text``` — inner skipped ✓; nested `{note}` > prose — flagged ✓. Sweep over current `docs/` tree: 28 new warnings surface across the existing pages (long prose inside admonition directives that the previous logic had been silently hiding). CI on PR scope is changed files only, so the new findings appear only when contributors touch those pages — they won't break this PR or future infra PRs. Tracked as item 5 of the rolling-side cleanup backlog from PR #2014 / #2019 / #2020 reviews. 🤖 Generated by [robots](https://vyos.io) (cherry picked from commit cd5759f26a1c18abc3c137234cf1ab503e2b26b7)
2026-05-14ci(doc-linter): fix RST code-block exit on short dedented linesYuriy Andamasov
The dedent check inside `handle_file_action()` was if in_rst_codeblock: if len(line) > rst_codeblock_indent and not line[rst_codeblock_indent].isspace(): in_rst_codeblock = False This worked only when the next line was at least `rst_codeblock_indent + 1` chars long — the indexing `line[rst_codeblock_indent]` requires that. A short dedented line (e.g., a single character at column 0 under a directive indented at column 4) failed the length guard and `in_rst_codeblock` stayed True. The block remained open longer than it should, suppressing line-length checks on subsequent prose until either EOF or the next `.. code-block::` reset the state. Replace with a leading-whitespace-count check: on any non-blank line, exit the block when leading-ws is <= the directive's column. Blank lines don't reset the block context. Test: a 3-line file with `.. code-block:: text` directive at col 0, one body line, a single `a` at col 0, then a 113-char line at col 0. With the old logic the long line is still treated as inside the code block and not flagged. With the new logic the single-`a` dedent exits the block and the long line is flagged as expected. Tracked as items 6 and 12 of the rolling-side cleanup backlog from PR #2014 / #2019 / #2020 reviews. 🤖 Generated by [robots](https://vyos.io) (cherry picked from commit 6628b2901933f67568ba68617ee27c6f843c6dcf)
2026-05-14ci(doc-linter): drop \s prefix from compressed-IPv6 regex branchYuriy Andamasov
The leading-compression group in `IPV6GROUPS` was r'(?:\s' + IPV6SEG + r':){1,7}:' The `\s` required whitespace before each hextet in the repeated group. In practice this meant compressed forms with leading hextets — `2001:db8::`, `64:ff9b::`, `fe80::1` — only matched when preceded by whitespace inside the line. The linter calls `lint_ipv6(line.strip())`, so at start-of-stripped-line there's no whitespace, and the address fell through to no match. Real-world impact: a documentation page mentioning `2001:4860:4860::8888` (Google DNS) or `64:ff9b::1` (NAT64 well-known prefix) at the start of a line silently passed the IPv6 documentation-address check. None of the other groups in `IPV6GROUPS` use a `\s` prefix. This one was inconsistent. Drop the `\s` so the branch matches compressed forms directly, like its peers. Verified with 6 hand-coded cases (RFC 3849 doc range, Google DNS, NAT64 prefix, mid-line and start-of-line positions). All pass. Tracked as item 3 of the rolling-side cleanup backlog from PR #2014 / #2019 / #2020 reviews. 🤖 Generated by [robots](https://vyos.io) (cherry picked from commit cc1b4d7c28272786e39a11b37a3ca22b80b12eec)
2026-05-14ci(doc-linter): check every IP on a line, not just the firstYuriy Andamasov
`lint_ipv4()` and `lint_ipv6()` used `re.search`, which returns only the first match. A line like Set DNS forwarder 192.0.2.1 then fall back to 8.8.8.8 flagged nothing because `192.0.2.1` (RFC 5737 documentation range) is allowed and the search stopped there. The real public IP `8.8.8.8` slipped through despite being exactly the case the linter was meant to catch. Switch both functions to `re.finditer` and walk every match: return on the first disallowed address; only return None when all matches on the line are allowed (private / multicast / non-global). Also fix the casing of "private space" in both error messages — was "private Space" with a stray capital. Verified with 7 hand-coded cases (allowed + public mixes, boundary cases, IPv6 RFC 3849 / Google DNS). All pass. Tracked as items 1, 2, and 10 of the rolling-side cleanup backlog from PR #2014 / #2019 / #2020 reviews. 🤖 Generated by [robots](https://vyos.io) (cherry picked from commit 85c0c1ea222d2c70662de5a03ecaf04b6498006e)
2026-05-14ci(doc-linter): delete lint_AS placeholderYuriy Andamasov
`lint_AS()` and its `NUMBER` regex were a placeholder for a future AS-number documentation-range check (RFC 5398). `lint_AS()` was never called from anywhere — it'd merely `pass` on `re.search` hit. Pure dead code that made it look like AS-number linting existed when it didn't. Delete: - the `NUMBER` regex constant - the `lint_AS()` function If/when AS-number linting is actually desired, implement it properly: hook into the lint loop in `handle_file_action()`, return the standard `(message, line, severity)` tuple on violations, and define the allowed AS ranges from `/^AGENTS.md/` (currently 64496–64511 and 65536–65551). Tracked as item 8 of the rolling-side cleanup backlog from PR #2014 / #2019 / #2020 reviews. 🤖 Generated by [robots](https://vyos.io) (cherry picked from commit 61ecb4e103c6a6225b3d71586f7f65e41c3e3510)
2026-05-14ci(doc-linter): delete lint_mac dead codeYuriy Andamasov
`lint_mac()` was called and its return value immediately overwritten with `None`: err_mac = lint_mac(cnt, line.strip()) # disable mac detection for the moment, too many false positives err_mac = None The comment is correct — MAC linting produced too many false positives — but the cleanup never landed. The dead call ran on every line for every linted file, and the function/MAC-regex/MAC-error-text all sat in the source as a misleading hint that MAC linting was a live feature. Delete: - the `MAC` regex constant - the `lint_mac()` function - the `err_mac = lint_mac(...)`, `err_mac = None`, and the `err_mac` entry in the tuple iterated by the error-collection loop in `handle_file_action()` Tracked as item 9 of the rolling-side cleanup backlog flagged across the PR #2014 / #2019 / #2020 reviews. When MAC linting is genuinely wanted again, recover the regex/function from git history and wire it in cleanly. 🤖 Generated by [robots](https://vyos.io) (cherry picked from commit ac9f61e5ae366774e51975b0faddd7ba07996762)
2026-05-14ci: inline doc-linter on circinus to match rollingYuriy Andamasov
The lint workflow on this branch referenced `vyos/.github/.github/workflows/lint-doc.yml@current`, which no longer exists in `vyos/.github` — `lint-doc.yml` was dropped from the central workflow set during the move toward repo-local linters. The reusable-workflow `uses:` line has been silently failing (no `doc-lint` check on recent circinus PRs). Mirror rolling's approach (inlined 2026-05-10): - Add `scripts/doc-linter.py` in this repo. Sourced from PR #2014's HEAD on rolling (the version that includes the `is_docs_path()` docs/-only scope guard so repo-root meta files like AGENTS.md and README.md are out of scope, plus the realpath / narrow-exception / Ruff fixes from that PR's review pass). - Replace `.github/workflows/lint-doc.yml` with an in-repo workflow that runs `python scripts/doc-linter.py "$FILES_MODIFIED"` against the changed-file list from `trilom/file-changes-action`. Same job shape as rolling. - Update the `## Lint` and `## CI` bullets in AGENTS.md to point at the in-repo paths. This restores actual lint coverage on circinus PRs and keeps the toolchain identical across rolling / circinus / sagitta. Behavior on the 11 remaining `.rst` pages is unchanged — the linter's SUPPORTED_EXTS already covers `.md`, `.rst`, and `.txt`. 🤖 Generated by [robots](https://vyos.io)