| Age | Commit message (Collapse) | Author |
|---|
|
Addresses Copilot review on PR #2023:
1. .. code-block:: tracking was triggered by a plain substring check, which
matched mid-line occurrences too. In MD prose like ``.. code-block::``
(docs/documentation.md:222) this set in_rst_codeblock=True spuriously and
could suppress line-length checks downstream. Replace with a leading-whitespace-
anchored regex and gate on file_ext in ('.rst', '.txt') or an open {eval-rst}
MyST fence so the directive opener is only recognized where it can actually
occur.
2. print('start') in main() was leftover debug noise — remove it.
|
|
removal with word-boundary prefix
Agent-Logs-Url: https://github.com/vyos/vyos-documentation/sessions/cdefcaf2-e89e-4090-b39a-15b385b774df
Co-authored-by: andamasov <12631358+andamasov@users.noreply.github.com>
|
|
Previous workflow:
env:
FILES_MODIFIED: ${{ steps.file_changes.outputs.files_modified }}
run: python scripts/doc-linter.py "$FILES_MODIFIED"
`trilom/file-changes-action`'s `files_modified` output is
modifications-only. A PR adding a new `.md`/`.rst` doc page passed
`files_added`, never `files_modified`, so a brand-new page with long
lines or real public IPs slipped past the linter entirely.
Workflow: also pass `files_added` and `files_renamed` as separate
positional args. Each output is a JSON array (action v1.2.4) and is
passed via env to avoid shell-quoting issues.
Linter: `main()` now accepts one OR multiple positional argv entries,
each a JSON array of paths. Arrays are merged and deduplicated before
linting. Single-arg invocations remain backward-compatible. Switched
from `ast.literal_eval` to `json.loads` — the action's outputs are
JSON, and `json.loads` is the right tool (and dodges
literal_eval-via-`eval`-substring linter warnings).
Test coverage:
- Two JSON arrays merge -> single linter run on union.
- Empty-string argv entry skipped (no `files_renamed` in many PRs).
- Malformed JSON -> falls back to walking DOCS_ROOT.
- No argv -> walks DOCS_ROOT.
- Single-arg invocation -> backward-compat preserved.
Tracked as item 7 of the rolling-side cleanup backlog from PR #2014 /
#2019 / #2020 reviews.
🤖 Generated by [robots](https://vyos.io)
|
|
`is_docs_path()` returned True for any path under `docs/`, including
the archived RST shadows under `docs/_rst_legacy/` and the build
output under `docs/_build/`. Sphinx excludes both from the build
(per `docs/conf.py`'s exclude_patterns) and AGENTS marks
`_rst_legacy` as reference-only. The linter shouldn't process either.
Add a `DOCS_EXCLUDED_SUBDIRS = ('_build', '_rst_legacy')` constant.
After confirming a path is under `docs/`, walk each excluded subtree
and reject the path if it's contained.
Also unify the auto-discover walk fallback to call `is_docs_path()`
for the filter — previously it had its own hand-rolled `"_build"
not in path` check that didn't handle `_rst_legacy` at all and would
have walked the entire legacy archive. Prune `dirs[:]` in-place at
each walk level so we don't descend into the excluded subtrees in
the first place — optimization on top of correctness. Reverted the
`_dirs` -> `dirs` rename here because we now mutate it.
Test coverage: 10 hand-coded `is_docs_path()` cases — all pass:
- `docs/configuration/foo.md` -> True
- `docs/_rst_legacy/foo.rst` -> False (was True)
- `docs/_rst_legacy/subdir/rst-foo.rst` -> False (was True)
- `docs/_build/html/index.html` -> False (was True)
- `docs/_include/foo.txt` -> True (live snippets stay in scope)
- `docs` -> True
- `AGENTS.md`, `README.md`, `.github/copilot-instructions.md`,
`scripts/doc-linter.py` -> False (already correct)
Tracked as item 4 of the rolling-side cleanup backlog from PR #2014
/ #2019 / #2020 reviews.
🤖 Generated by [robots](https://vyos.io)
|
|
The line-length skip was
test_line_length = not (in_md_fence or in_rst_codeblock)
`in_md_fence` was True for every MyST/Markdown fence regardless of
content type. That includes admonition directives like `:::{note}`,
`:::{warning}`, `:::{tip}` whose content is normal prose, not
preformatted code. Long lines in admonitions were silently skipped,
contradicting the documented 80-char rule which exempts code blocks
only.
Track an `is_code` property on each fence-stack entry. A fence is
code-bearing when:
- info string is empty (plain ``` per CommonMark), OR
- info string doesn't start with `{` (bare language tag like
`python`, `bash`, `yaml`), OR
- info string is `{<directive>}` and `<directive>` is in the
CODE_BEARING_DIRECTIVES set (`code-block`, `code`, `sourcecode`,
`cfgcmd`, `opcmd`, `cmdinclude`, `cmdincludemd`, `literalinclude`,
`parsed-literal`, `raw`, `command-output`, `eval-rst`).
Anything else is prose-bearing (`{note}`, `{warning}`, `{tip}`,
`{deprecated}`, `{seealso}`, …) and its content gets line-length
checked.
`in_md_code_fence` checks the topmost stack entry — the innermost
fence wins, so a `{note}` containing an inner `{code-block}` lints
the outer prose lines and skips the inner code-block body. The
classic `is_suppression_marker()` call still uses `in_md_fence`
because suppression markers are about "any fence depth" not
"code-bearing depth".
`{eval-rst}` is kept in CODE_BEARING_DIRECTIVES to preserve current
behavior — its body is RST and any line-length on nested
`.. code-block::` is handled by the separate RST tracker. Tightening
eval-rst is a separate change if wanted.
Test coverage:
- `_fence_is_code` classifier: 16 cases (code-like vs prose-like)
all pass.
- Integration: long line in `{note}` flagged ✓; long line in
```python``` not flagged ✓; long line in `{cfgcmd}` not flagged ✓;
nested `{note}` > ```text``` — inner skipped ✓; nested `{note}` >
prose — flagged ✓.
Sweep over current `docs/` tree: 28 new warnings surface across the
existing pages (long prose inside admonition directives that the
previous logic had been silently hiding). CI on PR scope is changed
files only, so the new findings appear only when contributors touch
those pages — they won't break this PR or future infra PRs.
Tracked as item 5 of the rolling-side cleanup backlog from PR #2014
/ #2019 / #2020 reviews.
🤖 Generated by [robots](https://vyos.io)
|
|
The dedent check inside `handle_file_action()` was
if in_rst_codeblock:
if len(line) > rst_codeblock_indent and not line[rst_codeblock_indent].isspace():
in_rst_codeblock = False
This worked only when the next line was at least
`rst_codeblock_indent + 1` chars long — the indexing
`line[rst_codeblock_indent]` requires that. A short dedented
line (e.g., a single character at column 0 under a directive
indented at column 4) failed the length guard and `in_rst_codeblock`
stayed True. The block remained open longer than it should,
suppressing line-length checks on subsequent prose until either
EOF or the next `.. code-block::` reset the state.
Replace with a leading-whitespace-count check: on any non-blank
line, exit the block when leading-ws is <= the directive's
column. Blank lines don't reset the block context.
Test: a 3-line file with `.. code-block:: text` directive at col
0, one body line, a single `a` at col 0, then a 113-char line
at col 0. With the old logic the long line is still treated as
inside the code block and not flagged. With the new logic the
single-`a` dedent exits the block and the long line is flagged
as expected.
Tracked as items 6 and 12 of the rolling-side cleanup backlog
from PR #2014 / #2019 / #2020 reviews.
🤖 Generated by [robots](https://vyos.io)
|
|
The leading-compression group in `IPV6GROUPS` was
r'(?:\s' + IPV6SEG + r':){1,7}:'
The `\s` required whitespace before each hextet in the repeated
group. In practice this meant compressed forms with leading
hextets — `2001:db8::`, `64:ff9b::`, `fe80::1` — only matched
when preceded by whitespace inside the line. The linter calls
`lint_ipv6(line.strip())`, so at start-of-stripped-line there's
no whitespace, and the address fell through to no match.
Real-world impact: a documentation page mentioning
`2001:4860:4860::8888` (Google DNS) or `64:ff9b::1` (NAT64
well-known prefix) at the start of a line silently passed the
IPv6 documentation-address check.
None of the other groups in `IPV6GROUPS` use a `\s` prefix.
This one was inconsistent. Drop the `\s` so the branch matches
compressed forms directly, like its peers.
Verified with 6 hand-coded cases (RFC 3849 doc range, Google
DNS, NAT64 prefix, mid-line and start-of-line positions). All
pass.
Tracked as item 3 of the rolling-side cleanup backlog from PR
#2014 / #2019 / #2020 reviews.
🤖 Generated by [robots](https://vyos.io)
|
|
`lint_ipv4()` and `lint_ipv6()` used `re.search`, which returns
only the first match. A line like
Set DNS forwarder 192.0.2.1 then fall back to 8.8.8.8
flagged nothing because `192.0.2.1` (RFC 5737 documentation
range) is allowed and the search stopped there. The real
public IP `8.8.8.8` slipped through despite being exactly
the case the linter was meant to catch.
Switch both functions to `re.finditer` and walk every match:
return on the first disallowed address; only return None when
all matches on the line are allowed (private / multicast /
non-global).
Also fix the casing of "private space" in both error
messages — was "private Space" with a stray capital.
Verified with 7 hand-coded cases (allowed + public mixes,
boundary cases, IPv6 RFC 3849 / Google DNS). All pass.
Tracked as items 1, 2, and 10 of the rolling-side cleanup
backlog from PR #2014 / #2019 / #2020 reviews.
🤖 Generated by [robots](https://vyos.io)
|
|
`lint_AS()` and its `NUMBER` regex were a placeholder for a
future AS-number documentation-range check (RFC 5398).
`lint_AS()` was never called from anywhere — it'd merely
`pass` on `re.search` hit. Pure dead code that made it look
like AS-number linting existed when it didn't.
Delete:
- the `NUMBER` regex constant
- the `lint_AS()` function
If/when AS-number linting is actually desired, implement it
properly: hook into the lint loop in `handle_file_action()`,
return the standard `(message, line, severity)` tuple on
violations, and define the allowed AS ranges from
`/^AGENTS.md/` (currently 64496–64511 and 65536–65551).
Tracked as item 8 of the rolling-side cleanup backlog from
PR #2014 / #2019 / #2020 reviews.
🤖 Generated by [robots](https://vyos.io)
|
|
`lint_mac()` was called and its return value immediately overwritten
with `None`:
err_mac = lint_mac(cnt, line.strip())
# disable mac detection for the moment, too many false positives
err_mac = None
The comment is correct — MAC linting produced too many false
positives — but the cleanup never landed. The dead call ran on every
line for every linted file, and the function/MAC-regex/MAC-error-text
all sat in the source as a misleading hint that MAC linting was a
live feature.
Delete:
- the `MAC` regex constant
- the `lint_mac()` function
- the `err_mac = lint_mac(...)`, `err_mac = None`, and the `err_mac`
entry in the tuple iterated by the error-collection loop in
`handle_file_action()`
Tracked as item 9 of the rolling-side cleanup backlog flagged across
the PR #2014 / #2019 / #2020 reviews. When MAC linting is genuinely
wanted again, recover the regex/function from git history and wire it
in cleanly.
🤖 Generated by [robots](https://vyos.io)
|
|
ValueError)
CodeRabbit / Ruff BLE001: the previous 'except Exception as e:' on the
explicit-file-list path caught any error, masking runtime failures
from handle_file_action() as silent fallback behavior. Only input
validation errors from ast.literal_eval(sys.argv[1]) should trigger
the fallback walk.
Refactor:
- Wrap only the parse step in try/except, catching just IndexError
(missing argv[1]), SyntaxError (malformed literal), and ValueError
(non-literal input).
- On parse failure, set files = None and dispatch to the DOCS_ROOT
walk via an explicit 'else' branch.
- On parse success, run the file loop outside the try so any errors
from handle_file_action() propagate normally and CI fails loudly.
Also drops the unused 'as e' (Ruff BLE001 noise) and the implicit
catch of TypeError (e.g. ast.literal_eval('42') returns an int and
'for file in 42:' would have been silently swallowed -> fallback
walk; now it raises clearly).
Verified scenarios:
- explicit file list (CI normal path) -> exit 0.
- no argv -> IndexError caught -> walks DOCS_ROOT.
- malformed argv ('not-a-list') -> SyntaxError caught -> walks DOCS_ROOT.
- explicit list with a non-existent file -> FileNotFoundError
propagates (previously silently triggered a fallback walk).
- explicit list with a non-list literal ('42') -> TypeError
propagates (programming error stays visible).
|
|
CodeRabbit nit (Ruff B007): the dirs variable from os.walk(DOCS_ROOT)
in the auto-discover fallback is unused. Renaming to _dirs makes the
intent explicit and silences the warning.
|
|
Three Copilot findings on ab497bf:
1. is_docs_path() docstring claimed paths 'resolve' under docs/, but
the implementation only normalized via abspath() — a symlink under
docs/ that points outside the tree would be treated as in-scope.
Switch both inputs to os.path.realpath() so symlinks are followed
to their real targets. The reverse case is also handled: if docs/
is itself a symlink (some CI checkouts), realpath() resolves it
consistently for both sides of the commonpath comparison.
Verified with a synthetic case: docs/poison.md -> /etc/hosts now
returns False (with abspath() it returned True).
2. The auto-discover fallback in main() still hardcoded
os.walk('docs') instead of using the new DOCS_ROOT constant.
Use DOCS_ROOT in both paths so the docs root is configured in
exactly one place.
3. Indentation inside 'for file in files:' was double-indented (8
spaces under the for, instead of 4) — pre-existing oddity from
before 65a8e9f, preserved through the is_docs_path() addition.
Normalize to a single indent level under the loop.
CI behavior unchanged: tj-actions/changed-files passes repo-relative
paths with no symlinks under docs/, which were already handled. The
realpath() switch only changes behavior in the symlink-escape case,
which was a bug.
|
|
CodeRabbit Pre-merge Docstring Coverage check reported 50% on
scripts/doc-linter.py (threshold 80%). Add minimal one-line docstrings
to each public function; no behavior change.
🤖 Generated by [robots](https://vyos.io)
|
|
CodeRabbit review on 28224f3 flagged that is_docs_path() introduced
in 65a8e9f only matched repo-relative path strings. An absolute path
to docs/... (e.g., from a local invocation that pre-resolves paths,
or from tooling that uses git ls-files --full-path) would silently
fail the docs/ check and the file would be skipped.
Rewrite the helper to use os.path.commonpath against an absolute
docs/ root computed on each call. Both inputs are normalized to
absolute form, so repo-relative and absolute callers produce the
same result. ValueError from commonpath (mixed Windows drives or
empty input) is caught and treated as 'not a docs path'.
abs_docs is recomputed per call rather than captured at import time
so the helper picks up the actual cwd at invocation, matching the
existing assumption that CI / local runs invoke the linter from the
repo root.
Verified against 12 edge cases:
- repo-relative docs paths (docs, docs/foo.md, docs/sub/dir/foo.md,
./docs/foo.md) -> True.
- repo-relative meta paths (AGENTS.md, README.md,
.github/copilot-instructions.md, docs_other/foo.md) -> False.
- absolute paths inside docs/ -> True; inside repo root but outside
docs/ -> False.
- traversal attempts (../other/foo.md, docs/../AGENTS.md) -> False.
CI behavior unchanged: tj-actions/changed-files passes repo-relative
paths, which were already handled by the previous logic.
|
|
The linter targets published documentation sources; the auto-discover
fallback already walks `docs/` only. CI was passing root-level meta
files (README.md, AGENTS.md, .github/copilot-instructions.md — the
last is a symlink to AGENTS.md) which forced docs-publication
conventions (80-char wrap, RFC IP rules, suppression markers) onto
project meta that has no business obeying them.
Add an `is_docs_path()` guard in `main()` so the explicit-file-list
path matches the auto-discover behavior — only files under `docs/`
are linted. AGENTS.md and the Copilot-instruction symlink are now
out of scope.
Verified:
- `python3 scripts/doc-linter.py "['AGENTS.md', 'README.md', '.github/copilot-instructions.md']"` → exit 0 (all skipped).
- `python3 scripts/doc-linter.py "['docs/_test_lint.md']"` with a real public IP → still errors as expected.
🤖 Generated by [robots](https://vyos.io)
|
|
Two issues from PR review:
1. MD/MyST fence tracking treated any longer same-char fence as a
closer, which would close `:::{note}` (3 cols) when seeing a
nested `::::{code-block}` (4 cols) opener inside it. Real bug
in `docs/configuration/interfaces/wireless.md:198–209` (currently
unobservable because inner code lines are <80 chars).
The "opener has info string / closer has none" heuristic is not
sufficient on its own: there are 2,826 bare-fence opens in the
tree, so info-string presence cannot distinguish opener from
closer.
Fix: stack-based tracking. A fence is treated as a closer only
when (a) the stack is non-empty, (b) char and length match the
top, AND (c) no info string follows. Anything else opens a new
(possibly nested) fence. The outermost fence's info string still
determines the `md_fence_is_eval_rst` flag.
2. `is_suppression_marker()` accepted `% stop_vyoslinter` in any
file outside an MD fence. Per AGENTS.md and the doc-linter
instructions, MyST `% ...` markers are only valid in `.md`
files; a stray `% stop_vyoslinter` in `.rst`/`.txt` should not
silently disable linting. Pass `file_ext` and gate the marker
forms accordingly: `% ...` only in `.md` outside fences;
`.. ...` in `.rst`/`.txt` outside RST code-blocks, or in `.md`
inside an `{eval-rst}` fence.
3. Drop the `not in_rst_codeblock` guard on `.. code-block::`
detection. Each occurrence resets the tracked indent (matches
`origin/rolling` baseline). Without this, code-block-inside-
code-block kept the outer indent and broke dedent detection
(verified regression: `_rst_legacy/configuration/system/
rst-syslog.rst:216` long-line warning was lost; restored).
Verified:
- All 7 original synthetic fixtures pass.
- New fixture `nested.md` (3-col outer wraps 4-col inner with long
line in between fences) produces exactly one warning at the line
outside both fences.
- New fixture `wrongmarker.rst` (`%` in `.rst`) — IP error fires
(marker correctly ignored).
- Full-tree run vs origin/rolling baseline: zero regressions on
pre-existing `.rst`/`.txt` warnings; all new output is `.md`.
🤖 Generated by [robots](https://vyos.io)
|
|
Agent-Logs-Url: https://github.com/vyos/vyos-documentation/sessions/5d679560-8a77-4735-b585-74c09293eea5
Co-authored-by: andamasov <12631358+andamasov@users.noreply.github.com>
|
|
Active docs are now MyST `.md`; the linter previously only inspected
`.rst` and `.txt`, so ~250 active pages were unchecked for IP usage and
line length on every PR.
scripts/doc-linter.py:
- Add `.md` to the extension filter (use `endswith` for correctness;
the prior 4-char slice silently skipped `.md` files).
- Track MyST/Markdown fenced code blocks (```` ``` ```` and `:::`) for
line-length exemption — same semantics as `.. code-block::` for RST.
- Recognize both suppression marker forms: `.. stop_vyoslinter` /
`.. start_vyoslinter` (RST and `.txt` includes) and `% stop_vyoslinter`
/ `% start_vyoslinter` (MyST). Both work in either context; pick the
form that matches the surrounding parser.
- Replace the brittle `try/finally: fp.close()` with a `with` block —
the previous form raised `UnboundLocalError` if `open()` itself
failed.
- Fix typo `forgett` → `forget`.
.github/instructions/rst-linter.instructions.md → doc-linter.instructions.md:
- Broaden `applyTo` from `**/*.rst` to `**/*.md,**/*.rst,**/*.txt`.
- Document MyST suppression syntax and fenced-code line-length
exemption.
- Note the parser-form rule for `{eval-rst}` blocks.
No regression on `.txt` includes: identical lint output verified
against the origin/rolling baseline on a sample of files.
Pre-existing IP violations exist in 14 `.md` files (e.g.
`configexamples/lac-lns.md` line 95 — a `8.8.8.8` already wrapped in
`% stop_vyoslinter`/`% start_vyoslinter`, correctly suppressed). PRs
touching unsuppressed violations will start failing CI; this is the
intent of enabling the check.
🤖 Generated by [robots](https://vyos.io)
|
|
The reusable lint-doc workflow at vyos/.github checks out vyos/.github
on the consumer's PR base.ref to source doc-linter.py — designed for
per-release-train linter rules. With this repo's default renamed
current → rolling and vyos/.github still on current, the checkout
errors with "fetch +refs/heads/rolling*: exit code 1".
Rather than chase branch parity across repos, move the linter where it
belongs: doc-linter.py is doc-specific and only consumed here. Inlining
removes the cross-repo coupling permanently and unblocks any future
branch renames in this repo without touching vyos/.github.
- scripts/doc-linter.py: copied byte-for-byte from
vyos/.github@current:.github/doc-linter.py (sha
3dc7c2fc16242e62b0ea7107f767577e999ca417 — identical across all four
release-train branches in vyos/.github, so no behavioral change).
- .github/workflows/lint-doc.yml: replaces `uses:
vyos/.github/.github/workflows/lint-doc.yml@current` with the inlined
steps. Same actions (bullfrogsec/bullfrog, trilom/file-changes-action,
setup-python) and the same final invocation, just sourcing the script
from this repo. Adds explicit minimal permissions (contents/pull-requests
read) and passes the file list via env var to follow the workflow-
injection guidance.
Follow-up: vyos/.github still hosts the now-orphaned doc-linter.py and
its reusable workflow — separate cleanup PR can delete them once any
other consumers migrate (none observed today; this repo was the only
caller).
🤖 Generated by [robots](https://vyos.io)
|
