diff options
| -rw-r--r-- | AGENTS.md | 183 | ||||
| -rw-r--r-- | README.md | 30 | ||||
| -rw-r--r-- | scripts/doc-linter.py | 55 |
3 files changed, 209 insertions, 59 deletions
@@ -1,12 +1,18 @@ # AGENTS.md -This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. +This file provides guidance to Claude Code (claude.ai/code) when working +with code in this repository. ## Project -VyOS user documentation, built with Sphinx and hosted on Read the Docs at https://docs.vyos.io. Sources are MyST Markdown (`.md`) for migrated pages and RST (`.rst`) for pages that haven't been migrated yet. Both formats are first-class to Sphinx. +VyOS user documentation, built with Sphinx and hosted on Read the Docs +at https://docs.vyos.io. Sources are MyST Markdown (`.md`) — the +migration off RST is complete, `source_suffix` in `docs/conf.py` is +`['.md']` only, and all canonical pages are `.md`. -Pre-migration RST shadows of migrated pages are archived under `docs/_rst_legacy/` for reference only — they are excluded from the build and not consulted by Sphinx. +Pre-migration RST originals are archived under `docs/_rst_legacy/` for +reference only — they are excluded from the build, not consulted by +Sphinx, and not indexed by Context7. Do not edit them. ## Build @@ -31,16 +37,17 @@ Output: `docs/_build/html/`. ## Lint -The repo doesn't ship a local lint config or pin a linter binary. CI runs -`vyoslinter` (`doc-linter.py` from the `vyos/.github` repo, via the -`lint-doc.yml` workflow) on changed files only — see the CI section -below. For local checks, manually grep for the rules in -[Source conventions](#source-conventions) (line length, address space, -suppression markers). +The repo doesn't ship a local lint config or pin a linter binary. CI +runs `scripts/doc-linter.py` (in-repo, invoked from +`.github/workflows/lint-doc.yml`) on changed files only, scoped to +`docs/` — see the CI section below. For local checks, manually grep +for the rules in [Source conventions](#source-conventions) (line +length, address space, suppression markers). ## Branches and versions -One long-lived branch per VyOS release line. Branch names are constellations sorted by area: +One long-lived branch per VyOS release line. Branch names are +constellations sorted by area: | Branch | VyOS version | |--------|--------------| @@ -50,49 +57,96 @@ One long-lived branch per VyOS release line. Branch names are constellations sor | `equuleus` | 1.3.x (legacy) | | `crux` | 1.2.x (legacy) | -PRs target `rolling`. After merge, request backports via a **post-merge comment** on the PR. Multiple branches go in a single command, space-separated: +PRs target `rolling`. After merge, request backports via a **post-merge +comment** on the PR. Multiple branches go in a single command, +space-separated: ```text @Mergifyio backport circinus sagitta ``` -Only **Maintainers team members** can invoke `@Mergifyio` commands — Mergify silently drops commands from anyone outside the team (no error reply). If a backport doesn't trigger, check team membership first. Ask a Maintainer to post the comment on your behalf. +Only **Maintainers team members** can invoke `@Mergifyio` commands — +Mergify silently drops commands from anyone outside the team (no error +reply). If a backport doesn't trigger, check team membership first. Ask +a Maintainer to post the comment on your behalf. -Mergify only reads commands from **PR comments** — mentions in the PR body are ignored. -Mergify is configured at the org level (no `.mergify.yml` in the repo). The PR template has a `## Backport` section to declare intent, but that does not trigger the backport; the comment does. +Mergify only reads commands from **PR comments** — mentions in the PR +body are ignored. +Mergify is configured at the org level (no `.mergify.yml` in the repo). +The PR template has a `## Backport` section to declare intent, but that +does not trigger the backport; the comment does. ## Architecture ### Sphinx config (`docs/conf.py`) -- `source_suffix = ['.rst', '.md']` — both formats build into the same site. +- `source_suffix = ['.md']` — Sphinx only picks up MyST Markdown + sources. The pre-migration RST originals under `docs/_rst_legacy/` + are not registered as a source extension and are excluded from the + build. - MyST extensions: `colon_fence`, `deflist`, `fieldlist`, `substitution`. -- `myst_fence_as_directive = ["cfgcmd", "opcmd", "cmdincludemd"]` — MyST fences with these names get parsed as if they were RST directives. This is how command pages stay format-portable. -- Custom modules live in `docs/_ext/` (only files listed in `extensions = [...]` in `conf.py` are actual Sphinx extensions; the others are support scripts loaded ad hoc): - - `vyos.py` (Sphinx extension, registered as `vyos`) — defines the `cfgcmd`, `opcmd`, `cmdinclude`, `cmdincludemd`, `cfgcmdlist`, `opcmdlist` directives and `cfgcmd`/`opcmd` roles that drive command coverage tracking. - - `autosectionlabel.py` (Sphinx extension, registered as `autosectionlabel`) — connects to `doctree-read` to register sections as labels. - - `testcoverage.py` — standalone helper that reads VyOS XML command definitions and exposes coverage stats; not a Sphinx extension. - - `releasenotes.py` — standalone release-notes/changelog generator script; not a Sphinx extension. +- `myst_fence_as_directive = ["cfgcmd", "opcmd", "cmdincludemd"]` — + MyST fences with these names get parsed as if they were RST + directives. This is how command pages stay format-portable. +- Custom modules live in `docs/_ext/` (only files listed in + `extensions = [...]` in `conf.py` are actual Sphinx extensions; the + others are support scripts loaded ad hoc): + - `vyos.py` (Sphinx extension, registered as `vyos`) — defines the + `cfgcmd`, `opcmd`, `cmdinclude`, `cmdincludemd`, `cfgcmdlist`, + `opcmdlist` directives and `cfgcmd`/`opcmd` roles that drive + command coverage tracking. + - `autosectionlabel.py` (Sphinx extension, registered as + `autosectionlabel`) — connects to `doctree-read` to register + sections as labels. + - `testcoverage.py` — standalone helper that reads VyOS XML command + definitions and exposes coverage stats; not a Sphinx extension. + - `releasenotes.py` — standalone release-notes/changelog generator + script; not a Sphinx extension. ### Source files -- `docs/<subdir>/<page>.md` — canonical MD source for migrated pages. -- `docs/<page>.rst` — canonical RST source for pages that have not been migrated yet (no `rst-` prefix, no MD sibling). -- `docs/_rst_legacy/<subdir>/rst-<page>.rst` — archived pre-migration RST shadows. Excluded from the Sphinx build and from the Context7 index. Reference only. +- `docs/<subdir>/<page>.md` — canonical MyST source for every page. + The migration off RST is complete. +- `docs/_include/<name>.txt` — shared RST snippets included into MyST + pages via `cmdincludemd`. Their content is parsed as RST so the + legacy templates keep working unchanged. +- `docs/_rst_legacy/<subdir>/rst-<page>.rst` — archived pre-migration + RST originals. Excluded from the Sphinx build and from the Context7 + index. Reference only. **Editing rules:** -- Migrated page (has `<page>.md`): edit the `.md`. Do not touch the archived shadow under `_rst_legacy/`. -- Non-migrated page (RST-only): edit the `.rst`. -- New page: write it as `.md` from the start. The `md-` prefix that earlier MyST migration commits used is gone — never add it. + +- Existing page: edit the `.md`. Do not touch the archived original + under `_rst_legacy/`. +- New page: write it as `.md` from the start. The `md-` prefix that + earlier MyST migration commits used is gone — never add it. +- `_include/*.txt` snippets stay RST — see the next section. ### Command directives -`.. cfgcmd::`, `.. opcmd::`, and `.. cmdinclude::` are the VyOS-specific Sphinx directives in RST. They are tracked for command coverage — do **not** convert them to plain `.. code-block::`. In MyST the same directives are written as fenced blocks: ```` ```{cfgcmd} set system ... ```` (enabled by `myst_fence_as_directive`). The MyST include directive is named `cmdincludemd` (not `cmdinclude`) so that template parsing follows MyST rules in MD pages and RST rules in RST pages — pick `cmdinclude` in `.rst`, `cmdincludemd` in `.md`. +The VyOS-specific Sphinx directives are `cfgcmd`, `opcmd`, and +`cmdincludemd`. In MyST pages they are written as fenced code blocks +with `{cfgcmd}`, `{opcmd}`, or `{cmdincludemd}` as the info string +(enabled by `myst_fence_as_directive`). They are tracked for command +coverage — do **not** replace them with plain `text` or `bash` +fences. + +For RST contexts (`{eval-rst}` blocks and `_include/*.txt` snippets), +the directives are written `.. cfgcmd::`, `.. opcmd::`, and +`.. cmdinclude::`. `cmdinclude` is the RST-side include form; +`cmdincludemd` is the MyST-side form. They resolve to the same +include logic but follow the host file's parser, so pick +`cmdinclude` in `.txt`/RST contexts and `cmdincludemd` in `.md`. ## Source conventions ### RST heading hierarchy +Applies only to RST contexts — `_include/*.txt` snippets and +`{eval-rst}` blocks inside MyST pages. Canonical pages are MyST and +use ATX `#` / `##` / `###` etc. headings; this hierarchy does not +apply to them. + ``` ##### Title (overline+underline, one per file) ***** Chapters @@ -102,15 +156,20 @@ Mergify is configured at the org level (no `.mergify.yml` in the repo). The PR t """"" Paragraphs ``` -The first heading in every RST file uses `#` overline+underline. Field lists (e.g., `:lastproofread:`) or labels may precede it. +The first heading in every embedded RST snippet that introduces a +title uses `#` overline+underline. Field lists (e.g., +`:lastproofread:`) or labels may precede it. ### Formatting -- 80-character line limit (exception: inside `.. code-block::` / fenced code blocks — `<pre>` preserves source verbatim). +- 80-character line limit (exception: inside `.. code-block::` / + fenced code blocks — `<pre>` preserves source verbatim). - American English. - Indent with 2 spaces. - Blank lines around headings. -- Inline code: use double backticks per RST convention (the [Inline markup](https://docutils.sourceforge.io/docs/user/rst/quickref.html#inline-markup) section of the docutils quick reference). +- Inline code: single backticks in MyST (the canonical form). Double + backticks only inside `{eval-rst}` blocks and `_include/*.txt` + snippets, per RST convention. ### IP addresses (linter-enforced) @@ -121,10 +180,13 @@ Allowed without suppression: - Loopback (`127.0.0.0/8`), link-local (`169.254.0.0/16`), `0.0.0.0/0` Allowed ASN: `64496-64511` (16-bit), `65536-65551` (32-bit). -Allowed MAC ranges: `00-53-00`–`00-53-FF` (unicast), `90-10-00`–`90-10-FF` (multicast). +Allowed MAC ranges: `00-53-00`–`00-53-FF` (unicast), +`90-10-00`–`90-10-FF` (multicast). **Requires `stop/start_vyoslinter` suppression:** -- Real public IPs (e.g., `8.8.8.8` for DNS examples). + +- Real public IPs (e.g., a DNS server's address in a DNS forwarder + example, or an upstream peer's address in an EBGP example). - NAT64 well-known prefix `64:ff9b::/96`. - Lines over 80 chars (URLs, certificate fingerprints). @@ -140,23 +202,38 @@ Allowed MAC ranges: `00-53-00`–`00-53-FF` (unicast), `90-10-00`–`90-10-FF` ( .. start_vyoslinter ``` -In MyST `.md` files use the comment form `% stop_vyoslinter` / `% start_vyoslinter` for top-level Markdown content. Inside `{eval-rst}` blocks (where the embedded content is parsed as RST) keep the RST form `.. stop_vyoslinter` / `.. start_vyoslinter` — the linter scans the source line literally and only the form that matches the surrounding parser is recognized. Likewise, `.txt` template files (included via `{include}` or `cmdincludemd`) keep the RST form. +In MyST `.md` files use the comment form `% stop_vyoslinter` / +`% start_vyoslinter` for top-level Markdown content. Inside +`{eval-rst}` blocks (where the embedded content is parsed as RST) +keep the RST form `.. stop_vyoslinter` / `.. start_vyoslinter` — the +linter scans the source line literally and only the form that matches +the surrounding parser is recognized. Likewise, `.txt` template files +(included via `{include}` or `cmdincludemd`) keep the RST form. -Markers must always come in pairs. Indentation may match the surrounding directive (indented inside a block) or sit at column 0 (top-level) — both are valid. +Markers must always come in pairs. Indentation may match the +surrounding directive (indented inside a block) or sit at column 0 +(top-level) — both are valid. ### Configuration page structure 1. **Theory** — what it is, when to use it, relevant RFCs. -2. **Configuration** — all CLI options as `.. cfgcmd::` (RST) or ```` ```{cfgcmd} ```` (MD). +2. **Configuration** — all CLI options as `.. cfgcmd::` directives + (in RST contexts) or `{cfgcmd}` fenced code blocks (in MD). 3. **Examples** — practical configurations with topology diagrams. 4. **Known issues** — problems and workarounds. 5. **Debugging** — log collection, `show` commands, state indicators. -### `.. TODO::` markers +### `{todo}` markers + +In MyST pages, write TODO markers as `{todo}` fenced directives +(triple-backtick or `:::` fenced blocks with `{todo}` as the info +string). In RST contexts (`{eval-rst}` blocks, `_include/*.txt` +snippets) use the RST form `.. TODO::`. Two valid uses: -Two valid uses: -1. **Tracking** marker on pages that still need `cfgcmd`/`opcmd` conversion — intentional. -2. **Stale** marker on pages that already have full content — should be removed. +1. **Tracking** marker on pages that still need `cfgcmd`/`opcmd` + conversion — intentional. +2. **Stale** marker on pages that already have full content — should + be removed. A PR that both adds and removes TODOs is not contradictory; intent matters. @@ -180,9 +257,10 @@ Files are shipped at the docs root for each version The render lives in `_write_llms_txt(app, exception)` in `docs/conf.py`, wired via `app.connect('build-finished', ...)`. -When adding new top-level sections to the docs, add a corresponding bullet in -`docs/_templates/llms.txt.j2`. Branch-specific differences (e.g. sagitta has -no `vpp/index.md` or `contributing/index.md`) live in that branch's copy of the template. +When adding new top-level sections to the docs, add a corresponding +bullet in `docs/_templates/llms.txt.j2`. Branch-specific differences +(e.g. sagitta has no `vpp/index.md` or `contributing/index.md`) live +in that branch's copy of the template. ## Read the Docs Layout @@ -209,8 +287,13 @@ serves and crawlers skip the redirect hop. ## CI -- **vyoslinter** (`doc-linter.py` from the `vyos/.github` repo, run via `lint-doc.yml`) — line length and IP rules, on changed files only. -- **Sphinx build** — runs on Read the Docs for every PR; preview URL appears as a check. +- **doc-linter** (`scripts/doc-linter.py` in-repo, invoked via + `.github/workflows/lint-doc.yml`) — line length and IP rules, on + changed files under `docs/` only. Repo-root meta files + (README.md, AGENTS.md, `.github/copilot-instructions.md`) are out + of scope. +- **Sphinx build** — runs on Read the Docs for every PR; preview URL + appears as a check. - **CLA check** — contributors must sign the VyOS CLA before merge. - **Conflict check** — fails the PR if it doesn't merge cleanly into base. @@ -220,13 +303,17 @@ Two bots run at separate stages — do not mix them: | Bot | When to trigger | How | |-----|-----------------|-----| -| **Copilot** | **Draft PRs only** | Comment `@copilot review` | -| **CodeRabbit** | **Ready-for-review PRs only** | Comment `@coderabbitai review` (auto-reviews are disabled on this repo) | +| **Copilot** | Draft PRs only | Comment `@copilot review` | +| **CodeRabbit** | Ready-for-review PRs only | Comment `@coderabbitai review` | + +Auto-reviews are disabled on this repo — both bots are triggered +manually via the comments shown above. Workflow: 1. Open PR as draft (`gh pr create --draft`). 2. Iterate; when complete, comment `@copilot review`. -3. Address Copilot threads, re-request after each fix round until Copilot is silent. +3. Address Copilot threads, re-request after each fix round until + Copilot is silent. 4. Flip to ready (`gh pr ready <num>`), then comment `@coderabbitai review`. 5. Address CodeRabbit threads the same way. @@ -3,10 +3,27 @@ Source for the VyOS user documentation hosted on Read the Docs at https://docs.vyos.io. -[](https://docs.vyos.io/en/rolling/?badge=rolling) +[![Documentation Status][badge]][rtd] The earlier wiki for VyOS 1.1.x and pre-1.2.0 docs is preserved on the -[Wayback Machine](https://web.archive.org/web/20200225171529/https://wiki.vyos.net/wiki/Main_Page). +[Wayback Machine][wayback]. + +[badge]: https://readthedocs.org/projects/vyos/badge/?version=rolling +[rtd]: https://docs.vyos.io/en/rolling/?badge=rolling +[wayback]: https://web.archive.org/web/2020/https://wiki.vyos.net/wiki/Main_Page + +## Source format + +Pages are [MyST Markdown](https://myst-parser.readthedocs.io/) (`.md`) and +are built with Sphinx — `source_suffix` in `docs/conf.py` lists `.md` only. +The pre-migration RST originals are archived under `docs/_rst_legacy/` +for reference; they are excluded from the build and should not be edited. + +VyOS-specific command directives (`cfgcmd`, `opcmd`, `cmdincludemd`) are +written as MyST fenced blocks and rendered as directives via +`myst_fence_as_directive`. Shared snippets under `docs/_include/*.txt` are +still RST — they are included into MyST pages through `cmdincludemd`, which +parses their content as RST so the legacy templates keep working unchanged. ## Branches @@ -16,7 +33,7 @@ constellation: | Branch | VyOS version | Role | |--------|--------------|------| -| `rolling` | 1.5+ rolling | Default branch — all new contributions target this. | +| `rolling` | 1.5+ rolling | Default branch — new contributions land here. | | `circinus` | 1.5.x | LTS docs. | | `sagitta` | 1.4.x | Previous LTS docs. | | `equuleus` | 1.3.x | Legacy. | @@ -60,6 +77,7 @@ Output lands in `docs/_build/html/`. ## Contributing -See [AGENTS.md](AGENTS.md) for the full contributor guide — source format -conventions (MyST Markdown for migrated pages, RST for the rest), CLI -directive syntax, IP-address rules, the linter, and the bot review workflow. +See [AGENTS.md](AGENTS.md) for the full contributor guide — MyST source +conventions, the VyOS command directives (`cfgcmd` / `opcmd` / +`cmdincludemd`), IP-address rules, the linter and its suppression markers, +and the Copilot/CodeRabbit bot review workflow. diff --git a/scripts/doc-linter.py b/scripts/doc-linter.py index 5ccff488..8e74600f 100644 --- a/scripts/doc-linter.py +++ b/scripts/doc-linter.py @@ -29,6 +29,32 @@ NUMBER = r"([\s']\d+[\s'])" SUPPORTED_EXTS = ('.md', '.rst', '.txt') +# Linter only applies to published documentation sources under docs/. Repo-root +# files (README.md, AGENTS.md, .github/copilot-instructions.md) are project +# meta, not docs content, and are out of scope. +DOCS_ROOT = 'docs' + + +def is_docs_path(path): + """Return True iff `path` resolves under the repo's `docs/` tree. + + Accepts both repo-relative and absolute paths. Both `path` and + `DOCS_ROOT` are resolved with `os.path.realpath`, so symlinks are + followed to their real targets — a symlink under `docs/` that + points outside the tree is correctly treated as out-of-scope, and + a `docs/` that is itself a symlink (e.g., in some CI checkouts) is + correctly treated as the docs root. Paths are normalized against + the linter's working directory (CI invokes it from the repo root; + the same is expected for local runs). + """ + abs_path = os.path.realpath(path) + abs_docs = os.path.realpath(DOCS_ROOT) + try: + return os.path.commonpath([abs_path, abs_docs]) == abs_docs + except ValueError: + # commonpath raises on mixed drives (Windows) or empty input. + return False + # MyST / Markdown fenced code block: leading whitespace + 3+ backticks or 3+ colons. # Same character and length-or-greater closes. MD_FENCE_RE = re.compile(r'^(\s*)(`{3,}|:{3,})(.*)$') @@ -67,6 +93,7 @@ def is_suppression_marker(line, kind, in_md_fence, in_rst_codeblock, def lint_mac(cnt, line): + """Flag MAC addresses outside the RFC 7042 documentation range.""" mac = re.search(MAC, line, re.I) if mac is not None: mac = mac.group() @@ -77,6 +104,7 @@ def lint_mac(cnt, line): def lint_ipv4(cnt, line): + """Flag IPv4 addresses outside RFC 5737 / private / multicast ranges.""" ip = re.search(IPV4ADDR, line, re.I) if ip is not None: ip = ipaddress.ip_address(ip.group().strip(' ')) @@ -91,6 +119,7 @@ def lint_ipv4(cnt, line): def lint_ipv6(cnt, line): + """Flag IPv6 addresses outside RFC 3849 / private / multicast ranges.""" ip = re.search(IPV6ADDR, line, re.I) if ip is not None: ip = ipaddress.ip_address(ip.group().strip(' ')) @@ -104,6 +133,7 @@ def lint_ipv6(cnt, line): def lint_AS(cnt, line): + """Placeholder for future AS-number documentation-range checks (RFC 5398).""" number = re.search(NUMBER, line, re.I) if number: pass @@ -111,11 +141,13 @@ def lint_AS(cnt, line): def lint_linelen(cnt, line): + """Warn when a line exceeds the 80-character docs convention.""" line = line.rstrip() if len(line) > 80: return (f"Line too long: len={len(line)}", cnt, 'warning') def handle_file_action(filepath): + """Run all lint checks on one file, respecting fence/code-block and suppression context.""" errors = [] file_ext = os.path.splitext(filepath)[1].lower() # Stack of open MD/MyST fences: (char, min_len). Supports nesting like @@ -215,16 +247,29 @@ def handle_file_action(filepath): def main(): + """Entry point: lint the changed-file list from argv, or fall back to walking `docs/`.""" bool_error = True print('start') + # Only the argv-parsing step is wrapped in try/except. Errors raised by + # handle_file_action() must propagate so CI failures stay visible instead + # of silently triggering a full docs/ walk. try: files = ast.literal_eval(sys.argv[1]) + except (IndexError, SyntaxError, ValueError): + # No argv or malformed list -> fall back to walking DOCS_ROOT. + files = None + + if files is not None: for file in files: - if file.endswith(SUPPORTED_EXTS) and "_build" not in file: - if handle_file_action(file) is False: - bool_error = False - except Exception as e: - for root, dirs, files in os.walk("docs"): + if ( + file.endswith(SUPPORTED_EXTS) + and "_build" not in file + and is_docs_path(file) + ): + if handle_file_action(file) is False: + bool_error = False + else: + for root, _dirs, files in os.walk(DOCS_ROOT): path = root.split(os.sep) for file in files: if file.endswith(SUPPORTED_EXTS) and "_build" not in path: |