Merge pull request #1874 from vyos/feat/docs-llms-txt-current

docs: render llms.txt at build time (current)

4 files changed, 157 insertions, 60 deletions

diff --git a/CLAUDE.md b/CLAUDE.md
index 60f7d13f..6c07c790 100644
--- a/CLAUDE.md
+++ b/CLAUDE.md
@@ -105,6 +105,50 @@ Each configuration page should contain:
 4. **Known issues** — problems and workarounds
 5. **Debugging** — log collection, `show` commands, state indicators
 
+## LLM-Facing Files (`llms.txt`, `llms-full.txt`)
+
+Both files are regenerated on every `html` and `readthedocs` builder run.
+The `dirhtml` builder is intentionally skipped — production publishes
+only via `html`/`readthedocs`, and we don't render llms.txt for builds
+we don't ship. Local `make dirhtml` is a developer convenience and
+won't emit `llms.txt`.
+
+Files are shipped at the docs root for each version
+(`https://docs.vyos.io/en/<version>/llms.txt`, `.../llms-full.txt`).
+
+- **`llms-full.txt`** — auto-generated by the `sphinx_llms_txt` extension from
+  the full corpus. No curation; configured by `llms_txt_file = False` (which
+  disables the extension's *index* output, not the full output).
+- **`llms.txt`** — curated overview rendered at build time from
+  `docs/_templates/llms.txt.j2`. URLs and the version line are interpolated
+  from `html_baseurl` and `release` so the file always matches the branch.
+  The render lives in `_write_llms_txt(app, exception)` in `docs/conf.py`,
+  wired via `app.connect('build-finished', ...)`.
+
+When adding new top-level sections to the docs, add a corresponding bullet in
+`docs/_templates/llms.txt.j2`. Branch-specific differences (e.g. sagitta has
+no `vpp/` or `contributing/index`) live in that branch's copy of the template.
+
+## Read the Docs Layout
+
+RTD slugs as of 2026-05-04 (verified via API):
+
+| Slug | Verbose | Branch | Role |
+|---|---|---|---|
+| `rolling` | current | `current` | canonical for rolling/next major |
+| `1.5` | circinus | `circinus` | canonical for current LTS |
+| `1.4` | sagitta | `sagitta` | canonical for previous LTS |
+| `1.3`, `1.2` | equuleus, crux | older | canonical for older releases |
+
+URL-level redirect aliases (resolve to the canonicals above):
+`/en/latest/* → /en/rolling/`, `/en/lts/* → /en/1.5/`,
+`/en/stable/* → /en/lts/`, `/en/circinus*`, `/en/sagitta*`,
+`/en/equuleus*`, `/en/crux*` → numeric slugs.
+
+`html_baseurl` per branch must point at the canonical (numeric or `rolling`),
+not the alias, so `<link rel="canonical">` and the sitemap match what RTD
+serves and crawlers skip the redirect hop.
+
 ## CI
 
 - **Linter** (`doc-linter.py` from `vyos/.github`): checks line length and IP addresses on changed files only
diff --git a/docs/_html_extra/llms.txt b/docs/_html_extra/llms.txt
deleted file mode 100644
index a8f5b002..00000000
--- a/docs/_html_extra/llms.txt
+++ /dev/null
@@ -1,52 +0,0 @@
-# VyOS
-
-> VyOS is a free, open-source network operating system based on Debian GNU/Linux.
-> It provides routing, firewall, and VPN functionality via a unified CLI using
-> `set`/`delete`/`show` command hierarchy. Current rolling release is 1.5.x (circinus).
-
-VyOS configuration follows a tree structure. All configuration commands start with
-`set` (to add/change) or `delete` (to remove). Changes are staged and applied with
-`commit`. The CLI hierarchy maps directly to the documentation structure.
-
-## Quick Start
-
-- [Quick Start Guide](https://docs.vyos.io/en/latest/quick-start.html): Minimal setup walkthrough
-- [CLI Overview](https://docs.vyos.io/en/latest/cli.html): Command-line interface usage
-
-## Configuration
-
-- [Firewall](https://docs.vyos.io/en/latest/configuration/firewall/index.html): Zone-based firewall, rules, groups
-- [Interfaces](https://docs.vyos.io/en/latest/configuration/interfaces/index.html): Ethernet, bonding, bridge, VLAN, tunnel, wireless
-- [Protocols](https://docs.vyos.io/en/latest/configuration/protocols/index.html): BGP, OSPF, IS-IS, static routing, MPLS
-- [VPN](https://docs.vyos.io/en/latest/configuration/vpn/index.html): IPsec, OpenVPN, WireGuard, L2TP, PPTP
-- [NAT](https://docs.vyos.io/en/latest/configuration/nat/index.html): Source NAT, destination NAT, NAT66
-- [System](https://docs.vyos.io/en/latest/configuration/system/index.html): DNS, NTP, syslog, users, task scheduler
-- [High Availability](https://docs.vyos.io/en/latest/configuration/highavailability/index.html): VRRP
-- [Load Balancing](https://docs.vyos.io/en/latest/configuration/loadbalancing/index.html): WAN and reverse proxy
-- [Containers](https://docs.vyos.io/en/latest/configuration/container/index.html): Podman-based container support
-- [PKI](https://docs.vyos.io/en/latest/configuration/pki/index.html): Certificate management
-- [Policy](https://docs.vyos.io/en/latest/configuration/policy/index.html): Route maps, prefix lists, access lists
-- [Traffic Policy](https://docs.vyos.io/en/latest/configuration/trafficpolicy/index.html): QoS and shaping
-- [Service](https://docs.vyos.io/en/latest/configuration/service/index.html): DHCP, DNS forwarding, SNMP, SSH, HTTPS API
-- [VRF](https://docs.vyos.io/en/latest/configuration/vrf/index.html): Virtual routing and forwarding
-
-## Operations
-
-- [Operational Commands](https://docs.vyos.io/en/latest/operation/index.html): Show, monitor, restart commands
-
-## Installation
-
-- [Installation Guide](https://docs.vyos.io/en/latest/installation/index.html): Bare metal, virtual, cloud deployments
-
-## Automation
-
-- [Automation](https://docs.vyos.io/en/latest/automation/index.html): Ansible, Terraform, HTTP API, NETCONF
-
-## Configuration Examples
-
-- [Blueprints](https://docs.vyos.io/en/latest/configexamples/index.html): Real-world topology examples
-
-## Optional
-
-- [Contributing](https://docs.vyos.io/en/latest/contributing/index.html): Development workflow
-- [VPP](https://docs.vyos.io/en/latest/vpp/index.html): Vector Packet Processing integration
diff --git a/docs/_templates/llms.txt.j2 b/docs/_templates/llms.txt.j2
new file mode 100644
index 00000000..1c6a3900
--- /dev/null
+++ b/docs/_templates/llms.txt.j2
@@ -0,0 +1,52 @@
+# VyOS
+
+> VyOS is a free, open-source network operating system based on Debian GNU/Linux.
+> It provides routing, firewall, and VPN functionality via a unified CLI using
+> `set`/`delete`/`show` command hierarchy. This documentation covers {{ release }}.
+
+VyOS configuration follows a tree structure. All configuration commands start with
+`set` (to add/change) or `delete` (to remove). Changes are staged and applied with
+`commit`. The CLI hierarchy maps directly to the documentation structure.
+
+## Quick Start
+
+- [Quick Start Guide]({{ baseurl }}quick-start.md): Minimal setup walkthrough
+- [CLI Overview]({{ baseurl }}cli.md): Command-line interface usage
+
+## Configuration
+
+- [Firewall]({{ baseurl }}configuration/firewall/index.md): Zone-based firewall, rules, groups
+- [Interfaces]({{ baseurl }}configuration/interfaces/index.md): Ethernet, bonding, bridge, VLAN, tunnel, wireless
+- [Protocols]({{ baseurl }}configuration/protocols/index.md): BGP, OSPF, IS-IS, static routing, MPLS
+- [VPN]({{ baseurl }}configuration/vpn/index.md): IPsec, OpenVPN, WireGuard, L2TP, PPTP
+- [NAT]({{ baseurl }}configuration/nat/index.md): Source NAT, destination NAT, NAT66
+- [System]({{ baseurl }}configuration/system/index.md): DNS, NTP, syslog, users, task scheduler
+- [High Availability]({{ baseurl }}configuration/highavailability/index.md): VRRP
+- [Load Balancing]({{ baseurl }}configuration/loadbalancing/index.md): WAN and reverse proxy
+- [Containers]({{ baseurl }}configuration/container/index.md): Podman-based container support
+- [PKI]({{ baseurl }}configuration/pki/index.md): Certificate management
+- [Policy]({{ baseurl }}configuration/policy/index.md): Route maps, prefix lists, access lists
+- [Traffic Policy]({{ baseurl }}configuration/trafficpolicy/index.md): QoS and shaping
+- [Service]({{ baseurl }}configuration/service/index.md): DHCP, DNS forwarding, SNMP, SSH, HTTPS API
+- [VRF]({{ baseurl }}configuration/vrf/index.md): Virtual routing and forwarding
+
+## Operations
+
+- [Operational Commands]({{ baseurl }}operation/index.md): Show, monitor, restart commands
+
+## Installation
+
+- [Installation Guide]({{ baseurl }}installation/index.md): Bare metal, virtual, cloud deployments
+
+## Automation
+
+- [Automation]({{ baseurl }}automation/index.md): Ansible, Terraform, HTTP API, NETCONF
+
+## Configuration Examples
+
+- [Blueprints]({{ baseurl }}configexamples/index.md): Real-world topology examples
+
+## Optional
+
+- [Contributing]({{ baseurl }}contributing/index.md): Development workflow
+- [VPP]({{ baseurl }}vpp/index.md): Vector Packet Processing integration
diff --git a/docs/conf.py b/docs/conf.py
index 76a00577..da462b0e 100644
--- a/docs/conf.py
+++ b/docs/conf.py
@@ -26,11 +26,16 @@ project = u'VyOS'
 copyright = u'2026, VyOS maintainers and contributors'
 author = u'VyOS maintainers and contributors'
 
-# The short X.Y version
-version = u'1.5'
+# The short X.Y version (rolling — next major; bumped at branch cut)
+version = u'rolling'
 
-# The full version, including alpha/beta/rc tags
-release = u'1.5.x (circinus)'
+# The full version, including alpha/beta/rc tags. The `current` branch
+# is the rolling tip and serves at /en/rolling/ on RTD; the literal
+# below is exposed in the page footer ("v: rolling (current)") and is
+# interpolated into _templates/llms.txt.j2 ("This documentation covers
+# {{ release }}."). Pin this to a value that names what the rolling
+# docs actually serve, not a stale LTS codename.
+release = u'rolling (current)'
 
 # -- General configuration ---------------------------------------------------
 
@@ -127,7 +132,7 @@ html_static_path = ['_static']
 
 html_extra_path = ['_html_extra']
 
-html_baseurl = 'https://docs.vyos.io/en/latest/'
+html_baseurl = 'https://docs.vyos.io/en/rolling/'
 
 _rtd_version_type = os.environ.get('READTHEDOCS_VERSION_TYPE', '')
 _github_version = (
@@ -144,11 +149,13 @@ html_context = {
     'conf_py_path': '/docs/',
 }
 
-# sphinx-sitemap: baseurl already includes /en/latest/, so skip lang+version
+# sphinx-sitemap: baseurl already includes /en/rolling/, so skip lang+version
 sitemap_url_scheme = '{link}'
 
-# sphinx-llms-txt: disable auto-generated llms.txt, keep curated one from
-# _html_extra; llms-full.txt is still auto-generated
+# sphinx-llms-txt: disable the package's auto-generated index llms.txt.
+# The curated llms.txt is rendered at build time from
+# _templates/llms.txt.j2 by the _write_llms_txt() build-finished hook
+# below; llms-full.txt is still auto-generated by sphinx-llms-txt.
 llms_txt_file = False
 
 # Custom sidebar templates, must be a dictionary that maps document names
@@ -246,6 +253,51 @@ def _prefer_webp(app):
             app.builder.supported_image_types = ['image/webp'] + types
 
 
+def _write_llms_txt(app, exception):
+    # Skip dirhtml: production publishes via the `html` / `readthedocs`
+    # builders only. The `.md` links in the curated template do
+    # actually resolve under `dirhtml` (`_copy_md_sources` puts `.md`
+    # files at their source-relative paths regardless of builder), but
+    # we still don't render llms.txt for builds we don't ship — local
+    # `make dirhtml` is a developer convenience, not a publish target.
+    if exception is not None or app.builder.name not in (
+            'html', 'readthedocs'):
+        return
+    if not app.config.html_baseurl:
+        # Fail loudly rather than rendering /quick-start.md etc. as a
+        # silently-broken root-relative URL — every supported branch
+        # sets html_baseurl, so a missing value is a regression.
+        raise RuntimeError(
+            'html_baseurl must be set to render llms.txt')
+    from pathlib import Path
+    from jinja2 import Environment, FileSystemLoader, StrictUndefined
+    tpl_dir = Path(app.srcdir) / '_templates'
+    out_path = Path(app.outdir) / 'llms.txt'
+    baseurl = app.config.html_baseurl.rstrip('/') + '/'
+    # FileSystemLoader + get_template (rather than from_string) makes
+    # Jinja tracebacks reference the real template filename and line
+    # number — useful when StrictUndefined trips on a typo in
+    # llms.txt.j2. StrictUndefined: missing template variables raise
+    # rather than silently render as empty strings, so a typo in
+    # llms.txt.j2 fails the build instead of shipping a half-blank
+    # llms.txt.
+    env = Environment(
+        loader=FileSystemLoader(str(tpl_dir)),
+        undefined=StrictUndefined,
+        keep_trailing_newline=True,
+        # Plain-text template (not HTML), so HTML autoescape is not
+        # appropriate. Setting autoescape=False explicitly to silence
+        # bandit/ruff S701 and document the intent.
+        autoescape=False,
+    )
+    template = env.get_template('llms.txt.j2')
+    rendered = template.render(
+        baseurl=baseurl,
+        release=app.config.release,
+    )
+    out_path.write_text(rendered, encoding='utf-8')
+
+
 def _copy_md_sources(app, exception):
     """Copy .md source files verbatim into the HTML output tree."""
     if exception is not None:
@@ -261,3 +313,4 @@ def _copy_md_sources(app, exception):
 def setup(app):
     app.connect('builder-inited', _prefer_webp)
     app.connect('build-finished', _copy_md_sources)
+    app.connect('build-finished', _write_llms_txt)