From b61087c396860bb507be1adea50244a8c8e3242c Mon Sep 17 00:00:00 2001 From: Yuriy Andamasov Date: Wed, 6 May 2026 17:47:58 +0300 Subject: feat: import MyST swap mechanism, llms.txt feature, and content from current Replaces the broken #1885/#1869/#1875 trio with a single PR for circinus. This PR: - Imports 254 md-*.md files from currents working MyST conversion (the broken converter that produced #1885s md-*.md files is not used here; currents files are correct as verified on /en/rolling/). - Adds the per-page RST-to-MyST swap mechanism: scripts/swap_sources.py, scripts/import_myst.py, tests, _swap.txt, _ext/vyos.py and Makefile swap-wrapped targets, .readthedocs.yml pre/post hooks. - Replaces 175 .jpg/.png with 187 .webp images for swapped pages. - Adds the llms.txt + sitemap feature (sphinx_llms_txt, sphinx_sitemap) with circinus-tailored html_baseurl https://docs.vyos.io/en/1.5/ and curated docs/_html_extra/llms.txt for 1.5.x. - Updates docs/_html_extra/robots.txt with AI crawler Allow rules and the /en/1.5/sitemap.xml reference. - Drops configuration/service/suricata from the swap set (no matching RST sibling on circinus; suricata is a current-only feature). Supersedes: - #1885 (broken md-*.md converter output) - #1869 (LLM doc adaptation backport, bundled here) - #1875 (llms.txt circinus, bundled here) Generated by robots https://vyos.io --- docs/_html_extra/llms.txt | 52 +++++++++++++++++++++++++++++++++++++++++++++ docs/_html_extra/robots.txt | 39 +++++++++++++++++++++++++++++----- 2 files changed, 86 insertions(+), 5 deletions(-) create mode 100644 docs/_html_extra/llms.txt (limited to 'docs/_html_extra') diff --git a/docs/_html_extra/llms.txt b/docs/_html_extra/llms.txt new file mode 100644 index 00000000..03d8562d --- /dev/null +++ b/docs/_html_extra/llms.txt @@ -0,0 +1,52 @@ +# VyOS + +> VyOS is a free, open-source network operating system based on Debian GNU/Linux. +> It provides routing, firewall, and VPN functionality via a unified CLI using +> `set`/`delete`/`show` command hierarchy. This documentation covers 1.5.x (circinus). + +VyOS configuration follows a tree structure. All configuration commands start with +`set` (to add/change) or `delete` (to remove). Changes are staged and applied with +`commit`. The CLI hierarchy maps directly to the documentation structure. + +## Quick Start + +- [Quick Start Guide](https://docs.vyos.io/en/1.5/quick-start.html): Minimal setup walkthrough +- [CLI Overview](https://docs.vyos.io/en/1.5/cli.html): Command-line interface usage + +## Configuration + +- [Firewall](https://docs.vyos.io/en/1.5/configuration/firewall/index.html): Zone-based firewall, rules, groups +- [Interfaces](https://docs.vyos.io/en/1.5/configuration/interfaces/index.html): Ethernet, bonding, bridge, VLAN, tunnel, wireless +- [Protocols](https://docs.vyos.io/en/1.5/configuration/protocols/index.html): BGP, OSPF, IS-IS, static routing, MPLS +- [VPN](https://docs.vyos.io/en/1.5/configuration/vpn/index.html): IPsec, OpenVPN, WireGuard, L2TP, PPTP +- [NAT](https://docs.vyos.io/en/1.5/configuration/nat/index.html): Source NAT, destination NAT, NAT66 +- [System](https://docs.vyos.io/en/1.5/configuration/system/index.html): DNS, NTP, syslog, users, task scheduler +- [High Availability](https://docs.vyos.io/en/1.5/configuration/highavailability/index.html): VRRP +- [Load Balancing](https://docs.vyos.io/en/1.5/configuration/loadbalancing/index.html): WAN and reverse proxy +- [Containers](https://docs.vyos.io/en/1.5/configuration/container/index.html): Podman-based container support +- [PKI](https://docs.vyos.io/en/1.5/configuration/pki/index.html): Certificate management +- [Policy](https://docs.vyos.io/en/1.5/configuration/policy/index.html): Route maps, prefix lists, access lists +- [Traffic Policy](https://docs.vyos.io/en/1.5/configuration/trafficpolicy/index.html): QoS and shaping +- [Service](https://docs.vyos.io/en/1.5/configuration/service/index.html): DHCP, DNS forwarding, SNMP, SSH, HTTPS API +- [VRF](https://docs.vyos.io/en/1.5/configuration/vrf/index.html): Virtual routing and forwarding + +## Operations + +- [Operational Commands](https://docs.vyos.io/en/1.5/operation/index.html): Show, monitor, restart commands + +## Installation + +- [Installation Guide](https://docs.vyos.io/en/1.5/installation/index.html): Bare metal, virtual, cloud deployments + +## Automation + +- [Automation](https://docs.vyos.io/en/1.5/automation/index.html): Ansible, Terraform, HTTP API, NETCONF + +## Configuration Examples + +- [Blueprints](https://docs.vyos.io/en/1.5/configexamples/index.html): Real-world topology examples + +## Optional + +- [Contributing](https://docs.vyos.io/en/1.5/contributing/index.html): Development workflow +- [VPP](https://docs.vyos.io/en/1.5/vpp/index.html): Vector Packet Processing integration diff --git a/docs/_html_extra/robots.txt b/docs/_html_extra/robots.txt index 627cb18e..98245d28 100644 --- a/docs/_html_extra/robots.txt +++ b/docs/_html_extra/robots.txt @@ -1,7 +1,36 @@ -User-agent: atlassian-bot -Disallow: - User-agent: * -Disallow: # Allow everything +Allow: / + +# AI Training Crawlers +User-agent: GPTBot +Allow: / + +User-agent: ClaudeBot +Allow: / + +User-agent: Google-Extended +Allow: / + +User-agent: CCBot +Allow: / + +User-agent: PerplexityBot +Allow: / + +# AI Search/Retrieval +User-agent: ChatGPT-User +Allow: / + +User-agent: Claude-SearchBot +Allow: / + +User-agent: Claude-User +Allow: / + +User-agent: OAI-SearchBot +Allow: / + +User-agent: Perplexity-User +Allow: / -Sitemap: https://docs.vyos.io/sitemap.xml \ No newline at end of file +Sitemap: https://docs.vyos.io/en/1.5/sitemap.xml -- cgit v1.2.3