From de78075957f7f01a268ca26a4927fd415f901472 Mon Sep 17 00:00:00 2001 From: Yuriy Andamasov Date: Mon, 30 Mar 2026 01:22:15 +0300 Subject: docs: add documentation modernization specs and project summary MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Six design specs covering LLM adaptation, RST→MyST migration, Cloudflare hosting, visual editor, mobile performance, and visitor tracking. Includes project summary PDF for team sharing. Co-Authored-By: Claude Opus 4.6 (1M context) --- ...26-03-29-cloudflare-hosting-migration-design.md | 316 ++ ...26-03-29-llm-documentation-adaptation-design.md | 207 + .../specs/2026-03-29-mobile-performance-design.md | 142 + .../specs/2026-03-29-project-summary.md | 196 + .../specs/2026-03-29-project-summary.pdf | 5496 ++++++++++++++++++++ .../2026-03-29-rst-to-myst-migration-design.md | 145 + .../specs/2026-03-29-visitor-tracking-design.md | 229 + .../specs/2026-03-29-visual-editor-design.md | 161 + 8 files changed, 6892 insertions(+) create mode 100644 docs/superpowers/specs/2026-03-29-cloudflare-hosting-migration-design.md create mode 100644 docs/superpowers/specs/2026-03-29-llm-documentation-adaptation-design.md create mode 100644 docs/superpowers/specs/2026-03-29-mobile-performance-design.md create mode 100644 docs/superpowers/specs/2026-03-29-project-summary.md create mode 100644 docs/superpowers/specs/2026-03-29-project-summary.pdf create mode 100644 docs/superpowers/specs/2026-03-29-rst-to-myst-migration-design.md create mode 100644 docs/superpowers/specs/2026-03-29-visitor-tracking-design.md create mode 100644 docs/superpowers/specs/2026-03-29-visual-editor-design.md (limited to 'docs/superpowers') diff --git a/docs/superpowers/specs/2026-03-29-cloudflare-hosting-migration-design.md b/docs/superpowers/specs/2026-03-29-cloudflare-hosting-migration-design.md new file mode 100644 index 00000000..88ac56cd --- /dev/null +++ b/docs/superpowers/specs/2026-03-29-cloudflare-hosting-migration-design.md @@ -0,0 +1,316 @@ +# Cloudflare Pages Hosting Migration + +**Date:** 2026-03-29 +**Status:** Approved +**Supersedes:** `docs/specs/2026-03-23-cloudflare-hosting-design.md` +**Depends on:** RST → MyST Migration (must merge before DNS cutover) +**Related:** LLM Adaptation (parallel, URL update at cutover), Mobile Performance (Phase 3), Visual Editor (Phase 3) +**Execution:** Phase 2 — after MyST Migration merges. LLM Adaptation can proceed in parallel. + +## Goal + +Migrate VyOS documentation hosting from ReadTheDocs to Cloudflare Pages, gaining: + +- Unlimited free bandwidth with 300+ edge PoP CDN +- Full control over cache headers (vs RTD's fixed 20-minute TTL) +- `_redirects` and `_headers` file support +- Cloudflare Workers + D1 as a future platform for the visual editor API +- No ads (RTD community tier injects EthicalAds) + +## Platform Selection + +Evaluated 7 hosting platforms. Cloudflare Pages won on every relevant criterion: + +| | Cloudflare Pages | GitHub Pages | AWS S3+CF | Netlify | Vercel | +|---|---|---|---|---|---| +| Cost | $0 | $0 | ~$0.50/mo | $0 (20 deploys/mo) | $20/mo (commercial) | +| Bandwidth | Unlimited | 100 GB soft | 1 TB free | ~30 GB (credits) | 100 GB | +| CDN | 300+ PoPs | Fastly, no control | 600+ PoPs | Fair | Good | +| Cache control | `_headers` file | None | Full | `_headers` file | `vercel.json` | +| Redirects | `_redirects` (2100) | None | CF Functions | `_redirects` | 2048 routes | +| Future web app | Workers + D1 | No | Lambda/ECS | Functions | Functions | + +GitHub Pages is the only credible alternative (free, simple) but lacks cache control, redirects, and serverless compute for the visual editor. + +## Architecture + +Unchanged from the original spec. Single CF Pages project, subdirectory-per-version: + +``` +Push to branch X + └─ GHA: checkout → Docker build → optimize images → Pagefind → stage → CF Pages deploy + └─ docs.vyos.io/en/{version}/ (HTML + downloads/) +``` + +## URL Structure + +``` +docs.vyos.io/en/1.5/ ← current (circinus, VyOS 1.5.x) +docs.vyos.io/en/1.4/ ← sagitta (VyOS 1.4.x) +docs.vyos.io/en/1.3/ ← equuleus (VyOS 1.3.x) +docs.vyos.io/en/1.2/ ← crux (VyOS 1.2.x) + +docs.vyos.io/en/1.5/downloads/VyOS-1.5.pdf +docs.vyos.io/en/1.5/downloads/VyOS-1.5.epub +``` + +### Redirects (`_redirects` at repo root) + +``` +/ /en/1.5/ 302 +/en/latest/* /en/1.5/:splat 302 +/en/stable/* /en/1.4/:splat 302 +/en/current/* /en/1.5/:splat 302 +/en/sagitta/* /en/1.4/:splat 301 +/en/equuleus/* /en/1.3/:splat 301 +/en/crux/* /en/1.2/:splat 301 +``` + +302 for aliases that change with releases; 301 for fixed historical branch names. + +## Cache Headers (`_headers` at repo root) + +``` +# Versioned static assets — immutable, cache forever +/en/*/_static/* + Cache-Control: public, max-age=31536000, immutable + +# Pagefind search index — changes on rebuild +/en/*/pagefind/* + Cache-Control: public, max-age=86400 + +# HTML pages — short browser cache, longer edge cache +/en/*/*.html + Cache-Control: public, max-age=3600, s-maxage=86400 + +# PDF/EPUB downloads — stable per version +/en/*/downloads/* + Cache-Control: public, max-age=604800 + +# LLM files — change on rebuild +/llms.txt + Cache-Control: public, max-age=86400 +/llms-full.txt + Cache-Control: public, max-age=86400 + +# Sitemap — change on rebuild +/en/*/sitemap.xml + Cache-Control: public, max-age=86400 +``` + +This directly addresses the RTD performance limitation (20-minute cache TTL on all assets). Static assets get 1-year immutable cache. HTML gets 1-hour browser / 24-hour edge. Downloads get 7-day cache. + +## GitHub Actions Workflow + +**New file:** `.github/workflows/deploy-docs.yml` + +```yaml +on: + push: + branches: [current, sagitta, equuleus, crux] + workflow_dispatch: + +concurrency: + group: deploy-docs + cancel-in-progress: false + +jobs: + build-and-deploy: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + with: + submodules: recursive + + - name: Set version + run: | + case "${{ github.ref_name }}" in + current) echo "VERSION=1.5" >> $GITHUB_ENV ;; + sagitta) echo "VERSION=1.4" >> $GITHUB_ENV ;; + equuleus) echo "VERSION=1.3" >> $GITHUB_ENV ;; + crux) echo "VERSION=1.2" >> $GITHUB_ENV ;; + *) echo "ERROR: unknown branch ${{ github.ref_name }}" && exit 1 ;; + esac + + - name: Restore other versions from cache + uses: actions/cache/restore@v4 + with: + path: dist + key: vyos-docs-${{ github.run_id }} + restore-keys: vyos-docs- + + - name: Authenticate to Docker Hub + uses: docker/login-action@v3 + with: + username: ${{ secrets.DOCKER_USER }} + password: ${{ secrets.DOCKER_TOKEN }} + + - name: Build HTML + run: | + docker run --rm -v "$PWD":/vyos -w /vyos/docs \ + vyos/vyos-documentation make html + + - name: Optimize images + run: | + docker run --rm -v "$PWD":/vyos -w /vyos/docs \ + vyos/vyos-documentation make optimize-images + + - name: Build PDF + run: | + docker run --rm -v "$PWD":/vyos -w /vyos/docs \ + vyos/vyos-documentation make latexpdf + + - name: Build EPUB + run: | + docker run --rm -v "$PWD":/vyos -w /vyos/docs \ + vyos/vyos-documentation make epub + + - name: Stage output + run: | + mkdir -p dist/en/${{ env.VERSION }}/downloads + cp -r docs/_build/html/. dist/en/${{ env.VERSION }}/ + cp docs/_build/latex/VyOS.pdf \ + dist/en/${{ env.VERSION }}/downloads/VyOS-${{ env.VERSION }}.pdf + cp docs/_build/epub/VyOS.epub \ + dist/en/${{ env.VERSION }}/downloads/VyOS-${{ env.VERSION }}.epub + if [ "${{ env.VERSION }}" = "1.5" ]; then + cp _redirects dist/_redirects + cp _headers dist/_headers + cp docs/_build/html/404.html dist/404.html + fi + + - name: Build Pagefind index + run: | + npx pagefind \ + --site dist/en/${{ env.VERSION }} \ + --output-path dist/en/${{ env.VERSION }}/pagefind + + - name: Deploy to Cloudflare Pages + uses: cloudflare/wrangler-action@v3 + with: + apiToken: ${{ secrets.CF_API_TOKEN }} + accountId: ${{ secrets.CF_ACCOUNT_ID }} + command: pages deploy dist --project-name=vyos-docs + + - name: Save merged dist to cache + uses: actions/cache/save@v4 + with: + path: dist + key: vyos-docs-${{ github.run_id }} + + - name: Verify all versions live + run: | + for VERSION in 1.5 1.4 1.3 1.2; do + STATUS=$(curl -s -o /dev/null -w "%{http_code}" \ + https://vyos-docs.pages.dev/en/$VERSION/) + if [ "$STATUS" != "200" ]; then + echo "ERROR: /en/$VERSION/ returned $STATUS" + exit 1 + fi + done +``` + +### Differences from original spec workflow: +- **Added `make optimize-images`** step between HTML build and PDF build (mobile performance spec) +- **Added `_headers` copy** to stage step alongside `_redirects` + +### Cache recovery workflow (`.github/workflows/rebuild-all.yml`) + +Unchanged from original spec. Runs weekly (Monday 04:00 UTC) and on `workflow_dispatch`. Triggers all 4 branches in sequence. + +### New secrets required + +| Secret | Scope | +|---|---| +| `CF_API_TOKEN` | Cloudflare API token, Pages:Edit permission | +| `CF_ACCOUNT_ID` | Cloudflare account ID | +| `DOCKER_USER` | Docker Hub username | +| `DOCKER_TOKEN` | Docker Hub access token (read-only) | + +## Search + +**Pagefind** — fully static, zero cost, per-version index. JS snippet in `docs/_templates/layout.html` replaces the RTD search widget. Future option: Algolia DocSearch (free for OSS). + +## Version Switcher + +JS snippet in template. Reads active version from URL path, switches to same page in target version with HEAD-request fallback to version root. + +## 404 Handling + +Root-level `dist/404.html` copied from the `current` branch's `sphinx-notfound-page` output. + +## Cross-Spec Interactions + +### LLM Adaptation Spec +- `html_baseurl` must be updated to `https://docs.vyos.io/en/1.5/` at DNS cutover +- Curated `llms.txt` URLs must be updated from `/en/latest/` to `/en/1.5/` +- `llms.txt`, `llms-full.txt`, `robots.txt`, `sitemap.xml` are served automatically from HTML build output +- `sphinx-sitemap` generates per-version sitemap at `/en/{version}/sitemap.xml` +- `robots.txt` Sitemap directive should reference `https://docs.vyos.io/en/1.5/sitemap.xml` + +### MyST Migration Spec +- No hosting impact — HTML output format is identical regardless of source format +- MyST migration should merge before DNS cutover to avoid two migrations on the live site + +### Visual Editor Spec +- Editor frontend (static files) can deploy as a separate CF Pages project at `edit.docs.vyos.io` +- Staging API has two options: + - **Cloudflare Workers + D1** — serverless, same platform, SQLite-compatible database. Consolidates infrastructure. + - **Separate VPS** — as currently specified in the visual editor spec +- Decision deferred to visual editor implementation. This spec notes both options. + +### Mobile Performance Spec +- `_headers` file enables cache header control (the primary RTD limitation) +- Image optimization (`make optimize-images`) integrated into GHA workflow +- CSS consolidation and JS deferring work independently of hosting platform but should be tested post-migration + +## Migration Plan (zero-downtime) + +### Step 0 — Bootstrap +Trigger `deploy-docs.yml` manually for each branch in order: `current` → `sagitta` → `equuleus` → `crux`. Seeds the cache chain. + +### Step 1 — Parallel build +Deploy to staging domain (`vyos-docs.pages.dev`). RTD remains live. + +### Step 2 — Validate on staging +- All 4 versioned paths render correctly +- Version switcher works +- Search returns scoped results per version +- PDF and EPUB downloads work +- `_redirects` route legacy URLs correctly +- `_headers` cache rules apply correctly +- 404 pages display correctly +- `llms.txt` and `llms-full.txt` accessible at root +- `sitemap.xml` generated per version + +### Step 3 — DNS cutover +- Point `docs.vyos.io` CNAME to CF Pages +- Update `html_baseurl` to `https://docs.vyos.io/en/1.5/` +- Update curated `llms.txt` URLs from `/en/latest/` to `/en/1.5/` +- Update `robots.txt` Sitemap directive to `https://docs.vyos.io/en/1.5/sitemap.xml` +- Submit sitemap to search engines + +### Step 4 — Decommission RTD +Remove `.readthedocs.yml`, disable RTD project. Retain RTD account 30 days as rollback. + +## Execution Order (All Five Specs) + +| Phase | Specs | Can start | +|---|---|---| +| Phase 1 (parallel) | LLM Adaptation + MyST Migration | Immediately | +| Phase 2 | Cloudflare Hosting Migration | After MyST Migration merges | +| Phase 3 (parallel) | Mobile Performance + Visual Editor | After Cloudflare Migration completes | + +## Risks + +- **Cache chain eviction** — GitHub caches expire after 7 days of inactivity. Mitigation: weekly `rebuild-all.yml` workflow. +- **Docker Hub rate limits** — authenticated pulls required. Mitigation: `DOCKER_USER`/`DOCKER_TOKEN` secrets. +- **DNS propagation** — some users may see RTD during propagation. Mitigation: both platforms serve valid content; RTD stays live until propagation completes. +- **File limit** — CF Pages free tier allows 20,000 files per deployment. 4 versions × ~2,000 files = ~8,000 files. Well within limits. + +## Maintenance + +- **GHA workflow**: Update when new VyOS release branches are created (add to branch list, version mapping, and VERSIONS array). +- **`_redirects`**: Update when `/en/latest/` should point to a new version. +- **`_headers`**: Rarely changes. +- **Secrets**: Rotate CF API token and Docker Hub token per their policies. diff --git a/docs/superpowers/specs/2026-03-29-llm-documentation-adaptation-design.md b/docs/superpowers/specs/2026-03-29-llm-documentation-adaptation-design.md new file mode 100644 index 00000000..0c02bf37 --- /dev/null +++ b/docs/superpowers/specs/2026-03-29-llm-documentation-adaptation-design.md @@ -0,0 +1,207 @@ +# LLM Documentation Adaptation Design + +**Date:** 2026-03-29 +**Status:** Approved +**Approach:** Hybrid — automated Sphinx extension + curated `llms.txt` +**Depends on:** None (independent of other specs; `conf.py` changes are non-conflicting) +**Related:** RST → MyST Migration spec, Cloudflare Hosting spec +**Execution:** Parallel with MyST Migration (trivial `conf.py` merge conflict in `extensions` list expected) + +## Goal + +Make VyOS documentation consumable by LLMs for two use cases: + +1. **Structured reference data** — enable LLMs (ChatGPT, Claude, etc.) to answer VyOS configuration questions accurately +2. **AI search discoverability** — make docs indexable by AI search products (Perplexity, Bing Chat, Google AI Overviews) + +Target audience: both end users (configuration/operations) and developers/contributors equally. + +## Design + +### 1. Sphinx Extension: `sphinx-llms-txt` + +Add [`sphinx-llms-txt`](https://pypi.org/project/sphinx-llms-txt/) (v0.7.1+) to the Sphinx build pipeline. + +**What it generates:** + +- `llms-full.txt` — entire documentation concatenated into a single markdown file for bulk LLM ingestion + +**Why this extension over `sphinx-llm` (NVIDIA):** + +- More mature (v0.7.1 vs v0.3.0) +- Lighter weight — post-processes during normal build rather than running a parallel markdown build +- Supports size limit configuration (useful for 258-file doc set) +- Available on conda-forge + +### 2. Curated `llms.txt` + +Hand-maintained file at `docs/_html_extra/llms.txt`, served at site root via the existing `html_extra_path` mechanism. + +**Content:** + +```markdown +# VyOS + +> VyOS is a free, open-source network operating system based on Debian GNU/Linux. +> It provides routing, firewall, and VPN functionality via a unified CLI using +> `set`/`delete`/`show` command hierarchy. Current rolling release is 1.5.x (circinus). + +VyOS configuration follows a tree structure. All configuration commands start with +`set` (to add/change) or `delete` (to remove). Changes are staged and applied with +`commit`. The CLI hierarchy maps directly to the documentation structure. + +## Quick Start + +- [Quick Start Guide](https://docs.vyos.io/en/latest/quick-start.html): Minimal setup walkthrough +- [CLI Overview](https://docs.vyos.io/en/latest/cli.html): Command-line interface usage + +## Configuration + +- [Firewall](https://docs.vyos.io/en/latest/configuration/firewall/index.html): Zone-based firewall, rules, groups +- [Interfaces](https://docs.vyos.io/en/latest/configuration/interfaces/index.html): Ethernet, bonding, bridge, VLAN, tunnel, wireless +- [Protocols](https://docs.vyos.io/en/latest/configuration/protocols/index.html): BGP, OSPF, IS-IS, static routing, MPLS +- [VPN](https://docs.vyos.io/en/latest/configuration/vpn/index.html): IPsec, OpenVPN, WireGuard, L2TP, PPTP +- [NAT](https://docs.vyos.io/en/latest/configuration/nat/index.html): Source NAT, destination NAT, NAT66 +- [System](https://docs.vyos.io/en/latest/configuration/system/index.html): DNS, NTP, syslog, users, task scheduler +- [High Availability](https://docs.vyos.io/en/latest/configuration/highavailability/index.html): VRRP +- [Load Balancing](https://docs.vyos.io/en/latest/configuration/loadbalancing/index.html): WAN and reverse proxy +- [Containers](https://docs.vyos.io/en/latest/configuration/container/index.html): Podman-based container support +- [PKI](https://docs.vyos.io/en/latest/configuration/pki/index.html): Certificate management +- [Policy](https://docs.vyos.io/en/latest/configuration/policy/index.html): Route maps, prefix lists, access lists +- [Traffic Policy](https://docs.vyos.io/en/latest/configuration/trafficpolicy/index.html): QoS and shaping +- [Service](https://docs.vyos.io/en/latest/configuration/service/index.html): DHCP, DNS forwarding, SNMP, SSH, HTTPS API +- [VRF](https://docs.vyos.io/en/latest/configuration/vrf/index.html): Virtual routing and forwarding + +## Operations + +- [Operational Commands](https://docs.vyos.io/en/latest/operation/index.html): Show, monitor, restart commands + +## Installation + +- [Installation Guide](https://docs.vyos.io/en/latest/installation/index.html): Bare metal, virtual, cloud deployments + +## Automation + +- [Automation](https://docs.vyos.io/en/latest/automation/index.html): Ansible, Terraform, HTTP API, NETCONF + +## Configuration Examples + +- [Blueprints](https://docs.vyos.io/en/latest/configexamples/index.html): Real-world topology examples + +## Optional + +- [Contributing](https://docs.vyos.io/en/latest/contributing/index.html): Development workflow +- [VPP](https://docs.vyos.io/en/latest/vpp/index.html): Vector Packet Processing integration +``` + +**Design decisions:** + +- Preamble explains `set`/`delete`/`commit` model — the most critical context for LLMs answering VyOS questions +- Sections mirror the documentation structure +- `## Optional` separates contributor/VPP content that most end-user queries won't need +- URLs use `/en/latest/` (ReadTheDocs canonical path for current branch). When the Cloudflare Pages migration happens, update URLs to `/en/1.5/` per that spec's URL structure. + +### 3. robots.txt Update + +Replace `docs/_html_extra/robots.txt` with explicit AI bot allowances: + +``` +User-agent: * +Allow: / + +# AI Training Crawlers +User-agent: GPTBot +Allow: / + +User-agent: ClaudeBot +Allow: / + +User-agent: Google-Extended +Allow: / + +User-agent: CCBot +Allow: / + +User-agent: PerplexityBot +Allow: / + +# AI Search/Retrieval +User-agent: ChatGPT-User +Allow: / + +User-agent: Claude-SearchBot +Allow: / + +User-agent: Claude-User +Allow: / + +User-agent: OAI-SearchBot +Allow: / + +User-agent: Perplexity-User +Allow: / + +Sitemap: https://docs.vyos.io/sitemap.xml +``` + +**Design decisions:** + +- Explicitly allow all AI bots (training and retrieval) — VyOS is open-source; broad LLM awareness benefits the project +- Named entries make intent explicit and allow selective blocking later if needed +- Removed `atlassian-bot` special case — covered by wildcard + +### 4. Sitemap Generation + +Add `sphinx-sitemap` to generate the `sitemap.xml` already referenced by `robots.txt`. This spec owns the `sphinx-sitemap` introduction; the Cloudflare hosting spec references it during DNS cutover but does not add it. + +- Requires `html_baseurl = 'https://docs.vyos.io/en/latest/'` in `conf.py` (update to `/en/1.5/` when Cloudflare migration happens) +- ReadTheDocs serves the generated sitemap automatically + +### 5. Build Pipeline Changes + +**`requirements.txt`** — add (pin to current stable versions at implementation time): + +``` +sphinx-llms-txt>=0.7.1 +sphinx-sitemap>=2.6.0 +``` + +**`conf.py`** — add to extensions: + +```python +'sphinx_llms_txt', +'sphinx_sitemap', +``` + +Add configuration: + +```python +html_baseurl = 'https://docs.vyos.io/en/latest/' +``` + +**No changes to:** + +- Custom extensions (`vyos.py`, `testcoverage.py`, `autosectionlabel.py`, `releasenotes.py`) +- RST source files +- Makefile (extensions hook into standard `make html`) + +**Docker image:** The `vyos/vyos-documentation` Docker image installs dependencies from `requirements.txt`. Adding `sphinx-llms-txt` and `sphinx-sitemap` to `requirements.txt` is sufficient — no Docker image rebuild needed unless dependencies are baked in at image build time. Verify at implementation time. + +**`conf.py` coordination:** The changes in this spec (adding extensions, `html_baseurl`) are to different config keys than the MyST migration spec's changes (`myst_enable_extensions`, `source_suffix`, `html_context`). They are non-conflicting and can be applied in any order. + +## Maintenance + +- **`llms.txt`**: Update when major documentation sections are added or reorganized. Expected frequency: a few times per year. +- **`llms-full.txt`**: Fully automated, regenerated on every build. +- **`robots.txt`**: Update when new AI crawlers emerge or policy changes. +- **`sitemap.xml`**: Fully automated. + +## Hosting + +Designed for the current ReadTheDocs setup. ReadTheDocs has built-in support for serving `llms.txt` and `llms-full.txt`. When the Cloudflare Pages migration happens (per the existing hosting spec), the static files will work as-is — they're just files in the build output. However, `html_baseurl` and the curated `llms.txt` URLs must be updated from `/en/latest/` to `/en/1.5/` to match the Cloudflare URL structure and avoid redirect chains. + +Note: After the RST → MyST migration, `llms-full.txt` content may change slightly since `sphinx-llms-txt` post-processes built HTML output. Verify output quality after migration. + +## Industry Reference + +This approach matches what Stripe, Vercel, Cloudflare, and Next.js use: a curated `llms.txt` entry point with automated full-content generation. Stripe notably includes LLM-specific instructions in their `llms.txt` (e.g., "check npm registry for latest versions rather than relying on memorized version numbers"), which informed our VyOS preamble explaining the `set`/`delete`/`commit` model. diff --git a/docs/superpowers/specs/2026-03-29-mobile-performance-design.md b/docs/superpowers/specs/2026-03-29-mobile-performance-design.md new file mode 100644 index 00000000..ac7c3012 --- /dev/null +++ b/docs/superpowers/specs/2026-03-29-mobile-performance-design.md @@ -0,0 +1,142 @@ +# Mobile Performance & Core Web Vitals Optimization + +**Date:** 2026-03-29 +**Status:** Approved +**Depends on:** Cloudflare Hosting Migration (cache headers require CF Pages; image optimization integrated into GHA workflow) +**Related:** LLM Adaptation spec (independent), Visual Editor spec (independent), RST → MyST Migration (safer to optimize post-migration codebase) +**Execution:** Phase 3 — after Cloudflare Migration completes. Can run in parallel with Visual Editor. + +## Goal + +Optimize the VyOS documentation site for mobile performance and Core Web Vitals (LCP, INP, CLS) to improve: + +1. Mobile user experience (reported complaints about slow loading) +2. Google search rankings via Core Web Vitals signals +3. Overall page load performance for all users + +## Current State (Audit Findings) + +The live site at docs.vyos.io has significant performance issues: + +| Issue | Impact | Size | +|---|---|---| +| Header iframe loading a full Next.js app | ~25 extra requests, layout shift | ~120 KB HTML + assets | +| `datatables.js` loaded synchronously on every page | Blocks rendering, most pages don't use it | 454 KB | +| 20 render-blocking CSS files (15 custom) | Waterfall of blocking requests | ~192 KB total | +| 13 synchronous JS files (none deferred) | Blocks interactivity | ~560 KB | +| 37 MB of unoptimized images (190 files) | No WebP, no lazy loading, no dimensions | Up to 2.6 MB per image | +| Google Fonts Archivo as TTF (not WOFF2) | Render-blocking external fetch | 5 font weights | +| FontAwesome 4.7 without `font-display: swap` | Flash of invisible text | 77 KB WOFF2 | +| 20-minute cache TTL on all assets | RTD platform limitation (not controllable) | — | + +## Approach + +Template overrides for HTML changes (script defer, preload hints, iframe optimization). External tooling (Pillow) as a post-build step for image optimization. CSS consolidation done manually once. + +## Design + +### 1. JavaScript Optimization + +**Defer all custom scripts** via template override in `docs/_templates/layout.html`. jQuery remains synchronous (sphinx_rtd_theme hard dependency). Everything else gets `defer`. + +| Script | Loading | Rationale | +|---|---|---| +| `jquery.js` | Synchronous | Theme hard dependency | +| `datatables.js` | `defer` | Only needs DOM-ready, not render-blocking | +| All other custom JS (`doctools.js`, `sphinx_highlight.js`, `design-tabs.js`, `theme.js`, `tables.js`, `codecopier.js`, `sidebar.js`, `footer.js`, etc.) | `defer` | None require synchronous execution | +| `readthedocs-addons.js` | Already `async` | No change needed | + +### 2. CSS Consolidation + +**Merge all 15 custom CSS files into a single `vyos.css`.** One-time manual task — concatenate in dependency order, verify no selector conflicts. + +Files to merge: +- `custom.css`, `lists.css`, `hints.css`, `headers.css`, `breadcrumbs.css`, `linkButtons.css`, `text.css`, `leftSidebar.css`, `scrolls.css`, `tables.css`, `running-on-bare-metal.css`, `code-snippets.css`, `separate-commands.css`, `configuration/index.css`, `datatables.css` + +Update `conf.py` — replace 15 entries in `html_css_files` with the single `vyos.css`. + +Theme-managed CSS files stay separate (`theme.css`, `pygments.css`, `design-style.*.min.css`, `graphviz.css`). + +**Result:** 20 CSS files → 6 CSS files. Eliminates 14 render-blocking requests. + +### 3. Font Optimization + +**Self-host Archivo as WOFF2:** +- Download WOFF2 files for 5 weights (400, 500, 600, 700, 800) +- Place in `docs/_static/fonts/` +- Add `@font-face` declarations to `vyos.css` with `font-display: swap` +- Remove the Google Fonts `` from the template + +Benefits: eliminates external DNS lookup + TLS handshake to `fonts.googleapis.com` and `fonts.gstatic.com`, serves WOFF2 instead of TTF (~30-50% smaller), removes render-blocking external request. + +**Add `font-display: swap` to FontAwesome** via CSS override: +```css +@font-face { + font-family: 'FontAwesome'; + font-display: swap; + /* re-declare src from theme.css */ +} +``` + +Dead Lato/Roboto Slab references in theme CSS are harmless (not loaded externally) — no action needed. + +### 4. Header Iframe Optimization + +The header iframe (`