Merge pull request #1 from vyos/claude/legacy-snapshot-phase4HEADproduction

docs: legacy 1.2/1.3 static snapshot + Worker deploy pipeline

1 files changed, 82 insertions, 1 deletions

diff --git a/README.md b/README.md
index 786cb53..d27f984 100644
--- a/README.md
+++ b/README.md
@@ -1 +1,82 @@
-# vyos-docs-legacy-snapshot
\ No newline at end of file
+# vyos-docs-legacy-snapshot
+
+Static snapshot of the legacy VyOS documentation versions **1.2** and **1.3**,
+taken from the live ReadTheDocs-hosted `docs.vyos.io/en/1.2/` and
+`docs.vyos.io/en/1.3/` trees before docs.vyos.io cuts over to Cloudflare
+Workers (per the `2026-07-02-cloudflare-migration-fresh-spec` design, §6 and
+§15a).
+
+## Why this repo exists (one-time nature)
+
+RTD's build pipeline for 1.2 and 1.3 will be retired as part of the docs
+platform migration. Those two versions are LTS/legacy and no longer receive
+doc changes, so instead of standing up a live Sphinx build for them on the
+new platform, we take **one static crawl** of the RTD-rendered output, scrub
+out RTD-specific chrome (platform JS, analytics, ad units, the RTD version
+flyout), inject our own version-picker bundle so the pages stay navigable
+into the new picker UI, and commit the result as a static asset tree. That
+tree is served by a small dedicated Worker (`vyos-docs-legacy`, deployed by
+Task 4.2) — no crawler, no Sphinx, no CI rebuild. This repo's `snapshot/`
+directory is the source of truth going forward; there is no "re-crawl on a
+schedule" step.
+
+**This is a one-time operation.** `snapshot/` is not meant to be regenerated
+routinely — 1.2 and 1.3 are frozen. Re-run `tools/snapshot.sh` only if:
+
+- The initial snapshot had a defect (missed pages, bad scrub, broken links)
+  that must be corrected from the still-live RTD source, or
+- RTD's hosting of 1.2/1.3 changes in a way that requires re-baselining
+  before RTD is decommissioned.
+
+Once RTD stops serving `docs.vyos.io/en/1.2/` and `/en/1.3/`, `tools/snapshot.sh`
+can no longer be re-run at all — the source it crawls will be gone. At that
+point this repo's `tools/` become historical record only.
+
+## Layout
+
+```text
+tools/
+  snapshot.sh       # crawl -> scrub -> picker/PDF -> parity -> sitemap -> pagefind
+  scrub.py          # HTML scrubbing (RTD chrome removal + URL localization + picker injection)
+  test_scrub.py     # TDD unit tests for scrub.py
+snapshot/           # committed output: en/1.2/, en/1.3/ (scrubbed HTML, static assets,
+                     # per-version pagefind/, per-version sitemap.xml)
+r2-staging/         # 1.3's PDF (29.2 MiB, over the 25 MiB Workers per-asset cap) —
+                     # uploaded to R2 separately by the Task 4.2 deploy flow, NOT
+                     # committed as a Worker static asset. Not committed to this repo
+                     # either (large binary; staged locally for the R2 upload step).
+crawl/               # scratch — raw wget mirror output, wiped and regenerated by every
+                     # snapshot.sh run, NEVER committed
+snapshot.new/        # scratch — build-then-swap staging dir; moved onto snapshot/ only
+                     # on full success, NEVER committed
+```
+
+## Re-running the snapshot
+
+Prerequisites: `wget`, `curl`, `python3.11+`, Node (for `npx pagefind`), and a
+local checkout of `vyos-documentation` (for the version-picker bundle files
+at `docs/_static/{js,css}/version-picker.*`).
+
+```bash
+DOCS_REPO=/path/to/vyos-documentation bash tools/snapshot.sh
+```
+
+`DOCS_REPO` defaults to `$HOME/GitHub/vyos-documentation`. The script is
+idempotent: it wipes `crawl/` and `snapshot.new/` before every run, crawls
+`/en/1.2/` and `/en/1.3/` from the **live** `docs.vyos.io`, scrubs every
+mirrored HTML file via `tools/scrub.py`, copies in the picker bundle,
+downloads 1.3's PDF into `r2-staging/` (validating its `%PDF-` header),
+hard-fails on any dangling internal link (parity check — the crawl graph is
+the source of truth since 1.2/1.3 never had a `sitemap.xml`), generates a
+`sitemap.xml` per version from the crawl inventory, builds a **per-version**
+Pagefind index (no cross-version search contamination), and — only once both
+`1.2` and `1.3` complete every step successfully — atomically swaps
+`snapshot.new/` onto `snapshot/`. A failed run therefore never leaves the
+committed `snapshot/` tree partially updated.
+
+Run the unit tests with:
+
+```bash
+python3.11 -m venv .venv && . .venv/bin/activate && pip install pytest
+python -m pytest tools/test_scrub.py -v
+```