docs: Cloudflare Workers hosting pipeline (apex, content workers, CI, previews) (#2140)

* docs-infra: scaffold Cloudflare workers workspace (versions.json v2, matrix, toolchain)

🤖 Generated by [robots](https://vyos.io)

* docs-infra: record full Phase-0 plan decision in workers/PLAN.md

🤖 Generated by [robots](https://vyos.io)

* docs-infra: shared content worker — asset serving, cache classes, X-Docs-Build, canary no-store

🤖 Generated by [robots](https://vyos.io)

* docs-infra: run worker script before assets; test fetch entrypoint

🤖 Generated by [robots](https://vyos.io)

* docs-infra: apex manifest loader + dispatch map + runtime binding guard (TDD)

🤖 Generated by [robots](https://vyos.io)

* docs-infra: apex redirects (aliases, PDF, trailing-slash) + special paths (TDD)

🤖 Generated by [robots](https://vyos.io)

* docs-infra: PDF redirect honors pdf:null and preserves query

🤖 Generated by [robots](https://vyos.io)

* docs-infra: apex UA gate — allowlist-wins, log-only AI crawlers, empty block list at launch

🤖 Generated by [robots](https://vyos.io)

* docs-infra: apex router (pipeline §3.2), themed 404/503, /kb seam, env configs + congruence test

🤖 Generated by [robots](https://vyos.io)

* docs-infra: add missing-User-Agent regression test for apex UA gate

🤖 Generated by [robots](https://vyos.io)

* docs-infra: R2-streaming preview worker — MIME map, noindex, no-store (TDD)

🤖 Generated by [robots](https://vyos.io)

* docs-infra: preview 404 no-store + fetch handler tests

🤖 Generated by [robots](https://vyos.io)

* docs-infra: bootstrap script — binding-target workers must exist before apex deploys

🤖 Generated by [robots](https://vyos.io)

* docs-infra: apex run_worker_first, lockfile for npm ci, PDF Location from manifest

🤖 Generated by [robots](https://vyos.io)

* docs-infra: derive html_baseurl from DOCS_VERSION_SLUG with RTD fallback (canonical gate prereq)

🤖 Generated by [robots](https://vyos.io)

* docs-infra: version picker + status banner + language scaffold (vanilla JS, TDD pure core)

🤖 Generated by [robots](https://vyos.io)

* docs-infra: picker preserves query+hash across switch; valid breadcrumb markup

🤖 Generated by [robots](https://vyos.io)

* docs-infra: Pagefind search wrapper with runtime base-path + preview prefix handling (TDD)

🤖 Generated by [robots](https://vyos.io)

* docs-infra: pagefind wrapper — asset-failure notice + UI stylesheet load

🤖 Generated by [robots](https://vyos.io)

* docs-infra: gate Pagefind searchbox to CF builds (RTD keeps stock search until cutover)

🤖 Generated by [robots](https://vyos.io)

* docs-infra: deploy sanity gates — limits, critical pages, count-delta, canonical (TDD)

🤖 Generated by [robots](https://vyos.io)

* docs-infra: hermetic gate tests via fixture versions.json

🤖 Generated by [robots](https://vyos.io)

* docs-infra: docs-build workflow — candidate/smoke/promote two-stage deploy + registry + rollback

Two-stage CF Workers pipeline: build in pinned container, assemble artifact,
sanity gates, deploy candidate, scoped pre-traffic smoke via canary apex,
promote (rollback-id capture, hostname purge, registry upload), post-promote
probe + auto-rollback. DOCS_CF_LIVE repo variable gates every docs.vyos.io
production interaction pre-cutover.

scripts/docs_gates/smoke.py adds one authorized check beyond the spec: the
version's index.html probe asserts the #vyos-search mount div is present in
the response body, guarding CI silently forgetting DOCS_VERSION_SLUG (which
would otherwise ship stock RTD search without the Pagefind gate noticing).

🤖 Generated by [robots](https://vyos.io)

* docs-infra: build docs image in-workflow with buildx cache (v4.1 — digest pin dropped)

Plan v4.1 amendment: the ghcr.io digest-pinned image does not exist (workflow
would hard-fail at the first docker step on every push). Replace the BUILD_IMAGE
env placeholder with an in-workflow docker build from docker/Dockerfile via
docker/setup-buildx-action@v3 + docker/build-push-action@v6 (context: docker/,
load: true, tags: docs-build:local, GHA cache from/to). The checked-out commit
is the pin; buildx GHA cache keeps repeat builds cheap. Sphinx-build step swaps
to docs-build:local; inner script unchanged.

🤖 Generated by [robots](https://vyos.io)

* docs-infra: apex/preview deploy workflow — canary auto, production behind environment approval

* docs-infra: apex-deploy concurrency guard (per-ref, cancel-in-progress)

🤖 Generated by [robots](https://vyos.io)

* docs-infra: fork-safe PR preview pipeline — approval record, R2 prefixes, label consumption, cleanup

* docs-infra: nightly preview sweep — pipefail + per-prefix failure isolation

🤖 Generated by [robots](https://vyos.io)

* docs-infra: nightly canary QA — per-entry sweep + URL-parity corpus vs RTD

🤖 Generated by [robots](https://vyos.io)

* docs-infra: parity sweep scoped to CF-built versions; transport-error resilience

🤖 Generated by [robots](https://vyos.io)

* docs-infra: one-off bootstrap workflow (binding targets — runs once on this push)

🤖 Generated by [robots](https://vyos.io)

* docs-infra: remove one-off bootstrap workflow (bootstrap complete)

🤖 Generated by [robots](https://vyos.io)

* docs-infra: one-off canary apex + preview deploy (route targets for Task 3.6 step 2c)

🤖 Generated by [robots](https://vyos.io)

* docs-infra: remove one-off canary deploy workflow (targets live)

🤖 Generated by [robots](https://vyos.io)

* docs-infra: address Phase-0 CodeRabbit findings (canonical gate, error caching, registry pointer, validation)

🤖 Generated by [robots](https://vyos.io)

* docs-infra: strengthen manifest tests (full dispatch iteration, mutation-free validate)

🤖 Generated by [robots](https://vyos.io)

* docs-infra: address GitHub CodeRabbit review (pointer-after-probe, fail-closed sweeps, block-precedence UA gate, preview hardening)

🤖 Generated by [robots](https://vyos.io)

* docs-infra: adversarial review fixes — error no-store, probe retry, PR-list membership, preview dotted-segment

🤖 Generated by [robots](https://vyos.io)

* docs-infra: serve oversized legacy PDF from R2 via apex (spec §5 fallback)

The 1.3 PDF (29.2 MiB) exceeds the 25 MiB static-asset cap and is absent
from the legacy content Worker's build, so /_/downloads/en/1.3/pdf/ (and
the picker's PDF link) 301'd into a dead-end 404 post-cutover. Add the R2
object fallback spec §5 already documented but never implemented: a
DOCS_PDFS R2 bucket binding on the apex Worker, a manifest pdf_r2_key
field (1.3 only), and a router step ahead of version dispatch that streams
the object with its own cache class (canary/error still force no-store).

🤖 Generated by [robots](https://vyos.io)

* docs-infra: PDF R2 fallback honors Range + If-None-Match, preserves ETag

🤖 Generated by [robots](https://vyos.io)

1 files changed, 61 insertions, 0 deletions

diff --git a/workers/preview/src/index.ts b/workers/preview/src/index.ts
new file mode 100644
index 00000000..5c945175
--- /dev/null
+++ b/workers/preview/src/index.ts
@@ -0,0 +1,61 @@
+export interface Env { PREVIEWS: R2Bucket }
+
+const MIME: Record<string, string> = {
+  html: "text/html; charset=utf-8", css: "text/css", js: "text/javascript",
+  json: "application/json", svg: "image/svg+xml", png: "image/png", jpg: "image/jpeg",
+  gif: "image/gif", ico: "image/x-icon", txt: "text/plain; charset=utf-8",
+  xml: "application/xml", pdf: "application/pdf", woff2: "font/woff2", woff: "font/woff",
+};
+
+export function mimeFor(key: string): string {
+  const ext = key.split(".").pop() ?? "";
+  return MIME[ext] ?? "application/octet-stream";
+}
+
+export function keyFor(pathname: string): string {
+  let key = pathname.replace(/^\//, "");
+  if (key.endsWith("/") || key === "" ) key += "index.html";
+  return key;
+}
+
+export default {
+  async fetch(request: Request, env: Env): Promise<Response> {
+    const key = keyFor(new URL(request.url).pathname);
+    let obj: R2ObjectBody | null = null;
+    let resolvedKey = key;
+    try {
+      obj = await env.PREVIEWS.get(key);
+      if (!obj && !key.split("/").pop()?.includes(".")) {
+        // Extensionless directory URL with no trailing slash (e.g. /en/rolling/cli) —
+        // keyFor() only appends index.html for trailing-slash/empty paths, so probe the
+        // directory's index.html before 404ing. Check the LAST path segment only — a dot
+        // anywhere earlier (e.g. version segment "1.4" in /pr-42/en/1.4/cli) must not skip
+        // the probe for an otherwise-extensionless final segment.
+        resolvedKey = `${key}/index.html`;
+        obj = await env.PREVIEWS.get(resolvedKey);
+      }
+    } catch {
+      // Transient R2/binding error on either probe — fail closed with a controlled 503
+      // instead of letting an unhandled exception surface as a raw worker error.
+      return new Response("preview temporarily unavailable", {
+        status: 503,
+        headers: { "X-Robots-Tag": "noindex", "Cache-Control": "no-store" },
+      });
+    }
+    if (!obj) {
+      return new Response("preview not found", {
+        status: 404,
+        // no-store on the 404 too — a cached 404 would persist past the preview upload
+        headers: { "X-Robots-Tag": "noindex", "Cache-Control": "no-store" },
+      });
+    }
+    return new Response(obj.body, {
+      headers: {
+        "content-type": obj.httpMetadata?.contentType ?? mimeFor(resolvedKey),
+        "X-Robots-Tag": "noindex",
+        "Cache-Control": "no-store",
+        "X-Content-Type-Options": "nosniff",
+      },
+    });
+  },
+} satisfies ExportedHandler<Env>;