ci: port the Cloudflare Workers docs pipeline to circinus (slug 1.5) (#2208)

* ci: port Cloudflare Workers docs pipeline files to circinus (verbatim)

Copies the branch-agnostic half of the docs.vyos.io Cloudflare Workers
pipeline from `rolling` at 8cb568bf, byte-identical:

  - .github/workflows/docs-build.yml
  - workers/ (entire tree)
  - scripts/docs_gates/
  - docker/im-convert.sh
  - docs/_static/js/version-picker.js, js/pagefind-wrapper.js,
    css/version-picker.css (new files, no circinus counterpart)
  - docs/_templates/breadcrumbs.html, searchbox.html (new files)

docs-build.yml already triggers on push to [rolling, circinus, sagitta]
and resolves `circinus` -> worker vyos-docs-v15-en / slug 1.5 from
workers/matrix.json; the files simply did not exist on this branch, so
slug 1.5 still serves the bootstrap placeholder.

workers/versions.json + workers/matrix.json are deliberately identical
across all three branches and must be kept in sync.

Advances: IS-572

* ci: wire circinus docs build into the Cloudflare Workers pipeline

Hand-merges the CF-specific hunks onto circinus's own docker/Dockerfile
and docs/conf.py rather than clobbering them with rolling's versions —
circinus keeps its own content-driven history in both files.

docker/Dockerfile:
  - imagemagick + librsvg2-bin (sphinx.ext.imgconverter backend) and
    poppler-utils (pdfinfo, used by docs-build.yml's PDF page-count
    completeness check), installed --no-install-recommends
  - install docker/im-convert.sh as /usr/local/bin/im-convert

docs/conf.py:
  - enable sphinx.ext.imgconverter + image_converter = 'im-convert' so
    the LaTeX/PDF builder stops silently dropping .webp/.svg images
  - register js/version-picker.js + css/version-picker.css
    unconditionally (degrades silently on ReadTheDocs)
  - _vyos_cf_build gate off the raw DOCS_VERSION_SLUG env var; only CF
    builds load js/pagefind-wrapper.js, and html_context['vyos_cf_build']
    lets _templates/searchbox.html fall back to the stock Sphinx
    searchbox via the "!" bang-include on RTD

The RTD path stays the default in both files: circinus continues
building on ReadTheDocs until RTD sunset, and every CF feature activates
only when DOCS_VERSION_SLUG is present. .readthedocs.yml is untouched.

circinus keeps its own version/release/html_title/source_suffix and its
hardcoded html_baseurl (its CF slug is also `1.5`, so rolling's
DOCS_VERSION_SLUG/READTHEDOCS_VERSION resolution block is a no-op here
and was deliberately not ported).

Advances: IS-572

* ci: IS-572: re-sync ported Cloudflare Workers pipeline files with rolling

The category-1 files in this port are byte-identical copies from `rolling`.
`rolling` has since moved: [vyos-documentation#2209](https://github.com/vyos/vyos-documentation/pull/2209)
merged as `3a1c6c30`, thirteen rounds of hardening on exactly these files.

Re-take all 14 category-1 paths from `origin/rolling` via
`git checkout origin/rolling -- <paths>`, so byte-identity holds by
construction rather than by hand-editing:

  .github/workflows/docs-build.yml
  scripts/docs_gates/{gates,parity,smoke,test_gates,test_parity,test_smoke}.py
  workers/.gitignore
  workers/apex/src/{index,special,uagate}.ts
  workers/apex/test/{router,uagate}.test.ts
  workers/apex/ua-policy.json

Thirteen of the fourteen carry
[vyos-documentation#2209](https://github.com/vyos/vyos-documentation/pull/2209)
exactly — the pre-change tree was byte-identical to `3a1c6c30^` for those
paths. `workers/.gitignore` additionally picks up the one-line `test-results/`
entry from
[vyos-documentation#2212](https://github.com/vyos/vyos-documentation/pull/2212);
inert on circinus, since only the deliberately-unported `apex-deploy.yml`
writes that directory.

Deliberate exclusions are unchanged: `docs-canary-qa.yml` (cron runs on the
default branch only, so it is not ported even though
[vyos-documentation#2209](https://github.com/vyos/vyos-documentation/pull/2209)
touched it on `rolling`), `apex-deploy.yml`, and the `docs-preview-*`
workflows. `docs/conf.py` stays hand-merged and circinus-specific, with its
ReadTheDocs fallback intact.

🤖 Generated by [robots](https://vyos.io)

3 files changed, 165 insertions, 0 deletions

diff --git a/workers/preview/src/index.ts b/workers/preview/src/index.ts
new file mode 100644
index 00000000..5c945175
--- /dev/null
+++ b/workers/preview/src/index.ts
@@ -0,0 +1,61 @@
+export interface Env { PREVIEWS: R2Bucket }
+
+const MIME: Record<string, string> = {
+  html: "text/html; charset=utf-8", css: "text/css", js: "text/javascript",
+  json: "application/json", svg: "image/svg+xml", png: "image/png", jpg: "image/jpeg",
+  gif: "image/gif", ico: "image/x-icon", txt: "text/plain; charset=utf-8",
+  xml: "application/xml", pdf: "application/pdf", woff2: "font/woff2", woff: "font/woff",
+};
+
+export function mimeFor(key: string): string {
+  const ext = key.split(".").pop() ?? "";
+  return MIME[ext] ?? "application/octet-stream";
+}
+
+export function keyFor(pathname: string): string {
+  let key = pathname.replace(/^\//, "");
+  if (key.endsWith("/") || key === "" ) key += "index.html";
+  return key;
+}
+
+export default {
+  async fetch(request: Request, env: Env): Promise<Response> {
+    const key = keyFor(new URL(request.url).pathname);
+    let obj: R2ObjectBody | null = null;
+    let resolvedKey = key;
+    try {
+      obj = await env.PREVIEWS.get(key);
+      if (!obj && !key.split("/").pop()?.includes(".")) {
+        // Extensionless directory URL with no trailing slash (e.g. /en/rolling/cli) —
+        // keyFor() only appends index.html for trailing-slash/empty paths, so probe the
+        // directory's index.html before 404ing. Check the LAST path segment only — a dot
+        // anywhere earlier (e.g. version segment "1.4" in /pr-42/en/1.4/cli) must not skip
+        // the probe for an otherwise-extensionless final segment.
+        resolvedKey = `${key}/index.html`;
+        obj = await env.PREVIEWS.get(resolvedKey);
+      }
+    } catch {
+      // Transient R2/binding error on either probe — fail closed with a controlled 503
+      // instead of letting an unhandled exception surface as a raw worker error.
+      return new Response("preview temporarily unavailable", {
+        status: 503,
+        headers: { "X-Robots-Tag": "noindex", "Cache-Control": "no-store" },
+      });
+    }
+    if (!obj) {
+      return new Response("preview not found", {
+        status: 404,
+        // no-store on the 404 too — a cached 404 would persist past the preview upload
+        headers: { "X-Robots-Tag": "noindex", "Cache-Control": "no-store" },
+      });
+    }
+    return new Response(obj.body, {
+      headers: {
+        "content-type": obj.httpMetadata?.contentType ?? mimeFor(resolvedKey),
+        "X-Robots-Tag": "noindex",
+        "Cache-Control": "no-store",
+        "X-Content-Type-Options": "nosniff",
+      },
+    });
+  },
+} satisfies ExportedHandler<Env>;
diff --git a/workers/preview/test/preview.test.ts b/workers/preview/test/preview.test.ts
new file mode 100644
index 00000000..90530690
--- /dev/null
+++ b/workers/preview/test/preview.test.ts
@@ -0,0 +1,93 @@
+import { describe, it, expect } from "vitest";
+import worker, { mimeFor, keyFor } from "../src/index";
+import type { Env } from "../src/index";
+
+describe("preview worker helpers (§10)", () => {
+  it("derives R2 key from path, defaulting directory to index.html", () => {
+    expect(keyFor("/pr-42/en/rolling/")).toBe("pr-42/en/rolling/index.html");
+    expect(keyFor("/pr-42/en/rolling/cli/index.html")).toBe("pr-42/en/rolling/cli/index.html");
+  });
+  it("extension→MIME fallback map (nosniff-safe)", () => {
+    expect(mimeFor("a.html")).toBe("text/html; charset=utf-8");
+    expect(mimeFor("a.css")).toBe("text/css");
+    expect(mimeFor("a.js")).toBe("text/javascript");
+    expect(mimeFor("a.json")).toBe("application/json");
+    expect(mimeFor("a.svg")).toBe("image/svg+xml");
+    expect(mimeFor("a.woff2")).toBe("font/woff2");
+    expect(mimeFor("a.unknown")).toBe("application/octet-stream");
+  });
+});
+
+// Fake R2Bucket mock following the makeEnv() precedent in apex/test/router.test.ts —
+// only the `get` surface the handler consumes; miniflare's real R2 not needed here.
+function makeEnv(objects: Record<string, { body: string; contentType?: string }>): Env {
+  return {
+    PREVIEWS: {
+      get: async (key: string) => {
+        const hit = objects[key];
+        if (!hit) return null;
+        return {
+          body: hit.body,
+          httpMetadata: hit.contentType ? { contentType: hit.contentType } : undefined,
+        };
+      },
+    } as unknown as R2Bucket,
+  };
+}
+
+const get = (path: string, env: Env) =>
+  worker.fetch(new Request(`https://docs-preview.vyos.io${path}`), env);
+
+describe("preview worker fetch entrypoint (§10)", () => {
+  it("serves a found object with uploader contentType + noindex/no-store/nosniff headers", async () => {
+    const env = makeEnv({
+      "pr-42/en/rolling/index.html": { body: "<h1>preview</h1>", contentType: "text/html; charset=utf-8" },
+    });
+    const r = await get("/pr-42/en/rolling/", env);
+    expect(r.status).toBe(200);
+    expect(await r.text()).toBe("<h1>preview</h1>");
+    expect(r.headers.get("content-type")).toBe("text/html; charset=utf-8");
+    expect(r.headers.get("X-Robots-Tag")).toBe("noindex");
+    expect(r.headers.get("Cache-Control")).toBe("no-store");
+    expect(r.headers.get("X-Content-Type-Options")).toBe("nosniff");
+  });
+  it("missing object → 404 with noindex AND no-store (404s must never be cached)", async () => {
+    const r = await get("/pr-42/en/rolling/missing.html", makeEnv({}));
+    expect(r.status).toBe(404);
+    expect(r.headers.get("X-Robots-Tag")).toBe("noindex");
+    expect(r.headers.get("Cache-Control")).toBe("no-store");
+  });
+  it("object without httpMetadata.contentType falls back to mimeFor(key)", async () => {
+    const env = makeEnv({ "pr-42/en/rolling/style.css": { body: "body{}" } });
+    const r = await get("/pr-42/en/rolling/style.css", env);
+    expect(r.status).toBe(200);
+    expect(r.headers.get("content-type")).toBe("text/css");
+  });
+  it("extensionless directory URL with no trailing slash serves the directory's index.html", async () => {
+    const env = makeEnv({
+      "pr-42/en/rolling/cli/index.html": { body: "<h1>cli</h1>", contentType: "text/html; charset=utf-8" },
+    });
+    const r = await get("/pr-42/en/rolling/cli", env);
+    expect(r.status).toBe(200);
+    expect(await r.text()).toBe("<h1>cli</h1>");
+    expect(r.headers.get("content-type")).toBe("text/html; charset=utf-8");
+  });
+  it("extensionless directory URL with a dotted version segment earlier in the path still probes index.html (dot-check is last-segment-only)", async () => {
+    const env = makeEnv({
+      "pr-42/en/1.4/cli/index.html": { body: "<h1>cli 1.4</h1>", contentType: "text/html; charset=utf-8" },
+    });
+    const r = await get("/pr-42/en/1.4/cli", env);
+    expect(r.status).toBe(200);
+    expect(await r.text()).toBe("<h1>cli 1.4</h1>");
+    expect(r.headers.get("content-type")).toBe("text/html; charset=utf-8");
+  });
+  it("a transient R2/binding error on either probe degrades to a controlled 503 (never an unhandled exception)", async () => {
+    const throwingEnv: Env = {
+      PREVIEWS: { get: async () => { throw new Error("R2 unavailable"); } } as unknown as R2Bucket,
+    };
+    const r = await get("/pr-42/en/rolling/cli", throwingEnv);
+    expect(r.status).toBe(503);
+    expect(r.headers.get("X-Robots-Tag")).toBe("noindex");
+    expect(r.headers.get("Cache-Control")).toBe("no-store");
+  });
+});
diff --git a/workers/preview/wrangler.jsonc b/workers/preview/wrangler.jsonc
new file mode 100644
index 00000000..fbdb4aba
--- /dev/null
+++ b/workers/preview/wrangler.jsonc
@@ -0,0 +1,11 @@
+{
+  "$schema": "node_modules/wrangler/config-schema.json",
+  "name": "vyos-docs-preview",
+  "main": "src/index.ts",
+  "compatibility_date": "2026-07-01",
+  "workers_dev": false,
+  "preview_urls": false,
+  // Route managed manually (v4.2 amendment — CI token has no Workers Routes group):
+  // operator creates docs-preview.vyos.io/* → vyos-docs-preview once, alongside the canary route.
+  "r2_buckets": [{ "binding": "PREVIEWS", "bucket_name": "vyos-docs-previews" }]
+}