diff options
| author | Yuriy Andamasov <yuriy@vyos.io> | 2026-07-10 17:14:13 +0300 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2026-07-10 16:14:13 +0200 |
| commit | 21689ef59b0eb34b1a29eda739dc10d33d25b44a (patch) | |
| tree | 95883c2ee70d4c09562ae3e0289e18f34999240c /.github/workflows | |
| parent | 6d2d28d4411ff52ad0f9314cf3696336748ab89d (diff) | |
| download | vyos-documentation-21689ef59b0eb34b1a29eda739dc10d33d25b44a.tar.gz vyos-documentation-21689ef59b0eb34b1a29eda739dc10d33d25b44a.zip | |
docs: Cloudflare Workers hosting pipeline (apex, content workers, CI, previews) (#2140)
* docs-infra: scaffold Cloudflare workers workspace (versions.json v2, matrix, toolchain)
๐ค Generated by [robots](https://vyos.io)
* docs-infra: record full Phase-0 plan decision in workers/PLAN.md
๐ค Generated by [robots](https://vyos.io)
* docs-infra: shared content worker โ asset serving, cache classes, X-Docs-Build, canary no-store
๐ค Generated by [robots](https://vyos.io)
* docs-infra: run worker script before assets; test fetch entrypoint
๐ค Generated by [robots](https://vyos.io)
* docs-infra: apex manifest loader + dispatch map + runtime binding guard (TDD)
๐ค Generated by [robots](https://vyos.io)
* docs-infra: apex redirects (aliases, PDF, trailing-slash) + special paths (TDD)
๐ค Generated by [robots](https://vyos.io)
* docs-infra: PDF redirect honors pdf:null and preserves query
๐ค Generated by [robots](https://vyos.io)
* docs-infra: apex UA gate โ allowlist-wins, log-only AI crawlers, empty block list at launch
๐ค Generated by [robots](https://vyos.io)
* docs-infra: apex router (pipeline ยง3.2), themed 404/503, /kb seam, env configs + congruence test
๐ค Generated by [robots](https://vyos.io)
* docs-infra: add missing-User-Agent regression test for apex UA gate
๐ค Generated by [robots](https://vyos.io)
* docs-infra: R2-streaming preview worker โ MIME map, noindex, no-store (TDD)
๐ค Generated by [robots](https://vyos.io)
* docs-infra: preview 404 no-store + fetch handler tests
๐ค Generated by [robots](https://vyos.io)
* docs-infra: bootstrap script โ binding-target workers must exist before apex deploys
๐ค Generated by [robots](https://vyos.io)
* docs-infra: apex run_worker_first, lockfile for npm ci, PDF Location from manifest
๐ค Generated by [robots](https://vyos.io)
* docs-infra: derive html_baseurl from DOCS_VERSION_SLUG with RTD fallback (canonical gate prereq)
๐ค Generated by [robots](https://vyos.io)
* docs-infra: version picker + status banner + language scaffold (vanilla JS, TDD pure core)
๐ค Generated by [robots](https://vyos.io)
* docs-infra: picker preserves query+hash across switch; valid breadcrumb markup
๐ค Generated by [robots](https://vyos.io)
* docs-infra: Pagefind search wrapper with runtime base-path + preview prefix handling (TDD)
๐ค Generated by [robots](https://vyos.io)
* docs-infra: pagefind wrapper โ asset-failure notice + UI stylesheet load
๐ค Generated by [robots](https://vyos.io)
* docs-infra: gate Pagefind searchbox to CF builds (RTD keeps stock search until cutover)
๐ค Generated by [robots](https://vyos.io)
* docs-infra: deploy sanity gates โ limits, critical pages, count-delta, canonical (TDD)
๐ค Generated by [robots](https://vyos.io)
* docs-infra: hermetic gate tests via fixture versions.json
๐ค Generated by [robots](https://vyos.io)
* docs-infra: docs-build workflow โ candidate/smoke/promote two-stage deploy + registry + rollback
Two-stage CF Workers pipeline: build in pinned container, assemble artifact,
sanity gates, deploy candidate, scoped pre-traffic smoke via canary apex,
promote (rollback-id capture, hostname purge, registry upload), post-promote
probe + auto-rollback. DOCS_CF_LIVE repo variable gates every docs.vyos.io
production interaction pre-cutover.
scripts/docs_gates/smoke.py adds one authorized check beyond the spec: the
version's index.html probe asserts the #vyos-search mount div is present in
the response body, guarding CI silently forgetting DOCS_VERSION_SLUG (which
would otherwise ship stock RTD search without the Pagefind gate noticing).
๐ค Generated by [robots](https://vyos.io)
* docs-infra: build docs image in-workflow with buildx cache (v4.1 โ digest pin dropped)
Plan v4.1 amendment: the ghcr.io digest-pinned image does not exist (workflow
would hard-fail at the first docker step on every push). Replace the BUILD_IMAGE
env placeholder with an in-workflow docker build from docker/Dockerfile via
docker/setup-buildx-action@v3 + docker/build-push-action@v6 (context: docker/,
load: true, tags: docs-build:local, GHA cache from/to). The checked-out commit
is the pin; buildx GHA cache keeps repeat builds cheap. Sphinx-build step swaps
to docs-build:local; inner script unchanged.
๐ค Generated by [robots](https://vyos.io)
* docs-infra: apex/preview deploy workflow โ canary auto, production behind environment approval
* docs-infra: apex-deploy concurrency guard (per-ref, cancel-in-progress)
๐ค Generated by [robots](https://vyos.io)
* docs-infra: fork-safe PR preview pipeline โ approval record, R2 prefixes, label consumption, cleanup
* docs-infra: nightly preview sweep โ pipefail + per-prefix failure isolation
๐ค Generated by [robots](https://vyos.io)
* docs-infra: nightly canary QA โ per-entry sweep + URL-parity corpus vs RTD
๐ค Generated by [robots](https://vyos.io)
* docs-infra: parity sweep scoped to CF-built versions; transport-error resilience
๐ค Generated by [robots](https://vyos.io)
* docs-infra: one-off bootstrap workflow (binding targets โ runs once on this push)
๐ค Generated by [robots](https://vyos.io)
* docs-infra: remove one-off bootstrap workflow (bootstrap complete)
๐ค Generated by [robots](https://vyos.io)
* docs-infra: one-off canary apex + preview deploy (route targets for Task 3.6 step 2c)
๐ค Generated by [robots](https://vyos.io)
* docs-infra: remove one-off canary deploy workflow (targets live)
๐ค Generated by [robots](https://vyos.io)
* docs-infra: address Phase-0 CodeRabbit findings (canonical gate, error caching, registry pointer, validation)
๐ค Generated by [robots](https://vyos.io)
* docs-infra: strengthen manifest tests (full dispatch iteration, mutation-free validate)
๐ค Generated by [robots](https://vyos.io)
* docs-infra: address GitHub CodeRabbit review (pointer-after-probe, fail-closed sweeps, block-precedence UA gate, preview hardening)
๐ค Generated by [robots](https://vyos.io)
* docs-infra: adversarial review fixes โ error no-store, probe retry, PR-list membership, preview dotted-segment
๐ค Generated by [robots](https://vyos.io)
* docs-infra: serve oversized legacy PDF from R2 via apex (spec ยง5 fallback)
The 1.3 PDF (29.2 MiB) exceeds the 25 MiB static-asset cap and is absent
from the legacy content Worker's build, so /_/downloads/en/1.3/pdf/ (and
the picker's PDF link) 301'd into a dead-end 404 post-cutover. Add the R2
object fallback spec ยง5 already documented but never implemented: a
DOCS_PDFS R2 bucket binding on the apex Worker, a manifest pdf_r2_key
field (1.3 only), and a router step ahead of version dispatch that streams
the object with its own cache class (canary/error still force no-store).
๐ค Generated by [robots](https://vyos.io)
* docs-infra: PDF R2 fallback honors Range + If-None-Match, preserves ETag
๐ค Generated by [robots](https://vyos.io)
Diffstat (limited to '.github/workflows')
| -rw-r--r-- | .github/workflows/apex-deploy.yml | 74 | ||||
| -rw-r--r-- | .github/workflows/docs-build.yml | 340 | ||||
| -rw-r--r-- | .github/workflows/docs-canary-qa.yml | 47 | ||||
| -rw-r--r-- | .github/workflows/docs-preview-approve.yml | 35 | ||||
| -rw-r--r-- | .github/workflows/docs-preview-build.yml | 26 | ||||
| -rw-r--r-- | .github/workflows/docs-preview-cleanup.yml | 43 | ||||
| -rw-r--r-- | .github/workflows/docs-preview-deploy.yml | 86 |
7 files changed, 651 insertions, 0 deletions
diff --git a/.github/workflows/apex-deploy.yml b/.github/workflows/apex-deploy.yml new file mode 100644 index 00000000..1528221b --- /dev/null +++ b/.github/workflows/apex-deploy.yml @@ -0,0 +1,74 @@ +name: Apex + preview worker deploy + +on: + push: + branches: [rolling] + paths: ["workers/**"] + workflow_dispatch: {} + +concurrency: + group: apex-deploy-${{ github.ref_name }} + # false (not true): a mid-job cancel can leave the apex/preview worker pair + # half-updated; queuing subsequent runs is safe. + cancel-in-progress: false + +permissions: + contents: read + +jobs: + test: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + - uses: actions/setup-node@v4 + with: { node-version: 22 } + - run: cd workers && npm ci && npx vitest run + + deploy-canary: + needs: test + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + - uses: actions/setup-node@v4 + with: { node-version: 22 } + - name: Preflight โ every service-binding target must already exist (Task 1.8) + env: + CLOUDFLARE_API_TOKEN: ${{ secrets.CLOUDFLARE_API_TOKEN_DOCS }} + CLOUDFLARE_ACCOUNT_ID: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }} + run: | + set -eu + cd workers && npm ci + missing=0 + for w in vyos-docs-rolling-en vyos-docs-v15-en vyos-docs-v14-en vyos-docs-legacy; do + for suffix in "" "-candidate"; do + npx wrangler deployments list --name "$w$suffix" --json >/dev/null 2>&1 \ + || { echo "::error::binding target $w$suffix does not exist โ run workers/bootstrap.sh first (Task 1.8)"; missing=1; } + done + done + exit $missing + - name: Deploy apex (canary) + preview + env: + CLOUDFLARE_API_TOKEN: ${{ secrets.CLOUDFLARE_API_TOKEN_DOCS }} + CLOUDFLARE_ACCOUNT_ID: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }} + run: | + cd workers && npm ci + npx wrangler deploy --config apex/wrangler.jsonc --env canary \ + --var APEX_BUILD_SHA:'${{ github.sha }}' + npx wrangler deploy --config preview/wrangler.jsonc + + deploy-production: + needs: deploy-canary + runs-on: ubuntu-latest + environment: docs-production # requires reviewer approval (ยง7.2) + steps: + - uses: actions/checkout@v4 + - uses: actions/setup-node@v4 + with: { node-version: 22 } + - name: Deploy apex (production) + env: + CLOUDFLARE_API_TOKEN: ${{ secrets.CLOUDFLARE_API_TOKEN_DOCS }} + CLOUDFLARE_ACCOUNT_ID: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }} + run: | + cd workers && npm ci + npx wrangler deploy --config apex/wrangler.jsonc --env production \ + --var APEX_BUILD_SHA:'${{ github.sha }}' diff --git a/.github/workflows/docs-build.yml b/.github/workflows/docs-build.yml new file mode 100644 index 00000000..fc9a2239 --- /dev/null +++ b/.github/workflows/docs-build.yml @@ -0,0 +1,340 @@ +name: Docs build + deploy (Cloudflare Workers) + +on: + push: + branches: [rolling, circinus, sagitta] + workflow_dispatch: + inputs: + skip_pdf: + description: "SKIP_PDF: carry forward last-good PDF from registry (broken-LaTeX escape hatch)" + type: boolean + default: false + +concurrency: + group: docs-build-${{ github.ref_name }} + # false (not true): a cancel mid-promote would leave production unverified and the + # registry pointer stale โ queued runs are safe because the check_head guard (ยง7.1) + # skips any run whose SHA is no longer the branch tip. + cancel-in-progress: false + +permissions: + contents: read + +env: + REGISTRY_BUCKET: vyos-docs-artifacts + +jobs: + build-deploy: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + + - name: Resolve matrix entry + id: matrix + run: | + set -eu + entry=$(jq -r --arg b "${{ github.ref_name }}" '.[$b] // empty' workers/matrix.json) + [ -n "$entry" ] || { echo "branch not in matrix"; exit 1; } + echo "worker=$(echo "$entry" | jq -r .worker)" >> "$GITHUB_OUTPUT" + echo "slug=$(echo "$entry" | jq -r .slug)" >> "$GITHUB_OUTPUT" + echo "pdf=$(jq -r --arg s "$(echo "$entry" | jq -r .slug)" \ + '.versions[] | select(.slug==$s) | .pdf // empty' workers/versions.json)" >> "$GITHUB_OUTPUT" + + # Build image in-workflow from docker/Dockerfile (plan v4.1: digest pin dropped โ + # no published ghcr.io image exists; the checked-out commit IS the pin, and the + # buildx GHA cache makes repeat builds cheap). + - name: Set up Docker Buildx + uses: docker/setup-buildx-action@v3 + + - name: Build docs image (buildx GHA cache) + uses: docker/build-push-action@v6 + with: + context: docker/ + load: true + tags: docs-build:local + cache-from: type=gha + cache-to: type=gha,mode=max + + - name: Build HTML + PDF in in-workflow-built container + run: | + set -eu + docker run --rm -v "$PWD:/src" -w /src \ + -e DOCS_VERSION_SLUG="${{ steps.matrix.outputs.slug }}" \ + -e DOCS_VERSION_BRANCH="${{ github.ref_name }}" \ + docs-build:local bash -c ' + cd docs && make html + if [ "${{ inputs.skip_pdf }}" != "true" ]; then + make latexpdf || make latexpdf # retry once (ยง14 LaTeX flakiness) + fi' + + - name: Assemble artifact (nest under en/<slug>/, ยง7.1) + run: | + set -eu + slug='${{ steps.matrix.outputs.slug }}' + mkdir -p "dist/assets/en/$slug" + cp -r docs/_build/html/. "dist/assets/en/$slug/" + npx --yes pagefind@1.5.2 --site "dist/assets/en/$slug" + + - name: PDF into artifact (build or registry carry-forward) + env: + CLOUDFLARE_API_TOKEN: ${{ secrets.CLOUDFLARE_API_TOKEN_DOCS }} + CLOUDFLARE_ACCOUNT_ID: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }} + DOCS_CF_LIVE: ${{ vars.DOCS_CF_LIVE }} + run: | + set -eu + slug='${{ steps.matrix.outputs.slug }}' + dest="dist/assets/en/$slug/vyos-documentation.pdf" + if [ "${{ inputs.skip_pdf }}" = "true" ]; then + # ยง5 + ยง7.1.4: SKIP_PDF refuses to run when registry is stale vs production. + # Registry layout is pointer-indirected (ยง7.1.4 atomicity): resolve + # $slug/latest.json โ sha, then read the sha-scoped generation. + cd workers && npm ci && npx wrangler r2 object get "$REGISTRY_BUCKET/$slug/latest.json" --file /tmp/latest.json --remote && cd .. + reg_sha=$(jq -r .sha /tmp/latest.json) + if [ "$DOCS_CF_LIVE" = "true" ]; then + prod_sha=$(curl -sI "https://docs.vyos.io/en/$slug/" | tr -d '\r' | awk -F': ' 'tolower($1)=="x-docs-build"{print $2}') + [ "$reg_sha" = "$prod_sha" ] || { echo "registry stale ($reg_sha != $prod_sha) โ SKIP_PDF refused"; exit 1; } + else + echo "DOCS_CF_LIVE=false โ pre-cutover: trusting registry meta without production comparison" + fi + cd workers && npx wrangler r2 object get "$REGISTRY_BUCKET/$slug/$reg_sha/pdf" --file "../$dest" --remote && cd .. + else + cp docs/_build/latex/*.pdf "$dest" + fi + + - name: Fetch previous-build metadata (for count-delta gate) + env: + CLOUDFLARE_API_TOKEN: ${{ secrets.CLOUDFLARE_API_TOKEN_DOCS }} + CLOUDFLARE_ACCOUNT_ID: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }} + run: | + set -eo pipefail + set -u + slug='${{ steps.matrix.outputs.slug }}' + cd workers + # Missing-object (no pointer yet, or the pointed-at generation's meta.json is + # gone) is the legitimate bootstrap case โ this slug has never shipped before, + # so proceed with no previous-meta.json (the count-delta gate below no-ops when + # the file is absent). Any OTHER failure (auth, network, transient API error) + # must FAIL the workflow โ silently swallowing those would bypass the + # count-collapse gate instead of just skipping it on a genuinely first deploy. + fetch_optional() { + set +e + out=$(npx wrangler r2 object get "$1" --file "$2" --remote 2>&1) + rc=$? + set -e + echo "$out" + [ "$rc" -eq 0 ] && return 0 + if echo "$out" | grep -qiE 'does not exist|not found|no such key|404'; then + return 1 + fi + echo "::error::registry fetch of $1 failed for a reason other than 'missing object' โ failing the build (see spec ยง7.1 count-delta gate)" + exit "$rc" + } + if fetch_optional "$REGISTRY_BUCKET/$slug/latest.json" /tmp/latest.json; then + reg_sha=$(jq -r .sha /tmp/latest.json) + if [ -z "$reg_sha" ] || [ "$reg_sha" = "null" ]; then + echo "::error::registry pointer latest.json is malformed (no .sha)" + exit 1 + fi + if ! fetch_optional "$REGISTRY_BUCKET/$slug/$reg_sha/meta.json" ../previous-meta.json; then + echo "::notice::registry pointer names $reg_sha but its meta.json is missing โ treating as bootstrap" + fi + else + echo "::notice::no registry entry yet for $slug โ treating as bootstrap (no previous-meta)" + fi + + - name: Sanity gates (deploy blockers, ยง7.1) + run: | + python -m scripts.docs_gates.gates \ + --artifact dist/assets --slug '${{ steps.matrix.outputs.slug }}' \ + --versions workers/versions.json \ + $( [ -f previous-meta.json ] && echo --previous-meta previous-meta.json ) + + - name: check_head guard (ยง7.1) + run: | + set -eu + remote=$(git ls-remote origin "refs/heads/${{ github.ref_name }}" | cut -f1) + [ "$remote" = "${{ github.sha }}" ] || { echo "branch moved โ skipping deploy"; exit 78; } + + - name: Install workers deps + run: cd workers && npm ci + + - name: Deploy CANDIDATE + env: + CLOUDFLARE_API_TOKEN: ${{ secrets.CLOUDFLARE_API_TOKEN_DOCS }} + CLOUDFLARE_ACCOUNT_ID: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }} + run: | + cd workers && npx wrangler deploy --config branch/wrangler.$( \ + case '${{ steps.matrix.outputs.slug }}' in rolling) echo rolling;; 1.5) echo v15;; 1.4) echo v14;; esac \ + ).jsonc \ + --name '${{ steps.matrix.outputs.worker }}-candidate' \ + --var DOCS_BUILD_SHA:'${{ github.sha }}' --var DOCS_ENV:canary + + - name: Pre-traffic smoke via canary apex (ยง7.1.2) + id: smoke + run: | + set -eu + # NOTE: never use the GHA "cond AND format(...)" expression trick for optional + # args here โ an empty/false result renders the literal string "false" into the + # shell. (Any double-curly GHA expression written literally in this script gets + # template-expanded by GitHub Actions before the shell ever runs it, so such an + # example cannot even be spelled out in a comment here.) Plain shell instead: + pdf_arg="" + if [ -n "${{ steps.matrix.outputs.pdf }}" ]; then + pdf_arg="--pdf ${{ steps.matrix.outputs.pdf }}" + fi + python -m scripts.docs_gates.smoke \ + --host docs-next.vyos.io --slug '${{ steps.matrix.outputs.slug }}' \ + --expect-sha '${{ github.sha }}' \ + --access-id '${{ secrets.CF_ACCESS_CLIENT_ID }}' \ + --access-secret '${{ secrets.CF_ACCESS_CLIENT_SECRET }}' \ + $pdf_arg + + - name: Candidate reset on smoke failure (ยง7.1.3) + if: failure() && steps.smoke.conclusion == 'failure' + env: + CLOUDFLARE_API_TOKEN: ${{ secrets.CLOUDFLARE_API_TOKEN_DOCS }} + CLOUDFLARE_ACCOUNT_ID: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }} + DOCS_CF_LIVE: ${{ vars.DOCS_CF_LIVE }} + run: | + set -eu + slug='${{ steps.matrix.outputs.slug }}' + cd workers + # Registry layout is pointer-indirected (ยง7.1.4 atomicity): resolve + # $slug/latest.json โ sha, then read the sha-scoped generation. + npx wrangler r2 object get "$REGISTRY_BUCKET/$slug/latest.json" --file /tmp/latest.json --remote \ + || { echo "::warning::no registry entry yet (first deploys) โ candidate reset skipped"; exit 0; } + reg_sha=$(jq -r .sha /tmp/latest.json) + if [ "$DOCS_CF_LIVE" = "true" ]; then + prod_sha=$(curl -sI "https://docs.vyos.io/en/$slug/" | tr -d '\r' | awk -F': ' 'tolower($1)=="x-docs-build"{print $2}') + [ "$reg_sha" = "$prod_sha" ] || { echo "::warning::registry stale โ candidate reset SKIPPED (manual registry-repair needed)"; exit 0; } + fi + npx wrangler r2 object get "$REGISTRY_BUCKET/$slug/$reg_sha/tar.zst" --file /tmp/lastgood.tar.zst --remote + rm -rf ../dist/assets && mkdir -p ../dist/assets + tar --zstd -xf /tmp/lastgood.tar.zst -C ../dist/assets + npx wrangler deploy --config branch/wrangler.$(case "$slug" in rolling) echo rolling;; 1.5) echo v15;; 1.4) echo v14;; esac).jsonc \ + --name '${{ steps.matrix.outputs.worker }}-candidate' \ + --var DOCS_BUILD_SHA:"$reg_sha" --var DOCS_ENV:canary + + - name: PROMOTE (capture rollback id โ deploy โ purge โ registry, ยง7.1.4) + id: promote + env: + CLOUDFLARE_API_TOKEN: ${{ secrets.CLOUDFLARE_API_TOKEN_DOCS }} + CLOUDFLARE_ACCOUNT_ID: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }} + CF_ZONE_ID: ${{ vars.CF_ZONE_ID_VYOS_IO }} + run: | + set -eu + slug='${{ steps.matrix.outputs.slug }}' + worker='${{ steps.matrix.outputs.worker }}' + cd workers + # Rollback target: verify this JSON shape on the FIRST real deploy โ + # `npx wrangler deployments list --name "$worker" --json | jq .` โ and adjust + # the jq path if wrangler's output differs (their JSON shape has churned across + # majors; the ID needed is the CURRENT deployment's, i.e. the newest entry). + # FIRST-DEPLOY BOOTSTRAP: the production Worker does not exist before the very + # first promote โ `deployments list` fails/returns empty. That is a valid state: + # rollback_id stays empty, the deploy CREATES the Worker, and the post-promote + # step knows an empty id means "nothing to roll back to". + if deps=$(npx wrangler deployments list --name "$worker" --json 2>/dev/null); then + rollback_id=$(echo "$deps" | jq -r '.[0].id // empty') + else + rollback_id="" + echo "first deploy for $worker โ no prior deployment; rollback disabled for this run" + fi + echo "rollback_id=$rollback_id" >> "$GITHUB_OUTPUT" + npx wrangler deploy --config branch/wrangler.$(case "$slug" in rolling) echo rolling;; 1.5) echo v15;; 1.4) echo v14;; esac).jsonc \ + --name "$worker" --var DOCS_BUILD_SHA:'${{ github.sha }}' --var DOCS_ENV:production + if [ "${{ vars.DOCS_CF_LIVE }}" = "true" ]; then + # hostname-scoped purge (ยง3.3; all-plans since 2025-04) + curl -sf -X POST "https://api.cloudflare.com/client/v4/zones/$CF_ZONE_ID/purge_cache" \ + -H "Authorization: Bearer $CLOUDFLARE_API_TOKEN" -H "Content-Type: application/json" \ + --data '{"hosts":["docs.vyos.io"]}' + else + echo "DOCS_CF_LIVE=false โ pre-cutover: skipping production hostname purge" + fi + # registry generation upload โ SHA-scoped immutable objects (ยง7.1.4). The + # $slug/latest.json POINTER is deliberately NOT published here: it moves to a + # dedicated step AFTER the post-promote probe/auto-rollback below, so a probe + # failure (and the rollback it triggers) leaves the pointer on the previous + # good generation instead of advancing it to a generation that just got rolled + # back. Uploading straight to $slug/last-good.* let a concurrent reader observe + # a half-written generation (e.g. new tar.zst, still-old meta.json) mid upload. + # Uploading under $slug/<sha>/* first โ a key no prior generation ever reused โ + # keeps readers of the OLD pointer seeing a full, untouched old generation. + cd .. && tar --zstd -cf lastgood.tar.zst -C dist/assets . + page_count=$(find "dist/assets/en/$slug" -name '*.html' | wc -l | tr -d ' ') + sha='${{ github.sha }}' + printf '{"sha":"%s","page_count":%s}' "$sha" "$page_count" > lastgood.meta.json + printf '{"sha":"%s"}' "$sha" > latest.json + cd workers + upload_generation() { + npx wrangler r2 object put "$REGISTRY_BUCKET/$slug/$sha/tar.zst" --file ../lastgood.tar.zst --remote && + npx wrangler r2 object put "$REGISTRY_BUCKET/$slug/$sha/pdf" --file "../dist/assets/en/$slug/vyos-documentation.pdf" --remote && + npx wrangler r2 object put "$REGISTRY_BUCKET/$slug/$sha/meta.json" --file ../lastgood.meta.json --remote + } + if ! (upload_generation || upload_generation); then + echo "::error::registry generation upload failed twice โ REGISTRY STALE (see spec ยง7.1.4)" + exit 1 + fi + + - name: Post-promote probe + auto-rollback (ยง7.1.5) + env: + CLOUDFLARE_API_TOKEN: ${{ secrets.CLOUDFLARE_API_TOKEN_DOCS }} + CLOUDFLARE_ACCOUNT_ID: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }} + CF_ZONE_ID: ${{ vars.CF_ZONE_ID_VYOS_IO }} + DOCS_CF_LIVE: ${{ vars.DOCS_CF_LIVE }} + run: | + set -eu + if [ "$DOCS_CF_LIVE" != "true" ]; then + echo "DOCS_CF_LIVE=false โ pre-cutover: docs.vyos.io still serves RTD; skipping production probe" + exit 0 + fi + slug='${{ steps.matrix.outputs.slug }}' + # Retry loop: the purge above is asynchronous, so an immediate probe can still + # observe a stale edge response and trigger a false rollback. Poll up to 6 times + # (60s budget) before concluding the promote genuinely failed. + got="" + for attempt in 1 2 3 4 5 6; do + got=$(curl -sI "https://docs.vyos.io/en/$slug/" | tr -d '\r' | awk -F': ' 'tolower($1)=="x-docs-build"{print $2}') + if [ "$got" = '${{ github.sha }}' ]; then + break + fi + if [ "$attempt" -lt 6 ]; then + echo "::notice::post-promote probe attempt $attempt/6 saw stale edge (got $got) โ retrying in 10s" + sleep 10 + fi + done + if [ "$got" != '${{ github.sha }}' ]; then + if [ -z '${{ steps.promote.outputs.rollback_id }}' ]; then + echo "::error::post-promote probe failed (got $got) on FIRST deploy โ no prior version to roll back to; investigate manually" + exit 1 + fi + echo "::error::post-promote probe failed (got $got) โ rolling back" + cd workers && npx wrangler rollback '${{ steps.promote.outputs.rollback_id }}' \ + --name '${{ steps.matrix.outputs.worker }}' --yes + curl -sf -X POST "https://api.cloudflare.com/client/v4/zones/$CF_ZONE_ID/purge_cache" \ + -H "Authorization: Bearer $CLOUDFLARE_API_TOKEN" -H "Content-Type: application/json" \ + --data '{"hosts":["docs.vyos.io"]}' + exit 1 + fi + + - name: Publish registry pointer (ยง7.1.4) + env: + CLOUDFLARE_API_TOKEN: ${{ secrets.CLOUDFLARE_API_TOKEN_DOCS }} + CLOUDFLARE_ACCOUNT_ID: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }} + run: | + set -eu + slug='${{ steps.matrix.outputs.slug }}' + sha='${{ github.sha }}' + cd workers + # Runs only after the post-promote probe/auto-rollback step above has passed (or + # was a pre-cutover no-op). Repointing $slug/latest.json is the sole remaining + # step of the ยง7.1.4 atomic-publish sequence โ the sha-scoped generation objects + # were already uploaded in PROMOTE. If this fails, the sha-scoped objects landed + # under $slug/$sha/ but latest.json still points at the previous generation. + publish_pointer() { + npx wrangler r2 object put "$REGISTRY_BUCKET/$slug/latest.json" --file ../latest.json --remote + } + if ! (publish_pointer || publish_pointer); then + echo "::error::registry pointer publish failed twice โ sha-scoped objects landed under $slug/$sha/ but latest.json still points at the previous generation (see spec ยง7.1.4)" + exit 1 + fi diff --git a/.github/workflows/docs-canary-qa.yml b/.github/workflows/docs-canary-qa.yml new file mode 100644 index 00000000..7f3be7c0 --- /dev/null +++ b/.github/workflows/docs-canary-qa.yml @@ -0,0 +1,47 @@ +name: Docs canary QA (nightly per-entry sweep + parity) +on: + schedule: + - cron: "45 2 * * *" + workflow_dispatch: {} +permissions: + contents: read +jobs: + qa: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + - name: Per-entry sweep (one page + PDF per versions.json entry, ยง3.4 gate c) + run: | + set -eu + # Capture-first (not pipe/process-substitution-into-while): a malformed + # versions.json must fail this step loudly. Piping/process-substituting jq + # directly into the while loop hides a jq failure from `set -e` (the loop just + # sees empty stdin and "passes" with zero iterations) โ capture the output into + # a variable first so a non-zero jq exit trips `set -e` before the loop runs. + versions=$(jq -ce '.versions[]' workers/versions.json) + fail=0 + while read -r v; do + slug=$(echo "$v" | jq -r .slug); pdf=$(echo "$v" | jq -r '.pdf // empty') + if ! python -m scripts.docs_gates.smoke --host docs-next.vyos.io --slug "$slug" \ + --expect-sha SKIP --access-id '${{ secrets.CF_ACCESS_CLIENT_ID }}' \ + --access-secret '${{ secrets.CF_ACCESS_CLIENT_SECRET }}' \ + ${pdf:+--pdf "$pdf"}; then + echo "sweep-fail=$slug" | tee -a "$GITHUB_STEP_SUMMARY" + fail=1 + fi + done <<< "$versions" + exit $fail # a broken advertised version MUST fail nightly QA (ยง3.4 gate c) + # --expect-sha SKIP = presence-only assert; implemented in smoke.docs_build_ok (Task 3.2). + - name: URL parity vs RTD (pre-cutover truth) + run: | + # --slugs pinned to CF-built versions: 1.3/1.2 have no RTD sitemaps (ยง15a.5); + # legacy parity is the snapshot repo's crawl-inventory job (defense in depth + # alongside the matching parity.py DEFAULT_SLUGS) + python -m scripts.docs_gates.parity \ + --sitemap-host docs.vyos.io --probe-host docs-next.vyos.io \ + --slugs rolling,1.5,1.4 \ + --access-id '${{ secrets.CF_ACCESS_CLIENT_ID }}' \ + --access-secret '${{ secrets.CF_ACCESS_CLIENT_SECRET }}' + - uses: actions/upload-artifact@v4 + if: always() + with: { name: parity-report, path: parity-report.json } diff --git a/.github/workflows/docs-preview-approve.yml b/.github/workflows/docs-preview-approve.yml new file mode 100644 index 00000000..f0fb713b --- /dev/null +++ b/.github/workflows/docs-preview-approve.yml @@ -0,0 +1,35 @@ +name: Docs preview approval record +on: + pull_request_target: + types: [labeled, synchronize] +permissions: + statuses: write + pull-requests: write + issues: write # `gh pr edit --remove-label` mutates labels via the issues API +jobs: + strip-on-push: + if: github.event.action == 'synchronize' + runs-on: ubuntu-latest + steps: + - name: Remove preview label on every new push (ยง10 one-label-one-deploy) + run: gh pr edit '${{ github.event.pull_request.number }}' --remove-label preview --repo '${{ github.repository }}' || true + env: { GH_TOKEN: "${{ github.token }}" } + record-approval: + if: github.event.action == 'labeled' && github.event.label.name == 'preview' + runs-on: ubuntu-latest + steps: + - name: Verify labeler permission; record approval on exact head SHA + env: + GH_TOKEN: "${{ github.token }}" + SENDER: "${{ github.event.sender.login }}" + run: | + set -eu + perm=$(gh api "repos/${{ github.repository }}/collaborators/$SENDER/permission" --jq .permission) + case "$perm" in admin|write|maintain) ;; *) + gh pr edit '${{ github.event.pull_request.number }}' --remove-label preview --repo '${{ github.repository }}' + echo "::error::labeler lacks write permission โ label removed"; exit 1;; + esac + sha='${{ github.event.pull_request.head.sha }}' + gh api -X POST "repos/${{ github.repository }}/statuses/$sha" \ + -f state=success -f context=docs-preview-approved \ + -f description="preview approved by $SENDER" diff --git a/.github/workflows/docs-preview-build.yml b/.github/workflows/docs-preview-build.yml new file mode 100644 index 00000000..29682979 --- /dev/null +++ b/.github/workflows/docs-preview-build.yml @@ -0,0 +1,26 @@ +name: Docs preview build +on: + pull_request: + types: [opened, synchronize, reopened, labeled] + paths: ["docs/**", "workers/versions.json"] +permissions: + contents: read +jobs: + build: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + with: + persist-credentials: false # untrusted PR code runs in this job; token is read-only but no reason to leave it on disk + - name: Build HTML (no PDF in previews, ยง10) + run: | + pip install -r requirements.txt + cd docs && DOCS_VERSION_SLUG=rolling make html + - name: Pagefind + package + run: | + mkdir -p out/en/rolling && cp -r docs/_build/html/. out/en/rolling/ + npx --yes pagefind --site out/en/rolling + echo '${{ github.event.pull_request.number }}' > out/PR_NUMBER + echo '${{ github.event.pull_request.head.sha }}' > out/HEAD_SHA + - uses: actions/upload-artifact@v4 + with: { name: docs-preview, path: out/, retention-days: 3 } diff --git a/.github/workflows/docs-preview-cleanup.yml b/.github/workflows/docs-preview-cleanup.yml new file mode 100644 index 00000000..384b762f --- /dev/null +++ b/.github/workflows/docs-preview-cleanup.yml @@ -0,0 +1,43 @@ +name: Docs preview cleanup +on: + pull_request_target: + types: [closed] + schedule: + - cron: "17 3 * * *" # nightly sweep (ยง10) +permissions: + contents: read + pull-requests: read +jobs: + cleanup: + runs-on: ubuntu-latest + steps: + - name: Delete R2 prefixes for closed PRs (bulk, S3 API) + env: + AWS_ACCESS_KEY_ID: ${{ secrets.R2_ACCESS_KEY_ID }} + AWS_SECRET_ACCESS_KEY: ${{ secrets.R2_SECRET_ACCESS_KEY }} + AWS_DEFAULT_REGION: auto + R2_ENDPOINT: https://${{ secrets.CLOUDFLARE_ACCOUNT_ID }}.r2.cloudflarestorage.com + GH_TOKEN: ${{ github.token }} + run: | + set -euo pipefail + if [ "${{ github.event_name }}" = "pull_request_target" ]; then + aws s3 rm "s3://vyos-docs-previews/pr-${{ github.event.pull_request.number }}/" \ + --recursive --endpoint-url "$R2_ENDPOINT" --only-show-errors + else + # nightly: every prefix whose PR is closed. Capture-first (not pipe-into-while): + # pipefail + set -e fail the job when the listing itself fails (a piped while + # would swallow it via empty stdin), and the per-prefix failure flag survives + # (a piped while runs in a subshell, so `failed=1` there would be lost). + prefixes=$(aws s3api list-objects-v2 --bucket vyos-docs-previews --delimiter / \ + --endpoint-url "$R2_ENDPOINT" --query 'CommonPrefixes[].Prefix' --output text \ + | tr '\t' '\n' | sed -n 's|^pr-\([0-9]*\)/$|\1|p') + failed=0 + while read -r n; do + [ -n "$n" ] || continue + state=$(gh pr view "$n" --repo '${{ github.repository }}' --json state --jq .state || echo GONE) + [ "$state" = "OPEN" ] || aws s3 rm "s3://vyos-docs-previews/pr-$n/" \ + --recursive --endpoint-url "$R2_ENDPOINT" --only-show-errors \ + || { echo "::error::SWEEP-FAIL: pr-$n/ not deleted"; failed=1; } + done <<< "$prefixes" + exit "$failed" + fi diff --git a/.github/workflows/docs-preview-deploy.yml b/.github/workflows/docs-preview-deploy.yml new file mode 100644 index 00000000..3e3de0d3 --- /dev/null +++ b/.github/workflows/docs-preview-deploy.yml @@ -0,0 +1,86 @@ +name: Docs preview deploy +on: + workflow_run: + workflows: ["Docs preview build"] + types: [completed] +permissions: + contents: read + pull-requests: write + issues: write # label removal + PR comment go through the issues API + statuses: read + actions: read # cross-run artifact download +jobs: + deploy: + if: github.event.workflow_run.conclusion == 'success' + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 # BASE repo code only โ never PR head (ยง10) + - uses: actions/download-artifact@v4 + with: + name: docs-preview + path: out + run-id: ${{ github.event.workflow_run.id }} + github-token: ${{ github.token }} + - name: Authorization gate (approval record + SHA match, ยง10) + id: gate + env: { GH_TOKEN: "${{ github.token }}" } + run: | + set -eu + # TRUST MODEL (ยง10): out/PR_NUMBER + out/HEAD_SHA come from the UNPRIVILEGED + # run and are attacker-controlled. Trusted anchors: the workflow_run event's + # head_sha, and the approval record (commit status) written by the privileged + # approve workflow. The artifact is accepted only when all three agree. + event_sha='${{ github.event.workflow_run.head_sha }}' + art_sha=$(cat out/HEAD_SHA) + [ "$art_sha" = "$event_sha" ] || { echo "::error::artifact SHA ($art_sha) != workflow_run head_sha ($event_sha)"; exit 1; } + # Validate the artifact's claimed PR number BEFORE it's used anywhere โ it comes + # from the unprivileged run and must be a bare digit string. + art_pr=$(cat out/PR_NUMBER) + case "$art_pr" in + ''|*[!0-9]*) echo "::error::artifact PR_NUMBER is not a bare digit string: '$art_pr'"; exit 1 ;; + esac + # Resolve EVERY PR associated with the trusted head SHA โ a single commit can be + # the head of more than one open PR, so picking just .[0].number is not reliable. + # Never trust out/PR_NUMBER for authorization on its own: require it to be a + # MEMBER of this trusted list. The membership check happens in bash (below), not + # inside a jq filter, so the untrusted artifact value is never interpolated into + # jq code. + pr_list=$(gh api "repos/${{ github.repository }}/commits/$event_sha/pulls" --jq '.[].number') + [ -n "$pr_list" ] || { echo "no open PR for $event_sha"; exit 78; } + pr="" + while IFS= read -r candidate; do + if [ "$candidate" = "$art_pr" ]; then + pr="$candidate" + break + fi + done <<< "$pr_list" + [ -n "$pr" ] || { echo "::error::artifact PR number ($art_pr) is not among the trusted PRs for $event_sha ($pr_list)"; exit 1; } + state=$(gh pr view "$pr" --repo '${{ github.repository }}' --json state,labels) + echo "$state" | jq -e '.state == "OPEN"' >/dev/null || { echo "PR closed"; exit 78; } + echo "$state" | jq -e '.labels[].name | select(. == "preview")' >/dev/null || { echo "no preview label"; exit 78; } + gh api "repos/${{ github.repository }}/commits/$event_sha/statuses" \ + --jq '.[] | select(.context=="docs-preview-approved" and .state=="success")' | grep -q . \ + || { echo "::error::no approval record on $event_sha โ artifact predates/postdates labeling"; exit 1; } + echo "pr=$pr" >> "$GITHUB_OUTPUT" + - name: Upload to R2 prefix (bulk, S3 API โ ยง10 MIME via content-type detection) + env: + AWS_ACCESS_KEY_ID: ${{ secrets.R2_ACCESS_KEY_ID }} + AWS_SECRET_ACCESS_KEY: ${{ secrets.R2_SECRET_ACCESS_KEY }} + AWS_DEFAULT_REGION: auto + R2_ENDPOINT: https://${{ secrets.CLOUDFLARE_ACCOUNT_ID }}.r2.cloudflarestorage.com + run: | + set -eu + pr='${{ steps.gate.outputs.pr }}' + rm -f out/PR_NUMBER out/HEAD_SHA + # aws s3 sync = parallel bulk upload + per-file Content-Type guessing. + # Sequential per-file `wrangler r2 object put` was rejected: one Node + # process per file ร thousands of files โ runner-timeout territory. + aws s3 sync out "s3://vyos-docs-previews/pr-$pr/" \ + --endpoint-url "$R2_ENDPOINT" --delete --only-show-errors + - name: Consume label + comment URL (ยง10) + env: { GH_TOKEN: "${{ github.token }}" } + run: | + pr='${{ steps.gate.outputs.pr }}' + gh pr edit "$pr" --remove-label preview --repo '${{ github.repository }}' || true + gh pr comment "$pr" --repo '${{ github.repository }}' \ + --body "๐ Docs preview: https://docs-preview.vyos.io/pr-$pr/en/rolling/ (label consumed โ re-apply \`preview\` to redeploy after new pushes)" |
