From a783e56b774795c1f4bd8a6088dfcce96a307c5f Mon Sep 17 00:00:00 2001 From: Yuriy Andamasov Date: Fri, 25 Sep 2026 17:11:19 +0200 Subject: ci: port the Cloudflare Workers docs pipeline to circinus (slug 1.5) (#2208) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * ci: port Cloudflare Workers docs pipeline files to circinus (verbatim) Copies the branch-agnostic half of the docs.vyos.io Cloudflare Workers pipeline from `rolling` at 8cb568bf, byte-identical: - .github/workflows/docs-build.yml - workers/ (entire tree) - scripts/docs_gates/ - docker/im-convert.sh - docs/_static/js/version-picker.js, js/pagefind-wrapper.js, css/version-picker.css (new files, no circinus counterpart) - docs/_templates/breadcrumbs.html, searchbox.html (new files) docs-build.yml already triggers on push to [rolling, circinus, sagitta] and resolves `circinus` -> worker vyos-docs-v15-en / slug 1.5 from workers/matrix.json; the files simply did not exist on this branch, so slug 1.5 still serves the bootstrap placeholder. workers/versions.json + workers/matrix.json are deliberately identical across all three branches and must be kept in sync. Advances: IS-572 * ci: wire circinus docs build into the Cloudflare Workers pipeline Hand-merges the CF-specific hunks onto circinus's own docker/Dockerfile and docs/conf.py rather than clobbering them with rolling's versions — circinus keeps its own content-driven history in both files. docker/Dockerfile: - imagemagick + librsvg2-bin (sphinx.ext.imgconverter backend) and poppler-utils (pdfinfo, used by docs-build.yml's PDF page-count completeness check), installed --no-install-recommends - install docker/im-convert.sh as /usr/local/bin/im-convert docs/conf.py: - enable sphinx.ext.imgconverter + image_converter = 'im-convert' so the LaTeX/PDF builder stops silently dropping .webp/.svg images - register js/version-picker.js + css/version-picker.css unconditionally (degrades silently on ReadTheDocs) - _vyos_cf_build gate off the raw DOCS_VERSION_SLUG env var; only CF builds load js/pagefind-wrapper.js, and html_context['vyos_cf_build'] lets _templates/searchbox.html fall back to the stock Sphinx searchbox via the "!" bang-include on RTD The RTD path stays the default in both files: circinus continues building on ReadTheDocs until RTD sunset, and every CF feature activates only when DOCS_VERSION_SLUG is present. .readthedocs.yml is untouched. circinus keeps its own version/release/html_title/source_suffix and its hardcoded html_baseurl (its CF slug is also `1.5`, so rolling's DOCS_VERSION_SLUG/READTHEDOCS_VERSION resolution block is a no-op here and was deliberately not ported). Advances: IS-572 * ci: IS-572: re-sync ported Cloudflare Workers pipeline files with rolling The category-1 files in this port are byte-identical copies from `rolling`. `rolling` has since moved: [vyos-documentation#2209](https://github.com/vyos/vyos-documentation/pull/2209) merged as `3a1c6c30`, thirteen rounds of hardening on exactly these files. Re-take all 14 category-1 paths from `origin/rolling` via `git checkout origin/rolling -- `, so byte-identity holds by construction rather than by hand-editing: .github/workflows/docs-build.yml scripts/docs_gates/{gates,parity,smoke,test_gates,test_parity,test_smoke}.py workers/.gitignore workers/apex/src/{index,special,uagate}.ts workers/apex/test/{router,uagate}.test.ts workers/apex/ua-policy.json Thirteen of the fourteen carry [vyos-documentation#2209](https://github.com/vyos/vyos-documentation/pull/2209) exactly — the pre-change tree was byte-identical to `3a1c6c30^` for those paths. `workers/.gitignore` additionally picks up the one-line `test-results/` entry from [vyos-documentation#2212](https://github.com/vyos/vyos-documentation/pull/2212); inert on circinus, since only the deliberately-unported `apex-deploy.yml` writes that directory. Deliberate exclusions are unchanged: `docs-canary-qa.yml` (cron runs on the default branch only, so it is not ported even though [vyos-documentation#2209](https://github.com/vyos/vyos-documentation/pull/2209) touched it on `rolling`), `apex-deploy.yml`, and the `docs-preview-*` workflows. `docs/conf.py` stays hand-merged and circinus-specific, with its ReadTheDocs fallback intact. 🤖 Generated by [robots](https://vyos.io) --- docker/Dockerfile | 18 ++++++++++++++ docker/im-convert.sh | 68 ++++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 86 insertions(+) create mode 100644 docker/im-convert.sh (limited to 'docker') diff --git a/docker/Dockerfile b/docker/Dockerfile index fee5c91c..8598b808 100644 --- a/docker/Dockerfile +++ b/docker/Dockerfile @@ -27,6 +27,16 @@ RUN apt-get update && apt-get install -y \ curl \ dos2unix +# PDF-build toolchain extras, installed with --no-install-recommends (Trivy +# DS-0029) on a separate line — the texlive line above deliberately keeps +# recommends, which carry font packages LaTeX needs: +# - imagemagick + librsvg2-bin: sphinx.ext.imgconverter via docker/im-convert.sh +# - poppler-utils: pdfinfo for the docs-build workflow's page-count validation +RUN apt-get update && apt-get install -y --no-install-recommends \ + imagemagick \ + librsvg2-bin \ + poppler-utils + RUN pip3 install --break-system-packages \ Sphinx \ sphinx-rtd-theme \ @@ -54,4 +64,12 @@ COPY entrypoint.sh /usr/local/bin/entrypoint.sh # "no such file or directory" RUN dos2unix /usr/local/bin/entrypoint.sh +# sphinx.ext.imgconverter's `image_converter` command (docs/conf.py sets +# image_converter = 'im-convert'). Routes .svg sources to rsvg-convert +# directly, since Debian's imagemagick package is built --without-rsvg and +# its built-in SVG coder can't handle embedded base64 raster +# elements. See docker/im-convert.sh for the full rationale. +COPY im-convert.sh /usr/local/bin/im-convert +RUN dos2unix /usr/local/bin/im-convert && chmod +x /usr/local/bin/im-convert + ENTRYPOINT ["/usr/local/bin/entrypoint.sh"] diff --git a/docker/im-convert.sh b/docker/im-convert.sh new file mode 100644 index 00000000..7930b8be --- /dev/null +++ b/docker/im-convert.sh @@ -0,0 +1,68 @@ +#!/bin/bash +# Wrapper invoked by sphinx.ext.imgconverter as the `image_converter` command +# (docs/conf.py sets image_converter = 'im-convert'). +# +# Sphinx calls it as: im-convert "[0]" "" (the "[0]" frame-index +# suffix is ImageMagick syntax for "first frame/page", appended unconditionally +# by sphinx/ext/imgconverter.py). +# +# Debian's `imagemagick` package is built --without-rsvg (LGPL licensing), so +# its own SVG coder is a minimal libxml2-based renderer rather than a wrapper +# around librsvg. That built-in coder cannot handle SVGs with an embedded +# base64-encoded raster element (common in diagrams exported from +# draw.io/diagrams.net): it fails with +# "convert-im6.q16: unable to open image `image/png;base64,...'" +# because it tries to open the data-URI payload as if it were a file path. +# `rsvg-convert` (from librsvg2-bin) handles the same file correctly. +# +# Route .svg sources through rsvg-convert directly; everything else (webp, +# gif, pdf, ...) goes through ImageMagick's `convert` as before — webp support +# is compiled into this ImageMagick build, so that path needs no delegate. +# +# Output size is constrained: the docs pipeline's per-file size gate +# (scripts/docs_gates/gates.py) enforces the Cloudflare Workers 25 MiB +# asset cap on the built PDF, and naive full-resolution truecolor PNG +# conversion inflates it to ~183 MiB (the ~170 source .webp files are +# lossy-compressed; decoded to truecolor PNG they explode ~6x). The raster +# policy below (cap at 1000px on the long edge — only shrinking, never +# enlarging — plus non-dithered 128-color palette quantization and max PNG +# compression) brings the summed image payload to ~10 MiB (~15 MiB final +# PDF). Sphinx's imgconverter fixes the destination extension to .png (from +# the conversion rule's target mimetype) and pdflatex picks its decoder by +# extension, so switching photographic sources to JPEG is not available +# here — resolution + palette are the levers. 1000px at the PDF's ~6.3in +# text width is ~158 dpi; spot-checked legible on screenshots, diagrams, +# and photos alike. +set -euo pipefail + +# sphinx.ext.imgconverter's is_available() probes the converter with a +# single-arg call (`im-convert -version`) before ever doing a real +# conversion. Delegate straight to ImageMagick's own -version so the probe +# succeeds — is_available()'s result is cached at the class level for the +# whole build, so a failed probe here silently disables ALL conversion +# (webp included), not just SVG. +if [ "$#" -lt 2 ]; then + exec convert "$@" +fi + +src=$1 +dst=$2 + +# Strip ImageMagick's trailing frame-index suffix, e.g. +# "/path/to/file.svg[0]" -> "/path/to/file.svg". +plain_src=${src%\[*\]} + +case "$plain_src" in + *.svg|*.SVG) + exec rsvg-convert -o "$dst" "$plain_src" + ;; + *) + # See the size-cap rationale in the header comment. `1000x1000>` + # only shrinks images larger than 1000px on either edge; `+dither` + # disables dithering (IM6 semantics: -dither enables, +dither + # disables), which quantizes flat UI colors cleanly AND compresses + # far better than dithered output; `-quality 95` for PNG means + # zlib level 9 + adaptive row filtering. + exec convert "$src" -resize '1000x1000>' -strip +dither -colors 128 -quality 95 "$dst" + ;; +esac -- cgit v1.2.3