diff options
| author | Yuriy Andamasov <yuriy@vyos.io> | 2026-05-06 14:40:28 +0300 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2026-05-06 12:40:28 +0100 |
| commit | 4b36114e053ee11d0cb264a1e4cfe4692d78f194 (patch) | |
| tree | be4ecc665eb3f1d556a37e768eed14989fec57b6 /scripts | |
| parent | 21a554bd4f9156e41f1c73ba6b7223bb63b3a4ef (diff) | |
| download | vyos-documentation-4b36114e053ee11d0cb264a1e4cfe4692d78f194.tar.gz vyos-documentation-4b36114e053ee11d0cb264a1e4cfe4692d78f194.zip | |
Add incremental RST-to-MyST swap mechanism (#1857)
* feat: add swap_sources.py for incremental RST-to-MyST migration
Pre-build swap/restore script that renames md-{name}.md → {name}.md
before Sphinx builds and restores after. Includes state tracking,
exclude file generation, collision detection, and partial-failure
rollback. 10 tests cover all specified behaviors plus rollback path.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat: add import_myst.py for importing MyST files from myst/* branches
Adds scripts/import_myst.py with import_page, git_show, list_myst_files,
list_rst_files, and do_import. Imported files are written as md-{name}.md
alongside existing RST files; importing is decoupled from swap activation.
Adds tests/test_import_myst.py covering single-page write, identical-skip,
warn-on-different-without-force, force-overwrite, and nested-path creation.
All 5 tests pass on Python 3.9.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat: add MyST swap exclude patterns and directive config to conf.py
🤖 Generated by [robots](https://vyos.io)
* feat: add swap-wrapped rendering targets to Makefile
🤖 Generated by [robots](https://vyos.io)
* feat: add swap pre/post build hooks for ReadTheDocs
🤖 Generated by [robots](https://vyos.io)
* feat: add empty _swap.txt, remove atexit from swap script
The atexit handler in --swap mode caused immediate restore on process
exit, breaking standalone usage. Makefile trap and RTD post_build
handle restore reliably.
🤖 Generated by [robots](https://vyos.io)
* feat: activate quick-start as MyST canary via swap mechanism
Imports docs/md-quick-start.md from origin/myst/current and adds
quick-start to docs/_swap.txt. Validates the swap pipeline end-to-end
on one page: import_myst pulls the MD via git show, swap_sources
renames md-quick-start.md to quick-start.md, sphinx-build renders
quick-start.html with zero MD-specific warnings, and restore reverses
the rename cleanly.
🤖 Generated by [robots](https://vyos.io)
* feat: activate 106 visual-validated canaries via swap
Imports 105 MD files (plus quick-start already present) from
origin/myst/current and adds them to docs/_swap.txt. The selection
is the BackstopJS visual-passers cohort: pages with <5% rendered
diff vs the live RST docs at docs.vyos.io/en/latest/, filtered to
those with an RST counterpart on current and no cmdincludemd usage
(template-format reconciliation pending).
Local sphinx-build with all 106 swapped: succeeded with 100
warnings (vs 95 baseline). The 5 new warnings are all undefined
cross-reference labels, not build failures:
- contributing/development.md (missing 'coding-guidelines')
- operation/upgrade-recovery.md (3 missing 'how_it_works' /
'cancelling_recovery')
- vpp/configuration/dataplane/{buffers,memory,unix}.md (missing
'vpp_config_dataplane_*' labels)
Source list: ~/.claude/projects/-Users-vybot-GitHub-vyos-documentation/docs/2026-04-29-myst-conversion-audit/visual-passers-under-5pct.txt
BackstopJS report: claude/gifted-hertz-74b9f9 worktree
(visual-compare/), 2026-04-23 vs vyos--1838.org.readthedocs.build.
🤖 Generated by [robots](https://vyos.io)
* fix: re-import 4 canary md-*.md files with xref label fixes
Re-imports the dash-form-corrected versions of:
- contributing/md-development.md (added (coding-guidelines)= anchor)
- operation/md-upgrade-recovery.md (3 ref renames: how_it_works /
cancelling_recovery -> dash form)
- vpp/configuration/dataplane/md-buffers.md (vpp_config_dataplane_physmem
-> vpp-config-dataplane-physmem)
- vpp/configuration/dataplane/md-unix.md
(vpp_config_dataplane_interface_rx_mode
-> vpp-config-dataplane-interface-rx-mode)
Source: origin/myst/current commit 59fbe3ea. Verified locally: clean
swap-build no longer reports any of the 5 target labels (1 of 6 —
vpp-config-hugepages — remains because system.md isn't in the canary
swap list; that anchor lives there).
🤖 Generated by [robots](https://vyos.io)
* fix: re-add 4 canary md-*.md files deleted by 242b334a
Commit 242b334a accidentally staged deletions instead of modifications
because the working tree had unprefixed *.md files left over from an
incomplete swap-restore cycle. Re-imports the same 4 files from
origin/myst/current with the xref label fixes applied:
- contributing/md-development.md — (coding-guidelines)= anchor
- operation/md-upgrade-recovery.md — how_it_works → how-it-works,
cancelling_recovery → cancelling-recovery
- vpp/configuration/dataplane/md-buffers.md — vpp_config_dataplane_physmem
→ vpp-config-dataplane-physmem
- vpp/configuration/dataplane/md-unix.md — vpp_config_dataplane_interface_rx_mode
→ vpp-config-dataplane-interface-rx-mode
Source: origin/myst/current commit 59fbe3ea.
🤖 Generated by [robots](https://vyos.io)
* fix: resolve remaining xref label gaps in swap-active build
Three small additions clear the cross-reference warnings tied to
underscore-vs-dash label form mismatches and the vpp-config-hugepages
reference that previously needed system.md in the canary set.
- system.rst: add .. _vpp-config-hugepages: alongside the existing
underscore label so memory.md references resolve regardless of
whether system.md is swap-active.
- md-lcp.md: add (vpp_config_dataplane_lcp_ignore-kernel-routes)=
alongside dash form (carries upstream from myst/current 079fa786).
- md-memory.md: add (vpp_config_dataplane_memory)= alongside dash
form (also from myst/current 079fa786).
Local clean swap-build with 106 canaries:
before: 305 warnings, 8 undefined-label entries in our scope
after: 300 warnings, 0 undefined-label entries in our scope
Remaining undefined-label warnings (release-notes, prepare_commit)
are in documentation.rst and unrelated to the canary swap mechanism.
🤖 Generated by [robots](https://vyos.io)
* fix: re-add md-lcp.md and md-memory.md (deleted by 870c9e7e)
Same disaster pattern as 242b334a: a swap-restore cycle left
unprefixed *.md files in the working tree, and the subsequent
git add staged deletions instead of modifications. Restoring the
two affected md-*.md files from origin/myst/current 079fa786
(which has the dual underscore+dash anchors needed for the
swap-active build).
🤖 Generated by [robots](https://vyos.io)
* feat: expand canaries to 114; refresh 3 with cfgcmd body fix
Adds 8 new visual-validated canaries from the post-cfgcmd-fix
BackstopJS run (2026-04-29):
- configuration/policy/as-path-list
- configuration/policy/community-list
- configuration/policy/extcommunity-list
- configuration/policy/large-community-list
- configuration/policy/local-route
- configuration/policy/prefix-list
- configuration/service/salt-minion
- configuration/system/updates
Refreshes 3 existing canaries whose MD content changed via the
cfgcmd/opcmd single-line body fix on myst/current fc19ab5c:
- configuration/firewall/global-options
- configuration/firewall/groups
- configuration/policy/route
All 11 sourced from origin/myst/current. Net: 106 -> 114 canaries.
🤖 Generated by [robots](https://vyos.io)
* fix: re-import md-cloud-init.md (block 3 fix from myst/current)
🤖 Generated by [robots](https://vyos.io)
* feat(swap): import .md files and webp transition from myst/current
Selective import from origin/myst/current (cf9c9b34):
- Add/update 255 .md files (full MyST conversion plus webp ref updates)
- Delete 175 PNG/JPG from docs/_static/images (webp twins already present)
- Delete 5 autotest topology.png (webp twins already present)
Preserved on swap (untouched):
- All .rst files (incremental swap pattern)
- conf.py, _ext/, _include/*.txt, .gitignore
- 115 canary md-*.md files
- 7 superpowers/specs/*.md design docs
- Logos vyos-logo.png / vyos-logo-icon.png (referenced by conf.py)
🤖 Generated by [robots](https://vyos.io)
* chore(swap): remove canary md-*.md files and docs/superpowers
- Remove 115 canary md-*.md files (incremental swap helpers no longer needed)
- Remove 8 files under docs/superpowers (project planning/design docs that
shouldn't ship in the documentation tree)
🤖 Generated by [robots](https://vyos.io)
* docs: address Copilot review feedback on imported MyST pages
Fix issues flagged by Copilot review on PR #1857 (the same content lives
in myst/current as the canonical source):
Real bugs:
- site-2-site-cisco.md: replace curly quote (U+2019) with ASCII apostrophe
- rsa-keys.md: fix typo "key-pair nam>>" → "key-pair name>"
- vmware.md: lowercase admonition directive (:::{NOTE} → :::{note})
- vpp/configuration/nat/index.md: remove blank line inside {include} fence
Grammar:
- vpp/configuration/interfaces/loopback.md: "bounded" → "bound"
- vpp/configuration/sflow.md: "VyOS support" → "VyOS supports"
- vpp/requirements.md: "bypass" → "bypasses"
- vpp/configuration/dataplane/interface.md: "configures" → "configure"
CI linter (IP addresses):
- nmp.md: wrap 8.8.8.8 example with stop/start_vyoslinter
- lac-lns.md: wrap LNS config block (contains 8.8.8.8)
- wan-load-balancing.md: wrap whole file (illustrative non-RFC IPs)
- policy/examples.md: replace 192.0.1.1 with RFC 5737 192.0.2.1
🤖 Generated by [robots](https://vyos.io)
* fix(swap): address Copilot review feedback on swap infrastructure
Category D — drop obsolete canary mechanism settings:
- conf.py: remove '**/md-*.md' from exclude_patterns (no canaries left)
- Makefile: replace malformed '*/_build/*' with '$(BUILDDIR)/**' and drop
the '*/md-*' ignore (canary files no longer exist)
Category C — script robustness:
- import_myst.py:
* list_myst_files() now raises SystemExit on git ls-tree failure instead
of silently returning [] (would have masked typo'd --source refs)
* list_rst_files() skips _build/ when scanning for .rst stems
* import_page() rejects stems containing '..' or absolute paths and
re-checks that the resolved destination stays under docs_dir
* --dry-run uses a separate "would_import" counter; summary line now
distinguishes dry-run from actual imports
- swap_sources.py:
* parse_swap_list() reads with explicit encoding='utf-8'
* do_restore() validates state file version + entry shape before
renaming files; raises with actionable message on corruption
* State file reads/writes use explicit encoding='utf-8' throughout
_swap.txt:
- Wrap long comment line to satisfy 80-character doc-linter limit
🤖 Generated by [robots](https://vyos.io)
* refactor(swap): rename imported .md files to md- prefix for swap mechanism
Restore the canary file naming convention that swap_sources.py expects:
the imported MyST pages now live as docs/<dir>/md-<name>.md alongside
the existing docs/<dir>/<name>.rst, so swap_sources.py --swap can rename
them into place at build time.
- 254 .md files renamed (every page with a matching .rst counterpart)
- 2 MyST-only pages left at their final names (no .rst exists, no swap
needed): docs/copyright.md, docs/automation/terraform/terraformvyos.md
All 114 stems listed in docs/_swap.txt now have a corresponding
md-<name>.md source file ready to swap in.
🤖 Generated by [robots](https://vyos.io)
* docs: address CodeRabbit review feedback on imported MyST pages
Fix issues flagged by CodeRabbit on PR #1857. All issues are pre-existing
in the upstream RST docs and inherited by the MyST conversion.
Real bugs:
- inter-vrf-routing-vrf-lite.md: invalid IPv6 next-hop "2001:db8::*" →
"2001:db8::1"
- ipsec-pa-route-based.md: vendor mislabel "Cisco" → "Palo Alto"
(header on line 39 and "Monitoring on Cisco side" section heading)
- bgp-ipv6-unnumbered.md: AS number mismatch between configuration and
verification output for both routers (Router A: 65020 → 64496;
Router B: 65021 → 64499)
- qos.md: class 30 used "match ADDRESS20" instead of ADDRESS30 — broke
the documented pattern (classes 10/20/30 → ADDRESS10/20/30)
Security:
- OpenVPN_with_LDAP.md: redact full PEM private key material from the
three "set pki ... private key '...'" lines and from the embedded
OpenVPN client <key> block; replace with <REDACTED> / ...REDACTED...
placeholders. Public certificates retained.
🤖 Generated by [robots](https://vyos.io)
* feat(swap): default to serving MyST for all swapped pages
Replace the previously-curated 114-stem _swap.txt with the full set of
254 imported md-prefixed pages, so MD is served by default at build
time. To revert any specific page back to RST, remove its stem from
_swap.txt (or comment it out).
🤖 Generated by [robots](https://vyos.io)
* fix(ext): handle RST fallback in CmdInclude when _renderer absent
`cmdincludemd` is in `myst_fence_as_directive`, so MyST routes
fence blocks through `render_fence → render_restructuredtext →
MockRSTParser`. In that path `self.state` is a plain docutils Body
with no `_renderer`, crashing the build.
Fall back to `nested_parse` when `_renderer` is unavailable so the
directive works in both MyST and RST/MockRSTParser contexts.
🤖 Generated by [robots](https://vyos.io)
* feat(conf): copy .md sources into HTML output for plain-text serving
Adds a build-finished hook that mirrors every .md file from the Sphinx
source tree into the HTML output directory verbatim, making unrendered
MyST sources accessible alongside HTML renders at the same URL path.
🤖 Generated by [robots](https://vyos.io)
* docs: address review feedback from PR #1857
Fix conversion artifacts, typos, grammar errors, and technical
inaccuracies flagged by automated code review (Copilot + CodeRabbit).
Infrastructure: add root-level md-*.md exclusion to conf.py,
fix sphinx-autobuild ignore globs in Makefile.
Content: fix curly quotes, invalid Go panic() calls, shell quoting
in cURL examples, incorrect firewall command paths, typos across
22 documentation files, remove duplicate sections.
🤖 Generated by [robots](https://vyos.io)
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Diffstat (limited to 'scripts')
| -rw-r--r-- | scripts/import_myst.py | 270 | ||||
| -rw-r--r-- | scripts/swap_sources.py | 327 |
2 files changed, 597 insertions, 0 deletions
diff --git a/scripts/import_myst.py b/scripts/import_myst.py new file mode 100644 index 00000000..b9dd7898 --- /dev/null +++ b/scripts/import_myst.py @@ -0,0 +1,270 @@ +#!/usr/bin/env python3 +"""import_myst.py — Import converted MD files from myst/* branches. + +Usage +----- + python scripts/import_myst.py --source myst/current --all + python scripts/import_myst.py --source myst/current --pages quick-start configuration/firewall/zone + python scripts/import_myst.py --source myst/current --all --force + python scripts/import_myst.py --source myst/current --all --dry-run + +Each imported file is written as md-{name}.md alongside the existing RST +file. Importing does not activate a page — use swap_sources.py for that. +""" + +import argparse +import subprocess +import sys +import warnings +from pathlib import Path +from typing import Optional + + +# --------------------------------------------------------------------------- +# Constants +# --------------------------------------------------------------------------- + +REPO_ROOT = Path(__file__).resolve().parent.parent +DOCS_DIR = REPO_ROOT / "docs" + + +# --------------------------------------------------------------------------- +# Core helpers +# --------------------------------------------------------------------------- + +def git_show(ref: str, path: str) -> Optional[bytes]: + """Run `git show {ref}:{path}` and return raw bytes, or None on error.""" + result = subprocess.run( + ["git", "show", f"{ref}:{path}"], + capture_output=True, + cwd=REPO_ROOT, + ) + if result.returncode != 0: + return None + return result.stdout + + +def list_myst_files(source: str) -> list[str]: + """Return list of MD stems from `git ls-tree -r --name-only {source} -- docs/`. + + Stems are relative to docs/ with the .md extension stripped. + Only .md files are returned. + """ + result = subprocess.run( + ["git", "ls-tree", "-r", "--name-only", source, "--", "docs/"], + capture_output=True, + text=True, + cwd=REPO_ROOT, + ) + if result.returncode != 0: + raise SystemExit( + f"import_myst: git ls-tree {source!r} failed " + f"(exit {result.returncode}): {result.stderr.strip()}" + ) + + stems = [] + for line in result.stdout.splitlines(): + line = line.strip() + if not line.endswith(".md"): + continue + # Strip leading "docs/" prefix and .md extension + rel = line[len("docs/"):] if line.startswith("docs/") else line + stem = rel[:-3] # strip ".md" + stems.append(stem) + return stems + + +def list_rst_files(docs_dir: Path) -> set[str]: + """Return set of stems (relative to docs_dir, no extension) for all *.rst files.""" + build_dir = (docs_dir / "_build").resolve() + stems = set() + for rst in docs_dir.rglob("*.rst"): + try: + rst.resolve().relative_to(build_dir) + continue # inside _build/, skip + except ValueError: + pass + rel = rst.relative_to(docs_dir) + stem = str(rel)[:-4] # strip ".rst" + stems.add(stem) + return stems + + +def import_page(stem: str, md_content: bytes, docs_dir: Path, force: bool) -> bool: + """Write md-{name}.md for the given stem. + + - Creates parent dirs as needed. + - Skips (returns False) if identical content already exists. + - Without --force: warns and skips (returns False) if different content exists. + - With --force: overwrites (returns True). + - Returns True on successful write. + + Refuses (raises ValueError) if *stem* would resolve outside *docs_dir*. + """ + p = Path(stem) + if p.is_absolute() or ".." in p.parts: + raise ValueError(f"import_myst: refusing unsafe stem {stem!r}") + name = p.name + parent = p.parent + if str(parent) == ".": + target_dir = docs_dir + else: + target_dir = docs_dir / parent + + dest = target_dir / f"md-{name}.md" + docs_root = docs_dir.resolve() + if not dest.resolve().parent.is_relative_to(docs_root): + raise ValueError(f"import_myst: refusing stem {stem!r} — escapes docs/") + + if dest.exists(): + existing = dest.read_bytes() + if existing == md_content: + return False # identical — skip silently + if not force: + warnings.warn( + f"import_myst: {dest} exists with different content; " + f"use --force to overwrite", + stacklevel=2, + ) + return False + # force=True — fall through to write + + target_dir.mkdir(parents=True, exist_ok=True) + dest.write_bytes(md_content) + return True + + +# --------------------------------------------------------------------------- +# do_import +# --------------------------------------------------------------------------- + +def do_import( + source: str, + pages: Optional[list], + all_pages: bool, + force: bool, + dry_run: bool, + docs_dir: Path, +) -> None: + """Orchestrate import from source branch. + + If --all: list MD files from source, intersect with RST stems in docs_dir, + import only those that have an RST counterpart. + If --pages: import only the listed stems. + """ + docs_dir = Path(docs_dir) + + if all_pages: + myst_stems = list_myst_files(source) + rst_stems = list_rst_files(docs_dir) + candidate_stems = [s for s in myst_stems if s in rst_stems] + else: + candidate_stems = list(pages or []) + + imported = 0 + skipped = 0 + would_import = 0 + + for stem in candidate_stems: + # Path inside the branch: docs/{stem}.md + ref_path = f"docs/{stem}.md" + content = git_show(source, ref_path) + if content is None: + print( + f"import_myst: WARNING — could not fetch {source}:{ref_path}", + file=sys.stderr, + ) + skipped += 1 + continue + + if dry_run: + print(f" (dry-run) would import {stem}") + would_import += 1 + continue + + wrote = import_page(stem, content, docs_dir, force) + if wrote: + imported += 1 + else: + skipped += 1 + + if dry_run: + print( + f"import_myst: (dry-run) would import {would_import}, " + f"skipped {skipped}.", + file=sys.stderr, + ) + else: + print( + f"import_myst: imported {imported}, skipped {skipped}.", + file=sys.stderr, + ) + + +# --------------------------------------------------------------------------- +# CLI +# --------------------------------------------------------------------------- + +def main(argv=None): + parser = argparse.ArgumentParser( + description="Import MyST MD files from a myst/* branch with md- prefix." + ) + parser.add_argument( + "--source", + required=True, + help="Git ref to import from (e.g. myst/current)", + ) + + pages_group = parser.add_mutually_exclusive_group(required=True) + pages_group.add_argument( + "--pages", + nargs="+", + metavar="STEM", + help="Specific page stems to import (e.g. quick-start configuration/firewall/zone)", + ) + pages_group.add_argument( + "--all", + action="store_true", + dest="all_pages", + help="Import all MD files that have a matching RST counterpart", + ) + + parser.add_argument( + "--force", + action="store_true", + help="Overwrite existing md- files even if content differs", + ) + parser.add_argument( + "--dry-run", + action="store_true", + help="Print what would be imported without writing files", + ) + parser.add_argument( + "--docs-dir", + type=Path, + default=None, + help="Path to docs/ directory (default: auto-detected from script location)", + ) + + args = parser.parse_args(argv) + + if args.docs_dir is None: + docs_dir = DOCS_DIR + else: + docs_dir = args.docs_dir.resolve() + + if not docs_dir.is_dir(): + parser.error(f"docs directory not found: {docs_dir}") + + do_import( + source=args.source, + pages=args.pages, + all_pages=args.all_pages, + force=args.force, + dry_run=args.dry_run, + docs_dir=docs_dir, + ) + + +if __name__ == "__main__": + main() diff --git a/scripts/swap_sources.py b/scripts/swap_sources.py new file mode 100644 index 00000000..683d5ef9 --- /dev/null +++ b/scripts/swap_sources.py @@ -0,0 +1,327 @@ +#!/usr/bin/env python3 +"""swap_sources.py — Pre-build swap/restore for incremental RST-to-MyST migration. + +Usage +----- + python scripts/swap_sources.py --swap # rename md-{name}.md → {name}.md + python scripts/swap_sources.py --restore # undo renames + python scripts/swap_sources.py --dry-run # show what would be swapped + python scripts/swap_sources.py --status # show current swap state + python scripts/swap_sources.py --swap --docs-dir /path/to/docs + +Swap list +--------- + docs/_swap.txt — one stem per line, relative to docs/, comments (#) skipped. + Example: + # migrated pages + configuration/firewall/zone + quick-start + +State files (in docs/_build/, gitignored) +----------------------------------------- + _swap_state.json — records completed renames + _swap_exclude.txt — RST paths to pass to Sphinx exclude_patterns +""" + +import argparse +import json +import os +import sys +from pathlib import Path + + +# --------------------------------------------------------------------------- +# Constants +# --------------------------------------------------------------------------- + +STATE_FILE = "_swap_state.json" +EXCLUDE_FILE = "_swap_exclude.txt" +SWAP_LIST_FILE = "_swap.txt" +STATE_VERSION = 1 + + +# --------------------------------------------------------------------------- +# Core helpers +# --------------------------------------------------------------------------- + +def parse_swap_list(path: Path) -> list: + """Read *path* and return a list of stems (comments/blanks skipped). + + Returns [] if the file does not exist. + """ + if not path.exists(): + return [] + stems = [] + for line in path.read_text(encoding="utf-8").splitlines(): + stripped = line.strip() + if stripped and not stripped.startswith("#"): + stems.append(stripped) + return stems + + +def _build_dir(docs_dir: Path) -> Path: + build = docs_dir / "_build" + build.mkdir(parents=True, exist_ok=True) + return build + + +def _state_path(docs_dir: Path) -> Path: + return docs_dir / "_build" / STATE_FILE + + +def _exclude_path(docs_dir: Path) -> Path: + return docs_dir / "_build" / EXCLUDE_FILE + + +def _resolve_stem_paths(docs_dir: Path, stem: str): + """Return (md_from, md_to, rst_path) as absolute Paths for a given stem.""" + p = Path(stem) + name = p.name + parent = p.parent # may be PosixPath('.') + if str(parent) == ".": + target_dir = docs_dir + else: + target_dir = docs_dir / parent + + md_from = target_dir / f"md-{name}.md" # source: md-prefixed + md_to = target_dir / f"{name}.md" # destination: plain .md + rst_path = target_dir / f"{name}.rst" # RST that will be excluded + return md_from, md_to, rst_path + + +# --------------------------------------------------------------------------- +# do_swap +# --------------------------------------------------------------------------- + +def do_swap(docs_dir: Path) -> None: + """Rename md-{name}.md → {name}.md for each stem in _swap.txt. + + Validates collisions, records state, writes exclude file. + Rolls back all completed renames on any failure. + """ + docs_dir = Path(docs_dir) + swap_list = parse_swap_list(docs_dir / SWAP_LIST_FILE) + if not swap_list: + print("swap_sources: no stems in swap list, nothing to do.", file=sys.stderr) + return + + # Stale state: warn and auto-restore first + state_path = _state_path(docs_dir) + if state_path.exists(): + print( + "swap_sources: WARNING — stale swap state detected. Auto-restoring before new swap.", + file=sys.stderr, + ) + do_restore(docs_dir) + + # Validate all stems before touching the filesystem + planned = [] # list of (md_from, md_to, rst_path, stem) + for stem in swap_list: + md_from, md_to, rst_path = _resolve_stem_paths(docs_dir, stem) + + if not md_from.exists(): + print( + f"swap_sources: skipping {stem!r} — md- source file not found: {md_from}", + file=sys.stderr, + ) + continue + + if not rst_path.exists(): + print( + f"swap_sources: skipping {stem!r} — RST file not found: {rst_path}", + file=sys.stderr, + ) + continue + + if md_to.exists(): + raise RuntimeError( + f"collision: both RST and unprefixed .md exist for stem {stem!r}. " + f"Remove {md_to} before swapping." + ) + + planned.append((md_from, md_to, rst_path, stem)) + + # Execute renames with rollback on failure + completed = [] # list of (md_from, md_to) that succeeded + try: + for md_from, md_to, rst_path, stem in planned: + os.rename(md_from, md_to) + completed.append((md_from, md_to)) + except Exception as exc: + # Rollback completed renames in reverse order + for from_path, to_path in reversed(completed): + try: + os.rename(to_path, from_path) + except Exception as rollback_exc: + print( + f"swap_sources: ROLLBACK ERROR for {to_path}: {rollback_exc}", + file=sys.stderr, + ) + raise RuntimeError(f"swap_sources: rename failed, rolled back. Cause: {exc}") from exc + + if not completed: + return + + # Build state and exclude data using relative paths (relative to docs_dir) + _build_dir(docs_dir) + swaps = [] + exclude_lines = [] + for md_from, md_to, rst_path, stem in planned: + rel_from = md_from.relative_to(docs_dir) + rel_to = md_to.relative_to(docs_dir) + rel_rst = rst_path.relative_to(docs_dir) + swaps.append({ + "stem": stem, + "md_from": str(rel_from), + "md_to": str(rel_to), + "rst_excluded": str(rel_rst), + }) + exclude_lines.append(str(rel_rst)) + + state = {"version": STATE_VERSION, "swaps": swaps} + state_path.write_text(json.dumps(state, indent=2), encoding="utf-8") + _exclude_path(docs_dir).write_text("\n".join(exclude_lines) + "\n", encoding="utf-8") + + print(f"swap_sources: swapped {len(completed)} file(s).", file=sys.stderr) + + +# --------------------------------------------------------------------------- +# do_restore +# --------------------------------------------------------------------------- + +def do_restore(docs_dir: Path) -> None: + """Undo all renames recorded in _swap_state.json (reverse order).""" + docs_dir = Path(docs_dir) + state_path = _state_path(docs_dir) + + if not state_path.exists(): + return # no-op + + try: + state = json.loads(state_path.read_text(encoding="utf-8")) + except (json.JSONDecodeError, OSError) as exc: + raise RuntimeError( + f"swap_sources: cannot read state file {state_path}: {exc}. " + f"Inspect or delete it manually." + ) from exc + + version = state.get("version") + if version != STATE_VERSION: + raise RuntimeError( + f"swap_sources: state file version {version!r} does not match " + f"expected {STATE_VERSION!r}. Inspect {state_path} manually." + ) + + swaps = state.get("swaps", []) + + for entry in reversed(swaps): + if not isinstance(entry, dict) or "md_from" not in entry or "md_to" not in entry: + raise RuntimeError( + f"swap_sources: malformed entry in {state_path}: {entry!r}. " + f"Inspect manually." + ) + md_from = docs_dir / entry["md_from"] # original source (md-prefixed) + md_to = docs_dir / entry["md_to"] # current location (plain .md) + if md_to.exists(): + os.rename(md_to, md_from) + else: + print( + f"swap_sources: WARNING — expected swapped file not found: {md_to}", + file=sys.stderr, + ) + + # Clean up state and exclude files + state_path.unlink(missing_ok=True) + exclude_path = _exclude_path(docs_dir) + exclude_path.unlink(missing_ok=True) + + print(f"swap_sources: restored {len(swaps)} file(s).", file=sys.stderr) + + +# --------------------------------------------------------------------------- +# do_dry_run / do_status +# --------------------------------------------------------------------------- + +def do_dry_run(docs_dir: Path) -> None: + """Print what would be swapped without making any changes.""" + docs_dir = Path(docs_dir) + swap_list = parse_swap_list(docs_dir / SWAP_LIST_FILE) + if not swap_list: + print("(dry-run) No stems in swap list.") + return + + for stem in swap_list: + md_from, md_to, rst_path = _resolve_stem_paths(docs_dir, stem) + issues = [] + if not md_from.exists(): + issues.append("md- source missing") + if not rst_path.exists(): + issues.append("RST missing") + if md_to.exists(): + issues.append("COLLISION: plain .md already exists") + if issues: + print(f" SKIP {stem}: {', '.join(issues)}") + else: + print(f" SWAP {md_from.name} → {md_to.name} (in {md_from.parent.relative_to(docs_dir) if md_from.parent != docs_dir else '.'})") + + +def do_status(docs_dir: Path) -> None: + """Show current swap state.""" + docs_dir = Path(docs_dir) + state_path = _state_path(docs_dir) + if not state_path.exists(): + print("swap_sources: no active swap state.") + return + + state = json.loads(state_path.read_text(encoding="utf-8")) + swaps = state.get("swaps", []) + print(f"swap_sources: {len(swaps)} file(s) currently swapped:") + for entry in swaps: + print(f" {entry['md_from']} → {entry['md_to']} (excludes {entry['rst_excluded']})") + + +# --------------------------------------------------------------------------- +# CLI +# --------------------------------------------------------------------------- + +def main(argv=None): + parser = argparse.ArgumentParser( + description="Swap md-prefixed MyST files in place of RST files before Sphinx builds." + ) + group = parser.add_mutually_exclusive_group(required=True) + group.add_argument("--swap", action="store_true", help="Perform swap") + group.add_argument("--restore", action="store_true", help="Restore original files") + group.add_argument("--dry-run", action="store_true", help="Show what would be swapped") + group.add_argument("--status", action="store_true", help="Show current swap state") + + parser.add_argument( + "--docs-dir", + type=Path, + default=None, + help="Path to docs/ directory (default: auto-detected from script location)", + ) + + args = parser.parse_args(argv) + + if args.docs_dir is None: + # Default: repo_root/docs where repo_root = parent of scripts/ + repo_root = Path(__file__).resolve().parent.parent + docs_dir = repo_root / "docs" + else: + docs_dir = args.docs_dir.resolve() + + if not docs_dir.is_dir(): + parser.error(f"docs directory not found: {docs_dir}") + + if args.swap: + do_swap(docs_dir) + elif args.restore: + do_restore(docs_dir) + elif args.dry_run: + do_dry_run(docs_dir) + elif args.status: + do_status(docs_dir) + + +if __name__ == "__main__": + main() |
