Add incremental RST-to-MyST swap mechanism (circinus) (#1867)

* feat(swap-circinus): add incremental RST-to-MyST swap mechanism

Backport of the swap mechanism from feat/incremental-myst-swap onto
the circinus release branch. Built directly on top of origin/circinus,
so the underlying RST tree is circinus's (not current's).

Mechanism:
- scripts/import_myst.py — import md from myst/* with md- prefix
- scripts/swap_sources.py — rename md-{name}.md → {name}.md before
  Sphinx builds, restore after; writes _build/_swap_state.json and
  _build/_swap_exclude.txt
- docs/Makefile — html/dirhtml/pdf/livehtml all run swap → build →
  trap restore; explicit `swap` and `restore` targets too
- docs/conf.py — MyST extensions enabled; swap exclude_patterns
  loader; _prefer_webp builder hook so html prefers webp over png

Content (all from origin/myst/circinus):
- 253 md-prefixed pages alongside each {name}.rst counterpart
- 1 plain MyST-only page kept at canonical name (docs/copyright.md,
  no .rst counterpart)
- 182 .webp images added (circinus release previously had only
  PNG/JPG; this PR brings webp into circinus alongside the originals)
- docs/_swap.txt populated with all 253 stems → MyST is served by
  default; revert a page by removing its stem from _swap.txt

🤖 Generated by [robots](https://vyos.io)

* fix(ext): handle RST fallback in CmdInclude when _renderer absent

`cmdincludemd` is in `myst_fence_as_directive`, so MyST routes
fence blocks through `render_fence → render_restructuredtext →
MockRSTParser`. In that path `self.state` is a plain docutils Body
with no `_renderer`, crashing the RTD build.

Fall back to `nested_parse` when `_renderer` is unavailable so the
directive works in both MyST and RST/MockRSTParser contexts.

🤖 Generated by [robots](https://vyos.io)

* feat(conf): copy .md sources into HTML output for plain-text serving

Adds a build-finished hook that mirrors every .md file from the Sphinx
source tree into the HTML output directory verbatim, making unrendered
MyST sources accessible alongside HTML renders at the same URL path.

🤖 Generated by [robots](https://vyos.io)

* docs: address review feedback (backport from PR #1857)

🤖 Generated by [robots](https://vyos.io)

* docs: port .readthedocs.yml jobs and swap-script tests from PR #1857

Parity backport from PR #1857 (current) — was missing on circinus.

- .readthedocs.yml: add build.jobs.pre_build / post_build hooks that run
  scripts/swap_sources.py --swap before the Sphinx build and --restore
  after. Without this, the swap mechanism ships but never runs on RTD
  builds for this branch — the swap is a silent no-op.
- tests/test_import_myst.py, tests/test_swap_sources.py: tests for the
  swap scripts. The scripts are identical to current's, so the same
  tests apply. Travels with the branch so CI catches per-branch
  regressions if the scripts ever drift.

🤖 Generated by [robots](https://vyos.io)

* fix(review): add conftest, filter md- from copy, fix _prefer_webp builder list, trailing newline

Agent-Logs-Url: https://github.com/vyos/vyos-documentation/sessions/64b17346-8739-4346-b619-84438cddef21

Co-authored-by: andamasov <12631358+andamasov@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: andamasov <12631358+andamasov@users.noreply.github.com>

1 files changed, 204 insertions, 0 deletions

diff --git a/docs/contributing/md-debugging.md b/docs/contributing/md-debugging.md
new file mode 100644
index 00000000..d3b4b513
--- /dev/null
+++ b/docs/contributing/md-debugging.md
@@ -0,0 +1,204 @@
+---
+lastproofread: '2025-12-05'
+---
+
+(debugging)=
+
+# Debugging
+
+Two flags are available to help debug configuration scripts. Configuration
+loading issues manifest during boot, so these flags are passed as kernel boot
+parameters.
+
+## ISO image build
+
+If you have trouble compiling your own ISO image or debugging Jenkins issues,
+follow the steps at {ref}`iso_build_issues`.
+
+## System Startup
+
+Debug system startup by examining the configuration file loading from
+`/config/config.boot`. Extend the kernel command-line in the bootloader to
+enable this.
+
+### Kernel
+
+- `vyos-debug` - Add this parameter to the Linux boot line to produce
+  timing results for script execution during commit. If you see an unexpected
+  delay during manual or boot commit, this parameter helps identify bottlenecks.
+  The internal flag is `VYOS_DEBUG`, found in [vyatta-cfg]. Output is directed
+  to `/var/log/vyatta/cfg-stdout.log`.
+- `vyos-config-debug` - During development, coding errors can cause commit
+  failures on boot, potentially preventing CLI initialization. This kernel boot
+  parameter ensures access to the system as user `vyos` and logs a Python
+  stack trace to `/tmp/boot-config-trace`. The file is created only if the
+  configuration load fails.
+
+## Live System
+
+Several flags can be set to change VyOS behavior at runtime. Toggle these flags
+using environment variables or by creating files.
+
+For each feature, create a file called `vyos.feature.debug` to enable it.
+If a parameter is required, place it as the first line inside the file.
+
+Place the file in `/tmp` for one-time debugging (the file is removed on
+reboot) or in `/config` to persist permanently.
+
+For example, `/tmp/vyos.ifconfig.debug` can be created to enable
+interface debugging.
+
+You can also enable debugging using environment variables.
+The environment variable name follows the convention `VYOS_FEATURE_DEBUG`.
+
+For example, `export VYOS_IFCONFIG_DEBUG=""` in your vbash has the same effect
+as `touch /tmp/vyos.ifconfig.debug`.
+
+- `ifconfig` - Display all commands and their responses from the OS on
+  screen for inspection.
+- `command` - Display all commands and their responses from the OS on screen
+  for inspection.
+- `developer` - When a command fails, start a PDB post-mortem session instead
+  of showing a standard error message. This allows developers to debug issues
+  interactively. Because the debugger waits for input, it can prevent the router
+  from booting, so only enable this permanently on production systems if you are
+  ready for potential boot failures.
+- `log` - Send all commands used by VyOS to a log file for inspection. This
+  is useful in rare cases when you need to see what the OS is doing, including
+  during boot. The default file is `/tmp/full-log`, but you can change it.
+
+:::{note}
+To retrieve debug output on the command line, disable `vyos-configd`
+in addition. You can do this one-time with
+`sudo systemctl stop vyos-configd`
+or permanently with `sudo systemctl disable vyos-configd`.
+:::
+
+### FRR
+
+Recent versions use the `vyos.frr` framework. The Python class is located in
+`vyos-1x:python/vyos/frr.py`. It includes an embedded debugger similar to the
+one in `vyos.ifconfig`.
+
+Enable debugging by running: `touch /tmp/vyos.frr.debug`
+
+### Debug Python code with PDB
+
+Sometimes it is useful to debug Python code interactively on the live system
+rather than in an IDE. You can do this using pdb.
+
+Assuming you want to debug a Python script called by an op-mode command, find
+the script by looking up the op-mode definitions, then edit it on the live
+system using vi:
+`vi /usr/libexec/vyos/op_mode/show_xyz.py`
+
+Insert the following statement right before the section where you want to
+investigate a problem (for example, a statement you see in a backtrace):
+`import pdb; pdb.set_trace()`
+
+Optionally, surround this statement with an `if` condition that triggers only
+for the conditions you are interested in.
+
+When you run `show xyz` and your condition triggers, you enter the Python
+debugger:
+
+```none
+> /usr/libexec/vyos/op_mode/show_nat_translations.py(109)process()
+-> rule_type = rule.get('type', '')
+(Pdb)
+```
+
+You can type `help` to get an overview of the available commands, and
+`help command` to get more information on each command.
+
+Common useful commands include:
+
+- examine variables using `pp(var)`
+- continue execution using `cont`
+- get a backtrace using `bt`
+
+### Config Migration Scripts
+
+Starting with VyOS 1.5, a new mechanism is used for config migration that
+improves migration performance. New migrators use only the new format with a
+`migration()` function.
+
+```python
+from vyos.configtree import ConfigTree
+base = ['vpn', 'ipsec']
+def migrate(config: ConfigTree) -> None:
+    if not config.exists(base):
+        # Nothing to do
+        return
+    # do your stuff here
+```
+
+New-style migration scripts can no longer run on their own. However, the new
+migration subsystem handler includes a test kit:
+
+```none
+vyos@vyos:~$ /usr/libexec/vyos/run-config-migration.py --help
+usage: run-config-migration.py [-h] [--test-script TEST_SCRIPT] [--output-file OUTPUT_FILE] [--force] config_file
+
+positional arguments:
+  config_file           configuration file to migrate
+
+options:
+  -h, --help            show this help message and exit
+  --test-script TEST_SCRIPT
+                        test named script
+  --output-file OUTPUT_FILE
+                        write to named output file instead of config file
+  --force               force run of all migration scripts
+```
+
+To test your migration, run:
+
+```none
+vyos@vyos:~$ /usr/libexec/vyos/run-config-migration.py --test-script /opt/vyatta/etc/config-migrate/migrate/quagga/11-to-12 --output-file /tmp/foo /tmp/static-route-basic
+vyos@vyos:~$ cat /tmp/foo
+```
+
+The file `/tmp/foo` contains the migrated configuration.
+
+### Configuration Error on System Boot
+
+Running the latest rolling releases sometimes exposes bugs due to edge cases
+missed in design. File these bugs via [Phabricator](https://vyos.dev/), but you can help narrow
+down the issue by following these steps:
+
+1. Log in to your VyOS system.
+2. Enter configuration mode: `configure`
+3. Reload your boot configuration: `load`
+
+You should see a Python backtrace that helps identify the issue. Attach it to
+the [Phabricator](https://vyos.dev/) task.
+
+### Boot Timing
+
+During the migration and rewrite of functionality from Perl to Python, system
+boot time increased significantly. You can analyze and graph boot time to see
+detailed call sequences during startup.
+
+This uses the `systemd-bootchart` package, which is installed by default on
+VyOS 1.3 (equuleus) and later. Configuration is versioned for comparable
+results. Refer to [bootchart.conf] for the configuration file.
+
+To enable boot time graphing, add the following to the kernel command line:
+`init=/usr/lib/systemd/systemd-bootchart`
+
+You can also make this permanent by editing `/boot/grub/grub.cfg`.
+
+## Priorities
+
+VyOS CLI depends heavily on priorities. Every CLI node has a corresponding
+`node.def` file and possibly an attached script. Nodes can have priorities,
+and on system bootup or any `commit` to the configuration, scripts execute
+from lowest to highest priority. This provides deterministic behavior.
+
+To debug priority issues or see script execution order, use the
+`/opt/vyatta/sbin/priority.pl` script, which lists the execution order of
+scripts.
+
+[bootchart.conf]: https://github.com/vyos/vyos-build/blob/current/data/live-build-config/includes.chroot/etc/systemd/bootchart.conf
+[vyatta-cfg]: https://github.com/vyos/vyatta-cfg