Add incremental RST-to-MyST swap mechanism (sagitta) (#1868)

* feat(swap-sagitta): add incremental RST-to-MyST swap mechanism

Backport of the swap mechanism from feat/incremental-myst-swap onto
the sagitta release branch. Built directly on top of origin/sagitta,
so the underlying RST tree is sagitta's (not current's).

Mechanism:
- scripts/import_myst.py — import md from myst/* with md- prefix
- scripts/swap_sources.py — rename md-{name}.md → {name}.md before
  Sphinx builds, restore after; writes _build/_swap_state.json and
  _build/_swap_exclude.txt
- docs/Makefile — html/dirhtml/pdf/livehtml all run swap → build →
  trap restore; explicit `swap` and `restore` targets too
- docs/conf.py — MyST extensions enabled; swap exclude_patterns
  loader; _prefer_webp builder hook so html prefers webp over png

Content:
- 202 md-prefixed pages from origin/myst/sagitta (md-{name}.md
  alongside each {name}.rst counterpart)
- 1 plain MyST-only page from myst/sagitta where no .rst exists
  (already at canonical name on sagitta: docs/copyright.md)
- 240 .webp images from myst/sagitta (added alongside the existing
  PNG/JPG so RST builds keep their assets)
- docs/_swap.txt populated with all 202 stems → MyST is served by
  default, revert a page by removing its stem from _swap.txt

🤖 Generated by [robots](https://vyos.io)

* feat(conf): copy .md sources into HTML output for plain-text serving

Adds a build-finished hook that mirrors every .md file from the Sphinx
source tree into the HTML output directory verbatim, making unrendered
MyST sources accessible alongside HTML renders at the same URL path.

🤖 Generated by [robots](https://vyos.io)

* docs: address review feedback (backport from PR #1857)

Fix conversion artifacts, typos, and technical inaccuracies applicable
to the sagitta branch: curly quotes, typos (deamonless, cammans,
amdifferent, trough), incorrect firewall command paths, missing closing
brace in zone-policy, peer name inconsistencies, hardcoded passwords
replaced with vault references, and md-*.md exclusion in conf.py.

🤖 Generated by [robots](https://vyos.io)

* docs: port .readthedocs.yml jobs, _ext/vyos.py fallback and swap-script tests from PR #1857

Parity backport from PR #1857 (current) — three pieces were missing on
sagitta.

- .readthedocs.yml: add build.jobs.pre_build / post_build hooks that run
  scripts/swap_sources.py --swap before the Sphinx build and --restore
  after. Without this, the swap mechanism ships but never runs on RTD
  builds for this branch — the swap is a silent no-op.
- docs/_ext/vyos.py: CmdInclude.run() now falls back to nested_parse()
  when self.state._renderer is not present. Required for cfgcmd /
  opcmd / cmdincludemd directives to render correctly when included
  from MyST pages (the swap mechanism's whole point). Sagitta-only
  delta on _ext/vyos.py (the path = str(path) line on 224) is
  intentionally untouched.
- tests/test_import_myst.py, tests/test_swap_sources.py: tests for the
  swap scripts. The scripts on this branch are byte-identical to
  current's, so the same tests apply. Travels with the branch so CI
  catches per-branch regressions if the scripts ever drift.

🤖 Generated by [robots](https://vyos.io)

* fix(conf): skip md-*.md staging files in _copy_md_sources

Agent-Logs-Url: https://github.com/vyos/vyos-documentation/sessions/919695a7-688d-41b9-89f0-540684625dbc

Co-authored-by: andamasov <12631358+andamasov@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: andamasov <12631358+andamasov@users.noreply.github.com>

1 files changed, 219 insertions, 0 deletions

diff --git a/docs/automation/md-command-scripting.md b/docs/automation/md-command-scripting.md
new file mode 100644
index 00000000..5c2d8f19
--- /dev/null
+++ b/docs/automation/md-command-scripting.md
@@ -0,0 +1,219 @@
+lastproofread  
+2023-01-16
+
+# Command Scripting
+
+VyOS supports executing configuration and operational commands non-interactively
+from shell scripts.
+
+To include VyOS specific functions and aliases you need to `source /opt/vyatta/etc/functions/script-template` files at the top of your script.
+
+``` none
+#!/bin/vbash
+source /opt/vyatta/etc/functions/script-template
+exit
+```
+
+## Run configuration commands
+
+Configuration commands are executed just like from a normal config session. For
+example, if you want to disable a BGP peer on VRRP transition to backup:
+
+``` none
+#!/bin/vbash
+source /opt/vyatta/etc/functions/script-template
+configure
+set protocols bgp system-as 65536
+set protocols bgp neighbor 192.168.2.1 shutdown
+commit
+exit
+```
+
+## Run operational commands
+
+Unlike a normal configuration session, all operational commands must be
+prepended with `run`, even if you haven't created a session with configure.
+
+``` none
+#!/bin/vbash
+source /opt/vyatta/etc/functions/script-template
+run show interfaces
+exit
+```
+
+## Run commands remotely
+
+Sometimes you simply want to execute a bunch of op-mode commands via SSH on
+a remote VyOS system.
+
+``` none
+ssh 192.0.2.1 'vbash -s' <<EOF
+source /opt/vyatta/etc/functions/script-template
+run show interfaces
+exit
+EOF
+```
+
+Will return:
+
+``` none
+Welcome to VyOS
+Codes: S - State, L - Link, u - Up, D - Down, A - Admin Down
+Interface        IP Address                        S/L  Description
+---------        ----------                        ---  -----------
+eth0             192.0.2.1/24                      u/u
+lo               127.0.0.1/8                       u/u
+                ::1/128
+```
+
+## Other script languages
+
+If you want to script the configs in a language other than bash you can have
+your script output commands and then source them in a bash script.
+
+Here is a simple example:
+
+``` python
+#!/usr/bin/env python3
+print("delete firewall group address-group somehosts")
+print("set firewall group address-group somehosts address '192.0.2.3'")
+print("set firewall group address-group somehosts address '203.0.113.55'")
+```
+
+``` none
+#!/bin/vbash
+source /opt/vyatta/etc/functions/script-template
+configure
+source < /config/scripts/setfirewallgroup.py
+commit
+```
+
+## Executing Configuration Scripts
+
+There is a pitfall when working with configuration scripts. It is tempting to
+call configuration scripts with "sudo" (i.e., temporary root permissions),
+because that's the common way on most Linux platforms to call system commands.
+
+On VyOS this will cause the following problem: After modifying the configuration
+via script like this once, it is not possible to manually modify the config
+anymore:
+
+``` none
+sudo ./myscript.sh # Modifies config
+configure
+set ... # Any configuration parameter
+```
+
+This will result in the following error message: `Set failed` If this happens,
+a reboot is required to be able to edit the config manually again.
+
+To avoid these problems, the proper way is to call a script with the
+`vyattacfg` group, e.g., by using the `sg` (switch group) command:
+
+``` none
+sg vyattacfg -c ./myscript.sh
+```
+
+To make sure that a script is not accidentally called without the `vyattacfg`
+group, the script can be safeguarded like this:
+
+``` none
+if [ "$(id -g -n)" != 'vyattacfg' ] ; then
+    exec sg vyattacfg -c "/bin/vbash $(readlink -f $0) $@"
+fi
+```
+
+## Executing pre-hooks/post-hooks Scripts
+
+VyOS has the ability to run custom scripts before and after each commit
+
+The default directories where your custom Scripts should be located are:
+
+``` none
+/config/scripts/commit/pre-hooks.d   - Directory with scripts that run before
+                                       each commit.
+
+/config/scripts/commit/post-hooks.d  - Directory with scripts that run after
+                                       each commit.
+```
+
+Scripts are run in alphabetical order. Their names must consist entirely of
+ASCII upper- and lower-case letters,ASCII digits, ASCII underscores, and
+ASCII minus-hyphens.No other characters are allowed.
+
+<div class="note">
+
+<div class="title">
+
+Note
+
+</div>
+
+Custom scripts are not executed with root privileges
+(Use sudo inside if this is necessary).
+
+</div>
+
+A simple example is shown below, where the ops command executed in
+the post-hook script is "show interfaces".
+
+``` none
+vyos@vyos# set interfaces ethernet eth1  address 192.0.2.3/24
+vyos@vyos# commit
+Codes: S - State, L - Link, u - Up, D - Down, A - Admin Down
+Interface        IP Address                        S/L  Description
+---------        ----------                        ---  -----------
+eth0             198.51.100.10/24                  u/u
+eth1             192.0.2.3/24                      u/u
+eth2             -                                 u/u
+eth3             -                                 u/u
+lo               203.0.113.5/24                    u/u
+```
+
+## Preconfig on boot
+
+The `/config/scripts/vyos-preconfig-bootup.script` script is called on boot
+before the VyOS configuration during boot process.
+
+Any modifications were done to work around unfixed bugs and implement
+enhancements that are not complete in the VyOS system can be placed here.
+
+The default file looks like this:
+
+``` none
+#!/bin/sh
+# This script is executed at boot time before VyOS configuration is applied.
+# Any modifications required to work around unfixed bugs or use
+# services not available through the VyOS CLI system can be placed here.
+```
+
+## Postconfig on boot
+
+The `/config/scripts/vyos-postconfig-bootup.script` script is called on boot
+after the VyOS configuration is fully applied.
+
+Any modifications were done to work around unfixed bugs and implement
+enhancements that are not complete in the VyOS system can be placed here.
+
+The default file looks like this:
+
+``` none
+#!/bin/sh
+# This script is executed at boot time after VyOS configuration is fully
+# applied. Any modifications required to work around unfixed bugs or use
+# services not available through the VyOS CLI system can be placed here.
+```
+
+<div class="hint">
+
+<div class="title">
+
+Hint
+
+</div>
+
+For configuration/upgrade management issues, modification of this
+script should be the last option. Always try to find solutions based on CLI
+commands first.
+
+</div>