Add incremental RST-to-MyST swap mechanism (sagitta) (#1868)

* feat(swap-sagitta): add incremental RST-to-MyST swap mechanism

Backport of the swap mechanism from feat/incremental-myst-swap onto
the sagitta release branch. Built directly on top of origin/sagitta,
so the underlying RST tree is sagitta's (not current's).

Mechanism:
- scripts/import_myst.py — import md from myst/* with md- prefix
- scripts/swap_sources.py — rename md-{name}.md → {name}.md before
  Sphinx builds, restore after; writes _build/_swap_state.json and
  _build/_swap_exclude.txt
- docs/Makefile — html/dirhtml/pdf/livehtml all run swap → build →
  trap restore; explicit `swap` and `restore` targets too
- docs/conf.py — MyST extensions enabled; swap exclude_patterns
  loader; _prefer_webp builder hook so html prefers webp over png

Content:
- 202 md-prefixed pages from origin/myst/sagitta (md-{name}.md
  alongside each {name}.rst counterpart)
- 1 plain MyST-only page from myst/sagitta where no .rst exists
  (already at canonical name on sagitta: docs/copyright.md)
- 240 .webp images from myst/sagitta (added alongside the existing
  PNG/JPG so RST builds keep their assets)
- docs/_swap.txt populated with all 202 stems → MyST is served by
  default, revert a page by removing its stem from _swap.txt

🤖 Generated by [robots](https://vyos.io)

* feat(conf): copy .md sources into HTML output for plain-text serving

Adds a build-finished hook that mirrors every .md file from the Sphinx
source tree into the HTML output directory verbatim, making unrendered
MyST sources accessible alongside HTML renders at the same URL path.

🤖 Generated by [robots](https://vyos.io)

* docs: address review feedback (backport from PR #1857)

Fix conversion artifacts, typos, and technical inaccuracies applicable
to the sagitta branch: curly quotes, typos (deamonless, cammans,
amdifferent, trough), incorrect firewall command paths, missing closing
brace in zone-policy, peer name inconsistencies, hardcoded passwords
replaced with vault references, and md-*.md exclusion in conf.py.

🤖 Generated by [robots](https://vyos.io)

* docs: port .readthedocs.yml jobs, _ext/vyos.py fallback and swap-script tests from PR #1857

Parity backport from PR #1857 (current) — three pieces were missing on
sagitta.

- .readthedocs.yml: add build.jobs.pre_build / post_build hooks that run
  scripts/swap_sources.py --swap before the Sphinx build and --restore
  after. Without this, the swap mechanism ships but never runs on RTD
  builds for this branch — the swap is a silent no-op.
- docs/_ext/vyos.py: CmdInclude.run() now falls back to nested_parse()
  when self.state._renderer is not present. Required for cfgcmd /
  opcmd / cmdincludemd directives to render correctly when included
  from MyST pages (the swap mechanism's whole point). Sagitta-only
  delta on _ext/vyos.py (the path = str(path) line on 224) is
  intentionally untouched.
- tests/test_import_myst.py, tests/test_swap_sources.py: tests for the
  swap scripts. The scripts on this branch are byte-identical to
  current's, so the same tests apply. Travels with the branch so CI
  catches per-branch regressions if the scripts ever drift.

🤖 Generated by [robots](https://vyos.io)

* fix(conf): skip md-*.md staging files in _copy_md_sources

Agent-Logs-Url: https://github.com/vyos/vyos-documentation/sessions/919695a7-688d-41b9-89f0-540684625dbc

Co-authored-by: andamasov <12631358+andamasov@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: andamasov <12631358+andamasov@users.noreply.github.com>

5 files changed, 524 insertions, 0 deletions

diff --git a/docs/operation/md-boot-options.md b/docs/operation/md-boot-options.md
new file mode 100644
index 00000000..73a8f62f
--- /dev/null
+++ b/docs/operation/md-boot-options.md
@@ -0,0 +1,56 @@
+# Boot Options
+
+<div class="warning">
+
+<div class="title">
+
+Warning
+
+</div>
+
+This function may be highly disruptive.
+It may cause major service interruption, so make sure you really
+need it and verify your input carefully.
+
+</div>
+
+VyOS has several kernel command line options to modify the normal boot
+process.
+To add an option, select the desired image in GRUB menu at load
+time, press **e**, edit the first line, and press **Ctrl-x** to boot when
+ready.
+
+<img src="/_static/images/boot-options.webp" class="align-center" style="width:80.0%" alt="image" />
+
+## Specify custom config file
+
+Tells the system to use specified file instead of `/config/config.boot`.
+If specified file does not exist or is not readable, fall back to
+default config. No additional verification is performed, so make sure
+you specify a valid config file.
+
+``` none
+vyos-config=/path/to/file
+```
+
+To load the *factory default* config, use:
+
+``` none
+vyos-config=/opt/vyatta/etc/config.boot.default
+```
+
+## Disable specific boot process steps
+
+These options disable some boot steps. Make sure you understand the
+`boot process <boot-steps>` well before using them!
+
+<div class="glossary">
+
+no-vyos-migrate  
+Do not perform config migration.
+
+no-vyos-firewall  
+Do not initialize default firewall chains, renders any firewall
+configuration unusable.
+
+</div>
diff --git a/docs/operation/md-index.md b/docs/operation/md-index.md
new file mode 100644
index 00000000..eed21dcd
--- /dev/null
+++ b/docs/operation/md-index.md
@@ -0,0 +1,10 @@
+# Operation Mode
+
+<div class="toctree" maxdepth="1" includehidden="">
+
+information
+boot-options
+password-recovery
+raid
+
+</div>
diff --git a/docs/operation/md-information.md b/docs/operation/md-information.md
new file mode 100644
index 00000000..c6f663a5
--- /dev/null
+++ b/docs/operation/md-information.md
@@ -0,0 +1,171 @@
+lastproofread  
+2021-07-07
+
+# Information
+
+VyOS features a rich set of operational level commands to retrieve arbitrary
+information about your running system.
+
+## Hardware
+
+### USB
+
+In the past serial interface have been defined as ttySx and ttyUSBx where x was
+an instance number of the serial interface. It was discovered that from system
+boot to system boot the mapping of USB based serial interfaces will differ,
+depending which driver was loaded first by the operating system. This will
+become rather painful if you not only have serial interfaces for a console
+server connected but in addition also a serial backed `wwan-interface`.
+
+To overcome this issue and the fact that in almost 50% of all cheap USB to
+serial converters there is no serial number programmed, the USB to serial
+interface is now directly identified by the USB root bridge and bus it connects
+to. This somehow mimics the new network interface definitions we see in recent
+Linux distributions.
+
+For additional details you can refer to <https://vyos.dev/T2490>.
+
+<div class="opcmd">
+
+show hardware usb
+
+Retrieve a tree like representation of all connected USB devices.
+
+<div class="note">
+
+<div class="title">
+
+Note
+
+</div>
+
+If a device is unplugged and re-plugged it will receive a new
+Port, Dev, If identification.
+
+</div>
+
+``` none
+vyos@vyos:~$ show hardware usb
+/:  Bus 03.Port 1: Dev 1, Class=root_hub, Driver=ehci-pci/2p, 480M
+    |__ Port 1: Dev 2, If 0, Class=Hub, Driver=hub/4p, 480M
+        |__ Port 3: Dev 4, If 0, Class=Vendor Specific Class, Driver=qcserial, 480M
+        |__ Port 3: Dev 4, If 2, Class=Vendor Specific Class, Driver=qcserial, 480M
+        |__ Port 3: Dev 4, If 3, Class=Vendor Specific Class, Driver=qcserial, 480M
+        |__ Port 3: Dev 4, If 8, Class=Vendor Specific Class, Driver=qmi_wwan, 480M
+/:  Bus 02.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/2p, 5000M
+/:  Bus 01.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/2p, 480M
+    |__ Port 1: Dev 2, If 0, Class=Vendor Specific Class, Driver=pl2303, 12M
+    |__ Port 2: Dev 3, If 0, Class=Hub, Driver=hub/4p, 480M
+        |__ Port 4: Dev 5, If 2, Class=Vendor Specific Class, Driver=ftdi_sio, 480M
+        |__ Port 4: Dev 5, If 0, Class=Vendor Specific Class, Driver=ftdi_sio, 480M
+        |__ Port 4: Dev 5, If 3, Class=Vendor Specific Class, Driver=ftdi_sio, 480M
+        |__ Port 4: Dev 5, If 1, Class=Vendor Specific Class, Driver=ftdi_sio, 480M
+        |__ Port 3: Dev 4, If 0, Class=Hub, Driver=hub/4p, 480M
+            |__ Port 3: Dev 6, If 0, Class=Hub, Driver=hub/4p, 480M
+                |__ Port 4: Dev 8, If 2, Class=Vendor Specific Class, Driver=ftdi_sio, 480M
+                |__ Port 4: Dev 8, If 0, Class=Vendor Specific Class, Driver=ftdi_sio, 480M
+                |__ Port 4: Dev 8, If 3, Class=Vendor Specific Class, Driver=ftdi_sio, 480M
+                |__ Port 4: Dev 8, If 1, Class=Vendor Specific Class, Driver=ftdi_sio, 480M
+            |__ Port 4: Dev 7, If 3, Class=Vendor Specific Class, Driver=ftdi_sio, 480M
+            |__ Port 4: Dev 7, If 1, Class=Vendor Specific Class, Driver=ftdi_sio, 480M
+            |__ Port 4: Dev 7, If 2, Class=Vendor Specific Class, Driver=ftdi_sio, 480M
+            |__ Port 4: Dev 7, If 0, Class=Vendor Specific Class, Driver=ftdi_sio, 480M
+```
+
+</div>
+
+<div class="opcmd">
+
+show hardware usb serial
+
+Retrieve a list and description of all connected USB serial devices. The
+device name displayed, e.g. <span class="title-ref">usb0b2.4p1.0</span> can be directly used when accessing
+the serial console as console-server device.
+
+``` none
+vyos@vyos$ show hardware usb serial
+Device           Model               Vendor
+------           ------              ------
+usb0b1.3p1.0     MC7710              Sierra Wireless, Inc.
+usb0b1.3p1.2     MC7710              Sierra Wireless, Inc.
+usb0b1.3p1.3     MC7710              Sierra Wireless, Inc.
+usb0b1p1.0       USB-Serial_Controller_D Prolific Technology, Inc.
+usb0b2.3.3.4p1.0 Quad_RS232-HS       Future Technology Devices International, Ltd
+usb0b2.3.3.4p1.1 Quad_RS232-HS       Future Technology Devices International, Ltd
+usb0b2.3.3.4p1.2 Quad_RS232-HS       Future Technology Devices International, Ltd
+usb0b2.3.3.4p1.3 Quad_RS232-HS       Future Technology Devices International, Ltd
+usb0b2.3.4p1.0   Quad_RS232-HS       Future Technology Devices International, Ltd
+usb0b2.3.4p1.1   Quad_RS232-HS       Future Technology Devices International, Ltd
+usb0b2.3.4p1.2   Quad_RS232-HS       Future Technology Devices International, Ltd
+usb0b2.3.4p1.3   Quad_RS232-HS       Future Technology Devices International, Ltd
+usb0b2.4p1.0     Quad_RS232-HS       Future Technology Devices International, Ltd
+usb0b2.4p1.1     Quad_RS232-HS       Future Technology Devices International, Ltd
+usb0b2.4p1.2     Quad_RS232-HS       Future Technology Devices International, Ltd
+usb0b2.4p1.3     Quad_RS232-HS       Future Technology Devices International, Ltd
+```
+
+</div>
+
+## Version
+
+<div class="opcmd">
+
+show version
+
+Return the current running VyOS version and build information. This includes
+also the name of the release train which is `crux` on VyOS 1.2, `equuleus`
+on VyOS 1.3 and `sagitta` on VyOS 1.4.
+
+``` none
+vyos@vyos:~$ show version  
+
+Version:          VyOS 1.4-rolling-202106270801
+Release Train:    sagitta
+
+Built by:         autobuild@vyos.net
+Built on:         Sun 27 Jun 2021 09:50 UTC
+Build UUID:       ab43e735-edcb-405a-9f51-f16a1b104e52
+Build Commit ID:  f544d75eab758f
+
+Architecture:     x86_64
+Boot via:         installed image
+System type:      KVM guest
+
+Hardware vendor:  QEMU
+Hardware model:   Standard PC (i440FX + PIIX, 1996)
+Hardware S/N:     
+Hardware UUID:    Unknown
+
+Copyright:        VyOS maintainers and contributors
+```
+
+</div>
+
+<div class="opcmd">
+
+show version kernel
+
+Return version number of the Linux Kernel used in this release.
+
+``` none
+vyos@vyos:~$ show version kernel
+5.10.46-amd64-vyos
+```
+
+</div>
+
+<div class="opcmd">
+
+show version frr
+
+Return version number of FRR (Free Range Routing - <https://frrouting.org/>)
+used in this release. This is the routing control plane and a successor to GNU
+Zebra and Quagga.
+
+> ``` none
+> vyos@vyos:~$ show version frr
+> FRRouting 7.5.1-20210625-00-gf07d935a2 (vyos).
+> Copyright 1996-2005 Kunihiro Ishiguro, et al.
+> ```
+
+</div>
diff --git a/docs/operation/md-password-recovery.md b/docs/operation/md-password-recovery.md
new file mode 100644
index 00000000..295de05c
--- /dev/null
+++ b/docs/operation/md-password-recovery.md
@@ -0,0 +1,22 @@
+# Password Recovery
+
+Using the console, restart the VyOS router. The GRUB menu appears.
+Select the relevant option from the GRUB menu and press Enter.
+The option must start with “Lost password change.”
+
+<figure>
+<img src="/_static/images/password-recovery-01.webp" width="600" />
+</figure>
+
+The stand-alone user-password recovery tool starts running and prompts
+you to reset the local system user password.
+
+``` console
+Do you wish to reset the admin password? (y or n) 
+y
+Which admin account do you want to reset?[vyos]
+my_username
+Enter my_username password: 
+Retype my_username password: 
+System will reboot in 10 seconds...
+```
diff --git a/docs/operation/md-raid.md b/docs/operation/md-raid.md
new file mode 100644
index 00000000..2009779d
--- /dev/null
+++ b/docs/operation/md-raid.md
@@ -0,0 +1,265 @@
+# RAID-1
+
+A Redundant Array of Independent Disks (RAID) uses two or more hard disk drives
+to improve disk speed, store more data, and/or provide fault tolerance.
+There are several storage schemes possible in a RAID array, each offering a
+different combination of storage, reliability, and/or performance.
+The VyOS system supports a “RAID 1” deployment. RAID 1 allows two or more
+disks to mirror one another to provide system fault tolerance. In a RAID 1
+solution, every sector of one disk is duplicated onto every sector of all
+disks in the array. Provided even one disk in the RAID 1 set is operational,
+the system continues to run, even through disk replacement (provided that the
+hardware supports in-service replacement of drives).
+RAID 1 can be implemented using special hardware or it can be implemented in
+software. The VyOS system supports software RAID 1 on two disks.
+The VyOS implementation of RAID 1 allows the following:
+
+- Detection and reporting of disk failure
+- The ability to maintain system operation with one failed disk
+- The ability to boot the system with one failed disk
+- The ability to replace a failed disk and initiate re-mirroring
+- The ability to monitor the status of remirroring
+
+## Installation Implications
+
+The VyOS systems installation utility provides several options for installing
+to a RAID 1 set. You can:
+
+- Use the install system to create the RAID 1 set
+- Use the underlying Linux commands to create a RAID 1 set before running the
+  install system command.
+- Use a previously-created RAID 1 set.
+
+<div class="note">
+
+<div class="title">
+
+Note
+
+</div>
+
+Before a permanent installation, VyOS runs a live installation
+
+</div>
+
+## Configuration
+
+### Single disk, install as normal
+
+When the VyOS system is installed, it automatically detects the presence of two
+disks not currently part of a RAID array. In these cases, the VyOS
+installation utility automatically offers you the option of configuring RAID 1
+mirroring for the drives, with the following prompt.
+
+``` none
+Would you like to configure RAID 1 mirroring on them?
+```
+
+- If you do not want to configure RAID 1 mirroring, enter “No” at the prompt
+  and continue with installation in the normal way.
+
+### Empty 2+ Disk
+
+If VyOS system detect two identical disks that are not currently part of a
+RAID-1 set, the VyOS installation utility automatically offers you the option
+of configuring RAID 1 mirroring for the drives, with the following prompt.
+
+``` none
+Would you like to configure RAID 1 mirroring on them? 
+```
+
+1 - To create a new RAID 1 array, enter “Yes” at the prompt. If the system
+detects a filesystem on the partitions being used for RAID 1 it will prompt you
+to indicate whether you want to continue creating the RAID 1 array.
+
+``` none
+Continue creating array?
+```
+
+2 - To overwrite the old filesystem, enter “Yes”.
+
+3 - The system informs you that all data on both drives will be erased. You are
+prompted to confirm that you want to continue
+
+``` none
+Are you sure you want to do this?
+```
+
+4 - Enter “Yes” at the prompt to retain the current VyOS configuration once
+installation is complete. Enter “No” to delete the current VyOS
+configuration.
+
+``` none
+Would you like me to save the data on it before I delete it?
+```
+
+5 - Enter “Yes” at the prompt to retain the current VyOS configuration once
+installation is complete. Enter “No” to delete the current VyOS configuration.
+
+6 - Continue with installation in the normal way.
+
+### Present RAID-1
+
+When the VyOS software on a system with a RAID 1 set already configured,
+the installation utility will detect the array and will display the following
+prompt:
+
+``` none
+Would you like to use this one? 
+```
+
+1 - To break apart the current RAID 1 set, enter “No” at the prompt. The
+
+installation utility detects that there are two identical disks and offers you
+the option of configuring RAID 1 mirroring on them, displaying the following
+prompt:
+
+``` none
+Would you like to configure RAID 1 mirroring on them? 
+```
+
+2 - To decline to set up a new RAID 1 configuration on the disks, enter “No”
+at the prompt. The system prompts you to indicate which partition you would
+like the system installed on.
+
+``` none
+Which partition should I install the root on? [sda1]: 
+```
+
+3 - Enter the partition where you would like the system installed. The system
+then prompts you to indicate whether you want to save the old configuration
+data. This represents the current VyOS configuration.
+
+``` none
+Would you like me to save the data on it before I delete it? 
+```
+
+4 - Enter “Yes” at the prompt to retain the current VyOS configuration once
+installation is complete. Enter “No” to delete the current VyOS configuration.
+
+5 - Continue with installation in the normal way.
+
+### Detecting and Replacing a Failed RAID 1 Disk
+
+The VyOS system automatically detects a disk failure within a RAID 1 set and
+reports it to the system console. You can verify the failure by issuing the
+show raid command.
+
+To replace a bad disk within a RAID 1 set, perform the following steps:
+
+1 - Remove the failed disk from the RAID 1 set by issuing the following
+command:
+
+<div class="opcmd">
+
+delete raid \<RAID‐1‐device\> member \<disk‐partition\>
+
+where RAID-1-device is the name of the RAID 1 device (for example, md0) and
+disk-partition is the name of the failed disk partition (for example, sdb2).
+
+</div>
+
+2- Physically remove the failed disk from the system. If the drives are not
+hot-swappable, then you must shut down the system before removing the disk.
+
+3 - Replace the failed drive with a drive of the same size or larger.
+
+4 - Format the new disk for RAID 1 by issuing the following command:
+
+<div class="opcmd">
+
+format disk \<disk‐device1\> like \<disk‐device2\>
+
+where disk-device1 is the replacement disk (for example, sdb) and
+disk-device2 is the existing healthy disk (for example, sda).
+
+</div>
+
+5-Add the replacement disk to the RAID 1 set by issuing the following command:
+
+<div class="opcmd">
+
+add raid \<RAID‐1‐device\> member \<disk‐partition\>
+
+where RAID-1-device is the name of the RAID 1 device (for example, md0) and
+disk-partition is the name of the replacement disk partition
+(for example, sdb2).
+
+</div>
+
+## Operation
+
+This part introduces how to add a disk partition to a RAID-1 set initiates
+mirror synchronization, check and display information.
+
+<div class="opcmd">
+
+add raid \<RAID‐1‐device\> member \<disk‐partition\>
+
+Use this command to add a member disk partition to the RAID 1 set. Adding a
+disk partition to a RAID 1 set initiates mirror synchronization, where all
+data on the existing member partition is copied to the new partition.
+
+</div>
+
+<div class="opcmd">
+
+format disk \<disk‐device1\> like \<disk‐device2\>
+
+This command is typically used to prepare a disk to be added to a preexisting
+RAID 1 set (of which disk-device2 is already a member).
+
+</div>
+
+<div class="opcmd">
+
+show raid \<RAID‐1‐device\>
+
+shows output for show raid md0 as sdb1 is being added to the RAID 1
+set and is in the process of being resynchronized.
+
+``` none
+vyos@vyos:~$ show raid md0
+/dev/md0:
+      Version : 00.90
+Creation Time : Wed Oct 29 09:19:09 2008
+   Raid Level : raid1
+   Array Size : 1044800 (1020.48 MiB 1069.88 MB)
+Used Dev Size : 1044800 (1020.48 MiB 1069.88 MB)
+ Raid Devices : 2
+Total Devices : 2
+Preferred Minor : 0
+  Persistence : Superblock is persistent
+  Update Time : Wed Oct 29 19:34:23 2008
+        State : active, degraded, recovering
+Active Devices : 1
+Working Devices : 2
+Failed Devices : 0
+Spare Devices : 1
+Rebuild Status : 17% complete
+         UUID : 981abd77:9f8c8dd8:fdbf4de4:3436c70f
+       Events : 0.103
+  Number   Major   Minor   RaidDevice State
+     0       8        1        0      active sync   /dev/sda1
+     2       8       17        1      spare rebuilding   /dev/sdb1
+```
+
+</div>
+
+<div class="opcmd">
+
+show raid \<RAID‐1‐device\>
+
+Use this command to display the formatting of a hard disk.
+
+``` none
+vyos@vyos:~$ show disk sda format
+Disk /dev/sda: 1073 MB, 1073741824 bytes
+85 heads, 9 sectors/track, 2741 cylinders
+Units = cylinders of 765 * 512 = 391680 bytes
+Disk identifier: 0x000b7179   
+ Device Boot      Start         End      Blocks   Id  System
+/dev/sda1               6        2737     1044922+  fd  Linux raid autodetect
+```
+
+</div>