README updated

1 files changed, 180 insertions, 28 deletions

diff --git a/README.md b/README.md
index 2e096c8..4beefae 100644
--- a/README.md
+++ b/README.md
@@ -16,9 +16,12 @@ The pipeline is split into two independent stages, each its own playbook,
 each independently triggerable:
 
 1. **Provision / deprovision the VM** (`site/hyperv_vm.yml`)
-   Creates (or removes) the Hyper-V VM from a golden VyOS image, attaches a
-   cloud-init seed ISO for Day-0 bootstrap, starts it, waits for SSH to come
-   up, and marks the device `active` in Netbox.
+   Creates the Hyper-V VM from a golden VyOS image for every device Netbox
+   marks `staged`, attaches a cloud-init seed ISO for Day-0 bootstrap,
+   starts it, waits for SSH to come up, and marks the device `active` in
+   Netbox. Devices marked `decommissioning` are reported only, unless the
+   run is explicitly invoked with `confirm_destroy=true` (see **Rundeck
+   integration** below), in which case that specific device is torn down.
 
 2. **Configure the appliance** (`site/vyos_configure.yml`)
    Applies Day-1 configuration over SSH (`network_cli`) using the
@@ -27,10 +30,10 @@ each independently triggerable:
    existing appliance never touches the VM lifecycle.
 
 Netbox is the source of truth throughout: VM sizing (memory/vCPU),
-interfaces and IP addressing, business-function role, and role-specific
-configuration data (via Config Context / Local Context Data) all come from
-Netbox, not hardcoded playbook values or manually maintained inventory
-files.
+interfaces and IP addressing, business-function role, lifecycle status,
+and role-specific configuration data (via Config Context / Local Context
+Data) all come from Netbox, not hardcoded playbook values or manually
+maintained inventory files.
 
 ### Day-0 / Day-1 split
 
@@ -72,7 +75,7 @@ files.
 - Python 3.x with `pip`
 - A Hyper-V host reachable over WinRM (HTTPS, port 5986), with a golden
   VyOS image already present
-- A Netbox instance with devices/VMs modeled, and an API token
+- A Netbox instance, set up per **Netbox setup** below
 - SSH reachability from the runner to provisioned VyOS appliances
 
 ## Setup
@@ -84,11 +87,13 @@ ansible-galaxy collection install -r requirements.yaml
 
 `requirements.txt` deliberately lists Python packages that specific
 collections' connection plugins need at runtime (e.g. `pywinrm`/`pypsrp`
-for `microsoft.hyperv`'s WinRM/PSRP plugins). `ansible-galaxy` only
-installs collection content — it does not read or install a collection's
-Python dependencies, even when a collection ships its own
-`requirements.txt` internally. Those have to be discovered and listed here
-by hand.
+for `microsoft.hyperv`'s WinRM/PSRP plugins, `pytz` for
+`netbox.netbox`'s inventory plugin). `ansible-galaxy` only installs
+collection content — it does not read or install a collection's Python
+dependencies, even when a collection ships its own `requirements.txt`
+internally. Those have to be discovered and listed here by hand; if a run
+fails with `No module named '<x>'`, add `<x>` here rather than assuming
+it's a one-off environment problem.
 
 ## Required environment variables
 
@@ -101,21 +106,169 @@ by hand.
 None of these are stored in the repo. See **Known limitations** re:
 credential handling.
 
+## Netbox setup
+
+Netbox holds every piece of data the pipeline needs — device/VM
+inventory, connection targeting, sizing, and role-specific configuration.
+Nothing about a specific appliance is meant to live in this repo.
+
+**1. Device Role — `hypervisor`**
+Devices → Device Roles → Add. Slug must be exactly `hypervisor` — this is
+what `group_by: device_roles` in `inventory/netbox.yml` turns into the
+`device_roles_hypervisor` inventory group, which `group_vars/device_roles_hypervisor.yml`
+matches by name.
+
+**2. Custom Field — `golden_vhdx_path`**
+Customization → Custom Fields → Add. Type: Text. Object type: `DCIM >
+Device`. This is where the Windows path to the golden VyOS image lives —
+set it on the Hyper-V device itself (Devices → your device → Edit).
+Surfaces in inventory under `custom_fields.golden_vhdx_path`.
+
+**3. The Hyper-V host device**
+Devices → Add Device, Role `hypervisor`. Add an interface, assign it an IP
+address, then set that IP as the device's **Primary IPv4** (Edit → Primary
+IPv4 address) — the inventory's `compose: ansible_host: primary_ip4.ip`
+only reads from this field specifically, not any IP merely attached to an
+interface.
+
+**4. Platform — `vyos`**
+Devices → Platforms → Add, slug `vyos`. Produces the `platforms_vyos`
+group.
+
+**5. Role — `edge_router`** (and similarly for other business functions as
+those roles get implemented)
+Same Role model, used for VMs. Slug `edge_router`.
+
+**6. Config Context — role-level baseline**
+Provisioning → Config Contexts → Add, assigned to Role `edge_router`:
+```json
+{
+  "vyos": {
+    "role": "edge_router",
+    "wan_dhcp": true,
+    "nat_enabled": true
+  }
+}
+```
+Every VM with this Role inherits this data automatically — surfaces
+flattened at the top level of that host's inventory vars (e.g. `vyos.role`),
+not nested under a `config_context` key, since `flatten_config_context: True`
+is set in `inventory/netbox.yml`.
+
+**7. Local Context Data — per-device override**
+On an individual VM's own edit page, to override the shared baseline for
+just that instance:
+```json
+{
+  "vyos": {
+    "wan_dhcp": false,
+    "wan_gateway": "203.0.113.1"
+  }
+}
+```
+Netbox merges this on top of the Role-level Config Context, highest
+precedence per key — every other `edge_router` VM without this override
+keeps the shared default.
+
+**8. The test/appliance VM itself**
+Virtualization → Virtual Machines → Add: Role `edge_router`, Platform
+`vyos`, Status `staged`. Add interfaces (e.g. `eth0` management with an
+assigned IP, `eth1` WAN with none for DHCP), and memory/vCPU — or leave
+those blank to exercise the role defaults' fallback (see `roles/hyperv_vm/defaults/main.yml`).
+
+**Verify before running anything:**
+```bash
+ansible-inventory -i inventory/netbox.yml --host <name>
+```
+Confirm `netbox_id`, `netbox_name`, `interfaces`, `custom_fields`, `vyos.*`,
+and `status.value` all render as expected for both the hypervisor and the
+appliance VM.
+
 ## Running locally
 
 ```bash
-# Provision (or deprovision) a VM
+# Reconcile: provisions every VM Netbox marks 'staged'.
+# Devices marked 'decommissioning' are reported only, never destroyed here.
 ansible-playbook -i inventory/netbox.yml site/hyperv_vm.yml \
   --forks 1 \
-  -e "vm_name=<name>" -e "appliance_state=present"
+  -l "<optional -l pattern, omit to reconcile everything staged>"
 
-# Configure an appliance
+# Destroy: only for a device already marked 'decommissioning' in Netbox.
+# -l is required — this never runs unscoped.
+ansible-playbook -i inventory/netbox.yml site/hyperv_vm.yml \
+  --forks 1 \
+  -l "<vm-name>" \
+  -e "confirm_destroy=true"
+
+# Configure: applies Day-1 config to already-provisioned appliance(s).
 ansible-playbook -i inventory/netbox.yml site/vyos_configure.yml \
-  -e "target_host=<name>"
+  -l "<optional -l pattern, omit to configure every active appliance>"
 ```
 
 `--forks 1` on the Hyper-V play is not optional — see the WinRM note
-below.
+below. Neither playbook takes a `target_host`/`appliance_state` variable;
+targeting is entirely via `-l` and Netbox's own `status` field.
+
+## Rundeck integration
+
+**Job Source Control (SCM):** Rundeck's own job definitions are versioned
+in a separate git repository from this one (job definitions vs. automation
+content are reviewed independently). Configured under Project Settings →
+Source Control Management, Git plugin, both Import and Export enabled.
+
+**`_setup_environment`** — an internal job (not meant to be run directly),
+referenced via job-ref from every payload job below. Clones this repo and
+installs dependencies fresh on every execution, since the runner is
+ephemeral:
+```bash
+rm -rf workspace
+git clone --branch main --depth 1 <this-repo-url> workspace
+cd workspace
+pip install -r requirements.txt --break-system-packages
+ansible-galaxy collection install -r requirements.yaml
+```
+
+**Three payload jobs**, each cloning fresh via `_setup_environment`, each
+ending with an explicit `rm -rf /tmp/@job.execid@` cleanup step:
+
+| Job | Playbook | Options |
+|---|---|---|
+| `Reconcile Appliance VMs` | `site/hyperv_vm.yml` | `netbox_api`, optional `limit` (plain text, maps to `-l`) |
+| `Destroy Appliance VM` | `site/hyperv_vm.yml` `-e confirm_destroy=true` | `netbox_api`, **required** `limit` — never runs unscoped |
+| `Configure Appliance` | `site/vyos_configure.yml` | `netbox_api`, optional `limit` |
+
+`Destroy Appliance VM` should have a narrower ACL than the other two — it's
+the one job in this set that permanently deletes a VM and its disk.
+Reconcile only ever provisions or reports; it never deletes anything on
+its own.
+
+**Secrets:** `netbox_token` and `hyperv_admin_password` are Key
+Storage-backed options (Input Type **Plain Text with Password Input**, not
+**Secure Remote Authentication** — the latter withholds the value from
+script steps entirely, which breaks the `export NETBOX_TOKEN=...` pattern
+these jobs rely on).
+
+**Script-step variable syntax:** inside inline script step bodies, use
+Rundeck's own `@option.foo@` / `@job.execid@` token syntax, not shell-style
+`${...}` — the latter is only substituted on native plugin step fields
+(e.g. the Git Clone step's Base Directory), not on script content, and
+will fail with `bad substitution` if used inside a script body.
+
+**Example step content** (Reconcile job):
+```bash
+cd /tmp/@job.execid@/vyos-hyperv-automation
+
+export NETBOX_API="@option.netbox_api@"
+export NETBOX_TOKEN="@option.netbox_token@"
+export HYPERV_ADMIN_PASSWORD="@option.hyperv_admin_password@"
+
+LIMIT_ARG=""
+if [ -n "@option.limit@" ]; then
+  LIMIT_ARG="-l @option.limit@"
+fi
+
+ansible-playbook -i inventory/netbox.yml site/hyperv_vm.yml --forks 1 $LIMIT_ARG
+```
 
 ## Known limitations (demo scope)
 
@@ -132,18 +285,17 @@ below.
   identical failures across multiple independent client libraries
   (Terraform's provider, raw `pywinrm`, `pypsrp`, and Ansible's own WinRM
   connection plugin).
-- **Single Hyper-V host.** `hyperv_target` defaults to one host; no
-  load-spreading logic across multiple hypervisors yet.
+- **Single Hyper-V host.** `hyperv_target` defaults to one host (matching
+  its Netbox device name); no load-spreading logic across multiple
+  hypervisors yet.
 - **No formal change management.** Requests are lodged by directly running
-  the relevant job with the appropriate parameters. There is currently no
-  separate request/approval step upstream of execution.
+  the relevant job with the appropriate parameters. `Destroy Appliance VM`
+  requires an explicit target and an explicit confirmation flag as its
+  only gate; there is no separate upstream approval workflow yet.
 - **Terraform is not used.** An earlier iteration of this pipeline used
   Terraform (`taliesins/hyperv` provider) for VM provisioning. This was
   descoped in favor of Ansible end-to-end (`microsoft.hyperv`) for a
   single, consistent tool and better idempotency around partial failures.
-- **Roles beyond `vyos_edge_router` are scaffolded but not implemented**
-  (`vyos_firewall`, `vyos_dual_wan`, `vyos_ha`, `vyos_ipsec_vpn`).
-
-## Rundeck integration
-
-_To be documented._
+- **`vyos_edge_router` is the only implemented business-function role.**
+  `vyos_firewall`, `vyos_dual_wan`, `vyos_ha`, `vyos_ipsec_vpn` are
+  scaffolded but not yet implemented.
\ No newline at end of file