summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
-rw-r--r--docs/vpp/configuration/dataplane/cpu.md21
-rw-r--r--docs/vpp/configuration/dataplane/system.md55
-rw-r--r--docs/vpp/requirements.md32
3 files changed, 104 insertions, 4 deletions
diff --git a/docs/vpp/configuration/dataplane/cpu.md b/docs/vpp/configuration/dataplane/cpu.md
index d92f6587..bd67b91e 100644
--- a/docs/vpp/configuration/dataplane/cpu.md
+++ b/docs/vpp/configuration/dataplane/cpu.md
@@ -35,6 +35,27 @@ This parameter defines the total number of CPU cores allocated to VPP.
```{cfgcmd} set vpp settings resource-allocation cpu-cores \<core-number\>
```
+:::{important}
+`cpu-cores` requires at least the same number of CPUs to be isolated from the
+kernel scheduler with `set system option kernel cpu isolate-cpus`. VPP takes
+its main core and its worker cores from the isolated set, and the commit is
+rejected when fewer CPUs are isolated than requested:
+
+```none
+Not enough isolated CPU cores available: 2 requested, but only 0 isolated.
+To isolate CPUs please use command
+"set system option kernel cpu isolate-cpus ..." save and reboot!
+```
+
+This applies even when `cpu-cores` is left at its default of `1`. VPP always
+takes its main core from the isolated set, so at least one CPU has to be
+isolated before the dataplane can be enabled at all.
+
+CPU isolation is a kernel option, so it has to be committed, saved and applied
+with a reboot **before** VPP is configured — the two cannot be set in the same
+commit. See {ref}`Optimal Configuration Example <vpp-config-setup-order>`.
+:::
+
The system automatically assigns cores using the following rules:
> - The first two CPU cores are always reserved for the operating system and
diff --git a/docs/vpp/configuration/dataplane/system.md b/docs/vpp/configuration/dataplane/system.md
index 51ee8f54..7ece3192 100644
--- a/docs/vpp/configuration/dataplane/system.md
+++ b/docs/vpp/configuration/dataplane/system.md
@@ -190,9 +190,24 @@ Disables all optional CPU mitigations for security vulnerabilities
platforms.
```
+(vpp-config-setup-order)=
+
### Optimal Configuration Example
-For a system with 4 CPU cores (0-3) where cores 2-3 are dedicated to VPP:
+For a system with 4 CPU cores (0-3) where cores 2-3 are dedicated to VPP.
+
+:::{important}
+Kernel options and VPP settings **cannot be applied in the same commit**.
+
+Everything under `system option kernel` only takes effect after a reboot,
+while the VPP configuration is validated against the *running* kernel. VPP is
+also committed before `system option`, so within a single commit the kernel
+options are never in place yet and the VPP part is rejected.
+
+Configure the kernel options first, save, reboot, and only then configure VPP.
+:::
+
+**Step 1 — kernel options**
```none
# Kernel CPU optimizations
@@ -201,12 +216,50 @@ set system option kernel cpu isolate-cpus '2-3'
set system option kernel cpu nohz-full '2-3'
set system option kernel cpu rcu-no-cbs '2-3'
+# Hugepages
+set system option kernel memory hugepage-size 2M hugepage-count '2048'
+
# System optimizations
set system option kernel disable-hpet
set system option kernel disable-mce
set system option kernel disable-power-saving
set system option kernel disable-softlockup
+```
+
+Commit, save and reboot:
+
+```none
+commit
+save
+exit
+reboot
+```
+
+**Step 2 — VPP configuration, after the reboot**
+```none
# VPP CPU assignment
set vpp settings resource-allocation cpu-cores '2'
+
+# Interfaces to attach to the dataplane
+set vpp settings interface eth0
+set vpp settings interface eth1
```
+
+```none
+commit
+save
+```
+
+:::{note}
+If the VPP part is committed too early, the commit fails with a message such
+as `Not enough free memory to start VPP!` or `Not enough isolated CPU cores
+available`, both of which point back at `set system option kernel ...`. That
+means the reboot from step 1 has not happened yet.
+
+Note that such a commit is *partial*: the `system option kernel` part is still
+applied even though the commit is reported as failed. The `set vpp ...`
+statements stay in the configuration session, but they are neither applied nor
+written by `save`, which writes the *running* configuration. They are lost on
+the reboot and have to be entered again.
+:::
diff --git a/docs/vpp/requirements.md b/docs/vpp/requirements.md
index 159f70d4..db53058a 100644
--- a/docs/vpp/requirements.md
+++ b/docs/vpp/requirements.md
@@ -52,6 +52,23 @@ prerequisites before enabling VPP:
- Recommended: 16 GB or more (especially for high throughput, many interfaces,
or large routing tables).
+- **Kernel Configuration**
+
+ Besides the hardware itself, VPP needs hugepages and isolated CPU cores.
+ Both are configured under `system option kernel` and only take effect after
+ a reboot, while VPP validates them against the *running* kernel.
+
+ :::{important}
+ The kernel options and the VPP configuration cannot be applied in the same
+ commit. Configure them under `system option kernel` first, `commit`, `save`
+ and reboot, and only then configure VPP.
+ :::
+
+ :::{seealso}
+ {ref}`VyOS Configuration for VPP <vpp_config_system>` — hugepages, CPU
+ isolation and kernel tuning, with a worked example of both steps.
+ :::
+
- **Network Interface Cards (NICs)**
:::{warning}
@@ -102,6 +119,13 @@ prerequisites before enabling VPP:
* - PCI ID
- 1af4:1000
- Red Hat, Inc. Virtio network device
+ (legacy ID)
+ - KVM-based hypervisors, including with
+ Open vSwitch; Google Cloud
+ * - PCI ID
+ - 1af4:1041
+ - Red Hat, Inc. Virtio network device
+ (modern ID)
- KVM-based hypervisors, including with
Open vSwitch; Google Cloud
* - PCI ID
@@ -125,7 +149,9 @@ prerequisites before enabling VPP:
```
:::{note}
- This option bypasses the hardware validation checks for the specified
- devices. Stability and performance are not guaranteed when using
- unsupported NICs or drivers.
+ This option is a single global switch, not a per-device one: it bypasses
+ the hardware validation check for every interface attached to VPP, not
+ only for the one that needed it, and for interfaces attached later as
+ well. Stability and performance are not guaranteed when using unsupported
+ NICs or drivers.
:::