diff options
| -rw-r--r-- | docs/vpp/configuration/dataplane/cpu.md | 21 | ||||
| -rw-r--r-- | docs/vpp/configuration/dataplane/system.md | 55 | ||||
| -rw-r--r-- | docs/vpp/requirements.md | 32 |
3 files changed, 104 insertions, 4 deletions
diff --git a/docs/vpp/configuration/dataplane/cpu.md b/docs/vpp/configuration/dataplane/cpu.md index d92f6587..bd67b91e 100644 --- a/docs/vpp/configuration/dataplane/cpu.md +++ b/docs/vpp/configuration/dataplane/cpu.md @@ -35,6 +35,27 @@ This parameter defines the total number of CPU cores allocated to VPP. ```{cfgcmd} set vpp settings resource-allocation cpu-cores \<core-number\> ``` +:::{important} +`cpu-cores` requires at least the same number of CPUs to be isolated from the +kernel scheduler with `set system option kernel cpu isolate-cpus`. VPP takes +its main core and its worker cores from the isolated set, and the commit is +rejected when fewer CPUs are isolated than requested: + +```none +Not enough isolated CPU cores available: 2 requested, but only 0 isolated. +To isolate CPUs please use command +"set system option kernel cpu isolate-cpus ..." save and reboot! +``` + +This applies even when `cpu-cores` is left at its default of `1`. VPP always +takes its main core from the isolated set, so at least one CPU has to be +isolated before the dataplane can be enabled at all. + +CPU isolation is a kernel option, so it has to be committed, saved and applied +with a reboot **before** VPP is configured — the two cannot be set in the same +commit. See {ref}`Optimal Configuration Example <vpp-config-setup-order>`. +::: + The system automatically assigns cores using the following rules: > - The first two CPU cores are always reserved for the operating system and diff --git a/docs/vpp/configuration/dataplane/system.md b/docs/vpp/configuration/dataplane/system.md index 51ee8f54..7ece3192 100644 --- a/docs/vpp/configuration/dataplane/system.md +++ b/docs/vpp/configuration/dataplane/system.md @@ -190,9 +190,24 @@ Disables all optional CPU mitigations for security vulnerabilities platforms. ``` +(vpp-config-setup-order)= + ### Optimal Configuration Example -For a system with 4 CPU cores (0-3) where cores 2-3 are dedicated to VPP: +For a system with 4 CPU cores (0-3) where cores 2-3 are dedicated to VPP. + +:::{important} +Kernel options and VPP settings **cannot be applied in the same commit**. + +Everything under `system option kernel` only takes effect after a reboot, +while the VPP configuration is validated against the *running* kernel. VPP is +also committed before `system option`, so within a single commit the kernel +options are never in place yet and the VPP part is rejected. + +Configure the kernel options first, save, reboot, and only then configure VPP. +::: + +**Step 1 — kernel options** ```none # Kernel CPU optimizations @@ -201,12 +216,50 @@ set system option kernel cpu isolate-cpus '2-3' set system option kernel cpu nohz-full '2-3' set system option kernel cpu rcu-no-cbs '2-3' +# Hugepages +set system option kernel memory hugepage-size 2M hugepage-count '2048' + # System optimizations set system option kernel disable-hpet set system option kernel disable-mce set system option kernel disable-power-saving set system option kernel disable-softlockup +``` + +Commit, save and reboot: + +```none +commit +save +exit +reboot +``` + +**Step 2 — VPP configuration, after the reboot** +```none # VPP CPU assignment set vpp settings resource-allocation cpu-cores '2' + +# Interfaces to attach to the dataplane +set vpp settings interface eth0 +set vpp settings interface eth1 ``` + +```none +commit +save +``` + +:::{note} +If the VPP part is committed too early, the commit fails with a message such +as `Not enough free memory to start VPP!` or `Not enough isolated CPU cores +available`, both of which point back at `set system option kernel ...`. That +means the reboot from step 1 has not happened yet. + +Note that such a commit is *partial*: the `system option kernel` part is still +applied even though the commit is reported as failed. The `set vpp ...` +statements stay in the configuration session, but they are neither applied nor +written by `save`, which writes the *running* configuration. They are lost on +the reboot and have to be entered again. +::: diff --git a/docs/vpp/requirements.md b/docs/vpp/requirements.md index 159f70d4..db53058a 100644 --- a/docs/vpp/requirements.md +++ b/docs/vpp/requirements.md @@ -52,6 +52,23 @@ prerequisites before enabling VPP: - Recommended: 16 GB or more (especially for high throughput, many interfaces, or large routing tables). +- **Kernel Configuration** + + Besides the hardware itself, VPP needs hugepages and isolated CPU cores. + Both are configured under `system option kernel` and only take effect after + a reboot, while VPP validates them against the *running* kernel. + + :::{important} + The kernel options and the VPP configuration cannot be applied in the same + commit. Configure them under `system option kernel` first, `commit`, `save` + and reboot, and only then configure VPP. + ::: + + :::{seealso} + {ref}`VyOS Configuration for VPP <vpp_config_system>` — hugepages, CPU + isolation and kernel tuning, with a worked example of both steps. + ::: + - **Network Interface Cards (NICs)** :::{warning} @@ -102,6 +119,13 @@ prerequisites before enabling VPP: * - PCI ID - 1af4:1000 - Red Hat, Inc. Virtio network device + (legacy ID) + - KVM-based hypervisors, including with + Open vSwitch; Google Cloud + * - PCI ID + - 1af4:1041 + - Red Hat, Inc. Virtio network device + (modern ID) - KVM-based hypervisors, including with Open vSwitch; Google Cloud * - PCI ID @@ -125,7 +149,9 @@ prerequisites before enabling VPP: ``` :::{note} - This option bypasses the hardware validation checks for the specified - devices. Stability and performance are not guaranteed when using - unsupported NICs or drivers. + This option is a single global switch, not a per-device one: it bypasses + the hardware validation check for every interface attached to VPP, not + only for the one that needed it, and for interfaces attached later as + well. Stability and performance are not guaranteed when using unsupported + NICs or drivers. ::: |
