1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
|
---
lastproofread: '2026-02-27'
---
(vpp_config_system)=
```{include} /_include/need_improvement.txt
```
# VyOS Configuration for VPP
(vpp-config-hugepages)=
## Hugepages
VPP uses hugepages for efficient memory management. Hugepages are larger
memory pages that reduce the overhead of page management and improve
performance for applications that require large amounts of memory.
Hugepages can be configured in VyOS using the following commands:
:::{warning}
Changes to hugepage settings require a system reboot to take effect.
Hugepages must be enabled before VPP configuration is applied.
:::
To enable hugepages:
```{cfgcmd} set system option kernel memory hugepage-size \<size\> hugepage-count '\<count\>'
```
Enables hugepages with the specified size and count. The size can be either
2MB or 1GB, and the count specifies the number of hugepages to allocate.
If your system has multiple NUMA nodes, the total amount of hugepages will be
divided equally among them.
## Resources Limits
:::{note}
By default, system will calculate and set the recommended values for
resource limits. Avoid tuning these values if you are not sure what you
are doing.
:::
During operations VPP utilizes a significant amount of system resources,
especially memory. There are two main settings that may need to be
adjusted to ensure VPP runs smoothly:
Maximum number of memory map areas a process may have:
```{cfgcmd} set system option resource-limits max-map-count \<value\>
```
Maximum shared memory segment size:
```{cfgcmd} set system option resource-limits shmmax \<value\>
```
Both settings are automatically calculated based on configured hugepages.
## Kernel Tuning
VPP performance greatly benefits from proper kernel tuning, especially
CPU isolation and disabling unnecessary kernel features. These
optimizations ensure dedicated CPU cores are available exclusively for
VPP dataplane processing without interference from the kernel scheduler
or other system processes.
:::{warning}
Kernel tuning changes require a system reboot to take effect.
Improper CPU isolation can lead to system instability if essential
system processes are starved of CPU resources.
:::
### CPU Isolation and Optimization
CPU isolation is crucial for VPP performance as it dedicates specific
CPU cores exclusively to VPP dataplane processing. The isolated cores are
removed from the kernel scheduler and will not run regular system
processes.
**Disable NMI Watchdog**
The NMI (Non-Maskable Interrupt) watchdog can interfere with VPP
performance by generating interrupts on isolated cores and is not
compatible with nohz-full mode:
```{cfgcmd} set system option kernel cpu disable-nmi-watchdog
Disables the NMI watchdog for detecting hard CPU lockups. This
prevents unnecessary interrupts on VPP worker cores.
```
**CPU Core Isolation**
```{cfgcmd} set system option kernel cpu isolate-cpus \<cpu-range\>
Isolates specified CPUs from the kernel scheduler. Isolated cores will
not run regular system processes and are dedicated to applications like
VPP.
The ``<cpu-range>`` can be:
* Single core: ``2``
* Range: ``2-5``
* Mixed: ``1,3-5,7``
:::{important}
Always reserve at least 2 cores for the operating system to ensure
system stability. For example, on a 4-core system, isolate cores
2-3 for VPP and leave cores 0-1 for the OS.
Assign the first isolated core as the VPP main core and the
remaining isolated cores as VPP worker cores. Ensure that VPP CPU
assignments match the isolated CPU range.
:::
```
**Adaptive-Tick Mode**
```{cfgcmd} set system option kernel cpu nohz-full \<cpu-range\>
Enables adaptive-tick mode (NO_HZ_FULL) for specified CPUs. This
causes the kernel to avoid sending scheduling-clock interrupts to CPUs
that have only one runnable task, significantly reducing interrupt
overhead for dedicated workloads like VPP.
Use the same CPU range as configured for ``isolate-cpus``.
```
**RCU Callback Offloading**
```{cfgcmd} set system option kernel cpu rcu-no-cbs \<cpu-range\>
Offloads Read-Copy-Update (RCU) callback processing from specified
CPUs. This ensures that RCU callbacks do not prevent the specified CPUs
from entering dyntick-idle or adaptive-tick mode, which is essential
for nohz-full functionality.
Use the same CPU range as configured for ``isolate-cpus``.
```
### System Optimization
Additional kernel optimizations can further improve VPP performance by
disabling unnecessary features and reducing system overhead.
**Disable High Precision Event Timer**
```{cfgcmd} set system option kernel disable-hpet
Disables the High Precision Event Timer (HPET). HPET can cause
additional interrupts and overhead that may impact VPP performance.
```
**Disable Machine Check Exceptions**
```{cfgcmd} set system option kernel disable-mce
Disables Machine Check Exception (MCE) reporting and handling. While
MCE provides hardware error detection, it can introduce latency in
high-performance scenarios.
```
**Disable CPU Power Saving**
```{cfgcmd} set system option kernel disable-power-saving
Disables CPU power saving mechanisms (C-states). This keeps CPU cores
at maximum performance levels, eliminating latency from power state
transitions.
```
**Disable Soft Lockup Detection**
```{cfgcmd} set system option kernel disable-softlockup
Disables the soft lockup detector for kernel threads. This prevents
false positives when VPP worker threads are busy processing packets.
```
**Disable CPU Mitigations**
```{cfgcmd} set system option kernel disable-mitigations
Disables all optional CPU mitigations for security vulnerabilities
(for example, Spectre, Meltdown). This may improve performance on some
platforms.
```
### Optimal Configuration Example
For a system with 4 CPU cores (0-3) where cores 2-3 are dedicated to VPP:
```none
# Kernel CPU optimizations
set system option kernel cpu disable-nmi-watchdog
set system option kernel cpu isolate-cpus '2-3'
set system option kernel cpu nohz-full '2-3'
set system option kernel cpu rcu-no-cbs '2-3'
# System optimizations
set system option kernel disable-hpet
set system option kernel disable-mce
set system option kernel disable-power-saving
set system option kernel disable-softlockup
# VPP CPU assignment
set vpp settings resource-allocation cpu-cores '2'
```
|
