From 5d6fa52b8985f8068314aba26878a1d7d5cb84e5 Mon Sep 17 00:00:00 2001 From: Yuriy Andamasov Date: Wed, 6 May 2026 20:42:32 +0300 Subject: feat: flip swap mechanism — MD as primary, RST as override (Phase 1) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit This is the first of three phases inverting the per-page swap mechanism so MD becomes the canonical primary and RST becomes the rare override. Phase 1 — file renames + conf.py exclude_patterns flip only: - Rename docs/**/md-.md to docs/**/.md (drop md- prefix) for all 254 stems previously listed in docs/_swap.txt - Rename docs/**/.rst to docs/**/rst-.rst (add rst- prefix) for the same 254 stems - Repurpose docs/_swap.txt as docs/_rst_overrides.txt; initially empty comment-only since no pages need the RST fallback right now - conf.py exclude_patterns flipped: rst-*.rst is now excluded by default instead of md-*.md - conf.py runtime-artifact references updated to _rst_override_state.json and _md_exclude.txt (Phase 2 will rewrite swap_sources.py to produce these names; for now no swap script runs because overrides list is empty) Phase 2 (next commit on this branch) will rewrite scripts/swap_sources.py with inverted rename direction, delete scripts/import_myst.py + tests, and update tests/test_swap_sources.py for the new semantics. Phase 3 will be the cleanup pass and ready-for-review flip. Generated by robots https://vyos.io --- docs/vpp/configuration/dataplane/buffers.md | 102 ++++++++++ docs/vpp/configuration/dataplane/buffers.rst | 103 ---------- docs/vpp/configuration/dataplane/cpu.md | 71 +++++++ docs/vpp/configuration/dataplane/cpu.rst | 74 ------- docs/vpp/configuration/dataplane/index.md | 36 ++++ docs/vpp/configuration/dataplane/index.rst | 35 ---- docs/vpp/configuration/dataplane/interface.md | 104 ++++++++++ docs/vpp/configuration/dataplane/interface.rst | 102 ---------- docs/vpp/configuration/dataplane/ipsec.md | 74 +++++++ docs/vpp/configuration/dataplane/ipsec.rst | 72 ------- docs/vpp/configuration/dataplane/ipv6.md | 46 +++++ docs/vpp/configuration/dataplane/ipv6.rst | 43 ----- docs/vpp/configuration/dataplane/l2learn.md | 35 ++++ docs/vpp/configuration/dataplane/l2learn.rst | 34 ---- docs/vpp/configuration/dataplane/lcp.md | 46 +++++ docs/vpp/configuration/dataplane/lcp.rst | 48 ----- docs/vpp/configuration/dataplane/logging.md | 59 ++++++ docs/vpp/configuration/dataplane/logging.rst | 58 ------ docs/vpp/configuration/dataplane/md-buffers.md | 102 ---------- docs/vpp/configuration/dataplane/md-cpu.md | 71 ------- docs/vpp/configuration/dataplane/md-index.md | 36 ---- docs/vpp/configuration/dataplane/md-interface.md | 104 ---------- docs/vpp/configuration/dataplane/md-ipsec.md | 74 ------- docs/vpp/configuration/dataplane/md-ipv6.md | 46 ----- docs/vpp/configuration/dataplane/md-l2learn.md | 35 ---- docs/vpp/configuration/dataplane/md-lcp.md | 46 ----- docs/vpp/configuration/dataplane/md-logging.md | 59 ------ docs/vpp/configuration/dataplane/md-memory.md | 142 -------------- docs/vpp/configuration/dataplane/md-system.md | 212 --------------------- docs/vpp/configuration/dataplane/md-unix.md | 57 ------ docs/vpp/configuration/dataplane/memory.md | 142 ++++++++++++++ docs/vpp/configuration/dataplane/memory.rst | 141 -------------- docs/vpp/configuration/dataplane/rst-buffers.rst | 103 ++++++++++ docs/vpp/configuration/dataplane/rst-cpu.rst | 74 +++++++ docs/vpp/configuration/dataplane/rst-index.rst | 35 ++++ docs/vpp/configuration/dataplane/rst-interface.rst | 102 ++++++++++ docs/vpp/configuration/dataplane/rst-ipsec.rst | 72 +++++++ docs/vpp/configuration/dataplane/rst-ipv6.rst | 43 +++++ docs/vpp/configuration/dataplane/rst-l2learn.rst | 34 ++++ docs/vpp/configuration/dataplane/rst-lcp.rst | 48 +++++ docs/vpp/configuration/dataplane/rst-logging.rst | 58 ++++++ docs/vpp/configuration/dataplane/rst-memory.rst | 141 ++++++++++++++ docs/vpp/configuration/dataplane/rst-system.rst | 208 ++++++++++++++++++++ docs/vpp/configuration/dataplane/rst-unix.rst | 55 ++++++ docs/vpp/configuration/dataplane/system.md | 212 +++++++++++++++++++++ docs/vpp/configuration/dataplane/system.rst | 208 -------------------- docs/vpp/configuration/dataplane/unix.md | 57 ++++++ docs/vpp/configuration/dataplane/unix.rst | 55 ------ 48 files changed, 1957 insertions(+), 1957 deletions(-) create mode 100644 docs/vpp/configuration/dataplane/buffers.md delete mode 100644 docs/vpp/configuration/dataplane/buffers.rst create mode 100644 docs/vpp/configuration/dataplane/cpu.md delete mode 100644 docs/vpp/configuration/dataplane/cpu.rst create mode 100644 docs/vpp/configuration/dataplane/index.md delete mode 100644 docs/vpp/configuration/dataplane/index.rst create mode 100644 docs/vpp/configuration/dataplane/interface.md delete mode 100644 docs/vpp/configuration/dataplane/interface.rst create mode 100644 docs/vpp/configuration/dataplane/ipsec.md delete mode 100644 docs/vpp/configuration/dataplane/ipsec.rst create mode 100644 docs/vpp/configuration/dataplane/ipv6.md delete mode 100644 docs/vpp/configuration/dataplane/ipv6.rst create mode 100644 docs/vpp/configuration/dataplane/l2learn.md delete mode 100644 docs/vpp/configuration/dataplane/l2learn.rst create mode 100644 docs/vpp/configuration/dataplane/lcp.md delete mode 100644 docs/vpp/configuration/dataplane/lcp.rst create mode 100644 docs/vpp/configuration/dataplane/logging.md delete mode 100644 docs/vpp/configuration/dataplane/logging.rst delete mode 100644 docs/vpp/configuration/dataplane/md-buffers.md delete mode 100644 docs/vpp/configuration/dataplane/md-cpu.md delete mode 100644 docs/vpp/configuration/dataplane/md-index.md delete mode 100644 docs/vpp/configuration/dataplane/md-interface.md delete mode 100644 docs/vpp/configuration/dataplane/md-ipsec.md delete mode 100644 docs/vpp/configuration/dataplane/md-ipv6.md delete mode 100644 docs/vpp/configuration/dataplane/md-l2learn.md delete mode 100644 docs/vpp/configuration/dataplane/md-lcp.md delete mode 100644 docs/vpp/configuration/dataplane/md-logging.md delete mode 100644 docs/vpp/configuration/dataplane/md-memory.md delete mode 100644 docs/vpp/configuration/dataplane/md-system.md delete mode 100644 docs/vpp/configuration/dataplane/md-unix.md create mode 100644 docs/vpp/configuration/dataplane/memory.md delete mode 100644 docs/vpp/configuration/dataplane/memory.rst create mode 100644 docs/vpp/configuration/dataplane/rst-buffers.rst create mode 100644 docs/vpp/configuration/dataplane/rst-cpu.rst create mode 100644 docs/vpp/configuration/dataplane/rst-index.rst create mode 100644 docs/vpp/configuration/dataplane/rst-interface.rst create mode 100644 docs/vpp/configuration/dataplane/rst-ipsec.rst create mode 100644 docs/vpp/configuration/dataplane/rst-ipv6.rst create mode 100644 docs/vpp/configuration/dataplane/rst-l2learn.rst create mode 100644 docs/vpp/configuration/dataplane/rst-lcp.rst create mode 100644 docs/vpp/configuration/dataplane/rst-logging.rst create mode 100644 docs/vpp/configuration/dataplane/rst-memory.rst create mode 100644 docs/vpp/configuration/dataplane/rst-system.rst create mode 100644 docs/vpp/configuration/dataplane/rst-unix.rst create mode 100644 docs/vpp/configuration/dataplane/system.md delete mode 100644 docs/vpp/configuration/dataplane/system.rst create mode 100644 docs/vpp/configuration/dataplane/unix.md delete mode 100644 docs/vpp/configuration/dataplane/unix.rst (limited to 'docs/vpp/configuration/dataplane') diff --git a/docs/vpp/configuration/dataplane/buffers.md b/docs/vpp/configuration/dataplane/buffers.md new file mode 100644 index 00000000..c9e38a54 --- /dev/null +++ b/docs/vpp/configuration/dataplane/buffers.md @@ -0,0 +1,102 @@ +--- +lastproofread: '2026-02-23' +--- + +(vpp-config-dataplane-buffers)= + +```{include} /_include/need_improvement.txt +``` + + +# VPP Dataplane Buffers Configuration + +Buffers are essential for handling network packets efficiently. Proper +configuration enhances performance and reliability, and is mandatory for +VPP to work. Buffers temporarily store packets during processing. Therefore, +their configuration must be in sync with NIC configuration, CPU threads, and +overall system resources. + +:::{important} +VPP buffers are allocated from the physical memory pool (`physmem`). The +total amount of memory available for buffer allocation is controlled by the +`physmem-max-size` setting, while the buffer configuration parameters +below control how that memory is used for buffer allocation. + +See {ref}`VPP Physical Memory Configuration ` +for details on configuring `physmem`. +::: + +## Buffer Configuration Parameters + +The following parameters can be configured for VPP buffers: + +### buffers-per-numa + +Number of buffers allocated per NUMA node. This setting optimizes +memory access patterns for multi-CPU systems. + +Typically, you need to tune this value if: +- The system has many interfaces +- NICs have many queues +- NICs have large descriptor sizes + +Set this value carefully to balance memory usage and performance. + +```{cfgcmd} set vpp settings resource-allocation buffers buffers-per-numa \ +``` + +The common approach for the calculation is to use the formula: + +```none +buffers-per-numa = (num-rx-queues * num-rx-desc) + (num-tx-queues * num-tx-desc) +``` + +Calculate this formula for each NIC and sum the results. Multiply the +total by 2.5 to get the minimum recommended value for +`buffers-per-numa`. + +Avoid setting this value too low to prevent packet drops. + +### data-size + +This value sets how much payload data can be stored in a single buffer +allocated by VPP. Larger values reduce buffer chains for large packets, +while smaller values conserve memory for environments handling mostly +small packets. + +```{cfgcmd} set vpp settings resource-allocation buffers data-size \ +``` + +Optimal size depends on the typical packet size in your network. If +unsure, use the largest MTU in your network plus overhead (for example, +128 bytes). + +### page-size + +A memory pages type used for buffer allocation. Common values are 4K, 2M, or 1G. + +Use page sizes configured in your system settings. + +```{cfgcmd} set vpp settings resource-allocation buffers page-size \ +``` + + +## Potential Issues and Troubleshooting + +Improper buffer configuration can lead to issues such as: + +- Increased latency and packet loss +- Inefficient CPU utilization +- Interface initialization failures + +Indicators of such issues are: + +- Errors during interfaces initialization in VPP logs +- Packet drops observed in VPP statistics + +To troubleshoot buffer-related issues, consider the following steps: + +- Review VPP logs for errors related to buffer allocation. Look for + error `-5` messages. +- Tune available buffers by adjusting the `buffers-per-numa` and + `data-size` parameters. diff --git a/docs/vpp/configuration/dataplane/buffers.rst b/docs/vpp/configuration/dataplane/buffers.rst deleted file mode 100644 index b08eb691..00000000 --- a/docs/vpp/configuration/dataplane/buffers.rst +++ /dev/null @@ -1,103 +0,0 @@ -:lastproofread: 2026-02-23 - -.. _vpp_config_dataplane_buffers: - -.. include:: /_include/need_improvement.txt - -################################### -VPP Dataplane Buffers Configuration -################################### - -Buffers are essential for handling network packets efficiently. Proper -configuration enhances performance and reliability, and is mandatory for -VPP to work. Buffers temporarily store packets during processing. Therefore, -their configuration must be in sync with NIC configuration, CPU threads, and -overall system resources. - -.. important:: - - VPP buffers are allocated from the physical memory pool (``physmem``). The - total amount of memory available for buffer allocation is controlled by the - ``physmem-max-size`` setting, while the buffer configuration parameters - below control how that memory is used for buffer allocation. - - See :ref:`VPP Physical Memory Configuration ` - for details on configuring ``physmem``. - -Buffer Configuration Parameters -=============================== - -The following parameters can be configured for VPP buffers: - -buffers-per-numa ----------------- - -Number of buffers allocated per NUMA node. This setting optimizes -memory access patterns for multi-CPU systems. - -Typically, you need to tune this value if: - -- The system has many interfaces -- NICs have many queues -- NICs have large descriptor sizes - -Set this value carefully to balance memory usage and performance. - -.. cfgcmd:: set vpp settings resource-allocation buffers buffers-per-numa - - -The common approach for the calculation is to use the formula: - -.. code-block:: none - - buffers-per-numa = (num-rx-queues * num-rx-desc) + (num-tx-queues * num-tx-desc) - -Calculate this formula for each NIC and sum the results. Multiply the -total by 2.5 to get the minimum recommended value for -``buffers-per-numa``. - -Avoid setting this value too low to prevent packet drops. - -data-size ---------- - -This value sets how much payload data can be stored in a single buffer -allocated by VPP. Larger values reduce buffer chains for large packets, -while smaller values conserve memory for environments handling mostly -small packets. - -.. cfgcmd:: set vpp settings resource-allocation buffers data-size - -Optimal size depends on the typical packet size in your network. If -unsure, use the largest MTU in your network plus overhead (for example, -128 bytes). - -page-size ---------- - -A memory pages type used for buffer allocation. Common values are 4K, 2M, or 1G. - -Use page sizes configured in your system settings. - -.. cfgcmd:: set vpp settings resource-allocation buffers page-size - -Potential Issues and Troubleshooting -==================================== - -Improper buffer configuration can lead to issues such as: - -- Increased latency and packet loss -- Inefficient CPU utilization -- Interface initialization failures - -Indicators of such issues are: - -- Errors during interfaces initialization in VPP logs -- Packet drops observed in VPP statistics - -To troubleshoot buffer-related issues, consider the following steps: - -- Review VPP logs for errors related to buffer allocation. Look for - error ``-5`` messages. -- Tune available buffers by adjusting the ``buffers-per-numa`` and - ``data-size`` parameters. diff --git a/docs/vpp/configuration/dataplane/cpu.md b/docs/vpp/configuration/dataplane/cpu.md new file mode 100644 index 00000000..d92f6587 --- /dev/null +++ b/docs/vpp/configuration/dataplane/cpu.md @@ -0,0 +1,71 @@ +--- +lastproofread: '2026-02-23' +--- + +(vpp-config-dataplane-cpu)= + +```{include} /_include/need_improvement.txt +``` + + +# VPP Dataplane CPU Configuration + +VPP can utilize multiple CPU cores for better packet processing +performance. Proper CPU configuration is essential for optimal +throughput and low latency. + +VPP CPU assignment is handled automatically. You specify how many CPU +cores VPP may use, and the system distributes them between the main +thread and worker threads. + +:::{important} +Review the system configuration settings page before changing CPU +settings: {doc}`system`. +::: + +If you don't configure CPU settings, VPP uses a single core for the +main thread and doesn't create worker threads. + +## CPU Configuration Parameters + +### `cpu-cores` + +This parameter defines the total number of CPU cores allocated to VPP. + +```{cfgcmd} set vpp settings resource-allocation cpu-cores \ +``` + +The system automatically assigns cores using the following rules: + +> - The first two CPU cores are always reserved for the operating system and +> other services. +> - The main VPP thread is assigned to the first available core after the +> reserved ones. +> - The remaining allocated cores are used for worker threads. + +For example: + +> - If cpu-cores is set to 1, VPP runs only a main thread. +> +> - If cpu-cores is set to 4, VPP uses: +> +> > - 1 core for the main thread +> > - 3 cores for worker threads + +Choose a value based on available hardware resources and expected +traffic load. Too few cores may limit performance, while too many can +negatively impact other system services. + +## Potential Issues and Troubleshooting + +Improper CPU configuration can lead to issues such as: + +- VPP underperformance when not enough cores are assigned, or kernel + underperformance when too many cores are assigned to VPP. +- Resource conflicts with other processes and services. + +Indicators of such issues are: + +- VPP or kernel forwarding performance is lower than expected +- Degraded performance of system components or services, such as DNS, + DHCP, and dynamic routing diff --git a/docs/vpp/configuration/dataplane/cpu.rst b/docs/vpp/configuration/dataplane/cpu.rst deleted file mode 100644 index 22158ce3..00000000 --- a/docs/vpp/configuration/dataplane/cpu.rst +++ /dev/null @@ -1,74 +0,0 @@ -:lastproofread: 2026-02-23 - -.. _vpp_config_dataplane_cpu: - -.. include:: /_include/need_improvement.txt - -############################### -VPP Dataplane CPU Configuration -############################### - -VPP can utilize multiple CPU cores for better packet processing -performance. Proper CPU configuration is essential for optimal -throughput and low latency. - -VPP CPU assignment is handled automatically. You specify how many CPU -cores VPP may use, and the system distributes them between the main -thread and worker threads. - -.. important:: - - Review the system configuration settings page before changing CPU - settings: :doc:`system`. - -If you don't configure CPU settings, VPP uses a single core for the -main thread and doesn't create worker threads. - -CPU Configuration Parameters -============================ - -``cpu-cores`` -^^^^^^^^^^^^^ - -This parameter defines the total number of CPU cores allocated to VPP. - -.. cfgcmd:: set vpp settings resource-allocation cpu-cores - -The system automatically assigns cores using the following rules: - - * The first two CPU cores are always reserved for the operating system and - other services. - - * The main VPP thread is assigned to the first available core after the - reserved ones. - - * The remaining allocated cores are used for worker threads. - -For example: - - * If cpu-cores is set to 1, VPP runs only a main thread. - - * If cpu-cores is set to 4, VPP uses: - - * 1 core for the main thread - - * 3 cores for worker threads - -Choose a value based on available hardware resources and expected -traffic load. Too few cores may limit performance, while too many can -negatively impact other system services. - -Potential Issues and Troubleshooting -==================================== - -Improper CPU configuration can lead to issues such as: - -- VPP underperformance when not enough cores are assigned, or kernel - underperformance when too many cores are assigned to VPP. -- Resource conflicts with other processes and services. - -Indicators of such issues are: - -- VPP or kernel forwarding performance is lower than expected -- Degraded performance of system components or services, such as DNS, - DHCP, and dynamic routing diff --git a/docs/vpp/configuration/dataplane/index.md b/docs/vpp/configuration/dataplane/index.md new file mode 100644 index 00000000..c9ad7746 --- /dev/null +++ b/docs/vpp/configuration/dataplane/index.md @@ -0,0 +1,36 @@ +--- +lastproofread: '2026-02-23' +--- + +(vpp-config-dataplane-index)= + +```{include} /_include/need_improvement.txt +``` + +# VPP Dataplane Core Configuration + +This section covers the core configuration options for the VPP dataplane in +VyOS. It includes settings for memory management, CPU allocation, hugepages, +and other essential parameters that influence the performance and behavior +of the VPP dataplane. + +Please review the general system configuration, before starting to configure +VPP. Without proper VyOS preconditions, VPP will not start or its efficiency +will be significantly degraded. + +```{toctree} +:includehidden: true +:maxdepth: 1 + +system +buffers +cpu +interface +ipsec +ipv6 +l2learn +lcp +logging +memory +unix +``` diff --git a/docs/vpp/configuration/dataplane/index.rst b/docs/vpp/configuration/dataplane/index.rst deleted file mode 100644 index 323d7588..00000000 --- a/docs/vpp/configuration/dataplane/index.rst +++ /dev/null @@ -1,35 +0,0 @@ -:lastproofread: 2026-02-23 - -.. _vpp_config_dataplane_index: - -.. include:: /_include/need_improvement.txt - -################################ -VPP Dataplane Core Configuration -################################ - -This section covers the core configuration options for the VPP dataplane in -VyOS. It includes settings for memory management, CPU allocation, hugepages, -and other essential parameters that influence the performance and behavior -of the VPP dataplane. - -Please review the general system configuration, before starting to configure -VPP. Without proper VyOS preconditions, VPP will not start or its efficiency -will be significantly degraded. - -.. toctree:: - :maxdepth: 1 - :includehidden: - - system - buffers - cpu - interface - ipsec - ipv6 - l2learn - lcp - logging - memory - unix - \ No newline at end of file diff --git a/docs/vpp/configuration/dataplane/interface.md b/docs/vpp/configuration/dataplane/interface.md new file mode 100644 index 00000000..02a65ff8 --- /dev/null +++ b/docs/vpp/configuration/dataplane/interface.md @@ -0,0 +1,104 @@ +--- +lastproofread: '2026-02-23' +--- + +(vpp-config-dataplane-interface)= + +```{include} /_include/need_improvement.txt +``` + + +# VPP Dataplane Interfaces Configuration + +Only Ethernet interfaces (physical or virtual) can be connected to the +VPP dataplane. Interfaces configured here act as a bridge between VPP +and the outside world, allowing VPP to send and receive network +packets. + +## Interface Configuration Parameters + +Interfaces connected to the VPP dataplane use the DPDK driver by default, +providing high performance and low latency. + +```{cfgcmd} set vpp settings interface \ +``` + +Some network interface cards (NICs) may not be compatible with the DPDK driver. + +### DPDK interface options + +This section shows how to configure DPDK-specific settings for an interface. + +```{cfgcmd} set vpp settings interface \ num-rx-queues \ +``` + +Specifies the number of receive queues for the interface. More queues +improve performance on multi-core systems by allowing parallel +processing of incoming packets. Each queue is assigned to a separate +CPU core. + +```{cfgcmd} set vpp settings interface \ num-tx-queues \ +``` + +Specifies the number of transmit queues for the interface. Similar to +receive queues, more transmit queues improve performance by enabling +parallel processing of outgoing packets. By default, the VPP Dataplane +has one TX queue per enabled CPU worker, or a single queue if no +workers are configured. + +:::{seealso} +{doc}`cpu` +::: +```{cfgcmd} set vpp settings interface \ num-rx-desc \ +``` + +Defines the size of each receive queue. Larger queue sizes accommodate +bursts of incoming traffic and reduce the likelihood of packet drops +during high traffic periods. + +```{cfgcmd} set vpp settings interface \ num-tx-desc \ +``` + +Defines the size of each transmit queue. Larger sizes help manage +bursts of outgoing traffic more effectively. + +## Global Interface Parameters + +(vpp-config-dataplane-interface-rx-mode)= + +### interface-rx-mode + +The `interface-rx-mode` parameter defines how VPP handles incoming +packets on interfaces. There are several modes available, each with its +own advantages and use cases: +- `interrupt`: In this mode, VPP relies on hardware interrupts to + notify it of incoming packets. This mode suits low to moderate + traffic loads and reduces CPU usage during idle periods. It is not + recommended for low-latency processing. Some NICs may not support + this mode. +- `polling`: In polling mode, VPP continuously checks the interface + for incoming packets. This mode is ideal for high-throughput + scenarios where low latency is critical, as it minimizes packet + waiting time. However, it can increase CPU usage, especially during + low traffic periods, as the polling process is always active. +- `adaptive`: Adaptive mode combines the benefits of interrupt and + polling modes. VPP starts in interrupt mode and switches to polling + mode when traffic load increases. + +```{cfgcmd} set vpp settings interface-rx-mode \ +``` + +Choose an rx-mode based on expected traffic patterns and performance +requirements of your network. + +## Potential Issues and Troubleshooting + +Improper interface configuration can lead to issues such as: + +- Failure to initialize the interface +- Poor performance due to suboptimal driver selection or settings + +Indicators of such issues are: + +- Failed commits after adding or modifying an interface settings +- Low throughput or high latency on the interface diff --git a/docs/vpp/configuration/dataplane/interface.rst b/docs/vpp/configuration/dataplane/interface.rst deleted file mode 100644 index e4556021..00000000 --- a/docs/vpp/configuration/dataplane/interface.rst +++ /dev/null @@ -1,102 +0,0 @@ -:lastproofread: 2026-02-23 - -.. _vpp_config_dataplane_interface: - -.. include:: /_include/need_improvement.txt - -###################################### -VPP Dataplane Interfaces Configuration -###################################### - -Only Ethernet interfaces (physical or virtual) can be connected to the -VPP dataplane. Interfaces configured here act as a bridge between VPP -and the outside world, allowing VPP to send and receive network -packets. - - -Interface Configuration Parameters -================================== - -Interfaces connected to the VPP dataplane use the DPDK driver by default, -providing high performance and low latency. - -.. cfgcmd:: set vpp settings interface - -Some network interface cards (NICs) may not be compatible with the DPDK driver. - -DPDK interface options ----------------------- - -This section shows how to configures DPDK-specific settings for an interface. - -.. cfgcmd:: set vpp settings interface num-rx-queues - -Specifies the number of receive queues for the interface. More queues -improve performance on multi-core systems by allowing parallel -processing of incoming packets. Each queue is assigned to a separate -CPU core. - -.. cfgcmd:: set vpp settings interface num-tx-queues - -Specifies the number of transmit queues for the interface. Similar to -receive queues, more transmit queues improve performance by enabling -parallel processing of outgoing packets. By default, the VPP Dataplane -has one TX queue per enabled CPU worker, or a single queue if no -workers are configured. - -.. seealso:: :doc:`cpu` - -.. cfgcmd:: set vpp settings interface num-rx-desc - -Defines the size of each receive queue. Larger queue sizes accommodate -bursts of incoming traffic and reduce the likelihood of packet drops -during high traffic periods. - -.. cfgcmd:: set vpp settings interface num-tx-desc - -Defines the size of each transmit queue. Larger sizes help manage -bursts of outgoing traffic more effectively. - -Global Interface Parameters -=========================== - -.. _vpp_config_dataplane_interface_rx_mode: - -interface-rx-mode ------------------ - -The ``interface-rx-mode`` parameter defines how VPP handles incoming -packets on interfaces. There are several modes available, each with its -own advantages and use cases: - -- ``interrupt``: In this mode, VPP relies on hardware interrupts to - notify it of incoming packets. This mode suits low to moderate - traffic loads and reduces CPU usage during idle periods. It is not - recommended for low-latency processing. Some NICs may not support - this mode. -- ``polling``: In polling mode, VPP continuously checks the interface - for incoming packets. This mode is ideal for high-throughput - scenarios where low latency is critical, as it minimizes packet - waiting time. However, it can increase CPU usage, especially during - low traffic periods, as the polling process is always active. -- ``adaptive``: Adaptive mode combines the benefits of interrupt and - polling modes. VPP starts in interrupt mode and switches to polling - mode when traffic load increases. - -.. cfgcmd:: set vpp settings interface-rx-mode - -Choose an rx-mode based on expected traffic patterns and performance -requirements of your network. - -Potential Issues and Troubleshooting -==================================== - -Improper interface configuration can lead to issues such as: - -- Failure to initialize the interface -- Poor performance due to suboptimal driver selection or settings - -Indicators of such issues are: - -- Failed commits after adding or modifying an interface settings -- Low throughput or high latency on the interface diff --git a/docs/vpp/configuration/dataplane/ipsec.md b/docs/vpp/configuration/dataplane/ipsec.md new file mode 100644 index 00000000..0a66221f --- /dev/null +++ b/docs/vpp/configuration/dataplane/ipsec.md @@ -0,0 +1,74 @@ +--- +lastproofread: '2026-02-23' +--- + +(vpp-config-dataplane-ipsec)= + +```{include} /_include/need_improvement.txt +``` + + +# VPP IPsec Configuration + +VPP supports IPsec (Internet Protocol Security) offloading from the +kernel, which speeds up cryptographic operations by leveraging VPP's +high-performance packet processing capabilities. + +IPsec does not require any specific configuration on VPP side. If both +sources and destinations of the IPsec traffic are reachable via VPP +interfaces, VPP will automatically offload the IPsec processing from +the kernel. IPsec tunnels are configured in the VPN configuration +section, see {ref}`ipsec_general`. + +## IPsec Configuration Parameters + +### enable IPsec acceleration + +When VPP is used for offloading IPsec, it creates a virtual interface to +connect to peers. The interface type is always 'ipsec', which is used for +IPsec tunnels. + +```{cfgcmd} set vpp settings ipsec-acceleration +``` + +Enabling this option allows VPP to handle IPsec traffic more efficiently by +offloading processing from the kernel. + +### netlink + +VPP uses netlink to receive IPsec event messages from the kernel. Proper +settings of the following parameters are crucial for ensuring that VPP can +process all such messages: + +```{cfgcmd} set vpp settings lcp netlink batch-delay-ms \ +``` + +This parameter specifies the delay in milliseconds between processing +batch netlink messages. + +```{cfgcmd} set vpp settings lcp netlink batch-size \ +``` + +This parameter specifies the maximum number of netlink messages to +process in a single batch. + +```{cfgcmd} set vpp settings lcp netlink rx-buffer-size \ +``` + +This parameter specifies the size of the receive buffer for netlink +socket. If you expect to offload many IPsec tunnels or get frequent and +intensive rekeying, you may need to increase this value. + +:::{note} +IPsec uses the same netlink parameters as LCP, so tuning them +affects both LCP and IPsec processing. +::: + +## Potential Issues and Troubleshooting + +Improper IPsec configuration can lead to various issues, including: + +- Failure to offload IPsec tunnels to VPP +- Lost IPsec event messages due to insufficient netlink buffer size or + batch settings +- IPsec states or SAs are not synchronized between kernel and VPP diff --git a/docs/vpp/configuration/dataplane/ipsec.rst b/docs/vpp/configuration/dataplane/ipsec.rst deleted file mode 100644 index 31734c86..00000000 --- a/docs/vpp/configuration/dataplane/ipsec.rst +++ /dev/null @@ -1,72 +0,0 @@ -:lastproofread: 2026-02-23 - -.. _vpp_config_dataplane_ipsec: - -.. include:: /_include/need_improvement.txt - -####################### -VPP IPsec Configuration -####################### - -VPP supports IPsec (Internet Protocol Security) offloading from the -kernel, which speeds up cryptographic operations by leveraging VPP's -high-performance packet processing capabilities. - -IPsec does not require any specific configuration on VPP side. If both -sources and destinations of the IPsec traffic are reachable via VPP -interfaces, VPP will automatically offload the IPsec processing from -the kernel. IPsec tunnels are configured in the VPN configuration -section, see :ref:`ipsec_general`. - -IPsec Configuration Parameters -============================== - -enable IPsec acceleration -^^^^^^^^^^^^^^^^^^^^^^^^^ - -When VPP is used for offloading IPsec, it creates a virtual interface to -connect to peers. The interface type is always 'ipsec', which is used for -IPsec tunnels. - -.. cfgcmd:: set vpp settings ipsec-acceleration - -Enabling this option allows VPP to handle IPsec traffic more efficiently by -offloading processing from the kernel. - -netlink -^^^^^^^ - -VPP uses netlink to receive IPsec event messages from the kernel. Proper -settings of the following parameters are crucial for ensuring that VPP can -process all such messages: - -.. cfgcmd:: set vpp settings lcp netlink batch-delay-ms - -This parameter specifies the delay in milliseconds between processing -batch netlink messages. - -.. cfgcmd:: set vpp settings lcp netlink batch-size - -This parameter specifies the maximum number of netlink messages to -process in a single batch. - -.. cfgcmd:: set vpp settings lcp netlink rx-buffer-size - -This parameter specifies the size of the receive buffer for netlink -socket. If you expect to offload many IPsec tunnels or get frequent and -intensive rekeying, you may need to increase this value. - -.. note:: - - IPsec uses the same netlink parameters as LCP, so tuning them - affects both LCP and IPsec processing. - -Potential Issues and Troubleshooting -==================================== - -Improper IPsec configuration can lead to various issues, including: - -- Failure to offload IPsec tunnels to VPP -- Lost IPsec event messages due to insufficient netlink buffer size or - batch settings -- IPsec states or SAs are not synchronized between kernel and VPP diff --git a/docs/vpp/configuration/dataplane/ipv6.md b/docs/vpp/configuration/dataplane/ipv6.md new file mode 100644 index 00000000..5f2ba3c5 --- /dev/null +++ b/docs/vpp/configuration/dataplane/ipv6.md @@ -0,0 +1,46 @@ +--- +lastproofread: '2026-02-26' +--- + +(vpp-config-dataplane-ipv6)= + +```{include} /_include/need_improvement.txt +``` + + +# VPP IPv6 Configuration + +VPP lets you configure resources allocated for IPv6 traffic processing +independently from IPv4. This helps ensure that in networks without IPv6 +traffic, resources are not wasted. If IPv6 traffic is present, especially +with large routing tables, you must allocate additional resources for IPv6 +processing to keep the dataplane stable. + +You can configure two main resources for IPv6 traffic processing: + +```{cfgcmd} set vpp settings resource-allocation ipv6 hash-buckets \ +``` + +This parameter configures the number of hash buckets used for IPv6 +routing. If you have a large IPv6 routing table, you may need to increase +this value to ensure efficient routing table performance and fast lookups. + +```{cfgcmd} set vpp settings resource-allocation ipv6 heap-size \ +``` + +This parameter configures the heap size used for IPv6 forwarding. If you +have a large IPv6 routing table, you may need to increase this value to +ensure the routing table can accommodate all routes. + +## Potential Issues and Troubleshooting + +Improper IPv6 configuration can lead to various issues, including: + +- Inefficient, slow routing table lookups and traffic processing due to + insufficient hash buckets +- Dataplane crashes or instability due to insufficient heap size when + handling a large number of IPv6 routes +- Overall dataplane instability when handling IPv6 traffic + +Consider increasing configuration values if you experience issues with +IPv6 traffic processing or if you have a large IPv6 routing table. diff --git a/docs/vpp/configuration/dataplane/ipv6.rst b/docs/vpp/configuration/dataplane/ipv6.rst deleted file mode 100644 index 94f38476..00000000 --- a/docs/vpp/configuration/dataplane/ipv6.rst +++ /dev/null @@ -1,43 +0,0 @@ -:lastproofread: 2026-02-26 - -.. _vpp_config_dataplane_ipv6: - -.. include:: /_include/need_improvement.txt - -###################### -VPP IPv6 Configuration -###################### - -VPP lets you configure resources allocated for IPv6 traffic processing -independently from IPv4. This helps ensure that in networks without IPv6 -traffic, resources are not wasted. If IPv6 traffic is present, especially -with large routing tables, you must allocate additional resources for IPv6 -processing to keep the dataplane stable. - -You can configure two main resources for IPv6 traffic processing: - -.. cfgcmd:: set vpp settings resource-allocation ipv6 hash-buckets - -This parameter configures the number of hash buckets used for IPv6 -routing. If you have a large IPv6 routing table, you may need to increase -this value to ensure efficient routing table performance and fast lookups. - -.. cfgcmd:: set vpp settings resource-allocation ipv6 heap-size - -This parameter configures the heap size used for IPv6 forwarding. If you -have a large IPv6 routing table, you may need to increase this value to -ensure the routing table can accommodate all routes. - -Potential Issues and Troubleshooting -==================================== - -Improper IPv6 configuration can lead to various issues, including: - -- Inefficient, slow routing table lookups and traffic processing due to - insufficient hash buckets -- Dataplane crashes or instability due to insufficient heap size when - handling a large number of IPv6 routes -- Overall dataplane instability when handling IPv6 traffic - -Consider increasing configuration values if you experience issues with -IPv6 traffic processing or if you have a large IPv6 routing table. diff --git a/docs/vpp/configuration/dataplane/l2learn.md b/docs/vpp/configuration/dataplane/l2learn.md new file mode 100644 index 00000000..2ce572a1 --- /dev/null +++ b/docs/vpp/configuration/dataplane/l2learn.md @@ -0,0 +1,35 @@ +--- +lastproofread: '2026-02-26' +--- + +(vpp-config-dataplane-l2learn)= + +```{include} /_include/need_improvement.txt +``` + +# VPP L2LEARN Configuration + +When VPP dataplane connects to an L2 domain, it learns MAC addresses of +devices in the domain. By default, the number of MAC addresses it can +learn is limited. + +You can configure the limit using the following command: + +```{cfgcmd} set vpp settings resource-allocation mac-limit \ +``` + +This parameter sets the maximum number of MAC addresses that can be +learned in the L2 domain. If you have many devices, you may need to +increase this limit to ensure VPP learns all MAC addresses. + +## Potential Issues and Troubleshooting + +Improper L2LEARN configuration can lead to various issues, including: + +- MAC address learning failure in the L2 domain if the limit is set too + low +- Increased packet loss or latency for devices that aren't learned +- Overall dataplane instability when handling L2 traffic + +Consider increasing the L2LEARN limit if you experience issues with MAC +address learning or if you have many devices in the L2 domain. diff --git a/docs/vpp/configuration/dataplane/l2learn.rst b/docs/vpp/configuration/dataplane/l2learn.rst deleted file mode 100644 index b204c41f..00000000 --- a/docs/vpp/configuration/dataplane/l2learn.rst +++ /dev/null @@ -1,34 +0,0 @@ -:lastproofread: 2026-02-26 - -.. _vpp_config_dataplane_l2learn: - -.. include:: /_include/need_improvement.txt - -######################### -VPP L2LEARN Configuration -######################### - -When VPP dataplane connects to an L2 domain, it learns MAC addresses of -devices in the domain. By default, the number of MAC addresses it can -learn is limited. - -You can configure the limit using the following command: - -.. cfgcmd:: set vpp settings resource-allocation mac-limit - -This parameter sets the maximum number of MAC addresses that can be -learned in the L2 domain. If you have many devices, you may need to -increase this limit to ensure VPP learns all MAC addresses. - -Potential Issues and Troubleshooting -==================================== - -Improper L2LEARN configuration can lead to various issues, including: - -- MAC address learning failure in the L2 domain if the limit is set too - low -- Increased packet loss or latency for devices that aren't learned -- Overall dataplane instability when handling L2 traffic - -Consider increasing the L2LEARN limit if you experience issues with MAC -address learning or if you have many devices in the L2 domain. diff --git a/docs/vpp/configuration/dataplane/lcp.md b/docs/vpp/configuration/dataplane/lcp.md new file mode 100644 index 00000000..a68247e1 --- /dev/null +++ b/docs/vpp/configuration/dataplane/lcp.md @@ -0,0 +1,46 @@ +--- +lastproofread: '2026-02-26' +--- + +```{include} /_include/need_improvement.txt +``` + +# VPP LCP Configuration + +Linux Control Plane (LCP) is a core component of VPP that lets you +offload various control plane functions to the Linux kernel. LCP provides +seamless integration with other VyOS components, letting you use system +components like DHCP clients and routing daemons together with the VPP +dataplane. + +VPP integration in VyOS relies heavily on LCP. Almost all control plane +functions are handled by other daemons and services, while VPP handles +high-performance packet forwarding exclusively. This approach also reduces +VPP management processing load, improving overall dataplane performance and +stability. + +VyOS integrates the kernel and VPP routing tables uniquely. By default, +all routes, even those not directly connected to VPP interfaces, are +imported from the kernel routing table to the VPP routing table, pointing +to the kernel. This lets you forward traffic to any destination known to +the kernel, even if VPP doesn't have a route to that destination. + +However, in some scenarios this behavior may not be desired. For example, +if you have many routes in the kernel routing table not directly connected +to VPP interfaces, and you don't need forwarding between those +destinations and destinations reachable via VPP, you can disable this +behavior using the following command: +(vpp_config_dataplane_lcp_ignore_kernel_routes)= + +```{cfgcmd} set vpp settings ignore-kernel-routes +``` + +Pay attention that disabling this option leads to loss of connectivity to +destinations if there are no direct routes in VPP routing table. + +## Potential Issues and Troubleshooting + +Disabling kernel route import can result in: + +- Loss of connectivity to certain destinations if kernel routes are ignored +- Incomplete route synchronization between the kernel and VPP diff --git a/docs/vpp/configuration/dataplane/lcp.rst b/docs/vpp/configuration/dataplane/lcp.rst deleted file mode 100644 index cb39439a..00000000 --- a/docs/vpp/configuration/dataplane/lcp.rst +++ /dev/null @@ -1,48 +0,0 @@ -:lastproofread: 2026-02-26 - -.. _vpp_config_dataplane_lcp: - -.. include:: /_include/need_improvement.txt - -##################### -VPP LCP Configuration -##################### - -Linux Control Plane (LCP) is a core component of VPP that lets you -offload various control plane functions to the Linux kernel. LCP provides -seamless integration with other VyOS components, letting you use system -components like DHCP clients and routing daemons together with the VPP -dataplane. - -VPP integration in VyOS relies heavily on LCP. Almost all control plane -functions are handled by other daemons and services, while VPP handles -high-performance packet forwarding exclusively. This approach also reduces -VPP management processing load, improving overall dataplane performance and -stability. - -VyOS integrates the kernel and VPP routing tables uniquely. By default, -all routes, even those not directly connected to VPP interfaces, are -imported from the kernel routing table to the VPP routing table, pointing -to the kernel. This lets you forward traffic to any destination known to -the kernel, even if VPP doesn't have a route to that destination. - -However, in some scenarios this behavior may not be desired. For example, -if you have many routes in the kernel routing table not directly connected -to VPP interfaces, and you don't need forwarding between those -destinations and destinations reachable via VPP, you can disable this -behavior using the following command: - -.. _vpp_config_dataplane_lcp_ignore-kernel-routes: - -.. cfgcmd:: set vpp settings ignore-kernel-routes - -Pay attention that disabling this option leads to loss of connectivity to -destinations if there are no direct routes in VPP routing table. - -Potential Issues and Troubleshooting -==================================== - -Disabling kernel route import can result in: - -- Loss of connectivity to certain destinations if kernel routes are ignored -- Incomplete route synchronization between the kernel and VPP diff --git a/docs/vpp/configuration/dataplane/logging.md b/docs/vpp/configuration/dataplane/logging.md new file mode 100644 index 00000000..50e6277b --- /dev/null +++ b/docs/vpp/configuration/dataplane/logging.md @@ -0,0 +1,59 @@ +--- +lastproofread: '2026-02-27' +--- + +(vpp-config-dataplane-logging)= + +```{include} /_include/need_improvement.txt +``` + + +# VPP Logging Configuration + +VPP logging is an important part of monitoring and troubleshooting +the performance and behavior of the VPP dataplane. + +VPP stores logs in two places: +- `/var/log/vpp.log` — This file contains logs related to daemon + startup and logs of commands executed directly via VPP CLI. Pay + attention: VyOS does not use VPP CLI for configuration, so this log + will not contain any configuration changes made via VyOS CLI and will + not be informative in most cases. +- System journal — contains logs related to the VPP daemon work, + including errors, warnings, and informational messages. It is the + main destination of logs generated by VPP. + +Logging detail level can be configured via the next command: + +```{cfgcmd} set vpp settings logging default-level \ +``` + +Where `` can be one of the following: + +- `emerg` (Emergency) - System is unusable. +- `alert` (Alert) - Immediate action required. +- `crit` (Critical) - Critical conditions. +- `error` (Error) - Error conditions. +- `warn` (Warning) - Warning conditions. +- `notice` (Notice) - Normal but significant. +- `info` (Informational) - Routine informational messages. +- `debug` (Debug) - Detailed debugging messages. +- `disabled` (Disabled) - Logging disabled. + +It is recommended to set logging level to `debug` only for +troubleshooting purposes, as it can generate a large volume of log +data. For regular operation, a level of `info` or `warn` is usually +sufficient. + +## Troubleshooting + +Improper logging configuration can lead to various issues, including: + +- Excessive log file sizes if the logging level is set too high + (for example, `debug`). +- Missing critical information if the logging level is set too low + (for example, `alert`). +- Performance degradation due to excessive logging overhead + +Consider adjusting the logging level if you experience issues mentioned +above. diff --git a/docs/vpp/configuration/dataplane/logging.rst b/docs/vpp/configuration/dataplane/logging.rst deleted file mode 100644 index 70d57028..00000000 --- a/docs/vpp/configuration/dataplane/logging.rst +++ /dev/null @@ -1,58 +0,0 @@ -:lastproofread: 2026-02-27 - -.. _vpp_config_dataplane_logging: - -.. include:: /_include/need_improvement.txt - -######################### -VPP Logging Configuration -######################### - -VPP logging is an important part of monitoring and troubleshooting -the performance and behavior of the VPP dataplane. - -VPP stores logs in two places: - -- ``/var/log/vpp.log`` — This file contains logs related to daemon - startup and logs of commands executed directly via VPP CLI. Pay - attention: VyOS does not use VPP CLI for configuration, so this log - will not contain any configuration changes made via VyOS CLI and will - not be informative in most cases. -- System journal — contains logs related to the VPP daemon work, - including errors, warnings, and informational messages. It is the - main destination of logs generated by VPP. - -Logging detail level can be configured via the next command: - -.. cfgcmd:: set vpp settings logging default-level - -Where ```` can be one of the following: - -- ``emerg`` (Emergency) - System is unusable. -- ``alert`` (Alert) - Immediate action required. -- ``crit`` (Critical) - Critical conditions. -- ``error`` (Error) - Error conditions. -- ``warn`` (Warning) - Warning conditions. -- ``notice`` (Notice) - Normal but significant. -- ``info`` (Informational) - Routine informational messages. -- ``debug`` (Debug) - Detailed debugging messages. -- ``disabled`` (Disabled) - Logging disabled. - -It is recommended to set logging level to ``debug`` only for -troubleshooting purposes, as it can generate a large volume of log -data. For regular operation, a level of ``info`` or ``warn`` is usually -sufficient. - -Troubleshooting -=============== - -Improper logging configuration can lead to various issues, including: - -- Excessive log file sizes if the logging level is set too high - (for example, ``debug``). -- Missing critical information if the logging level is set too low - (for example, ``alert``). -- Performance degradation due to excessive logging overhead - -Consider adjusting the logging level if you experience issues mentioned -above. diff --git a/docs/vpp/configuration/dataplane/md-buffers.md b/docs/vpp/configuration/dataplane/md-buffers.md deleted file mode 100644 index c9e38a54..00000000 --- a/docs/vpp/configuration/dataplane/md-buffers.md +++ /dev/null @@ -1,102 +0,0 @@ ---- -lastproofread: '2026-02-23' ---- - -(vpp-config-dataplane-buffers)= - -```{include} /_include/need_improvement.txt -``` - - -# VPP Dataplane Buffers Configuration - -Buffers are essential for handling network packets efficiently. Proper -configuration enhances performance and reliability, and is mandatory for -VPP to work. Buffers temporarily store packets during processing. Therefore, -their configuration must be in sync with NIC configuration, CPU threads, and -overall system resources. - -:::{important} -VPP buffers are allocated from the physical memory pool (`physmem`). The -total amount of memory available for buffer allocation is controlled by the -`physmem-max-size` setting, while the buffer configuration parameters -below control how that memory is used for buffer allocation. - -See {ref}`VPP Physical Memory Configuration ` -for details on configuring `physmem`. -::: - -## Buffer Configuration Parameters - -The following parameters can be configured for VPP buffers: - -### buffers-per-numa - -Number of buffers allocated per NUMA node. This setting optimizes -memory access patterns for multi-CPU systems. - -Typically, you need to tune this value if: -- The system has many interfaces -- NICs have many queues -- NICs have large descriptor sizes - -Set this value carefully to balance memory usage and performance. - -```{cfgcmd} set vpp settings resource-allocation buffers buffers-per-numa \ -``` - -The common approach for the calculation is to use the formula: - -```none -buffers-per-numa = (num-rx-queues * num-rx-desc) + (num-tx-queues * num-tx-desc) -``` - -Calculate this formula for each NIC and sum the results. Multiply the -total by 2.5 to get the minimum recommended value for -`buffers-per-numa`. - -Avoid setting this value too low to prevent packet drops. - -### data-size - -This value sets how much payload data can be stored in a single buffer -allocated by VPP. Larger values reduce buffer chains for large packets, -while smaller values conserve memory for environments handling mostly -small packets. - -```{cfgcmd} set vpp settings resource-allocation buffers data-size \ -``` - -Optimal size depends on the typical packet size in your network. If -unsure, use the largest MTU in your network plus overhead (for example, -128 bytes). - -### page-size - -A memory pages type used for buffer allocation. Common values are 4K, 2M, or 1G. - -Use page sizes configured in your system settings. - -```{cfgcmd} set vpp settings resource-allocation buffers page-size \ -``` - - -## Potential Issues and Troubleshooting - -Improper buffer configuration can lead to issues such as: - -- Increased latency and packet loss -- Inefficient CPU utilization -- Interface initialization failures - -Indicators of such issues are: - -- Errors during interfaces initialization in VPP logs -- Packet drops observed in VPP statistics - -To troubleshoot buffer-related issues, consider the following steps: - -- Review VPP logs for errors related to buffer allocation. Look for - error `-5` messages. -- Tune available buffers by adjusting the `buffers-per-numa` and - `data-size` parameters. diff --git a/docs/vpp/configuration/dataplane/md-cpu.md b/docs/vpp/configuration/dataplane/md-cpu.md deleted file mode 100644 index d92f6587..00000000 --- a/docs/vpp/configuration/dataplane/md-cpu.md +++ /dev/null @@ -1,71 +0,0 @@ ---- -lastproofread: '2026-02-23' ---- - -(vpp-config-dataplane-cpu)= - -```{include} /_include/need_improvement.txt -``` - - -# VPP Dataplane CPU Configuration - -VPP can utilize multiple CPU cores for better packet processing -performance. Proper CPU configuration is essential for optimal -throughput and low latency. - -VPP CPU assignment is handled automatically. You specify how many CPU -cores VPP may use, and the system distributes them between the main -thread and worker threads. - -:::{important} -Review the system configuration settings page before changing CPU -settings: {doc}`system`. -::: - -If you don't configure CPU settings, VPP uses a single core for the -main thread and doesn't create worker threads. - -## CPU Configuration Parameters - -### `cpu-cores` - -This parameter defines the total number of CPU cores allocated to VPP. - -```{cfgcmd} set vpp settings resource-allocation cpu-cores \ -``` - -The system automatically assigns cores using the following rules: - -> - The first two CPU cores are always reserved for the operating system and -> other services. -> - The main VPP thread is assigned to the first available core after the -> reserved ones. -> - The remaining allocated cores are used for worker threads. - -For example: - -> - If cpu-cores is set to 1, VPP runs only a main thread. -> -> - If cpu-cores is set to 4, VPP uses: -> -> > - 1 core for the main thread -> > - 3 cores for worker threads - -Choose a value based on available hardware resources and expected -traffic load. Too few cores may limit performance, while too many can -negatively impact other system services. - -## Potential Issues and Troubleshooting - -Improper CPU configuration can lead to issues such as: - -- VPP underperformance when not enough cores are assigned, or kernel - underperformance when too many cores are assigned to VPP. -- Resource conflicts with other processes and services. - -Indicators of such issues are: - -- VPP or kernel forwarding performance is lower than expected -- Degraded performance of system components or services, such as DNS, - DHCP, and dynamic routing diff --git a/docs/vpp/configuration/dataplane/md-index.md b/docs/vpp/configuration/dataplane/md-index.md deleted file mode 100644 index c9ad7746..00000000 --- a/docs/vpp/configuration/dataplane/md-index.md +++ /dev/null @@ -1,36 +0,0 @@ ---- -lastproofread: '2026-02-23' ---- - -(vpp-config-dataplane-index)= - -```{include} /_include/need_improvement.txt -``` - -# VPP Dataplane Core Configuration - -This section covers the core configuration options for the VPP dataplane in -VyOS. It includes settings for memory management, CPU allocation, hugepages, -and other essential parameters that influence the performance and behavior -of the VPP dataplane. - -Please review the general system configuration, before starting to configure -VPP. Without proper VyOS preconditions, VPP will not start or its efficiency -will be significantly degraded. - -```{toctree} -:includehidden: true -:maxdepth: 1 - -system -buffers -cpu -interface -ipsec -ipv6 -l2learn -lcp -logging -memory -unix -``` diff --git a/docs/vpp/configuration/dataplane/md-interface.md b/docs/vpp/configuration/dataplane/md-interface.md deleted file mode 100644 index 02a65ff8..00000000 --- a/docs/vpp/configuration/dataplane/md-interface.md +++ /dev/null @@ -1,104 +0,0 @@ ---- -lastproofread: '2026-02-23' ---- - -(vpp-config-dataplane-interface)= - -```{include} /_include/need_improvement.txt -``` - - -# VPP Dataplane Interfaces Configuration - -Only Ethernet interfaces (physical or virtual) can be connected to the -VPP dataplane. Interfaces configured here act as a bridge between VPP -and the outside world, allowing VPP to send and receive network -packets. - -## Interface Configuration Parameters - -Interfaces connected to the VPP dataplane use the DPDK driver by default, -providing high performance and low latency. - -```{cfgcmd} set vpp settings interface \ -``` - -Some network interface cards (NICs) may not be compatible with the DPDK driver. - -### DPDK interface options - -This section shows how to configure DPDK-specific settings for an interface. - -```{cfgcmd} set vpp settings interface \ num-rx-queues \ -``` - -Specifies the number of receive queues for the interface. More queues -improve performance on multi-core systems by allowing parallel -processing of incoming packets. Each queue is assigned to a separate -CPU core. - -```{cfgcmd} set vpp settings interface \ num-tx-queues \ -``` - -Specifies the number of transmit queues for the interface. Similar to -receive queues, more transmit queues improve performance by enabling -parallel processing of outgoing packets. By default, the VPP Dataplane -has one TX queue per enabled CPU worker, or a single queue if no -workers are configured. - -:::{seealso} -{doc}`cpu` -::: -```{cfgcmd} set vpp settings interface \ num-rx-desc \ -``` - -Defines the size of each receive queue. Larger queue sizes accommodate -bursts of incoming traffic and reduce the likelihood of packet drops -during high traffic periods. - -```{cfgcmd} set vpp settings interface \ num-tx-desc \ -``` - -Defines the size of each transmit queue. Larger sizes help manage -bursts of outgoing traffic more effectively. - -## Global Interface Parameters - -(vpp-config-dataplane-interface-rx-mode)= - -### interface-rx-mode - -The `interface-rx-mode` parameter defines how VPP handles incoming -packets on interfaces. There are several modes available, each with its -own advantages and use cases: -- `interrupt`: In this mode, VPP relies on hardware interrupts to - notify it of incoming packets. This mode suits low to moderate - traffic loads and reduces CPU usage during idle periods. It is not - recommended for low-latency processing. Some NICs may not support - this mode. -- `polling`: In polling mode, VPP continuously checks the interface - for incoming packets. This mode is ideal for high-throughput - scenarios where low latency is critical, as it minimizes packet - waiting time. However, it can increase CPU usage, especially during - low traffic periods, as the polling process is always active. -- `adaptive`: Adaptive mode combines the benefits of interrupt and - polling modes. VPP starts in interrupt mode and switches to polling - mode when traffic load increases. - -```{cfgcmd} set vpp settings interface-rx-mode \ -``` - -Choose an rx-mode based on expected traffic patterns and performance -requirements of your network. - -## Potential Issues and Troubleshooting - -Improper interface configuration can lead to issues such as: - -- Failure to initialize the interface -- Poor performance due to suboptimal driver selection or settings - -Indicators of such issues are: - -- Failed commits after adding or modifying an interface settings -- Low throughput or high latency on the interface diff --git a/docs/vpp/configuration/dataplane/md-ipsec.md b/docs/vpp/configuration/dataplane/md-ipsec.md deleted file mode 100644 index 0a66221f..00000000 --- a/docs/vpp/configuration/dataplane/md-ipsec.md +++ /dev/null @@ -1,74 +0,0 @@ ---- -lastproofread: '2026-02-23' ---- - -(vpp-config-dataplane-ipsec)= - -```{include} /_include/need_improvement.txt -``` - - -# VPP IPsec Configuration - -VPP supports IPsec (Internet Protocol Security) offloading from the -kernel, which speeds up cryptographic operations by leveraging VPP's -high-performance packet processing capabilities. - -IPsec does not require any specific configuration on VPP side. If both -sources and destinations of the IPsec traffic are reachable via VPP -interfaces, VPP will automatically offload the IPsec processing from -the kernel. IPsec tunnels are configured in the VPN configuration -section, see {ref}`ipsec_general`. - -## IPsec Configuration Parameters - -### enable IPsec acceleration - -When VPP is used for offloading IPsec, it creates a virtual interface to -connect to peers. The interface type is always 'ipsec', which is used for -IPsec tunnels. - -```{cfgcmd} set vpp settings ipsec-acceleration -``` - -Enabling this option allows VPP to handle IPsec traffic more efficiently by -offloading processing from the kernel. - -### netlink - -VPP uses netlink to receive IPsec event messages from the kernel. Proper -settings of the following parameters are crucial for ensuring that VPP can -process all such messages: - -```{cfgcmd} set vpp settings lcp netlink batch-delay-ms \ -``` - -This parameter specifies the delay in milliseconds between processing -batch netlink messages. - -```{cfgcmd} set vpp settings lcp netlink batch-size \ -``` - -This parameter specifies the maximum number of netlink messages to -process in a single batch. - -```{cfgcmd} set vpp settings lcp netlink rx-buffer-size \ -``` - -This parameter specifies the size of the receive buffer for netlink -socket. If you expect to offload many IPsec tunnels or get frequent and -intensive rekeying, you may need to increase this value. - -:::{note} -IPsec uses the same netlink parameters as LCP, so tuning them -affects both LCP and IPsec processing. -::: - -## Potential Issues and Troubleshooting - -Improper IPsec configuration can lead to various issues, including: - -- Failure to offload IPsec tunnels to VPP -- Lost IPsec event messages due to insufficient netlink buffer size or - batch settings -- IPsec states or SAs are not synchronized between kernel and VPP diff --git a/docs/vpp/configuration/dataplane/md-ipv6.md b/docs/vpp/configuration/dataplane/md-ipv6.md deleted file mode 100644 index 5f2ba3c5..00000000 --- a/docs/vpp/configuration/dataplane/md-ipv6.md +++ /dev/null @@ -1,46 +0,0 @@ ---- -lastproofread: '2026-02-26' ---- - -(vpp-config-dataplane-ipv6)= - -```{include} /_include/need_improvement.txt -``` - - -# VPP IPv6 Configuration - -VPP lets you configure resources allocated for IPv6 traffic processing -independently from IPv4. This helps ensure that in networks without IPv6 -traffic, resources are not wasted. If IPv6 traffic is present, especially -with large routing tables, you must allocate additional resources for IPv6 -processing to keep the dataplane stable. - -You can configure two main resources for IPv6 traffic processing: - -```{cfgcmd} set vpp settings resource-allocation ipv6 hash-buckets \ -``` - -This parameter configures the number of hash buckets used for IPv6 -routing. If you have a large IPv6 routing table, you may need to increase -this value to ensure efficient routing table performance and fast lookups. - -```{cfgcmd} set vpp settings resource-allocation ipv6 heap-size \ -``` - -This parameter configures the heap size used for IPv6 forwarding. If you -have a large IPv6 routing table, you may need to increase this value to -ensure the routing table can accommodate all routes. - -## Potential Issues and Troubleshooting - -Improper IPv6 configuration can lead to various issues, including: - -- Inefficient, slow routing table lookups and traffic processing due to - insufficient hash buckets -- Dataplane crashes or instability due to insufficient heap size when - handling a large number of IPv6 routes -- Overall dataplane instability when handling IPv6 traffic - -Consider increasing configuration values if you experience issues with -IPv6 traffic processing or if you have a large IPv6 routing table. diff --git a/docs/vpp/configuration/dataplane/md-l2learn.md b/docs/vpp/configuration/dataplane/md-l2learn.md deleted file mode 100644 index 2ce572a1..00000000 --- a/docs/vpp/configuration/dataplane/md-l2learn.md +++ /dev/null @@ -1,35 +0,0 @@ ---- -lastproofread: '2026-02-26' ---- - -(vpp-config-dataplane-l2learn)= - -```{include} /_include/need_improvement.txt -``` - -# VPP L2LEARN Configuration - -When VPP dataplane connects to an L2 domain, it learns MAC addresses of -devices in the domain. By default, the number of MAC addresses it can -learn is limited. - -You can configure the limit using the following command: - -```{cfgcmd} set vpp settings resource-allocation mac-limit \ -``` - -This parameter sets the maximum number of MAC addresses that can be -learned in the L2 domain. If you have many devices, you may need to -increase this limit to ensure VPP learns all MAC addresses. - -## Potential Issues and Troubleshooting - -Improper L2LEARN configuration can lead to various issues, including: - -- MAC address learning failure in the L2 domain if the limit is set too - low -- Increased packet loss or latency for devices that aren't learned -- Overall dataplane instability when handling L2 traffic - -Consider increasing the L2LEARN limit if you experience issues with MAC -address learning or if you have many devices in the L2 domain. diff --git a/docs/vpp/configuration/dataplane/md-lcp.md b/docs/vpp/configuration/dataplane/md-lcp.md deleted file mode 100644 index a68247e1..00000000 --- a/docs/vpp/configuration/dataplane/md-lcp.md +++ /dev/null @@ -1,46 +0,0 @@ ---- -lastproofread: '2026-02-26' ---- - -```{include} /_include/need_improvement.txt -``` - -# VPP LCP Configuration - -Linux Control Plane (LCP) is a core component of VPP that lets you -offload various control plane functions to the Linux kernel. LCP provides -seamless integration with other VyOS components, letting you use system -components like DHCP clients and routing daemons together with the VPP -dataplane. - -VPP integration in VyOS relies heavily on LCP. Almost all control plane -functions are handled by other daemons and services, while VPP handles -high-performance packet forwarding exclusively. This approach also reduces -VPP management processing load, improving overall dataplane performance and -stability. - -VyOS integrates the kernel and VPP routing tables uniquely. By default, -all routes, even those not directly connected to VPP interfaces, are -imported from the kernel routing table to the VPP routing table, pointing -to the kernel. This lets you forward traffic to any destination known to -the kernel, even if VPP doesn't have a route to that destination. - -However, in some scenarios this behavior may not be desired. For example, -if you have many routes in the kernel routing table not directly connected -to VPP interfaces, and you don't need forwarding between those -destinations and destinations reachable via VPP, you can disable this -behavior using the following command: -(vpp_config_dataplane_lcp_ignore_kernel_routes)= - -```{cfgcmd} set vpp settings ignore-kernel-routes -``` - -Pay attention that disabling this option leads to loss of connectivity to -destinations if there are no direct routes in VPP routing table. - -## Potential Issues and Troubleshooting - -Disabling kernel route import can result in: - -- Loss of connectivity to certain destinations if kernel routes are ignored -- Incomplete route synchronization between the kernel and VPP diff --git a/docs/vpp/configuration/dataplane/md-logging.md b/docs/vpp/configuration/dataplane/md-logging.md deleted file mode 100644 index 50e6277b..00000000 --- a/docs/vpp/configuration/dataplane/md-logging.md +++ /dev/null @@ -1,59 +0,0 @@ ---- -lastproofread: '2026-02-27' ---- - -(vpp-config-dataplane-logging)= - -```{include} /_include/need_improvement.txt -``` - - -# VPP Logging Configuration - -VPP logging is an important part of monitoring and troubleshooting -the performance and behavior of the VPP dataplane. - -VPP stores logs in two places: -- `/var/log/vpp.log` — This file contains logs related to daemon - startup and logs of commands executed directly via VPP CLI. Pay - attention: VyOS does not use VPP CLI for configuration, so this log - will not contain any configuration changes made via VyOS CLI and will - not be informative in most cases. -- System journal — contains logs related to the VPP daemon work, - including errors, warnings, and informational messages. It is the - main destination of logs generated by VPP. - -Logging detail level can be configured via the next command: - -```{cfgcmd} set vpp settings logging default-level \ -``` - -Where `` can be one of the following: - -- `emerg` (Emergency) - System is unusable. -- `alert` (Alert) - Immediate action required. -- `crit` (Critical) - Critical conditions. -- `error` (Error) - Error conditions. -- `warn` (Warning) - Warning conditions. -- `notice` (Notice) - Normal but significant. -- `info` (Informational) - Routine informational messages. -- `debug` (Debug) - Detailed debugging messages. -- `disabled` (Disabled) - Logging disabled. - -It is recommended to set logging level to `debug` only for -troubleshooting purposes, as it can generate a large volume of log -data. For regular operation, a level of `info` or `warn` is usually -sufficient. - -## Troubleshooting - -Improper logging configuration can lead to various issues, including: - -- Excessive log file sizes if the logging level is set too high - (for example, `debug`). -- Missing critical information if the logging level is set too low - (for example, `alert`). -- Performance degradation due to excessive logging overhead - -Consider adjusting the logging level if you experience issues mentioned -above. diff --git a/docs/vpp/configuration/dataplane/md-memory.md b/docs/vpp/configuration/dataplane/md-memory.md deleted file mode 100644 index 2465e3b3..00000000 --- a/docs/vpp/configuration/dataplane/md-memory.md +++ /dev/null @@ -1,142 +0,0 @@ ---- -lastproofread: '2026-02-27' ---- - -(vpp_config_dataplane_memory)= - -```{include} /_include/need_improvement.txt -``` - -# VPP Memory Configuration - -VPP heavily relies on hugepages for its memory management. Hugepages -are larger memory pages that reduce the overhead of page management and -improve performance for applications that require large amounts of -memory, such as VPP. - -VPP supports both 2MB and 1GB hugepages, but the default and most -commonly used size is 2MB. The choice of hugepage size can impact -performance, with larger pages generally providing better performance -for memory-intensive applications. - -Before configuring memory in VPP dataplane settings, you need to -ensure that hugepages are enabled and properly configured on your -system. - -:::{seealso} -{ref}`Hugepages in VyOS Configuration for VPP ` -::: - -To configure memory settings for VPP, you can use the following -commands in the VPP CLI: - -VPP uses a main heap as a central memory pool for FIB data structures -entry allocations. - -Efficient memory management is crucial for VPP's performance, and the -main heap plays a significant role in this. - -It can be configured using the following command: - -```{cfgcmd} set vpp settings resource-allocation memory main-heap-page-size \ -``` - -Sets the main heap page size for VPP. - -```{cfgcmd} set vpp settings resource-allocation memory main-heap-size \ -``` - -Sets the main heap size for VPP. -(vpp-config-dataplane-physmem)= - -## Physical Memory Configuration - -VPP uses physical memory for packet buffers and interface operations. -The `physmem` setting controls how much memory VPP can allocate for -these operations. - -```{cfgcmd} set vpp settings resource-allocation memory physmem-max-size \ -``` - -Sets the maximum amount of physical memory VPP can use for packet -processing and interface buffers. - -**Default**: 16GB (usually sufficient for most deployments) - -You may need to modify the value for high-throughput environments with -many interfaces, large packet buffers, very high packet rates, or -memory-constrained systems where you need to limit VPP's memory usage. - -**Physmem independent of main heap size** — physmem is for packet -buffers, main heap is for routing tables. - -:::{seealso} -- {ref}`Hugepages in VyOS Configuration for VPP ` -- {ref}`VPP Buffer Configuration ` - for - controlling buffer allocation within physmem -::: - -### Common configurations - -```none -# Reduce for memory-constrained systems -set vpp settings physmem max-size 4G - -# Increase for high-throughput environments -set vpp settings physmem max-size 32G -``` - -## Stats Memory Configuration - -VPP uses a dedicated statistics memory segment to store runtime -counters and telemetry data. This segment is used by the VPP CLI and -monitoring tools to access performance and status information. - -The statistics segment is allocated from hugepage memory and can be -configured independently from the main heap and physmem settings. - -You can configure statistics memory using the following commands: - -```{cfgcmd} set vpp settings resource-allocation memory stats page-size \ -``` - -Sets the hugepage page size used for the statistics memory segment. - -```{cfgcmd} set vpp settings resource-allocation memory stats size \ -``` - -Sets the total size of the statistics memory segment. - -Increasing this value may be required in large deployments with many -interfaces or enabled features that generate a high number of counters. - -Statistics memory is used only for telemetry and monitoring. It does -not affect packet buffer allocation or routing table memory. - -## Troubleshooting - -Improper configuration of main heap size can lead to performance -degradation or even system instability. If VPP runs out of memory in the -main heap, it may crash or exhibit erratic behavior. Symptoms you may -observe include: - -- Increased latency or packet loss -- Crashes or restarts of VPP processes, especially during routing table - population (for example, BGP session establishment) -- Error messages related to memory allocation failures - -You need to tune the main heap size based on expected FIB entries. Pay -attention: the same amount of routes with a single next-hop and with -multiple next-hops will consume different amounts of memory. - -For physmem, insufficient allocation can lead to packet drops, interface -initialization failures, and overall degraded performance. Symptoms -include: - -- Packet drops or failures to allocate buffers -- Increased latency or jitter in packet processing -- Crashes or restarts of VPP processes under heavy load - -You need to tune the physmem settings based on expected traffic patterns -and interface usage. Monitor memory usage closely and adjust the -configuration as needed to ensure optimal performance. diff --git a/docs/vpp/configuration/dataplane/md-system.md b/docs/vpp/configuration/dataplane/md-system.md deleted file mode 100644 index 51ee8f54..00000000 --- a/docs/vpp/configuration/dataplane/md-system.md +++ /dev/null @@ -1,212 +0,0 @@ ---- -lastproofread: '2026-02-27' ---- - -(vpp_config_system)= - -```{include} /_include/need_improvement.txt -``` - -# VyOS Configuration for VPP - -(vpp-config-hugepages)= - -## Hugepages - -VPP uses hugepages for efficient memory management. Hugepages are larger -memory pages that reduce the overhead of page management and improve -performance for applications that require large amounts of memory. - -Hugepages can be configured in VyOS using the following commands: - -:::{warning} -Changes to hugepage settings require a system reboot to take effect. - -Hugepages must be enabled before VPP configuration is applied. -::: - -To enable hugepages: - -```{cfgcmd} set system option kernel memory hugepage-size \ hugepage-count '\' -``` - -Enables hugepages with the specified size and count. The size can be either -2MB or 1GB, and the count specifies the number of hugepages to allocate. - -If your system has multiple NUMA nodes, the total amount of hugepages will be -divided equally among them. - -## Resources Limits - -:::{note} -By default, system will calculate and set the recommended values for -resource limits. Avoid tuning these values if you are not sure what you -are doing. -::: - -During operations VPP utilizes a significant amount of system resources, -especially memory. There are two main settings that may need to be -adjusted to ensure VPP runs smoothly: - -Maximum number of memory map areas a process may have: - -```{cfgcmd} set system option resource-limits max-map-count \ -``` - -Maximum shared memory segment size: - -```{cfgcmd} set system option resource-limits shmmax \ -``` - -Both settings are automatically calculated based on configured hugepages. - -## Kernel Tuning - -VPP performance greatly benefits from proper kernel tuning, especially -CPU isolation and disabling unnecessary kernel features. These -optimizations ensure dedicated CPU cores are available exclusively for -VPP dataplane processing without interference from the kernel scheduler -or other system processes. - -:::{warning} -Kernel tuning changes require a system reboot to take effect. - -Improper CPU isolation can lead to system instability if essential -system processes are starved of CPU resources. -::: - -### CPU Isolation and Optimization - -CPU isolation is crucial for VPP performance as it dedicates specific -CPU cores exclusively to VPP dataplane processing. The isolated cores are -removed from the kernel scheduler and will not run regular system -processes. - -**Disable NMI Watchdog** - -The NMI (Non-Maskable Interrupt) watchdog can interfere with VPP -performance by generating interrupts on isolated cores and is not -compatible with nohz-full mode: - -```{cfgcmd} set system option kernel cpu disable-nmi-watchdog - -Disables the NMI watchdog for detecting hard CPU lockups. This -prevents unnecessary interrupts on VPP worker cores. -``` - -**CPU Core Isolation** - -```{cfgcmd} set system option kernel cpu isolate-cpus \ - -Isolates specified CPUs from the kernel scheduler. Isolated cores will -not run regular system processes and are dedicated to applications like -VPP. - -The ```` can be: -* Single core: ``2`` -* Range: ``2-5`` -* Mixed: ``1,3-5,7`` - -:::{important} -Always reserve at least 2 cores for the operating system to ensure -system stability. For example, on a 4-core system, isolate cores -2-3 for VPP and leave cores 0-1 for the OS. - -Assign the first isolated core as the VPP main core and the -remaining isolated cores as VPP worker cores. Ensure that VPP CPU -assignments match the isolated CPU range. -::: -``` - -**Adaptive-Tick Mode** - -```{cfgcmd} set system option kernel cpu nohz-full \ - -Enables adaptive-tick mode (NO_HZ_FULL) for specified CPUs. This -causes the kernel to avoid sending scheduling-clock interrupts to CPUs -that have only one runnable task, significantly reducing interrupt -overhead for dedicated workloads like VPP. - -Use the same CPU range as configured for ``isolate-cpus``. -``` - -**RCU Callback Offloading** - -```{cfgcmd} set system option kernel cpu rcu-no-cbs \ - -Offloads Read-Copy-Update (RCU) callback processing from specified -CPUs. This ensures that RCU callbacks do not prevent the specified CPUs -from entering dyntick-idle or adaptive-tick mode, which is essential -for nohz-full functionality. - -Use the same CPU range as configured for ``isolate-cpus``. -``` - -### System Optimization - -Additional kernel optimizations can further improve VPP performance by -disabling unnecessary features and reducing system overhead. - -**Disable High Precision Event Timer** - -```{cfgcmd} set system option kernel disable-hpet - -Disables the High Precision Event Timer (HPET). HPET can cause -additional interrupts and overhead that may impact VPP performance. -``` - -**Disable Machine Check Exceptions** - -```{cfgcmd} set system option kernel disable-mce - -Disables Machine Check Exception (MCE) reporting and handling. While -MCE provides hardware error detection, it can introduce latency in -high-performance scenarios. -``` - -**Disable CPU Power Saving** - -```{cfgcmd} set system option kernel disable-power-saving - -Disables CPU power saving mechanisms (C-states). This keeps CPU cores -at maximum performance levels, eliminating latency from power state -transitions. -``` - -**Disable Soft Lockup Detection** - -```{cfgcmd} set system option kernel disable-softlockup - -Disables the soft lockup detector for kernel threads. This prevents -false positives when VPP worker threads are busy processing packets. -``` - -**Disable CPU Mitigations** - -```{cfgcmd} set system option kernel disable-mitigations - -Disables all optional CPU mitigations for security vulnerabilities -(for example, Spectre, Meltdown). This may improve performance on some -platforms. -``` - -### Optimal Configuration Example - -For a system with 4 CPU cores (0-3) where cores 2-3 are dedicated to VPP: - -```none -# Kernel CPU optimizations -set system option kernel cpu disable-nmi-watchdog -set system option kernel cpu isolate-cpus '2-3' -set system option kernel cpu nohz-full '2-3' -set system option kernel cpu rcu-no-cbs '2-3' - -# System optimizations -set system option kernel disable-hpet -set system option kernel disable-mce -set system option kernel disable-power-saving -set system option kernel disable-softlockup - -# VPP CPU assignment -set vpp settings resource-allocation cpu-cores '2' -``` diff --git a/docs/vpp/configuration/dataplane/md-unix.md b/docs/vpp/configuration/dataplane/md-unix.md deleted file mode 100644 index a1f6a1fd..00000000 --- a/docs/vpp/configuration/dataplane/md-unix.md +++ /dev/null @@ -1,57 +0,0 @@ ---- -lastproofread: '2026-02-27' ---- - -(vpp-config-dataplane-unix)= - -```{include} /_include/need_improvement.txt -``` - - -# VPP Unix Dataplane Configuration - -The UNIX configuration section is used to control VPP's interaction -with the underlying operating system, including operations scheduling. - -VPP relies on the polling mechanism to efficiently manage I/O operations -and system events. By default VPP continuously polls for events, which -leads to permanent 100% CPU usage by all cores assigned to VPP dataplane. -This is optimal for performance, but may not be desirable in all -environments, especially where power consumption is a concern or where VPP -is running inside a hypervisor, especially if the VM has burstable -thresholds and CPU usage limits. - -To mitigate this, VPP provides a configurable polling delay that allows -reducing CPU usage by introducing a delay between polling cycles. This -introduces a trade-off between CPU usage and latency, as longer delays -can lead to increased latency in processing events. - -You can configure the polling delay using the following command in the -VyOS CLI: - -```{cfgcmd} set vpp settings poll-sleep-usec \ -``` - -Sets the polling delay in microseconds. A value of 0 means no delay -(default), while higher values introduce a delay between polling cycles. - -## Troubleshooting - -Setting the polling delay too high can lead to increased latency and -reduced performance, as VPP may not respond to events as quickly. -Conversely, setting it too low may result in high CPU usage, which can be -problematic in resource-constrained environments. - -Symptoms of improper configuration may include: - -- Increased latency in packet processing -- Higher CPU usage than expected -- Packets lost due to buffer overruns - -If you do not need to reduce CPU usage, it is recommended to leave the -polling delay at its default value of 0 for optimal performance. - -If you need to reduce CPU usage, you may also consider using `interrupt` or -`adaptive` {ref}`DPDK driver modes `, -which can provide a balance between performance and resource utilization -without affecting polling behavior. diff --git a/docs/vpp/configuration/dataplane/memory.md b/docs/vpp/configuration/dataplane/memory.md new file mode 100644 index 00000000..2465e3b3 --- /dev/null +++ b/docs/vpp/configuration/dataplane/memory.md @@ -0,0 +1,142 @@ +--- +lastproofread: '2026-02-27' +--- + +(vpp_config_dataplane_memory)= + +```{include} /_include/need_improvement.txt +``` + +# VPP Memory Configuration + +VPP heavily relies on hugepages for its memory management. Hugepages +are larger memory pages that reduce the overhead of page management and +improve performance for applications that require large amounts of +memory, such as VPP. + +VPP supports both 2MB and 1GB hugepages, but the default and most +commonly used size is 2MB. The choice of hugepage size can impact +performance, with larger pages generally providing better performance +for memory-intensive applications. + +Before configuring memory in VPP dataplane settings, you need to +ensure that hugepages are enabled and properly configured on your +system. + +:::{seealso} +{ref}`Hugepages in VyOS Configuration for VPP ` +::: + +To configure memory settings for VPP, you can use the following +commands in the VPP CLI: + +VPP uses a main heap as a central memory pool for FIB data structures +entry allocations. + +Efficient memory management is crucial for VPP's performance, and the +main heap plays a significant role in this. + +It can be configured using the following command: + +```{cfgcmd} set vpp settings resource-allocation memory main-heap-page-size \ +``` + +Sets the main heap page size for VPP. + +```{cfgcmd} set vpp settings resource-allocation memory main-heap-size \ +``` + +Sets the main heap size for VPP. +(vpp-config-dataplane-physmem)= + +## Physical Memory Configuration + +VPP uses physical memory for packet buffers and interface operations. +The `physmem` setting controls how much memory VPP can allocate for +these operations. + +```{cfgcmd} set vpp settings resource-allocation memory physmem-max-size \ +``` + +Sets the maximum amount of physical memory VPP can use for packet +processing and interface buffers. + +**Default**: 16GB (usually sufficient for most deployments) + +You may need to modify the value for high-throughput environments with +many interfaces, large packet buffers, very high packet rates, or +memory-constrained systems where you need to limit VPP's memory usage. + +**Physmem independent of main heap size** — physmem is for packet +buffers, main heap is for routing tables. + +:::{seealso} +- {ref}`Hugepages in VyOS Configuration for VPP ` +- {ref}`VPP Buffer Configuration ` - for + controlling buffer allocation within physmem +::: + +### Common configurations + +```none +# Reduce for memory-constrained systems +set vpp settings physmem max-size 4G + +# Increase for high-throughput environments +set vpp settings physmem max-size 32G +``` + +## Stats Memory Configuration + +VPP uses a dedicated statistics memory segment to store runtime +counters and telemetry data. This segment is used by the VPP CLI and +monitoring tools to access performance and status information. + +The statistics segment is allocated from hugepage memory and can be +configured independently from the main heap and physmem settings. + +You can configure statistics memory using the following commands: + +```{cfgcmd} set vpp settings resource-allocation memory stats page-size \ +``` + +Sets the hugepage page size used for the statistics memory segment. + +```{cfgcmd} set vpp settings resource-allocation memory stats size \ +``` + +Sets the total size of the statistics memory segment. + +Increasing this value may be required in large deployments with many +interfaces or enabled features that generate a high number of counters. + +Statistics memory is used only for telemetry and monitoring. It does +not affect packet buffer allocation or routing table memory. + +## Troubleshooting + +Improper configuration of main heap size can lead to performance +degradation or even system instability. If VPP runs out of memory in the +main heap, it may crash or exhibit erratic behavior. Symptoms you may +observe include: + +- Increased latency or packet loss +- Crashes or restarts of VPP processes, especially during routing table + population (for example, BGP session establishment) +- Error messages related to memory allocation failures + +You need to tune the main heap size based on expected FIB entries. Pay +attention: the same amount of routes with a single next-hop and with +multiple next-hops will consume different amounts of memory. + +For physmem, insufficient allocation can lead to packet drops, interface +initialization failures, and overall degraded performance. Symptoms +include: + +- Packet drops or failures to allocate buffers +- Increased latency or jitter in packet processing +- Crashes or restarts of VPP processes under heavy load + +You need to tune the physmem settings based on expected traffic patterns +and interface usage. Monitor memory usage closely and adjust the +configuration as needed to ensure optimal performance. diff --git a/docs/vpp/configuration/dataplane/memory.rst b/docs/vpp/configuration/dataplane/memory.rst deleted file mode 100644 index c2d74991..00000000 --- a/docs/vpp/configuration/dataplane/memory.rst +++ /dev/null @@ -1,141 +0,0 @@ -:lastproofread: 2026-02-27 - -.. _vpp_config_dataplane_memory: - -.. include:: /_include/need_improvement.txt - -######################## -VPP Memory Configuration -######################## - -VPP heavily relies on hugepages for its memory management. Hugepages -are larger memory pages that reduce the overhead of page management and -improve performance for applications that require large amounts of -memory, such as VPP. - -VPP supports both 2MB and 1GB hugepages, but the default and most -commonly used size is 2MB. The choice of hugepage size can impact -performance, with larger pages generally providing better performance -for memory-intensive applications. - -Before configuring memory in VPP dataplane settings, you need to -ensure that hugepages are enabled and properly configured on your -system. - -.. seealso:: :ref:`Hugepages in VyOS Configuration for VPP - ` - -To configure memory settings for VPP, you can use the following -commands in the VPP CLI: - -VPP uses a main heap as a central memory pool for FIB data structures -entry allocations. - -Efficient memory management is crucial for VPP's performance, and the -main heap plays a significant role in this. - -It can be configured using the following command: - -.. cfgcmd:: set vpp settings resource-allocation memory main-heap-page-size - - -Sets the main heap page size for VPP. - -.. cfgcmd:: set vpp settings resource-allocation memory main-heap-size - -Sets the main heap size for VPP. - -.. _vpp_config_dataplane_physmem: - -Physical Memory Configuration -============================= - -VPP uses physical memory for packet buffers and interface operations. -The ``physmem`` setting controls how much memory VPP can allocate for -these operations. - -.. cfgcmd:: set vpp settings resource-allocation memory physmem-max-size - -Sets the maximum amount of physical memory VPP can use for packet -processing and interface buffers. - -**Default**: 16GB (usually sufficient for most deployments) - -You may need to modify the value for high-throughput environments with -many interfaces, large packet buffers, very high packet rates, or -memory-constrained systems where you need to limit VPP's memory usage. - -**Physmem independent of main heap size** — physmem is for packet -buffers, main heap is for routing tables. - -.. seealso:: - - - :ref:`Hugepages in VyOS Configuration for VPP ` - - :ref:`VPP Buffer Configuration ` - for - controlling buffer allocation within physmem - -Common configurations ---------------------- - -.. code-block:: none - - # Reduce for memory-constrained systems - set vpp settings physmem max-size 4G - - # Increase for high-throughput environments - set vpp settings physmem max-size 32G - -Stats Memory Configuration -========================== - -VPP uses a dedicated statistics memory segment to store runtime -counters and telemetry data. This segment is used by the VPP CLI and -monitoring tools to access performance and status information. - -The statistics segment is allocated from hugepage memory and can be -configured independently from the main heap and physmem settings. - -You can configure statistics memory using the following commands: - -.. cfgcmd:: set vpp settings resource-allocation memory stats page-size - -Sets the hugepage page size used for the statistics memory segment. - -.. cfgcmd:: set vpp settings resource-allocation memory stats size - -Sets the total size of the statistics memory segment. - -Increasing this value may be required in large deployments with many -interfaces or enabled features that generate a high number of counters. - -Statistics memory is used only for telemetry and monitoring. It does -not affect packet buffer allocation or routing table memory. - -Troubleshooting -=============== - -Improper configuration of main heap size can lead to performance -degradation or even system instability. If VPP runs out of memory in the -main heap, it may crash or exhibit erratic behavior. Symptoms you may -observe include: - -- Increased latency or packet loss -- Crashes or restarts of VPP processes, especially during routing table - population (for example, BGP session establishment) -- Error messages related to memory allocation failures - -You need to tune the main heap size based on expected FIB entries. Pay -attention: the same amount of routes with a single next-hop and with -multiple next-hops will consume different amounts of memory. - -For physmem, insufficient allocation can lead to packet drops, interface -initialization failures, and overall degraded performance. Symptoms -include: - -- Packet drops or failures to allocate buffers -- Increased latency or jitter in packet processing -- Crashes or restarts of VPP processes under heavy load - -You need to tune the physmem settings based on expected traffic patterns -and interface usage. Monitor memory usage closely and adjust the -configuration as needed to ensure optimal performance. diff --git a/docs/vpp/configuration/dataplane/rst-buffers.rst b/docs/vpp/configuration/dataplane/rst-buffers.rst new file mode 100644 index 00000000..b08eb691 --- /dev/null +++ b/docs/vpp/configuration/dataplane/rst-buffers.rst @@ -0,0 +1,103 @@ +:lastproofread: 2026-02-23 + +.. _vpp_config_dataplane_buffers: + +.. include:: /_include/need_improvement.txt + +################################### +VPP Dataplane Buffers Configuration +################################### + +Buffers are essential for handling network packets efficiently. Proper +configuration enhances performance and reliability, and is mandatory for +VPP to work. Buffers temporarily store packets during processing. Therefore, +their configuration must be in sync with NIC configuration, CPU threads, and +overall system resources. + +.. important:: + + VPP buffers are allocated from the physical memory pool (``physmem``). The + total amount of memory available for buffer allocation is controlled by the + ``physmem-max-size`` setting, while the buffer configuration parameters + below control how that memory is used for buffer allocation. + + See :ref:`VPP Physical Memory Configuration ` + for details on configuring ``physmem``. + +Buffer Configuration Parameters +=============================== + +The following parameters can be configured for VPP buffers: + +buffers-per-numa +---------------- + +Number of buffers allocated per NUMA node. This setting optimizes +memory access patterns for multi-CPU systems. + +Typically, you need to tune this value if: + +- The system has many interfaces +- NICs have many queues +- NICs have large descriptor sizes + +Set this value carefully to balance memory usage and performance. + +.. cfgcmd:: set vpp settings resource-allocation buffers buffers-per-numa + + +The common approach for the calculation is to use the formula: + +.. code-block:: none + + buffers-per-numa = (num-rx-queues * num-rx-desc) + (num-tx-queues * num-tx-desc) + +Calculate this formula for each NIC and sum the results. Multiply the +total by 2.5 to get the minimum recommended value for +``buffers-per-numa``. + +Avoid setting this value too low to prevent packet drops. + +data-size +--------- + +This value sets how much payload data can be stored in a single buffer +allocated by VPP. Larger values reduce buffer chains for large packets, +while smaller values conserve memory for environments handling mostly +small packets. + +.. cfgcmd:: set vpp settings resource-allocation buffers data-size + +Optimal size depends on the typical packet size in your network. If +unsure, use the largest MTU in your network plus overhead (for example, +128 bytes). + +page-size +--------- + +A memory pages type used for buffer allocation. Common values are 4K, 2M, or 1G. + +Use page sizes configured in your system settings. + +.. cfgcmd:: set vpp settings resource-allocation buffers page-size + +Potential Issues and Troubleshooting +==================================== + +Improper buffer configuration can lead to issues such as: + +- Increased latency and packet loss +- Inefficient CPU utilization +- Interface initialization failures + +Indicators of such issues are: + +- Errors during interfaces initialization in VPP logs +- Packet drops observed in VPP statistics + +To troubleshoot buffer-related issues, consider the following steps: + +- Review VPP logs for errors related to buffer allocation. Look for + error ``-5`` messages. +- Tune available buffers by adjusting the ``buffers-per-numa`` and + ``data-size`` parameters. diff --git a/docs/vpp/configuration/dataplane/rst-cpu.rst b/docs/vpp/configuration/dataplane/rst-cpu.rst new file mode 100644 index 00000000..22158ce3 --- /dev/null +++ b/docs/vpp/configuration/dataplane/rst-cpu.rst @@ -0,0 +1,74 @@ +:lastproofread: 2026-02-23 + +.. _vpp_config_dataplane_cpu: + +.. include:: /_include/need_improvement.txt + +############################### +VPP Dataplane CPU Configuration +############################### + +VPP can utilize multiple CPU cores for better packet processing +performance. Proper CPU configuration is essential for optimal +throughput and low latency. + +VPP CPU assignment is handled automatically. You specify how many CPU +cores VPP may use, and the system distributes them between the main +thread and worker threads. + +.. important:: + + Review the system configuration settings page before changing CPU + settings: :doc:`system`. + +If you don't configure CPU settings, VPP uses a single core for the +main thread and doesn't create worker threads. + +CPU Configuration Parameters +============================ + +``cpu-cores`` +^^^^^^^^^^^^^ + +This parameter defines the total number of CPU cores allocated to VPP. + +.. cfgcmd:: set vpp settings resource-allocation cpu-cores + +The system automatically assigns cores using the following rules: + + * The first two CPU cores are always reserved for the operating system and + other services. + + * The main VPP thread is assigned to the first available core after the + reserved ones. + + * The remaining allocated cores are used for worker threads. + +For example: + + * If cpu-cores is set to 1, VPP runs only a main thread. + + * If cpu-cores is set to 4, VPP uses: + + * 1 core for the main thread + + * 3 cores for worker threads + +Choose a value based on available hardware resources and expected +traffic load. Too few cores may limit performance, while too many can +negatively impact other system services. + +Potential Issues and Troubleshooting +==================================== + +Improper CPU configuration can lead to issues such as: + +- VPP underperformance when not enough cores are assigned, or kernel + underperformance when too many cores are assigned to VPP. +- Resource conflicts with other processes and services. + +Indicators of such issues are: + +- VPP or kernel forwarding performance is lower than expected +- Degraded performance of system components or services, such as DNS, + DHCP, and dynamic routing diff --git a/docs/vpp/configuration/dataplane/rst-index.rst b/docs/vpp/configuration/dataplane/rst-index.rst new file mode 100644 index 00000000..323d7588 --- /dev/null +++ b/docs/vpp/configuration/dataplane/rst-index.rst @@ -0,0 +1,35 @@ +:lastproofread: 2026-02-23 + +.. _vpp_config_dataplane_index: + +.. include:: /_include/need_improvement.txt + +################################ +VPP Dataplane Core Configuration +################################ + +This section covers the core configuration options for the VPP dataplane in +VyOS. It includes settings for memory management, CPU allocation, hugepages, +and other essential parameters that influence the performance and behavior +of the VPP dataplane. + +Please review the general system configuration, before starting to configure +VPP. Without proper VyOS preconditions, VPP will not start or its efficiency +will be significantly degraded. + +.. toctree:: + :maxdepth: 1 + :includehidden: + + system + buffers + cpu + interface + ipsec + ipv6 + l2learn + lcp + logging + memory + unix + \ No newline at end of file diff --git a/docs/vpp/configuration/dataplane/rst-interface.rst b/docs/vpp/configuration/dataplane/rst-interface.rst new file mode 100644 index 00000000..e4556021 --- /dev/null +++ b/docs/vpp/configuration/dataplane/rst-interface.rst @@ -0,0 +1,102 @@ +:lastproofread: 2026-02-23 + +.. _vpp_config_dataplane_interface: + +.. include:: /_include/need_improvement.txt + +###################################### +VPP Dataplane Interfaces Configuration +###################################### + +Only Ethernet interfaces (physical or virtual) can be connected to the +VPP dataplane. Interfaces configured here act as a bridge between VPP +and the outside world, allowing VPP to send and receive network +packets. + + +Interface Configuration Parameters +================================== + +Interfaces connected to the VPP dataplane use the DPDK driver by default, +providing high performance and low latency. + +.. cfgcmd:: set vpp settings interface + +Some network interface cards (NICs) may not be compatible with the DPDK driver. + +DPDK interface options +---------------------- + +This section shows how to configures DPDK-specific settings for an interface. + +.. cfgcmd:: set vpp settings interface num-rx-queues + +Specifies the number of receive queues for the interface. More queues +improve performance on multi-core systems by allowing parallel +processing of incoming packets. Each queue is assigned to a separate +CPU core. + +.. cfgcmd:: set vpp settings interface num-tx-queues + +Specifies the number of transmit queues for the interface. Similar to +receive queues, more transmit queues improve performance by enabling +parallel processing of outgoing packets. By default, the VPP Dataplane +has one TX queue per enabled CPU worker, or a single queue if no +workers are configured. + +.. seealso:: :doc:`cpu` + +.. cfgcmd:: set vpp settings interface num-rx-desc + +Defines the size of each receive queue. Larger queue sizes accommodate +bursts of incoming traffic and reduce the likelihood of packet drops +during high traffic periods. + +.. cfgcmd:: set vpp settings interface num-tx-desc + +Defines the size of each transmit queue. Larger sizes help manage +bursts of outgoing traffic more effectively. + +Global Interface Parameters +=========================== + +.. _vpp_config_dataplane_interface_rx_mode: + +interface-rx-mode +----------------- + +The ``interface-rx-mode`` parameter defines how VPP handles incoming +packets on interfaces. There are several modes available, each with its +own advantages and use cases: + +- ``interrupt``: In this mode, VPP relies on hardware interrupts to + notify it of incoming packets. This mode suits low to moderate + traffic loads and reduces CPU usage during idle periods. It is not + recommended for low-latency processing. Some NICs may not support + this mode. +- ``polling``: In polling mode, VPP continuously checks the interface + for incoming packets. This mode is ideal for high-throughput + scenarios where low latency is critical, as it minimizes packet + waiting time. However, it can increase CPU usage, especially during + low traffic periods, as the polling process is always active. +- ``adaptive``: Adaptive mode combines the benefits of interrupt and + polling modes. VPP starts in interrupt mode and switches to polling + mode when traffic load increases. + +.. cfgcmd:: set vpp settings interface-rx-mode + +Choose an rx-mode based on expected traffic patterns and performance +requirements of your network. + +Potential Issues and Troubleshooting +==================================== + +Improper interface configuration can lead to issues such as: + +- Failure to initialize the interface +- Poor performance due to suboptimal driver selection or settings + +Indicators of such issues are: + +- Failed commits after adding or modifying an interface settings +- Low throughput or high latency on the interface diff --git a/docs/vpp/configuration/dataplane/rst-ipsec.rst b/docs/vpp/configuration/dataplane/rst-ipsec.rst new file mode 100644 index 00000000..31734c86 --- /dev/null +++ b/docs/vpp/configuration/dataplane/rst-ipsec.rst @@ -0,0 +1,72 @@ +:lastproofread: 2026-02-23 + +.. _vpp_config_dataplane_ipsec: + +.. include:: /_include/need_improvement.txt + +####################### +VPP IPsec Configuration +####################### + +VPP supports IPsec (Internet Protocol Security) offloading from the +kernel, which speeds up cryptographic operations by leveraging VPP's +high-performance packet processing capabilities. + +IPsec does not require any specific configuration on VPP side. If both +sources and destinations of the IPsec traffic are reachable via VPP +interfaces, VPP will automatically offload the IPsec processing from +the kernel. IPsec tunnels are configured in the VPN configuration +section, see :ref:`ipsec_general`. + +IPsec Configuration Parameters +============================== + +enable IPsec acceleration +^^^^^^^^^^^^^^^^^^^^^^^^^ + +When VPP is used for offloading IPsec, it creates a virtual interface to +connect to peers. The interface type is always 'ipsec', which is used for +IPsec tunnels. + +.. cfgcmd:: set vpp settings ipsec-acceleration + +Enabling this option allows VPP to handle IPsec traffic more efficiently by +offloading processing from the kernel. + +netlink +^^^^^^^ + +VPP uses netlink to receive IPsec event messages from the kernel. Proper +settings of the following parameters are crucial for ensuring that VPP can +process all such messages: + +.. cfgcmd:: set vpp settings lcp netlink batch-delay-ms + +This parameter specifies the delay in milliseconds between processing +batch netlink messages. + +.. cfgcmd:: set vpp settings lcp netlink batch-size + +This parameter specifies the maximum number of netlink messages to +process in a single batch. + +.. cfgcmd:: set vpp settings lcp netlink rx-buffer-size + +This parameter specifies the size of the receive buffer for netlink +socket. If you expect to offload many IPsec tunnels or get frequent and +intensive rekeying, you may need to increase this value. + +.. note:: + + IPsec uses the same netlink parameters as LCP, so tuning them + affects both LCP and IPsec processing. + +Potential Issues and Troubleshooting +==================================== + +Improper IPsec configuration can lead to various issues, including: + +- Failure to offload IPsec tunnels to VPP +- Lost IPsec event messages due to insufficient netlink buffer size or + batch settings +- IPsec states or SAs are not synchronized between kernel and VPP diff --git a/docs/vpp/configuration/dataplane/rst-ipv6.rst b/docs/vpp/configuration/dataplane/rst-ipv6.rst new file mode 100644 index 00000000..94f38476 --- /dev/null +++ b/docs/vpp/configuration/dataplane/rst-ipv6.rst @@ -0,0 +1,43 @@ +:lastproofread: 2026-02-26 + +.. _vpp_config_dataplane_ipv6: + +.. include:: /_include/need_improvement.txt + +###################### +VPP IPv6 Configuration +###################### + +VPP lets you configure resources allocated for IPv6 traffic processing +independently from IPv4. This helps ensure that in networks without IPv6 +traffic, resources are not wasted. If IPv6 traffic is present, especially +with large routing tables, you must allocate additional resources for IPv6 +processing to keep the dataplane stable. + +You can configure two main resources for IPv6 traffic processing: + +.. cfgcmd:: set vpp settings resource-allocation ipv6 hash-buckets + +This parameter configures the number of hash buckets used for IPv6 +routing. If you have a large IPv6 routing table, you may need to increase +this value to ensure efficient routing table performance and fast lookups. + +.. cfgcmd:: set vpp settings resource-allocation ipv6 heap-size + +This parameter configures the heap size used for IPv6 forwarding. If you +have a large IPv6 routing table, you may need to increase this value to +ensure the routing table can accommodate all routes. + +Potential Issues and Troubleshooting +==================================== + +Improper IPv6 configuration can lead to various issues, including: + +- Inefficient, slow routing table lookups and traffic processing due to + insufficient hash buckets +- Dataplane crashes or instability due to insufficient heap size when + handling a large number of IPv6 routes +- Overall dataplane instability when handling IPv6 traffic + +Consider increasing configuration values if you experience issues with +IPv6 traffic processing or if you have a large IPv6 routing table. diff --git a/docs/vpp/configuration/dataplane/rst-l2learn.rst b/docs/vpp/configuration/dataplane/rst-l2learn.rst new file mode 100644 index 00000000..b204c41f --- /dev/null +++ b/docs/vpp/configuration/dataplane/rst-l2learn.rst @@ -0,0 +1,34 @@ +:lastproofread: 2026-02-26 + +.. _vpp_config_dataplane_l2learn: + +.. include:: /_include/need_improvement.txt + +######################### +VPP L2LEARN Configuration +######################### + +When VPP dataplane connects to an L2 domain, it learns MAC addresses of +devices in the domain. By default, the number of MAC addresses it can +learn is limited. + +You can configure the limit using the following command: + +.. cfgcmd:: set vpp settings resource-allocation mac-limit + +This parameter sets the maximum number of MAC addresses that can be +learned in the L2 domain. If you have many devices, you may need to +increase this limit to ensure VPP learns all MAC addresses. + +Potential Issues and Troubleshooting +==================================== + +Improper L2LEARN configuration can lead to various issues, including: + +- MAC address learning failure in the L2 domain if the limit is set too + low +- Increased packet loss or latency for devices that aren't learned +- Overall dataplane instability when handling L2 traffic + +Consider increasing the L2LEARN limit if you experience issues with MAC +address learning or if you have many devices in the L2 domain. diff --git a/docs/vpp/configuration/dataplane/rst-lcp.rst b/docs/vpp/configuration/dataplane/rst-lcp.rst new file mode 100644 index 00000000..cb39439a --- /dev/null +++ b/docs/vpp/configuration/dataplane/rst-lcp.rst @@ -0,0 +1,48 @@ +:lastproofread: 2026-02-26 + +.. _vpp_config_dataplane_lcp: + +.. include:: /_include/need_improvement.txt + +##################### +VPP LCP Configuration +##################### + +Linux Control Plane (LCP) is a core component of VPP that lets you +offload various control plane functions to the Linux kernel. LCP provides +seamless integration with other VyOS components, letting you use system +components like DHCP clients and routing daemons together with the VPP +dataplane. + +VPP integration in VyOS relies heavily on LCP. Almost all control plane +functions are handled by other daemons and services, while VPP handles +high-performance packet forwarding exclusively. This approach also reduces +VPP management processing load, improving overall dataplane performance and +stability. + +VyOS integrates the kernel and VPP routing tables uniquely. By default, +all routes, even those not directly connected to VPP interfaces, are +imported from the kernel routing table to the VPP routing table, pointing +to the kernel. This lets you forward traffic to any destination known to +the kernel, even if VPP doesn't have a route to that destination. + +However, in some scenarios this behavior may not be desired. For example, +if you have many routes in the kernel routing table not directly connected +to VPP interfaces, and you don't need forwarding between those +destinations and destinations reachable via VPP, you can disable this +behavior using the following command: + +.. _vpp_config_dataplane_lcp_ignore-kernel-routes: + +.. cfgcmd:: set vpp settings ignore-kernel-routes + +Pay attention that disabling this option leads to loss of connectivity to +destinations if there are no direct routes in VPP routing table. + +Potential Issues and Troubleshooting +==================================== + +Disabling kernel route import can result in: + +- Loss of connectivity to certain destinations if kernel routes are ignored +- Incomplete route synchronization between the kernel and VPP diff --git a/docs/vpp/configuration/dataplane/rst-logging.rst b/docs/vpp/configuration/dataplane/rst-logging.rst new file mode 100644 index 00000000..70d57028 --- /dev/null +++ b/docs/vpp/configuration/dataplane/rst-logging.rst @@ -0,0 +1,58 @@ +:lastproofread: 2026-02-27 + +.. _vpp_config_dataplane_logging: + +.. include:: /_include/need_improvement.txt + +######################### +VPP Logging Configuration +######################### + +VPP logging is an important part of monitoring and troubleshooting +the performance and behavior of the VPP dataplane. + +VPP stores logs in two places: + +- ``/var/log/vpp.log`` — This file contains logs related to daemon + startup and logs of commands executed directly via VPP CLI. Pay + attention: VyOS does not use VPP CLI for configuration, so this log + will not contain any configuration changes made via VyOS CLI and will + not be informative in most cases. +- System journal — contains logs related to the VPP daemon work, + including errors, warnings, and informational messages. It is the + main destination of logs generated by VPP. + +Logging detail level can be configured via the next command: + +.. cfgcmd:: set vpp settings logging default-level + +Where ```` can be one of the following: + +- ``emerg`` (Emergency) - System is unusable. +- ``alert`` (Alert) - Immediate action required. +- ``crit`` (Critical) - Critical conditions. +- ``error`` (Error) - Error conditions. +- ``warn`` (Warning) - Warning conditions. +- ``notice`` (Notice) - Normal but significant. +- ``info`` (Informational) - Routine informational messages. +- ``debug`` (Debug) - Detailed debugging messages. +- ``disabled`` (Disabled) - Logging disabled. + +It is recommended to set logging level to ``debug`` only for +troubleshooting purposes, as it can generate a large volume of log +data. For regular operation, a level of ``info`` or ``warn`` is usually +sufficient. + +Troubleshooting +=============== + +Improper logging configuration can lead to various issues, including: + +- Excessive log file sizes if the logging level is set too high + (for example, ``debug``). +- Missing critical information if the logging level is set too low + (for example, ``alert``). +- Performance degradation due to excessive logging overhead + +Consider adjusting the logging level if you experience issues mentioned +above. diff --git a/docs/vpp/configuration/dataplane/rst-memory.rst b/docs/vpp/configuration/dataplane/rst-memory.rst new file mode 100644 index 00000000..c2d74991 --- /dev/null +++ b/docs/vpp/configuration/dataplane/rst-memory.rst @@ -0,0 +1,141 @@ +:lastproofread: 2026-02-27 + +.. _vpp_config_dataplane_memory: + +.. include:: /_include/need_improvement.txt + +######################## +VPP Memory Configuration +######################## + +VPP heavily relies on hugepages for its memory management. Hugepages +are larger memory pages that reduce the overhead of page management and +improve performance for applications that require large amounts of +memory, such as VPP. + +VPP supports both 2MB and 1GB hugepages, but the default and most +commonly used size is 2MB. The choice of hugepage size can impact +performance, with larger pages generally providing better performance +for memory-intensive applications. + +Before configuring memory in VPP dataplane settings, you need to +ensure that hugepages are enabled and properly configured on your +system. + +.. seealso:: :ref:`Hugepages in VyOS Configuration for VPP + ` + +To configure memory settings for VPP, you can use the following +commands in the VPP CLI: + +VPP uses a main heap as a central memory pool for FIB data structures +entry allocations. + +Efficient memory management is crucial for VPP's performance, and the +main heap plays a significant role in this. + +It can be configured using the following command: + +.. cfgcmd:: set vpp settings resource-allocation memory main-heap-page-size + + +Sets the main heap page size for VPP. + +.. cfgcmd:: set vpp settings resource-allocation memory main-heap-size + +Sets the main heap size for VPP. + +.. _vpp_config_dataplane_physmem: + +Physical Memory Configuration +============================= + +VPP uses physical memory for packet buffers and interface operations. +The ``physmem`` setting controls how much memory VPP can allocate for +these operations. + +.. cfgcmd:: set vpp settings resource-allocation memory physmem-max-size + +Sets the maximum amount of physical memory VPP can use for packet +processing and interface buffers. + +**Default**: 16GB (usually sufficient for most deployments) + +You may need to modify the value for high-throughput environments with +many interfaces, large packet buffers, very high packet rates, or +memory-constrained systems where you need to limit VPP's memory usage. + +**Physmem independent of main heap size** — physmem is for packet +buffers, main heap is for routing tables. + +.. seealso:: + + - :ref:`Hugepages in VyOS Configuration for VPP ` + - :ref:`VPP Buffer Configuration ` - for + controlling buffer allocation within physmem + +Common configurations +--------------------- + +.. code-block:: none + + # Reduce for memory-constrained systems + set vpp settings physmem max-size 4G + + # Increase for high-throughput environments + set vpp settings physmem max-size 32G + +Stats Memory Configuration +========================== + +VPP uses a dedicated statistics memory segment to store runtime +counters and telemetry data. This segment is used by the VPP CLI and +monitoring tools to access performance and status information. + +The statistics segment is allocated from hugepage memory and can be +configured independently from the main heap and physmem settings. + +You can configure statistics memory using the following commands: + +.. cfgcmd:: set vpp settings resource-allocation memory stats page-size + +Sets the hugepage page size used for the statistics memory segment. + +.. cfgcmd:: set vpp settings resource-allocation memory stats size + +Sets the total size of the statistics memory segment. + +Increasing this value may be required in large deployments with many +interfaces or enabled features that generate a high number of counters. + +Statistics memory is used only for telemetry and monitoring. It does +not affect packet buffer allocation or routing table memory. + +Troubleshooting +=============== + +Improper configuration of main heap size can lead to performance +degradation or even system instability. If VPP runs out of memory in the +main heap, it may crash or exhibit erratic behavior. Symptoms you may +observe include: + +- Increased latency or packet loss +- Crashes or restarts of VPP processes, especially during routing table + population (for example, BGP session establishment) +- Error messages related to memory allocation failures + +You need to tune the main heap size based on expected FIB entries. Pay +attention: the same amount of routes with a single next-hop and with +multiple next-hops will consume different amounts of memory. + +For physmem, insufficient allocation can lead to packet drops, interface +initialization failures, and overall degraded performance. Symptoms +include: + +- Packet drops or failures to allocate buffers +- Increased latency or jitter in packet processing +- Crashes or restarts of VPP processes under heavy load + +You need to tune the physmem settings based on expected traffic patterns +and interface usage. Monitor memory usage closely and adjust the +configuration as needed to ensure optimal performance. diff --git a/docs/vpp/configuration/dataplane/rst-system.rst b/docs/vpp/configuration/dataplane/rst-system.rst new file mode 100644 index 00000000..6a2756fd --- /dev/null +++ b/docs/vpp/configuration/dataplane/rst-system.rst @@ -0,0 +1,208 @@ +:lastproofread: 2026-02-27 + +.. _vpp_config_system: + +.. include:: /_include/need_improvement.txt + +########################## +VyOS Configuration for VPP +########################## + +.. _vpp_config_hugepages: +.. _vpp-config-hugepages: + +Hugepages +========= + +VPP uses hugepages for efficient memory management. Hugepages are larger +memory pages that reduce the overhead of page management and improve +performance for applications that require large amounts of memory. + +Hugepages can be configured in VyOS using the following commands: + +.. warning:: + + Changes to hugepage settings require a system reboot to take effect. + + Hugepages must be enabled before VPP configuration is applied. + +To enable hugepages: + +.. cfgcmd:: set system option kernel memory hugepage-size hugepage-count + '' + +Enables hugepages with the specified size and count. The size can be either +2MB or 1GB, and the count specifies the number of hugepages to allocate. + +If your system has multiple NUMA nodes, the total amount of hugepages will be +divided equally among them. + +Resources Limits +================ + +.. note:: + + By default, system will calculate and set the recommended values for + resource limits. Avoid tuning these values if you are not sure what you + are doing. + +During operations VPP utilizes a significant amount of system resources, +especially memory. There are two main settings that may need to be +adjusted to ensure VPP runs smoothly: + +Maximum number of memory map areas a process may have: + +.. cfgcmd:: set system option resource-limits max-map-count + +Maximum shared memory segment size: + +.. cfgcmd:: set system option resource-limits shmmax + +Both settings are automatically calculated based on configured hugepages. + +Kernel Tuning +============= + +VPP performance greatly benefits from proper kernel tuning, especially +CPU isolation and disabling unnecessary kernel features. These +optimizations ensure dedicated CPU cores are available exclusively for +VPP dataplane processing without interference from the kernel scheduler +or other system processes. + +.. warning:: + + Kernel tuning changes require a system reboot to take effect. + + Improper CPU isolation can lead to system instability if essential system + processes are starved of CPU resources. + +CPU Isolation and Optimization +------------------------------- + +CPU isolation is crucial for VPP performance as it dedicates specific +CPU cores exclusively to VPP dataplane processing. The isolated cores are +removed from the kernel scheduler and will not run regular system +processes. + +**Disable NMI Watchdog** + +The NMI (Non-Maskable Interrupt) watchdog can interfere with VPP +performance by generating interrupts on isolated cores and is not +compatible with nohz-full mode: + +.. cfgcmd:: set system option kernel cpu disable-nmi-watchdog + + Disables the NMI watchdog for detecting hard CPU lockups. This + prevents unnecessary interrupts on VPP worker cores. + +**CPU Core Isolation** + +.. cfgcmd:: set system option kernel cpu isolate-cpus + + Isolates specified CPUs from the kernel scheduler. Isolated cores will + not run regular system processes and are dedicated to applications like + VPP. + + The ```` can be: + + * Single core: ``2`` + * Range: ``2-5`` + * Mixed: ``1,3-5,7`` + + .. important:: + + Always reserve at least 2 cores for the operating system to ensure + system stability. For example, on a 4-core system, isolate cores + 2-3 for VPP and leave cores 0-1 for the OS. + + Assign the first isolated core as the VPP main core and the + remaining isolated cores as VPP worker cores. Ensure that VPP CPU + assignments match the isolated CPU range. + +**Adaptive-Tick Mode** + +.. cfgcmd:: set system option kernel cpu nohz-full + + Enables adaptive-tick mode (NO_HZ_FULL) for specified CPUs. This + causes the kernel to avoid sending scheduling-clock interrupts to CPUs + that have only one runnable task, significantly reducing interrupt + overhead for dedicated workloads like VPP. + + Use the same CPU range as configured for ``isolate-cpus``. + +**RCU Callback Offloading** + +.. cfgcmd:: set system option kernel cpu rcu-no-cbs + + Offloads Read-Copy-Update (RCU) callback processing from specified + CPUs. This ensures that RCU callbacks do not prevent the specified CPUs + from entering dyntick-idle or adaptive-tick mode, which is essential + for nohz-full functionality. + + Use the same CPU range as configured for ``isolate-cpus``. + +System Optimization +-------------------- + +Additional kernel optimizations can further improve VPP performance by +disabling unnecessary features and reducing system overhead. + +**Disable High Precision Event Timer** + +.. cfgcmd:: set system option kernel disable-hpet + + Disables the High Precision Event Timer (HPET). HPET can cause + additional interrupts and overhead that may impact VPP performance. + +**Disable Machine Check Exceptions** + +.. cfgcmd:: set system option kernel disable-mce + + Disables Machine Check Exception (MCE) reporting and handling. While + MCE provides hardware error detection, it can introduce latency in + high-performance scenarios. + +**Disable CPU Power Saving** + +.. cfgcmd:: set system option kernel disable-power-saving + + Disables CPU power saving mechanisms (C-states). This keeps CPU cores + at maximum performance levels, eliminating latency from power state + transitions. + +**Disable Soft Lockup Detection** + +.. cfgcmd:: set system option kernel disable-softlockup + + Disables the soft lockup detector for kernel threads. This prevents + false positives when VPP worker threads are busy processing packets. + +**Disable CPU Mitigations** + +.. cfgcmd:: set system option kernel disable-mitigations + + Disables all optional CPU mitigations for security vulnerabilities + (for example, Spectre, Meltdown). This may improve performance on some + platforms. + +Optimal Configuration Example +----------------------------- + +For a system with 4 CPU cores (0-3) where cores 2-3 are dedicated to VPP: + +.. code-block:: none + + # Kernel CPU optimizations + set system option kernel cpu disable-nmi-watchdog + set system option kernel cpu isolate-cpus '2-3' + set system option kernel cpu nohz-full '2-3' + set system option kernel cpu rcu-no-cbs '2-3' + + # System optimizations + set system option kernel disable-hpet + set system option kernel disable-mce + set system option kernel disable-power-saving + set system option kernel disable-softlockup + + # VPP CPU assignment + set vpp settings resource-allocation cpu-cores '2' diff --git a/docs/vpp/configuration/dataplane/rst-unix.rst b/docs/vpp/configuration/dataplane/rst-unix.rst new file mode 100644 index 00000000..f31d6381 --- /dev/null +++ b/docs/vpp/configuration/dataplane/rst-unix.rst @@ -0,0 +1,55 @@ +:lastproofread: 2026-02-27 + +.. _vpp_config_dataplane_unix: + +.. include:: /_include/need_improvement.txt + +################################ +VPP Unix Dataplane Configuration +################################ + +The UNIX configuration section is used to control VPP's interaction +with the underlying operating system, including operations scheduling. + +VPP relies on the polling mechanism to efficiently manage I/O operations +and system events. By default VPP continuously polls for events, which +leads to permanent 100% CPU usage by all cores assigned to VPP dataplane. +This is optimal for performance, but may not be desirable in all +environments, especially where power consumption is a concern or where VPP +is running inside a hypervisor, especially if the VM has burstable +thresholds and CPU usage limits. + +To mitigate this, VPP provides a configurable polling delay that allows +reducing CPU usage by introducing a delay between polling cycles. This +introduces a trade-off between CPU usage and latency, as longer delays +can lead to increased latency in processing events. + +You can configure the polling delay using the following command in the +VyOS CLI: + +.. cfgcmd:: set vpp settings poll-sleep-usec + +Sets the polling delay in microseconds. A value of 0 means no delay +(default), while higher values introduce a delay between polling cycles. + +Troubleshooting +=============== + +Setting the polling delay too high can lead to increased latency and +reduced performance, as VPP may not respond to events as quickly. +Conversely, setting it too low may result in high CPU usage, which can be +problematic in resource-constrained environments. + +Symptoms of improper configuration may include: + +- Increased latency in packet processing +- Higher CPU usage than expected +- Packets lost due to buffer overruns + +If you do not need to reduce CPU usage, it is recommended to leave the +polling delay at its default value of 0 for optimal performance. + +If you need to reduce CPU usage, you may also consider using ``interrupt`` or +``adaptive`` :ref:`DPDK driver modes `, +which can provide a balance between performance and resource utilization +without affecting polling behavior. diff --git a/docs/vpp/configuration/dataplane/system.md b/docs/vpp/configuration/dataplane/system.md new file mode 100644 index 00000000..51ee8f54 --- /dev/null +++ b/docs/vpp/configuration/dataplane/system.md @@ -0,0 +1,212 @@ +--- +lastproofread: '2026-02-27' +--- + +(vpp_config_system)= + +```{include} /_include/need_improvement.txt +``` + +# VyOS Configuration for VPP + +(vpp-config-hugepages)= + +## Hugepages + +VPP uses hugepages for efficient memory management. Hugepages are larger +memory pages that reduce the overhead of page management and improve +performance for applications that require large amounts of memory. + +Hugepages can be configured in VyOS using the following commands: + +:::{warning} +Changes to hugepage settings require a system reboot to take effect. + +Hugepages must be enabled before VPP configuration is applied. +::: + +To enable hugepages: + +```{cfgcmd} set system option kernel memory hugepage-size \ hugepage-count '\' +``` + +Enables hugepages with the specified size and count. The size can be either +2MB or 1GB, and the count specifies the number of hugepages to allocate. + +If your system has multiple NUMA nodes, the total amount of hugepages will be +divided equally among them. + +## Resources Limits + +:::{note} +By default, system will calculate and set the recommended values for +resource limits. Avoid tuning these values if you are not sure what you +are doing. +::: + +During operations VPP utilizes a significant amount of system resources, +especially memory. There are two main settings that may need to be +adjusted to ensure VPP runs smoothly: + +Maximum number of memory map areas a process may have: + +```{cfgcmd} set system option resource-limits max-map-count \ +``` + +Maximum shared memory segment size: + +```{cfgcmd} set system option resource-limits shmmax \ +``` + +Both settings are automatically calculated based on configured hugepages. + +## Kernel Tuning + +VPP performance greatly benefits from proper kernel tuning, especially +CPU isolation and disabling unnecessary kernel features. These +optimizations ensure dedicated CPU cores are available exclusively for +VPP dataplane processing without interference from the kernel scheduler +or other system processes. + +:::{warning} +Kernel tuning changes require a system reboot to take effect. + +Improper CPU isolation can lead to system instability if essential +system processes are starved of CPU resources. +::: + +### CPU Isolation and Optimization + +CPU isolation is crucial for VPP performance as it dedicates specific +CPU cores exclusively to VPP dataplane processing. The isolated cores are +removed from the kernel scheduler and will not run regular system +processes. + +**Disable NMI Watchdog** + +The NMI (Non-Maskable Interrupt) watchdog can interfere with VPP +performance by generating interrupts on isolated cores and is not +compatible with nohz-full mode: + +```{cfgcmd} set system option kernel cpu disable-nmi-watchdog + +Disables the NMI watchdog for detecting hard CPU lockups. This +prevents unnecessary interrupts on VPP worker cores. +``` + +**CPU Core Isolation** + +```{cfgcmd} set system option kernel cpu isolate-cpus \ + +Isolates specified CPUs from the kernel scheduler. Isolated cores will +not run regular system processes and are dedicated to applications like +VPP. + +The ```` can be: +* Single core: ``2`` +* Range: ``2-5`` +* Mixed: ``1,3-5,7`` + +:::{important} +Always reserve at least 2 cores for the operating system to ensure +system stability. For example, on a 4-core system, isolate cores +2-3 for VPP and leave cores 0-1 for the OS. + +Assign the first isolated core as the VPP main core and the +remaining isolated cores as VPP worker cores. Ensure that VPP CPU +assignments match the isolated CPU range. +::: +``` + +**Adaptive-Tick Mode** + +```{cfgcmd} set system option kernel cpu nohz-full \ + +Enables adaptive-tick mode (NO_HZ_FULL) for specified CPUs. This +causes the kernel to avoid sending scheduling-clock interrupts to CPUs +that have only one runnable task, significantly reducing interrupt +overhead for dedicated workloads like VPP. + +Use the same CPU range as configured for ``isolate-cpus``. +``` + +**RCU Callback Offloading** + +```{cfgcmd} set system option kernel cpu rcu-no-cbs \ + +Offloads Read-Copy-Update (RCU) callback processing from specified +CPUs. This ensures that RCU callbacks do not prevent the specified CPUs +from entering dyntick-idle or adaptive-tick mode, which is essential +for nohz-full functionality. + +Use the same CPU range as configured for ``isolate-cpus``. +``` + +### System Optimization + +Additional kernel optimizations can further improve VPP performance by +disabling unnecessary features and reducing system overhead. + +**Disable High Precision Event Timer** + +```{cfgcmd} set system option kernel disable-hpet + +Disables the High Precision Event Timer (HPET). HPET can cause +additional interrupts and overhead that may impact VPP performance. +``` + +**Disable Machine Check Exceptions** + +```{cfgcmd} set system option kernel disable-mce + +Disables Machine Check Exception (MCE) reporting and handling. While +MCE provides hardware error detection, it can introduce latency in +high-performance scenarios. +``` + +**Disable CPU Power Saving** + +```{cfgcmd} set system option kernel disable-power-saving + +Disables CPU power saving mechanisms (C-states). This keeps CPU cores +at maximum performance levels, eliminating latency from power state +transitions. +``` + +**Disable Soft Lockup Detection** + +```{cfgcmd} set system option kernel disable-softlockup + +Disables the soft lockup detector for kernel threads. This prevents +false positives when VPP worker threads are busy processing packets. +``` + +**Disable CPU Mitigations** + +```{cfgcmd} set system option kernel disable-mitigations + +Disables all optional CPU mitigations for security vulnerabilities +(for example, Spectre, Meltdown). This may improve performance on some +platforms. +``` + +### Optimal Configuration Example + +For a system with 4 CPU cores (0-3) where cores 2-3 are dedicated to VPP: + +```none +# Kernel CPU optimizations +set system option kernel cpu disable-nmi-watchdog +set system option kernel cpu isolate-cpus '2-3' +set system option kernel cpu nohz-full '2-3' +set system option kernel cpu rcu-no-cbs '2-3' + +# System optimizations +set system option kernel disable-hpet +set system option kernel disable-mce +set system option kernel disable-power-saving +set system option kernel disable-softlockup + +# VPP CPU assignment +set vpp settings resource-allocation cpu-cores '2' +``` diff --git a/docs/vpp/configuration/dataplane/system.rst b/docs/vpp/configuration/dataplane/system.rst deleted file mode 100644 index 6a2756fd..00000000 --- a/docs/vpp/configuration/dataplane/system.rst +++ /dev/null @@ -1,208 +0,0 @@ -:lastproofread: 2026-02-27 - -.. _vpp_config_system: - -.. include:: /_include/need_improvement.txt - -########################## -VyOS Configuration for VPP -########################## - -.. _vpp_config_hugepages: -.. _vpp-config-hugepages: - -Hugepages -========= - -VPP uses hugepages for efficient memory management. Hugepages are larger -memory pages that reduce the overhead of page management and improve -performance for applications that require large amounts of memory. - -Hugepages can be configured in VyOS using the following commands: - -.. warning:: - - Changes to hugepage settings require a system reboot to take effect. - - Hugepages must be enabled before VPP configuration is applied. - -To enable hugepages: - -.. cfgcmd:: set system option kernel memory hugepage-size hugepage-count - '' - -Enables hugepages with the specified size and count. The size can be either -2MB or 1GB, and the count specifies the number of hugepages to allocate. - -If your system has multiple NUMA nodes, the total amount of hugepages will be -divided equally among them. - -Resources Limits -================ - -.. note:: - - By default, system will calculate and set the recommended values for - resource limits. Avoid tuning these values if you are not sure what you - are doing. - -During operations VPP utilizes a significant amount of system resources, -especially memory. There are two main settings that may need to be -adjusted to ensure VPP runs smoothly: - -Maximum number of memory map areas a process may have: - -.. cfgcmd:: set system option resource-limits max-map-count - -Maximum shared memory segment size: - -.. cfgcmd:: set system option resource-limits shmmax - -Both settings are automatically calculated based on configured hugepages. - -Kernel Tuning -============= - -VPP performance greatly benefits from proper kernel tuning, especially -CPU isolation and disabling unnecessary kernel features. These -optimizations ensure dedicated CPU cores are available exclusively for -VPP dataplane processing without interference from the kernel scheduler -or other system processes. - -.. warning:: - - Kernel tuning changes require a system reboot to take effect. - - Improper CPU isolation can lead to system instability if essential system - processes are starved of CPU resources. - -CPU Isolation and Optimization -------------------------------- - -CPU isolation is crucial for VPP performance as it dedicates specific -CPU cores exclusively to VPP dataplane processing. The isolated cores are -removed from the kernel scheduler and will not run regular system -processes. - -**Disable NMI Watchdog** - -The NMI (Non-Maskable Interrupt) watchdog can interfere with VPP -performance by generating interrupts on isolated cores and is not -compatible with nohz-full mode: - -.. cfgcmd:: set system option kernel cpu disable-nmi-watchdog - - Disables the NMI watchdog for detecting hard CPU lockups. This - prevents unnecessary interrupts on VPP worker cores. - -**CPU Core Isolation** - -.. cfgcmd:: set system option kernel cpu isolate-cpus - - Isolates specified CPUs from the kernel scheduler. Isolated cores will - not run regular system processes and are dedicated to applications like - VPP. - - The ```` can be: - - * Single core: ``2`` - * Range: ``2-5`` - * Mixed: ``1,3-5,7`` - - .. important:: - - Always reserve at least 2 cores for the operating system to ensure - system stability. For example, on a 4-core system, isolate cores - 2-3 for VPP and leave cores 0-1 for the OS. - - Assign the first isolated core as the VPP main core and the - remaining isolated cores as VPP worker cores. Ensure that VPP CPU - assignments match the isolated CPU range. - -**Adaptive-Tick Mode** - -.. cfgcmd:: set system option kernel cpu nohz-full - - Enables adaptive-tick mode (NO_HZ_FULL) for specified CPUs. This - causes the kernel to avoid sending scheduling-clock interrupts to CPUs - that have only one runnable task, significantly reducing interrupt - overhead for dedicated workloads like VPP. - - Use the same CPU range as configured for ``isolate-cpus``. - -**RCU Callback Offloading** - -.. cfgcmd:: set system option kernel cpu rcu-no-cbs - - Offloads Read-Copy-Update (RCU) callback processing from specified - CPUs. This ensures that RCU callbacks do not prevent the specified CPUs - from entering dyntick-idle or adaptive-tick mode, which is essential - for nohz-full functionality. - - Use the same CPU range as configured for ``isolate-cpus``. - -System Optimization --------------------- - -Additional kernel optimizations can further improve VPP performance by -disabling unnecessary features and reducing system overhead. - -**Disable High Precision Event Timer** - -.. cfgcmd:: set system option kernel disable-hpet - - Disables the High Precision Event Timer (HPET). HPET can cause - additional interrupts and overhead that may impact VPP performance. - -**Disable Machine Check Exceptions** - -.. cfgcmd:: set system option kernel disable-mce - - Disables Machine Check Exception (MCE) reporting and handling. While - MCE provides hardware error detection, it can introduce latency in - high-performance scenarios. - -**Disable CPU Power Saving** - -.. cfgcmd:: set system option kernel disable-power-saving - - Disables CPU power saving mechanisms (C-states). This keeps CPU cores - at maximum performance levels, eliminating latency from power state - transitions. - -**Disable Soft Lockup Detection** - -.. cfgcmd:: set system option kernel disable-softlockup - - Disables the soft lockup detector for kernel threads. This prevents - false positives when VPP worker threads are busy processing packets. - -**Disable CPU Mitigations** - -.. cfgcmd:: set system option kernel disable-mitigations - - Disables all optional CPU mitigations for security vulnerabilities - (for example, Spectre, Meltdown). This may improve performance on some - platforms. - -Optimal Configuration Example ------------------------------ - -For a system with 4 CPU cores (0-3) where cores 2-3 are dedicated to VPP: - -.. code-block:: none - - # Kernel CPU optimizations - set system option kernel cpu disable-nmi-watchdog - set system option kernel cpu isolate-cpus '2-3' - set system option kernel cpu nohz-full '2-3' - set system option kernel cpu rcu-no-cbs '2-3' - - # System optimizations - set system option kernel disable-hpet - set system option kernel disable-mce - set system option kernel disable-power-saving - set system option kernel disable-softlockup - - # VPP CPU assignment - set vpp settings resource-allocation cpu-cores '2' diff --git a/docs/vpp/configuration/dataplane/unix.md b/docs/vpp/configuration/dataplane/unix.md new file mode 100644 index 00000000..a1f6a1fd --- /dev/null +++ b/docs/vpp/configuration/dataplane/unix.md @@ -0,0 +1,57 @@ +--- +lastproofread: '2026-02-27' +--- + +(vpp-config-dataplane-unix)= + +```{include} /_include/need_improvement.txt +``` + + +# VPP Unix Dataplane Configuration + +The UNIX configuration section is used to control VPP's interaction +with the underlying operating system, including operations scheduling. + +VPP relies on the polling mechanism to efficiently manage I/O operations +and system events. By default VPP continuously polls for events, which +leads to permanent 100% CPU usage by all cores assigned to VPP dataplane. +This is optimal for performance, but may not be desirable in all +environments, especially where power consumption is a concern or where VPP +is running inside a hypervisor, especially if the VM has burstable +thresholds and CPU usage limits. + +To mitigate this, VPP provides a configurable polling delay that allows +reducing CPU usage by introducing a delay between polling cycles. This +introduces a trade-off between CPU usage and latency, as longer delays +can lead to increased latency in processing events. + +You can configure the polling delay using the following command in the +VyOS CLI: + +```{cfgcmd} set vpp settings poll-sleep-usec \ +``` + +Sets the polling delay in microseconds. A value of 0 means no delay +(default), while higher values introduce a delay between polling cycles. + +## Troubleshooting + +Setting the polling delay too high can lead to increased latency and +reduced performance, as VPP may not respond to events as quickly. +Conversely, setting it too low may result in high CPU usage, which can be +problematic in resource-constrained environments. + +Symptoms of improper configuration may include: + +- Increased latency in packet processing +- Higher CPU usage than expected +- Packets lost due to buffer overruns + +If you do not need to reduce CPU usage, it is recommended to leave the +polling delay at its default value of 0 for optimal performance. + +If you need to reduce CPU usage, you may also consider using `interrupt` or +`adaptive` {ref}`DPDK driver modes `, +which can provide a balance between performance and resource utilization +without affecting polling behavior. diff --git a/docs/vpp/configuration/dataplane/unix.rst b/docs/vpp/configuration/dataplane/unix.rst deleted file mode 100644 index f31d6381..00000000 --- a/docs/vpp/configuration/dataplane/unix.rst +++ /dev/null @@ -1,55 +0,0 @@ -:lastproofread: 2026-02-27 - -.. _vpp_config_dataplane_unix: - -.. include:: /_include/need_improvement.txt - -################################ -VPP Unix Dataplane Configuration -################################ - -The UNIX configuration section is used to control VPP's interaction -with the underlying operating system, including operations scheduling. - -VPP relies on the polling mechanism to efficiently manage I/O operations -and system events. By default VPP continuously polls for events, which -leads to permanent 100% CPU usage by all cores assigned to VPP dataplane. -This is optimal for performance, but may not be desirable in all -environments, especially where power consumption is a concern or where VPP -is running inside a hypervisor, especially if the VM has burstable -thresholds and CPU usage limits. - -To mitigate this, VPP provides a configurable polling delay that allows -reducing CPU usage by introducing a delay between polling cycles. This -introduces a trade-off between CPU usage and latency, as longer delays -can lead to increased latency in processing events. - -You can configure the polling delay using the following command in the -VyOS CLI: - -.. cfgcmd:: set vpp settings poll-sleep-usec - -Sets the polling delay in microseconds. A value of 0 means no delay -(default), while higher values introduce a delay between polling cycles. - -Troubleshooting -=============== - -Setting the polling delay too high can lead to increased latency and -reduced performance, as VPP may not respond to events as quickly. -Conversely, setting it too low may result in high CPU usage, which can be -problematic in resource-constrained environments. - -Symptoms of improper configuration may include: - -- Increased latency in packet processing -- Higher CPU usage than expected -- Packets lost due to buffer overruns - -If you do not need to reduce CPU usage, it is recommended to leave the -polling delay at its default value of 0 for optimal performance. - -If you need to reduce CPU usage, you may also consider using ``interrupt`` or -``adaptive`` :ref:`DPDK driver modes `, -which can provide a balance between performance and resource utilization -without affecting polling behavior. -- cgit v1.2.3