From 00faafed96d86a970bf566c79f1bfd304630ccaa Mon Sep 17 00:00:00 2001 From: zdc Date: Fri, 29 Aug 2025 19:53:13 +0300 Subject: VPP: Add comprehensive VPP documentation This commit introduces extensive documentation for VyOS VPP (Vector Packet Processing) Dataplane interfaces and features, including examples. Features documented: * VPP integration description, base requirements and limitations * VPP dataplane core settings * Features enabled in the dataplane: ACL, IPsec, NAT, sFlow * VPP interface types: bonding, bridge, GRE, IPIP, kernel, loopback, VXLAN, XConnect Note: This is an initial documentation version that will require ongoing improvements based on user feedback and questions, real-world deployment testing and validation, additional use cases and configuration scenarios discovered in production. Therefore, all pages are marked as "need improvement". --- docs/vpp/configuration/dataplane/buffers.rst | 78 +++++++++++++++++++++++ docs/vpp/configuration/dataplane/cpu.rst | 68 +++++++++++++++++++++ docs/vpp/configuration/dataplane/index.rst | 30 +++++++++ docs/vpp/configuration/dataplane/interface.rst | 85 ++++++++++++++++++++++++++ docs/vpp/configuration/dataplane/ipsec.rst | 56 +++++++++++++++++ docs/vpp/configuration/dataplane/ipv6.rst | 32 ++++++++++ docs/vpp/configuration/dataplane/l2learn.rst | 28 +++++++++ docs/vpp/configuration/dataplane/lcp.rst | 50 +++++++++++++++ docs/vpp/configuration/dataplane/logging.rst | 42 +++++++++++++ docs/vpp/configuration/dataplane/memory.rst | 49 +++++++++++++++ docs/vpp/configuration/dataplane/system.rst | 54 ++++++++++++++++ docs/vpp/configuration/dataplane/unix.rst | 36 +++++++++++ 12 files changed, 608 insertions(+) create mode 100644 docs/vpp/configuration/dataplane/buffers.rst create mode 100644 docs/vpp/configuration/dataplane/cpu.rst create mode 100644 docs/vpp/configuration/dataplane/index.rst create mode 100644 docs/vpp/configuration/dataplane/interface.rst create mode 100644 docs/vpp/configuration/dataplane/ipsec.rst create mode 100644 docs/vpp/configuration/dataplane/ipv6.rst create mode 100644 docs/vpp/configuration/dataplane/l2learn.rst create mode 100644 docs/vpp/configuration/dataplane/lcp.rst create mode 100644 docs/vpp/configuration/dataplane/logging.rst create mode 100644 docs/vpp/configuration/dataplane/memory.rst create mode 100644 docs/vpp/configuration/dataplane/system.rst create mode 100644 docs/vpp/configuration/dataplane/unix.rst (limited to 'docs/vpp/configuration/dataplane') diff --git a/docs/vpp/configuration/dataplane/buffers.rst b/docs/vpp/configuration/dataplane/buffers.rst new file mode 100644 index 00000000..b702ae69 --- /dev/null +++ b/docs/vpp/configuration/dataplane/buffers.rst @@ -0,0 +1,78 @@ +:lastproofread: 2025-09-04 + +.. _vpp_config_dataplane_buffers: + +.. include:: /_include/need_improvement.txt + +################################### +VPP Dataplane Buffers Configuration +################################### + +Buffers are essential for handling network packets efficiently, and proper configuration can enhance performance, reliability and is mandatory for VPP work. + +Buffers are used to temporarily store packets during processing, therefore their configuration should be in sync with NIC configuration, CPU threads, and overall system resources. + +Buffer Configuration Parameters +=============================== + +The following parameters can be configured for VPP buffers: + +buffers-per-numa +^^^^^^^^^^^^^^^^ + +Number of buffers allocated per NUMA node. This setting helps in optimizing memory access patterns for multi-CPU systems. +Usually it needs to be tuned if: +- there are a lot of interfaces in the system +- there are a lot of queues in NICs +- there are big descriptors size configured for NICs +The value should be set responsibly, overprovisioning can lead to issues with NICs configured with XDP driver. + +.. cfgcmd:: set vpp settings buffers buffers-per-numa + +The common approach for the calculation is to use the formula: + +.. code-block:: none + + buffers-per-numa = (num-rx-queues * num-rx-desc) + (num-tx-queues * num-tx-desc) + +This should be done for each NIC, and then sum the results for all NICs in the system needs to be multiplied by 2.5 and the result will be the minimum value for `buffers-per-numa`. + +Try to avoid setting this value too low to avoid packet drops. + +data-size +^^^^^^^^^ + +This value sets how much payload data can be stored in a single buffer allocated by VPP. +Making it larger can reduce buffer chains for big packets, while a smaller value can save memory for environments handling mostly small packets. + +.. cfgcmd:: set vpp settings buffers data-size + +Optimal size depends on the typical packet size in your network. If you are not sure, use the value of biggest MTU in your network plus some overhead (e.g., 128 bytes). + +page-size +^^^^^^^^^ + +A memory pages type used for buffer allocation. Common values are 4K, 2M, or 1G. + +Use pages that are configured in system settings. + +.. cfgcmd:: set vpp settings buffers page-size + +Potential Issues and Troubleshooting +------------------------------------ + +Improper buffer configuration can lead to various issues, including: + +- Increased latency and packet loss +- Inefficient CPU utilization +- Interfaces initialization failures + +Indicators of such issues are: + +- Errors during interfaces initialization in VPP logs +- Packet drops observed in VPP statistics + +To troubleshoot buffer-related issues, consider the following steps: + +- Review VPP logs for any errors related to buffer allocation (often the message may contains error `-5`). +- Tune available buffers by adjusting the `buffers-per-numa` and `data-size` parameters. diff --git a/docs/vpp/configuration/dataplane/cpu.rst b/docs/vpp/configuration/dataplane/cpu.rst new file mode 100644 index 00000000..97e0fd93 --- /dev/null +++ b/docs/vpp/configuration/dataplane/cpu.rst @@ -0,0 +1,68 @@ +:lastproofread: 2025-09-04 + +.. _vpp_config_dataplane_cpu: + +.. include:: /_include/need_improvement.txt + +############################### +VPP Dataplane CPU Configuration +############################### + +VPP can utilize multiple CPU cores to enhance packet processing performance. Proper CPU configuration is crucial for achieving optimal throughput and low latency. + +There are several parameters that can be configured to optimize CPU usage for VPP. + +CPU Configuration Parameters +============================ + +main-core +^^^^^^^^^ + +The main core is responsible for handling control plane operations and managing worker threads. It should be set to a core that is not heavily utilized by other processes. If not set, VPP will automatically select a core `1`. + +.. cfgcmd:: set vpp settings cpu main-core + +corelist-workers +^^^^^^^^^^^^^^^^ + +This parameter specifies the list of CPU cores that will be used as worker threads for packet processing. + +.. cfgcmd:: set vpp settings cpu corelist-workers + +The core list can be specified in various formats, such as a comma-separated list (e.g., 2,3,4) or a range (e.g., 2-4). It is advisable to select cores that are on the same NUMA node as the network interfaces to minimize latency. + +Automatic cores selection +------------------------- + +There is a possibility to let VPP select CPU cores automatically. This can be done by configuring the following two parameters: + +skip-cores +^^^^^^^^^^ + +This parameter allows you to specify number of first CPU cores that should be excluded from being used for main or worker threads. The main thread will be assigned to the first available core after the skipped ones, and worker threads will be assigned to subsequent cores. + +.. cfgcmd:: set vpp settings cpu skip-cores + +Exclude cores that are reserved for other critical system processes to ensure that VPP does not interfere with their operation. + +workers +^^^^^^^ + +This parameter allows you to specify the number of worker threads that should be created. Each worker thread will be assigned to a separate CPU core after the skipped and main ones. + +.. cfgcmd:: set vpp settings cpu workers + +The number of worker threads should be chosen based on the available CPU resources and the expected network load. A common approach is to set the number of workers to the number of available CPU cores minus the skipped and main cores. + +Potential Issues and Troubleshooting +==================================== + +Improper CPU configuration can lead to various issues, including: + +- Underperformance for both VPP (not enough cores were assigned) and kernel (too many cores were assigned to VPP) +- Resource conflicts with other processes and services + +Indicators of such issues are: + +- VPP or kernel forwarding performance is lower than expected +- Slower work of system components or services, including DNS, DHCP, dynamic routing, etc. diff --git a/docs/vpp/configuration/dataplane/index.rst b/docs/vpp/configuration/dataplane/index.rst new file mode 100644 index 00000000..90ecc6f6 --- /dev/null +++ b/docs/vpp/configuration/dataplane/index.rst @@ -0,0 +1,30 @@ +:lastproofread: 2025-09-04 + +.. _vpp_config_dataplane_index: + +.. include:: /_include/need_improvement.txt + +################################ +VPP Dataplane Core Configuration +################################ + +This section covers the core configuration options for the VPP dataplane in VyOS. It includes settings for memory management, CPU allocation, hugepages, and other essential parameters that influence the performance and behavior of the VPP dataplane. + +Please review the general system configuration, before starting to configure VPP. Without proper VyOS preconditions, VPP will not start or its efficiency will be significantly degraded. + +.. toctree:: + :maxdepth: 1 + :includehidden: + + system + buffers + cpu + interface + ipsec + ipv6 + l2learn + lcp + logging + memory + unix + \ No newline at end of file diff --git a/docs/vpp/configuration/dataplane/interface.rst b/docs/vpp/configuration/dataplane/interface.rst new file mode 100644 index 00000000..e8ab8e58 --- /dev/null +++ b/docs/vpp/configuration/dataplane/interface.rst @@ -0,0 +1,85 @@ +:lastproofread: 2025-09-04 + +.. _vpp_config_dataplane_interface: + +.. include:: /_include/need_improvement.txt + +###################################### +VPP Dataplane Interfaces Configuration +###################################### + +Only Ethernet interfaces (physical or virtual) can be connected to the VPP dataplane. Interfaces configured here act as a bridge between VPP and the outside world, allowing VPP to send and receive packets into the network. + + +Interface Configuration Parameters +================================== + +driver +^^^^^^ +The driver parameter specifies the type of driver used for the network interface. VPP supports two types of drivers that can be used for this: DPDK and XDP. The choice of driver depends on the specific use case, hardware capabilities, and performance requirements. Some NICs may support only one of these drivers. + +.. cfgcmd:: set vpp settings interface driver + +The DPDK driver is generally preferred for high-performance scenarios, while the XDP driver may be suitable for NICs that do not support DPDK. + +.. _vpp_config_dataplane_interface_rx_mode: + +rx-mode +^^^^^^^ + +The rx-mode parameter defines how VPP handles incoming packets on the interface. There are several modes available, each with its own advantages and use cases: + +- ``interrupt``: In this mode, VPP relies on hardware interrupts to notify it of incoming packets. This mode is suitable for low to moderate traffic loads and can help reduce CPU usage during idle periods. It is not recommended if low-latency processing is required. May not be supported by some NICs. +- ``polling``: In polling mode, VPP continuously checks the interface for incoming packets. This mode is ideal for high-throughput scenarios where low latency is critical, as it minimizes the time packets spend waiting to be processed. However, it can lead to higher CPU usage, especially during periods of low traffic, because the polling process is always active. +- ``adaptive``: Adaptive mode combines the benefits of both interrupt and polling modes. VPP starts in interrupt mode and switches to polling mode when the traffic load increases. + +.. cfgcmd:: set vpp settings interface rx-mode + +The choice of rx-mode should be based on the expected traffic patterns and performance requirements of the network environment. + +dpdk-options +^^^^^^^^^^^^ + +The dpdk-options section allows for the configuration of various DPDK-specific settings for the interface. + +.. cfgcmd:: set vpp settings interface dpdk-options