summaryrefslogtreecommitdiff
path: root/docs
diff options
context:
space:
mode:
authorLiudmylaNad <l.nadolina@vyos.io>2026-08-04 19:16:07 +0200
committerGitHub <noreply@github.com>2026-08-04 18:16:07 +0100
commit1d1fb3f579dfbe83f0e89a051b2d7ddce9289fac (patch)
treeeac5e57e849cb30b46d59506f6cab45bed4217da /docs
parentea6a61f3c36ce24bb8c966a5ae3f3b72934f644f (diff)
downloadvyos-documentation-1d1fb3f579dfbe83f0e89a051b2d7ddce9289fac.tar.gz
vyos-documentation-1d1fb3f579dfbe83f0e89a051b2d7ddce9289fac.zip
docs: Update Conntrack sync page to VyOS 1.5 standards (#2147)
* docs: Update Conntrack sync page to VyOS 1.5 standards * Minor corrections * Update conntrack-sync.md * Use "active/backup" terminology consistent with official conntrackd docs Co-authored-by: Daniil Baturin <daniil@baturin.org> --------- Co-authored-by: Daniil Baturin <daniil@baturin.org>
Diffstat (limited to 'docs')
-rw-r--r--docs/configuration/service/conntrack-sync.md431
1 files changed, 313 insertions, 118 deletions
diff --git a/docs/configuration/service/conntrack-sync.md b/docs/configuration/service/conntrack-sync.md
index 61fa7034..6c471720 100644
--- a/docs/configuration/service/conntrack-sync.md
+++ b/docs/configuration/service/conntrack-sync.md
@@ -1,200 +1,357 @@
+---
+myst:
+ html_meta:
+ description: |
+ Conntrack sync preserves active sessions during failover between
+ active and backup routers in a VyOS high-availability pair by
+ continuously syncing conntrack entries between them.
+ keywords: conntrack-sync, connection-tracking, ha, high-availability, vrrp
+---
+
(conntrack-sync)=
-# Conntrack Sync
-
-One of the important features built on top of the Netfilter framework is
-connection tracking. Connection tracking allows the kernel to keep track of all
-logical network connections or sessions, and thereby relate all of the packets
-which may make up that connection. NAT relies on this information to translate
-all related packets in the same way, and iptables can use this information to
-act as a stateful firewall.
-
-The connection state however is completely independent of any upper-level
-state, such as TCP's or SCTP's state. Part of the reason for this is that when
-merely forwarding packets, i.e. no local delivery, the TCP engine may not
-necessarily be invoked at all. Even connectionless-mode transmissions such as
-UDP, IPsec (AH/ESP), GRE and other tunneling protocols have, at least, a pseudo
-connection state. The heuristic for such protocols is often based upon a preset
-timeout value for inactivity, after whose expiration a Netfilter connection is
-dropped.
-
-Each Netfilter connection is uniquely identified by a (layer-3 protocol, source
-address, destination address, layer-4 protocol, layer-4 key) tuple. The layer-4
-key depends on the transport protocol; for TCP/UDP it is the port numbers, for
-tunnels it can be their tunnel ID, but otherwise is just zero, as if it were
-not part of the tuple. To be able to inspect the TCP port in all cases, packets
-will be mandatorily defragmented.
-
-It is possible to use either Multicast or Unicast to sync conntrack traffic.
-Most examples below show Multicast, but unicast can be specified by using the
-"peer" keyword after the specified interface, as in the following example:
-
-{cfgcmd}`set service conntrack-sync interface eth0 peer 192.168.0.250`
+# Conntrack sync
+
+Conntrack Sync (Connection Tracking Synchronization) is used to
+preserve active sessions during a failover between the active and
+backup routers in a high-availability
+({abbr}`HA (High Availability)`) pair.
+
+Each active session on the active router is tracked as a local conntrack
+entry, and for some protocols, expect entries are also created for
+anticipated follow-up connections (for example, the FTP data channel
+or SIP media streams). Conntrack sync continuously copies these
+entries to backup routers over one or more dedicated interfaces, so it
+holds a current picture of every active session in near real time.
+This traffic is carried over IPv4, either as multicast (the default)
+or unicast to a configured peer address.
+
+When a failover occurs, a backup router takes over all active
+sessions and maintains them in their current state, without dropping
+or resetting them. A takeover is triggered by VRRP (Virtual Router
+Redundancy Protocol), which detects when the current active becomes
+unavailable and transitions one of the backup routers to active.
## Configuration
-```{cfgcmd} set service conntrack-sync accept-protocol
+### Failover integration
-Accept only certain protocols: You may want to replicate the state of flows
-depending on their layer 4 protocol.
+```{cfgcmd} set service conntrack-sync failover-mechanism vrrp sync-group \<name\>
-Protocols are: tcp, sctp, dccp, udp, icmp and ipv6-icmp.
+**Bind the conntrack-sync service to the specified VRRP sync-group.**
+
+This setting is mandatory. The referenced group must already be
+configured under `high-availability vrrp sync-group`.
```
+Example:
-```{cfgcmd} set service conntrack-sync event-listen-queue-size \<size\>
+```none
+set service conntrack-sync failover-mechanism vrrp sync-group syncgrp
+```
-The daemon doubles the size of the netlink event socket buffer size if it
-detects netlink event message dropping. This clause sets the maximum buffer
-size growth that can be reached.
+### Sync transport
-Queue size for listening to local conntrack events in MB.
+```{cfgcmd} set service conntrack-sync interface \<name\>
+
+**Configure the interface used to exchange conntrack state with the
+peer.**
+
+The interface must have an IPv4 address assigned.
+
+Repeat the command to configure multiple interfaces. In this case,
+interfaces must use the same transport mode (either all multicast or
+all unicast).
```
+Example:
-```{cfgcmd} set service conntrack-sync expect-sync \<all|ftp|h323|nfs|sip|sqlnet\>
+```none
+set service conntrack-sync interface eth1
+```
+
+```{cfgcmd} set service conntrack-sync interface \<name\> peer \<address\>
-Protocol for which expect entries need to be synchronized.
+**Configure the peer's IPv4 address for unicast sync on the specified
+interface.**
+
+Setting `peer` switches this interface from multicast (the default) to
+unicast mode.
+
+Repeat the command for all interfaces configured for conntrack state
+exchange, since mixing unicast and multicast interfaces causes commit
+to fail.
```
+Example:
+
+```none
+set service conntrack-sync interface eth1 peer 192.0.2.2
+```
+
+```{cfgcmd} set service conntrack-sync interface \<name\> port \<1-65535\>
+
+**Configure the UDP port used on the specified sync interface.**
-```{cfgcmd} set service conntrack-sync failover-mechanism vrrp sync-group \<group\>
+In multicast mode, it is the UDP port used by the multicast group. In
+unicast mode, it is the UDP destination port on the peer.
-Failover mechanism to use for conntrack-sync.
+The default port is 3780.
+```
+
+Example:
-Only VRRP is supported. Required option.
+```none
+set service conntrack-sync interface eth1 port 3781
```
+```{cfgcmd} set service conntrack-sync listen-address \<address\>
+
+**Configure the local IPv4 address the router listens on for unicast
+sync traffic.**
-```{cfgcmd} set service conntrack-sync ignore-address \<x.x.x.x\>
+The command applies only when conntrack sync operates in unicast mode.
-IP addresses or networks for which local conntrack entries will not be synced
+The listen address must be assigned to the sync interface. Using an
+address from another interface may cause conntrack sync to fail
+silently.
+
+Repeat the command to configure multiple listen addresses.
```
+Example:
-```{cfgcmd} set service conntrack-sync interface \<name\>
+```none
+set service conntrack-sync listen-address 192.0.2.1
+```
+
+```{cfgcmd} set service conntrack-sync mcast-group \<ipv4-multicast-address\>
+
+**Configure the IPv4 multicast group both routers in the HA pair join
+to exchange sync traffic.**
+
+The command applies only when conntrack sync operates in multicast
+mode. All routers must be configured with the same group.
+
+The default is 225.0.0.50.
+```
+
+Example:
-Interface to use for syncing conntrack entries.
+```none
+set service conntrack-sync mcast-group 225.0.0.60
```
+### Sync scope
+
+```{cfgcmd} set service conntrack-sync accept-protocol \<tcp | udp | icmp | icmp6 | sctp | dccp\>
-```{cfgcmd} set service conntrack-sync interface \<name\> port \<port\>
+**Configure the protocol whose local conntrack entries are
+synchronized with the HA peer.**
-Port number used by connection.
+Repeat the command to configure multiple protocols. When omitted,
+conntrack entries for all tracked protocols are synchronized.
```
+Example:
+
+```none
+set service conntrack-sync accept-protocol tcp
+set service conntrack-sync accept-protocol udp
+set service conntrack-sync accept-protocol icmp
+```
+
+```{cfgcmd} set service conntrack-sync ignore-address \<address | prefix\>
+
+**Exclude local conntrack entries involving the specified IP address
+or prefix from being synchronized with the HA peer.**
-```{cfgcmd} set service conntrack-sync listen-address \<ipv4address\>
+Accepts IPv4 and IPv6 addresses or prefixes.
-Local IPv4 addresses for service to listen on.
+Repeat the command to configure multiple values.
```
+Example:
-```{cfgcmd} set service conntrack-sync mcast-group \<x.x.x.x\>
+```none
+set service conntrack-sync ignore-address 192.0.2.0/24
+set service conntrack-sync ignore-address 2001:db8::/32
+```
+
+```{cfgcmd} set service conntrack-sync expect-sync \<all | ftp | h323 | nfs | sip | sqlnet\>
+
+**Configure the protocol whose expect entries are synchronized with
+the HA peer.**
-Multicast group to use for syncing conntrack entries.
+Repeat the command to configure multiple protocols. Use `all` to
+synchronize expect entries for all supported protocols. `all` cannot
+be combined with any other value.
-Defaults to 225.0.0.50.
+When omitted, no expect entries are synchronized.
```
+Example:
-```{cfgcmd} set service conntrack-sync interface \<name\> peer \<address\>
+```none
+set service conntrack-sync expect-sync ftp
+set service conntrack-sync expect-sync sip
+```
+
+### Buffers and timers
+
+```{cfgcmd} set service conntrack-sync event-listen-queue-size \<0-4294967295\>
+
+**Configure the maximum size of the conntrack event buffer (in megabytes).**
+
+The buffer starts at 2 MB and grows up to the specified value if
+events arrive faster than they can be processed.
+
+The default is 8.
+```
+
+Example:
-Peer to send unicast UDP conntrack sync entries to, if not using Multicast
-configuration from above.
+```none
+set service conntrack-sync event-listen-queue-size 16
```
+```{cfgcmd} set service conntrack-sync sync-queue-size \<0-4294967295\>
+
+**Configure the size of the queue that holds sync messages sent between peers (in megabytes)**
-```{cfgcmd} set service conntrack-sync sync-queue-size \<size\>
+The same value is applied to both directions, in either multicast or
+unicast mode.
-Queue size for syncing conntrack entries in MB.
+The default is 1.
```
+Example:
-```{cfgcmd} set service conntrack-sync disable-external-cache
+```none
+set service conntrack-sync sync-queue-size 4
+```
+
+```{cfgcmd} set service conntrack-sync purge-timeout \<1-2147483647\>
-This disables the external cache and directly injects the flow-states into the
-in-kernel Connection Tracking System of the backup firewall.
+**Configure the delay, in seconds, before synchronized entries are
+purged after a handover.**
+
+The default is 60.
```
+```{note}
+If your setup allows a recovered router to reclaim the active role,
+set the VRRP `preempt-delay` to at least this value on the VRRP
+`sync-group` bound to the conntrack sync service. This gives the
+recovered router time to receive the current conntrack entries before
+taking over.
+```
-```{cfgcmd} set service conntrack-sync purge-timeout \<timeout\>
+Example:
-Timeout (in seconds) for purging synchronized entries on handover events.
+```none
+set service conntrack-sync purge-timeout 60
+```
-On handover, ``conntrackd -t`` is invoked, which schedules a conntrack table
-flush after ``<timeout>`` seconds to purge stale (“zombie”) entries and
-reduce clashes when multiple handovers occur in a short period.
-The default is 60 seconds.
+### Behavior
+
+```{cfgcmd} set service conntrack-sync disable-external-cache
+
+**Inject conntrack entries directly to backup routers' kernel connection
+tracking table as they arrive, rather than holding them in the
+external cache used by default.**
```
-:::{note}
-In VRRP stateful firewall deployments, align VRRP timing with this
-behavior: because synchronized conntrack state is purged after the purge
-timeout, set **VRRP preempt-delay** to ≥ **purge-timeout** so mastership
-can be restored before conntrack state is purged.
-:::
+Example:
+
+```none
+set service conntrack-sync disable-external-cache
+```
```{cfgcmd} set service conntrack-sync disable-syslog
-Disable connection logging via Syslog.
+**Disable syslog logging of conntrack-sync operational events.**
```
+Example:
+
+```none
+set service conntrack-sync disable-syslog
+```
```{cfgcmd} set service conntrack-sync startup-resync
-Order conntrackd to request a complete conntrack table resync against
-the other node at startup.
+**Request a full copy of conntrack entries from the HA peer when the
+conntrack sync service starts.**
+```
+
+Example:
+
+```none
+set service conntrack-sync startup-resync
```
## Operation
-```{opcmd} show conntrack table \<ipv4|ipv6\>
+### Show
-Make sure conntrack is enabled by running and show connection tracking table.
+```{opcmd} show conntrack table \<ipv4 | ipv6\>
-:::{code-block} none
-vyos@vyos:~$ show conntrack table ipv4
-Id Original src Original dst Original packets Original bytes Reply src Reply dst Reply packets Reply bytes Protocol State Timeout Mark Zone
----------- -------------- -------------- ------------------ ---------------- ------------ -------------- --------------- ------------- ---------- ----------- --------- ------ ------
-282920088 10.0.2.2:41724 10.0.2.15:22 79 8741 10.0.2.15:22 10.0.2.2:41724 52 8495 tcp TIME_WAIT 3 0
-102466872 192.168.50.1 192.168.50.4 31953 2684052 192.168.50.4 192.168.50.1 31953 2684052 icmp 29 0 110
-1445684978 10.0.2.2:37762 10.0.2.15:22 111 9969 10.0.2.15:22 10.0.2.2:37762 86 13323 tcp ESTABLISHED 431999 0
-3302843234 10.0.2.2:37758 10.0.2.15:22 2612 3845685 10.0.2.15:22 10.0.2.2:37758 254 17447 tcp TIME_WAIT 11 0
-:::
-:::{note}
-If the table is empty and you have a warning message, it means
-conntrack is not enabled. To enable conntrack, just create a NAT or a firewall
-rule.
-{cfgcmd}`set firewall global-options state-policy established action accept`
-:::
+**Show conntrack entries for the specified address family (IPv4 or
+IPv6).**
```
+Example output:
-```{opcmd} show conntrack table \<ipv4|ipv6\> vrf \<vrf-name\>
+```none
+vyos@vyos:~$ show conntrack table ipv4
+Id Original src Original dst Original packets Original bytes Reply src Reply dst Reply packets Reply bytes Protocol State Timeout Mark Zone
+---------- ---------------- ---------------- ------------------ ---------------- ---------------- ---------------- --------------- ------------- ---------- ----------- --------- ------ ------
+282920088 192.0.2.2:41724 192.0.2.15:22 79 8741 192.0.2.15:22 192.0.2.2:41724 52 8495 tcp TIME_WAIT 3 0
+102466872 198.51.100.1 198.51.100.4 31953 2684052 198.51.100.4 198.51.100.1 31953 2684052 icmp 29 0 110
+1445684978 192.0.2.2:37762 192.0.2.15:22 111 9969 192.0.2.15:22 192.0.2.2:37762 86 13323 tcp ESTABLISHED 431999 0
+3302843234 192.0.2.2:37758 192.0.2.15:22 2612 3845685 192.0.2.15:22 192.0.2.2:37758 254 17447 tcp TIME_WAIT 11 0
+```
-Show connection tracking table filtered by VRF name. Only entries originating
-from the specified VRF are displayed.
+```{note}
+If the table is empty and you see a warning message, conntrack is
+not enabled. To enable it, create a NAT or a firewall rule, for
+example: `set firewall global-options state-policy established
+action accept`.
```
+```{opcmd} show conntrack-sync cache external [main]
+
+**Show the regular conntrack entries in the external cache.**
+
+The `main` keyword is optional and produces the same output.
+```
-```{opcmd} show conntrack-sync cache external
+```{opcmd} show conntrack-sync cache external expect
-Show connection syncing external cache entries
+**Show the expect entries in the external cache.**
```
+```{opcmd} show conntrack-sync cache internal [main]
-```{opcmd} show conntrack-sync cache internal
+**Show the regular conntrack entries in the internal cache.**
-Show connection syncing internal cache entries
+The `main` keyword is optional and produces the same output.
```
+```{opcmd} show conntrack-sync cache internal expect
+
+**Show the expect entries in the internal cache.**
+```
```{opcmd} show conntrack-sync statistics
-Retrieve current statistics of connection tracking subsystem.
+**Show operational statistics for the conntrack sync service.**
+```
+
+Example output:
-:::{code-block} none
+```none
vyos@vyos:~$ show conntrack-sync statistics
Main Table Statistics:
@@ -220,34 +377,71 @@ multicast traffic (active device=eth0.5):
message tracking:
0 Malformed msgs 263 Lost msgs
-:::
```
+
+```{opcmd} show conntrack statistics
+
+**Show operational statistics for the kernel connection tracking
+subsystem.**
+```
+
```{opcmd} show conntrack-sync status
-Retrieve current status of connection tracking subsystem.
+**Show the current operational status of the conntrack sync
+service.**
+```
+
+Example output:
-:::{code-block} none
+```none
vyos@vyos:~$ show conntrack-sync status
sync-interface : eth0.5
failover-mechanism : vrrp [sync-group GEFOEKOM]
last state transition : no transition yet!
ExpectationSync : disabled
-:::
+```
+
+### Restart/reset
+
+```{opcmd} restart conntrack-sync
+
+**Restart the conntrack sync service. On restart, the local cache is
+cleared.**
+```
+
+```{opcmd} reset conntrack-sync external-cache
+
+**Clear the external cache and request a fresh copy of conntrack
+entries from the HA peer.**
+```
+
+```{opcmd} reset conntrack-sync internal-cache
+
+**Clear the internal cache and request a fresh copy of conntrack
+entries from the HA peer.**
```
## Example
-The next example is a simple configuration of conntrack-sync.
+The following shows how to configure a two-node HA pair with
+conntrack sync.
:::{figure} /_static/images/service_conntrack_sync-schema.webp
-:alt: Conntrack Sync Example
-:scale: 60 %
+:alt: Conntrack sync example
+:scale: 80 %
+Conntrack sync example
:::
-Now configure conntrack-sync service on `router1` **and** `router2`
+Apply the following configuration on both `router1` and `router2`. The
+only difference between the two nodes is the VRRP priority: use `200`
+on `router1` (which becomes VRRP master, i.e., active) and `100` on `router2` (which
+becomes backup).
```none
-set high-availability vrrp group internal virtual-address ... etc ...
+set high-availability vrrp group internal interface 'eth1'
+set high-availability vrrp group internal vrid '10'
+set high-availability vrrp group internal priority '200'
+set high-availability vrrp group internal virtual-address '192.0.2.254/24'
set high-availability vrrp sync-group syncgrp member 'internal'
set service conntrack-sync accept-protocol 'tcp'
set service conntrack-sync accept-protocol 'udp'
@@ -257,11 +451,12 @@ set service conntrack-sync interface 'eth0'
set service conntrack-sync mcast-group '225.0.0.50'
```
-On the active router, you should have information in the internal-cache of
-conntrack-sync. The same current active connections number should be shown in
-the external-cache of the standby router
+After commit, the active router populates its internal cache with
+local conntrack entries and sends them to the backup, which stores
+them in its external cache. Running `show conntrack-sync statistics`
+on each peer reflects this asymmetry.
-On active router run:
+On the active router:
```none
$ show conntrack-sync statistics
@@ -292,7 +487,7 @@ message tracking:
0 Malformed msgs 0 Lost msgs
```
-On standby router run:
+On the backup router:
```none
$ show conntrack-sync statistics