summaryrefslogtreecommitdiff
path: root/docs/configuration/loadbalancing/wan.md
blob: a19bbfaeed751d112ea909438075404d7f765ed6 (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
---
lastproofread: '2026-04-06'
---

# WAN load balancing

```{todo}
Convert raw command blocks in this file to cfgcmd/opcmd
directives for command coverage tracking.
```

The load balancer distributes outbound traffic across two or more
interfaces. If a path fails, the load balancer balances traffic across the
remaining healthy paths. When a path recovers, it is automatically added back
to the routing table. The load balancer adds routes for each path and
distributes traffic based on interface health and weight.

In a minimal configuration, the following must be provided:
> - An interface with a `nexthop`.
> - One rule with a LAN (inbound-interface) and the WAN (interface).

The following examples uses two DHCP WAN interfaces and one LAN (`eth2`):

```none
set load-balancing wan interface-health eth0 nexthop 'dhcp'
set load-balancing wan interface-health eth1 nexthop 'dhcp'
set load-balancing wan rule 1 inbound-interface 'eth2'
set load-balancing wan rule 1 interface eth0
set load-balancing wan rule 1 interface eth1
```

:::{note}
Do not use WAN load balancing with dynamic routing protocols. This
feature creates customized routing tables and firewall rules that are
incompatible with routing protocols.
:::

## Load balancing rules

You define interfaces, their weight, and the traffic type to balance in
numbered rule sets. The load balancer executes rules in numerical order
against outgoing packets. When a packet matches a rule, it is sent through the
specified interface. Packets that do not match any rule use the system routing
table. You cannot change rule numbers.

Create a load balancing rule, it can be a number between 1 and 9999:

```none
vyos@vyos# set load-balancing wan rule 1
Possible completions:
description             Description for this rule
> destination           Destination
exclude                 Exclude packets matching this rule from wan load balance
failover                Enable failover for packets matching this rule from wan load balance
inbound-interface       Inbound interface name (e.g., "eth0") [REQUIRED]
+> interface            Interface name [REQUIRED]
> limit                 Enable packet limit for this rule
per-packet-balancing    Option to match traffic per-packet instead of the default, per-flow
protocol                Protocol to match
> source                Source information
```


### Interface weight

By default, the load balancer distributes outbound
traffic randomly across available interfaces. You can assign weights to
interfaces to influence the distribution. If `eth0` has more bandwidth
than `eth1`, you can assign a higher weight to `eth0` to send more
traffic through it:

```none
set load-balancing wan rule 1 interface eth0 weight 2
set load-balancing wan rule 1 interface eth1 weight 1
```

In this example,\`\`eth0\`\` receives 66% of traffic, and `eth1` receives
33% of traffic.

### Rate limit

Set a packet rate limit for a rule to apply it to traffic above or below a
specified threshold. To configure rate limiting, use:

```none
set load-balancing wan rule <rule> limit <parameter>
```

- `burst`: Number of packets allowed to overshoot the limit within `period`.
  Default 5.
- `period`: Time window for rate calculation. Possible values:
  `second` (one second), `minute` (one minute), `hour` (one hour).
  Default is `second`.
- `rate`: Number of packets. Default: `5`.
- `threshold`: `below` or `above` the specified rate limit.

### Flow and packet-based balancing

The load balancer balances outgoing traffic by flow. A connection tracking
table tracks flows by source address, destination address, and port. Each
flow is assigned to an interface based on the balancing rules, and subsequent
packets use the same interface. This ensures packets arrive in order when links
have different speeds.

Packet-based balancing can improve balance across interfaces when packet
order is not critical. Enable per-packet balancing for a rule with:

```none
set load-balancing wan rule <rule> per-packet-balancing
```


### Exclude traffic

To exclude traffic from load balancing, traffic matching an exclude rule
bypasses load balancing and uses the system routing table instead:

```none
set load-balancing wan rule <rule> exclude
```


## Health checks

The load balancer periodically checks the health of interfaces and paths by
sending ICMP packets (ping) to remote destinations, performing TTL tests, or
executing a user-defined script. If an interface fails the health check, the
load balancer removes it from its interface pool.
To enable health checking for an interface:

```none
vyos@vyos# set load-balancing wan interface-health <interface>
Possible completions:
failure-count    Failure count
nexthop          Outbound interface nexthop address. Can be 'dhcp or ip address' [REQUIRED]
success-count    Success count
+> test          Rule number
```

Specify the nexthop on the path to the destination. You can set
`ipv4-address` to `dhcp`.

```none
set load-balancing wan interface-health <interface> nexthop <ipv4-address>
```

Set the number of health check failures before the load balancer marks an
interface as unavailable (range 1-10, default 1). Or set the number of
successful health checks before adding an interface back to the pool
(range 1-10, default 1).

```none
set load-balancing wan interface-health <interface> failure-count <number>
set load-balancing wan interface-health <interface> success-count <number>
```

Configure each health check in its own test. Tests are numbered and processed
in numeric order. You can define multiple tests for multi-target health
checking:

```none
vyos@vyos# set load-balancing wan interface-health eth1 test 0
Possible completions:
resp-time    Ping response time (seconds)
target       Health target address
test-script  Path to user defined script
ttl-limit    Ttl limit (hop count)
type         WLB test type
```

- `resp-time`: The maximum response time for ping in seconds. Range
  1-30, default `5`.
- `target`: The target to receive ICMP packets. The address can be an IPv4
  address or hostname.
- `test-script`: A user-defined script must return 0 to succeed and
  non-zero to fail. Scripts reside in `/config/scripts`. For other locations,
  provide the full path.
- `ttl-limit`: For the UDP TTL limit test, specify the hop count limit.
  The limit must be shorter than the path length. The test succeeds when an
  ICMP time-expired message is returned. Default `1`.
- `type`: Specify the test type: `ping`, `ttl`, or a user-defined
  script.

## Source NAT rules

By default, interfaces in a load balancing pool replace the source IP of
each outgoing packet with their own address to ensure replies arrive on the
same interface. The load balancer handles this through automatically generated
Source NAT (SNAT) rules applied only to balanced traffic. To disable the
automatic generation of SNAT rules when this behavior is not desired, use:

```none
set load-balancing wan disable-source-nat
```


## Sticky connections

Inbound connections to a WAN interface can be improperly handled when
replies are sent back to the client.

```{image} /_static/images/sticky-connections.webp
:align: center
:width: 80%
```

When responding to an incoming packet, you may want to ensure the response
leaves from the same interface as the incoming packet. Enable sticky
connections in the load balancer to do this:

```none
set load-balancing wan sticky-connections inbound
```


## Failover

In failover mode, one interface is primary and other interfaces are
secondary or spare. The load balancer uses only the primary interface. If it
fails, a secondary interface from the available pool takes over. The load
balancer selects the primary interface based on its weight and health. Other
interfaces become secondary. Secondary interfaces are chosen based on their
weight and health. You can also select interface roles based on rule order by
including interfaces in balancing rules and ordering those rules accordingly.
To enable failover mode, create a failover rule:

```none
set load-balancing wan rule <number> failover
```

Existing sessions do not automatically fail over to a new path. Flush the
session table on each connection state change to enable failover:

```none
set load-balancing wan flush-connections
```

:::{warning}
Flushing the session table causes other connections to revert from
flow-based to packet-based balancing until each flow is reestablished.
:::

## Script execution

Run a script when an interface state changes. Scripts run from the
`/config/scripts` directory. To use a script in another location,
specify the full path:

```none
set load-balancing wan hook script-name
```

Two environment variables are available:
- `WLB_INTERFACE_NAME=[interfacename]`: Interface to be monitored
- `WLB_INTERFACE_STATE=[ACTIVE|FAILED]`: Interface state

:::{warning}
Blocking call with no timeout: VyOS becomes unresponsive if the
script does not return.
:::

## Handling and monitoring

The following command shows WAN load balancer information including test
types and targets. The character at the start of each line indicates the test
state:
- `+` successful.
- `-` failed.
- A blank indicates that no test has been carried out.

```none
vyos@vyos:~$ show wan-load-balance
Interface:  eth0
Status:  failed
Last Status Change:  Tue Jun 11 20:12:19 2019
-Test:  ping  Target:
    Last Interface Success:  55s
    Last Interface Failure:  0s
    # Interface Failure(s):  5

Interface:  eth1
Status:  active
Last Status Change:  Tue Jun 11 20:06:42 2019
+Test:  ping  Target:
    Last Interface Success:  0s
    Last Interface Failure:  6m26s
    # Interface Failure(s):  0
```

Show connection data of load balanced traffic:

```none
vyos@vyos:~$ show wan-load-balance connection
conntrack v1.4.2 (conntrack-tools): 3 flow entries have been shown.
Type    State           Src                     Dst                     Packets Bytes
tcp     TIME_WAIT       10.1.1.13:38040         203.0.113.2:80          203.0.113.2  192.168.188.71
udp                     10.1.1.13:41891         198.51.100.3:53         198.51.100.3 192.168.188.71
udp                     10.1.1.13:55437         198.51.100.3:53         198.51.100.3 192.168.188.71
```


### Restart

```none
restart wan-load-balance
```