Merge pull request #1016 from sever-sever/reverse-proxy

Add load-balancing reverse-proxy documentation and examples

3 files changed, 612 insertions, 311 deletions

diff --git a/docs/configuration/loadbalancing/index.rst b/docs/configuration/loadbalancing/index.rst
index 18f01347..382bd0d7 100644
--- a/docs/configuration/loadbalancing/index.rst
+++ b/docs/configuration/loadbalancing/index.rst
@@ -1,315 +1,12 @@
-:lastproofread: 2023-01-27
-
 .. _load-balancing:
 
-WAN load balancing
-==================
-
-Outbound traffic can be balanced between two or more outbound interfaces.
-If a path fails, traffic is balanced across the remaining healthy paths,
-a recovered path is automatically added back to the routing table and used by 
-the load balancer. The load balancer automatically adds routes for each path to
-the routing table and balances traffic across the configured interfaces,
-determined by interface health and weight.
-
-
-In a minimal configuration, the following must be provided:
-
- * an interface with a nexthop
- * one rule with a LAN (inbound-interface) and the WAN (interface).
-
-Let's assume we have two DHCP WAN interfaces and one LAN (eth2):
-
-.. code-block:: none
-
-    set load-balancing wan interface-health eth0 nexthop 'dhcp'
-    set load-balancing wan interface-health eth1 nexthop 'dhcp'
-    set load-balancing wan rule 1 inbound-interface 'eth2'
-    set load-balancing wan rule 1 interface eth0
-    set load-balancing wan rule 1 interface eth1
-
-.. note::
-
-    WAN Load Balacing should not be used when dynamic routing protocol is
-    used/needed. This feature creates customized routing tables and firewall
-    rules, that makes it incompatible to use with routing protocols.
-
-Balancing Rules
----------------
-
-Interfaces, their weight and the type of traffic to be balanced are defined in
-numbered balancing rule sets. The rule sets are executed in numerical order
-against outgoing packets. In case of a match the packet is sent through an
-interface specified in the matching rule. If a packet doesn't match any rule
-it is sent by using the system routing table. Rule numbers can't be changed.
-
-Create a load balancing rule, it can be a number between 1 and 9999:
-
-.. code-block:: none
-
-    vyos@vyos# set load-balancing wan rule 1
-    Possible completions:
-    description             Description for this rule
-    > destination           Destination
-    exclude                 Exclude packets matching this rule from wan load balance
-    failover                Enable failover for packets matching this rule from wan load balance
-    inbound-interface       Inbound interface name (e.g., "eth0") [REQUIRED]
-    +> interface            Interface name [REQUIRED]
-    > limit                 Enable packet limit for this rule
-    per-packet-balancing    Option to match traffic per-packet instead of the default, per-flow
-    protocol                Protocol to match
-    > source                Source information
-
-Interface weight
-****************
-
-Let's expand the example from above and add weight to the interfaces.
-The bandwidth from eth0 is larger than eth1. Per default, outbound traffic is
-distributed randomly across available interfaces. Weights can be assigned to
-interfaces to influence the balancing.
-
-.. code-block:: none
-
-    set load-balancing wan rule 1 interface eth0 weight 2
-    set load-balancing wan rule 1 interface eth1 weight 1
-
-66% of traffic is routed to eth0, eth1 gets 33% of traffic.
-
-Rate limit
-**********
-
-A packet rate limit can be set for a rule to apply the rule to traffic above or
-below a specified threshold. To configure the rate limiting use:
-
-.. code-block:: none
-
-    set load-balancing wan rule <rule> limit <parameter>
-
-* ``burst``: Number of packets allowed to overshoot the limit within ``period``.
-  Default 5.
-* ``period``: Time window for rate calculation. Possible values:
-  ``second`` (one second), ``minute`` (one minute), ``hour`` (one hour).
-  Default is ``second``.
-* ``rate``: Number of packets. Default 5.
-* ``threshold``: ``below`` or ``above`` the specified rate limit.
-
-Flow and packet-based balancing
-*******************************
-
-Outgoing traffic is balanced in a flow-based manner.
-A connection tracking table is used to track flows by their source address,
-destination address and port. Each flow is assigned to an interface according
-to the defined balancing rules and subsequent packets are sent through the
-same interface. This has the advantage that packets always arrive in order if
-links with different speeds are in use.
-
-Packet-based balancing can lead to a better balance across interfaces when out
-of order packets are no issue. Per-packet-based balancing can be set for a
-balancing rule with:
-
-.. code-block:: none
-
-    set load-balancing wan rule <rule> per-packet-balancing
-
-Exclude traffic
-***************
-
-To exclude traffic from load balancing, traffic matching an exclude rule is not
-balanced but routed through the system routing table instead:
-
-.. code-block:: none
-
-    set load-balancing wan rule <rule> exclude
-
-
-Health checks
--------------
-
-The health of interfaces and paths assigned to the load balancer is
-periodically checked by sending ICMP packets (ping) to remote destinations,
-a TTL test or the execution of a user defined script. If an interface fails the
-health check it is removed from the load balancer's pool of interfaces.
-To enable health checking for an interface:
-
-.. code-block:: none
-
-    vyos@vyos# set load-balancing wan interface-health <interface>
-    Possible completions:
-    failure-count    Failure count
-    nexthop          Outbound interface nexthop address. Can be 'dhcp or ip address' [REQUIRED]
-    success-count    Success count
-    +> test          Rule number
-
-Specify nexthop on the path to the destination, ``ipv4-address`` can be set to
-``dhcp``
-
-.. code-block:: none
-
-    set load-balancing wan interface-health <interface> nexthop <ipv4-address>
-
-Set the number of health check failures before an interface is marked as
-unavailable, range for number is 1 to 10, default 1. Or set the number of
-successful health checks before an interface is added back to the interface
-pool, range for number is 1 to 10, default 1.
-
-.. code-block:: none
-
-    set load-balancing wan interface-health <interface> failure-count <number>
-    set load-balancing wan interface-health <interface> success-count <number>
-
-Each health check is configured in its own test, tests are numbered and
-processed in numeric order. For multi target health checking multiple tests
-can be defined:
-
-.. code-block:: none
-
-    vyos@vyos# set load-balancing wan interface-health eth1 test 0
-    Possible completions:
-    resp-time    Ping response time (seconds)
-    target       Health target address
-    test-script  Path to user defined script
-    ttl-limit    Ttl limit (hop count)
-    type         WLB test type
-
-* ``resp-time``: the maximum response time for ping in seconds.
-  Range 1...30, default 5
-* ``target``: the target to be sent ICMP packets to, address can be an IPv4
-  address or hostname
-* ``test-script``: A user defined script must return 0 to be considered
-  successful and non-zero to fail. Scripts are located in /config/scripts,
-  for different locations the full path needs to be provided
-* ``ttl-limit``: For the UDP TTL limit test the hop count limit must be
-  specified. The limit must be shorter than the path length, an ICMP time
-  expired message is needed to be returned for a successful test. default 1
-* ``type``: Specify the type of test. type can be ping, ttl or a user defined
-  script
-
-Source NAT rules
-----------------
-
-Per default, interfaces used in a load balancing pool replace the source IP
-of each outgoing packet with its own address to ensure that replies arrive on
-the same interface. This works through automatically generated source NAT (SNAT)
-rules, these rules are only applied to balanced traffic. In cases where this
-behaviour is not desired, the automatic generation of SNAT rules can be
-disabled:
-
-.. code-block:: none
-
-    set load-balancing wan disable-source-nat
-
-Sticky Connections
-------------------
-Inbound connections to a WAN interface can be improperly handled when the reply
-is sent back to the client.
-
-.. image:: /_static/images/sticky-connections.jpg
-   :width: 80%
-   :align: center
-
-
-Upon reception of an incoming packet, when a response is sent, it might be
-desired to ensure that it leaves from the same interface as the inbound one.
-This can be achieved by enabling sticky connections in the load balancing:
-
-.. code-block:: none
-
-    set load-balancing wan sticky-connections inbound
-
-Failover
---------
-
-In failover mode, one interface is set to be the primary interface and other
-interfaces are secondary or spare. Instead of balancing traffic across all
-healthy interfaces, only the primary interface is used and in case of failure,
-a secondary interface selected from the pool of available interfaces takes over.
-The primary interface is selected based on its weight and health, others become
-secondary interfaces. Secondary interfaces to take over a failed primary
-interface are chosen from the load balancer's interface pool, depending
-on their weight and health. Interface roles can also be selected based on rule
-order by including interfaces in balancing rules and ordering those rules
-accordingly. To put the load balancer in failover mode, create a failover rule:
-
-.. code-block:: none
-
-    set load-balancing wan rule <number> failover
-
-Because existing sessions do not automatically fail over to a new path,
-the session table can be flushed on each connection state change:
-
-.. code-block:: none
-
-    set load-balancing wan flush-connections
-
-.. warning::
-
-    Flushing the session table will cause other connections to fall back from
-    flow-based to packet-based balancing until each flow is reestablished.
-
-Script execution
-----------------
-
-A script can be run when an interface state change occurs. Scripts are run
-from /config/scripts, for a different location specify the full path:
-
-.. code-block:: none
-
-    set load-balancing wan hook script-name
-
-Two environment variables are available:
-
-* ``WLB_INTERFACE_NAME=[interfacename]``: Interface to be monitored
-* ``WLB_INTERFACE_STATE=[ACTIVE|FAILED]``: Interface state
-
-.. warning::
-
-    Blocking call with no timeout. System will become unresponsive if script
-    does not return!
-
-Handling and monitoring
------------------------
-
-
-Show WAN load balancer information including test types and targets.
-A character at the start of each line depicts the state of the test
-
-* ``+`` successful
-* ``-`` failed
-* a blank indicates that no test has been carried out
-
-.. code-block:: none
-
-    vyos@vyos:~$ show wan-load-balance
-    Interface:  eth0
-    Status:  failed
-    Last Status Change:  Tue Jun 11 20:12:19 2019
-    -Test:  ping  Target:
-        Last Interface Success:  55s
-        Last Interface Failure:  0s
-        # Interface Failure(s):  5
-
-    Interface:  eth1
-    Status:  active
-    Last Status Change:  Tue Jun 11 20:06:42 2019
-    +Test:  ping  Target:
-        Last Interface Success:  0s
-        Last Interface Failure:  6m26s
-        # Interface Failure(s):  0
-
-Show connection data of load balanced traffic:
-
-.. code-block:: none
-
-    vyos@vyos:~$ show wan-load-balance connection
-    conntrack v1.4.2 (conntrack-tools): 3 flow entries have been shown.
-    Type    State           Src                     Dst                     Packets Bytes
-    tcp     TIME_WAIT       10.1.1.13:38040         203.0.113.2:80          203.0.113.2  192.168.188.71
-    udp                     10.1.1.13:41891         198.51.100.3:53         198.51.100.3 192.168.188.71
-    udp                     10.1.1.13:55437         198.51.100.3:53         198.51.100.3 192.168.188.71
-
-Restart
-*******
+##############
+Load-balancing
+##############
 
-.. code-block:: none
+.. toctree::
+   :maxdepth: 1
+   :includehidden:
 
-    restart wan-load-balance
+   wan
+   reverse-proxy
diff --git a/docs/configuration/loadbalancing/reverse-proxy.rst b/docs/configuration/loadbalancing/reverse-proxy.rst
new file mode 100644
index 00000000..24f26af0
--- /dev/null
+++ b/docs/configuration/loadbalancing/reverse-proxy.rst
@@ -0,0 +1,289 @@
+
+.. _load-balancing:
+
+#############
+Reverse-proxy
+#############
+.. include:: /_include/need_improvement.txt
+
+VyOS reverse-proxy is balancer and proxy server that provides
+high-availability, load balancing and proxying for TCP (level 4)
+and HTTP-based (level 7) applications.
+
+Configuration
+=============
+
+
+Service configuration is responsible for binding to a specific port,
+while the backend configuration determines the type of load balancing
+to be applied and specifies the real servers to be utilized.
+
+Service
+-------
+
+.. cfgcmd:: set load-balancing reverse-proxy service <name> listen-address
+   <address>
+
+  Set service to bind on IP address, by default listen on any IPv4 and IPv6
+
+.. cfgcmd:: set load-balancing reverse-proxy service <name> port
+   <port>
+
+  Create service `<name>` to listen on <port>
+
+.. cfgcmd:: set load-balancing reverse-proxy service <name> mode
+   <tcp|http>
+
+  Configure service `<name>` mode TCP or HTTP
+
+.. cfgcmd:: set load-balancing reverse-proxy service <name> backend
+   <name>
+
+  Configure service `<name>` to use the backend <name>
+
+.. cfgcmd:: set load-balancing reverse-proxy service <name> ssl
+   certificate <name>
+
+  Set SSL certeficate <name> for service <name>
+
+
+Rules
+^^^^^
+Rules allow to control and route incoming traffic to specific backend based
+on predefined conditions. Rules allow to define matching criteria and
+perform action accordingly.
+
+.. cfgcmd:: set load-balancing reverse-proxy service <name> rule <rule>
+   domain-name <name>
+
+  Match domain name
+
+.. cfgcmd:: set load-balancing reverse-proxy service <name> rule <rule>
+   ssl <sni>
+
+  SSL match Server Name Indication (SNI) option:
+   * ``req-ssl-sni`` SSL Server Name Indication (SNI) request match
+   * ``ssl-fc-sni`` SSL frontend connection Server Name Indication match
+   * ``ssl-fc-sni-end`` SSL frontend match end of connection Server Name 
+   Indication
+
+.. cfgcmd:: set load-balancing reverse-proxy service <name> rule <rule>
+   url-path <match> <url>
+
+  Allows to define URL path matching rules for a specific service.
+
+  With this command, you can specify how the URL path should be matched 
+  against incoming requests.
+
+  The available options for <match> are:
+   * ``begin`` Matches the beginning of the URL path
+   * ``end`` Matches the end of the URL path.
+   * ``exact`` Requires an exactly match of the URL path
+
+.. cfgcmd:: set load-balancing reverse-proxy service <name> rule <rule>
+   set backend <name>
+
+  Assign a specific backend to a rule
+
+.. cfgcmd:: set load-balancing reverse-proxy service <name> rule <rule>
+   redirect-location <url>
+
+  Redirect URL to a new location
+
+
+Backend
+-------
+
+.. cfgcmd:: set load-balancing reverse-proxy backend <name> balance
+   <balance>
+
+  Load-balancing algorithms to be used for distributind requests among the
+  vailable servers
+
+  Balance algorithms:
+   * ``source-address`` Distributes requests based on the source IP address
+     of the client
+   * ``round-robin`` Distributes requests in a circular manner,
+     sequentially sending each request to the next server in line
+   * ``least-connection`` Distributes requests tp tje server wotj the fewest 
+     active connections
+
+.. cfgcmd:: set load-balancing reverse-proxy backend <name> mode
+   <mode>
+
+  Configure backend `<name>` mode TCP or HTTP
+
+.. cfgcmd:: set load-balancing reverse-proxy backend <name> parameters
+   http-check
+
+  Enable layer 7 HTTP health check
+
+.. cfgcmd:: set load-balancing reverse-proxy backend <name> server
+   <name> address <x.x.x.x>
+
+  Set the address of the backend server to which the incoming traffic will
+  be forwarded
+
+.. cfgcmd:: set load-balancing reverse-proxy backend <name> server
+   <name> port <port>
+
+  Set the address of the backend port
+
+.. cfgcmd:: set load-balancing reverse-proxy backend <name> server
+   <name> check
+
+  Active health check backend server
+
+.. cfgcmd:: set load-balancing reverse-proxy backend <name> server
+   <name> send-proxy
+
+  Send a Proxy Protocol version 1 header (text format)
+
+.. cfgcmd:: set load-balancing reverse-proxy backend <name> server
+   <name> send-proxy-v2
+
+  Send a Proxy Protocol version 2 header (binary format)
+
+
+
+Gloabal
+-------
+
+Global parameters
+
+.. cfgcmd:: set load-balancing reverse-proxy global-parameters max-connections
+   <num>
+
+  Limit maximum number of connections
+
+.. cfgcmd:: set load-balancing reverse-proxy global-parameters ssl-bind-ciphers
+   <ciphers>
+
+  Limit allowed cipher algorithms used during SSL/TLS handshake
+
+.. cfgcmd:: set load-balancing reverse-proxy global-parameters tls-version-min
+   <version>
+
+  Specify the minimum required TLS version 1.2 or 1.3
+
+
+Redirect HTTP to HTTPS
+======================
+Configure the load-balancing reverse-proxy service for HTTP.
+
+This configuration listen on port 80 and redirect incoming
+requests to HTTPS:
+
+.. code-block:: none
+
+    set load-balancing reverse-proxy service http port '80'
+    set load-balancing reverse-proxy service http redirect-http-to-https
+
+The name of the service can be different, in this example it is only for 
+convenience.
+
+
+Examples
+========
+
+Level 4 balancing
+-----------------
+
+This configuration enables the TCP reverse proxy for the "my-tcp-api" service.
+Incoming TCP connections on port 8888 will be load balanced across the backend
+servers (srv01 and srv02) using the round-robin load-balancing algorithm.
+
+.. code-block:: none
+
+    set load-balancing reverse-proxy service my-tcp-api backend 'bk-01'
+    set load-balancing reverse-proxy service my-tcp-api mode 'tcp'
+    set load-balancing reverse-proxy service my-tcp-api port '8888'
+
+    set load-balancing reverse-proxy backend bk-01 balance 'round-robin'
+    set load-balancing reverse-proxy backend bk-01 mode 'tcp'
+
+    set load-balancing reverse-proxy backend bk-01 server srv01 address '192.0.2.11'
+    set load-balancing reverse-proxy backend bk-01 server srv01 port '8881'
+    set load-balancing reverse-proxy backend bk-01 server srv02 address '192.0.2.12'
+    set load-balancing reverse-proxy backend bk-01 server srv02 port '8882'
+
+Balancing based on domain name
+------------------------------
+The following configuration demonstrates how to use VyOS
+to achieve load balancing based on the domain name.
+
+The HTTP service listen on TCP port 80.
+
+Rule 10 matches requests with the domain name ``node1.example.com`` forwards
+to the backend ``bk-api-01``
+
+Rule 20 matches requests with the domain name ``node2.example.com`` forwards
+to the backend ``bk-api-02``
+
+.. code-block:: none
+
+    set load-balancing reverse-proxy service http description 'bind app listen on 443 port'
+    set load-balancing reverse-proxy service http mode 'tcp'
+    set load-balancing reverse-proxy service http port '80'
+
+    set load-balancing reverse-proxy service http rule 10 domain-name 'node1.example.com'
+    set load-balancing reverse-proxy service http rule 10 set backend 'bk-api-01'
+    set load-balancing reverse-proxy service http rule 20 domain-name 'node2.example.com'
+    set load-balancing reverse-proxy service http rule 20 set backend 'bk-api-02'
+
+    set load-balancing reverse-proxy backend bk-api-01 description 'My API-1'
+    set load-balancing reverse-proxy backend bk-api-01 mode 'tcp'
+    set load-balancing reverse-proxy backend bk-api-01 server api01 address '127.0.0.1'
+    set load-balancing reverse-proxy backend bk-api-01 server api01 port '4431'
+    set load-balancing reverse-proxy backend bk-api-02 description 'My API-2'
+    set load-balancing reverse-proxy backend bk-api-02 mode 'tcp'
+    set load-balancing reverse-proxy backend bk-api-02 server api01 address '127.0.0.2'
+    set load-balancing reverse-proxy backend bk-api-02 server api01 port '4432'
+
+
+Terminate SSL
+-------------
+The following configuration reverse-proxy terminate SSL.
+
+The ``http`` service is lestens on port 80 and force redirects from HTTP to
+HTTPS.
+
+The ``https`` service listens on port 443 with backend `bk-default` to 
+handle HTTPS traffic. It uses certificate named ``cert`` for SSL termination.
+
+Rule 10 matches requests with the exact URL path ``/.well-known/xxx``
+and redirects to location ``/certs/``.
+
+Rule 20 matches requests with URL paths ending in ``/mail`` or exact
+path ``/email/bar`` redirect to location ``/postfix/``.
+
+Additional global parameters are set, including the maximum number
+connection limit of 4000 and a minimum TLS version of 1.3.
+
+
+.. code-block:: none
+
+    set load-balancing reverse-proxy service http description 'Force redirect to HTTPS'
+    set load-balancing reverse-proxy service http port '80'
+    set load-balancing reverse-proxy service http redirect-http-to-https
+
+    set load-balancing reverse-proxy service https backend 'bk-default'
+    set load-balancing reverse-proxy service https description 'listen on 443 port'
+    set load-balancing reverse-proxy service https mode 'http'
+    set load-balancing reverse-proxy service https port '443'
+    set load-balancing reverse-proxy service https ssl certificate 'cert'
+
+    set load-balancing reverse-proxy service https rule 10 url-path exact '/.well-known/xxx'
+    set load-balancing reverse-proxy service https rule 10 set redirect-location '/certs/'
+    set load-balancing reverse-proxy service https rule 20 url-path end '/mail'
+    set load-balancing reverse-proxy service https rule 20 url-path exact '/email/bar'
+    set load-balancing reverse-proxy service https rule 20 set redirect-location '/postfix/'
+
+    set load-balancing reverse-proxy backend bk-default description 'Default backend'
+    set load-balancing reverse-proxy backend bk-default mode 'http'
+    set load-balancing reverse-proxy backend bk-default server sr01 address '192.0.2.23'
+    set load-balancing reverse-proxy backend bk-default server sr01 port '80'
+
+    set load-balancing reverse-proxy global-parameters max-connections '4000'
+    set load-balancing reverse-proxy global-parameters tls-version-min '1.3'
+
diff --git a/docs/configuration/loadbalancing/wan.rst b/docs/configuration/loadbalancing/wan.rst
new file mode 100644
index 00000000..18f01347
--- /dev/null
+++ b/docs/configuration/loadbalancing/wan.rst
@@ -0,0 +1,315 @@
+:lastproofread: 2023-01-27
+
+.. _load-balancing:
+
+WAN load balancing
+==================
+
+Outbound traffic can be balanced between two or more outbound interfaces.
+If a path fails, traffic is balanced across the remaining healthy paths,
+a recovered path is automatically added back to the routing table and used by 
+the load balancer. The load balancer automatically adds routes for each path to
+the routing table and balances traffic across the configured interfaces,
+determined by interface health and weight.
+
+
+In a minimal configuration, the following must be provided:
+
+ * an interface with a nexthop
+ * one rule with a LAN (inbound-interface) and the WAN (interface).
+
+Let's assume we have two DHCP WAN interfaces and one LAN (eth2):
+
+.. code-block:: none
+
+    set load-balancing wan interface-health eth0 nexthop 'dhcp'
+    set load-balancing wan interface-health eth1 nexthop 'dhcp'
+    set load-balancing wan rule 1 inbound-interface 'eth2'
+    set load-balancing wan rule 1 interface eth0
+    set load-balancing wan rule 1 interface eth1
+
+.. note::
+
+    WAN Load Balacing should not be used when dynamic routing protocol is
+    used/needed. This feature creates customized routing tables and firewall
+    rules, that makes it incompatible to use with routing protocols.
+
+Balancing Rules
+---------------
+
+Interfaces, their weight and the type of traffic to be balanced are defined in
+numbered balancing rule sets. The rule sets are executed in numerical order
+against outgoing packets. In case of a match the packet is sent through an
+interface specified in the matching rule. If a packet doesn't match any rule
+it is sent by using the system routing table. Rule numbers can't be changed.
+
+Create a load balancing rule, it can be a number between 1 and 9999:
+
+.. code-block:: none
+
+    vyos@vyos# set load-balancing wan rule 1
+    Possible completions:
+    description             Description for this rule
+    > destination           Destination
+    exclude                 Exclude packets matching this rule from wan load balance
+    failover                Enable failover for packets matching this rule from wan load balance
+    inbound-interface       Inbound interface name (e.g., "eth0") [REQUIRED]
+    +> interface            Interface name [REQUIRED]
+    > limit                 Enable packet limit for this rule
+    per-packet-balancing    Option to match traffic per-packet instead of the default, per-flow
+    protocol                Protocol to match
+    > source                Source information
+
+Interface weight
+****************
+
+Let's expand the example from above and add weight to the interfaces.
+The bandwidth from eth0 is larger than eth1. Per default, outbound traffic is
+distributed randomly across available interfaces. Weights can be assigned to
+interfaces to influence the balancing.
+
+.. code-block:: none
+
+    set load-balancing wan rule 1 interface eth0 weight 2
+    set load-balancing wan rule 1 interface eth1 weight 1
+
+66% of traffic is routed to eth0, eth1 gets 33% of traffic.
+
+Rate limit
+**********
+
+A packet rate limit can be set for a rule to apply the rule to traffic above or
+below a specified threshold. To configure the rate limiting use:
+
+.. code-block:: none
+
+    set load-balancing wan rule <rule> limit <parameter>
+
+* ``burst``: Number of packets allowed to overshoot the limit within ``period``.
+  Default 5.
+* ``period``: Time window for rate calculation. Possible values:
+  ``second`` (one second), ``minute`` (one minute), ``hour`` (one hour).
+  Default is ``second``.
+* ``rate``: Number of packets. Default 5.
+* ``threshold``: ``below`` or ``above`` the specified rate limit.
+
+Flow and packet-based balancing
+*******************************
+
+Outgoing traffic is balanced in a flow-based manner.
+A connection tracking table is used to track flows by their source address,
+destination address and port. Each flow is assigned to an interface according
+to the defined balancing rules and subsequent packets are sent through the
+same interface. This has the advantage that packets always arrive in order if
+links with different speeds are in use.
+
+Packet-based balancing can lead to a better balance across interfaces when out
+of order packets are no issue. Per-packet-based balancing can be set for a
+balancing rule with:
+
+.. code-block:: none
+
+    set load-balancing wan rule <rule> per-packet-balancing
+
+Exclude traffic
+***************
+
+To exclude traffic from load balancing, traffic matching an exclude rule is not
+balanced but routed through the system routing table instead:
+
+.. code-block:: none
+
+    set load-balancing wan rule <rule> exclude
+
+
+Health checks
+-------------
+
+The health of interfaces and paths assigned to the load balancer is
+periodically checked by sending ICMP packets (ping) to remote destinations,
+a TTL test or the execution of a user defined script. If an interface fails the
+health check it is removed from the load balancer's pool of interfaces.
+To enable health checking for an interface:
+
+.. code-block:: none
+
+    vyos@vyos# set load-balancing wan interface-health <interface>
+    Possible completions:
+    failure-count    Failure count
+    nexthop          Outbound interface nexthop address. Can be 'dhcp or ip address' [REQUIRED]
+    success-count    Success count
+    +> test          Rule number
+
+Specify nexthop on the path to the destination, ``ipv4-address`` can be set to
+``dhcp``
+
+.. code-block:: none
+
+    set load-balancing wan interface-health <interface> nexthop <ipv4-address>
+
+Set the number of health check failures before an interface is marked as
+unavailable, range for number is 1 to 10, default 1. Or set the number of
+successful health checks before an interface is added back to the interface
+pool, range for number is 1 to 10, default 1.
+
+.. code-block:: none
+
+    set load-balancing wan interface-health <interface> failure-count <number>
+    set load-balancing wan interface-health <interface> success-count <number>
+
+Each health check is configured in its own test, tests are numbered and
+processed in numeric order. For multi target health checking multiple tests
+can be defined:
+
+.. code-block:: none
+
+    vyos@vyos# set load-balancing wan interface-health eth1 test 0
+    Possible completions:
+    resp-time    Ping response time (seconds)
+    target       Health target address
+    test-script  Path to user defined script
+    ttl-limit    Ttl limit (hop count)
+    type         WLB test type
+
+* ``resp-time``: the maximum response time for ping in seconds.
+  Range 1...30, default 5
+* ``target``: the target to be sent ICMP packets to, address can be an IPv4
+  address or hostname
+* ``test-script``: A user defined script must return 0 to be considered
+  successful and non-zero to fail. Scripts are located in /config/scripts,
+  for different locations the full path needs to be provided
+* ``ttl-limit``: For the UDP TTL limit test the hop count limit must be
+  specified. The limit must be shorter than the path length, an ICMP time
+  expired message is needed to be returned for a successful test. default 1
+* ``type``: Specify the type of test. type can be ping, ttl or a user defined
+  script
+
+Source NAT rules
+----------------
+
+Per default, interfaces used in a load balancing pool replace the source IP
+of each outgoing packet with its own address to ensure that replies arrive on
+the same interface. This works through automatically generated source NAT (SNAT)
+rules, these rules are only applied to balanced traffic. In cases where this
+behaviour is not desired, the automatic generation of SNAT rules can be
+disabled:
+
+.. code-block:: none
+
+    set load-balancing wan disable-source-nat
+
+Sticky Connections
+------------------
+Inbound connections to a WAN interface can be improperly handled when the reply
+is sent back to the client.
+
+.. image:: /_static/images/sticky-connections.jpg
+   :width: 80%
+   :align: center
+
+
+Upon reception of an incoming packet, when a response is sent, it might be
+desired to ensure that it leaves from the same interface as the inbound one.
+This can be achieved by enabling sticky connections in the load balancing:
+
+.. code-block:: none
+
+    set load-balancing wan sticky-connections inbound
+
+Failover
+--------
+
+In failover mode, one interface is set to be the primary interface and other
+interfaces are secondary or spare. Instead of balancing traffic across all
+healthy interfaces, only the primary interface is used and in case of failure,
+a secondary interface selected from the pool of available interfaces takes over.
+The primary interface is selected based on its weight and health, others become
+secondary interfaces. Secondary interfaces to take over a failed primary
+interface are chosen from the load balancer's interface pool, depending
+on their weight and health. Interface roles can also be selected based on rule
+order by including interfaces in balancing rules and ordering those rules
+accordingly. To put the load balancer in failover mode, create a failover rule:
+
+.. code-block:: none
+
+    set load-balancing wan rule <number> failover
+
+Because existing sessions do not automatically fail over to a new path,
+the session table can be flushed on each connection state change:
+
+.. code-block:: none
+
+    set load-balancing wan flush-connections
+
+.. warning::
+
+    Flushing the session table will cause other connections to fall back from
+    flow-based to packet-based balancing until each flow is reestablished.
+
+Script execution
+----------------
+
+A script can be run when an interface state change occurs. Scripts are run
+from /config/scripts, for a different location specify the full path:
+
+.. code-block:: none
+
+    set load-balancing wan hook script-name
+
+Two environment variables are available:
+
+* ``WLB_INTERFACE_NAME=[interfacename]``: Interface to be monitored
+* ``WLB_INTERFACE_STATE=[ACTIVE|FAILED]``: Interface state
+
+.. warning::
+
+    Blocking call with no timeout. System will become unresponsive if script
+    does not return!
+
+Handling and monitoring
+-----------------------
+
+
+Show WAN load balancer information including test types and targets.
+A character at the start of each line depicts the state of the test
+
+* ``+`` successful
+* ``-`` failed
+* a blank indicates that no test has been carried out
+
+.. code-block:: none
+
+    vyos@vyos:~$ show wan-load-balance
+    Interface:  eth0
+    Status:  failed
+    Last Status Change:  Tue Jun 11 20:12:19 2019
+    -Test:  ping  Target:
+        Last Interface Success:  55s
+        Last Interface Failure:  0s
+        # Interface Failure(s):  5
+
+    Interface:  eth1
+    Status:  active
+    Last Status Change:  Tue Jun 11 20:06:42 2019
+    +Test:  ping  Target:
+        Last Interface Success:  0s
+        Last Interface Failure:  6m26s
+        # Interface Failure(s):  0
+
+Show connection data of load balanced traffic:
+
+.. code-block:: none
+
+    vyos@vyos:~$ show wan-load-balance connection
+    conntrack v1.4.2 (conntrack-tools): 3 flow entries have been shown.
+    Type    State           Src                     Dst                     Packets Bytes
+    tcp     TIME_WAIT       10.1.1.13:38040         203.0.113.2:80          203.0.113.2  192.168.188.71
+    udp                     10.1.1.13:41891         198.51.100.3:53         198.51.100.3 192.168.188.71
+    udp                     10.1.1.13:55437         198.51.100.3:53         198.51.100.3 192.168.188.71
+
+Restart
+*******
+
+.. code-block:: none
+
+    restart wan-load-balance