arrange loadbalancing, high availability, qos

3 files changed, 1623 insertions, 0 deletions

diff --git a/docs/configuration/highavailability/index.rst b/docs/configuration/highavailability/index.rst
new file mode 100644
index 00000000..ad714597
--- /dev/null
+++ b/docs/configuration/highavailability/index.rst
@@ -0,0 +1,158 @@
+.. _high-availability:
+
+High availability
+=================
+
+VRRP (Virtual Redundancy Protocol) provides active/backup redundancy for routers.
+Every VRRP router has a physical IP/IPv6 address, and a virtual address.
+On startup, routers elect the master, and the router with the highest priority becomes the master and assigns the virtual address to its interface.
+All routers with lower priorities become backup routers. The master then starts sending keepalive packets to notify other routers that it's available.
+If the master fails and stops sending keepalive packets, the router with the next highest priority becomes the new master and takes over the virtual address.
+
+VRRP keepalive packets use multicast, and VRRP setups are limited to a single datalink layer segment.
+You can setup multiple VRRP groups (also called virtual routers). Virtual routers are identified by a VRID (Virtual Router IDentifier).
+If you setup multiple groups on the same interface, their VRIDs must be unique, but it's possible (even if not recommended for readability reasons) to use duplicate VRIDs on different interfaces.
+
+Basic setup
+-----------
+
+VRRP groups are created with the ``set high-availability vrrp group $GROUP_NAME`` commands.
+The required parameters are interface, vrid, and virtual-address.
+
+minimal config
+
+.. code-block:: none
+
+  set high-availability vrrp group Foo vrid 10
+  set high-availability vrrp group Foo interface eth0
+  set high-availability vrrp group Foo virtual-address 192.0.2.1/24
+
+You can verify your VRRP group status with the operational mode ``run show vrrp`` command:
+
+.. code-block:: none
+
+  vyos@vyos# run show vrrp
+  Name        Interface      VRID  State    Last Transition
+  ----------  -----------  ------  -------  -----------------
+  Foo         eth1             10  MASTER   2s
+
+IPv6 support
+------------
+
+The ``virtual-address`` parameter can be either an IPv4 or IPv6 address, but you cannot mix IPv4 and IPv6 in the same group, and will need to create groups with different VRIDs specially for IPv4 and IPv6.
+
+Disabling a VRRP group
+----------------------
+
+You can disable a VRRP group with ``disable`` option:
+
+.. code-block:: none
+
+  set high-availability vrrp group Foo disable
+
+A disabled group will be removed from the VRRP process and your router will not participate in VRRP for that VRID. It will disappear from operational mode commands output, rather than enter the backup state.
+
+Setting VRRP group priority
+---------------------------
+
+VRRP priority can be set with ``priority`` option:
+
+.. code-block:: none
+
+  set high-availability vrrp group Foo priority 200
+
+The priority must be an integer number from 1 to 255. Higher priority value increases router's precedence in the master elections.
+
+Sync groups
+-----------
+
+A sync group allows VRRP groups to transition together.
+
+.. code-block:: none
+
+    edit high-availability vrrp
+    set sync-group MAIN member VLAN9
+    set sync-group MAIN member VLAN20
+
+In the following example, when VLAN9 transitions, VLAN20 will also transition:
+
+.. code-block:: none
+
+    vrrp {
+        group VLAN9 {
+            interface eth0.9
+            virtual-address 10.9.1.1/24
+            priority 200
+            vrid 9
+        }
+        group VLAN20 {
+            interface eth0.20
+            priority 200
+            virtual-address 10.20.20.1/24
+            vrid 20
+        }
+        sync-group MAIN {
+            member VLAN20
+            member VLAN9
+        }
+    }
+
+
+.. warning:: All items in a sync group should be similarly configured.  If one VRRP group is set to a different premption delay or priority, it would result in an endless transition loop.
+
+
+Preemption
+----------
+
+VRRP can use two modes: preemptive and non-preemptive. In the preemptive mode, if a router with a higher priority fails and then comes back, routers with lower priority will give up their master status. In non-preemptive mode, the newly elected master will keep the master status and the virtual address indefinitely.
+
+By default VRRP uses preemption. You can disable it with the "no-preempt" option:
+
+.. code-block:: none
+
+  set high-availability vrrp group Foo no-preempt
+
+You can also configure the time interval for preemption with the "preempt-delay" option. For example, to set the higher priority router to take over in 180 seconds, use:
+
+.. code-block:: none
+
+  set high-availability vrrp group Foo preempt-delay 180
+
+Unicast VRRP
+------------
+
+By default VRRP uses multicast packets. If your network does not support multicast for whatever reason, you can make VRRP use unicast communication instead.
+
+.. code-block:: none
+
+  set high-availability vrrp group Foo peer-address 192.0.2.10
+  set high-availability vrrp group Foo hello-source-address 192.0.2.15
+
+Scripting
+---------
+
+VRRP functionality can be extended with scripts. VyOS supports two kinds of scripts: health check scripts and transition scripts. Health check scripts execute custom checks in addition to the master router reachability.
+Transition scripts are executed when VRRP state changes from master to backup or fault and vice versa and can be used to enable or disable certain services, for example.
+
+Health check scripts
+^^^^^^^^^^^^^^^^^^^^
+
+This setup will make the VRRP process execute the ``/config/scripts/vrrp-check.sh script`` every 60 seconds, and transition the group to the fault state if it fails (i.e. exits with non-zero status) three times:
+
+.. code-block:: none
+
+  set high-availability vrrp group Foo health-check script /config/scripts/vrrp-check.sh
+  set high-availability vrrp group Foo health-check interval 60
+  set high-availability vrrp group Foo health-check failure-count 3
+
+Transition scripts
+^^^^^^^^^^^^^^^^^^
+
+Transition scripts can help you implement various fixups, such as starting and stopping services, or even modifying the VyOS config on VRRP transition.
+This setup will make the VRRP process execute the ``/config/scripts/vrrp-fail.sh`` with argument ``Foo`` when VRRP fails, and the ``/config/scripts/vrrp-master.sh`` when the router becomes the master:
+
+.. code-block:: none
+
+  set high-availability vrrp group Foo transition-script backup "/config/scripts/vrrp-fail.sh Foo"
+  set high-availability vrrp group Foo transition-script fault "/config/scripts/vrrp-fail.sh Foo"
+  set high-availability vrrp group Foo transition-script master "/config/scripts/vrrp-master.sh Foo"
diff --git a/docs/configuration/loadbalancing/index.rst b/docs/configuration/loadbalancing/index.rst
new file mode 100644
index 00000000..6b0bede9
--- /dev/null
+++ b/docs/configuration/loadbalancing/index.rst
@@ -0,0 +1,263 @@
+.. _load-balancing:
+
+WAN load balancing
+==================
+
+Outbound traffic can be balanced between two or more outbound interfaces.
+If a path fails, traffic is balanced across the remaining healthy paths, a recovered path is automatically added back to the routing table and used by the load balancer.
+The load balancer automatically adds routes for each path to the routing table and balances traffic across the configured interfaces, determined by interface health and weight.
+
+
+In a minimal, configuration the following must be provided:
+
+ * a interface with a nexthop
+ * one rule with a LAN (inbound-interface) and the WAN (interface).
+
+Let's assume we have two DHCP WAN interfaces and one LAN (eth2):
+
+.. code-block:: none
+
+    set load-balancing wan interface-health eth0 nexthop 'dhcp'
+    set load-balancing wan interface-health eth1 nexthop 'dhcp'
+    set load-balancing wan rule 1 inbound-interface 'eth2'
+    set load-balancing wan rule 1 interface eth0
+    set load-balancing wan rule 1 interface eth1
+
+Balancing Rules
+---------------
+
+Interfaces, their weight and the type of traffic to be balanced are defined in numbered balancing rule sets.
+The rule sets are executed in numerical order against outgoing packets. In case of a match the packet is sent through an interface specified in the matching rule.
+If a packet doesn't match any rule it is sent by using the system routing table. Rule numbers can't be changed.
+
+Create a load balancing rule, rule can be a number between 1 and 9999:
+
+.. code-block:: none
+
+    vyos@vyos# set load-balancing wan rule 1
+    Possible completions:
+    description             Description for this rule
+    > destination           Destination
+    exclude                 Exclude packets matching this rule from wan load balance
+    failover                Enable failover for packets matching this rule from wan load balance
+    inbound-interface       Inbound interface name (e.g., "eth0") [REQUIRED]
+    +> interface            Interface name [REQUIRED]
+    > limit                 Enable packet limit for this rule
+    per-packet-balancing    Option to match traffic per-packet instead of the default, per-flow
+    protocol                Protocol to match
+    > source                Source information
+
+Interface weight
+****************
+
+Let's expand the example from above and add a weight to the interfaces. The bandwidth from eth0 is larger than eth1.
+Per default outbound traffic is distributed randomly across available interfaces. Weights can be assigned to interfaces to influence the balancing.
+
+.. code-block:: none
+
+    set load-balancing wan rule 1 interface eth0 weight 2
+    set load-balancing wan rule 1 interface eth1 weight 1
+
+66% traffic is routed to eth0 and eth1 get 33% of traffic.
+
+Rate limit
+**********
+
+A packet rate limit can be set for a rule to apply the rule to traffic above or below a specified threshold.
+To configure the rate limiting use:
+
+.. code-block:: none
+
+    set load-balancing wan rule <rule> limit <parameter>
+
+* ``burst``: Number of packets allowed to overshoot the limit within ``period``. Default 5.
+* ``period``: Time window for rate calculation. Possible values: ``second`` (one second), ``minute`` (one minute), ``hour`` (one hour). Default is ``second``.
+* ``rate``: Number of packets. Default 5.
+* ``threshold``: ``below`` or ``above`` the specified rate limit.
+
+Flow and packet-based balancing
+*******************************
+
+Outgoing traffic is balanced in a flow-based manner.
+A connection tracking table is used to track flows by their source address, destination address and port.
+Each flow is assigned to an interface according to the defined balancing rules and subsequent packets are sent through the same interface.
+This has the advantage that packets always arrive in order if links with different speeds are in use.
+
+Packet-based balancing can lead to a better balance across interfaces when out of order packets are no issue. Per-packet-based balancing can be set for a balancing rule with:
+
+.. code-block:: none
+
+    set load-balancing wan rule <rule> per-packet-balancing
+
+Exclude traffic
+***************
+
+To exclude traffic from load balancing, traffic matching an exclude rule is not balanced but routed through the system routing table instead:
+
+.. code-block:: none
+
+    set load-balancing wan rule <rule> exclude
+
+
+Health checks
+-------------
+
+The health of interfaces and paths assigned to the load balancer is periodically checked by sending ICMP packets (ping) to remote destinations, a TTL test or the execution of a user defined script.
+If an interface fails the health check it is removed from the load balancer's pool of interfaces. To enable health checking for an interface:
+
+.. code-block:: none
+
+    vyos@vyos# set load-balancing wan interface-health <interface>
+    Possible completions:
+    failure-count    Failure count
+    nexthop          Outbound interface nexthop address. Can be 'dhcp or ip address' [REQUIRED]
+    success-count    Success count
+    +> test          Rule number
+
+Specify nexthop on the path to destination, ``ipv4-address`` can be set to ``dhcp``
+
+.. code-block:: none
+
+    set load-balancing wan interface-health <interface> nexthop <ipv4-address>
+
+Set the number of health check failures before an interface is marked as unavailable, range for number is 1 to 10, default 1.
+Or set the number of successful health checks before an interface is added back to the interface pool, range for number is 1 to 10, default 1.
+
+.. code-block:: none
+
+    set load-balancing wan interface-health <interface> failure-count <number>
+    set load-balancing wan interface-health <interface> success-count <number>
+
+Each health check is configured in its own test, tests are numbered and processed in numeric order.
+For multi target health checking multiple tests can be defined:
+
+.. code-block:: none
+
+    vyos@vyos# set load-balancing wan interface-health eth1 test 0
+    Possible completions:
+    resp-time    Ping response time (seconds)
+    target       Health target address
+    test-script  Path to user defined script
+    ttl-limit    Ttl limit (hop count)
+    type         WLB test type
+
+* ``resp-time``: the maximum response time for ping in seconds. Range 1...30, default 5
+* ``target``: the target to be sent ICMP packets to, address can be an IPv4 address or hostname
+* ``test-script``: A user defined script must return 0 to be considered successful and non-zero to fail. Scripts are located in /config/scripts, for different locations the full path needs to be provided
+* ``ttl-limit``: For the UDP TTL limit test the hop count limit must be specified. The limit must be shorter than the path length, an ICMP time expired message is needed to be returned for a successful test. default 1
+* ``type``: Specify the type of test. type can be ping, ttl or a user defined script
+
+Source NAT rules
+----------------
+
+Per default, interfaces used in a load balancing pool replace the source IP of each outgoing packet with its own address to ensure that replies arrive on the same interface.
+This works through automatically generated source NAT (SNAT) rules, these rules are only applied to balanced traffic. In cases where this behaviour is not desired, the automatic generation of SNAT rules can be disabled:
+
+.. code-block:: none
+
+    set load-balancing wan disable-source-nat
+
+Sticky Connections
+------------------
+Inbound connections to a WAN interface can be improperly handled when the reply is sent back to the client.
+
+.. image:: /_static/images/sticky-connections.jpg
+   :width: 80%
+   :align: center
+
+
+Upon reception of an incoming packet, when a response is sent, it might be desired to ensure that it leaves from the same interface as the inbound one.
+This can be achieved by enabling sticky connections in the load balancing:
+
+.. code-block:: none
+
+    set load-balancing wan sticky-connections inbound
+
+Failover
+--------
+
+In failover mode, one interface is set to be the primary interface and other interfaces are secondary or spare.
+Instead of balancing traffic across all healthy interfaces, only the primary interface is used and in case of failure, a secondary interface selected from the pool of available interfaces takes over.
+The primary interface is selected based on its weight and health, others become secondary interfaces.
+Secondary interfaces to take over a failed primary interface are chosen from the load balancer's interface pool, depending on their weight and health.
+Interface roles can also be selected based on rule order by including interfaces in balancing rules and ordering those rules accordingly. To put the load balancer in failover mode, create a failover rule:
+
+.. code-block:: none
+
+    set load-balancing wan rule <number> failover
+
+Because existing sessions do not automatically fail over to a new path, the session table can be flushed on each connection state change:
+
+.. code-block:: none
+
+    set load-balancing wan flush-connections
+
+.. warning::
+
+    Flushing the session table will cause other connections to fall back from flow-based to packet-based balancing until each flow is reestablished.
+
+Script execution
+----------------
+
+A script can be run when an interface state change occurs. Scripts are run from /config/scripts, for a different location specify the full path:
+
+.. code-block:: none
+
+    set load-balancing wan hook script-name
+
+Two environment variables are available:
+
+* ``WLB_INTERFACE_NAME=[interfacename]``: Interface to be monitored
+* ``WLB_INTERFACE_STATE=[ACTIVE|FAILED]``: Interface state
+
+.. warning::
+
+    Blocking call with no timeout. System will become unresponsive if script does not return!
+
+Handling and monitoring
+-----------------------
+
+
+Show WAN load balancer information including test types and targets.
+A character at the start of each line depicts the state of the test
+
+* ``+`` successful
+* ``-`` failed
+* a blank indicates that no test has been carried out
+
+.. code-block:: none
+
+    vyos@vyos:~$ show wan-load-balance
+    Interface:  eth0
+    Status:  failed
+    Last Status Change:  Tue Jun 11 20:12:19 2019
+    -Test:  ping  Target:
+        Last Interface Success:  55s
+        Last Interface Failure:  0s
+        # Interface Failure(s):  5
+
+    Interface:  eth1
+    Status:  active
+    Last Status Change:  Tue Jun 11 20:06:42 2019
+    +Test:  ping  Target:
+        Last Interface Success:  0s
+        Last Interface Failure:  6m26s
+        # Interface Failure(s):  0
+
+Show connection data of load balanced traffic:
+
+.. code-block:: none
+
+    vyos@vyos:~$ show wan-load-balance connection
+    conntrack v1.4.2 (conntrack-tools): 3 flow entries have been shown.
+    Type    State           Src                     Dst                     Packets Bytes
+    tcp     TIME_WAIT       10.1.1.13:38040         203.0.113.2:80          203.0.113.2  192.168.188.71
+    udp                     10.1.1.13:41891         198.51.100.3:53         198.51.100.3 192.168.188.71
+    udp                     10.1.1.13:55437         198.51.100.3:53         198.51.100.3 192.168.188.71
+
+Restart
+*******
+
+.. code-block:: none
+
+    restart wan-load-balance
diff --git a/docs/configuration/trafficpolicy/index.rst b/docs/configuration/trafficpolicy/index.rst
new file mode 100644
index 00000000..babccd6f
--- /dev/null
+++ b/docs/configuration/trafficpolicy/index.rst
@@ -0,0 +1,1202 @@
+.. _qos:
+
+##############
+Traffic Policy
+##############
+
+
+***
+QoS
+***
+
+The generic name of Quality of Service or Traffic Control involves
+things like shaping traffic, scheduling or dropping packets, which
+are the kind of things you may want to play with when you have, for
+instance, a bandwidth bottleneck in a link and you want to somehow
+prioritize some type of traffic over another.
+
+tc_ is a powerful tool for Traffic Control found at the Linux kernel.
+However, its configuration is often considered a cumbersome task.
+Fortunately, VyOS eases the job through its CLI, while using ``tc`` as
+backend.
+
+
+How to make it work
+===================
+
+In order to have VyOS Traffic Control working you need to follow 2
+steps:
+
+ 1. **Create a traffic policy**.
+
+ 2. **Apply the traffic policy to an interface ingress or egress**.
+    
+
+But before learning to configure your policy, we will warn you
+about the different units you can use and also show you what *classes*
+are and how they work, as some policies may require you to configure
+them.
+
+
+Units
+=====
+
+When configuring your traffic policy, you will have to set data rate
+values, watch out the units you are managing, it is easy to get confused
+with the different prefixes and suffixes you can use. VyOS will always
+show you the different units you can use.
+
+Prefixes
+--------
+
+They can be **decimal** prefixes.
+
+   .. code-block:: none
+
+	kbit  (10^3)    kilobit per second
+	mbit  (10^6)    megabit per second
+	gbit  (10^9)    gigabit per second
+ 	tbit  (10^12)   terabit per second
+	
+	kbps  (8*10^3)  kilobyte per second
+	mbps  (8*10^6)  megabyte per second
+	gbps  (8*10^9)  gigabyte per second
+	tbps  (8*10^12) terabyte per second
+
+Or **binary** prefixes.
+
+   .. code-block:: none
+
+	kibit (2^10 = 1024)    kibibit per second
+	mibit (2^20 = 1024^2)  mebibit per second
+	gibit (2^30 = 1024^3)  gibibit per second
+        tbit  (2^40 = 1024^4)  tebibit per second
+
+	kibps (1024*8)	       kibibyte (KiB) per second
+	mibps (1024^2*8)       mebibyte (MiB) per second
+        gibps (1024^3*8)       gibibyte (GiB) per second
+        tibps (1024^4*8)       tebibyte (TiB) per second
+
+
+Suffixes
+--------
+
+A *bit* is written as **bit**,
+   
+   .. code-block:: none
+
+        kbit (kilobits per second)
+        mbit (megabits per second)
+        gbit (gigabits per second)
+	tbit (terabits per second)
+
+while a *byte* is written as a single **b**.
+
+   .. code-block:: none
+
+        kbps (kilobytes per second)
+        mbps (megabytes per second)
+        gbps (gigabytes per second)
+
+
+
+
+.. _classes:
+
+Classes
+=======
+
+In the :ref:`creating_a_traffic_policy` section you will see that
+some of the policies use *classes*. Those policies let you distribute
+traffic into different classes according to different parameters you can
+choose. So, a class is just a specific type of traffic you select.
+
+The ultimate goal of classifying traffic is to give each class a
+different treatment.
+
+
+Matching traffic
+----------------
+
+In order to define which traffic goes into which class, you define
+filters (that is, the matching criteria). Packets go through these matching rules
+(as in the rules of a firewall) and, if a packet matches the filter, it
+is assigned to that class.
+
+In VyOS, a class is identified by a number you can choose when
+configuring it.
+
+
+.. note:: The meaning of the Class ID is not the same for every type of
+   policy. Normally policies just need a meaningless number to identify
+   a class (Class ID), but that does not apply to every policy.
+   The the number of a class in a Priority Queue it does not only
+   identify it, it also defines its priority.
+
+
+.. code-block:: none
+
+  set traffic-policy <policy> <policy-name> class <class-ID> match <class-matching-rule-name>
+
+
+In the command above, we set the type of policy we are going to
+work with and the name we choose for it; a class (so that we can
+differentiate some traffic) and an identifiable number for that class;
+then we configure a matching rule (or filter) and a name for it.
+
+A class can have multiple match filters:
+
+.. code-block:: none
+
+  set traffic-policy shaper MY-SHAPER class 30 match HTTP
+  set traffic-policy shaper MY-SHAPER class 30 match HTTPs
+
+A match filter can contain multiple criteria and will match traffic if
+all those criteria are true.
+
+For example:
+
+.. code-block:: none
+
+  set traffic-policy shaper MY-SHAPER class 30 match HTTP ip protocol tcp
+  set traffic-policy shaper MY-SHAPER class 30 match HTTP ip source port 80
+
+This will match TCP traffic with source port 80.
+
+There are many parameters you will be able to use in order to match the
+traffic you want for a class:
+
+ - **Ethernet (protocol, destination address or source address)**
+ - **Interface name**
+ - **IPv4 (DSCP value, maximum packet length, protocol, source address,**
+   **destination address, source port, destination port or TCP flags)**
+ - **IPv6 (DSCP value, maximum payload length, protocol, source address,**
+   **destination address, source port, destination port or TCP flags)**
+ - **Firewall mark**
+ - **VLAN ID**
+
+When configuring your filter, you can use the ``Tab`` key to see the many
+different parameters you can configure.
+
+
+.. code-block:: none
+
+   vyos@vyos# set traffic-policy shaper MY-SHAPER class 30 match MY-FIRST-FILTER 
+   Possible completions:
+      description  Description for this match
+    > ether        Ethernet header match
+      interface    Interface name for this match
+    > ip           Match IP protocol header
+    > ipv6         Match IPV6 header
+      mark         Match on mark applied by firewall
+      vif          Virtual Local Area Network (VLAN) ID for this match
+ 
+ 
+
+As shown in the example above, one of the possibilities to match packets
+is based on marks done by the firewall, `that can give you a great deal of flexibility`_.
+
+You can also write a description for a filter:
+
+.. code-block:: none
+
+  set traffic-policy shaper MY-SHAPER class 30 match MY-FIRST-FILTER description "My filter description"
+
+
+
+.. note:: An IPv4 TCP filter will only match packets with an IPv4 header length of
+   20 bytes (which is the majority of IPv4 packets anyway).
+
+
+.. note:: IPv6 TCP filters will only match IPv6 packets with no header extension, see
+   https://en.wikipedia.org/wiki/IPv6_packet#Extension_headers
+
+
+Default
+-------
+
+Often you will also have to configure your *default* traffic in the same
+way you do with a class. *Default* can be considered a class as it
+behaves like that. It contains any traffic that did not match any
+of the defined classes, so it is like an open class, a class without
+matching filters.
+
+
+Class treatment
+---------------
+
+Once a class has a filter configured, you will also have to define what
+you want to do with the traffic of that class, what specific
+Traffic-Control treatment you want to give it. You will have different
+possibilities depending on the Traffic Policy you are configuring.
+
+.. code-block:: none
+
+   vyos@vyos# set traffic-policy shaper MY-SHAPER class 30 
+   Possible completions:
+      bandwidth    Bandwidth used for this class
+      burst        Burst size for this class (default: 15kb)
+      ceiling      Bandwidth limit for this class
+      codel-quantum
+                   fq-codel - Number of bytes used as 'deficit' (default 1514)
+      description  Description for this traffic class
+      flows        fq-codel - Number of flows (default 1024)
+      interval     fq-codel - Interval (milliseconds) used to measure the delay (default 100)
+   +> match        Class matching rule name
+      priority     Priority for usage of excess bandwidth
+      queue-limit  Maximum queue size (packets)
+      queue-type   Queue type for this class
+      set-dscp     Change the Differentiated Services (DiffServ) field in the IP header
+      target       fq-codel - Acceptable minimum queue delay (milliseconds)
+   
+
+For instance, with :code:`set traffic-policy shaper MY-SHAPER class 30 set-dscp EF`
+you would be modifying the DSCP field value of packets in that class to
+Expedite Forwarding.
+
+
+  DSCP values as per :rfc:`2474` and :rfc:`4595`:
+
+  +---------+------------+--------+------------------------------+
+  | Binary  | Configured |  Drop  | Description                  |
+  | value   | value      |  rate  |                              |
+  +=========+============+========+==============================+
+  | 101110  |     46     |   -    | Expedited forwarding (EF)    |
+  +---------+------------+--------+------------------------------+
+  | 000000  |     0      |   -    | Best effort traffic, default |
+  +---------+------------+--------+------------------------------+
+  | 001010  |     10     | Low    | Assured Forwarding(AF) 11    |
+  +---------+------------+--------+------------------------------+
+  | 001100  |     12     | Medium | Assured Forwarding(AF) 12    |
+  +---------+------------+--------+------------------------------+
+  | 001110  |     14     | High   | Assured Forwarding(AF) 13    |
+  +---------+------------+--------+------------------------------+
+  | 010010  |     18     | Low    | Assured Forwarding(AF) 21    |
+  +---------+------------+--------+------------------------------+
+  | 010100  |     20     | Medium | Assured Forwarding(AF) 22    |
+  +---------+------------+--------+------------------------------+
+  | 010110  |     22     | High   | Assured Forwarding(AF) 23    |
+  +---------+------------+--------+------------------------------+
+  | 011010  |     26     | Low    | Assured Forwarding(AF) 31    |
+  +---------+------------+--------+------------------------------+
+  | 011100  |     28     | Medium | Assured Forwarding(AF) 32    |
+  +---------+------------+--------+------------------------------+
+  | 011110  |     30     | High   | Assured Forwarding(AF) 33    |
+  +---------+------------+--------+------------------------------+
+  | 100010  |     34     | Low    | Assured Forwarding(AF) 41    |
+  +---------+------------+--------+------------------------------+
+  | 100100  |     36     | Medium | Assured Forwarding(AF) 42    |
+  +---------+------------+--------+------------------------------+
+  | 100110  |     38     | High   | Assured Forwarding(AF) 43    |
+  +---------+------------+--------+------------------------------+
+
+
+
+
+.. _embed:
+
+Embedding one policy into another one
+-------------------------------------
+
+Often we need to embed one policy into another one. It is possible to do
+so on classful policies, by attaching a new policy into a class. For
+instance, you might want to apply different policies to the different
+classes of a Round-Robin policy you have configured.
+
+A common example is the case of some policies which, in order to be
+effective, they need to be applied to an interface that is directly
+connected where the bottleneck is. If your router is not
+directly connected to the bottleneck, but some hop before it, you can
+emulate the bottleneck by embedding your non-shaping policy into a
+classful shaping one so that it takes effect.
+
+You can configure a policy into a class through the ``queue-type``
+setting.
+
+.. code-block:: none
+
+   set traffic-policy shaper FQ-SHAPER bandwidth 4gbit
+   set traffic-policy shaper FQ-SHAPER default bandwidth 100%
+   set traffic-policy shaper FQ-SHAPER default queue-type fq-codel
+
+As shown in the last command of the example above, the `queue-type`
+setting allows these combinations. You will be able to use it
+in many policies.
+
+.. note:: Some policies already include other embedded policies inside.
+   That is the case of Shaper_: each of its classes use fair-queue
+   unless you change it.
+
+.. _creating_a_traffic_policy:
+
+
+Creating a traffic policy
+=========================
+
+VyOS lets you control traffic in many different ways, here we will cover
+every possibility. You can configure as many policies as you want, but
+you will only be able to apply one policy per interface and direction
+(inbound or outbound).
+
+Some policies can be combined, you will be able to embed_ a different
+policy that will be applied to a class of the main policy. 
+
+.. hint:: **If you are looking for a policy for your outbound traffic**
+   but you don't know which one you need and you don't want to go
+   through every possible policy shown here, **our bet is that highly
+   likely you are looking for a** Shaper_ **policy and you want to**
+   :ref:`set its queues <embed>` **as FQ-CoDel**.
+
+Drop Tail
+---------
+
+| **Queueing discipline:** PFIFO (Packet First In First Out).
+| **Applies to:** Outbound traffic.
+
+This the simplest queue possible you can apply to your traffic. Traffic
+must go through a finite queue before it is actually sent. You must
+define how many packets that queue can contain.
+
+When a packet is to be sent, it will have to go through that queue, so
+the packet will be placed at the tail of it. When the packet completely
+goes through it, it will be dequeued emptying its place in the queue and
+being eventually handed to the NIC to be actually sent out.
+
+Despite the Drop-Tail policy does not slow down packets, if many packets
+are to be sent, they could get dropped when trying to get enqueued at
+the tail. This can happen if the queue has still not been able to
+release enough packets from its head.
+
+This is the policy that requieres the lowest resources for the same
+amount of traffic. But **very likely you do not need it as you cannot
+get much from it. Sometimes it is used just to enable logging.**
+
+.. cfgcmd:: set traffic-policy drop-tail <policy-name> queue-limit <number-of-packets>
+
+   Use this command to configure a drop-tail policy (PFIFO). Choose a
+   unique name for this policy and the size of the queue by setting the
+   number of packets it can contain (maximum 4294967295).
+
+
+Fair Queue
+----------
+
+| **Queueing discipline:** SFQ (Stochastic Fairness Queuing).
+| **Applies to:** Outbound traffic.
+
+Fair Queue is a work-conserving scheduler which schedules the
+transmission of packets based on flows, that is, it balances traffic
+distributing it through different sub-queues in order to ensure
+fairness so that each flow is able to send data in turn, preventing any
+single one from drowning out the rest.
+
+
+.. cfgcmd:: set traffic-policy fair-queue <policy-name>
+
+   Use this command to create a Fair-Queue policy and give it a name.
+   It is based on the Stochastic Fairness Queueing and can be applied to
+   outbound traffic.
+
+In order to separate traffic, Fair Queue uses a classifier based on
+source address, destination address and source port. The algorithm
+enqueues packets to hash buckets based on those tree parameters.
+Each of these buckets should represent a unique flow. Because multiple
+flows may get hashed to the same bucket, the hashing algorithm is
+perturbed at configurable intervals so that the unfairness lasts only
+for a short while. Perturbation may however cause some inadvertent
+packet reordering to occur. An advisable value could be 10 seconds.
+
+One of the uses of Fair Queue might be the mitigation of Denial of
+Service attacks.
+
+.. cfgcmd:: set traffic-policy fair-queue <policy-name> hash-interval <seconds>`
+
+   Use this command to define a Fair-Queue policy, based on the
+   Stochastic Fairness Queueing, and set the number of seconds at which
+   a new queue algorithm perturbation will occur (maximum 4294967295).
+
+When dequeuing, each hash-bucket with data is queried in a round robin
+fashion. You can configure the length of the queue.
+
+.. cfgcmd:: set traffic-policy fair-queue <policy-name> queue-limit <limit>
+
+   Use this command to define a Fair-Queue policy, based on the
+   Stochastic Fairness Queueing, and set the number of maximum packets
+   allowed to wait in the queue. Any other packet will be dropped.
+
+.. note:: Fair Queue is a non-shaping (work-conserving) policy, so it
+   will only be useful if your outgoing interface is really full. If it
+   is not, VyOS will not own the queue and Fair Queue will have no
+   effect. If there is bandwidth available on the physical link, you can
+   embed_ Fair-Queue into a classful shaping policy to make sure it owns
+   the queue.
+
+
+
+.. _FQ-CoDel:
+
+FQ-CoDel
+--------
+
+| **Queueing discipline** Fair/Flow Queue CoDel.
+| **Applies to:** Outbound Traffic.
+
+The FQ-CoDel policy distributes the traffic into 1024 FIFO queues and
+tries to provide good service between all of them. It also tries to keep
+the length of all the queues short.
+
+FQ-CoDel fights bufferbloat and reduces latency without the need of
+complex configurations. It has become the new default Queueing
+Discipline for the interfaces of some GNU/Linux distributions.
+
+It uses a stochastic model to classify incoming packets into
+different flows and is used to provide a fair share of the bandwidth to
+all the flows using the queue. Each flow is managed by the CoDel
+queuing  discipline. Reordering within a flow is avoided since Codel
+internally uses a FIFO queue.
+
+FQ-CoDel is based on a modified Deficit Round Robin (DRR_) queue
+scheduler with the CoDel Active Queue Management (AQM) algorithm
+operating on each queue.
+
+
+.. note:: FQ-Codel is a non-shaping (work-conserving) policy, so it
+   will only be useful if your outgoing interface is really full. If it
+   is not, VyOS will not own the queue and FQ-Codel will have no
+   effect. If there is bandwidth available on the physical link, you can
+   embed_ FQ-Codel into a classful shaping policy to make sure it owns
+   the queue. If you are not sure if you need to embed your FQ-CoDel
+   policy into a Shaper, do it.
+
+
+FQ-CoDel is tuned to run ok with its default parameters at 10Gbit
+speeds. It might work ok too at other speeds without configuring
+anything, but here we will explain some cases when you might want to
+tune its parameters.
+
+When running it at 1Gbit and lower, you may want to reduce the
+`queue-limit` to 1000 packets or less. In rates like 10Mbit, you may
+want to set it to 600 packets.
+
+If you are using FQ-CoDel embedded into Shaper_ and you have large rates
+(100Mbit and above), you may consider increasing `quantum` to 8000 or
+higher so that the scheduler saves CPU.
+
+On low rates (below 40Mbit) you may want to tune `quantum` down to
+something like 300 bytes.
+
+At very low rates (below 3Mbit), besides tuning `quantum` (300 keeps
+being ok) you may also want to increase `target` to something like 15ms
+and increase `interval` to something around 150 ms.
+
+
+.. cfgcmd:: set traffic-policy fq-codel <policy name> codel-quantum <bytes>
+
+   Use this command to configure an fq-codel policy, set its name and
+   the maximum number of bytes (default: 1514) to be dequeued from a
+   queue at once.
+
+.. cfgcmd:: set traffic-policy fq-codel <policy name> flows <number-of-flows>
+
+   Use this command to configure an fq-codel policy, set its name and
+   the number of sub-queues (default: 1024) into which packets are
+   classified.
+
+.. cfgcmd:: set traffic-policy fq-codel <policy name> interval <miliseconds>
+
+   Use this command to configure an fq-codel policy, set its name and
+   the time period used by the control loop of CoDel to detect when a
+   persistent queue is developing, ensuring that the measured minimum
+   delay does not become too stale (default: 100ms).
+
+.. cfgcmd:: set traffic-policy fq-codel <policy-name> queue-limit <number-of-packets>`
+
+   Use this command to configure an fq-codel policy, set its name, and
+   define a hard limit on the real queue size. When this limit is
+   reached, new packets are dropped (default: 10240 packets).
+
+.. cfgcmd:: set traffic-policy fq-codel <policy-name> target <miliseconds>`
+
+   Use this command to configure an fq-codel policy, set its name, and
+   define the acceptable minimum standing/persistent queue delay. This
+   minimum delay is identified by tracking the local minimum queue delay
+   that packets experience (default: 5ms).
+
+
+Example
+^^^^^^^
+
+A simple example of an FQ-CoDel policy working inside a Shaper one.
+
+
+.. code-block:: none
+
+   set traffic-policy shaper FQ-CODEL-SHAPER bandwidth 2gbit
+   set traffic-policy shaper FQ-CODEL-SHAPER default bandwidth 100%
+   set traffic-policy shaper FQ-CODEL-SHAPER default queue-type fq-codel
+
+
+
+Limiter
+-------
+
+| **Queueing discipline:** Ingress policer.
+| **Applies to:** Inbound traffic.
+
+Limiter is one of those policies that uses classes_ (Ingress qdisc is
+actually a classless policy but filters do work in it).
+
+The limiter performs basic ingress policing of traffic flows. Multiple
+classes of traffic can be defined and traffic limits can be applied to
+each class. Although the policer uses a token bucket mechanism
+internally, it does not have the capability to delay a packet as a
+shaping mechanism does. Traffic exceeding the defined bandwidth limits
+is directly dropped. A maximum allowed burst can be configured too.
+
+You can configure classes (up to 4090) with different settings and a
+default policy which will be applied to any traffic not matching any of
+the configured classes.
+
+
+.. note:: In the case you want to apply some kind of **shaping** to your
+  **inbound** traffic, check the ingress-shaping_ section.
+
+
+.. cfgcmd:: set traffic-policy limiter <policy-name> class <class ID> match <match-name> description <description>
+
+   Use this command to configure an Ingress Policer, defining its name,
+   a class identifier (1-4090), a class matching rule name and its
+   description.
+
+
+Once the matching rules are set for a class, you can start configuring
+how you want matching traffic to behave.
+
+
+.. cfgcmd:: set traffic-policy limiter <policy-name> class <class-ID> bandwidth <rate>
+
+   Use this command to configure an Ingress Policer, defining its name,
+   a class identifier (1-4090) and the maximum allowed bandwidth for
+   this class.
+
+
+.. cfgcmd:: set traffic-policy limiter <policy-name> class <class-ID> burst <burst-size>
+
+   Use this command to configure an Ingress Policer, defining its name,
+   a class identifier (1-4090) and the burst size in bytes for this
+   class (default: 15).
+
+
+.. cfgcmd:: set traffic-policy limiter <policy-name> default bandwidth <rate>
+
+   Use this command to configure an Ingress Policer, defining its name
+   and the maximum allowed bandwidth for its default policy.
+
+
+.. cfgcmd:: set traffic-policy limiter <policy-name> default burst <burst-size>
+
+   Use this command to configure an Ingress Policer, defining its name
+   and the burst size in bytes (default: 15) for its default policy.
+
+
+.. cfgcmd:: set traffic-policy limiter <policy-name> class <class ID> priority <value>
+
+   Use this command to configure an Ingress Policer, defining its name,
+   a class identifier (1-4090), and the priority (0-20, default 20) in
+   which the rule is evaluated (the lower the number, the higher the
+   priority).
+
+ 
+
+Network Emulator
+----------------
+
+| **Queueing discipline:** netem (Network Emulator) + TBF (Token Bucket Filter).
+| **Applies to:** Outbound traffic.
+
+VyOS Network Emulator policy emulates the conditions you can suffer in a
+real network. You will be able to configure things like rate, burst,
+delay, packet loss, packet corruption or packet reordering.
+
+This could be helpful if you want to test how an application behaves
+under certain network conditions.
+
+
+.. cfgcmd:: set traffic-policy network-emulator <policy-name> bandwidth <rate>
+   
+   Use this command to configure the maximum rate at which traffic will
+   be shaped in a Network Emulator policy. Define the name of the policy
+   and the rate.
+
+.. cfgcmd:: set traffic-policy network-emulator <policy-name> burst <burst-size>
+   
+   Use this command to configure the burst size of the traffic in a
+   Network Emulator policy. Define the name of the Network Emulator
+   policy and its traffic burst size (it will be configured through the
+   Token Bucket Filter qdisc). Default:15kb. It will only take effect if
+   you have configured its bandwidth too.
+
+.. cfgcmd:: set traffic-policy network-emulator <policy-name> network-delay <delay>
+   
+   Use this command to configure a Network Emulator policy defining its
+   name and the fixed amount of time you want to add to all packet going
+   out of the interface. The latency will be added through the
+   Token Bucket Filter qdisc. It will only take effect if you have
+   configured its bandwidth too. You can use secs, ms and us. Default:
+   50ms.
+
+.. cfgcmd:: set traffic-policy network-emulator <policy-name> packet-corruption <percent>
+   
+   Use this command to emulate noise in a Network Emulator policy. Set
+   the policy name and the percentage of corrupted packets you want. A
+   random error will be introduced in a random position for the chosen
+   percent of packets.
+
+.. cfgcmd:: set traffic-policy network-emulator <policy-name> packet-loss <percent>`
+   
+   Use this command to emulate packet-loss conditions in a Network
+   Emulator policy. Set the policy name and the percentage of loss
+   packets your traffic will suffer.
+
+.. cfgcmd:: set traffic-policy network-emulator <policy-name> packet-reordering <percent>`
+   
+   Use this command to emulate packet-reordering conditions in a Network
+   Emulator policy. Set the policy name and the percentage of reordered
+   packets your traffic will suffer.
+
+.. cfgcmd:: set traffic-policy network-emulator <policy-name> queue-limit <limit>
+   
+   Use this command to define the length of the queue of your Network
+   Emulator policy. Set the policy name and the maximum number of
+   packets (1-4294967295) the queue may hold queued at a time.
+
+
+
+Priority Queue
+--------------
+
+| **Queueing discipline:** PRIO.
+| **Applies to:** Outbound traffic.
+
+
+The Priority Queue is a classful scheduling policy. It does not delay
+packets (Priority Queue is not a shaping policy), it simply dequeues
+packets according to their priority.
+
+.. note:: Priority Queue, as other non-shaping policies, is only useful
+   if your outgoing interface is really full. If it is not, VyOS will
+   not own the queue and Priority Queue will have no effect. If there is
+   bandwidth available on the physical link, you can embed_ Priority
+   Queue into a classful shaping policy to make sure it owns the queue.
+   In that case packets can be prioritized based on DSCP.
+
+Up to seven queues -defined as classes_ with different priorities- can
+be configured. Packets are placed into queues based on associated match
+criteria. Packets are transmitted from the queues in priority order. If
+classes with a higher priority are being filled with packets
+continuously, packets from lower priority classes will only be
+transmitted after traffic volume from higher priority classes decreases.
+
+
+.. note:: In Priority Queue we do not define clases with a meaningless
+   class ID number but with a class priority number (1-7). The lower the
+   number, the higher the priority.
+
+
+As with other policies, you can define different type of matching rules
+for your classes:
+
+.. code-block:: none
+
+   vyos@vyos# set traffic-policy priority-queue MY-PRIO class 3 match MY-MATCH-RULE 
+   Possible completions:
+      description  Description for this match
+    > ether        Ethernet header match
+      interface    Interface name for this match
+    > ip           Match IP protocol header
+    > ipv6         Match IPV6 header
+      mark         Match on mark applied by firewall
+      vif          Virtual Local Area Network (VLAN) ID for this match
+
+
+As with other policies, you can embed_ other policies into the classes 
+(and default) of your Priority Queue policy through the ``queue-type``
+setting:
+
+.. code-block:: none
+
+   vyos@vyos# set traffic-policy priority-queue MY-PRIO class 3 queue-type 
+   Possible completions:
+      fq-codel     Fair Queue Codel
+      fair-queue   Stochastic Fair Queue (SFQ)
+      drop-tail    First-In-First-Out (FIFO)
+      priority     Priority queueing based on DSCP
+      random-detect
+                   Random Early Detection (RED)
+
+
+.. cfgcmd:: set traffic-policy priority-queue <policy-name> class <class-ID>  queue-limit <limit>`
+
+   Use this command to configure a Priority Queue policy, set its name,
+   set a class with a priority from 1 to 7 and define a hard limit on
+   the real queue size. When this limit is reached, new packets are
+   dropped.
+
+
+
+.. _Random-Detect:
+
+Random-Detect
+-------------
+
+
+| **Queueing discipline:** Generalized Random Early Drop.
+| **Applies to:** Outbound traffic.
+
+A simple Random Early Detection (RED) policy would start randomly
+dropping packets from a queue before it reaches its queue limit thus
+avoiding congestion. That is good for TCP connections as the gradual
+dropping of packets acts as a signal for the sender to decrease its
+transmission rate.
+
+In contrast to simple RED, VyOS' Random-Detect uses a Generalized Random
+Early Detect policy that provides different virtual queues based on the
+IP Precedence value so that some virtual queues can drop more packets
+than others. 
+
+This is achieved by using the first three bits of the ToS (Type of
+Service) field to categorize data streams and, in accordance with the
+defined precedence parameters, a decision is made.
+
+IP precedence as defined in :rfc:`791`:
+
+ +------------+----------------------+
+ | Precedence |      Priority        |
+ +============+======================+
+ |      7     | Network Control      |
+ +------------+----------------------+
+ |      6     | Internetwork Control |
+ +------------+----------------------+
+ |      5     | CRITIC/ECP           |
+ +------------+----------------------+
+ |      4     | Flash Override       |
+ +------------+----------------------+
+ |      3     | Flash                |
+ +------------+----------------------+
+ |      2     | Immediate            |
+ +------------+----------------------+
+ |      1     | Priority             |
+ +------------+----------------------+
+ |      0     | Routine              |
+ +------------+----------------------+
+
+
+Random-Detect could be useful for heavy traffic. One use of this
+algorithm might be to prevent a backbone overload. But only for TCP
+(because dropped packets could be retransmitted), not for UDP.
+
+
+.. cfgcmd:: set traffic-policy random-detect <policy-name> bandwidth <bandwidth>
+
+   Use this command to configure a Random-Detect policy, set its name
+   and set the available bandwidth for this policy. It is used for
+   calculating the average queue size after some idle time. It should be
+   set to the bandwidth of your interface. Random Detect is not a
+   shaping policy, this command will not shape.
+
+.. cfgcmd:: set traffic-policy random-detect <policy-name> precedence <IP-precedence-value> average-packet <bytes>
+   
+   Use this command to configure a Random-Detect policy and set its
+   name, then state the IP Precedence for the virtual queue you are
+   configuring and what the size of its average-packet should be
+   (in bytes, default: 1024).
+
+.. note:: When configuring a Random-Detect policy: **the higher the
+   precedence number, the higher the priority**.
+
+.. cfgcmd:: set traffic-policy random-detect <policy-name> precedence <IP-precedence-value> mark-probability <value>
+   
+   Use this command to configure a Random-Detect policy and set its
+   name, then state the IP Precedence for the virtual queue you are
+   configuring and what its mark (drop) probability will be. Set the
+   probability by giving the N value of the fraction 1/N (default: 10).
+
+
+.. cfgcmd:: set traffic-policy random-detect <policy-name> precedence <IP-precedence-value> maximum-threshold <packets>
+   
+   Use this command to configure a Random-Detect policy and set its
+   name, then state the IP Precedence for the virtual queue you are
+   configuring and what its maximum threshold for random detection will
+   be (from 0 to 4096 packets, default: 18). At this size, the marking
+   (drop) probability is maximal.
+
+.. cfgcmd:: set traffic-policy random-detect <policy-name> precedence <IP-precedence-value> minimum-threshold <packets>
+   
+   Use this command to configure a Random-Detect policy and set its
+   name, then state the IP Precedence for the virtual queue you are
+   configuring and what its minimum threshold for random detection will
+   be (from 0 to 4096 packets).  If this value is exceeded, packets
+   start being eligible for being dropped.
+
+
+The default values for the minimum-threshold depend on IP precedence:
+
+ +------------+-----------------------+
+ | Precedence | default min-threshold |
+ +============+=======================+
+ |      7     |         16            |
+ +------------+-----------------------+
+ |      6     |         15            |
+ +------------+-----------------------+
+ |      5     |         14            |
+ +------------+-----------------------+
+ |      4     |         13            |
+ +------------+-----------------------+
+ |      3     |         12            |
+ +------------+-----------------------+
+ |      2     |         11            |
+ +------------+-----------------------+
+ |      1     |         10            |
+ +------------+-----------------------+
+ |      0     |          9            |
+ +------------+-----------------------+
+
+
+.. cfgcmd:: set traffic-policy random-detect <policy-name> precedence <IP-precedence-value> queue-limit <packets>
+   
+   Use this command to configure a Random-Detect policy and set its
+   name, then name the IP Precedence for the virtual queue you are
+   configuring and what the maximum size of its queue will be (from 1 to
+   1-4294967295 packets). Packets are dropped when the current queue
+   length reaches this value.
+
+
+If the average queue size is lower than the **min-threshold**, an
+arriving packet will be placed in the queue.
+
+In the case the average queue size is between **min-threshold** and
+**max-threshold**, then an arriving packet would be either dropped or
+placed in the queue, it will depend on the defined **mark-probability**.
+
+If the current queue size is larger than **queue-limit**,
+then packets will be dropped. The average queue size depends on its
+former average size and its current one.
+
+If **max-threshold** is set but **min-threshold is not, then
+**min-threshold** is scaled to 50% of **max-threshold**.
+
+In principle, values must be
+:code:`min-threshold` < :code:`max-threshold` < :code:`queue-limit`.
+
+
+
+
+Rate Control
+------------
+
+| **Queueing discipline:** Tocken Bucket Filter.
+| **Applies to:** Outbound traffic.
+
+Rate-Control is a classless policy that limits the packet flow to a set
+rate. It is a pure shaper, it does not schedule traffic. Traffic is
+filtered based on the expenditure of tokens. Tokens roughly correspond
+to bytes.
+
+Short bursts can be allowed to exceed the limit. On creation, the
+Rate-Control traffic is stocked with tokens which correspond to the
+amount of traffic that can be burst in one go. Tokens arrive at a steady
+rate, until the bucket is full.
+
+.. cfgcmd:: set traffic-policy rate-control <policy-name> bandwidth <rate>
+
+   Use this command to configure a Rate-Control policy, set its name
+   and the rate limit you want to have.
+
+.. cfgcmd:: set traffic-policy rate-control <policy-name> burst <burst-size>
+
+   Use this command to configure a Rate-Control policy, set its name
+   and the size of the bucket in bytes which will be available for
+   burst.
+
+
+As a reference: for 10mbit/s on Intel, you might need at least 10kbyte
+buffer if you want to reach your configured rate.
+
+A very small buffer will soon start dropping packets.
+
+.. cfgcmd:: set traffic-policy rate-control <policy-name> latency 
+
+   Use this command to configure a Rate-Control policy, set its name
+   and the maximum amount of time a packet can be queued (default: 50
+   ms).
+
+
+Rate-Control is a CPU-friendly policy. You might consider using it when
+you just simply want to slow traffic down.
+
+.. _DRR:
+
+Round Robin
+-----------
+
+| **Queueing discipline:** Deficit Round Robin.
+| **Applies to:** Outbound traffic.
+
+The round-robin policy is a classful scheduler that divides traffic in
+different classes_ you can configure (up to 4096). You can embed_ a
+new policy into each of those classes (default included).
+ 
+Each class is assigned a deficit counter (the number of bytes that a
+flow is allowed to transmit when it is its turn) initialized to quantum.
+Quantum is a parameter you configure which acts like a credit of fix
+bytes the counter receives on each round. Then the Round-Robin policy
+starts moving its Round Robin pointer through the queues. If the deficit
+counter is greater than the packet's size at the head of the queue, this
+packet will be sent and the value of the counter will be decremented by
+the packet size. Then, the size of the next packet will be compared to
+the counter value again, repeating the process. Once the queue is empty
+or the value of the counter is insufficient, the Round-Robin pointer
+will move to the next queue. If the queue is empty, the value of the
+deficit counter is reset to 0. 
+
+At every round, the deficit counter adds the quantum so that even large
+packets will have their opportunity to be dequeued.
+
+
+.. cfgcmd:: set traffic-policy round-robin <policy name> class
+   <class-ID> quantum <packets>
+
+   Use this command to configure a Round-Robin policy, set its name, set
+   a class ID, and the quantum for that class. The deficit counter will
+   add that value each round.
+
+.. cfgcmd:: set traffic-policy round-robin <policy name> class
+   <class ID> queue-limit <packets>
+
+   Use this command to configure a Round-Robin policy, set its name, set
+   a class ID, and the queue size in packets.
+
+As with other policies, Round-Robin can embed_ another policy into a
+class through the ``queue-type`` setting.
+
+.. code-block:: none
+
+   vyos@vyos# set traffic-policy round-robin DRR class 10 queue-type 
+   Possible completions:
+      fq-codel     Fair Queue Codel
+      fair-queue   Stochastic Fair Queue (SFQ)
+      drop-tail    First-In-First-Out (FIFO)
+      priority     Priority queueing based on DSCP
+            
+
+
+
+.. _Shaper:
+
+Shaper
+------
+
+| **Queueing discipline:** Hierarchical Token Bucket.
+| **Applies to:** Outbound traffic.
+
+
+The Shaper policy does not guarantee a low delay, but it does guarantee
+bandwidth to different traffic classes and also lets you decide how to
+allocate more traffic once the guarantees are met.
+
+Each class can have a guaranteed part of the total bandwidth defined for
+the whole policy, so all those shares together should not be higher
+than the policy's whole bandwidth.
+
+If guaranteed traffic for a class is met and there is room for more
+traffic, the ceiling parameter can be used to set how much more
+bandwidth could be used. If guaranteed traffic is met and there are
+several classes willing to use their ceilings, the priority parameter
+will establish the order in which that additional traffic will be
+allocated. Priority can be any number from 0 to 7. The lower the number,
+the higher the priority.
+
+
+.. cfgcmd:: set traffic-policy shaper <policy-name> bandwidth <rate>
+
+   Use this command to configure a Shaper policy, set its name
+   and the maximum bandwidth for all combined traffic.
+
+
+.. cfgcmd:: set traffic-policy shaper <policy-name> class <class-ID> bandwidth <rate>
+
+   Use this command to configure a Shaper policy, set its name, define
+   a class and set the guaranteed traffic you want to allocate to that
+   class.
+
+.. cfgcmd:: set traffic-policy shaper <policy-name> class <class-ID> burst <bytes>
+
+   Use this command to configure a Shaper policy, set its name, define
+   a class and set the size of the `tocken bucket`_ in bytes, which will
+   be available to be sent at ceiling speed (default: 15Kb).
+
+.. cfgcmd:: set traffic-policy shaper <policy-name> class <class-ID> ceiling <bandwidth>
+
+   Use this command to configure a Shaper policy, set its name, define
+   a class and set the maximum speed possible for this class. The
+   default ceiling value is the bandwidth value.
+
+.. cfgcmd:: set traffic-policy shaper <policy-name> class <class-ID> priority <0-7>
+
+   Use this command to configure a Shaper policy, set its name, define
+   a class and set the priority for usage of available bandwidth once
+   guarantees have been met. The lower the priority number, the higher
+   the priority. The default priority value is 0, the highest priority.
+
+
+As with other policies, Shaper can embed_ other policies into its
+classes through the ``queue-type`` setting and then configure their
+parameters.
+
+
+.. code-block:: none
+
+   vyos@vyos# set traffic-policy shaper HTB class 10 queue-type 
+   Possible completions:
+      fq-codel     Fair Queue Codel
+      fair-queue   Stochastic Fair Queue (SFQ)
+      drop-tail    First-In-First-Out (FIFO)
+      priority     Priority queueing based on DSCP
+      random-detect
+                   Random Early Detection (RED)
+
+
+.. code-block:: none
+
+   vyos@vyos# set traffic-policy shaper HTB class 10 
+   Possible completions:
+      bandwidth    Bandwidth used for this class
+      burst        Burst size for this class (default: 15kb)
+      ceiling      Bandwidth limit for this class
+      codel-quantum
+                   fq-codel - Number of bytes used as 'deficit' (default 1514)
+      description  Description for this traffic class
+      flows        fq-codel - Number of flows (default 1024)
+      interval     fq-codel - Interval (milliseconds) used to measure the delay (default 100)
+   +> match        Class matching rule name
+      priority     Priority for usage of excess bandwidth
+      queue-limit  Maximum queue size (packets)
+      queue-type   Queue type for this class
+      set-dscp     Change the Differentiated Services (DiffServ) field in the IP header
+      target       fq-codel - Acceptable minimum queue delay (milliseconds)
+
+
+
+.. note:: If you configure a class for **VoIP traffic**, don't give it any
+   *ceiling*, otherwise new VoIP calls could start when the link is
+   available and get suddenly dropped when other classes start using
+   their assigned *bandwidth* share.
+
+
+Example
+^^^^^^^
+
+A simple example of Shaper using priorities.
+
+
+.. code-block:: none
+
+   set traffic-policy shaper MY-HTB bandwidth '50mbit'
+   set traffic-policy shaper MY-HTB class 10 bandwidth '20%'
+   set traffic-policy shaper MY-HTB class 10 match DSCP ip dscp 'EF'
+   set traffic-policy shaper MY-HTB class 10 queue-type 'fq-codel'
+   set traffic-policy shaper MY-HTB class 20 bandwidth '10%'
+   set traffic-policy shaper MY-HTB class 20 ceiling '50%'
+   set traffic-policy shaper MY-HTB class 20 match PORT666 ip destination port '666'
+   set traffic-policy shaper MY-HTB class 20 priority '3'
+   set traffic-policy shaper MY-HTB class 20 queue-type 'fair-queue'
+   set traffic-policy shaper MY-HTB class 30 bandwidth '10%'
+   set traffic-policy shaper MY-HTB class 30 ceiling '50%'
+   set traffic-policy shaper MY-HTB class 30 match ADDRESS30 ip source address '192.168.30.0/24'
+   set traffic-policy shaper MY-HTB class 30 priority '5'
+   set traffic-policy shaper MY-HTB class 30 queue-type 'fair-queue'
+   set traffic-policy shaper MY-HTB default bandwidth '10%'
+   set traffic-policy shaper MY-HTB default ceiling '100%'
+   set traffic-policy shaper MY-HTB default priority '7'
+   set traffic-policy shaper MY-HTB default queue-type 'fair-queue'
+
+
+Applying a traffic policy
+=========================
+
+Once a traffic-policy is created, you can apply it to an interface:
+
+.. code-block:: none
+
+  set interfaces etherhet eth0 traffic-policy out WAN-OUT
+
+You can only apply one policy per interface and direction, but you could
+reuse a policy on different interfaces and directions:
+
+.. code-block:: none
+
+  set interfaces ethernet eth0 traffic-policy in WAN-IN
+  set interfaces etherhet eth0 traffic-policy out WAN-OUT
+  set interfaces etherhet eth1 traffic-policy in LAN-IN
+  set interfaces etherhet eth1 traffic-policy out LAN-OUT
+  set interfaces ethernet eth2 traffic-policy in LAN-IN
+  set interfaces ethernet eth2 traffic-policy out LAN-OUT
+  set interfaces etherhet eth3 traffic-policy in TWO-WAY-POLICY
+  set interfaces etherhet eth3 traffic-policy out TWO-WAY-POLICY
+  set interfaces etherhet eth4 traffic-policy in TWO-WAY-POLICY
+  set interfaces etherhet eth4 traffic-policy out TWO-WAY-POLICY
+
+Getting queueing information
+----------------------------
+
+.. opcmd:: show queueing <interface-type> <interface-name>
+
+   Use this command to see the queueing information for an interface.
+   You will be able to see a packet counter (Sent, Dropped, Overlimit
+   and Backlog) per policy and class configured.
+
+
+
+.. _ingress-shaping:
+
+The case of ingress shaping
+===========================
+
+| **Applies to:** Inbound traffic.
+
+For the ingress traffic of an interface, there is only one policy you
+can directly apply, a **Limiter** policy. You cannot apply a shaping
+policy directly to the ingress traffic of any interface because shaping
+only works for outbound traffic.
+
+This workaround lets you apply a shaping policy to the ingress traffic
+by first redirecting it to an in-between virtual interface
+(`Intermediate Functional Block`_). There, in that virtual interface,
+you will be able to apply any of the policies that work for outbound
+traffic, for instance, a shaping one.
+
+That is how it is possible to do the so-called "ingress shaping".
+
+
+.. code-block:: none
+
+   set traffic-policy shaper MY-INGRESS-SHAPING bandwidth 1000kbit
+   set traffic-policy shaper MY-INGRESS-SHAPING default bandwidth 1000kbit
+   set traffic-policy shaper MY-INGRESS-SHAPING default queue-type fair-queue
+   
+   set interfaces input ifb0 traffic-policy out MY-INGRESS-SHAPING
+   set interfaces ethernet eth0 redirect ifb0
+
+.. warning::
+
+  Do not configure IFB as the first step. First create everything else
+  of your traffic-policy, and then you can configure IFB.
+  Otherwise you might get the ``RTNETLINK answer: File exists`` error,
+  which can be solved with ``sudo ip link delete ifb0``.
+
+
+.. _that can give you a great deal of flexibility: https://blog.vyos.io/using-the-policy-route-and-packet-marking-for-custom-qos-matches
+.. _tc: https://en.wikipedia.org/wiki/Tc_(Linux)
+.. _tocken bucket: https://en.wikipedia.org/wiki/Token_bucket
+.. _HFSC: https://en.wikipedia.org/wiki/Hierarchical_fair-service_curve
+.. _Intermediate Functional Block: https://www.linuxfoundation.org/collaborate/workgroups/networking/ifb