summaryrefslogtreecommitdiff
path: root/docs/configuration/interfaces/bonding.rst
diff options
context:
space:
mode:
authorrebortg <github@ghlr.de>2020-11-29 21:52:28 +0100
committerrebortg <github@ghlr.de>2020-11-29 21:52:28 +0100
commit4abded8025a47990e58cd0a5fe9b96e38f4a3715 (patch)
tree2a6ebaa9dcf96a10e95577ae446c3897b0938fd4 /docs/configuration/interfaces/bonding.rst
parente7f01e6efc8578603592ff86c031d46f1f1f9d82 (diff)
downloadvyos-documentation-4abded8025a47990e58cd0a5fe9b96e38f4a3715.tar.gz
vyos-documentation-4abded8025a47990e58cd0a5fe9b96e38f4a3715.zip
arrange: interfaces
Diffstat (limited to 'docs/configuration/interfaces/bonding.rst')
-rw-r--r--docs/configuration/interfaces/bonding.rst560
1 files changed, 560 insertions, 0 deletions
diff --git a/docs/configuration/interfaces/bonding.rst b/docs/configuration/interfaces/bonding.rst
new file mode 100644
index 00000000..7faddd6f
--- /dev/null
+++ b/docs/configuration/interfaces/bonding.rst
@@ -0,0 +1,560 @@
+.. _bond-interface:
+
+#######################
+Bond / Link Aggregation
+#######################
+
+The bonding interface provides a method for aggregating multiple network
+interfaces into a single logical "bonded" interface, or LAG, or ether-channel,
+or port-channel. The behavior of the bonded interfaces depends upon the mode;
+generally speaking, modes provide either hot standby or load balancing services.
+Additionally, link integrity monitoring may be performed.
+
+*************
+Configuration
+*************
+
+Common interface configuration
+==============================
+
+.. cmdinclude:: ../_include/interface-common-with-dhcp.txt
+ :var0: bond
+ :var1: bond0
+
+Member Interfaces
+=================
+
+.. cfgcmd:: set interfaces bonding <interface> member interface <member>
+
+ Enslave `<member>` interface to bond `<interface>`.
+
+Bond options
+============
+
+.. cfgcmd:: set interfaces bonding <interface> mode <802.3ad | active-backup |
+ broadcast | round-robin | transmit-load-balance | adaptive-load-balance |
+ xor-hash>
+
+ Specifies one of the bonding policies. The default is 802.3ad. Possible
+ values are:
+
+ * ``802.3ad`` - IEEE 802.3ad Dynamic link aggregation. Creates aggregation
+ groups that share the same speed and duplex settings. Utilizes all slaves
+ in the active aggregator according to the 802.3ad specification.
+
+ Slave selection for outgoing traffic is done according to the transmit
+ hash policy, which may be changed from the default simple XOR policy via
+ the :cfgcmd:`hash-policy` option, documented below.
+
+ .. note:: Not all transmit policies may be 802.3ad compliant, particularly
+ in regards to the packet mis-ordering requirements of section 43.2.4
+ of the 802.3ad standard.
+
+ * ``active-backup`` - Active-backup policy: Only one slave in the bond is
+ active. A different slave becomes active if, and only if, the active slave
+ fails. The bond's MAC address is externally visible on only one port
+ (network adapter) to avoid confusing the switch.
+
+ When a failover occurs in active-backup mode, bonding will issue one or
+ more gratuitous ARPs on the newly active slave. One gratuitous ARP is
+ issued for the bonding master interface and each VLAN interfaces
+ configured above it, provided that the interface has at least one IP
+ address configured. Gratuitous ARPs issued for VLAN interfaces are tagged
+ with the appropriate VLAN id.
+
+ This mode provides fault tolerance. The :cfgcmd:`primary` option,
+ documented below, affects the behavior of this mode.
+
+ * ``broadcast`` - Broadcast policy: transmits everything on all slave
+ interfaces.
+
+ This mode provides fault tolerance.
+
+ * ``round-robin`` - Round-robin policy: Transmit packets in sequential
+ order from the first available slave through the last.
+
+ This mode provides load balancing and fault tolerance.
+
+ * ``transmit-load-balance`` - Adaptive transmit load balancing: channel
+ bonding that does not require any special switch support.
+
+ Incoming traffic is received by the current slave. If the receiving slave
+ fails, another slave takes over the MAC address of the failed receiving
+ slave.
+
+ * ``adaptive-load-balance`` - Adaptive load balancing: includes
+ transmit-load-balance plus receive load balancing for IPV4 traffic, and
+ does not require any special switch support. The receive load balancing
+ is achieved by ARP negotiation. The bonding driver intercepts the ARP
+ Replies sent by the local system on their way out and overwrites the
+ source hardware address with the unique hardware address of one of the
+ slaves in the bond such that different peers use different hardware
+ addresses for the server.
+
+ Receive traffic from connections created by the server is also balanced.
+ When the local system sends an ARP Request the bonding driver copies and
+ saves the peer's IP information from the ARP packet. When the ARP Reply
+ arrives from the peer, its hardware address is retrieved and the bonding
+ driver initiates an ARP reply to this peer assigning it to one of the
+ slaves in the bond. A problematic outcome of using ARP negotiation for
+ balancing is that each time that an ARP request is broadcast it uses the
+ hardware address of the bond. Hence, peers learn the hardware address
+ of the bond and the balancing of receive traffic collapses to the current
+ slave. This is handled by sending updates (ARP Replies) to all the peers
+ with their individually assigned hardware address such that the traffic
+ is redistributed. Receive traffic is also redistributed when a new slave
+ is added to the bond and when an inactive slave is re-activated. The
+ receive load is distributed sequentially (round robin) among the group
+ of highest speed slaves in the bond.
+
+ When a link is reconnected or a new slave joins the bond the receive
+ traffic is redistributed among all active slaves in the bond by initiating
+ ARP Replies with the selected MAC address to each of the clients. The
+ updelay parameter (detailed below) must be set to a value equal or greater
+ than the switch's forwarding delay so that the ARP Replies sent to the
+ peers will not be blocked by the switch.
+
+ * ``xor-hash`` - XOR policy: Transmit based on the selected transmit
+ hash policy. The default policy is a simple [(source MAC address XOR'd
+ with destination MAC address XOR packet type ID) modulo slave count].
+ Alternate transmit policies may be selected via the :cfgcmd:`hash-policy`
+ option, described below.
+
+ This mode provides load balancing and fault tolerance.
+
+.. cfgcmd:: set interfaces bonding <interface> min-links <0-16>
+
+ Specifies the minimum number of links that must be active before asserting
+ carrier. It is similar to the Cisco EtherChannel min-links feature. This
+ allows setting the minimum number of member ports that must be up (link-up
+ state) before marking the bond device as up (carrier on). This is useful for
+ situations where higher level services such as clustering want to ensure a
+ minimum number of low bandwidth links are active before switchover.
+
+ This option only affects 802.3ad mode.
+
+ The default value is 0. This will cause carrier to be asserted (for 802.3ad
+ mode) whenever there is an active aggregator, regardless of the number of
+ available links in that aggregator.
+
+ .. note:: Because an aggregator cannot be active without at least one
+ available link, setting this option to 0 or to 1 has the exact same
+ effect.
+
+.. cfgcmd:: set interfaces bonding <interface> hash-policy <policy>
+
+ * **layer2** - Uses XOR of hardware MAC addresses and packet type ID field
+ to generate the hash. The formula is
+
+ .. code-block:: none
+
+ hash = source MAC XOR destination MAC XOR packet type ID
+ slave number = hash modulo slave count
+
+ This algorithm will place all traffic to a particular network peer on
+ the same slave.
+
+ This algorithm is 802.3ad compliant.
+
+ * **layer2+3** - This policy uses a combination of layer2 and layer3
+ protocol information to generate the hash. Uses XOR of hardware MAC
+ addresses and IP addresses to generate the hash. The formula is:
+
+ .. code-block:: none
+
+ hash = source MAC XOR destination MAC XOR packet type ID
+ hash = hash XOR source IP XOR destination IP
+ hash = hash XOR (hash RSHIFT 16)
+ hash = hash XOR (hash RSHIFT 8)
+
+ And then hash is reduced modulo slave count.
+
+ If the protocol is IPv6 then the source and destination addresses are
+ first hashed using ipv6_addr_hash.
+
+ This algorithm will place all traffic to a particular network peer on the
+ same slave. For non-IP traffic, the formula is the same as for the layer2
+ transmit hash policy.
+
+ This policy is intended to provide a more balanced distribution of traffic
+ than layer2 alone, especially in environments where a layer3 gateway
+ device is required to reach most destinations.
+
+ This algorithm is 802.3ad compliant.
+
+ * **layer3+4** - This policy uses upper layer protocol information, when
+ available, to generate the hash. This allows for traffic to a particular
+ network peer to span multiple slaves, although a single connection will
+ not span multiple slaves.
+
+ The formula for unfragmented TCP and UDP packets is
+
+ .. code-block:: none
+
+ hash = source port, destination port (as in the header)
+ hash = hash XOR source IP XOR destination IP
+ hash = hash XOR (hash RSHIFT 16)
+ hash = hash XOR (hash RSHIFT 8)
+
+ And then hash is reduced modulo slave count.
+
+ If the protocol is IPv6 then the source and destination addresses are
+ first hashed using ipv6_addr_hash.
+
+ For fragmented TCP or UDP packets and all other IPv4 and IPv6 protocol
+ traffic, the source and destination port information is omitted. For
+ non-IP traffic, the formula is the same as for the layer2 transmit hash
+ policy.
+
+ This algorithm is not fully 802.3ad compliant. A single TCP or UDP
+ conversation containing both fragmented and unfragmented packets will see
+ packets striped across two interfaces. This may result in out of order
+ delivery. Most traffic types will not meet this criteria, as TCP rarely
+ fragments traffic, and most UDP traffic is not involved in extended
+ conversations. Other implementations of 802.3ad may or may not tolerate
+ this noncompliance.
+
+.. cfgcmd:: set interfaces bonding <interface> primary <interface>
+
+ An `<interface>` specifying which slave is the primary device. The specified
+ device will always be the active slave while it is available. Only when the
+ primary is off-line will alternate devices be used. This is useful when one
+ slave is preferred over another, e.g., when one slave has higher throughput
+ than another.
+
+ The primary option is only valid for active-backup, transmit-load-balance,
+ and adaptive-load-balance mode.
+
+.. cfgcmd:: set interfaces bonding <interface> arp-monitor interval <time>
+
+ Specifies the ARP link monitoring `<time>` in seconds.
+
+ The ARP monitor works by periodically checking the slave devices to determine
+ whether they have sent or received traffic recently (the precise criteria
+ depends upon the bonding mode, and the state of the slave). Regular traffic
+ is generated via ARP probes issued for the addresses specified by the
+ :cfgcmd:`arp-monitor target` option.
+
+ If ARP monitoring is used in an etherchannel compatible mode (modes
+ round-robin and xor-hash), the switch should be configured in a mode that
+ evenly distributes packets across all links. If the switch is configured to
+ distribute the packets in an XOR fashion, all replies from the ARP targets
+ will be received on the same link which could cause the other team members
+ to fail.
+
+ A value of 0 disables ARP monitoring. The default value is 0.
+
+.. cfgcmd:: set interfaces bonding <interface> arp-monitor target <address>
+
+ Specifies the IP addresses to use as ARP monitoring peers when
+ :cfgcmd:`arp-monitor interval` option is > 0. These are the targets of the
+ ARP request sent to determine the health of the link to the targets.
+
+ Multiple target IP addresses can be specified. At least one IP address must
+ be given for ARP monitoring to function.
+
+ The maximum number of targets that can be specified is 16. The default value
+ is no IP addresses.
+
+VLAN
+====
+
+.. cmdinclude:: ../_include/interface-vlan-8021q.txt
+ :var0: bond
+ :var1: bond0
+
+*******
+Example
+*******
+
+The following configuration on VyOS applies to all following 3rd party vendors.
+It creates a bond with two links and VLAN 10, 100 on the bonded interfaces with
+a per VIF IPv4 address.
+
+.. code-block:: none
+
+ # Create bonding interface bond0 with 802.3ad LACP
+ set interfaces bonding bond0 hash-policy 'layer2'
+ set interfaces bonding bond0 mode '802.3ad'
+
+ # Add the required vlans and IPv4 addresses on them
+ set interfaces bonding bond0 vif 10 address 192.168.0.1/24
+ set interfaces bonding bond0 vif 100 address 10.10.10.1/24
+
+ # Add the member interfaces to the bonding interface
+ set interfaces bonding bond0 member interface eth1
+ set interfaces bonding bond0 member interface eth2
+
+Cisco Catalyst
+==============
+
+Assign member interfaces to PortChannel
+
+.. code-block:: none
+
+ interface GigabitEthernet1/0/23
+ description VyOS eth1
+ channel-group 1 mode active
+ !
+ interface GigabitEthernet1/0/24
+ description VyOS eth2
+ channel-group 1 mode active
+ !
+
+A new interface becomes present ``Port-channel1``, all configuration like
+allowed VLAN interfaces, STP will happen here.
+
+.. code-block:: none
+
+ interface Port-channel1
+ description LACP Channel for VyOS
+ switchport trunk encapsulation dot1q
+ switchport trunk allowed vlan 10,100
+ switchport mode trunk
+ spanning-tree portfast trunk
+ !
+
+
+Juniper EX Switch
+=================
+
+For a headstart you can use the below example on how to build a bond with two
+interfaces from VyOS to a Juniper EX Switch system.
+
+.. code-block:: none
+
+ # Create aggregated ethernet device with 802.3ad LACP and port speeds of 10gbit/s
+ set interfaces ae0 aggregated-ether-options link-speed 10g
+ set interfaces ae0 aggregated-ether-options lacp active
+
+ # Create layer 2 on the aggregated ethernet device with trunking for our vlans
+ set interfaces ae0 unit 0 family ethernet-switching port-mode trunk
+
+ # Add the required vlans to the device
+ set interfaces ae0 unit 0 family ethernet-switching vlan members 10
+ set interfaces ae0 unit 0 family ethernet-switching vlan members 100
+
+ # Add the two interfaces to the aggregated ethernet device, in this setup both
+ # ports are on the same switch (switch 0, module 1, port 0 and 1)
+ set interfaces xe-0/1/0 ether-options 802.3ad ae0
+ set interfaces xe-0/1/1 ether-options 802.3ad ae0
+
+ # But this can also be done with multiple switches in a stack, a virtual
+ # chassis on Juniper (switch 0 and switch 1, module 1, port 0 on both switches)
+ set interfaces xe-0/1/0 ether-options 802.3ad ae0
+ set interfaces xe-1/1/0 ether-options 802.3ad ae0
+
+Aruba/HP
+========
+
+For a headstart you can use the below example on how to build a bond,port-channel
+with two interfaces from VyOS to a Aruba/HP 2510G switch.
+
+.. code-block:: none
+
+ # Create trunk with 2 member interfaces (interface 1 and 2) and LACP
+ trunk 1-2 Trk1 LACP
+
+ # Add the required vlans to the trunk
+ vlan 10 tagged Trk1
+ vlan 100 tagged Trk1
+
+Arista EOS
+==========
+
+When utilizing VyOS in an environment with Arista gear you can use this blue
+print as an initial setup to get an LACP bond / port-channel operational between
+those two devices.
+
+Lets assume the following topology:
+
+.. figure:: /_static/images/vyos_arista_bond_lacp.png
+ :alt: VyOS Arista EOS setup
+
+**R1**
+
+ .. code-block:: none
+
+ interfaces {
+ bonding bond10 {
+ hash-policy layer3+4
+ member {
+ interface eth1
+ interface eth2
+ }
+ mode 802.3ad
+ vif 100 {
+ address 192.0.2.1/30
+ address 2001:db8::1/64
+ }
+ }
+
+**R2**
+
+ .. code-block:: none
+
+ interfaces {
+ bonding bond10 {
+ hash-policy layer3+4
+ member {
+ interface eth1
+ interface eth2
+ }
+ mode 802.3ad
+ vif 100 {
+ address 192.0.2.2/30
+ address 2001:db8::2/64
+ }
+ }
+
+**SW1**
+
+ .. code-block:: none
+
+ !
+ vlan 100
+ name FOO
+ !
+ interface Port-Channel10
+ switchport trunk allowed vlan 100
+ switchport mode trunk
+ spanning-tree portfast
+ !
+ interface Port-Channel20
+ switchport mode trunk
+ no spanning-tree portfast auto
+ spanning-tree portfast network
+ !
+ interface Ethernet1
+ channel-group 10 mode active
+ !
+ interface Ethernet2
+ channel-group 10 mode active
+ !
+ interface Ethernet3
+ channel-group 20 mode active
+ !
+ interface Ethernet4
+ channel-group 20 mode active
+ !
+
+**SW2**
+
+ .. code-block:: none
+
+ !
+ vlan 100
+ name FOO
+ !
+ interface Port-Channel10
+ switchport trunk allowed vlan 100
+ switchport mode trunk
+ spanning-tree portfast
+ !
+ interface Port-Channel20
+ switchport mode trunk
+ no spanning-tree portfast auto
+ spanning-tree portfast network
+ !
+ interface Ethernet1
+ channel-group 10 mode active
+ !
+ interface Ethernet2
+ channel-group 10 mode active
+ !
+ interface Ethernet3
+ channel-group 20 mode active
+ !
+ interface Ethernet4
+ channel-group 20 mode active
+ !
+
+.. note:: When using EVE-NG to lab this environment ensure you are using e1000
+ as the desired driver for your VyOS network interfaces. When using the regular
+ virtio network driver no LACP PDUs will be sent by VyOS thus the port-channel
+ will never become active!
+
+*********
+Operation
+*********
+
+.. opcmd:: show interfaces bonding
+
+ Show brief interface information.
+
+ .. code-block:: none
+
+ vyos@vyos:~$ show interfaces bonding
+ Codes: S - State, L - Link, u - Up, D - Down, A - Admin Down
+ Interface IP Address S/L Description
+ --------- ---------- --- -----------
+ bond0 - u/u my-sw1 int 23 and 24
+ bond0.10 192.168.0.1/24 u/u office-net
+ bond0.100 10.10.10.1/24 u/u management-net
+
+
+.. opcmd:: show interfaces bonding <interface>
+
+ Show detailed information on given `<interface>`
+
+ .. code-block:: none
+
+ vyos@vyos:~$ show interfaces bonding bond5
+ bond5: <NO-CARRIER,BROADCAST,MULTICAST,MASTER,UP> mtu 1500 qdisc noqueue state DOWN group default qlen 1000
+ link/ether 00:50:56:bf:ef:aa brd ff:ff:ff:ff:ff:ff
+ inet6 fe80::e862:26ff:fe72:2dac/64 scope link tentative
+ valid_lft forever preferred_lft forever
+
+ RX: bytes packets errors dropped overrun mcast
+ 0 0 0 0 0 0
+ TX: bytes packets errors dropped carrier collisions
+ 0 0 0 0 0 0
+
+.. opcmd:: show interfaces bonding <interface> detail
+
+ Show detailed information about the underlaying physical links on given
+ bond `<interface>`.
+
+ .. code-block:: none
+
+ vyos@vyos:~$ show interfaces bonding bond5 detail
+ Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011)
+
+ Bonding Mode: IEEE 802.3ad Dynamic link aggregation
+ Transmit Hash Policy: layer2 (0)
+ MII Status: down
+ MII Polling Interval (ms): 100
+ Up Delay (ms): 0
+ Down Delay (ms): 0
+
+ 802.3ad info
+ LACP rate: slow
+ Min links: 0
+ Aggregator selection policy (ad_select): stable
+
+ Slave Interface: eth1
+ MII Status: down
+ Speed: Unknown
+ Duplex: Unknown
+ Link Failure Count: 0
+ Permanent HW addr: 00:50:56:bf:ef:aa
+ Slave queue ID: 0
+ Aggregator ID: 1
+ Actor Churn State: churned
+ Partner Churn State: churned
+ Actor Churned Count: 1
+ Partner Churned Count: 1
+
+ Slave Interface: eth2
+ MII Status: down
+ Speed: Unknown
+ Duplex: Unknown
+ Link Failure Count: 0
+ Permanent HW addr: 00:50:56:bf:19:26
+ Slave queue ID: 0
+ Aggregator ID: 2
+ Actor Churn State: churned
+ Partner Churn State: churned
+ Actor Churned Count: 1
+ Partner Churned Count: 1
+