appendix: ha: use 80 character line break

1 files changed, 157 insertions, 81 deletions

diff --git a/docs/appendix/examples/ha.rst b/docs/appendix/examples/ha.rst
index ed11de2d..702cb2b2 100644
--- a/docs/appendix/examples/ha.rst
+++ b/docs/appendix/examples/ha.rst
@@ -2,40 +2,60 @@
 High Availability Walkthrough
 #############################
 
-This document walks you through a complete HA setup of two VyOS machines. This design is based on a VM as the primary router, and a physical machine as a backup, using VRRP, BGP, OSPF and conntrack sharing.
+This document walks you through a complete HA setup of two VyOS machines. This
+design is based on a VM as the primary router, and a physical machine as a
+backup, using VRRP, BGP, OSPF and conntrack sharing.
 
-The aim of this document is to walk you through setting everything up so you end up at a point where you can reboot any machine and not lose more than a few seconds worth of connectivity.
+The aim of this document is to walk you through setting everything up so you
+and up at a point where you can reboot any machine and not lose more than a few
+seconds worth of connectivity.
 
 Design
 ======
 
-This is based on a real life, in production design. One of the complex issues is ensuring you have redundant data INTO your network.
-We do this with a pair of Cisco Nexus switches, and using Virtual PortChannels that are spanned across them.
-This as an added bonus, also allows for complete switch failure without an outage. How you achieve this yourself is left as an exercise to the reader but our setup is documented here.
+This is based on a real life, in production design. One of the complex issues
+is ensuring you have redundant data INTO your network. We do this with a pair
+of Cisco Nexus switches, and using Virtual PortChannels that are spanned across
+them. This as an added bonus, also allows for complete switch failure without
+an outage. How you achieve this yourself is left as an exercise to the reader
+but our setup is documented here.
 
 Walkthrough suggestion
 ----------------------
 
-The ``commit`` command is implied after every section. If you make an error, ``commit`` will warn you and you can fix it before getting too far into things. Please ensure you commit early and commit often.
+The ``commit`` command is implied after every section. If you make an error,
+``commit`` will warn you and you can fix it before getting too far into things.
+Please ensure you commit early and commit often.
 
-If you are following through this document, it is strongly suggested you complete the entire document, ONLY doing the virtual router1 steps, and then come back and walk through it AGAIN on the backup hardware router.
+If you are following through this document, it is strongly suggested you
+complete the entire document, ONLY doing the virtual router1 steps, and then
+come back and walk through it AGAIN on the backup hardware router.
 
-This ensures you don't go to fast, or miss a step. However, it will make your life easier to configure the fixed IP address and default route now on the hardware router.
+This ensures you don't go to fast, or miss a step. However, it will make your
+life easier to configure the fixed IP address and default route now on the
+hardware router.
 
 Example Network
 ---------------
 
-In this document, we have been allocated 203.0.113.0/24 by our upstream provider, which we are publishing on VLAN100.
+In this document, we have been allocated 203.0.113.0/24 by our upstream
+provider, which we are publishing on VLAN100.
 
-They want us to establish a BGP session to their routers on 192.0.2.11 and 192.0.2.12 from our routers 192.0.2.21 and 192.0.2.22. They are AS 65550 and we are AS65551.
+They want us to establish a BGP session to their routers on 192.0.2.11 and
+192.0.2.12 from our routers 192.0.2.21 and 192.0.2.22. They are AS 65550 and
+we are AS65551.
 
-Our routers are going to have a floating IP address of 203.0.113.1, and use .2 and .3 as their fixed IPs.
+Our routers are going to have a floating IP address of 203.0.113.1, and use
+.2 and .3 as their fixed IPs.
 
 We are going to use 10.200.201.0/24 for an 'internal' network on VLAN201.
 
-When traffic is originated from the 10.200.201.0/24 network, it will be masqueraded to 203.0.113.1
+When traffic is originated from the 10.200.201.0/24 network, it will be
+masqueraded to 203.0.113.1
 
-For connection between sites, we are running a WireGuard link to two REMOTE routers, and using OSPF over those links to distribute routes. That remote site is expected to send traffic from anything in 10.201.0.0/16
+For connection between sites, we are running a WireGuard link to two REMOTE
+routers, and using OSPF over those links to distribute routes. That remote
+site is expected to send traffic from anything in 10.201.0.0/16
 
 VLANs
 -----
@@ -61,35 +81,48 @@ Note that router1 is a VM that runs on one of the compute nodes.
 Network Cabling
 ---------------
 
-* From Datacenter - This connects into port 1 on both switches, and is tagged as VLAN 50
+* From Datacenter - This connects into port 1 on both switches, and is tagged
+  as VLAN 50
 * Cisco VPC Crossconnect - Ports 39 and 40 bonded between each switch
 * Hardware Router - Port 8 of each switch
 * compute1 - Port 9 of each switch
 * compute2 - Port 10 of each switch
 * compute3 - Port 11 of each switch
 
-This is ignoring the extra Out-of-band management networking, which should be on totally different switches, and a different feed into the rack, and is out of scope of this.
+This is ignoring the extra Out-of-band management networking, which should be
+on totally different switches, and a different feed into the rack, and is out
+of scope of this.
 
-Note about VMware
------------------
+.. note:: Our implementation uses VMware's Distributed Port Groups, which allows
+  VMware to use LACP. This is a part of the ENTERPRISE licence, and is not
+  available on a Free licence. If you are implementing this and do not have
+  access to DPGs, you should not use VMware, and use some other virtualization
+  platform instead.
 
-Our implementation uses VMware's Distributed Port Groups, which allows VMware to use LACP. This is a part of the ENTERPRISE licence, and is not available on a Free licence. If you are implementing this and do not have access to DPGs, you should not use VMware, and use some other virtualization platform instead.
 
 Basic Setup (via console)
 =========================
 
-Create your router1 VM so it is able to withstand a VM Host failing, or a network link failing. Using VMware, this is achieved by enabling vSphere DRS, vSphere Availability, and creating a Distributed Port Group that uses LACP.
+Create your router1 VM so it is able to withstand a VM Host failing, or a
+network link failing. Using VMware, this is achieved by enabling vSphere DRS,
+vSphere Availability, and creating a Distributed Port Group that uses LACP.
 
-Many other Hypervisors do this, and I'm hoping that this document will be expanded to document how to do this for others.
+Many other Hypervisors do this, and I'm hoping that this document will be
+expanded to document how to do this for others.
 
-Create an 'All VLANs' network group, that passes all trunked traffic through to the VM. Attach this network group to router1 as eth0.
+Create an 'All VLANs' network group, that passes all trunked traffic through
+to the VM. Attach this network group to router1 as eth0.
 
-**VMware Note:** You must DISABLE SECURITY on this Port group. Make sure that ``Promiscuous Mode``\ , ``MAC address changes`` and ``Forged transmits`` are enabled. All of these will be done as part of failover.
+.. note:: VMware: You must DISABLE SECURITY on this Port group. Make sure that
+   ``Promiscuous Mode``\ , ``MAC address changes`` and ``Forged transmits`` are
+   enabled. All of these will be done as part of failover.
 
 Bonding on Hardware Router
 --------------------------
 
-Create a LACP bond on the hardware router. We are assuming that eth0 and eth1 are connected to port 8 on both switches, and that those ports are configured as a Port-Channel.
+Create a LACP bond on the hardware router. We are assuming that eth0 and eth1
+are connected to port 8 on both switches, and that those ports are configured
+as a Port-Channel.
 
 .. code-block:: none
 
@@ -100,58 +133,66 @@ Create a LACP bond on the hardware router. We are assuming that eth0 and eth1 ar
    set interfaces bonding bond0 mode '802.3ad'
 
 
-
 Assign external IP addresses
 ----------------------------
 
-VLAN 100 and 201 will have floating IP addresses, but VLAN50 does not, as this is talking directly to upstream. Create our IP address on vlan50.
+VLAN 100 and 201 will have floating IP addresses, but VLAN50 does not, as this
+is talking directly to upstream. Create our IP address on vlan50.
 
-For the hardware router, replace ``eth0`` with ``bond0``. As (almost) every command is identical, this will not be specified unless different things need to be performed on different hosts.
+For the hardware router, replace ``eth0`` with ``bond0``. As (almost) every
+command is identical, this will not be specified unless different things need
+to be performed on different hosts.
 
 .. code-block:: none
 
    set interfaces ethernet eth0 vif 50 address '192.0.2.21/24'
 
-
 In this case, the hardware router has a different IP, so it would be
 
 .. code-block:: none
 
    set interfaces ethernet bond0 vif 50 address '192.0.2.22/24'
 
+Add (temporary) default route
+-----------------------------
 
-Add (temporary) default route, and enable SSH
----------------------------------------------
-
-It is assumed that the routers provided by upstream are capable of acting as a default router. Add that as a static route, and enable SSH so you can now SSH into the routers, rather than using the console.
+It is assumed that the routers provided by upstream are capable of acting as a
+default router, add that as a static route.
 
 .. code-block:: none
 
    set protocols static route 0.0.0.0/0 next-hop 192.0.2.11
-   set service ssh
    commit
    save
 
 
-At this point you should be able to SSH into both of them, and will no longer need access to the console (unless you break something!)
+Enable SSH
+----------
 
-Configure Floating IPs
-----------------------
+Enable SSH so you can now SSH into the routers, rather than using the console.
+
+.. code-block:: none
+
+   set service ssh
+   commit
+   save
 
-Now you can SSH into the routers, it makes it a lot easier to copy-and-paste configurations.
+At this point you should be able to SSH into both of them, and will no longer
+need access to the console (unless you break something!)
 
-We need to set up the fixed and floating IPs.
 
 VRRP Configuration
 ==================
 
+We are setting up VRRP so that it does NOT fail back when a machine returns into
+service, and it prioritizes router1 over router2.
 
-We are setting up VRRP so that it does NOT fail back when a machine returns into service, and it prioritizes router1 over router2.
-
-Internal network
+Internal Network
 ----------------
 
-This has a floating IP address of 10.200.201.1/24, using virtual router ID 201. The difference between them is the interface name, hello-source-address, and peer-address.
+This has a floating IP address of 10.200.201.1/24, using virtual router ID 201.
+The difference between them is the interface name, hello-source-address, and
+peer-address.
 
 **router1**
 
@@ -181,11 +222,13 @@ This has a floating IP address of 10.200.201.1/24, using virtual router ID 201.
    set high-availability vrrp group int vrid '201'
 
 
-Public network
+Public Network
 --------------
 
 This has a floating IP address of 203.0.113.1/24, using virtual router ID 113.
-The virtual router ID is just a random number between 1 and 254, and can be set to whatever you want. Best practices suggest you try to keep them unique enterprise-wide.
+The virtual router ID is just a random number between 1 and 254, and can be set
+to whatever you want. Best practices suggest you try to keep them unique
+enterprise-wide.
 
 **router1**
 
@@ -200,7 +243,6 @@ The virtual router ID is just a random number between 1 and 254, and can be set
    set high-availability vrrp group public virtual-address '203.0.113.1/24'
    set high-availability vrrp group public vrid '113'
 
-
 **router2**
 
 .. code-block:: none
@@ -215,20 +257,23 @@ The virtual router ID is just a random number between 1 and 254, and can be set
    set high-availability vrrp group public vrid '113'
 
 
-Create vrrp sync-group
+Create VRRP sync-group
 ----------------------
 
-The sync group is used to replicate connection tracking. It needs to be assigned to a random VRRP group, and we are creating a sync group called ``sync`` using the vrrp group ``int``.
+The sync group is used to replicate connection tracking. It needs to be assigned
+to a random VRRP group, and we are creating a sync group called ``sync`` using
+the vrrp group ``int``.
 
 .. code-block:: none
 
    set high-availability vrrp sync-group sync member 'int'
 
-
 Testing
 -------
 
-At this point, you should be able to see both IP addresses when you run ``show interfaces``\ , and ``show vrrp`` should show both interfaces in MASTER state (and SLAVE state on router2).
+At this point, you should be able to see both IP addresses when you run
+``show interfaces``\ , and ``show vrrp`` should show both interfaces in MASTER
+state (and SLAVE state on router2).
 
 .. code-block:: none
 
@@ -245,8 +290,11 @@ You should be able to ping to and from all the IPs you have allocated.
 NAT and conntrack-sync
 ======================
 
-Masquerade Traffic originating from 10.200.201.0/24 that is heading out the public interface.
-Note we explicitly exclude the primary upstream network so that BGP or OSPF traffic doesn't accidentally get NAT'ed.
+Masquerade Traffic originating from 10.200.201.0/24 that is heading out the
+public interface.
+
+.. note:: We explicitly exclude the primary upstream network so that BGP or
+   OSPF traffic doesn't accidentally get NAT'ed.
 
 .. code-block:: none
 
@@ -259,7 +307,9 @@ Note we explicitly exclude the primary upstream network so that BGP or OSPF traf
 Configure conntrack-sync and disable helpers
 --------------------------------------------
 
-Most conntrack modules cause more problems than they're worth, especially in a complex network. Turn them off by default, and if you need to turn them on later, you can do so.
+Most conntrack modules cause more problems than they're worth, especially in a
+complex network. Turn them off by default, and if you need to turn them on
+later, you can do so.
 
 .. code-block:: none
 
@@ -270,8 +320,8 @@ Most conntrack modules cause more problems than they're worth, especially in a c
    set system conntrack modules sip disable
    set system conntrack modules tftp disable
 
-
-Now enable replication between nodes. Replace eth0.201 with bond0.201 on the hardware router.
+Now enable replication between nodes. Replace eth0.201 with bond0.201 on the
+hardware router.
 
 .. code-block:: none
 
@@ -282,25 +332,33 @@ Now enable replication between nodes. Replace eth0.201 with bond0.201 on the har
    set service conntrack-sync mcast-group '224.0.0.50'
    set service conntrack-sync sync-queue-size '8'
 
-
 Testing
 -------
 
-The simplest way to test is to look at the connection tracking stats on the standby hardware router with the command ``show conntrack-sync statistics``. The numbers should be very close to the numbers on the primary router.
+The simplest way to test is to look at the connection tracking stats on the
+standby hardware router with the command ``show conntrack-sync statistics``.
+The numbers should be very close to the numbers on the primary router.
 
-When you have both routers up, you should be able to establish a connection from a NAT'ed machine out to the internet, reboot the active machine, and that connection should be preserved, and will not drop out.
+When you have both routers up, you should be able to establish a connection
+from a NAT'ed machine out to the internet, reboot the active machine, and that
+connection should be preserved, and will not drop out.
 
 OSPF Over WireGuard
 ===================
 
-Wireguard doesn't have the concept of an up or down link, due to its design. This complicates AND simplifies using it for network transport, as for reliable state detection you need to use SOMETHING to detect when the link is down.
+Wireguard doesn't have the concept of an up or down link, due to its design.
+This complicates AND simplifies using it for network transport, as for reliable
+state detection you need to use SOMETHING to detect when the link is down.
 
-If you use a routing protocol itself, you solve two problems at once.  This is only a basic example, and is provided as a starting point.
+If you use a routing protocol itself, you solve two problems at once. This is
+only a basic example, and is provided as a starting point.
 
 Configure Wireguard
 -------------------
 
-There is plenty of instructions and documentation on setting up Wireguard. The only important thing you need to remember is to only use one WireGuard interface per OSPF connection.
+There is plenty of instructions and documentation on setting up Wireguard. The
+only important thing you need to remember is to only use one WireGuard
+interface per OSPF connection.
 
 We use small /30's from 10.254.60/24 for the point-to-point links.
 
@@ -349,21 +407,25 @@ This is connecting back to the STATIC IP of router1, not the floating.
    set interfaces wireguard wg01 peer ROUTER1 pubkey 'CKwMV3ZaLntMule2Kd3G7UyVBR7zE8/qoZgLb82EE2Q='
    set interfaces wireguard wg01 port '50001'
 
-
-Test wireguard link
--------------------
+Test WireGuard
+--------------
 
 Make sure you can ping 10.254.60.1 and .2 from both routers.
 
 Create Export Filter
 --------------------
 
-We only want to export the networks we know we should be exporting. Always whitelist your route filters, both importing and exporting.
-A good rule of thumb is **'If you are not the default router for a network, don't advertise it'**. This means we explicitly do not want to
-advertise the 192.0.2.0/24 network (but do want to advertise 10.200.201.0 and 203.0.113.0, which we ARE the default route for).
-This filter is applied to ``redistribute connected``.  If we WERE to advertise it, the remote machines would see 192.0.2.21 available
-via their default route, establish the connection, and then OSPF would say '192.0.2.0/24 is available via this tunnel', at which point
-the tunnel would break, OSPF would drop the routes, and then 192.0.2.0/24 would be reachable via default again. This is called 'flapping'.
+We only want to export the networks we know we should be exporting. Always
+whitelist your route filters, both importing and exporting. A good rule of
+thumb is **'If you are not the default router for a network, don't advertise
+it'**. This means we explicitly do not want to advertise the 192.0.2.0/24
+network (but do want to advertise 10.200.201.0 and 203.0.113.0, which we ARE
+the default route for). This filter is applied to ``redistribute connected``.
+If we WERE to advertise it, the remote machines would see 192.0.2.21 available
+via their default route, establish the connection, and then OSPF would say
+'192.0.2.0/24 is available via this tunnel', at which point the tunnel would
+break, OSPF would drop the routes, and then 192.0.2.0/24 would be reachable via
+default again. This is called 'flapping'.
 
 .. code-block:: none
 
@@ -384,7 +446,9 @@ the tunnel would break, OSPF would drop the routes, and then 192.0.2.0/24 would
 Create Import Filter
 --------------------
 
-We only want to import networks we know about. Our OSPF peer should only be advertising networks in the 10.201.0.0/16 range. Note that this is an INVERSE MATCH. You deny in access-list 100 to accept the route.
+We only want to import networks we know about. Our OSPF peer should only be
+advertising networks in the 10.201.0.0/16 range. Note that this is an INVERSE
+MATCH. You deny in access-list 100 to accept the route.
 
 .. code-block:: none
 
@@ -405,7 +469,9 @@ Enable OSPF
 -----------
 
 Every router **must** have a unique router-id.
-The 'reference-bandwidth' is used because when OSPF was originally designed, the idea of a link faster than 1gbit was unheard of, and it does not scale correctly.
+The 'reference-bandwidth' is used because when OSPF was originally designed,
+the idea of a link faster than 1gbit was unheard of, and it does not scale
+correctly.
 
 .. code-block:: none
 
@@ -421,16 +487,19 @@ The 'reference-bandwidth' is used because when OSPF was originally designed, the
 Test OSPF
 ---------
 
-When you have enabled OSPF on both routers, you should be able to see each other with the command ``show ip ospf neighbour``.
-The state must be 'Full' or '2-Way', if it is not then there is a network connectivity issue between the hosts. This is often caused by NAT or MTU issues.
-You should not see any new routes (unless this is the second pass) in the output of ``show ip route``
+When you have enabled OSPF on both routers, you should be able to see each
+other with the command ``show ip ospf neighbour``. The state must be 'Full'
+or '2-Way', if it is not then there is a network connectivity issue between the
+hosts. This is often caused by NAT or MTU issues. You should not see any new
+routes (unless this is the second pass) in the output of ``show ip route``
 
 Advertise connected routes
 ==========================
 
-As a reminder, only advertise routes that you are the default router for. This is why we are NOT announcing the
-192.0.2.0/24 network, because if that was announced into OSPF, the other routers would try to connect to that
-network over a tunnel that connects to that network!
+As a reminder, only advertise routes that you are the default router for. This
+is why we are NOT announcing the 192.0.2.0/24 network, because if that was
+announced into OSPF, the other routers would try to connect to that network
+over a tunnel that connects to that network!
 
 .. code-block:: none
 
@@ -443,12 +512,14 @@ You should now be able to see the advertised network on the other host.
 Duplicate configuration
 -----------------------
 
-At this pont you now need to create the X link between all four routers. Use a different /30 for each link.
+At this pont you now need to create the X link between all four routers. Use a
+different /30 for each link.
 
 Priorities
 ----------
 
-Set the cost on the secondary links to be 200. This means that they will not be used unless the primary links are down.
+Set the cost on the secondary links to be 200. This means that they will not
+be used unless the primary links are down.
 
 .. code-block:: none
 
@@ -461,12 +532,16 @@ This will be visible in 'show ip route'.
 BGP
 ===
 
-BGP is an extremely complex network protocol. An example is provided here. Note, again, router id's must be unique.
+BGP is an extremely complex network protocol. An example is provided here.
+
+.. note:: Router id's must be unique.
 
 **router1**
 
 
-The ``redistribute ospf`` command is there purely as an example of how this can be expanded. In this walkthrough, it will be filtered by BGPOUT rule 10000, as it is not 203.0.113.0/24.
+The ``redistribute ospf`` command is there purely as an example of how this can
+be expanded. In this walkthrough, it will be filtered by BGPOUT rule 10000, as
+it is not 203.0.113.0/24.
 
 .. code-block:: none
 
@@ -501,4 +576,5 @@ The ``redistribute ospf`` command is there purely as an example of how this can
 
 **router2**
 
-This is identical, but you use the BGPPREPENDOUT route-map to advertise the route with a longer path.
+This is identical, but you use the BGPPREPENDOUT route-map to advertise the
+route with a longer path.