summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
-rw-r--r--docs/configexamples/index.rst3
-rw-r--r--docs/configexamples/l3vpn-hub-and-spoke.rst1130
2 files changed, 1132 insertions, 1 deletions
diff --git a/docs/configexamples/index.rst b/docs/configexamples/index.rst
index b1f34801..3a3c2849 100644
--- a/docs/configexamples/index.rst
+++ b/docs/configexamples/index.rst
@@ -18,6 +18,7 @@ This chapter contains various configuration examples:
ha
wan-load-balancing
pppoe-ipv6-basic
+ l3vpn-hub-and-spoke
Configuration Blueprints (autotest)
@@ -45,4 +46,4 @@ The process will do the following steps:
:maxdepth: 1
autotest/L3VPN_EVPN/L3VPN_EVPN
- autotest/Wireguard/Wireguard \ No newline at end of file
+ autotest/Wireguard/Wireguard
diff --git a/docs/configexamples/l3vpn-hub-and-spoke.rst b/docs/configexamples/l3vpn-hub-and-spoke.rst
new file mode 100644
index 00000000..a2520ea4
--- /dev/null
+++ b/docs/configexamples/l3vpn-hub-and-spoke.rst
@@ -0,0 +1,1130 @@
+
+##############################################
+L3VPN for Hub-and-Spoke connectivity with VyOS
+##############################################
+
+IP/MPLS technology is widely used by various service providers and large
+enterprises in order to achieve better network scalability, manageability
+and flexibility. It also provides the possibility to deliver different
+services for the customers in a seamless manner.
+Layer 3 VPN (L3VPN) is a type of VPN mode that is built and delivered
+through OSI layer 3 networking technologies. Often the border gateway
+protocol (BGP) is used to send and receive VPN-related data that is
+responsible for the control plane. L3VPN utilizes virtual routing and
+forwarding (VRF) techniques to receive and deliver user data as well as
+separate data planes of the end-users. It is built using a combination of
+IP- and MPLS-based information. Generally, L3VPNs are used to send data
+on back-end VPN infrastructures, such as for VPN connections between data
+centres, HQs and branches.
+
+An L3VPN consists of multiple access links, multiple VPN routing and
+forwarding (VRF) tables, and multiple MPLS paths or multiple P2MP LSPs.
+An L3VPN can be configured to connect two or more customer sites.
+In hub-and-spoke MPLS L3VPN environments, the spoke routers need to have
+unique Route Distinguishers (RDs). In order to use the hub site as a
+transit point for connectivity in such an environment, the spoke sites
+export their routes to the hub. Spokes can talk to hubs, but never have
+direct paths to other spokes. All traffic between spokes is controlled
+and delivered over the hub site.
+
+
+To deploy a Layer3 VPN with MPLS on VyOS, we should meet a couple
+requirements in order to properly implement the solution.
+We'll use the following nodes in our LAB environment:
+
+* 2 x Route reflectors (VyOS-RRx)
+* 4 x Provider routers (VyOS-Px)
+* 3 x Provider Edge (VyOs-PEx)
+* 3 x Customer Edge (VyOS-CEx)
+
+The following software was used in the creation of this document:
+
+* Operating system: VyOS
+* Version: 1.4-rolling-202110310317
+* Image name: vyos-1.4-rolling-202110310317-amd64.iso
+
+**NOTE:** VyOS Router (tested with VyOS 1.4-rolling-202110310317)
+– The configurations below are specifically for VyOS 1.4.x.
+
+General information can be found in the :ref:`l3vpn-vrfs` chapter.
+
+
+
+********
+Topology
+********
+.. image:: /_static/images/L3VPN_hub_spoke.png
+ :width: 80%
+ :align: center
+ :alt: Network Topology Diagram
+
+
+
+*****************
+How does it work?
+*****************
+
+As we know the main assumption of L3VPN “Hub and Spoke” is, that the traffic
+between spokes have to pass via hub, in our scenario VyOS-PE2 is the Hub PE
+and the VyOS-CE1-HUB is the central customer office device that is responsible
+for controlling access between all spokes and announcing its network prefixes
+(100.100.100.100/32). VyOS-PE2 has the main VRF (its name is BLUE_HUB), its
+own Route-Distinguisher(RD) and route-target import/export lists.
+Multiprotocol-BGP(MP-BGP) delivers L3VPN related control-plane information to
+the nodes across network where PEs Spokes import the route-target 60535:1030
+(this is export route-target of vrf BLUE_HUB) and export its own route-target
+60535:1011(this is vrf BLUE_SPOKE export route-target). Therefore, the
+Customer edge nodes can only learn the network prefixes of the HUB site
+[100.100.100.100/32]. For this example VyOS-CE1 has network prefixes
+[80.80.80.80/32] / VyOS-CE2 has network prefixes [90.90.90.90/32].
+Route-Reflector devices VyOS-RR1 and VyOS-RR2 are used to simplify network
+routes exchange and minimize iBGP peerings between devices.
+
+L3VPN configuration parameters table:
+
++----------+-------+------------+-----------------+-------------+-------------+
+| Node | Role | VRF | RD | RT import | RT export |
++----------+-------+------------+-----------------+-------------+-------------+
+| VyOS-PE2 | Hub | BLUE_HUB | 10.80.80.1:1011 | 65035:1011 | 65035:1030 |
+| | | | | 65035:1030 | |
++----------+-------+------------+-----------------+-------------+-------------+
+| VyOS-PE1 | Spoke | BLUE_SPOKE | 10.50.50.1:1011 | 65035:1030 | 65035:1011 |
++----------+-------+------------+-----------------+-------------+-------------+
+| VyOS-PE3 | Spoke | BLUE_SPOKE | 10.60.60.1:1011 | 65035:1030 | 65035:1011 |
++----------+-------+------------+-----------------+-------------+-------------+
+
+
+
+*************
+Configuration
+*************
+
+
+
+Step-1: Configuring IGP and enabling MPLS LDP
+=====================================
+
+At the first step we need to configure the IP/MPLS backbone network using OSPF as
+IGP protocol and LDP as label-switching protocol for the base connectivity between
+**P** (rovider), **P** (rovider) **E** (dge) and **R** (oute) **R** (eflector) nodes:
+
+- VyOS-P1:
+
+.. code-block:: none
+
+ # interfaces
+ set interfaces dummy dum10 address '3.3.3.3/32'
+ set interfaces ethernet eth0 address '172.16.30.1/24'
+ set interfaces ethernet eth1 address '172.16.40.1/24'
+ set interfaces ethernet eth2 address '172.16.90.1/24'
+ set interfaces ethernet eth3 address '172.16.10.1/24'
+ set interfaces ethernet eth5 address '172.16.100.1/24'
+
+ # protocols ospf+ldp
+ set protocols mpls interface 'eth1'
+ set protocols mpls interface 'eth2'
+ set protocols mpls interface 'eth3'
+ set protocols mpls interface 'eth5'
+ set protocols mpls interface 'eth0'
+ set protocols mpls ldp discovery transport-ipv4-address '3.3.3.3'
+ set protocols mpls ldp interface 'eth0'
+ set protocols mpls ldp interface 'eth1'
+ set protocols mpls ldp interface 'eth2'
+ set protocols mpls ldp interface 'eth3'
+ set protocols mpls ldp interface 'eth5'
+ set protocols mpls ldp router-id '3.3.3.3'
+ set protocols ospf area 0 network '0.0.0.0/0'
+ set protocols ospf parameters abr-type 'cisco'
+ set protocols ospf parameters router-id '3.3.3.3
+
+
+- VyOS-P2:
+
+.. code-block:: none
+
+ # interfaces
+ set interfaces dummy dum10 address '4.4.4.4/32'
+ set interfaces ethernet eth0 address '172.16.30.2/24'
+ set interfaces ethernet eth1 address '172.16.20.1/24'
+ set interfaces ethernet eth2 address '172.16.120.1/24'
+ set interfaces ethernet eth3 address '172.16.60.1/24'
+
+ # protocols ospf+ldp
+ set protocols mpls interface 'eth1'
+ set protocols mpls interface 'eth2'
+ set protocols mpls interface 'eth3'
+ set protocols mpls interface 'eth0'
+ set protocols mpls ldp discovery transport-ipv4-address '4.4.4.4'
+ set protocols mpls ldp interface 'eth0'
+ set protocols mpls ldp interface 'eth1'
+ set protocols mpls ldp interface 'eth2'
+ set protocols mpls ldp interface 'eth3'
+ set protocols mpls ldp router-id '4.4.4.4'
+ set protocols ospf area 0 network '0.0.0.0/0'
+ set protocols ospf parameters abr-type 'cisco'
+ set protocols ospf parameters router-id '4.4.4.4'
+
+- VyOS-P3:
+
+.. code-block:: none
+
+ # interfaces
+ set interfaces dummy dum10 address '5.5.5.5/32'
+ set interfaces ethernet eth0 address '172.16.110.1/24'
+ set interfaces ethernet eth1 address '172.16.40.2/24'
+ set interfaces ethernet eth2 address '172.16.50.1/24'
+ set interfaces ethernet eth3 address '172.16.70.1/24'
+
+ # protocols ospf + ldp
+ set protocols mpls interface 'eth1'
+ set protocols mpls interface 'eth2'
+ set protocols mpls interface 'eth3'
+ set protocols mpls interface 'eth0'
+ set protocols mpls ldp discovery transport-ipv4-address '5.5.5.5'
+ set protocols mpls ldp interface 'eth0'
+ set protocols mpls ldp interface 'eth1'
+ set protocols mpls ldp interface 'eth2'
+ set protocols mpls ldp interface 'eth3'
+ set protocols mpls ldp router-id '5.5.5.5'
+ set protocols ospf area 0 network '0.0.0.0/0'
+ set protocols ospf parameters abr-type 'cisco'
+ set protocols ospf parameters router-id '5.5.5.5'
+
+- VyOS-P4:
+
+.. code-block:: none
+
+ # interfaces
+ set interfaces dummy dum10 address '6.6.6.6/32'
+ set interfaces ethernet eth0 address '172.16.80.2/24'
+ set interfaces ethernet eth1 address '172.16.130.1/24'
+ set interfaces ethernet eth2 address '172.16.50.2/24'
+ set interfaces ethernet eth3 address '172.16.60.2/24'
+ set interfaces ethernet eth5 address '172.16.140.1/24'
+
+
+ # protocols ospf + ldp
+ set protocols mpls interface 'eth1'
+ set protocols mpls interface 'eth2'
+ set protocols mpls interface 'eth3'
+ set protocols mpls interface 'eth0'
+ set protocols mpls interface 'eth5'
+ set protocols mpls ldp discovery transport-ipv4-address '6.6.6.6'
+ set protocols mpls ldp interface 'eth0'
+ set protocols mpls ldp interface 'eth1'
+ set protocols mpls ldp interface 'eth2'
+ set protocols mpls ldp interface 'eth3'
+ set protocols mpls ldp interface 'eth5'
+ set protocols mpls ldp router-id '6.6.6.6'
+ set protocols ospf area 0 network '0.0.0.0/0'
+ set protocols ospf parameters abr-type 'cisco'
+ set protocols ospf parameters router-id '6.6.6.6'
+
+- VyOS-PE1:
+
+.. code-block:: none
+
+ # interfaces
+ set interfaces dummy dum10 address '7.7.7.7/32'
+ set interfaces ethernet eth0 address '172.16.90.2/24'
+
+ # protocols ospf + ldp
+ set protocols mpls interface 'eth0'
+ set protocols mpls ldp discovery transport-ipv4-address '7.7.7.7'
+ set protocols mpls ldp interface 'eth0'
+ set protocols mpls ldp router-id '7.7.7.7'
+ set protocols ospf area 0 network '0.0.0.0/0'
+ set protocols ospf parameters abr-type 'cisco'
+ set protocols ospf parameters router-id '7.7.7.7'
+
+- VyOS-PE2:
+
+.. code-block:: none
+
+ # interfaces
+ set interfaces dummy dum10 address '8.8.8.8/32'
+ set interfaces ethernet eth0 address '172.16.110.2/24'
+ set interfaces ethernet eth1 address '172.16.100.2/24'
+ set interfaces ethernet eth2 address '172.16.80.1/24'
+
+ # protocols ospf + ldp
+ set protocols mpls interface 'eth0'
+ set protocols mpls interface 'eth1'
+ set protocols mpls ldp discovery transport-ipv4-address '8.8.8.8'
+ set protocols mpls ldp interface 'eth0'
+ set protocols mpls ldp interface 'eth1'
+ set protocols mpls ldp router-id '8.8.8.8'
+ set protocols ospf area 0 network '0.0.0.0/0'
+ set protocols ospf parameters abr-type 'cisco'
+ set protocols ospf parameters router-id '8.8.8.8'
+
+- VyOS-PE3:
+
+.. code-block:: none
+
+ # interfaces
+ set interfaces dummy dum10 address '10.10.10.10/32'
+ set interfaces ethernet eth0 address '172.16.140.2/24'
+
+ # protocols ospf + ldp
+ set protocols mpls interface 'eth0'
+ set protocols mpls ldp discovery transport-ipv4-address '10.10.10.10'
+ set protocols mpls ldp interface 'eth0'
+ set protocols mpls ldp router-id '10.10.10.10'
+ set protocols ospf area 0 network '0.0.0.0/0'
+ set protocols ospf parameters abr-type 'cisco'
+ set protocols ospf parameters router-id '10.10.10.10'
+
+- VyOS-RR1:
+
+.. code-block:: none
+
+ # interfaces
+ set interfaces ethernet eth1 address '172.16.20.2/24'
+ set interfaces ethernet eth2 address '172.16.10.2/24'
+ set interfaces dummy dum10 address '1.1.1.1/32'
+
+ # protocols ospf + ldp
+ set protocols mpls interface 'eth1'
+ set protocols mpls interface 'eth2'
+ set protocols mpls ldp discovery transport-ipv4-address '1.1.1.1'
+ set protocols mpls ldp interface 'eth1'
+ set protocols mpls ldp interface 'eth2'
+ set protocols mpls ldp router-id '1.1.1.1'
+ set protocols ospf area 0 network '0.0.0.0/0'
+ set protocols ospf parameters abr-type 'cisco'
+ set protocols ospf parameters router-id '1.1.1.1'
+
+- VyOS-RR2:
+
+.. code-block:: none
+
+ # interfaces
+ set interfaces ethernet eth0 address '172.16.80.1/24'
+ set interfaces ethernet eth1 address '172.16.70.2/24'
+ set interfaces dummy dum10 address '2.2.2.2/32'
+
+ # protocols ospf + ldp
+ set protocols mpls interface 'eth0'
+ set protocols mpls interface 'eth1'
+ set protocols mpls ldp discovery transport-ipv4-address '2.2.2.2'
+ set protocols mpls ldp interface 'eth1'
+ set protocols mpls ldp interface 'eth0'
+ set protocols mpls ldp router-id '2.2.2.2'
+ set protocols ospf area 0 network '0.0.0.0/0'
+ set protocols ospf parameters abr-type 'cisco'
+ set protocols ospf parameters router-id '2.2.2.2'
+
+
+
+Step-2: Configuring iBGP for L3VPN control-plane
+================================================
+
+At this step we are going to enable iBGP protocol on MPLS nodes and
+Route Reflectors (two routers for redundancy) that will deliver IPv4
+VPN (L3VPN) routes between them:
+
+- VyOS-RR1:
+
+.. code-block:: none
+
+ set protocols bgp local-as '65001'
+ set protocols bgp neighbor 7.7.7.7 address-family ipv4-vpn route-reflector-client
+ set protocols bgp neighbor 7.7.7.7 peer-group 'RR_VPNv4'
+ set protocols bgp neighbor 8.8.8.8 address-family ipv4-vpn route-reflector-client
+ set protocols bgp neighbor 8.8.8.8 peer-group 'RR_VPNv4'
+ set protocols bgp neighbor 9.9.9.9 address-family ipv4-vpn route-reflector-client
+ set protocols bgp neighbor 9.9.9.9 peer-group 'RR_VPNv4'
+ set protocols bgp neighbor 10.10.10.10 address-family ipv4-vpn route-reflector-client
+ set protocols bgp neighbor 10.10.10.10 peer-group 'RR_VPNv4'
+ set protocols bgp parameters cluster-id '1.1.1.1'
+ set protocols bgp parameters default no-ipv4-unicast
+ set protocols bgp parameters log-neighbor-changes
+ set protocols bgp parameters router-id '1.1.1.1'
+ set protocols bgp peer-group RR_VPNv4 remote-as '65001'
+ set protocols bgp peer-group RR_VPNv4 update-source 'dum10'
+
+- VyOS-RR2:
+
+.. code-block:: none
+
+ set protocols bgp local-as '65001'
+ set protocols bgp neighbor 7.7.7.7 address-family ipv4-vpn route-reflector-client
+ set protocols bgp neighbor 7.7.7.7 peer-group 'RR_VPNv4'
+ set protocols bgp neighbor 8.8.8.8 address-family ipv4-vpn route-reflector-client
+ set protocols bgp neighbor 8.8.8.8 peer-group 'RR_VPNv4'
+ set protocols bgp neighbor 9.9.9.9 address-family ipv4-vpn route-reflector-client
+ set protocols bgp neighbor 9.9.9.9 peer-group 'RR_VPNv4'
+ set protocols bgp neighbor 10.10.10.10 address-family ipv4-vpn route-reflector-client
+ set protocols bgp neighbor 10.10.10.10 peer-group 'RR_VPNv4'
+ set protocols bgp parameters cluster-id '1.1.1.1'
+ set protocols bgp parameters default no-ipv4-unicast
+ set protocols bgp parameters log-neighbor-changes
+ set protocols bgp parameters router-id '2.2.2.2'
+ set protocols bgp peer-group RR_VPNv4 remote-as '65001'
+ set protocols bgp peer-group RR_VPNv4 update-source 'dum10'
+
+- VyOS-PE1:
+
+.. code-block:: none
+
+ set protocols bgp local-as '65001'
+ set protocols bgp neighbor 1.1.1.1 address-family ipv4-vpn nexthop-self
+ set protocols bgp neighbor 1.1.1.1 peer-group 'RR_VPNv4'
+ set protocols bgp neighbor 2.2.2.2 address-family ipv4-vpn nexthop-self
+ set protocols bgp neighbor 2.2.2.2 peer-group 'RR_VPNv4'
+ set protocols bgp parameters default no-ipv4-unicast
+ set protocols bgp parameters log-neighbor-changes
+ set protocols bgp parameters router-id '7.7.7.7'
+ set protocols bgp peer-group RR_VPNv4 remote-as '65001'
+ set protocols bgp peer-group RR_VPNv4 update-source 'dum10'
+
+- VyOS-PE2:
+
+.. code-block:: none
+
+ set protocols bgp local-as '65001'
+ set protocols bgp neighbor 1.1.1.1 address-family ipv4-vpn nexthop-self
+ set protocols bgp neighbor 1.1.1.1 peer-group 'RR_VPNv4'
+ set protocols bgp neighbor 2.2.2.2 address-family ipv4-vpn nexthop-self
+ set protocols bgp neighbor 2.2.2.2 peer-group 'RR_VPNv4'
+ set protocols bgp parameters default no-ipv4-unicast
+ set protocols bgp parameters log-neighbor-changes
+ set protocols bgp parameters router-id '8.8.8.8'
+ set protocols bgp peer-group RR_VPNv4 remote-as '65001'
+ set protocols bgp peer-group RR_VPNv4 update-source 'dum10'
+
+- VyOS-PE3:
+
+.. code-block:: none
+
+ set protocols bgp local-as '65001'
+ set protocols bgp neighbor 1.1.1.1 address-family ipv4-vpn nexthop-self
+ set protocols bgp neighbor 1.1.1.1 peer-group 'RR_VPNv4'
+ set protocols bgp neighbor 2.2.2.2 address-family ipv4-vpn nexthop-self
+ set protocols bgp neighbor 2.2.2.2 peer-group 'RR_VPNv4'
+ set protocols bgp parameters default no-ipv4-unicast
+ set protocols bgp parameters log-neighbor-changes
+ set protocols bgp parameters router-id '10.10.10.10'
+ set protocols bgp peer-group RR_VPNv4 remote-as '65001'
+ set protocols bgp peer-group RR_VPNv4 update-source 'dum10'
+
+
+
+Step-3: Configuring L3VPN VRFs on PE nodes
+==========================================
+
+This section provides configuration steps for setting up VRFs on our
+PE nodes including CE facing interfaces, BGP, rd and route-target
+import/export based on the pre-defined parameters.
+
+- VyOS-PE1:
+
+.. code-block:: none
+
+ # VRF settings
+ set vrf name BLUE_SPOKE table '200'
+ set vrf name BLUE_SPOKE protocols bgp address-family ipv4-unicast export vpn
+ set vrf name BLUE_SPOKE protocols bgp address-family ipv4-unicast import vpn
+ set vrf name BLUE_SPOKE protocols bgp address-family ipv4-unicast label vpn export 'auto'
+ set vrf name BLUE_SPOKE protocols bgp address-family ipv4-unicast network 10.50.50.0/24
+ set vrf name BLUE_SPOKE protocols bgp address-family ipv4-unicast rd vpn export '10.50.50.1:1011'
+ set vrf name BLUE_SPOKE protocols bgp address-family ipv4-unicast redistribute connected
+ set vrf name BLUE_SPOKE protocols bgp address-family ipv4-unicast route-target vpn export '65035:1011'
+ set vrf name BLUE_SPOKE protocols bgp address-family ipv4-unicast route-target vpn import '65035:1030'
+ set vrf name BLUE_SPOKE protocols bgp local-as '65001'
+ set vrf name BLUE_SPOKE protocols bgp neighbor 10.50.50.2 address-family ipv4-unicast as-override
+ set vrf name BLUE_SPOKE protocols bgp neighbor 10.50.50.2 remote-as '65035'
+
+ # interfaces
+ set interfaces ethernet eth3 address '10.50.50.1/24'
+ set interfaces ethernet eth3 vrf 'BLUE_SPOKE'
+
+- VyOS-PE2:
+
+.. code-block:: none
+
+ # VRF settings
+ set vrf name BLUE_HUB table '400'
+ set vrf name BLUE_HUB protocols bgp address-family ipv4-unicast export vpn
+ set vrf name BLUE_HUB protocols bgp address-family ipv4-unicast import vpn
+ set vrf name BLUE_HUB protocols bgp address-family ipv4-unicast label vpn export 'auto'
+ set vrf name BLUE_HUB protocols bgp address-family ipv4-unicast network 10.80.80.0/24
+ set vrf name BLUE_HUB protocols bgp address-family ipv4-unicast rd vpn export '10.80.80.1:1011'
+ set vrf name BLUE_HUB protocols bgp address-family ipv4-unicast redistribute connected
+ set vrf name BLUE_HUB protocols bgp address-family ipv4-unicast route-target vpn export '65035:1030'
+ set vrf name BLUE_HUB protocols bgp address-family ipv4-unicast route-target vpn import '65035:1011 65050:2011 65035:1030'
+ set vrf name BLUE_HUB protocols bgp local-as '65001'
+ set vrf name BLUE_HUB protocols bgp neighbor 10.80.80.2 address-family ipv4-unicast as-override
+ set vrf name BLUE_HUB protocols bgp neighbor 10.80.80.2 remote-as '65035'
+
+ # interfaces
+ set interfaces ethernet eth3 address '10.80.80.1/24'
+ set interfaces ethernet eth3 vrf 'BLUE_HUB'
+
+- VyOS-PE3:
+
+.. code-block:: none
+
+ # VRF settings
+ set vrf name BLUE_SPOKE table '200'
+ set vrf name BLUE_SPOKE protocols bgp address-family ipv4-unicast export vpn
+ set vrf name BLUE_SPOKE protocols bgp address-family ipv4-unicast import vpn
+ set vrf name BLUE_SPOKE protocols bgp address-family ipv4-unicast label vpn export 'auto'
+ set vrf name BLUE_SPOKE protocols bgp address-family ipv4-unicast network 10.60.60.0/24
+ set vrf name BLUE_SPOKE protocols bgp address-family ipv4-unicast rd vpn export '10.60.60.1:1011'
+ set vrf name BLUE_SPOKE protocols bgp address-family ipv4-unicast redistribute connected
+ set vrf name BLUE_SPOKE protocols bgp address-family ipv4-unicast route-target vpn export '65035:1011'
+ set vrf name BLUE_SPOKE protocols bgp address-family ipv4-unicast route-target vpn import '65035:1030'
+ set vrf name BLUE_SPOKE protocols bgp local-as '65001'
+ set vrf name BLUE_SPOKE protocols bgp neighbor 10.60.60.2 address-family ipv4-unicast as-override
+ set vrf name BLUE_SPOKE protocols bgp neighbor 10.60.60.2 remote-as '65035'
+
+ # interfaces
+ set interfaces ethernet eth3 address '10.60.60.1/24'
+ set interfaces ethernet eth3 vrf 'BLUE_SPOKE'
+
+
+
+Step-4: Configuring CE nodes
+============================
+
+Dynamic routing used between CE and PE nodes and eBGP peering
+established for the route exchanging between them. All routes
+received by PEs are then exported to L3VPN and delivered from
+Spoke sites to Hub and vise-versa based on previously
+configured L3VPN parameters.
+
+- VyOS-CE1-SPOKE:
+
+.. code-block:: none
+
+ # interfaces
+ set interfaces dummy dum20 address '80.80.80.80/32'
+ set interfaces ethernet eth0 address '10.50.50.2/24'
+
+ # BGP for peering with PE
+ set protocols bgp 65035 address-family ipv4-unicast network 80.80.80.80/32
+ set protocols bgp 65035 neighbor 10.50.50.1 ebgp-multihop '2'
+ set protocols bgp 65035 neighbor 10.50.50.1 remote-as '65001'
+ set protocols bgp 65035 neighbor 10.50.50.1 update-source 'eth0'
+ set protocols bgp 65035 parameters default no-ipv4-unicast
+ set protocols bgp 65035 parameters log-neighbor-changes
+ set protocols bgp 65035 parameters router-id '10.50.50.2'
+
+- VyOS-CE1-HUB:
+
+.. code-block:: none
+
+ # interfaces
+ set interfaces dummy dum20 address '100.100.100.100/32'
+ set interfaces ethernet eth0 address '10.80.80.2/24'
+
+ # BGP for peering with PE
+ set protocols bgp 65035 address-family ipv4-unicast network 100.100.100.100/32
+ set protocols bgp 65035 address-family ipv4-unicast redistribute connected
+ set protocols bgp 65035 neighbor 10.80.80.1 ebgp-multihop '2'
+ set protocols bgp 65035 neighbor 10.80.80.1 remote-as '65001'
+ set protocols bgp 65035 neighbor 10.80.80.1 update-source 'eth0'
+ set protocols bgp 65035 parameters default no-ipv4-unicast
+ set protocols bgp 65035 parameters log-neighbor-changes
+ set protocols bgp 65035 parameters router-id '10.80.80.2'
+
+- VyOS-CE2-SPOKE:
+
+.. code-block:: none
+
+ # interfaces
+ set interfaces dummy dum20 address '90.90.90.90/32'
+ set interfaces ethernet eth0 address '10.60.60.2/24'
+
+ # BGP for peering with PE
+ set protocols bgp 65035 address-family ipv4-unicast network 90.90.90.90/32
+ set protocols bgp 65035 neighbor 10.60.60.1 ebgp-multihop '2'
+ set protocols bgp 65035 neighbor 10.60.60.1 remote-as '65001'
+ set protocols bgp 65035 neighbor 10.60.60.1 update-source 'eth0'
+ set protocols bgp 65035 parameters default no-ipv4-unicast
+ set protocols bgp 65035 parameters log-neighbor-changes
+ set protocols bgp 65035 parameters router-id '10.60.60.2'
+
+
+
+Step-5: Verification
+====================
+
+This section describes verification commands for MPLS/BGP/LDP
+protocols and L3VPN related routes as well as diagnosis and
+reachability checks between CE nodes.
+
+Let’s check IPv4 routing and MPLS information on provider nodes
+(same procedure for all P nodes):
+
+- “show ip ospf neighbor” for checking ospf relationship
+
+.. code-block:: none
+
+ vyos@VyOS-P1:~$ show ip ospf neighbor
+
+ Neighbor ID Pri State Dead Time Address Interface RXmtL RqstL DBsmL
+ 4.4.4.4 1 Full/Backup 34.718s 172.16.30.2 eth0:172.16.30.1 0 0 0
+ 5.5.5.5 1 Full/Backup 35.132s 172.16.40.2 eth1:172.16.40.1 0 0 0
+ 7.7.7.7 1 Full/Backup 34.764s 172.16.90.2 eth2:172.16.90.1 0 0 0
+ 1.1.1.1 1 Full/Backup 35.642s 172.16.10.2 eth3:172.16.10.1 0 0 0
+ 8.8.8.8 1 Full/Backup 35.484s 172.16.100.2 eth5:172.16.100.1 0 0 0
+
+- “show mpls ldp neighbor “ for checking ldp neighbors
+
+.. code-block:: none
+
+ vyos@VyOS-P1:~$ show mpls ldp neighbor
+ AF ID State Remote Address Uptime
+ ipv4 1.1.1.1 OPERATIONAL 1.1.1.1 07w5d06h
+ ipv4 4.4.4.4 OPERATIONAL 4.4.4.4 09w3d00h
+ ipv4 5.5.5.5 OPERATIONAL 5.5.5.5 09w2d23h
+ ipv4 7.7.7.7 OPERATIONAL 7.7.7.7 03w0d01h
+ ipv4 8.8.8.8 OPERATIONAL 8.8.8.8 01w3d02h
+
+- “show mpls ldp binding” for checking mpls label assignment
+
+.. code-block:: none
+
+ vyos@VyOS-P1:~$ show mpls ldp discovery
+ AF Destination Nexthop Local Label Remote Label In Use
+ ipv4 1.1.1.1/32 1.1.1.1 23 imp-null yes
+ ipv4 1.1.1.1/32 4.4.4.4 23 20 no
+ ipv4 1.1.1.1/32 5.5.5.5 23 17 no
+ ipv4 1.1.1.1/32 7.7.7.7 23 16 no
+ ipv4 1.1.1.1/32 8.8.8.8 23 16 no
+ ipv4 2.2.2.2/32 1.1.1.1 20 16 no
+ ipv4 2.2.2.2/32 4.4.4.4 20 22 no
+ ipv4 2.2.2.2/32 5.5.5.5 20 24 yes
+ ipv4 2.2.2.2/32 7.7.7.7 20 17 no
+ ipv4 2.2.2.2/32 8.8.8.8 20 17 no
+ ipv4 3.3.3.3/32 1.1.1.1 imp-null 17 no
+ ipv4 3.3.3.3/32 4.4.4.4 imp-null 16 no
+ ipv4 3.3.3.3/32 5.5.5.5 imp-null 18 no
+ ipv4 3.3.3.3/32 7.7.7.7 imp-null 18 no
+ ipv4 3.3.3.3/32 8.8.8.8 imp-null 18 no
+ ipv4 4.4.4.4/32 1.1.1.1 16 18 no
+ ipv4 4.4.4.4/32 4.4.4.4 16 imp-null yes
+ ipv4 4.4.4.4/32 5.5.5.5 16 19 no
+ ipv4 4.4.4.4/32 7.7.7.7 16 19 no
+ ipv4 4.4.4.4/32 8.8.8.8 16 19 no
+ ipv4 5.5.5.5/32 1.1.1.1 21 19 no
+ ipv4 5.5.5.5/32 4.4.4.4 21 17 no
+ ipv4 5.5.5.5/32 5.5.5.5 21 imp-null yes
+ ipv4 5.5.5.5/32 7.7.7.7 21 20 no
+ ipv4 5.5.5.5/32 8.8.8.8 21 20 no
+ ipv4 6.6.6.6/32 1.1.1.1 17 20 no
+ ipv4 6.6.6.6/32 4.4.4.4 17 23 yes
+ ipv4 6.6.6.6/32 5.5.5.5 17 21 yes
+ ipv4 6.6.6.6/32 7.7.7.7 17 21 no
+ ipv4 6.6.6.6/32 8.8.8.8 17 21 no
+ ipv4 7.7.7.7/32 1.1.1.1 22 21 no
+ ipv4 7.7.7.7/32 4.4.4.4 22 18 no
+ ipv4 7.7.7.7/32 5.5.5.5 22 20 no
+ ipv4 7.7.7.7/32 7.7.7.7 22 imp-null yes
+ ipv4 7.7.7.7/32 8.8.8.8 22 22 no
+ ipv4 8.8.8.8/32 1.1.1.1 24 22 no
+ ipv4 8.8.8.8/32 4.4.4.4 24 19 no
+ ipv4 8.8.8.8/32 5.5.5.5 24 16 no
+ ipv4 8.8.8.8/32 7.7.7.7 24 22 no
+ ipv4 8.8.8.8/32 8.8.8.8 24 imp-null yes
+ ipv4 9.9.9.9/32 1.1.1.1 18 23 no
+ ipv4 9.9.9.9/32 4.4.4.4 18 21 yes
+ ipv4 9.9.9.9/32 5.5.5.5 18 22 no
+ ipv4 9.9.9.9/32 7.7.7.7 18 23 no
+ ipv4 9.9.9.9/32 8.8.8.8 18 23 no
+ ipv4 10.10.10.10/32 1.1.1.1 19 24 no
+ ipv4 10.10.10.10/32 4.4.4.4 19 24 yes
+ ipv4 10.10.10.10/32 5.5.5.5 19 23 yes
+ ipv4 10.10.10.10/32 7.7.7.7 19 24 no
+ ipv4 10.10.10.10/32 8.8.8.8 19 24 no
+
+Now we’re checking iBGP status and routes from route-reflector
+nodes to other devices:
+
+- “show bgp ipv4 vpn summary” for checking BGP VPNv4 neighbors:
+
+.. code-block:: none
+
+ vyos@VyOS-RR1:~$ show bgp ipv4 vpn summary
+ BGP router identifier 1.1.1.1, local AS number 65001 vrf-id 0
+ BGP table version 0
+ RIB entries 9, using 1728 bytes of memory
+ Peers 4, using 85 KiB of memory
+ Peer groups 1, using 64 bytes of memory
+
+ Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd PfxSnt
+ 7.7.7.7 4 65001 7719 7733 0 0 0 5d07h56m 2 10
+ 8.8.8.8 4 65001 7715 7724 0 0 0 5d08h28m 4 10
+ 9.9.9.9 4 65001 7713 7724 0 0 0 5d08h28m 2 10
+ 10.10.10.10 4 65001 7713 7724 0 0 0 5d08h28m 2 10
+
+ Total number of neighbors 4
+
+- “show bgp ipv4 vpn” for checking all VPNv4 prefixes information:
+
+.. code-block:: none
+
+ vyos@VyOS-RR1:~$ show bgp ipv4 vpn
+ BGP table version is 2, local router ID is 1.1.1.1, vrf id 0
+ Default local pref 100, local AS 65001
+ Status codes: s suppressed, d damped, h history, * valid, > best, = multipath,
+ i internal, r RIB-failure, S Stale, R Removed
+ Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self
+ Origin codes: i - IGP, e - EGP, ? - incomplete
+
+ Network Next Hop Metric LocPrf Weight Path
+ Route Distinguisher: 10.50.50.1:1011
+ *>i10.50.50.0/24 7.7.7.7 0 100 0 i
+ UN=7.7.7.7 EC{65035:1011} label=80 type=bgp, subtype=0
+ *>i80.80.80.80/32 7.7.7.7 0 100 0 65035 i
+ UN=7.7.7.7 EC{65035:1011} label=80 type=bgp, subtype=0
+ Route Distinguisher: 10.60.60.1:1011
+ *>i10.60.60.0/24 10.10.10.10 0 100 0 i
+ UN=10.10.10.10 EC{65035:1011} label=80 type=bgp, subtype=0
+ *>i90.90.90.90/32 10.10.10.10 0 100 0 65035 i
+ UN=10.10.10.10 EC{65035:1011} label=80 type=bgp, subtype=0
+ Route Distinguisher: 10.80.80.1:1011
+ *>i10.80.80.0/24 8.8.8.8 0 100 0 i
+ UN=8.8.8.8 EC{65035:1030} label=80 type=bgp, subtype=0
+ *>i100.100.100.100/32
+ 8.8.8.8 0 100 0 65035 i
+ UN=8.8.8.8 EC{65035:1030} label=80 type=bgp, subtype=0
+ Route Distinguisher: 172.16.80.1:2011
+ *>i10.110.110.0/24 8.8.8.8 0 100 0 65050 i
+ UN=8.8.8.8 EC{65050:2011} label=81 type=bgp, subtype=0
+ *>i172.16.80.0/24 8.8.8.8 0 100 0 i
+ UN=8.8.8.8 EC{65050:2011} label=81 type=bgp, subtype=0
+ Route Distinguisher: 172.16.100.1:2011
+ *>i10.210.210.0/24 9.9.9.9 0 100 0 65050 i
+ UN=9.9.9.9 EC{65050:2011} label=80 type=bgp, subtype=0
+ *>i172.16.100.0/24 9.9.9.9 0 100 0 i
+ UN=9.9.9.9 EC{65050:2011} label=80 type=bgp, subtype=0
+
+- “show bgp ipv4 vpn x.x.x.x/x” for checking best path selected
+ for specific VPNv4 destination
+
+.. code-block:: none
+
+ vyos@VyOS-RR1:~$ show bgp ipv4 vpn 100.100.100.100/32
+ BGP routing table entry for 10.80.80.1:1011:100.100.100.100/32
+ not allocated
+ Paths: (1 available, best #1)
+ Advertised to non peer-group peers:
+ 7.7.7.7 8.8.8.8 9.9.9.9 10.10.10.10
+ 65035, (Received from a RR-client)
+ 8.8.8.8 from 8.8.8.8 (8.8.8.8)
+ Origin incomplete, metric 0, localpref 100, valid, internal, best (First path received)
+ Extended Community: RT:65035:1030
+ Remote label: 80
+ Last update: Tue Oct 19 13:45:32 202
+
+Also we can verify how PE devices receives VPNv4 networks from the RRs
+and installing them to the specific customer VRFs:
+
+- “show bgp ipv4 vpn summary” for checking iBGP neighbors against
+ route-reflector devices:
+
+.. code-block:: none
+
+ vyos@VyOS-PE1:~$ show bgp ipv4 vpn summary
+ BGP router identifier 7.7.7.7, local AS number 65001 vrf-id 0
+ BGP table version 0
+ RIB entries 9, using 1728 bytes of memory
+ Peers 2, using 43 KiB of memory
+ Peer groups 1, using 64 bytes of memory
+
+ Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd PfxSnt
+ 1.1.1.1 4 65001 8812 8794 0 0 0 01:18:42 8 2
+ 2.2.2.2 4 65001 8800 8792 0 0 0 6d02h27m 8 2
+
+- “show bgp vrf all” for checking all the prefix learning on BGP
+ within VRFs:
+
+.. code-block:: none
+
+ vyos@VyOS-PE1:~$ show bgp vrf all
+
+ Instance default:
+ No BGP prefixes displayed, 0 exist
+
+ Instance BLUE_SPOKE:
+ BGP table version is 8, local router ID is 10.50.50.1, vrf id 6
+ Default local pref 100, local AS 65001
+ Status codes: s suppressed, d damped, h history, * valid, > best, = multipath,
+ i internal, r RIB-failure, S Stale, R Removed
+ Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self
+ Origin codes: i - IGP, e - EGP, ? - incomplete
+
+ Network Next Hop Metric LocPrf Weight Path
+ * 10.50.50.0/24 0.0.0.0 0 32768 ?
+ *> 0.0.0.0 0 32768 i
+ *> 10.80.80.0/24 8.8.8.8@0< 0 100 0 i
+ * 8.8.8.8@0< 0 100 0 i
+ *> 80.80.80.80/32 10.50.50.2 0 0 65035 i
+ *> 100.100.100.100/32
+ 8.8.8.8@0< 0 100 0 65035 ?
+ * 8.8.8.8@0< 0 100 0 65035 ?
+
+- “show bgp vrf BLUE_SPOKE summary” for checking EBGP neighbor
+ information between PE and CE:
+
+.. code-block:: none
+
+ vyos@VyOS-PE1:~$ show bgp vrf BLUE_SPOKE summary
+
+
+ IPv4 Unicast Summary:
+ BGP router identifier 10.50.50.1, local AS number 65001 vrf-id 6
+ BGP table version 8
+ RIB entries 7, using 1344 bytes of memory
+ Peers 1, using 21 KiB of memory
+
+ Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd PfxSnt
+ 10.50.50.2 4 65035 9019 9023 0 0 0 6d06h12m 1 4
+
+ Total number of neighbors 1
+
+- “show ip route vrf BLUE_SPOKE” for viewing the RIB in our Spoke PE.
+ Using this command we are also able to check the transport and
+ customer label (inner/outer) for Hub network prefix (100.100.100.100/32):
+
+.. code-block:: none
+
+ vyos@VyOS-PE1:~$ show ip route vrf BLUE_SPOKE
+
+ Codes: K - kernel route, C - connected, S - static, R - RIP,
+ O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
+ T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
+ F - PBR, f - OpenFabric,
+ > - selected route, * - FIB route, q - queued, r - rejected, b - backup
+
+ VRF BLUE_SPOKE:
+ K>* 0.0.0.0/0 [255/8192] unreachable (ICMP unreachable), 03w0d23h
+ C>* 10.50.50.0/24 is directly connected, eth3, 03w0d23h
+ B> 10.80.80.0/24 [200/0] via 8.8.8.8 (vrf default) (recursive), label 80, weight 1, 04:22:00
+ * via 172.16.90.1, eth0 (vrf default), label 24/80, weight 1, 04:22:00
+ B>* 80.80.80.80/32 [20/0] via 10.50.50.2, eth3, weight 1, 6d05h30m
+ B> 100.100.100.100/32 [200/0] via 8.8.8.8 (vrf default) (recursive), label 80, weight 1, 04:22:00
+ * via 172.16.90.1, eth0 (vrf default), label 24/80, weight 1, 04:22:00
+
+- “show bgp ipv4 vpn x.x.x.x/32” for checking the best-path to the
+ specific VPNv4 destination including extended community and
+ remotelabel information. This procedure is the same on all Spoke nodes:
+
+.. code-block:: none
+
+ vyos@VyOS-PE1:~$ show bgp ipv4 vpn 100.100.100.100/32
+ BGP routing table entry for 10.80.80.1:1011:100.100.100.100/32
+ not allocated
+ Paths: (2 available, best #1)
+ Not advertised to any peer
+ 65035
+ 8.8.8.8 from 1.1.1.1 (8.8.8.8)
+ Origin incomplete, metric 0, localpref 100, valid, internal, best (Neighbor IP)
+ Extended Community: RT:65035:1030
+ Originator: 8.8.8.8, Cluster list: 1.1.1.1
+ Remote label: 80
+ Last update: Tue Oct 19 13:45:26 2021
+ 65035
+ 8.8.8.8 from 2.2.2.2 (8.8.8.8)
+ Origin incomplete, metric 0, localpref 100, valid, internal
+ Extended Community: RT:65035:1030
+ Originator: 8.8.8.8, Cluster list: 1.1.1.1
+ Remote label: 80
+ Last update: Wed Oct 13 12:39:34 202
+
+Now, let’s check routing information on out Hub PE:
+- “show bgp ipv4 vpn summary” for checking iBGP neighbors again
+ VyOS-RR1/RR2
+
+.. code-block:: none
+
+ vyos@VyOS-PE2:~$ show bgp ipv4 vpn summary
+ BGP router identifier 8.8.8.8, local AS number 65001 vrf-id 0
+ BGP table version 0
+ RIB entries 9, using 1728 bytes of memory
+ Peers 2, using 43 KiB of memory
+ Peer groups 1, using 64 bytes of memory
+
+ Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd PfxSnt
+ 1.1.1.1 4 65001 15982 15949 0 0 0 05:41:28 6 4
+ 2.2.2.2 4 65001 9060 9054 0 0 0 6d06h47m 6 4
+
+ Total number of neighbors
+
+- “show bgp vrf all” for checking all the prefixes learning on BGP
+
+.. code-block:: none
+
+ vyos@VyOS-PE2:~$ show bgp vrf all
+
+ Instance default:
+ No BGP prefixes displayed, 0 exist
+
+ Instance BLUE_HUB:
+ BGP table version is 50, local router ID is 10.80.80.1, vrf id 8
+ Default local pref 100, local AS 65001
+ Status codes: s suppressed, d damped, h history, * valid, > best, = multipath,
+ i internal, r RIB-failure, S Stale, R Removed
+ Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self
+ Origin codes: i - IGP, e - EGP, ? - incomplete
+
+ Network Next Hop Metric LocPrf Weight Path
+ *> 10.50.50.0/24 7.7.7.7@0< 0 100 0 i
+ * 7.7.7.7@0< 0 100 0 i
+ *> 10.60.60.0/24 10.10.10.10@0< 0 100 0 i
+ * 10.10.10.10@0< 0 100 0 i
+ * 10.80.80.0/24 10.80.80.2 0 0 65035 ?
+ * 0.0.0.0 0 32768 i
+ *> 0.0.0.0 0 32768 ?
+ *> 10.110.110.0/24 172.16.80.2@9< 0 0 65050 i
+ *> 10.210.210.0/24 9.9.9.9@0< 0 100 0 65050 i
+ * 9.9.9.9@0< 0 100 0 65050 i
+ *> 80.80.80.80/32 7.7.7.7@0< 0 100 0 65035 i
+ * 7.7.7.7@0< 0 100 0 65035 i
+ *> 90.90.90.90/32 10.10.10.10@0< 0 100 0 65035 i
+ * 10.10.10.10@0< 0 100 0 65035 i
+ *> 100.100.100.100/32
+ 10.80.80.2 0 0 65035 ?
+ *> 172.16.80.0/24 0.0.0.0@9< 0 32768 ?
+ 0.0.0.0@9< 0 32768 i
+ *> 172.16.100.0/24 9.9.9.9@0< 0 100 0 i
+ * 9.9.9.9@0< 0 100 0 i
+
+- “show bgp vrf BLUE_HUB summary” for checking EBGP neighbor
+ CE Hub device
+
+.. code-block:: none
+
+ vyos@VyOS-PE2:~$ show bgp vrf BLUE_HUB summary
+
+ IPv4 Unicast Summary:
+ BGP router identifier 10.80.80.1, local AS number 65001 vrf-id 8
+ BGP table version 50
+ RIB entries 19, using 3648 bytes of memory
+ Peers 1, using 21 KiB of memory
+
+ Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd PfxSnt
+ 10.80.80.2 4 65035 15954 15972 0 0 0 01w4d01h 2 10
+
+- “show ip route vrf BLUE_HUB” to view the RIB in our Hub PE.
+ With this command we are able to check the transport and
+ customer label (inner/outer) for network spokes prefixes
+ 80.80.80.80/32 - 90.90.90.90/32
+
+.. code-block:: none
+
+ vyos@VyOS-PE2:~$ show ip route vrf BLUE_HUB
+ Codes: K - kernel route, C - connected, S - static, R - RIP,
+ O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
+ T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
+ F - PBR, f - OpenFabric,
+ > - selected route, * - FIB route, q - queued, r - rejected, b - backup
+ VRF BLUE_HUB:
+ K>* 0.0.0.0/0 [255/8192] unreachable (ICMP unreachable), 01w4d01h
+ B> 10.50.50.0/24 [200/0] via 7.7.7.7 (vrf default) (recursive), label 144, weight 1, 05:53:15
+ * via 172.16.100.1, eth1 (vrf default), label 22/144, weight 1, 05:53:15
+ B> 10.60.60.0/24 [200/0] via 10.10.10.10 (vrf default) (recursive), label 144, weight 1, 05:53:15
+ * via 172.16.110.1, eth0 (vrf default), label 23/144, weight 1, 05:53:15
+ C>* 10.80.80.0/24 is directly connected, eth3, 01w4d01h
+ B>* 10.110.110.0/24 [200/0] via 172.16.80.2, eth2 (vrf GREEN), weight 1, 01w4d01h
+ B> 10.210.210.0/24 [200/0] via 9.9.9.9 (vrf default) (recursive), label 144, weight 1, 05:53:15
+ * via 172.16.100.1, eth1 (vrf default), label 18/144, weight 1, 05:53:15
+ * via 172.16.110.1, eth0 (vrf default), label 22/144, weight 1, 05:53:15
+ B> 80.80.80.80/32 [200/0] via 7.7.7.7 (vrf default) (recursive), label 144, weight 1, 05:53:15
+ * via 172.16.100.1, eth1 (vrf default), label 22/144, weight 1, 05:53:15
+ B> 90.90.90.90/32 [200/0] via 10.10.10.10 (vrf default) (recursive), label 144, weight 1, 05:53:15
+ * via 172.16.110.1, eth0 (vrf default), label 23/144, weight 1, 05:53:15
+ B>* 100.100.100.100/32 [20/0] via 10.80.80.2, eth3, weight 1, 01w4d01h
+ B>* 172.16.80.0/24 [200/0] is directly connected, eth2 (vrf GREEN), weight 1, 01w4d01h
+ B> 172.16.100.0/24 [200/0] via 9.9.9.9 (vrf default) (recursive), label 144, weight 1, 05:53:15
+ * via 172.16.100.1, eth1 (vrf default), label 18/144, weight 1, 05:53:15
+ * via 172.16.110.1, eth0 (vrf default), label 22/144, weight 1, 05:53:15
+
+- “show bgp ipv4 vpn x.x.x.x/32” for checking best-path,
+ extended community and remote label of specific destination
+
+.. code-block:: none
+
+ vyos@VyOS-PE2:~$ show bgp ipv4 vpn 80.80.80.80/32
+ BGP routing table entry for 10.50.50.1:1011:80.80.80.80/32
+ not allocated
+ Paths: (2 available, best #1)
+ Not advertised to any peer
+ 65035
+ 7.7.7.7 from 1.1.1.1 (7.7.7.7)
+ Origin IGP, metric 0, localpref 100, valid, internal, best (Neighbor IP)
+ Extended Community: RT:65035:1011
+ Originator: 7.7.7.7, Cluster list: 1.1.1.1
+ Remote label: 144
+ Last update: Tue Oct 19 13:45:30 2021
+ 65035
+ 7.7.7.7 from 2.2.2.2 (7.7.7.7)
+ Origin IGP, metric 0, localpref 100, valid, internal
+ Extended Community: RT:65035:1011
+ Originator: 7.7.7.7, Cluster list: 1.1.1.1
+ Remote label: 144
+ Last update: Wed Oct 13 12:39:37 2021
+
+ vyos@VyOS-PE2:~$ show bgp ipv4 vpn 90.90.90.90/32
+ BGP routing table entry for 10.60.60.1:1011:90.90.90.90/32
+ not allocated
+ Paths: (2 available, best #1)
+ Not advertised to any peer
+ 65035
+ 10.10.10.10 from 1.1.1.1 (10.10.10.10)
+ Origin IGP, metric 0, localpref 100, valid, internal, best (Neighbor IP)
+ Extended Community: RT:65035:1011
+ Originator: 10.10.10.10, Cluster list: 1.1.1.1
+ Remote label: 144
+ Last update: Tue Oct 19 13:45:30 2021
+ 65035
+ 10.10.10.10 from 2.2.2.2 (10.10.10.10)
+ Origin IGP, metric 0, localpref 100, valid, internal
+ Extended Community: RT:65035:1011
+ Originator: 10.10.10.10, Cluster list: 1.1.1.1
+ Remote label: 144
+ Last update: Wed Oct 13 12:45:44 2021
+
+Finally, let’s check the reachability between CEs:
+
+- VyOS-CE1-SPOKE -----> VyOS-CE-HUB
+
+
+.. code-block:: none
+
+ # check rib
+ vyos@VyOS-CE1-SPOKE:~$ show ip route
+ Codes: K - kernel route, C - connected, S - static, R - RIP,
+ O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
+ T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
+ F - PBR, f - OpenFabric,
+ > - selected route, * - FIB route, q - queued, r - rejected, b - backup
+
+ B 10.50.50.0/24 [20/0] via 10.50.50.1 inactive, weight 1, 6d07h53m
+ C>* 10.50.50.0/24 is directly connected, eth0, 09w0d00h
+ B>* 10.80.80.0/24 [20/0] via 10.50.50.1, eth0, weight 1, 6d07h53m
+ C>* 80.80.80.80/32 is directly connected, dum20, 09w0d00h
+ B>* 100.100.100.100/32 [20/0] via 10.50.50.1, eth0, weight 1, 6d07h53m
+
+ # check icmp
+ vyos@VyOS-CE1-SPOKE:~$ ping 100.100.100.100 interface 80.80.80.80
+ PING 100.100.100.100 (100.100.100.100) from 80.80.80.80 : 56(84) bytes of data.
+ 64 bytes from 100.100.100.100: icmp_seq=1 ttl=62 time=6.52 ms
+ 64 bytes from 100.100.100.100: icmp_seq=2 ttl=62 time=4.13 ms
+ 64 bytes from 100.100.100.100: icmp_seq=3 ttl=62 time=4.04 ms
+ 64 bytes from 100.100.100.100: icmp_seq=4 ttl=62 time=4.03 ms
+ ^C
+ --- 100.100.100.100 ping statistics ---
+ 4 packets transmitted, 4 received, 0% packet loss, time 8ms
+ rtt min/avg/max/mdev = 4.030/4.680/6.518/1.064 ms
+
+ # check network path
+ vyos@VyOS-CE1-SPOKE:~$ traceroute 100.100.100.100
+ traceroute to 100.100.100.100 (100.100.100.100), 30 hops max, 60 byte packets
+ 1 10.50.50.1 (10.50.50.1) 1.041 ms 1.252 ms 1.835 ms
+ 2 * * *
+ 3 100.100.100.100 (100.100.100.100) 9.225 ms 9.159 ms 9.121 m
+
+- VyOS-CE-HUB -------> VyOS-CE1-SPOKE
+- VyOS-CE-HUB -------> VyOS-CE2-SPOKE
+
+.. code-block:: none
+
+ # check rib
+ vyos@VyOS-CE-HUB:~$ show ip route
+ Codes: K - kernel route, C - connected, S - static, R - RIP,
+ O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
+ T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
+ F - PBR, f - OpenFabric,
+ > - selected route, * - FIB route, q - queued, r - rejected, b - backup
+
+ B>* 10.50.50.0/24 [20/0] via 10.80.80.1, eth0, weight 1, 6d08h04m
+ B>* 10.60.60.0/24 [20/0] via 10.80.80.1, eth0, weight 1, 6d08h35m
+ C>* 10.80.80.0/24 is directly connected, eth0, 01w6d07h
+ B>* 10.110.110.0/24 [20/0] via 10.80.80.1, eth0, weight 1, 01w4d02h
+ B>* 10.210.210.0/24 [20/0] via 10.80.80.1, eth0, weight 1, 6d08h35m
+ B>* 80.80.80.80/32 [20/0] via 10.80.80.1, eth0, weight 1, 6d08h04m
+ B>* 90.90.90.90/32 [20/0] via 10.80.80.1, eth0, weight 1, 6d08h35m
+ C>* 100.100.100.100/32 is directly connected, dum20, 01w6d07h
+ B>* 172.16.80.0/24 [20/0] via 10.80.80.1, eth0, weight 1, 01w4d02h
+ B>* 172.16.100.0/24 [20/0] via 10.80.80.1, eth0, weight 1, 6d08h35m
+
+ # check icmp
+ vyos@VyOS-CE-HUB:~$ ping 80.80.80.80 interface 100.100.100.100 c 4
+ PING 80.80.80.80 (80.80.80.80) from 100.100.100.100 : 56(84) bytes of data.
+ 64 bytes from 80.80.80.80: icmp_seq=1 ttl=62 time=3.31 ms
+ 64 bytes from 80.80.80.80: icmp_seq=2 ttl=62 time=4.23 ms
+ 64 bytes from 80.80.80.80: icmp_seq=3 ttl=62 time=3.89 ms
+ 64 bytes from 80.80.80.80: icmp_seq=4 ttl=62 time=3.22 ms
+
+ --- 80.80.80.80 ping statistics ---
+ 4 packets transmitted, 4 received, 0% packet loss, time 9ms
+ rtt min/avg/max/mdev = 3.218/3.661/4.226/0.421 ms
+
+ vyos@VyOS-CE-HUB:~$ ping 90.90.90.90 interface 100.100.100.100 c 4
+ PING 90.90.90.90 (90.90.90.90) from 100.100.100.100 : 56(84) bytes of data.
+ 64 bytes from 90.90.90.90: icmp_seq=1 ttl=62 time=7.46 ms
+ 64 bytes from 90.90.90.90: icmp_seq=2 ttl=62 time=4.43 ms
+ 64 bytes from 90.90.90.90: icmp_seq=3 ttl=62 time=4.60 ms
+ ^C
+ --- 90.90.90.90 ping statistics ---
+ 3 packets transmitted, 3 received, 0% packet loss, time 6ms
+ rtt min/avg/max/mdev = 4.430/5.498/7.463/1.391 ms
+
+ # check network path
+ vyos@VyOS-CE-HUB:~$ traceroute 80.80.80.80
+ traceroute to 80.80.80.80 (80.80.80.80), 30 hops max, 60 byte packets
+ 1 10.80.80.1 (10.80.80.1) 1.563 ms 1.341 ms 1.075 ms
+ 2 * * *
+ 3 80.80.80.80 (80.80.80.80) 8.125 ms 8.019 ms 7.781 ms
+
+ vyos@VyOS-CE-HUB:~$ traceroute 90.90.90.90
+ traceroute to 90.90.90.90 (90.90.90.90), 30 hops max, 60 byte packets
+ 1 10.80.80.1 (10.80.80.1) 1.305 ms 1.137 ms 1.097 ms
+ 2 * * *
+ 3 * * *
+ 4 90.90.90.90 (90.90.90.90) 9.358 ms 9.325 ms 9.292 ms
+
+- VyOS-CE2-SPOKE -------> VyOS-CE-HUB
+
+.. code-block:: none
+
+ # check rib
+ vyos@rt-ce2-SPOKE:~$ show ip route
+ Codes: K - kernel route, C - connected, S - static, R - RIP,
+ O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
+ T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
+ F - PBR, f - OpenFabric,
+ > - selected route, * - FIB route, q - queued, r - rejected, b - backup
+
+ B 10.60.60.0/24 [20/0] via 10.60.60.1 inactive, weight 1, 02w6d00h
+ C>* 10.60.60.0/24 is directly connected, eth0, 02w6d00h
+ B>* 10.80.80.0/24 [20/0] via 10.60.60.1, eth0, weight 1, 6d08h46m
+ C>* 90.90.90.90/32 is directly connected, dum20, 02w6d00h
+ B>* 100.100.100.100/32 [20/0] via 10.60.60.1, eth0, weight 1, 6d08h46m
+
+ # check icmp
+ vyos@rt-ce2-SPOKE:~$ ping 100.100.100.100 interface 90.90.90.90 c 4
+ PING 100.100.100.100 (100.100.100.100) from 90.90.90.90 : 56(84) bytes of data.
+ 64 bytes from 100.100.100.100: icmp_seq=1 ttl=62 time=4.97 ms
+ 64 bytes from 100.100.100.100: icmp_seq=2 ttl=62 time=4.45 ms
+ 64 bytes from 100.100.100.100: icmp_seq=3 ttl=62 time=4.20 ms
+ 64 bytes from 100.100.100.100: icmp_seq=4 ttl=62 time=4.29 ms
+
+ --- 100.100.100.100 ping statistics ---
+ 4 packets transmitted, 4 received, 0% packet loss, time 9ms
+ rtt min/avg/max/mdev = 4.201/4.476/4.971/0.309 ms
+
+ # check network path
+ vyos@rt-ce2-SPOKE:~$ traceroute 100.100.100.100
+ traceroute to 100.100.100.100 (100.100.100.100), 30 hops max, 60 byte packets
+ 1 10.60.60.1 (10.60.60.1) 1.343 ms 1.190 ms 1.152 ms
+ 2 * * *
+ 3 * * *
+ 4 100.100.100.100 (100.100.100.100) 7.504 ms 7.480 ms 7.488 ms
+
+**Note:** At the moment, trace mpls doesn’t show labels/paths. So we’ll see * * * for the transit routers of the mpls backbone.