Campus Core

The first step is to create the OSPF process for each VRF tenant. You could also create a single OSPF process then nest the tenants under it. But I like to have a separate process per tenant. Let’s configure Leaf-01 first (see Listing 5-1) and then Leaf-02 (see Listing 5-2) .

After validating that all the required configurations are in place, let’s confirm if you can see OSPF adjacency for both tenants on both leafs facing the core switch (see Figures 5-2, 5-3, 5-4, and 5-5).

You can confirm that the tenants on both Leaf-01 and Leaf-02 have OSPF adjacency to the campus core.

Interface Redistribution to OSPF

Let’s confirm that the OSPF routes were built at the campus core switch from the VXLAN tenants.

Figure 5-6 shows the campus core route table.

The next step is to filter any tenant-A and tenant-B routes from being advertised between each other by using the campus core. Let’s do some route filtering to accomplish that. Let’s look at Leaf-01 and Leaf-02. The routes from tenant-B are shown in tenant-A and vice versa.

Figure 5-7 shows Leaf-01 tenant-A and tenant-B .

The tenants can reach each other through the campus core switch. You need to avoid the tenant’s from seeing each other. The campus core network is only allowed to reach both tenants.

Let’s now add a route filter on Leaf-01 to filter incoming OSPF routes from tenants leaked in OSPF via the campus core. In summary, you create a route map with the “deny” action, and then match all the interfaces participating in the OSPF process.

OSPF Inbound Route Filtering Configuration

Figure 5-8 shows the post-change result on Leaf-01 in tenant-A and tenant-B.

You no longer see routes advertised from the campus core to the tenants. You need to be aware of an active OSPF peering to both Leaf-01 and Leaf-02. To avoid any asymmetric routing issues, you must make sure that only one OSPF path is active at a time. How do you accomplish this? First, you need to identify which leaf becomes the active path for the campus core. I chose Leaf-01 to become the active OSPF path to the campus network. On the campus core side, you need to configure OSPF costing to influence routes to be preferred to either OSPF neighbor, in this case, Leaf-01. Do a simple OSPF costing configuration that is lower on the VLAN interfaces peering to Leaf-01. VLANs 101 and 201 are the designated OSPF peering interfaces between Leaf-01 and the campus core. You’ll see more of this in the next section.

Adding the static OSPF cost values, you now confirm that only one active OSPF route is active for all advertised tenant networks. In my configuration, I set the lowest cost value possible to elect Leaf-01 as the primary path. If Leaf-01 goes down, Leaf-02 starts advertising its path. Let’s verify (see Figure 5-10).

As you can see, there is one active route path via Leaf-01. Asymmetric routing issues are very common to face in VXLAN spines and leafs since the route paths are more elastic. I can route in many directions, but I still need to make sure that the traffic flow remains symmetric. In a nutshell, you send traffic one way, but the return traffic has a better path advertised via another leaf or so. This causes your asymmetric route flow issue.

A better way to explain it is by describing an asymmetric and symmetric flow sample. The diagrams presented in Figure 5-11 and Figure 5-12 are two examples of fabric efficiency. The diagram shown in Figure 5-11 is the asymmetric route issue using a firewall between the two tenants. Figure 5-12 is a symmetric route flow sample with the same scenario but traffic correctly pinned in one direction.

Asymmetric Route Issues in VXLAN BGP EVPN

Figure 5-11 describes each hop between a client tenant PC trying to communicate to the web servers tenant. By following each hop in the diagram, you see that the return traffic does not flow using the same path as the source request traffic from the client PC.

Symmetric Routing in VXLAN BGP EVPN

By confirming a symmetric route path, you guarantee that all requests and replies follow the same route path. In the diagram shown in Figure 5-12, you see the same scenario covered in the asymmetric route discussion. The difference is that the server traffic returns to the client using the same route path. How do you accomplish this? By making sure that you only configure a lower cost or distance in the dynamic route protocol, you are implementing so the traffic returns in the same direction it came from.

Routes Before Changes

Figure 5-13 illustrates tenant-A VRF on Leaf-01.

Figure 5-14 illustrates tenant-A VRF on Leaf-02.

In Leaf-02, the preferred path is still local to the campus core. You want to re-route the traffic from all leafs to prefer Leaf-01 then Leaf-01 forward to the campus core. You need to fix this by redistributing OSPF to BGP and tweaking the BGP distance, so it is preferred via Leaf-01 using EVPN type-5 route advertisements.

You have accomplished a few things. First, you redistributed all OSPF routes learned in Leaf-01 to the VXLAN fabric via BGP EVPN. Second, you modified the iBGP distance in Leaf-02, so routes advertised in BGP are preferred over OSPF. OSPF’s AD is 110, and BGP is 200. I’ve lowered the BGP distance to 90, so it is preferred over OSPF. Let’s validate route 10.100.10.0/24 in Leaf-02 (see Figure 5-15).

Let’s apply it on Leaf-03 and confirm that the distance change is now reflected on the tenant-A VRF routes (see Figure 5-16).

Tenant: Italy

You need to perform this configuration to receive the MPLS subnets on the Italy tenant VRF route table. The ISP informed that if you peer to them using private AS numbers, you won’t have an AS path or network reachability since those AS numbers are already in use in other locations.

Now that you have all the information you need, let’s enable BGP and OSPF under the tenants and allow connectivity to the external networks. Let’s begin configuring the connection for Italy’s tenant. Then you do the same for France. Please refer to the diagram in Figure 5-17 for the logical connectivity to the external sites.

When you enable tenants with dynamic routing protocols, you need to designate either an existing routing process or create a new one per tenant. I prefer to work with independent routing processes because if there is an issue with one process, the other tenants aren’t impacted.

You already have BGP running for the VXLAN EVPN address family. You need to configure a BGP IPv4 address family for the Italy tenant. You need to add a unique BGP AS number for the Italy tenant. You need to use the same AS number to peer the tenants. That might be an issue since you don’t control what tenants assign to their BGP community from AS number perspective. Instead, allow them as or override the AS number, so you don’t have conflicts with any other possible BGP neighbor in the AS path using the same number.

Enabling BGP for a Tenant VRF

To enable BGP in a VRF, you need to add the VRF under the global BGP router process. Add the BGP neighbor and inform BGP to use an interface to source the hello message. You want to use the layer 3 interface that faces the MPLS router. Next, enable the AFI that you are working with. In our case, it is IPv4 unicast. With the information provided by the service provider, you need to establish adjacency to the MPLS router, which is connected to DC1-N9500-Leaf-01. Let’s use the information provided and perform the configurations.

Listing 5-9 shows the Italy tenant BGP configuration on Leaf-01.

int eth1/6

no switchport

vrf member Italy

ip address 145.11.24.26/30

no shut

Router BGP 65501

  vrf Italy

    router-id 145.11.24.26

    address-family ipv4 unicast

    neighbor 145.11.24.25

      remote-as 43321

      local-as 54423

      update-source Ethernet1/6

      address-family ipv4 unicast

Listing 5-9

Italy Tenant BGP Configuration on Leaf-01

Now let’s do the verifications. You want to first confirm that there is a BGP neighbor adjacency with the MPLS router. To perform this check, type in Leaf-01, as follows. It is illustrated it Figure 5-18.

show bgp ipv4 unicast summary vrf Italy

You can confirm adjacency to the MPLS network. The show command output indicates a neighbor of 145.11.24.25, which is the BGP router ID for the MPLS router. You also have received three prefixes if you look at the State/PfxRcd column. That should give us an indication that the remote networks are advertised on the Italy VRF route table. Let’s confirm with the following command. It is illustrated it Figure 5-19.

 show ip route vrf Italy

The MPLS routes are advertised in the Italy tenant VRF, but you need to make sure that the routes are available across all leafs. If the L3VNI is properly configured and assigned to the Italy tenant, you should have the routes across the VXLAN fabric. Let’s confirm if you see the routes in Leaf-03 (see Figure 5-20).

Amazing. EVPN is doing its job. The MPLS networks are distributed in L3VNI to the other leafs. The next hop is the VTEP IP for Leaf-01.

Enabling OSPF for a Tenant VRF

Enabling OSPF for a tenant VRF is quite similar to BGP. The only difference with OSPF is that you can create a new OSPF router process and assign it to the tenant. In BGP, you only run a single BGP process. Once you create the OSPF process, you add the tenant VRF under it and configure your OSPF router ID. Here’s the catch. With BGP, you don’t have to redistribute the external networks so EVPN can re-advertise to all my leafs. In OSPF, you need to perform redistribution to announce the networks to the tenant in all leafs. Okay, let’s tackle the request to peer OSPF for the France tenant.

The diagram in Figure 5-21 should help you understand the connectivity layout for both sites. Site 1 router is homed to Leaf-01 via eth1/5. The site 2 router is homed to Leaf-02 via eth1/5. To be successful, all fabric leafs need to have advertised site 1 and site 2 networks in the France tenant VRF.

Tenant: France

Let’s apply the configurations to both Leaf-01 and Leaf-02 and verify if you are successfully receiving routes from site 1 and site 2. Let’s confirm that you see the routes in the border leaf by using the following command. Then you should also see all the other leafs via EVPN (see Figure 5-22).

show ip ospf neighbors vrf France

You have established full OSPF adjacency to site 1’s router 172.16.1.1. Now let’s confirm if the routes were received from the site 1 router in the France tenant on Leaf-01 using the following command (see Figure 5-23).

show ip route vrf France

You learned the routes from site 1 in the France tenant on the border leaf. Now you also performed a route redistribution from OSPF to BGP. You should have the routes learned fabric wide in the France tenant. Let’s verify on Leaf-03 to confirm if this is the case using the following command (see Figure 5-24).

show ip route vrf France

Now that you applied the configurations on Leaf-02, let’s confirm if you are receiving the routes from site 2. But first, using the following command, let’s make sure that there is OSPF adjacency to site 2 (see Figure 5-25).

show ip ospf neighbors vrf France

After confirming OSPF adjacency to site 2 via Leaf-02 on the France tenant, let’s verify if the routes were received from site 2 by using the following command (see Figure 5-26).

show ip route ospf-France vrf France

Yes, routes are being received from site 2 to the France tenant. Finally, since you performed route redistribution from OSPF to BGP, let’s confirm that other leafs have received the routes via Leaf-02 L3VNI in EVPN.

Let’s confirm in Leaf-03 (see Figure 5-27).

Excellent! You can confirm that you have the routes from site 1 via EVPN to the L3VNI for Leaf-01 (10.10.10.10) and the routes from site 2 via EVPN to for Leaf-02’s L3VNI. The France tenant has end-to-end reachability to both sites across the entire VXLAN fabric. Success!