There are plenty of documents online discussing Multipath BGP, but few seem to address the need for using eiBGP for load sharing (ie. eBGP and iBGP at the same time).

Use case with Routing Loops : Simple Multipath eiBGP without MPLS

Multipath eBGP is related to load balacing using routes learned from External BGP Peers, while iBGP is related to Internal BGP Peers.

Multipath eiBGP is load sharing using routes leaned from External and Internal BGP at the same time.

Let’s take the simple design below :

As we can see, in this design, let’s imagine we have two CE Routers and that we enabled multipath => there will be a routing loop.

Router A and B will both have two routes for a same destination :

• One route pointing (Next-Hop) to the External BGP
• One route pointing (Next-Hop) to the Internal BGP

Since they are pointing to each other, there will be some cases (depending on the ECMP hash function) where a routing loop will occur.

It’s also mentioned on the Cisco website, this risk exists, although the article discussing it has a typo regarding PE and CE :

https://www.cisco.com/c/en/us/td/docs/ios-xml/ios/iproute_bgp/configuration/xe-17/irg-xe-17-book/eibgp-multipath-for-non-vrf-interfaces-ipv-ipv.html

Some providers/companies use eiBGP only on the “Primary” CE router especialy when they have an active/passive multihomed design, which ensures that they will benefit from the backup CE WAN link. However, they also accept that when an incident occurs, the service may be degraded depending on the bandwidth usage of both links (which is where QoS may comes into play).

Note : When enabling multipath eiBGP, behind the scenes, both routes are installed in the routing table with BGP AD 20, even if they are learned through iBGP. This seems to be the way it has been implemented on Cisco IOS (it is a trick to prevent the router from preferring iBGP over eBGP).

RP/0/RP0/CPU0:PE1#show route vrf ABC 10.24.24.0/24 Routing entry for 10.24.24.0/24 Known via “bgp 1234”, distance 20, metric 0 type internal and external Routing Descriptor Blocks 10.11.11.11, from 10.11.11.11, BGP multi path Nexthop in Vrf: “default”, Table: “default”, IPv4 Unicast, Table Id: 0xe0000000 Route metric is 0 10.22.22.22, from 10.22.22.22, BGP external, BGP multi path Route metric is 0

Use case without Routing Loops : eiBGP With MPLS

Generaly speaking, service provider does not need to enable eiBGP feature on an MPLS VPN Network because  PE to PE communication will work with ECMP already (Provider will in most case uses different RD on each PE so it can load balance trafic for Multihomed Custumer site using Multipath iBGP)

But there is some cases were this is necessary for example when one PE connect two Sites (and so different CE), here, the trafic will follow the eBGP path because both sites (CE) are directly connected to the same PE.

eiBGP function make more sense in this scenario because it has been designed for an MPLS deploiement in the first place.

Let’s suppose that we have two routers with MPLS enabled between them and we want to enable eiBGP. In that specific case, we will be able to do so on both routers without risking any routing loops. Why? Because when MPLS is enabled on both routers, the router does a label lookup only and eBGP multipath does not cause loops.

Packets sent from the PE router on an eBGP path leave as IP packets, and packets sent on an iBGP path leave as MPLS packets.

For instance, below we see that when a packet is received with label 57635 from PE1, then PE2 will not check it’s routing table, instead it will check the label, decide to remove the label and send it using the interface Te/02 and next-hop “10.123.123.1”.

RP/0/RP0/CPU0:PE2_ROUTER#show mpls forwarding labels 57635 Local Outgoing Prefix Outgoing Next Hop Bytes Label Label or ID Interface Switched —— ———– —————— ———— ————— ———— 57635 Unlabelled 10.1.1.0/24[V] Te0/2 10.123.123.1 63819

Of course, it only make sense to implement this feature if the iBGP link capacity (Back to Back) is equal or greater than the the WAN Link + you have to take into consideration the latency between PE-PE link if it is significant.

Another example, is a company that might have an MPLS Core Routers on which they may be Multihomed to some service providers (or others WANs) CE, and this is where it can make sense to enable this feature

In summary :

• The eiBGP feature operates only when routers have the MPLS Label allocation set to “Per Prefix” (or “Per CE/NH when available) and not “Per VRF”, as with the latter, the incoming label cannot uniquely identify the outgoing interface.
  • Per-prefix and per-CE modes perform a single lookup for arriving packets. The LFIB entry directly references the next-hop, along with the egress interface and L2 rewrite information.
  • In per-VRF mode, the LFIB entry references the VRF, requiring an additional lookup on the IP packet to determine the next-hop, which may lead to forwarding loops.
• If you have a deployment with MPLS (remember you can enable MPLS without VPN/VRF), you can safely use the eiBGP feature to benefit from the load balancing mechanism depending on your design (especially if you have a PE-PE link with iBGP VPNv4 peers with multiples peerings).
• If you have a simple multihomed CE-CE deploiement, you have to asset if it is worth it to enable MPLS/VPNv4 between them (may be they are not MPLS capable, so do you need to change the HW, buy a new licence…). Or you may choose to enable eiBGP multipath only on one on them at least for the primary router.
• Adding a Layer 3 layer device upstream in the sandwitch can also be a solution to do the load balancing
• Remember that to  enable Multipath in iBGP, the following condition must be met “The IGP metric to the BGP next hop must be equal to the best-path IGP metric, unless the router is configured for unequal-cost iBGP multipath”
  • Well, you don’t have to care about “unequal cost” to the Next-Hop when you enable the eiBGP multipath, the “unequal-cost” to the next-hop will be enabled by default with eiBGP (even if it is not seen in the configuration)
• When enabling Multipath you will generaly see the benefits from it, but keep in mind that it will depend on the hash algorithm and in general for the same “flow/state” (same IP SRC/DST Port SRC/DST) the same path will be used, so all tunnels, big and long backups will not be load balanced
• Another solution to use “without MPLS” is to apply a policy-based routing (PBR) on the CE-CE link with some conditions, and set the next-hop to the eBGP peer in order to avoid any loop
• You also may be tempted to think about tweaking the BGP AS (with local-as) and enable multipath without the eiBGP feature, but I will pass 🙂

Have you implemented eBGP yet? Are there any other scenarios besides what we have presented that you can think of? We would be interested in hearing your thoughts on this, you can leave a comment below.

Are you considering implementing Multipath Routing in your network and have questions about it? Alternatively, do you want to optimize your network’s bandwidth by enabling load balancing? If so, we can assist you. Please don’t hesitate to reach out to us through the CONTACT-US page or by emailing us at [email protected] for advise assistance.

Mehdi SFAR (CCDE 2021:3, CCIE #51583)