Gandi Kitchen

Home > Network > When Null0 and BGP May Cause Problems

When Null0 and BGP May Cause Problems

Let us take the following simplified pair of autonomous systems as an example. On the left, AS 1 and on the right AS 2;  These two networks have connectivity to the internet as well as a direct peering relation between them.  Under normal circumstances, the traffic between the client in AS1 and the server in AS2 will follow the shortest AS-Path which in this case will be over the peering session.

In this example, the routers have static routes to null0 to 'pin' up the prefixes that they advertise in BGP for their respective ASNs.  The only significant difference in this example between the two networks, is that the connectivity between R2 in AS1 and the rest of the AS1 network has a single connection.  After all, it is just a peering router, and less critical than central core and internet transit, right?  Well, let's have a look for a moment.

Let us imagine that the connectivity between R2 and the rest of the AS1 network is severed.  The routes to the network in AS 2 will then only be seen via the internet across the transit connection, as seen in the BGP advertisements.  Under correct configuration, traffic between the two networks should now be flowing via the Internet across their respective transit connections, as in the following diagram.

In this example, however, R2's connection to the peering session between the two ASNs is still active, and as such the BGP session is still active between the two.  Since R2 has statically 'pinned' the network prefixes via a route to null0, these prefixes are still advertised to R3 in AS 2.  

As a result, the return traffic, following the shortest AS-path, will be sent via R3 to R2.  Unfortunately, due to the break in connectivity between R2 and the rest of its network, the client destination is unreachable, resulting in a partial "black hole" instigated by the break in connectivity and exacerbated by what is likely to be either an administrative "oversight", or a lack of understanding of the network topology at the time of configuration.

So how can we mitigate this problem?

There are three primary ways in which this problem can be mitigated.  The first method would involve adding a separate redundant connection from R2 to the rest of the AS 1 network (similar to that visible on R3 in AS 2 in the above diagrams), thereby mitigating (or at least reducing, while not completely eliminating) the risk of R2 becoming isolated and generating the partial black hole in the first place.  Depending on the distance and geographic location of R2, however, this may prove to be costly.

An alternative would be to tie the aggregated prefixes only in the core (which itself should already have redundant connectivity between the critical equipment), and advertise these with an internal routing protocol, such as OSPF or IS-IS, to the rest of the the border devices.  

As a result here, if R2 loses connectivity to the rest of the AS1 network, it no longer receives the internal protocol advertisements for the aggregated prefixes, and since these prefixes are no longer in the routing table of R2, it will cease to advertise them to any peers in BGP.  As a result, the return traffic from AS 2 to AS 1 would follow the only remaining path via the internet across the transit connections.

The third alternative involves more complex configuration, and of course assumes a dynamic internal routing protocol within the AS.  Under this scenario, the BGP advertisements would be conditioned on the existence (or non-existence, according to preferences) of another prefix in the routing table; this other prefix being received via the internal routing protocol.  This is sometimes referred to as "BGP Conditional Route Injection".  On Cisco equipment, this involves using advertise-maps within the BGP configuration, and there is documentation on this available on Cisco's website.

Aside from these methods, a fourth possibility would be to bind the static routes to null0 against a track object which in turn is associated with an IP SLA monitor.  For example, the router can be configured to regularly poll an IP address elsewhere within the network either via PING or another protocol.  By binding the this IP SLA monitor and track object against the static route to null0, this static route will remain in the routing table only if the IP SLA monitor returns successful.  (Thresholds may be configured as well to guard against transient false negatives...)  If the SLA monitor fails, then the route is withdrawn from the routing table, and thereby suppressing the prefix from being advertised in BGP.