Routing Security

There is Still Hope for BGP Route Leak Prevention

  • By Mingwei Zhang
  • 26 Apr 2023
BGP route leak

Route leaks occur when BGP prefixes are propagated in a way that goes against the expected topology relationships of BGP. For example, this can happen when a route learned from one transit provider is announced to another transit provider or a lateral peer (peer-peer-peer), or when a route learned from one lateral peer is announced to another lateral peer or a transit provider (see RFC7908). These leaks often result from a misconfiguration or the absence of BGP route filtering, or inadequate coordination between autonomous systems (ASes).

I have worked closely on detecting route leaks and we shipped a route-leak detection system under the Cloudflare Radar platform for free public consumption. In short, the system detects potential route leaks by first inferring the inter-AS relationships on a per-prefix-basis and examines each announced AS path to detect valley-free violations. However, detecting route leaks can only help operators so much; prevention should be the ultimate solution.

RFC9234

RFC9234 documents an active route-leak prevention approach that defines new BGP capacities (BGP Roles) exchanged during the eBGP session open time and allows the BGP routers to understand AS relationships between local and remote ASes, and thus prevent the propagation of route leaks. RFC9234 also defines a new BGP attributes type, only-to-customer, which tells the receiving BGP routers whether some routes should never be announced to another provider.

When a pair of eBGP routers both implement RFC9234, they will first confirm the BGP role of each other.

Local AS RoleRemote AS Role
ProviderCustomer
CustomerProvider
RSRS-Client
RS-ClientRS
PeerPeer
Table 1 — Allowed pairs of role capabilities. (Adapted from RFC9234)

With roles defined, it handles the only-to-customer attributes as follows:

The following ingress procedure applies to the processing of the OTC
   Attribute on route receipt:

   1.  If a route with the OTC Attribute is received from a Customer or
       an RS-Client, then it is a route leak and MUST be considered
       ineligible (see Section 3).

   2.  If a route with the OTC Attribute is received from a Peer (i.e.,
       remote AS with a Peer Role) and the Attribute has a value that is
       not equal to the remote (i.e., Peer's) AS number, then it is a
       route leak and MUST be considered ineligible.

   3.  If a route is received from a Provider, a Peer, or an RS and the
       OTC Attribute is not present, then it MUST be added with a value
       equal to the AS number of the remote AS.

   The following egress procedure applies to the processing of the OTC
   Attribute on route advertisement:

   1.  If a route is to be advertised to a Customer, a Peer, or an RS-
       Client (when the sender is an RS), and the OTC Attribute is not
       present, then when advertising the route, an OTC Attribute MUST
       be added with a value equal to the AS number of the local AS.

   2.  If a route already contains the OTC Attribute, it MUST NOT be
       propagated to Providers, Peers, or RSes.

In essence, RFC9234 uses BGP roles and OTC attributes to make sure routes received from a provider or a peer can only be propagated to customers.

Only-to-customer Attribute in the Wild

RFC9234 graduated as an official RFC in May 2022, almost one year ago. There have not been many discussions about the deployment of RFC9234, not to mention actual measurements of how widely it is deployed on the Internet.

To observe the deployment of RFC9234, I added the support for RFC9234 to the BGPKIT parser a while back and wrote a quick example code to demonstrate how to parse a RIB file and look for messages that contain only-to-customer attributes.

The following is what I saw from parsing a single RIB dump file from route-views2 RIB dump (file link).

OTC found: 15562 for path 23673:6939:15562
A|1678932008|203.189.128.233|23673|67.221.245.0/24|23673 6939 15562|IGP|203.189.128.233|0|0||NAG||

OTC found: 15562 for path 293:6939:15562
A|1678932008|198.129.33.85|293|67.221.245.0/24|293 6939 15562|IGP|198.129.33.85|0|710||NAG||

OTC found: 15562 for path 11686:6939:15562
A|1678932008|96.4.0.55|11686|67.221.245.0/24|11686 6939 15562|IGP|96.4.0.55|0|0|11686:195|NAG||

OTC found: 15562 for path 18106:6939:15562
A|1678932008|202.73.40.45|18106|67.221.245.0/24|18106 6939 15562|IGP|202.73.40.45|0|0|6939:2000|NAG||

OTC found: 15562 for path 34224:6939:15562
A|1678932008|94.156.252.18|34224|67.221.245.0/24|34224 6939 15562|IGP|94.156.252.18|0|0|34224:333|NAG||

OTC found: 15562 for path 20130:6939:15562
A|1678932008|140.192.8.16|20130|67.221.245.0/24|20130 6939 15562|IGP|140.192.8.16|0|0||NAG||

OTC found: 15562 for path 6939:15562
A|1678932008|64.71.137.241|6939|67.221.245.0/24|6939 15562|IGP|64.71.137.241|0|0||NAG||

OTC found: 212068 for path 11686:6939:212068
A|1678932020|96.4.0.55|11686|139.28.205.0/24|11686 6939 212068|IGP|96.4.0.55|0|0|11686:195|NAG||

OTC found: 212068 for path 23673:6939:212068
A|1678932020|203.189.128.233|23673|139.28.205.0/24|23673 6939 212068|IGP|203.189.128.233|0|0||NAG||

OTC found: 212068 for path 6939:212068
A|1678932020|64.71.137.241|6939|139.28.205.0/24|6939 212068|IGP|64.71.137.241|0|0||NAG||

OTC found: 212068 for path 20130:6939:212068
A|1678932020|140.192.8.16|20130|139.28.205.0/24|20130 6939 212068|IGP|140.192.8.16|0|0||NAG||

OTC found: 20555 for path 6939:20555
A|1678932039|64.71.137.241|6939|213.135.44.0/22|6939 20555|IGP|64.71.137.241|0|0||NAG||

OTC found: 20555 for path 11686:6939:20555
A|1678932039|96.4.0.55|11686|213.135.44.0/22|11686 6939 20555|IGP|96.4.0.55|0|0|11686:195|NAG||

OTC found: 20555 for path 23673:6939:20555
A|1678932039|203.189.128.233|23673|213.135.44.0/22|23673 6939 20555|IGP|203.189.128.233|0|0||NAG||

OTC found: 20555 for path 20130:6939:20555
A|1678932039|140.192.8.16|20130|213.135.44.0/22|20130 6939 20555|IGP|140.192.8.16|0|0||NAG||

OTC found: 20555 for path 6939:20555
A|1678932039|64.71.137.241|6939|213.135.48.0/23|6939 20555|IGP|64.71.137.241|0|0||NAG||

OTC found: 20555 for path 11686:6939:20555
A|1678932039|96.4.0.55|11686|213.135.48.0/23|11686 6939 20555|IGP|96.4.0.55|0|0|11686:195|NAG||

OTC found: 20555 for path 23673:6939:20555
A|1678932039|203.189.128.233|23673|213.135.48.0/23|23673 6939 20555|IGP|203.189.128.233|0|0||NAG||

OTC found: 20555 for path 20130:6939:20555
A|1678932039|140.192.8.16|20130|213.135.48.0/23|20130 6939 20555|IGP|140.192.8.16|0|0||NAG||
| asn    | as_name   | org_name                                           | org_country |
|--------|-----------|----------------------------------------------------|-------------|
| 6939   | HURRICANE | Hurricane Electric LLC                             | US          |
| 15562  | SNIJDERS  | Job Snijders                                       | NL          |
| 20555  | WSISIZ-AS | Wyzsza Szkola Informatyki Stosowanej i Zarzadzania | PL          |
| 212068 | SHINRA-AS | Shinra Electric Power Company Limited              | GB          |

From this output, we can see that at least 4 ASes (AS6939, AS15562, AS20555, AS212068) implement RFC9234 and the messages containing OTC attributes propagated to the route collector and got preserved into MRT files.

The code that generates this result is also very simple. It’s pretty much just a loop of messages and checking if the only_to_customer attribute is present in the message.

use bgpkit_parser::BgpkitParser;

fn main() {
    for elem in BgpkitParser::new(
        "http://archive.routeviews.org/bgpdata/2023.03/RIBS/rib.20230316.0200.bz2",
    )
    .unwrap()
    {
        if let Some(otc) = elem.only_to_customer {
            println!(
                "OTC found: {} for path {}\n{}\n",
                &otc,
                &elem.as_path.as_ref().unwrap(),
                &elem
            );
        }
    }
}

If you would like to try out this example yourself, you can run the following command (assuming you have Rust toolchain installed):

git clone https://github.com/bgpkit/bgpkit-parser
cd bgpkit-parser
cargo run --release --example only-to-customer

Finally, thanks to Job Snijders for shedding more light on the source of OTC attributes—I did not know YYCIX before, pretty cool exchange point!

Mingwei Zhang is a Senior Systems Engineer at Cloudflare.

Adapted from repost which first appeared on Request for Commits

Related Content

Leave a Comment