MANRS Implementation Guide


4.4. Filtering – Preventing propagation of incorrect routing information

Relevant MANRS expected actions:

  • Network operator defines a clear routing policy and implements a system that ensures correctness of their own announcements and announcements from their customers to adjacent networks with prefix and AS-path granularity.
  • Network operator applies due diligence when checking the correctness of its customer’s announcements, specifically that the customer legitimately holds the ASN and the address space it announces.

Most important is to secure the inbound routing advertisements, particularly from customer networks, through the use of explicit prefix-level filters or equivalent mechanisms.

Secondarily, AS-path filters might be used to require that the customer network be explicit about which Autonomous Systems (ASes) are downstream of that customer. Alternately, AS-path filters that block announcements by customers of ASes with which the provider has a settlement-free relationship can prevent some types of routing “leaks”.

Keep in mind that a common error is a typo in the announced IP addresses, which causes the wrong addresses to be announced from an allowed ASN. Filtering customer BGP announcements by AS-path filters alone is therefore insufficient to prevent catastrophic routing problems at a systemic level.

Similarly, filtering outbound announcements towards non-customer peers alone is insufficient where the operator provides services to multi-homed customers. Consider for example the scenario where a customer advertises a set of longer prefixes to only one of several transit providers: those longer prefixes will be learnt by that customer’s other providers via settlement- free peering or from their own transits, but since the prefix belongs to a valid customer, the prefixes will not be filtered outbound towards non-customers if those filters are based only on prefix matching. The operator in question will therefore advertise transit reachability to those more specific prefixes even though their customer never announced those directly to them. Operators should only announce transit for the prefixes that their customer announced directly to them, not prefixes learned through other paths.

Before building filters it is important to apply due diligence and check whether the information provided by the customer about their identity and resources is correct. Filters can check whether AS64501 is allowed to announce 192.0.2.0/24, but only verification of identity can determine whether your customer is really the holder of AS64501.

There are several ways to build these filter

  1. Use Internet Routing Registries (IRRs) and require the customers to register route objects
  2. Use Resource Public Key Infrastructure (RPKI) and require the customers to create Route Origin Authorizations (ROAs)
  3. Use an internal database with the information provided as part of the provisioning process

This document will only focus on the first two cases, since case 3 is proprietary.

In general, an observed prefix in the routing system may be partially validated as originating in the correct origin ASN (i.e. against only the rightmost ASN in the AS_PATH attribute), or fully validated against the entire ordered set of ASNs in the AS_PATH. These are referred to respectively as origin validation and path validation. While origin and path validation are related concepts, and may be performed through the use of an overlapping set of tools, they differ significantly with respect to the authoritative source of validation information as described below, and thus in the information that needs to be maintained by a MANRS participant.

4.4.1. Using an IRR and require the customers to register route objects

Internet Routing Registries are central places where routing information is published. They document which ASNs are allowed to announce which IP addresses and what the policies are for exchanging routes between ASNs. This information can be used in router configurations to validate the received routes.

Note that not all IRRs are authenticated against authoritative sources. The AFRINIC, APNIC and RIPE NCC IRRs only allow users to enter resource records that match the resources they have allocated. Other IRRs may have less strict authentication mechanisms to prevent their users from entering fake data.

4.4.1.1. Using the IRR to produce prefix filters

For the purposes of this section, we assume that this refers to prefix-filters used in the following circumstances (a non-exhaustive list):

  1. Specific-prefix outbound filtering of your network to peers and upstreams (required)
  2. Specific-prefix inbound filtering from customers (required).
  3. Specific-prefix Inbound filtering of peers to your network (recommended).

The authors note that (1) should be achievable, just given the operator’s own records.
(2) is desirable and should be trivially synthesizable from the operator’s AS-SET in the IRR (assuming registrations of customer networks are done in the IRR) and (3) is generally seen as a good thing, if achievable given the size and complexity of the registration of the networks of the peer.

Whether (3) is performed or not it should be the case that unwanted networks (such as BOGON networks, and the operator’s own prefixes) are filtered at all times.

It should be noted that a fourth scenario, specific inbound prefix filtering of full or partial upstream (for those relying on 3rd party connectivity) is out of scope of this document (but the point concerning BOGON filtering made above for (3) should be considered for all connections nonetheless)

In a typical scenario an operator will require their customers to register their expected announcements as route objects in a selected IRR. For the example network topology on fig.1 the network AS64500 will ask AS64501 to register the following objects:


    route:            192.0.2.0/24
    descr:            Cust 64501
    origin:           AS64501
    mnt-by:           MAINT-AS64501
    created:          2015-09-27T12:14:23Z
    last-modified:    2015-09-27T12:14:23Z
    source:           RIPE

    route6:           2001:db8:1001::/48
    descr:            Cust 64501
    origin:           AS64501 
    mnt-by:           MAINT-AS64501
    created:          2015-09-27T12:14:23Z
    last-modified:    2015-09-27T12:14:23Z
    source:           RIPE

And similar objects for their other customer – AS64502.

AS64500 itself will register the following objects:


    route:            203.0.113.0/24
    descr:            Provider 64500
    origin:           AS64500
    mnt-by:           MAINT-AS64500
    created:          2012-10-27T12:14:23Z
    last-modified:    2016-02-27T12:33:15Z
    source:           RIPE

    route6:           2001:db8:1000::/36
    descr:            Provider 64500
    origin:           AS64500
    mnt-by:           MAINT-AS64500
    created:          2012-10-27T12:14:23Z
    last-modified:    2016-02-27T12:33:15Z
    source:           RIPE

    as-set:           AS64500:AS-CUSTOMERS
    members:          AS64501
    members:          AS64502
    mnt-by:           MAINT-AS64500
    created:          2012-10-27T12:14:23Z
    last-modified:    2016-02-27T12:33:15Z
    source:           RIPE

    as-set:           AS64500:AS-ALL
    members:          AS64500
    members:          AS64500:AS-CUSTOMERS
    mnt-by:           MAINT-AS64500
    created:          2012-10-27T12:14:23Z
    last-modified:    2016-02-27T12:33:15Z
    source:           RIPE

    aut-num:          AS64500
    descr:            Provider 64500
    mp-import:        from AS64500:AS-CUSTOMERS
                      accept PeerAS AND <^PeerAS++$>
    mp-export:        to AS64500:AS-CUSTOMERS announce ANY
    mp-import:        from AS64510 accept ANY except FLTR-BOGONS
    mp-export:        to AS64510 announce AS64500:AS-ALL 
    mnt-by:           MAINT-AS64500
    created:          2012-10-27T12:14:23Z
    last-modified:    2016-02-27T12:33:15Z
    source:           RIPE 

Now, a clever tool could parse the aut-num object that documents the AS64500 policy, collect all referenced objects, extract prefixes and build necessary ingress and egress filters.

Examples of software to generate router configuration for filters from IRR data, are the IRRToolset, BGPQ3 and IRRPT. All are open source.

4.4.1.2. Prefix filter configuration tools

The tools range in feature-set (and corresponding complexity) from prefix-list generator (BGPQ3) to router configuration template processor (IRRToolset).

4.4.1.2.1. BGPQ3 example

BGPQ3 is a simple command line tool that connects to the IRR database and builds prefix-lists from the collected data.

To create a Cisco IOS formatted prefix-list for all the IPv4 prefixes in an AS-SET:


    $ bgpq3 -4 -l AS64500-v4 AS64500:AS-ALL
    no ip prefix-list AS64500-v4
    ip prefix-list AS64500-v4 permit 203.0.113.0/24
    ip prefix-list AS64500-v4 permit 192.0.2.0/24
    ip prefix-list AS64500-v4 permit 198.51.100.0/24

To create a Cisco IOS formatted prefix-list for all the IPv6 prefixes in an AS-SET:


    $ bgpq3 -6 -l AS64500-v6 AS64500 AS64500:AS-CUSTOMERS 
    no ipv6 prefix-list AS64500-v6
    ipv6 prefix-list AS64500-v6 permit 2001:db8:1000::/36 
    ipv6 prefix-list AS64500-v6 permit 2001:db8:1001::/48
    ipv6 prefix-list AS64500-v6 permit 2001:db8:2002::/48

The -l option defines the name of the prefix-list and the -4/-6 options select IPv4/IPv6. As you can see in the IPv6 example it is possible to specify multiple AS-SETS on the command line. BGPQ3 will automatically combine them.

4.4.1.2.2. IRRPT example

IRRPT generates prefix-lists just like BGPQ3 but it uses configuration files for options and ASNs, it keeps track of changed prefixes between runs and it can alert administrators to such changes.

After downloading IRRPT first run the configure.php script to automatically set the most basic options in the configuration file. Then adapt the configuration file in conf/irrpt.conf to your needs. Configuration options include email addresses, default router configuration syntax style etc. Finally define the AS-SET to use for each ASN you peer with in conf/irrdb.conf. Each ASN can only have one AS-SET associated with it. It is therefore not possible to use different AS-SETs for IPv4 and IPv6 or to configure a combination of AS-SETs.

When all configuration files have been updated run the irrpt_fetch tool. It will fetch all prefixes included in the specified AS-SET for each ASN and send emails to notify administrators about changes:


    $ ./bin/irrpt_fetch 
    Processing AS64500 (Record 1)
        - Importing ./db/64500
        - Importing ./db/64500.4
        - Importing ./db/64500.6
        - Importing ./db/64500.agg
        - Importing ./db/64500.4.agg
        - Importing ./db/64500.6.agg
        - Sending update notification to [email protected]

After fetching all the prefixes IRRPT can now generate router configuration scripts:


    $ ./bin/irrpt_pfxgen 64500
    conf t
    no ip prefix-list CUSTOMER:64500
    no ip prefix-list CUSTOMERv6:64500
    ip prefix-list CUSTOMER:64500 permit 203.0.113.0/24
    ip prefix-list CUSTOMER:64500 permit 192.0.2.0/24
    ip prefix-list CUSTOMER:64500 permit 198.51.100.0/24
    ipv6 prefix-list CUSTOMERv6:64500 permit 2001:db8:1000::/36 le 48 
    ipv6 prefix-list CUSTOMERv6:64500 permit 2001:db8:1001::/48 le 48 
    ipv6 prefix-list CUSTOMERv6:64500 permit 2001:db8:2002::/48 le 48 
    end
    write mem

IRRPT by default allows more-specific prefixes, up to a configured maximum prefix length. In this example the maximum prefix lengths were configured as /24 for IPv4 and /48 for IPv6.

4.4.1.3. Router provisioning tools

Once the configuration is generated, a platform or means must be selected that can transmit and deploy this configuration to live infrastructure. Popular means of orchestrating include SNMP + (T)FTP , vty marshaling (usually over SSH) and NETCONF (again, usually over SSH). In terms of platforms, open source orchestration stack Ansible and Cisco’s Network Service Orchestrator provide more complete solutions.

4.4.1.4. A word of caution

Great caution must still be exercised before deploying this configuration on live infrastructure. A simple mistake can cause traffic to be incorrectly filtered, especially when providing transit services to customers.

The authors recommend the following mechanisms for ensuring that bad filters (or even just bad filter configuration) do not end up being deployed:

Simple syntax checks

  • Ensure the filters generated are not empty, are well formed and do not have syntax errors. Note that some routing platforms treat any empty prefix list as an implicit “allow any”.
  • Ensure the filters do not refer to impossible addresses and masks.
  • Ensure they will not inadvertently block anything which is explicitly configured to pass.

Delta checks

  • Ensure that if a filter changes by a delta of more than (n) percent (where n is an internally agreed number, for example, 20%), then prevent the filter from being deployed and prevent any further filters from being built and/or deployed until a human reviews the output.

Beacon prefixes

  • Ensure that special or important prefixes to the organisation are permitted to pass.
  • Ensure that bogon prefixes are never allowed through the filter.

4.4.1.5. Outsourcing the filtering

Some IXPs will provide route-servers that are pre-configured to perform IRR filtering.
At the time of writing, these are known to be AMS-IX, DE-CIX, LONAP, INEX, NWAX, LyonIX, FIXO-IX, and MSK-IX. Route servers with IRR filters are becoming more and more popular so check with the IXPs that you connect to if they offer this service.

Outsourcing of egress filters at an IXP is not MANRS compliant. That would be like claiming that you can litter and still be well behaved because someone else is going to clean it up for you. Being well behaved means not littering in the first place. To be MANRS compliant you have to make sure that your own network is well behaved. Operators may choose to rely on this for ingress filtering at IXPs (section 4.4.1.1 point 3) at their own risk.

The advantages of doing IRR filtering at IXP route servers:

  • Saves time and effort for the network operator
  • IXPs usually provide multiple route servers running different software for redundancy
  • Professionally maintained route servers, so smaller chance of mistakes

The disadvantage of using IRR filtering at IXP route servers:

  • Egress filtering in IXP route servers is not enough to be MANRS compliant
  • Less control over the filtering
  • Extra dependency on a 3rd party
  • Difficult to make exceptions

4.4.1.6. Additional reading

4.4.2. Using RPKI to validate route origins

The basic idea of validating route announcements with RPKI is the same as for the IRR. A network operator registers their announcements in the form of ROAs and they are subsequently used by operators to either generate filters (pseudo-IRR) or to tag/validate announcements using more advanced techniques like the RPKI-to-router protocol.

RPKI works with trust anchors that are the starting points for verifying ROAs. All RIRs publish trust anchors for the address space they are authoritative for. ARIN requires users to agree to a Relying Party agreement, the other RIRs make their trust anchors available with no strings attached.

4.4.2.1. Using the RIPE NCC RPKI Validator

The RIPE NCC RPKI Validator is implemented in Java and provides a nice web interface to manually query the data it has collected. Download the validator from the RIPE NCC website, unpack it and run:

    rpki-validator.sh start

This will start the validation software which provides a web server on port 8080. The web interface will let you keep an eye on the synchronization with the trust anchors:

It also allows you to query the database of ROAs, check the ROAs against the BGP routing table (as seen from the RIPE NCC Route Collectors) and export the ROAs as ROUTE/ROUTE6 objects for integration with tools that can use IRR data.

The validator also provides an RPKI-to-Router interface on port 8282. See below on how to make a router use this service.

4.4.2.2. Using the Dragon Research Labs RPKI Toolkit

The easiest way to use the RP Toolkit is to run it on Ubuntu Xenial or Debian Jessie using the provided APT repositories. Install the Relying Party (Validator) software with

    apt install rpki-rp

This will install all the dependencies and install a cron-job to update the local cache every hour. Statistics are written as HTML files to /var/www/rcynic, which can usually be accessed through the web server as http:///rcynic/:

The rp-rp package also provides an RPKI-to-Router interface through xinetd on port 323. See below on how to make a router use this service.

4.4.2.3. Connecting a router to the RPKI-to-Router interface

The RIPE NCC website provides extensive examples on how to configure routers to use an RPKI validator. The following basic examples are based on their documentation.

4.4.2.3.1. Juniper JunOS

First connect the router to the RPKI validator. In the following example the router with address 192.0.2.1 connects to an RPKI Validator with address 192.0.2.2:


    routing-options { 
        validation {
            group rpki-validator { 
                session 192.0.2.2 {
                    refresh-time 120; 
                    hold-time 180;
                    port 8282;
                    local-address 192.0.2.1;
                }
            }
        }
    }

The router will now keep track of which routes are valid according to the ROAs, which routes are invalid and which routes aren’t covered by ROAs. You can use that in your routing policy:


    policy - options {
        policy - statement validation {
            term valid {
                from {
                    protocol bgp;
                    validation - database valid;
                }
                then {
                    validation - state valid;
                    community add origin - validation - state - valid;
                    next policy;
                }
            }
                term invalid {

                ...etc...

You can use the commands under show validation to check the sessions to the validator, the received data etc:


    > show validation statistics
    Total RV records: 1487
    Total Replication RV records: 2946
      Prefix entries: 1382
      Origin-AS entries: 1487
    Memory utilization: 440802 bytes
    Policy origin-validation requests: 13065187
      Valid: 35605
      Invalid: 37896 
      Unknown: 12991686
    BGP import policy reevaluation notifications: 27306 
      inet.0, 27306
      inet6.0, 0

4.4.2.3.2. Cisco IOS XE

The RPKI validator can be configured under the BGP routing process:


    router bgp 64500
      bgp rpki server tcp 192.0.2.2 port 8282 refresh 600

The router ll now keep track of which routes are valid according to the ROAs, which routes are invalid and which routes aren’t covered by ROAs. You can use that in your routing policy:


    route-map rpki permit 10 
      match rpki valid
      set local-preference 999
    ... etc ...

You can use the commands under show bgp rpki to check the sessions to the validator, the received data etc:


    #show bgp ipv4 unicast rpki servers
    BGP SOVC neighbor is 192.0.2.2/8282 connected to port 8282
    Flags 64, Refresh time is 60, Serial number is 60, Session ID is 914 
    InQ has 0 messages, OutQ has 0 messages, formatted msg 30
    Session IO flags 3, Session flags 4008
      Neighbor Statistics: 
        Prefixes 25773 
        Connection attempts: 250 
        Connection failures: 244 
        Errors sent: 0
        Errors received: 2
    
    Connection state is ESTAB, I/O status: 1, unread input bytes: 0 
    Connection ECN Disabled, Mininum incoming TTL 0, Outgoing TTL 255 
    Local host: 192.0.2.1, Local port: 46721
    Foreign host: 192.0.2.2, Foreign port: 8282
    ... etc ...

4.4.2.3.3. Cisco IOS XR

The RPKI vadator can be configured under the BGP routing process:


    router bgp 64500
      rpki server 192.0.2.2
        transport tcp port 8282

The router ll now keep track of which routes are valid according to the ROAs, which routes are invalid and which routes aren’t covered by ROAs. You can use that in your routing policy:


    route-policy rpki
      if validation-state is valid then
        set local-preference 999 
      endif
      ... etc ...

You can usehe commands under show bgp rpki to check the sessions to the validator, the received data etc:


    # show bgp rpki summary
    RPKI cache-servers configured: 1 
    RPKI global knobs
      Origin-AS validation is ENABLED globally
      Origin-AS validity WILL NOT affect bestpath selection globally 
      Origin-AS validity signaling towards iBGP is DISABLED globally
    RPKI database
      Total IPv4 net/path: 23854/24733 
      Total IPv6 net/path: 3368/3507

4.4.2.4. Additional reading

Technical documentation on RPKI and the underlying protocols:

4.4.3. Path validation

Path validation seeks to validate the AS_PATH attribute of an observed prefix announcement. For these purposes, a path is considered valid if it is consistent with the intended routing policies of each AS traversed by the path. Such validation therefore requires that an authoritative description of each ASNs routing policy be made publicly available.

The automated validation (with certain limitations) of an observed AS_PATH can be achieved by logically “chaining together” the documented policies of the individual ASs in the path. MANRS participants should therefore express their own policy as thoroughly as possible to facilitate validation by 3rd party networks.

At the very least, as noted above, any operator that provides transit services to potentially multi- homed customer networks (irrespective of whether those customers operate an AS) should implement a filtering mechanism on outbound announcements towards its non-customer peers to ensure that only customer prefixes that are being learnt directly from customers (rather than via some other intermediate network) are propagated onwards.

Whilst it is possible to construct AS_PATH filters describing all the possible valid customer paths, such an approach may scale poorly over time given the tools available today. An alternative approach is to append a community attribute to routes received from customers as they are imported, and to configure outbound filters towards non-customer peers to match only routes carrying the correct community attribute (in addition to prefix-based matching as described above)