IXP peering platform: an environment to take care of

By Flavio Luciani, MANRS ambassador, and Carmen Denis Polanco, MANRS fellow

Internet Exchange Points (IXPs) are delicate infrastructure and play an important role in the Internet ecosystem. They are strategic points where network operators meet and where thousands of Border Gateway Protocol (BGP) sessions take place.

We can describe IXPs as physical infrastructure made up of several elements through which IXPs themselves provide different interconnection services to their members. One of the most essential components of an IXP is the peering platform. It is the heart of every IXP, the equipment where the peering interconnections happen.  Typically, the peering platforms act like a plain Layer 2 ethernet network and the Virtual LAN dedicated to the exchange is known as “peering LAN“.

The main purpose of the peering platform is to allow Internet service provders (ISPs) to exchange Internet protocol (IP) traffic between their Autonomous Systems and those of their customers.  The only function that routers connected to the IXP should accomplish is to establish BGP peerings and to forward IP traffic. Features that do not perform BGP-related functions which are active on ISP routers might be not only useless, but potentially dangerous to the operations of the IXP.

Like in all common environments, there must be rules that all should observe to ensure that the IXPs run safely for the benefit of all participants. Although not strictly related to routing security, applying hygiene at Layer 2 can contribute to the stability of the IXP and consequently to the routing itself.

Requirements for the better functioning of the peering platform

IXPs typically publish the policy about allowed traffic in their technical regulations. Although it may depend on the IXP, there are common rules shared by all IXPs.

Each port of the peering platform is filtered on the ethertype field so that only packets with proper payload are allowed:

• 0x0800 – IPv4
• 0x0806 – ARP
• 0x86dd – IPv6

Behind each port, only one MAC address should be visible (MAC locking mechanism). Usually, for each peering connection, the MAC address of its router interface is statically “locked” to the switch port where it is connected. All packets coming from other MAC addresses are dropped by the port.

This requirement protects the platform against potential loops that can sometimes happen (for example a third party layer-2 provider supplying more than one member with remote connections to the IXP).

Together with MAC Locking, IXPs could enforce Broadcast Storm Control mechanisms on their equipment, in order to prevent an accidentally generated broadcast storm to affect the entire exchange functionality. As a general rule (and it depends on the implementation), the broadcast level is limited to 10% of available bandwidth on each port. 

Proxy-ARP on the router’s interface is not allowed. A router configured with Proxy ARP answers to all ARP queries passing through the network which is dangerous as it could lead to the hijacking of packets destined for other ISPs. 

Frames forwarded to the peering platform must not be addressed to a multicast or broadcast MAC destination address. There are two exceptions:

• broadcast ARP packets
• multicast ICMPv6 Neighbour Discovery packets

Traffic related to Link Local Protocols is not allowed and it should not be forwarded through the peering platform (except for ARP and IPv6 ND protocols):

• IRDP
• ICMP redirects
• IEEE 802 Spanning Tree (BPDU filtering)
• Vendor proprietary protocols. These include, but are not limited to:
  • Discovery protocols: CDP, EDP, LLDP etc.
  • VLAN/trunking protocols: VTP, DTP
  • Interior routing protocol broadcasts (e.g. OSPF, ISIS, IGRP, EIGRP)
  • BOOTP/DHCP
  • PIM-SM
  • PIM-DM
  • DVMRP
  • ICMPv6 ND-RA
  • UDLD
  • L2 Keepalives

None of this should limit adoption of other security best practices on other network devices, such as application of mechanisms like Infrastructure Access Control Lists (iACL) and/or VLAN Access Control Lists (VACL).

Even though this article focuses on the importance of security at the Link Local Protocols level of the Peering core platform, it is essential to fully adopt protections in the management, control and data planes of a network infrastructure.

As network administrators we should remember that security is a process, not a state, and our infrastructure and/or services can be affected at the weakest link. Where possible, it is useful to filter the traffic listed above in order to guarantee high levels of security and robustness to the whole peering platform, and to avoid potential problems to the smooth flow of traffic between members.

Given that the peering platform is one of the essential elements of the IXPs, this aspect has been taken into serious consideration by MANRS. Indeed, in the MANRS IXP program the Action 3 requires IXPs to take care of the peering platform by applying hygiene measures in order to filter traffic potentially harmful to the operation of the platform itself. 

Editor’s note: This is a guest post by a MANRS ambassador and fellow. Viewpoints expressed in this post are those of the authors’ and may or may not reflect official positions.