VXLAN Comes of Age with BGP-EVPN

by Mike Cohen & Harry Petty

Back in 2011, when software-defined networking was in its infancy, Arista, Broadcom, Cisco, Citrix, and Red Hat joined together to rethink multi-tenancy and segmentation in the cloud datacenter.  The output of this collaboration was Virtual eXtensible Local Area Networks (VXLAN). VXLAN introduced an overlay scheme that expands the layer-2 network address space from 4K to 16 million, largely solving the scaling issues operators were seeing in VLAN-based environments.  More importantly, a broad range of networking hardware, ASICS, and hypervisor vendors  backed the proposal, creating an opportunity for long-term hardware support and smooth interoperability.   In fact, the ONUG virtual networks overlay working group has since called out VXLAN support in its very first solution requirement.

However, while VXLAN offered a solid data plane solution, it needed to be paired with an equally solid control plane. The initial solution, based on data-driven flood-and-learn behavior, introduced  scaling challenges  of a control plane and turned off some operators who did not already run multicast-based networks. Meantime, another multivendor solution, Ethernet VPN (EVPN), has emerged to offer a strong end-to-end solution for datacenter VXLAN networks. EVPN uses a new address family, L2VPN EVPN,  of Multi-protocol BGP control plane to distribute VXLAN EVPN routes that include both Layer-3 Host IP routes and Layer-2 MAC routes. Multi-protocol BGP has a proven track record for operating Internet-scale IP networks with multi-tenancy support. Cisco and other leading industry network vendors and operators have proposed  specifications for a BGP MPLS based Ethernet VPN (RFC 7432) and extentions of RFC 7432 to enable BGP control plane for VXLAN encapsulation, draft-ietf-bess-evpn-overlay to the Internet Engineering Task Force (IETF). Further, the European Advanced Networking Test Center (EANTC)  recently released a report on interoperability testing of EVPN uses cases incorporating submissions from leading switch vendors. The EVPN-based control plane offers a number of advantages over the flood-and-learn mechanism used by VXLAN today, including:

  • Discover and authenticate VXLAN tunnel endpoints (VTEP) dynamically
  • End to End mobility with optimal forwarding with distributed any cast gateway
  • BGP protocol-driven learning to distribute both MAC and IP addresses to avoid the need for the flood-and-learn mechanism for unknown host learning
  • Manage broadcast and multicast packets with either a multicast or unicast core (using ingress replication)
  • Terminate Address Resolution Protocol (ARP) requests early and minimize flooding

The BGP-EVPN control plane offers a standards-based, scalable solution for multi-tenancy and host mobility for VXLAN overlay networks. It can be supported on both virtual as well as physical networking devices as well, allowing software-based overlay mechanisms to build tight integration with physical switch vendors.

cisco blog figure 1

Just as we saw data plane standards like VXLAN gain popularity to enable overlay networks, it appears we may be witnessing another building block of open software-defined networking technology emerge.  In fact, leading edge vendors already have solutions that can be deployed today.

For More Information:

White Paper – Cisco Border Gateway Protocol Control Plane for Virtual Extensible LAN

Blog – VXLAN/EVPN: Standards based Overlay with Control-Plane

Video – Python shell for VXLAN EVPN config management


Author Info

RPV_0317 (1)Mike Cohen

Director of Product Management, Cisco Systems

Mike Cohen is Director of Product Management at Cisco Systems where he leads a team focused on developing open source policy-based solutions.  Mike began his career as an early engineer on VMware’s hypervisor team and subsequently worked in infrastructure product management on Google and Big Switch Networks.  Mike holds a BSE in Electrical Engineering from Princeton University and an MBA from Harvard Business School.

AQ75453[6] Harry Petty

Director – Marketing, Data Center, and Cloud Networks, Cisco Systems

Harry Petty is Director – Marketing, Data Center and Cloud Networks, at Cisco Systems where he focuses on SDN, Application Centric Infrastructure, and open ACI ecosystem solutions.  Harry began his early career as a switching software engineer, later worked in server product management at Groupe Bull, and data center LAN and SAN marketing at Brocade.  Harry has a BS in Mathematics from University of Dayton and an MBA from Booth Graduate School of Business at the University of Chicago.