Most enterprise WANs are built using MPLS VPN service, sometimes using two connections from different providers for diversity and often with Internet VPN for backup. MPLS VPN is easy to order and manage, very reliable and most performance issues can be fixed simply by throwing more money to the provider for circuit upgrades. The chief downside is its high cost, approximately 30 – 100 times more expensive per Mbps than Internet bandwidth.

Recently, some network administrators have found themselves scrambling to upgrade MPLS VPN bandwidth across their networks, and the added cost is setting off alarms. What has changed? Are these costs warranted? Is there a better way?

To understand what’s at work, consider the network traffic patterns for remote branch offices:

  • Branch-to-DC- Traffic between the branch office and the corporate data center (DC), such as email, ERP, file sharing, Citrix/VDI.
  • Branch-to-Branch- Traffic between branch offices, such as IP telephony, video conferencing, and peer-to-peer collaboration software.
  • Branch-to-Internet- Traffic between users or servers at the branch office and resources on the Internet. This includes web surfing, but also cloud-hosted applications like, Office 365, Google Docs, etc.

Below are the two WAN usage trends I’ve been seeing:


graph2In both cases, branch-to-Internet traffic is on the rise. In one case, branch-to-DC traffic is even dropping as applications move to the cloud.

For the enterprise, this is an issue because branch-to-Internet traffic is typically backhauled across the MPLS VPN to a DC for corporate security policy enforcement – firewalls, content filters, data loss prevention, etc. In the past, the cost of this was absorbed or ignored because branch-to-Internet was limited to casual web surfing. But today’s branch-to-Internet traffic explosion is due to cloud-hosted applications, rich media, conferencing, guest/BYOD traffic, mobile devices and “app store” software distribution.

Recently, I helped audit a client’s MPLS VPN and found many branches with >80% Internet traffic share. The client had been in an arms race, paying to upgrade both branch MPLS VPN circuits and their data center Internet circuits to carry branch-to-Internet traffic.

What is the alternative?

The Internet isn’t becoming less important, so every organization’s long-term WAN strategy must be to move some of the branch-to-Internet traffic off MPLS VPN and on to branch office Internet service. Here’s where to start:

First, measure the volume of branch-to-Internet traffic and calculate the cost of sending it over MPLS VPN. Then, research branch Internet service prices to determine the potential savings. I’ve had clients arrive at savings of between $500 and $1000/month per branch.

Second, work with security stakeholders to identify acceptable ways to decentralize security, such as:

  • Allow guest/BYOD traffic to reach the Internet directly, using a router or in-branch firewall.
  • Use a cloud web security service, such as those from Symantec, Cisco or Websense that do web filtering and Internet security in the cloud for a per-seat fee. Be sure to do a proof-of-concept since cloud security management will have fewer controls than local solutions, and there may also be technical challenges (user identification, SSL traffic, web filter categories, etc).
  • White-list heavily used Internet sites so that they can be reached directly from the branch using a router or in-branch firewall. For example, these might include Windows Update, Apple App Store, YouTube, Amazon Web Services (AWS) and so on. IP address whitelists are simple, but can be difficult to maintain. URL whitelists are more flexible and can be implemented with proxy servers and proxy auto-config (PAC).
  • Ask your MPLS VPN provider about their cloud firewall service offerings. In my experience, this is a dead-end discussion since it does nothing to reduce the branch MPLS VPN circuit costs. But it is very easy to implement, and that may justify the cost.

Third, look at the branch network design and figure out how the Internet service should attach. This can be complicated if:

  • The MPLS provider owns and manages the MPLS VPN router
  • The branch has inline appliances for WAN acceleration, traffic shaping, security or monitoring branch hardware fault tolerance is required (e.g., dual WAN routers)
  • The security folks don’t allow the use of router-based firewalls; • Proxy servers are used.

It’s also worth considering what it would take to implement a “next-gen” WAN, one that gracefully supports multiple WAN circuits with routing policies based on application performance, link utilization, path availability, time-of-day, etc. Cisco’s Intelligent WAN framework delivers some of these capabilities today, and this is an area where software-defined networking (SDN) is expected to help in the coming years.

Fourth, if this seems too daunting a task for your IT staff to take on, there are managed service alternatives. Glue Networks offers cloud management of complex next-gen WANs, while Meraki sells cloud-managed security and VPN appliances that are ideal for simpler environments like retail and kiosks.

Finally, it’s a good exercise to evaluate the continued use of MPLS VPN. Many enterprises get by fine without it, and much of the rationale for MPLS VPN is debatable:

  • Rationale MPLS VPN gives us private, secure connectivity.
  • Retort MPLS VPN is unencrypted. Mistakes by the provider (often while provisioning new sites) can impact security and stability of the whole WAN. Many networks are now encrypting traffic before sending it over the MPLS VPN.
  • Rationale MPLS VPN allows us to have a single provider for ordering, billing and support.
  • Retort The same provider will happily sell you Internet service, too.
  • Rationale End-to-end QoS is needed for IP telephony, video and business-critical applications.
  • Retort While this is a valid point, there’s plenty of anecdotal evidence that voice, video and business apps can work well on the Internet without end-to-end QoS. QoS is too complex a subject to explore deeply, but here are some useful facts:
  1. If all Internet circuits are from one provider, WAN performance will be quite similar to MPLS VPN from that same provider.
  2. When a circuit moves from low-speed T1 to high-speed Ethernet, some of the need for QoS goes away. For example, latency and jitter are naturally low on Ethernet.
  3. Although the Internet doesn’t support QoS, customer-owned WAN routers can still provide priority and shaping guarantees for voice and video.
  4. Modern audio/video software can be configured to tolerate Internet packet loss, jitter and latency.
  5. Many Internet circuits have higher download speeds than upload speeds. This can make QoS less important since branches can’t overwhelm each other.
  • Rationale MPLS VPN has four or five nines of reliability
  • Retort For the same price as 10Mbps MPLS VPN service, you could get two 200Mbps Internet connections from different providers, and even better reliability.

In conclusion, the recent need for MPLS VPN bandwidth upgrades is a wake-up call that few enterprises can ignore. The solution is simple, doable, cost-effective and inevitable. But it does require buy-in from security stakeholders and requires the network administrator to detach somewhat from the warm comfort of a single MPLS VPN provider.