I recently had an opportunity to experiment with a VeloCloud Edge ‘gateway’ before/while deploying it, and thought I would write up the ‘quirks and features’ that I ran across. Some background, I’m used to deploying solutions that provide many ‘nerd-knobs’ and the ability to look at logs and flows with relative ease. VeloCloud, from a customer’s standpoint, is very much a hands-off, let-it-do-its-thing solution. There’s pros and cons to this approach, depending on what your needs are, and how far you are willing to trust the magical
black white box to do what you want it to do. In my experience, VeloCloud certainly gets the job done, however sometimes I wish that I could have more visibility into what’s going on behind the curtain.
First quirk is the limitation on how WAN interfaces can be configured. On the Edge appliances, only ‘routed’ interfaces can be used for WAN connections. Typically these routed ports are labeled as ‘GE1’, ‘GE2’, SFP1, etc… using the ‘LAN’ ports are not available, as those are considered ‘Switched’ ports. What this means is on a smaller appliance, such as the 520 model, you’re limited to 2 copper WAN ports out of the box. Probably suitable for most smaller branches, but if you need more, then you’ll need to install SFP modules to enable the use of the other 2 ‘routed’ interfaces.
Supposedly there’s a way to configure 2 WAN connections on a single port, however it seems to be designed around an ISP connection that provides both an Internet WAN connection, and a private WAN connection, such as an MPLS or Metro Ethernet, over the same physical connection. One connection would be either untagged, or tagged with a VLAN ID. The second connection would of course be tagged with a second VLAN ID. This is not an unusual configuration for SD-WAN and Firewall deployments in multi-internet deployments, especially when dealing with high-availability configurations, or multi-gateway/SD-WAN environments, that need to share the internet connections between multiple devices. Typically this would be done by connecting the internet connections into switch(s), each one on a specific VLAN. I attempted to setup the Edge gateway with both internet connections on one physical interface (with one or both connections tagged with a VLAN ID), however I was unable to get both working reliably. I switched to the standard setup (one port per WAN connection), and installed a copper SFP module into the Edge so I could also have the use of a ‘Routed’ LAN interface. More on Routed LAN interfaces below.
On the topic of LAN interfaces, we have a two types. The first type are ‘Switched’ ports on the back of the Edge. You configure these ports in the same way as a basic Layer 2 switch… each port can be setup in ‘Access’ or ‘Trunk’ mode and be configured with any VLAN interfaces that you configure. However, keep in mind these VLAN interfaces cannot be used with OSPF to peer with other OSPF routers. OSPF can be enabled on that interface, but only in passive mode.
The second type of LAN interfaces is a ‘Routed’ interface. This cannot be shared with a WAN interface (no router-on-a-stick allowed here!), however there is more control available over the interface, including having the ability to enable OSPF for peering with other routers. You CAN create a sub-interface on a ‘Routed’ interface, however OSPF cannot be enabled in any capacity on that interface.
Side note: I believe BGP is not bound by these interface limitations, however I did not have a BGP router available at the time to see if that’s true (including VLAN interfaces). This would make setting up dynamic routing a bit more flexible in an environment that can utilize BGP.
Other quirks I’ve found:
- Packet captures are allowed, but it takes a few minutes to start capturing on the specified physical interface, and you’re limited to only a 120 seconds. This makes it tricky to coordinate reproducing an issue with a user, since you may be ready NOW, or in 2 minutes. And if it’s an intermittent issue…
- Some changes require the VeloCloud services to restart. This may cause traffic to stop flowing temporarily.
- Edge Gateways have a limited number of tunnels they can support. Don’t expect to be able to support concurrent branch-to-branch tunnels between ALL your branches, if you’ve got more than a few dozen branches with dual internet connections at each site. You can do dynamic branch to branch tunnels, and traffic will dynamically ‘float’ to one of those branch-to-branch tunnels once it’s established, but be careful with this. Dynamic VPN works in simple environments where branch-to-branch communication is rare (such as the occasional VoIP call, etc). But if you don’t limit what traffic is allowed between branches, you could run into situations where some process behind the scenes (network vulnerability scans, Windows Update peer-to-peer sharing, roaming users that have ALL the printers installed locally on their laptop for ALL branches, etc) will trigger the creation of dynamic VPNs between ALL the branches. The Edges will run out of memory or CPU and start to drop traffic, OSPF will drop routes intermittently as the OSPF process competes for resources, etc. Make sure your appliances are SIZED CORRECTLY not only for the number of SITES, but also the number of internet connections at each site. If someone gets the bright idea of slapping a 4g card in each Edge Gateway for a backup connection… you’ve now added N tunnels from each branch to your hubs.
- You can’t ping the LAN interface IPs of the VeloCloud Edge Gateways… at least out of the box through the SD-WAN. This makes troubleshooting connectivity issues more exciting.
In a future post I’ll do a comparison between VeloCloud and SilverPeak (since I have experience with both solutions) and how the two differ in deployment flexibility and manageability.