[UPDATE] My home lab now runs Palo Alto VM100 virtual firewalls with a Unifi Security Gateway. I’ll leave this post up for anyone who may be interested in my experiences working with VyOS[/UPDATE]
I’ve just completed a project implementing VyOS routers into the home lab. In this post I’ll detail why and how I use VyOS. VyOS is a fork of what was Vyatta Core, the opensource community edition of Vyatta. When Brocade acquired Vyatta they discontinued the core edition but in 2013 VyOS was born as an opensource project. Last year I took Brocade’s promotional offer for online training and free exam voucher for their Vyatta vrouter engineer. I was instantly hooked on Vyatta as an OS but i’m a huge fan of opensource so opted to go with VyOS, plus its free 🙂
For some time I was running a Palo Alto PA 200 as the main firewall in my home lab. The vendor stat’s show the throughput performance is around 100mbps and 50mbps with threat enabled. Depending on the type of traffic / applications traversing the firewall I’ve actually had more than that through, In fact, on one test using iperf I got 360Mbps. I use the Palo frequently for testing funcionality for customers and remote access to the lab via Global Protect. The throughput started to cause me performance issues that needed resolving. My IP surveillance and HDMI over IP applications and can on occasion hit 300Mbps of traffic. The little PA 200 was breaking into a sweat quite regularly.
To lighten the load in the interim I moved some of the vlan’s off the Palo and onto my Cisco SG300 switches as SVI’s. This helped significantly as the SG300 can forward at line rate and non blocking. I used ACL’s to filter traffic between vlan’s especially for the IoT devices but this approach becomes an annoyance due to the time consuming nature of working with stateless packet filters. The SG300’s do not support first hop redundancy protocols such as HSRP or VRRP either. I knew this when I purchased them and was happy to run with SVI’s on one switch only for a while as I’d always planned to use a virtual router / firewall as an inter vlan router. I’ve been working towards eliminating a single point of failure in the home and this is where the VyOS routers come in.
I’ve so far built 4 VyOS virtual routers (two on each of my hypervisors) configured as zone based statefull firewalls. Two of them run in the internet edge and two running as LAN / WAN routers. Each pair is clustered and sync the conntrack state stables between them. VRRP provides the gateway for various hosts and OSPF takes care of the routing between the VyOS routers, Palo Alto firewall and various other kit in the lab. If I loose a hypervisor or a physical switch, VRRP and OSPF takes care of the fail over to the routers on the other Hypervisor.
Performance
I’ve had the VyOS routers in place now for a few weeks and they’ve been rock solid. Each router has a single cpu core and 512GB ram assigned to it. I can almost get line rate throughput with a fairly sizeable statefull firewall policy. That’s just superb. Below is the result of a simple test I ran from a Windows 2012 server to a shared folder on my NAS thro
ugh one of the VyOS routers.
The NAS has constant traffic running through it so I was never going to hit Line rate but it was pretty close! I also ran an iperf test and was able to reach 983Mbps. Almost 1Gbps of read speed through a virtual statefull firewall with 512GB of ram and a single cpu core ? thats just awesome 🙂 I’m fairly confident they’ll easily cope with much more than I’ve thrown at them. I’ve read that some people have had up to 6 Gbps although I’m not sure how much resource they assigned to them. When I run either of the LAN Speed test or iperf I’ve never seen a spike in CPU to RAM during the test. One important point to make here though is VyOS is a layer 4 firewall. The Palo Alto is a Next Generation (Layer 7) firewall which inspects the payload of each and every packet which is significantly more expensive on compute power to achieve.
Configuring VyOS
At the time of writing this post there was no GUI to configure VyOS, its CLI only. Don’t let that scare you though as the CLI syntax is beautiful to work with. Its similar to Junos and you can prep your changes, review before committing them and even have VyOS roll them back after a configurable amount of time if you don’t confirm the commit. That’s really handy if there’s a risk that your change may break your access to the device. If your used to zone based firewalls from vendors like Palo Alto, Fortinet, Juniper etc configuring VyOS as zone based statefull firewall is a bit of a learning curve but well worth the effort.
What challenges did I face?
I did meet some challenges when deploying VyOS. The main one for me was not being able to nat behind a VRRP address. I needed to nat as my cable modem does not support static routes, you can configure a subnet on the LAN interface only. If I could NAT traffic towards the cable modem behind the VRRP address of the active VyOS router there would be a route back from the cable modem to the active router. Using my google fu I learnt that a patched version of VyOS addresses this issue but I decided to wait until that feature is rolled into a main stream release. As a workaround I turned off RFC 3768 mode for VRRP on the VyOS routers. Traffic is still NAT’ed behind the active VyOS routers interface address but under fail over I just have to be patient while the TCP connections time out and re-establish (the tcp 5 tuple match breaks due to the change in NAT address) For a home lab that’s perfectly acceptable to me.
Summary
I’m delighted I went down the VyOS route. VyOS is awesome to work with, fast, reliable and free! You can run them as routers only which takes minutes to provision them. In my case it took me a about 3 weeks of evenings (quite a few hours) as I enabled the zone based firewall functionality and needed to built and test the policy. At some point in the future another project is to implement a 4G router to act as a fail over to my cable provider.