The latest thing that is getting a lot of attention is WireGuard - the new shooting star in terms of VPN. But is it as great as it sounds? I would like to discuss some thoughts, have a look at the implementation and tell you why WireGuard is not a solution that will replace IPsec or OpenVPN.

In this article I would like to debunk the myths. It is a long read. If you are in need of a tea of coffee, now is the time to make it. Thanks to Peter for proof-reading my chaotic thoughts.

I do not want to discredit the developers of WireGuard for their efforts or for their ideas. It is a working piece of technology, but I personally think that it is being presented as something entirely different - as a replacement for IPsec and OpenVPN which it simply is not.

As a side-note, I think that the media is responsible for this and not the WireGuard project itself.

There has not been much positive news around the Linux kernel recently. They have reported of crushing processor vulnerabilities that have been mitigated in software, Linus Torvalds using too harsh language and just boring developer things. The scheduler or a zero-copy network stack are not very approachable topics for a glossy magazine. WireGuard is.

It sounds great on paper, it is exciting new technology.

Let's have a closer look.

The WireGuard Whitepaper

This article is based on the WireGuard Whitepaper written by Jason Donenfeld. In there he explains the concept, objective and technical implementation in the Linux kernel.

The first sentence says:

WireGuard [...] aims to replace both IPsec for most use cases, as well as popular user space and/or TLS-based solutions like OpenVPN, while being more secure, more performant, and easier to use.

Of course the biggest selling points of some new technology are that it is easy. VPNs also need to be performant and secure.

What else would they say here?

If that is not your design goal, then you will probably have no chance. However, those are exactly the same goals for literally every other VPN technology, too.

The more interesting part of that sentence is "for most use cases". That has been removed by the press and here we are in the middle of this mess.

Will WireGuard replace my (IPsec) site-to-site VPN?

No. There is no chance the big vendors like Cisco, Juniper, etc. will pick up WireGuard. They do not jump onto trains like this unless there is a big necessity.

I will later cover a couple of reasons why they probably can't ship WireGuard even if they wanted to.

Will WireGuard replace my roadwarrior from my laptop to the data centre?

No. Right now, WireGuard has a huge backlog of features that it needs to implement to be suitable for this use-case. It does not, for example, allow using a dynamic IP address on the server side of the tunnel which breaks a whole use-case.

IPFire is often used on a cheap Internet uplink like a DSL or cable connection. For branch offices and small to even medium businesses, this makes sense and saves money on an expensive fibre uplink which isn't needed by everyone. However, DSL, cable, LTE and all other sorts of wireless connections come with dynamic IP addresses. Sometimes it does not change that often, but it still is dynamic.

There is a sub-project called "wg-dynamic" which adds a user-space daemon to overcome this shortcoming. A massive design flaw because of the use-cases stated above, but it is even worse that IPv6 is all about dynamic addressing.

From a distributor's point of view, this is also quite disappointing. One of the design goals was to keep the protocol lean and simple.

Unfortunately it has become too simple that more software is now necessary to make it work in the real world.

So it is easy to use then?

Not yet. I am not saying that WireGuard won't reach a point where it is a good alternative for host-to-net connections, but so far it just is an alpha version of what it aims to be.

But what does it actually try to solve? Is IPsec really hard to use?

No, it clearly is not if the vendor has done their homework right and provides an interface that is easy to use - like IPFire.

To setup an IPsec VPN tunnel you will need to have five pieces of information that you will have to type in: Your own public IP address, the public IP address of your peer, the subnets that you want to make available through this VPN for each side and a pre-shared-key. That way, the VPN is set up within minutes and it is compatible with every vendor out there.

Unfortunately there are some exceptions out there. Everyone who has ever tried to create an IPsec tunnel to an OpenBSD machine can probably tell a tale of that. There are some more black sheep out there, but there are just as many good examples.

On Protocol Complexity

The end-user does not have to worry about the complexity of the protocol.

If that was an issue we would have definitely gone rid of SIP and H.323, FTP and other protocols that don't cope well with NAT and are decades old.

There are reasons why IPsec is more complex than WireGuard: It does so many more things. User-authentication using username/password or a SIM card with EAP. It is extensible that new cryptographic primitives can be added.

WireGuard does not have that.

That means WireGuard will break at some point, because one of the cryptographic primitives will weaken or entirely break at some point. The author says:

Finally, WireGuard is cryptographically opinionated. It intentionally lacks cipher and protocol agility. If holes are found in the underlying primitives, all endpoints will be required to update. As shown by the continuing torrent of SSL/TLS vulnerabilities, cipher agility increases complexity monumentally.

The last sentence is absolutely true.

Negotiating what cipher to use makes protocols like IKE and TLS more complex. But is it too complex? Indeed vulnerabilities have often been found in the protocol handshake. There is no alternative to this.

On Ignoring Real-World Problems

Imagine you have a VPN server with 200 road warrior clients somewhere out there in the world - which is a very normal use-case. If you were to change the cipher you are using from one day to the next one, you would need to upgrade your WireGuard software on all those laptops, phones, etc. at the same time. That is literally impossible. Administrators who have tried this needed months to deploy configuration changes. Sometimes even middle-sized companies need years to conduce a process like this.

IPsec and OpenVPN offer cipher negotiation. So for a while you will turn on the new cipher and gradually update all clients. After that is done you will turn off using the broken cipher and you are done. The clients won't even notice.

This is a very common task in large deployments and OpenVPN has been a pain with this, too. Compatibility matters and although you are using some weaker cipher, for many this is no reason to shut down their business and cut off hundreds of sales people from doing their job.

WireGuard made their protocol easier, but entirely unusable for people who are not in permanent control of both peers. In my experience this is by far the most likely case.

Cryptography!

But what are all those exciting new ciphers that WireGuard is using?

WireGuard utilizes Curve25519 for key exchange, ChaCha20 for encryption, and Poly1305 for data authentication, SipHash for hashtable keys, and BLAKE2s for hashing.

ChaCha20-Poly1305 is standardised for both, IPsec and OpenVPN (through TLS).

It is obvious that Daniel J. Bernstein's work is being used here a lot. BLAKE2 is the successor of BLAKE, a SHA-3 finalist which did not win because of its close similarity to SHA-2. If SHA-2 would have been broken, there is a good chance that BLAKE breaks, too.

IPsec and OpenVPN do not need SipHash because of their design. So the only thing that is currently not usable with those is BLAKE2 until it is standardised. That is not much of a disadvantage, because VPNs use HMACs for integrity, which still is considered strong even with MD5.

I would conclude that practically the same cryptography is available for all VPNs here. Therefore WireGuard is not more or less secure than the others when it comes to encryption or data integrity.

But that isn't the focus, according to the whitepaper. Speed is.

Is WireGuard faster than other VPN solutions?

Simple answer: It isn't.

ChaCha20 is a stream cipher which are easier to implement in software. They encrypt one bit at a time. Block ciphers like AES encrypt a block of 128 bits at a time. That would need many more transistors when implemented in hardware, so larger processors come with AES-NI - an instruction set extension that performs some tasks of the encryption process to speed it up.

AES-NI was not expected to land in smartphones and ChaCha20 was developed as a lightweight and battery-saving alternative. So it might surprise you that every smartphone that you can buy today has some sort of acceleration for AES and perform it faster and more energy-efficiently than ChaCha20.

Obviously virtually every desktop/server CPU bought in the last couple of years has AES-NI.

Hence I expect AES to outperform ChaCha20 in every single scenario. The WireGuard whitepaper mentions due to AVX512, ChaCha20-Poly1305 will outperform AES-NI1, but that instruction set extension will only be available on large processors which again won't help with smaller and mobile hardware that will always be faster with AES-NI.

I am not sure if this could have been foreseen when WireGuard was drafted, but today it is already a disadvantage that it is nailed to one cipher which might not perform very well.

IPsec allows you to freely choose what cipher is best for your setup. Certainly that is necessary when you want to transfer 10G or more over a VPN connection.

Issues integrating into Linux

So although WireGuard has chosen modern cryptography, this has already causes loads of problems. Instead of using what the Linux kernel provides, the integration of WireGuard was delayed by years due to lack of those primitives in Linux.

I am not sure what the situation is like in other operating systems, but probably not very different.

What does the real world look like?

Unfortunately every time, when a customer asks me to help them setting up a VPN, the credentials that they are getting are using old ciphers. 3DES in combination with MD5 is a common candidate as well as AES-256 with SHA1. Although the latter is better, it is still not what I would like to use today.

For the key exchange RSA is always being used, which is slow, but still secure enough.

These customers have connections with customs authorities or other government agencies and large corporations in the world which names we all know. They all use an order form that has been created decades ago and an option for "SHA-512" has simply never been added. It is isn't always the technology that is stopping innovation, but slow corporate processes.

It hurts me to see this because IPsec has support for Elliptic Curves since around 2005. Curve25519 is newer and available, too. Alternatives to AES like Camellia and ChaCha20 are also specified, but obviously not all of them are implemented by the big vendors like Cisco and so on.

That is however what people are using out there. There is a lot of Cisco boxes out there, and those that are not Cisco are made to work with Cisco. They are the market leader in this segment and not very much interested in driving innovation.

This is awful, but we won't see that change with WireGuard. The vendors will probably not see any performance issues with their existing choice of ciphers, no interoperability issues when using IKEv2 - and therefore there is little pressure for them to offer alternatives.

If you have fallen victim of any of this, have you thought about replacing your Cisco?

Benchmarks

The whitepaper benchmarks WireGuard. Although it is not a scientific paper, I would still expect a scientific approach to a benchmark. It is worthless if it cannot be repeated and it is worthless when it does not examine the software in a realistic lab setup.

In the Linux implementation, WireGuard is gaining an advantage by using GSO - Generic Segmentation Offloading. It creates a huge packet of 64 kilobytes and encrypts or decrypts it in one go. That way, overhead of initialising and calling cryptographic operations is being saved. If you want to maximise throughput that is a good idea to do.

However, things are again not so easy in reality. Sending such a large packet to the network adapter will require that it is being cut into many smaller packets. Usually of 1500 bytes. 64 kilobytes would result in 45 packets (1480 bytes payload and 20 bytes IP header per packet). Those will then block the adapter for quite some time, because they all will be sent in one go. Packets that should be prioritised like VoIP calls will have to wait.

So the high throughput that WireGuard claims to achieve is being bought by making other applications slower. This needs to go and the WireGuard team has already acknowledged it2.

But they went further.

The benchmark in the whitepaper shows a throughput of 1011 MBit/s.

Impressive.

Especially because the maximum theoretical throughput of a one Gigabit Ethernet link is 966 MBit/s at a packet size of 1500 bytes, minus 20 bytes IP header, 8 bytes UDP header and 16 bytes WireGuard header, another IP header in the encapsulated packet and another TCP header of 20 bytes. Where is this extra bandwidth coming from?

With jumbo frames enabled and the GSO advantages from above, the theoretical maximum at 9000 bytes frame size would be 1014 MBit/s. Normally, that won't be achievable in reality because there is more overhead. So I can only assume that the benchmark was made using even larger jumbo frames at 64 kilobytes with a theoretical maximum of 1023 MBit/s, which is supported by some network adapters. However, that is absolutely not practical in reality, but can be used with a direct link between the two systems executing the benchmark.

Since a VPN is usually built to connect two hosts over the Internet, which supports no jumbo frames at all, this is not a fair benchmark. It is an unrealistic lab setup which could never be applied in the real world.

Not even inside a data center I would expect to be able to transmit frames larger than 9000 bytes.

This benchmark is absolutely broken and I think that the author has discredited himself a lot for obvious reasons.

A last glimmer of hope

WireGuard's website talks a lot about containers. And clearly this is what this is designed for.

A simple and fast VPN, that takes nothing to set up and can be rolled out and configured by large orchestration tools like Amazon would have them for their cloud. They are using latest hardware features like AVX512 to be fast, but not tie themselves to x86 or another architecture.

They optimise for throughput and packets that are larger than 9000 bytes - that would be jumbo frames being encapsulated when two containers are talking to each other, or to backup, snapshot and deploy container images. Even dynamic IP addresses will not matter in this scenario.

Well done. Brilliant implementation and very lean and almost elegant protocol.

But it simply is not made for a world outside of the data centre in which you control everything. Otherwise you start making compromises on the design and implementation of the protocol.

Conclusion

It is not very hard for me to conclude that WireGuard is not ready - yet.

It has been drafted as a lightweight and fast solution to some problems of the existing solutions. Unfortunately it has sacrificed many features that will be relevant for many users and therefore will not be able to replace IPsec and OpenVPN.

For that we need to add at least IP address assignment, pushing configuration like DNS servers and routes. Obviously this needs cipher negotiation, too.

Security is my top priority and I have currently no reason to believe that IKE or TLS are intrinsically broken. Modern ciphers are supported in both of them and they have all been audited over decades. Just because something is newer, does not mean that it is better.

Interoperability is important when you connect with third parties that you do not control. IPsec is established as the de-facto standard and is virtually supported everywhere. It works. As it even looks, WireGuard might not even be compatible with itself in the future.

Cryptography breaks and things need to be replaced and updated.

Denying any of this and still wanting to use WireGuard to connect your iPhone to your home is a masterclass of putting your head in the sand.


  1. https://moderncrypto.org/mail-archive/noise/2016/000699.html 

  2. https://www.wireguard.com/todo/ 


Enhancements to our DNS Resolver

by Michael Tremer, February 12

Today, we have taken some important changes on our DNS Resolver into production. Having released support for DNS-over-TLS in 2018, we have now added TCP Fast Open and TLSv1.3.

Lightning Wire Labs is managing a DNS Resolver to provide an alternative to the large corporation who are trying to get the global DNS system under their control and use it for marketing purposes.

To not fall behind the technical development, we have now enabled some new features on our resolver to make it ready for the new DNS changes that are going to land with IPFire 2.25 - Core Update 141 very soon.

DNS-over-TLS

We are supporting DNS-over-TLS, for almost two years now, but with only few users. This is not surprising since IPFire did not DNS-over-TLS in the past, but this will now change.

We support TLSv1.3 and require at least TLSv1.2. ChaCha20-Poly1305, AES-GCM, Curve25519, and smaller ECSDA certificates are of course not missing either.

TCP Fast Open

For users who have an ISP that is filtering UDP queries or breaks DNSSEC in one way or the other, we are supporting TCP of course.

Since TCP requires a full 3-way handshake before any data can be sent, there is a small performance impact. To combat that, we now support TCP Fast Open which allows to send the DNS query with the first packet, even before the TCP connection is fully open.

This way, queries over TCP are just as fast as those over UDP.

How to use it?

The server is available at 2001:678:b28::54 and 81.3.27.54.

If you are using TLS, enter recursor01.dns.lightningwirelabs.com or recursor01.dns.ipfire.org as TLS hostname.


You will have seen that we have just release an announcement for testing the next release of IPFire - IPFire 2.25 - Core Update 141. The major release number has changed as well as a Core Update has been skipped. But why?

Rolling, rolling, rolling...

IPFire is a rolling release. There are very few, but some systems that have been updated all the way through since 2007, when the first release of IPFire 2 was published. Despite some bugs during the update process, it is never necessary to reinstall your firewall. And why would you do that? We have replaced the whole base system underneath it not only once, but countless times.

IPFire is a modern distribution with its roots somewhere in the past. However, sometimes we need to break things. On purpose. We have removed old crypto that is dangerously broken and we have removed features that virtually nobody has been using any more - simply because the world looked different in 2007 than in 2017.

Bump to IPFire 2.25

This time, the reason for bumping the release to 2.25 is that we have upgraded to GCC 9. A new compiler brings some new libraries and changes some other things that are not backwards-compatible. So add-ons compiled with the new compiler won't work on older systems. We create a new directory on the server with everything compiled with the new compiler every time this happens. It is as simple as that.

But where has Core Update 140 gone?

It does exist. You will see it when you upgrade from Core Update 139 or older. The update is a regular update as any other, but it does not bring any new files. It only performs cleanup tasks...

You might have seen that your /boot or / filling up. In the past, with a kernel size of about one megabyte we thought a /boot partition of 20 MB is plenty. Now, a single kernel is 20MB. In the meantime we have increase the size of that partition to 64MB and that again is not enough.

The same goes for the / partition. It used to be 2GB no matter how large the disk is. The remaining space was allocated to /var. Right now, we are shipping almost half a gigabyte of firmware blobs and about 1GB of system libraries. With a couple of add-ons, 2GB is very tight.

Core Update 140 therefore aggressively cleans up what we can get rid of. Old versions of system libraries, etc. For some users that might give you a little bit more breathing space, but ultimately there is a chance that you will have to reinstall to get the new partition layout.

On a newly installed system you will get a /boot partition of 128MB and a / partition as large as the remaining disk space. That will give you all the space you need for any add-ons.

With using the backup functionality, IPFire is quickly reinstalled within about 15 minutes and comes back up just as it has been before. If you have not already done so, I once again would like to recommend to upgrade from 32 bit to 64 bit if your hardware allows it. If not, consider getting new hardware.

The update process would automatically stop if there is not enough space available to install Core Update 141 and log a message.

Apologies for this, but it is hard to predict more than a decade of software development.