Transport Layer

We all know TCP and UDP. Some of you may know SCTP, and maybe QUIC.

But what makes Fenrir different?

To understand it, let’s try to look at a couple of headers.

UDP

The simplest one.

Just a couple of ports and the checksum of the payload. Checksum is optional, you can use all zeroes if you don’t need it

Very simple, 8 bytes total, no features whatsoever.

TCP

Now we go into something a bit more advanced.

Now there is lots that can be said (and won’t be said. there’s no space here). Thanks to the sequence number we can reconstruct a full stream, the acknowledgement lets us know what the other part has received, there are flags for connection setup and teardown, some optional stuff for future ideas…

Then there’s the urgent pointer, which is a kind of application-level qos.

The “Options” field is actually commonly used in connection setup to modify the initial congestion window, and for a lot of other things.

Although the sequence number is 32 bits (and can wrap), the window size is only 16 bits. This can lead to slow connections for satellite communications: The [Bandwidth-Delay-Product] (http://en.wikipedia.org/wiki/Bandwidth-delay_product) tells us that on a (not uncommon) 500ms RTT link with standard TCP we can only get about 128kbytes/sec out of a connection.

This header has a couple of limitations:

• we are not independent from the IP layer, as the TCP uses both ports and the ip to track the connection.
• acks, window advertising must be done with each packet.
• the urgent pointer is a poor qos method.
• you have a stream of bytes, there’s no way to know where a message begins or ends

And of course, there’s no security at all.

SCTP

This was a big improvement on TCP.

The header looks like this:

Much simpler. For now. This design is very similar to what we will find in Fenrir. The “chunk” section can be repeated, so that multiple chunks can be sent in a single packet.

For everything that we have to do, we simply put a new header in a “chunk” and add all we want. So a data header is another 16 bytes, then we have INIT chunks, SACK and a lot more.

The idea of managing chunks is a new and very powerful idea from SCTP, but I think they made the headers a little more complicated than it needed to be. That, and the fact that it has to run on top of IP, so it must have a kernel space implementation, pretty much blocked its diffusion.

This protocol also introduces the possibility of handling message boundaries directly, so that the user does not have to do trick to guess the start/end of his/her messages.

SCTP is the first big protocol to support both reliable and unreliable delivery, but it still as no security, and the header in the chunks (not shown in picture) can be pretty complex.

QUIC

The next step is QUIC.

The header should look something like this (not much information I’m afraid…)

The first thing that we notice is the variable-length fields. This means that we are throwing away any alignment optimization (not a big deal) but it might also make parsing a little more complicated, especially with the optional fields.

Just like SCTP, QUIC supports streams (not shown here, can’t find what the header looks like), but keeps the “0” stream for connection management.

The improvement over SCTP is that the headers are extremely small (even though this is an incomplete version), that the handshake includes TLS exchange of data, but -more importantly- we finally get rid of the source/destination ports, and the only thing that identifies a connection is its GUID. This is a great advance we keep in Fenrir.

This is actually due to two reasons:

• avoiding port makes the protocol really independent from the lower IP protocol.
• QUIC still needs UDP underneath

A completely new feature of this protocol is the FEC (Forward Error Correction). Basically it works like a raid, where every 2 packets or so, you send an other packet which is the xor of the previous 2 packets. This lets you lose any of the two packets and still reconstruct it without additional retransmission.

The Fenrir Way

Now that you have seen the most of what can be included in a packet, and what can be left optional, let’s look at how we do things in Fenrir.

The packet looks like this:

And that’s it.

The padding and its position will be discussed in an other article, for now just try to look at its simplicity:

• The connection id is the only thing we need in cleartext. it’s 4 bytes, meaning each computer can have a maximum of 4294967296 connections. If you think your computer can handle that much, grab a calculator and try to imagine how much space you would waste with just a couple of 64-bit pointers to track only the most basic things.
• The padding is at the beginning of the ciphertext, to make analysis and known-plaintext attacks even more difficult, but forget it for now.
• Next we have the streams. Each stream has:
  • an identifier (65k streams should be enough for everybody, just like ram)
  • the length of the user data, so we can put more streams in a single packet.
  • a couple of flags to know where the start/end of the user message is.
  • a stream counter to know how to reconstruct the sent data,

As with QUIC, we still need to run on top of UDP. but only to bypass firewall restrictions. The goal is to move come code into kernel space, but just enough to dispatch the packet to the right userspace application. Which means that the kernel space code will be extremely short and simple, and anything that can go wrong will be likely into userspace only.

But what about the connection management?

There really is no need to keep all that stuff in the main packet. Instead of having different stream types, and complicating things like SCTP, we simply have to choose one stream id. That stream id will be used to carry whatever information we want, so there’s no need to track anything else.

This is a bit like the QUIC version, except is more explicit (I still have trouble understanding how QUIC really works), we do not have always the same stream id for connection management, and the header is always 8 bytes.

ACKs can be sent only once in a while to save bandwidth, so we do not need space for them at every packet (think TCP).

The SCTP verification tag is used to be sure that we are talking to the right person. we have crypto for that.

The version of the protocol does not change with every packet, so the “QUIC version” seems pretty useless, if not for handshakes. So we track that only in the handshake. FEC data does not change at every packet either, so there’s no need for a full byte there. There are also different ways to do FEC, what is good today might not be good tomorrow, so we need to be able to extend it. Let’s decide what to use during handshake. Same for checksum.

The fun part is that we don’t really need a lot of fixed space for this in the handshake, we just need to decide what to use. The rest can be connection-dependent

Alignment

As you may have noticed, all headers, except for QUIC, are aligned to 4 bytes.

This was necessary, since in older processors misaligned data was really bad This is not so true today, but it can still cause more cache miss and is can still slow things down.

QUIC seems to completely ignore this. In Fenrir we have something somewhere in the middle: the only thing that can cause misalignment is the padding, right at the beginning of the ciphertext.

Worry not though, as there are options during the handshake to let you decide whether Fenrir should align data or not.

Also, having a fixed length header (the stream header) surely helps to make parsing quicker and easier.

Is this all?

Well, there’s much more, but it includes how this interacts with encryption, too. I’ll probably edit this post in the future.

Feel free to ask clarifications :)

-Luke