After some time, I was finally able to design a way for Fenrir to handle multicast transmissions.
It kinda easy once you think about it.
We will use two transmissions. One will be the multicast one. The other will be a normal unicast transmission, on which we will agree on things like key renewal, and where we can ask for more repair data (RaptorQ), so that we can have reliable multicast transmissions (HA! top that if you can!).
…But all of this comes after we have defined multicast, which brings its own set of problems.
Multicast -per se- is not a big problem.
It is handled mainly by the network protocol (IP), which can deliver the same packet to multiple destinations. We don’t have to do anything for that.
But contrary to the IP layer, we need to maintain a session between packets, and we have to handle all security related concerns. Here lie the problems.
We could naively think that each multicast packet is completely independent from the others. But this will just force the developer to introduce a counter in his/her protocol to be able to at least reorder the packets. This would re-introduce the session we took away.
Making the developer reimplement parts of the OSI layer, when he should just work at layer 7 is stupid. We need to keep a session between packets. We also need a session for the crypto stuff, but more on that later.
The connection ID
In Fenrir the connection id is decided by the receiving party. This means that we actually have 2 connection id per connection, but since each one is used only for receiving, both can use the full 32 bit space for connections. This however can not be done with multicast. We would have to synchronize multiple receivers on the same connection id. Obviously error prone, and it does not even feel safe.
We could somehow make the connection id track the sender connection instead of the receiver. But now we have to somehow synchronize multiple servers to be sure they do not use the same connection id. Or use big connection ids, so that the probability of this happening is really low. It’s the same problem as before, we have just moved it to the senders. And it’s not safe either.
But what do we mean by safe? Let’s enter into the crypto (it will be easy) to understand what is fundamentally different in multicast security.
Let’s assume we have solved our connection id problems. We have already exchanged keys, and now we can use the usual layer of symmetric encryption plus the usual HMAC if the cypher is not an AEAD one.
…then again, maybe not? The usual way this is done is that we generate a couple of keys from the key exchange, both parties have the keys, and now both of them can send and receive with those keys.
See where this is going? Consider the following example: One server is sending a multicast transmission to two clients. This means that the same keys must be shared with both clients. Well, duh. But this also means that now one of the two clients can create packets that will be accepted by the other clients! After all, both know everything and exactly the same things about the connection!
Truth is, few bothered to implement safe multicast transmissions. Multicast connections have always been unauthenticated and unencrypted, or encryption was used only when the receiver could not freely communicate with others (think about TV broadcasts).
The solution is simple, and forced: multicast transmissions must use private/public key signing. For every packet.
Ok, but why the digression from the connection id?
Because now we have the perfect multicast connection id! The public key itself will be the id for the connection. Of course, public keys can be quite long, depending on the algorithm used. But we can generate a much shorter hash that will act as an index, so that the receiver will immediately know which public key to use for verification.
In Fenrir we will reserve one connection id for multicast transmissions. When the multicast connection id is found, we will find the next 128 bits as the SHA3 of the public key. It is yet undecided whether we could shorten this to 64 bits. SHA3 can manage this, and the number of multicast connections is actually limited to the number of unicast connections, since each multicast connection will be aided by a unicast connection. This means that we could shorten the multicast connection id to 32 bits. But this will obviously generate too many clashes. We will probably stick with 64 bit hashes, and make sure that multiple keys can be tried in the case of a hash clash.