ed2k Obfuscation

Journal started Oct 28, 2006

It behooves me to note that eMule has figured out a method of obfuscating their protocol(s) so that nasty ISPs won't be able to block p2p by scanning the network's content for eMule streams. No specs seem to be out in public, so I'm going to see what I can glean from examining the source.

First off, there are server flags, depending on if you're connected to the server via TCP or UDP. Things such as how the server supporting compression over TCP is 0x1 and the server supporting the getsources request over UDP is 0x1. Added to those flags, is by TCP connections, 0x400: server supports TCP obfuscation. By UDP connections 0x200: server supports UDP obfuscation, 0x400: server supports TCP obfuscation. So obviously it seems if you're communicating with a server and it has 0x400 in the flags, and you're connecting via TCP, you can start negotiating obfuscation. Same with UDP except it's 0x200.

The server now has a 'tcp obfuscation port', that is a uint16 port that is presumably different from the non-obfuscated port. It also has a UDP obfuscation port of a similar name. Why eMule didn't simply implement TLS I have no idea, but they seem to think that dedicated obfuscation ports are somehow better.

One final note about the server, it seems that the UDP 'getsources' command uses TCP somehow, so when the server checks over UDP for getting sources, it should check the for the TCP obfuscation flag before using TCP to get those sources.

Now, when a client connects via UDP to the server it receives the server's flags. If those flags contain the 0x200 bit then the packet's size will be greater than or equal to 40. After the 32nd byte there will be a uint16, the UDP obfuscation port. After the 34th byte, another 2 bytes for the TCP obfuscation port. After that, a uint32, 4 bytes, that is the encryption session key. Any bytes after that (greater than 40) are ignored for now.

I should note that this ^^^ packet with the information about the server is not encrypted or obfuscated in any way, so the lords might be able to exploit that and block servers based on their initial handshake. If you can get that handshake across though, you have the obfuscated ports, and the obfuscated key. And the obfuscation flag.

The above handshake comes during the global server status result message, indicated by opcode 0x97, the second byte of the packet. The first byte is always 0xe3, to mark the packet as an ed2k packet. If even during an obfuscated session, an obfuscated packet comes in with 0x97 as the second byte, the client is expected to check for obfuscation support (it may have been removed) and update their obfuscation key to what bytes 36-40 report.

The above handshake may be called a 'crypt ping reply' which makes sense because you're testing the server's ability to encrypt, or obfuscate. At your leisure (1st packet sent) clients should send a crypt ping in order to see if that server supports obfuscation. A crypt ping is a packet that starts with a uint32, a challenge number that must be verified by the server. After those 4 bytes, randomly between 0 and 16 bytes of random data is added as padding. Here's the odd thing now: instead of sending this packet to the server's UDP port, you send it to the server's UDP port plus 12. Presumably the server will be listening at that port for crypt pings. Why aren't we using TLS again? Once a server receives this crypt ping, it is expected to send you an opcode 0x97 packet as described above. eMule gives it 20 seconds to send that packet before giving up and assuming the server doesn't support obfuscation.

One additional note, the server's going to send 0x97 with obfuscation not supported if you have lowID (that is, you don't have a clear public IP address).

Ok, now you have the key and the server supports encryption. Now you can consider it an 'encrypted datagram socket'. (We're not onto TCP just yet.) Fill the following structure:

struct {
uint32 encryption_key;
byte magic = 0x6b;
uint16 random_data;
};

Also this is the form of an RC4 key:

struct RC4_Key_Struct{
uint8 abyState[256];
uint8 byX;
uint8 byY;
};

Save the above random_data for later. Now take the MD5 (16 bit binary) hash of that, and get an RC4 key with that. (This is different from your above encryption key.) You're ready to prepare your encrypted packet. (TLS, please, TLS over UDP, anything!) The first byte is a random number that is NOT the 'ed2k protocol' indicator (0xe3). If it is, the server will think it's an unobfuscated packet and botch the whole thing up. After that, append the two byte random_data you saved above. After that, everything should be RC4 encrypted, using the key you calculated above. The server knows the encryption_key and since you added the random_data, it can regenerate the RC4 key. RC4 encrypt, and append, the following to your packet:

Then you're done. Send it like you would an ordinary packet, but confident that it's so screwed up only the server's going to be able to make sense of it. Note that your packet will be 8 bytes longer than unobfuscated because of the extra keys and magic you added into it.

To receive an encrypted packet, you do the above, just in reverse! Some things to consider: if the initial magic is not a type you recognize, try to decrypt it, but recognize it might just be accidental garbage. The server will send you a packet composed of the following: a uint16 of random_data, and RC4 encrypted data after that. To get the RC4 key you use the MD5 hash of the following structure:

struct {
byte[16] your_user_hash;
byte magic = 0xA5
uint16 random_data;
};

What should result from unencrypting the RC4 encrypted data starting from the 3rd byte of the packet is the 'udp sync server magic' or 0x13EF24D5. After the 7th byte, it should decrypt to the padding length, and then that many bytes of padding. If the padding goes past the packet length, you know you've got a bad packet here. (With the 0x395F2EC1 though, you only get one bogon every 256 garbage packets.) Finally after the padding, you should be able to decrypt the normal packet data you originally wanted at the tail end of this packet.

Obfuscating over UDP to clients is very similar. The only difference is you use the 'udp sync client magic' or 0x395F2EC1 and how you prepare the data for the RC4 key. Sending:

struct {
byte[16] their_user_hash;
uint32 your_ip_address;
byte magic = 0x5b;
uint16 random_key;
};

And receiving:

struct {
byte[16] your_user_hash;
uint32 their_ip_address;
byte magic = 0x5b;
uint16 random_key;
};

The TCP handshake is oodles more complicated, so I'll just paste what I snarfed from EncryptedStreamSocket.cpp:

Basic Obfuscated Handshake Protocol Client <-> Client:
-Keycreation:
- Client A (Outgoing connection):
Sendkey:	Md5()  21
Receivekey: Md5() 21
- Client B (Incomming connection):
Sendkey:	Md5() 21
Receivekey: Md5()  21
NOTE: First 1024 Bytes are discarded

- Handshake -> The handshake is encrypted - except otherwise noted - by the Keys created above -> Handshake is blocking - do not start sending an answer before the request is completly received (this includes the random bytes) -> EncryptionMethod = 0 is Obfusication and the only supported right now Client A: Client B: -> The basic handshake is finished here, if an additional/different EncryptionMethod was selected it may continue negotiating details for this one

- Overhead: 18-48 (~33) Bytes + 2 * IP/TCP Headers per Connection

- Security for Basic Obfusication: - Random looking stream, very limited protection against passive eavesdropping single connections

- Additional Comments: - RandomKeyPart is needed to make multiple connections between two clients look different (but still random), since otherwise the same key would be used and RC4 would create the same output. Since the key is a MD5 hash it doesnt weakens the key if that part is known - Why DH-KeyAgreement isn't used as basic obfusication key: It doesn't offers substantial more protection against passive connection based protocol identification, it has about 200 bytes more overhead, needs more CPU time, we cannot say if the received data is junk, unencrypted or part of the keyagreement before the handshake is finished without loosing the complete randomness, it doesn't offers substantial protection against eavesdropping without added authentification

Basic Obfuscated Handshake Protocol Client <-> Server: - RC4 Keycreation: - Client (Outgoing connection): Sendkey: Md5() 97 Receivekey: Md5() 97 - Server (Incomming connection): Sendkey: Md5() 97 Receivekey: Md5() 97

gnutls_transport_set_ptr (session, (gnutls_transport_ptr_t) sd);
ret = gnutls_handshake (session);
if (ret < 0)
{
// Note if an attacker spoofs this, we really should fail,
// but we must support unencrypted ed2k so...
/*
fprintf (stderr, "*** Handshake failed\n");
gnutls_perror (ret);
goto end;
*/
supportsObfuscation = false;
} else {
supportsObfuscation = true;
obfuscated = true;
//etc...
}