Decrypt https Traffic with Wireshark

Wireshark, an interesting open source network sniffer, can not only read network traffic, but can further decrypt https traffic provided you have the private key! Let’s learn more about decrypting https traffic using this tool.
To keep a network traffic sniffer from revealing login credentials, secure protocols such as https, ssh, sftp, etc, are used instead of their respective clear text protocols like http, telnet and ftp. Secure protocols encrypt traffic travelling between two end points; thus all the traffic available on the network is unreadable unless and until decrypted.

To understand secure protocols, let me try to explain the basics of encryption and decryption in simple terms.

Encryption and decryption
Encryption is a technique used to make readable (clear text) data unreadable (cipher text), typically using complex mathematical algorithms. Decryption is the exact reverse of the process of encryption.

The two main types of encryption
Symmetric key encryption: A single key is used to encrypt and to decrypt. Examples of the symmetric key algorithm are the advance encryption standard (AES), data encryption standard (DES), etc. The typical length of a key is 128 bytes onwards, making encryption-decryption fast. However, the key must be known to both sides for successful operation.

Asymmetric key encryption: A key pair is used for encryption and decryption. Text encrypted by one key can be decrypted only by the second key and vice versa. Thus, the same key cannot perform encryption and its corresponding decryption. Examples of the asymmetric key encryption algorithm are Rivest, Shamir Adleman (RSA), ElGamal, Elliptic curve cryptography, etc.

Some important points about asymmetric key encryption are:

1. Asymmetric key encryption and decryption is slow compared to symmetric key encryption. For example, RSA encryption with a 1024-bit key is about 250 times slower compared to AES encryption with a 128-bit key.
2. One key (a private key) is kept confidential and the other key (the public one) is distributed.
3. Text encrypted by the sender’s private key can be decrypted by the sender’s public key. Thus, practically anybody can decrypt this since this key is distributed. However, please note that this ensures the authenticity of the sender—the private key is held confidential by the corresponding sender.
4. Text encrypted by a receiver’s public key can be decrypted only by the receiver’s private key; thus messages encrypted using the public key of the receiver can be decrypted only by the corresponding receiver with the corresponding private key. The https protocol uses this feature.

The https protocol

As discussed in earlier Wireshark articles, it is definitely possible to sniff (eavesdrop) network traffic leading to compromise of sensitive information such as login names, passwords, etc. If clear text protocol is in use. The https protocol is used to guard against possibility of clear text information between client (browser) and server getting compromised.

The https protocol uses the secure socket layer (SSL) or its successor, the transport layer security (TLS) to encrypt traffic between the Web server and the client (browser). SSL, in turn, uses an asymmetric key RSA algorithm for encryption and decryption. When a user sends a browser request to an https website, encrypted communication is established as follows:

1. The browser sends an https request for a secure session towards the server’s TCP 443 port (or on a different port for servers running on non-standard ports).
2. The server sends back its digital certificate – which contains its public key.
3. The browser authenticates this certificate with the certifying authority (CA) – the provider of the digital certificate, before continuing further. If the certificate is not authenticated by the CA or self generated, an error like ‘This connection is untrusted’ is received. This could be a man-in-the-middle (MITM) type of attack, where the attacker makes two independent connections – one with the user and the other with the server, to capture the complete traffic in decrypted format. The user should abort such a connection, or may proceed further only if the origin of the certificate is assured.
4. The client generates a random sequence of characters to be used as a symmetric key.
5. The client encrypts these characters using the server’s public key and sends it to the server, thus ensuring that only the corresponding server (or private key) can decrypt it.
6. The server decrypts the symmetric key using its private key.
7. Now, this set of characters is known only to the client and the server. They are used as the symmetric key for encrypting all further traffic between client and server.
8. The client and the server start encrypted communication using this symmetric key.

You may wonder about the jugglery involved in using a ‘hybrid’ technique. This is done merely to improve speed since the symmetric key provides much faster encryption and decryption.

Now, we have sufficient information to understand the capturing and decrypting of https traffic using Wireshark. Start Wireshark and browse any https website–you will definitely notice that the data part of the capture is encrypted. To decrypt data, we must have the private key of the https server.

So, let us proceed with the following steps. This article uses VMware ESXi 5.0 as the target system to decrypt https traffic using Wireshark.
A. Enable VMware SSH:

• Use the VMware console keyboard
• Press the F2 key
• Provide your login credentials
• Select troubleshooting options
• Enable ESXi shellB.

B. Copy the private key file using the SSH command:

ssh root@vmwareesxi:/etc/vmware/ssl/rui.key .

C. Start Wireshark, and let traffic capture begin – you may use capture filter for the VMware ESXi host.
D. Open the browser, then open the VMware ESXi https Web page and browse a few links.
E. Capture traffic and stop.
F. Now, apply the private key to the SSL traffic from this host (as shown in Figure 1):

• Edit – Preferences
• Protocols
• SSL
• RSA keys list: Edit
• Insert the required values: IP address, port number, protocol and key file path+name
• Textbox2 – RSA private key format – RSA key does not require password. The password field is for the file in the PKCS#12 format.

G. Start the browser and open https://vmwareesxi

• Right-click any packet and look at what’s shown under ‘Follow TCP stream’ – this will show the encrypted https traffic as seen in Figure 2.
• Use ‘Follow SSL stream’ to view the traffic in the decrypted format as depicted in Figure 3.
  If all the steps are followed correctly, the captured traffic will be seen in a decrypted format.

Protecting against https RSA decryption

This decryption method is successful only if RSA is used to encrypt and decrypt the symmetric key – which is sent across from the client to the server via the network (in encrypted form). So is there a way out? Yes, definitely.
The Diffie Hellman (DH) key exchange method plays a vital role in ensuring the security of https. DH creates a shared secret between two systems without actually transmitting the ‘encryption key’ across the wire. However, DH is only a key exchange method; it does not authenticate the server nor does it encrypt the data.

By combining RSA, which provides server authentication and DH (which enables the creation of a shared encryption key at the client and the server ends), the result is excellent protection against https decryption even if a private key is available.
RSA and DH in combination are used by various open source distros such as IPCop and OpenFiler, thus making sure that a compromised private key is not sufficient for successful decryption.

Diagnosing decryption related issues

1. Make sure that you capture all the traffic right from the first packet in order to enable decryption. A point to ponder over is that the encryption key is transmitted in encrypted format between client and server before the actual data transfer. If you miss this traffic, the key will not be available and further decryption will not be possible.
2. Use Wireshark decryption logs, which can be enabled by specifying the log file name under Edit – Preferences – Protocols – SSL – (Pre)-Master-Secret. This file can then be checked to find the decryption status. Please see Box 3 and Box 4 for unsuccessful and successful decryption logs.

In this article, I have tried to describe various encryption techniques, DH and RSA only to enable a basic understanding of the topics. For those who want to know more, please search the Web, especially Wikipedia.

An interesting note: Websites using the https protocol are called ‘secure’ websites only to the extent that the communication between browser and Web server is encrypted and, further, the server is authenticated by CA – but nothing beyond that! It does not guarantee that the Web applications or the site is secure enough.

References
[1] DH with RSA: http://security.stackexchange.com/questions/35471/is-there-any-particular-reason-to-use-diffie-hellman-over-rsa-for-key-exchange
[2] One of the commercial products affected by Heartbleed was VMware ESXi (versions 5.0, 5.1 and 5.5) – its security update was released immediately. Read VMware’s advisory for Heartbleed at: http://www.vmware.com/security/advisories/VMSA-2014-0006.html
[3] OpenSSL Heartbleed security notice and update instructions for Ubuntu: http://www.ubuntu.com/usn/usn-2165-1/