HTTPS is a secure version of the HTTP. The "S" means that it is secured by Secure Socket Layer/Transport Layer Security (SSL/TLS). It is used when you connect to your bank account, webmail service, or any other service that runs over HTTP and requires security.
In this recipe, we will see how it works and what can fail when we are using HTTPS communications.
Port mirror to the suspected device or link that forwards traffic from several devices, and start capture. HTTPS works with the TCP port 443, and this is what we should watch.
To monitor HTTPS sessions, perform the following steps:
- HTTPS session establishment can be done in four or five steps. It is described in the How it works... section of this recipe.
- Watch the order of the packet in the session establishment, and make sure the messages you get are according to the order shown in the following figure (in brackets you'll see what should be shown in the packet):
- There are some common errors that are described in
RFC 2246:- close_notify: This message notifies the recipient that the sender has finished sending messages on this connection. The session can be resumed later.
- unexpected_message: This alert is returned if an inappropriate message was received. This is a critical error that can indicate a bad implementation on one of the sides.
- bad_record_mac: This alert is returned if a record is received with incorrect Message Authentication Code (MAC). This is a critical error that can indicate a bad implementation on one of the sides.
- decryption_failed: This alert is returned if a TLS Ciphertext was decrypted in the wrong way. This is a critical message that can indicate a bad implementation on one of the sides.
- record_overflow: This alert is returned if a TLS Ciphertext record was received with a length longer than the allowed length. This is a fatal error, and it usually indicates a bad implementation on one of the sides.
- decompression_failure: This message indicates that a decompression function received a wrong input. This is a critical error that can indicate a bad implementation on one of the sides.
- handshake_failure: Reception of this alert message indicates a negotiation error that occurred when the sender was unable to negotiate the set of security parameters, given the options available. This is a critical error that can indicate a bad implementation on one of the sides.
- bad_certificate: This is a certificate error. It occurs when a certificate is corrupt, contains signatures that were not verified correctly, or any other error.
- unsupported_certificate: This indicates that the received certificate was not of the supported type.
- certificate_revoked: This indicates that a certificate was canceled by its signer.
- certificate_expired: This indicates an invalid certificate or a certificate that has expired.
- certificate_unknown: This tells that a certificate was not accepted due to an unspecified reason.
- illegal_parameter: This tells that a field in the handshake process was out of range or inconsistent with other fields. This is a critical error that can indicate a bad implementation on one of the sides.
- unknown_ca: This indicates that a valid certificate was received, but was not accepted because it couldn't be matched with a known, trusted CA. This is a critical error, and should be checked with the certificate issuer.
- access_denied: This tells that a valid certificate was received, but it was not approved by the access control of the receiver, and the sender decided not to proceed with negotiation.
- decode_error: This tells that a message was too long and, therefore, could not be decoded. This is a critical error that can indicate a bad implementation on one of the sides.
- decrypt_error: This indicates that a handshake cryptographic operation failed, including the ones that failed due to signature verification, key exchange, or validation of a finished message.
- export_restriction: This tells that a negotiation which is not compliant with export restrictions was detected.
- protocol_version: This tells that the protocol version which the client has attempted to negotiate is not supported.
- insufficient_security: This is returned when a negotiation has failed because the server required ciphers with higher security than those supported by the client.
- internal_error: This is an internal error not related to the peer of the connection.
- user_canceled: This tells that the handshake was canceled for a reason other than a protocol failure.
- no_renegotiation: This is sent by the client or the server in response to a hello request after the initial handshaking.
In each one of the failures mentioned, the connection will not be established.
SSL and TLS are protocols that secure a specific application, for example, HTTP, SMTP, Telnet, and others. SSL Versions 1, 2, and 3 were developed by Netscape in the mid 90s for their Navigator browser, while TLS is a standard from the IETF (RFC 2246, RFC 4492, RFC 5246, RFC 6176, and others). TLS 1.0 was first introduced in RFC 2246 in January 1999 as an upgrade of the SSL Version 3.0 (third paragraph at http://tools.ietf.org/html/rfc2246).
The TLS handshake protocol involves the following procedures for establishing a TLS connection:
- Exchange hello messages to agree on the algorithms to work with, and exchange random values for the key generation.
- Exchange the necessary cryptographic parameters to allow the client and the server to agree on a premaster secret key.
- Exchange certificates and cryptographic information to allow the client and server to authenticate each other.
- Generate a master secret key from the premaster secret and exchanged random values.
- Allow the client and server to verify that their peer has calculated the same security parameters and that the handshake occurred without being tampered with by an attacker.
These procedures are performed in the following order:
- Select cryptographic algorithms:
- The Client Hello message (marked as 1 in the following screenshot)
- The Server Hello message (marked as 2 in the following screenshot)
- Authenticate the server and exchange key (marked as 3 in the following screenshot).
- Authenticate the client and exchange key (marked as 4 in the following screenshot).
- Complete the handshake (marked as 5 in the following screenshot).
Let's see how it works. In the preceding screenshot, we see how TCP SSL/TLS establishes a connection (packets 157-158-159) and packet 160 starts the TLS handshake. Let us see the details:
- In packet
160, the client sends a Client Hello message that starts the negotiation. - The server answers with a Server Hello message.
- The server sends a certificate to the client.
- The client takes the certificate and generates a premaster key.
- The server creates the master key, and the conversation begins. This is an optional message.
Tip
This refers to a mechanism (defined in
RFC 4507) that enables the TLS server to resume sessions and avoid keeping the per-client session state. The TLS server encapsulates the session state into a ticket and forwards it to the client. The client can subsequently resume a session using the obtained ticket. This happens, for example, when you re-open a connection to your webmail account (Gmail, Hotmail, and so on) and is common to these scenarios.
Communication between the client and the server will start after step 4 or 5.
Let's look at each one of them:
In step 1, packet 160 is a Client Hello message which is the first packet in the TLS handshake. Some of the parameters that we can see are shown in the following screenshot:
- The area highlighted as 1 shows that the content of the packet is a handshake (
ssl.record.content_type == 22). - The area highlighted as 2 shows that the packet is a Client Hello message sent from the client to the web server. This message starts the handshake.
- The area highlighted as 3 shows the highest SSL and TLS version supported by the client.
- The area highlighted as 4 shows the client time that will be used in the key generation process.
- The area highlighted as 5 shows the random data that is generated by the client for use in the key generation process.
- The area highlighted as 6 shows the ciphers supported by the client. The ciphers are listed in order of preference.
- The area highlighted as 7 shows the data compression methods that are supported by the client.
As shown in the following screenshot, Packet 162 is a Server Hello message, which includes the following details:
- The area highlighted as 1 shows that the content of the packet is a handshake (
ssl.record.content_type == 22). - The area highlighted as 2 shows the TLS version that will be used in this session.
- The area highlighted as 3 shows that the packet is a Server Hello message sent from the server to the client.
- The area highlighted as 4 shows the server time used in the key generation process.
- The area highlighted as 5 shows the random data that is generated by the server for use in the key generation process.
- The area highlighted as 6 shows the cipher suite to be used in this conversation. It is chosen from the list of ciphers sent by the client.
- The area highlighted as 7 shows the data compression method that will be used for the session.
The next packet is the response from the server issuing a certificate:
- The area highlighted as 1 shows that the server sends the Certificate command, which includes the server's certificate. By clicking on the (+) sign on the left of this line and digging into the details, you will see the certificate issuer, validity time, algorithm, and other data.
- The area highlighted as 2 shows that the server sends the Server Key Exchange command (usually Diffie-Hellman), including the required parameters (public key, signature, and so on).
- The area highlighted as 3 shows that the server sends the Server Hello Done command. This command indicates that the server has completed this phase of the SSL handshake. The next step is the client authentication.
The next packet (packet 165 in this example) is the response from the server, issuing a certificate.
- The area marked as 1 shows that the client sends the Client Key Exchange command. This command contains the premaster secret that was created by the client and was then encrypted using the server's public key. The symmetric encryption keys are generated by the client and the server, based on the data exchanged in the client and server hello messages.
- The area marked as 2 shows that the client sends the Change Cipher Spec notification to the server. This is done in order to indicate that the client will start using the new session keys for hashing and encryption.
The last step is when the server sends a New Session Ticket to the client, and it will look like the example in the following screenshot:
I've been asked several times if it is possible to decrypt sessions that are encrypted with SSL/TLS. Well it's possible if you have the private key, which is provided to you by the server you connect to; and to get it is not an easy thing to do.
There are methods to hijack this key, and in some cases they will work. It is not an obvious thing to do, and in any case it is not in the goal of this book. If you get the private key, you simply add it in the protocol list in the preferences window and continue from there. Additional details about this feature can be obtained from http://wiki.wireshark.org/SSL, as well as from many other websites and blogs.