OCSP Stapling on Nginx

When connecting to a server, clients should verify the validity of the server certificate using either a Certificate Revocation List (CRL), or an Online Certificate Status Protocol (OCSP) record. The problem with CRL is that the lists have grown huge and takes forever to download.

OCSP is much more lightweight, as only one record is retrieved at a time. But the side effect is that OCSP requests must be made to a 3rd party OCSP responder when connecting to a server, which adds latency and potential failures. In fact, the OCSP responders operated by CAs are often so unreliable that browser will fail silently if no response is received in a timely manner. This reduces security, by allowing an attacker to DoS an OCSP responder to disable the validation.

The solution is to allow the server to send its cached OCSP record during the TLS handshake, therefore bypassing the OCSP responder. This mechanism saves a roundtrip between the client and the OCSP responder, and is called OCSP Stapling.

The server will send a cached OCSP response only if the client requests it, by announcing support for the status_request TLS extension in its CLIENT HELLO.

Most servers will cache OCSP response for up to 48 hours. At regular intervals, the server will connect to the OCSP responder of the CA to retrieve a fresh OCSP record. The location of the OCSP responder is taken from the Authority Information Access field of the signed certificate.

What is OCSP Stapling

OCSP stapling is defined in the IETF RFC 6066. The term “stapling” is a popular term used to describe how the OCSP response is obtained by the web server. The web server caches the response from the CA that issued the certificate. When an SSL/TLS handshake is initiated, the response is returned by the web server to the client by attaching the cached OCSP response to the CertificateStatus message. To make use of OCSP stapling, a client must include the “status_request” extension with its SSL/TSL Client “Hello” message.

OCSP stapling presents several advantages including the following:

• The relying party receives the status of the web servers certificate when it is needed (during the SSL/TLS handshake).
• No additional HTTP connection needs to be set up with the issuing CA.
• OCSP stapling provides added security by reducing the number of attack vectors.

Read one of the following links for more information on OCSP and OCSP stapling.

Requirements

You need at least nginx 1.3.7 for this to work. This is not available in the current Ubuntu LTS releases (12.04), it has 1.1.19 and on CentOS you need EPEL or the official repositories. However, it is easy to install the latest version of nginx.

You also need create a firewall exception to allow your server to make outbound connections to the upstream OCSP’s. You can view all OCSP URI’s from a website using this one liner:

OLDIFS=$IFS; IFS=':' certificates=$(openssl s_client -connect google.com:443 -showcerts -tlsextdebug -tls1 2>&1 </dev/null | sed -n '/-----BEGIN/,/-----END/ {/-----BEGIN/ s/^/:/; p}'); for certificate in ${certificates#:}; do echo $certificate | openssl x509 -noout -ocsp_uri; done; IFS=$OLDIFS

It results for google.com in:

http://clients1.google.com/ocsp
http://gtglobal-ocsp.geotrust.com

nginx Configuration

Add the below configuration to your https (443) server block:

ssl_stapling on;
ssl_stapling_verify on;
resolver 8.8.8.8 8.8.4.4 valid=300s;
resolver_timeout 5s;

For the OCSP stapling to work, the certificate of the server certificate issuer should be known. If the ssl_certificate file does not contain intermediate certificates, the certificate of the server certificate issuer should be present in the ssl_trusted_certificate file.

My certificate for raymii.org is issues by Positive CA 2. That certificate is issued by Addtrust External CA Root. In my nginxssl_certificate file all these certificates are present. If that for you is not the case, create a file with the certificate chain and use it like so:

  ssl_trusted_certificate /etc/ssl/certs/domain.chain.stapling.pem;

Before version 1.1.7, only a single name server could be configured. Specifying name servers using IPv6 addresses is supported starting from versions 1.3.1 and 1.2.2. By default, nginx will look up both IPv4 and IPv6 addresses while resolving. If looking up of IPv6 addresses is not desired, the ipv6=off parameter can be specified. Resolving of names into IPv6 addresses is supported starting from version 1.5.8.

By default, nginx caches answers using the TTL value of a response. The (optional) valid parameter allows overrides it to be 5 minutes. Before version 1.1.9, tuning of caching time was not possible, and nginx always cached answers for the duration of 5 minutes.

Restart your nginx to load the new configuration:

service nginx restart

And it should work. Let’s test it.

Testing it

Fire up a terminal and use the following OpenSSL command to connect to your website:

openssl s_client -connect example.org:443 -tls1 -tlsextdebug -status

In the response, look for the following:

OCSP response:
======================================
OCSP Response Data:
    OCSP Response Status: successful (0x0)
    Response Type: Basic OCSP Response
    Version: 1 (0x0)
    Responder Id: 99E4405F6B145E3E05D9DDD36354FC62B8F700AC
    Produced At: Feb  3 04:25:39 2014 GMT
    Responses:
    Certificate ID:
      Hash Algorithm: sha1
      Issuer Name Hash: 0226EE2F5FA2810834DACC3380E680ACE827F604
      Issuer Key Hash: 99E4405F6B145E3E05D9DDD36354FC62B8F700AC
      Serial Number: C1A3D8D00D72FCE483CD84759E9EC0BC
    Cert Status: good
    This Update: Feb  3 04:25:39 2014 GMT
    Next Update: Feb  7 04:25:39 2014 GMT

That means it is working. If you get a response like below, it is not working:

OCSP response: no response sent

You can also use the SSL Labs test to see if OCSP stapling works.

Sources

• nginx documentation for resolver
• nginx documentation for ssl_stapling

OCSP Stapling on Apache

When connecting to a server, clients should verify the validity of the server certificate using either a Certificate Revocation List (CRL), or an Online Certificate Status Protocol (OCSP) record. The problem with CRL is that the lists have grown huge and takes forever to download.

OCSP is much more lightweight, as only one record is retrieved at a time. But the side effect is that OCSP requests must be made to a 3rd party OCSP responder when connecting to a server, which adds latency and potential failures. In fact, the OCSP responders operated by CAs are often so unreliable that browser will fail silently if no response is received in a timely manner. This reduces security, by allowing an attacker to DoS an OCSP responder to disable the validation.

The solution is to allow the server to send its cached OCSP record during the TLS handshake, therefore bypassing the OCSP responder. This mechanism saves a roundtrip between the client and the OCSP responder, and is called OCSP Stapling.

The server will send a cached OCSP response only if the client requests it, by announcing support for the status_request TLS extension in its CLIENT HELLO.

Most servers will cache OCSP response for up to 48 hours. At regular intervals, the server will connect to the OCSP responder of the CA to retrieve a fresh OCSP record. The location of the OCSP responder is taken from the Authority Information Access field of the signed certificate.

What is OCSP Stapling

OCSP stapling is defined in the IETF RFC 6066. The term “stapling” is a popular term used to describe how the OCSP response is obtained by the web server. The web server caches the response from the CA that issued the certificate. When an SSL/TLS handshake is initiated, the response is returned by the web server to the client by attaching the cached OCSP response to the CertificateStatus message. To make use of OCSP stapling, a client must include the “status_request” extension with its SSL/TSL Client “Hello” message.

OCSP stapling presents several advantages including the following:

• The relying party receives the status of the web servers certificate when it is needed (during the SSL/TLS handshake).
• No additional HTTP connection needs to be set up with the issuing CA.
• OCSP stapling provides added security by reducing the number of attack vectors.

Read one of the following links for more information on OCSP and OCSP stapling.

Requirements

You need at least Apache 2.3.3 and later plus OpenSSL 0.9.8h or later for this to work. This is not available in the current Ubuntu LTS releases (12.04), it has 2.2.22 and CentOS 6 has 2.2.15. Either search for PPA’s/unofficial repositories or compile them yourself.

You also need create a firewall exception to allow your server to make outbound connections to the upstream OCSP’s. You can view all OCSP URI’s from a website using this one liner:

OLDIFS=$IFS; IFS=':' certificates=$(openssl s_client -connect google.com:443 -showcerts -tlsextdebug -tls1 2>&1 </dev/null | sed -n '/-----BEGIN/,/-----END/ {/-----BEGIN/ s/^/:/; p}'); for certificate in ${certificates#:}; do echo $certificate | openssl x509 -noout -ocsp_uri; done; IFS=$OLDIFS

It results for google.com in:

http://clients1.google.com/ocsp
http://gtglobal-ocsp.geotrust.com

Replace google.com with your domain. Also note that you need the GNU version of sed and bash. It does not work on OS X or BSD.

Apache Configuration

Add the below configuration to your virtualhost:

SSLUseStapling on
SSLStaplingCache "shmcb:logs/stapling-cache(150000)"

Here’s the explanation for the two lines:

SSLUseStapling

OCSP stapling relieves the client of querying the OCSP responder on its own, but it should be noted that with the RFC 6066 specification, the server's CertificateStatus reply may only include an OCSP response for a single cert. For server certificates with intermediate CA certificates in their chain (the typical case nowadays), stapling in its current implementation therefore only partially achieves the stated goal of "saving roundtrips and resources" - see also RFC 6961 (TLS Multiple Certificate Status Extension). 

SSLStaplingCache

Configures the cache used to store OCSP responses which get included in the TLS handshake if SSLUseStapling is enabled. Configuration of a cache is mandatory for OCSP stapling. With the exception of none and nonenotnull, the same storage types are supported as with SSLSessionCache

The shmbc part:

This makes use of a high-performance cyclic buffer (approx. size bytes in size) inside a shared memory segment in RAM (established via /path/to/datafile) to synchronize the local OpenSSL memory caches of the server processes. This is the recommended session cache. To use this, ensure that mod_socache_shmcb is loaded.

You can also give a few more options. For example, a freshness timeout, how old the OCSP response can be:

SSLStaplingResponseMaxAge 900

This lets the response only be max 15 minutes old (900 seconds).

If your apache server is behind a HTTP proxy and you need to do your OCSP queries through a proxy you can use SSLStaplingForceURL. This overrides the URL provided by the certificate:

SSLStaplingForceURL http://internal-proxy.example.org

Restart your apache to load the new configuration:

service apache2 restart

And it should work. Let’s test it.

Testing it

Fire up a terminal and use the following OpenSSL command to connect to your website:

openssl s_client -connect example.org:443 -tls1 -tlsextdebug -status

In the response, look for the following:

OCSP response:
======================================
OCSP Response Data:
    OCSP Response Status: successful (0x0)
    Response Type: Basic OCSP Response
    Version: 1 (0x0)
    Responder Id: 99E4405F6B145E3E05D9DDD36354FC62B8F700AC
    Produced At: Feb  3 04:25:39 2014 GMT
    Responses:
    Certificate ID:
      Hash Algorithm: sha1
      Issuer Name Hash: 0226EE2F5FA2810834DACC3380E680ACE827F604
      Issuer Key Hash: 99E4405F6B145E3E05D9DDD36354FC62B8F700AC
      Serial Number: C1A3D8D00D72FCE483CD84759E9EC0BC
    Cert Status: good
    This Update: Feb  3 04:25:39 2014 GMT
    Next Update: Feb  7 04:25:39 2014 GMT

That means it is working. If you get a response like below, it is not working:

OCSP response: no response sent

You can also use the SSL Labs test to see if OCSP stapling works.

Sources

• Apache documentation for SSLUseStapling
• Apache documentation for SSLStaplingCache
• Apache documentation for SSLStaplingForceURL
• Apache documentation for SSLStaplingResponderTimeout

How To Configure OCSP Stapling on Linux

How To Configure OCSP Stapling

OCSP (Online Certificate Status Protocol) is a protocol for checking if a SSL certificate has been revoked. It was created as an alternative to CRL to reduce the SSL negotiation time. With CRL (Certificate Revocation List) the browser downloads a list of revoked certificate serial numbers and verifies the current certificate, which increases the SSL negotiation time. In OCSP the browser sends a request to a OCSP URL and receives a response containing the validity status of the certificate.

OCSP has major two issues:

• Privacy – Since OCSP requires the browser to contact the CA to confirm certificate validity it compromises privacy. The CA knows what website is being accessed and who accessed it.
• Load – If a HTTPS website gets lots of visitors the CA’s OCSP server has to handle all the OCSP requests made by the visitors.

When OCSP stapling is implemented the certificate holder queries the OCSP server themselves and caches the response. This response is “stapled” with the TLS/SSL Handshake via the Certificate Status Request extension response. As a result the CA’s servers are not burdened with requests and browsers no longer need to disclose users’ browsing habits to any third party.

Check for OCSP stapling support

OCSP stapling is supported on

• Apache HTTP Server (>=2.3.3)
• Nginx (>=1.3.7)

Please check the version of your installation with the following commands before proceeding.

Apache:

apache2 -v

Nginx:

nginx -v

Implement OCSP

If you followed our tutorial on How to create your own certificate authority you can now use this section to extend the capabilities of your CA. If not, you can skip to the next section

Assuming that you already have an OpenSSL Certificate Authority set up, you will need to make a couple of changes to your openssl.cnf file.

Add a new line to the usr_cert stanza

[ usr_cert ]
authorityInfoAccess = OCSP;URI:http://<uri to server>

Create a new stanza

[ v3_OCSP ]
basicConstraints = CA:FALSE
keyUsage = nonRepudiation, digitalSignature, keyEncipherment
extendedKeyUsage = OCSPSigning

For this example, the OCSP server will be running on n0where.net on port 8888, so the authorityInfoAccess line will look like:

authorityInfoAccess = OCSP;URI:http://n0where.net:8888

This line will add a new attribute to issued certs that tells clients where the CA’s OCSP server is located so it can check the validity of the cert. The new v3 template assigns a neccesary attribute “OCSPSigning” to any certificate issued under this template. We will need to issue an OCSP signing certificate to the OCSP server with the OCSPSigning attribute, otherwise signature verification will fail when a cert is being checked. This is the first thing we will do:

openssl req -new -nodes -out n0where.net.csr -keyout n0where.net.key -extensions v3_OCSP

Sign the request with the CA signing key:

openssl ca -in auth.n0where.net.csr -out auth.n0where.net.crt -extensions v3_OCSP

OpenSSL should show the signing request, look for this in the X509v3 extensions:

X509v3 Extended Key Usage:
OCSP Signing

Sign and commit the request. Now, issue a throwaway cert and sign it

openssl req -new -nodes -out dummy.n0where.net.csr -keyout dummy.n0where.net.key 
openssl ca -in dummy.n0where.net.csr -out dummy.n0where.net.crt

Next, start up the OCSP server.

openssl ocsp -index /etc/pki/CA/index.txt -port 8888 -rsigner n0where.net.crt -rkey n0where.net.key -CA /etc/pki/CA/cacert.pem -text -out log.txt

Once the dummy cert has been been issued and the OCSP server started, we can test the cert using the “openssl ocsp” command. To verify a certificate with OpenSSL, the command syntax is:

openssl ocsp -CAfile <cafile pem> -issuer <issuing ca pem> -cert <certificate to check> -url <url to OCSP server> -resp_text

Test our dummy file:

openssl ocsp -CAfile cacert.pem -issuer cacert.pem -cert dummy.n0where.net.crt -url http://n0where.net:8888 -resp_text

There’s going to be a large block of text flooding the screen. Some of the more important text:

OCSP Response Data:
OCSP Response Status: successful (0×0)
Response Type: Basic OCSP Response
…
Certificate ID:
Hash Algorithm: sha1
Issuer Name Hash: 922CD93C975EDC121DB25B1A55BA9B544E06F9B3
Issuer Key Hash: 322A8DBF79BE1A934543DC4F24FC69220A2803BA
Serial Number: 06
Cert Status: good
…
Response verify OK
dummy.n0where.net.crt: good
This Update: Nov 17 15:55:54 2014 GMT

Now revoke the cert, regenerate the CRL and restart the OCSP server (the server must be restarted every time a cert is issued or revoked). If the OCSP signing certificate was not issued with the OCSPSigning attribute, OpenSSL will gripe that the verification did not work properly. Reissue the signing cert with the OCSPSigning attribute for the server.

openssl ca -revoke /etc/pki/CA/newcerts/06.pem 
openssl ca -gencrl -out /etc/pki/CA/crl.pem

Now we can verify the certificate again:

openssl ocsp -CAfile /etc/pki/CA/cacert.pem -issuer /etc/pki/CA/cacert.pem -cert dummy.n0where.net.crt -url http://n0where.net:8888 -resp_text 

OCSP Response Status: successful (0×0) 
Response Type: Basic OCSP Response
… 
Certificate ID:
Hash Algorithm: sha1
Issuer Name Hash: 922CD93C975EDC121DB25B1A55BA9B544E06F9B3
Issuer Key Hash: 322A8DBF79BE1A934543DC4F24FC69220A2803BA
Serial Number: 06
Cert Status: revoked
Revocation Time: Nov 17 16:07:36 2014 GMT
This Update: Nov 17 16:12:08 2014 GMT
…
Response verify OK
dummy.n0where.net.crt: revoked
This Update: Nov 17 16:12:08 2014 GMT
Revocation Time: Nov 17 16:07:36 2014 GMT

If you were to install this cert on a website, and the CA certificate was installed, any modern browser should refuse to connect to the site as the cert has been revoked.

Configuring OCSP Stapling on Apache

Edit the SSL virtual hosts file and place these lines inside the <VirtualHost></VirtualHost> directive.

sudo nano /etc/apache2/sites-enabled/n0where.net.ssl.conf 

SSLCACertificateFile /etc/ssl/ca-certs.pem
SSLUseStapling on

A cache location has to be specified outside <VirtualHost></VirtualHost>.

sudo nano /etc/apache2/sites-enabled/n0where.net.ssl.conf

SSLStaplingCache shmcb:/tmp/stapling_cache(128000)

The virtual host file will look this:

/etc/apache2/sites-enabled/n0where.net.ssl.conf

<IfModule mod_ssl.c>
    SSLStaplingCache shmcb:/tmp/stapling_cache(128000)
    <VirtualHost *:443>

            ServerAdmin admin@n0where.net
            ServerName n0where.net
            DocumentRoot /var/www

            SSLEngine on

            SSLCertificateFile /etc/apache2/ssl/n0where.net/apache.crt
            SSLCertificateKeyFile /etc/apache2/ssl/n0where.net/apache.key

            SSLCACertificateFile /etc/ssl/ca-certs.pem
            SSLUseStapling on
    </VirtualHost>
</IfModule>

Do a configtest to check for errors.

apachectl -t

Reload if Syntax OK is displayed.

service apache2 reload

Access the website on IE (on Vista and above) or Firefox 26+ and check the error log.

tail /var/log/apache2/error.log

If the file defined in the SSLCACertificateFile directive is missing, a certificate error similar to the following is displayed.

[Mon Nov 17 17:36:44.055900 2014] [ssl:error] [pid 1491:tid 139921007208320] AH02217: ssl_stapling_init_cert: Can't retrieve issuer certificate!
[Mon Nov 17 23:36:44.056018 2014] [ssl:error] [pid 1491:tid 139921007208320] AH02235: Unable to configure server certificate for stapling

Configuring OCSP stapling on Nginx

Edit the SSL virtual hosts file and place the following directives inside the server {}section.

sudo nano /etc/nginx/sites-enabled/n0where.net.ssl 

ssl_stapling on;
ssl_stapling_verify on;
ssl_trusted_certificate /etc/ssl/private/ca-certs.pem;

Nginx virtual host file will look like this:

/etc/nginx/sites-enabled/n0where.net.ssl

server {

        listen   443;
        server_name n0where.net;

        root /usr/share/nginx/www;
        index index.html index.htm;

        ssl on;
        ssl_certificate /etc/nginx/ssl/n0where.net/server.crt;
        ssl_certificate_key /etc/nginx/ssl/n0where.net/server.key;

        ssl_stapling on;
        ssl_stapling_verify on;
        ssl_trusted_certificate /etc/ssl/private/ca-certs.pem;
}

Do a configtest to see if everything is correct.

service nginx configtest

Then reload the nginx service.

service nginx reload

Access the website on IE (on Vista and above) or Firefox 26+ and check the error log.

tail /var/log/nginx/error.log

If the file defined in ssl_trusted_certificate is missing a certificate an error similar to the following is displayed:

2014/11/17 17:55:16 [error] 1580#0: OCSP_basic_verify() failed (SSL: error:27069065:OCSP routines:OCSP_basic_verify:certificate verify error:Verify error:unable to get local issuer certificate) while requesting certificate status, responder: {your certificate provider}

If no such errors are displayed proceed to the next step.

Testing OCSP Stapling

This command’s output displays a section which says if your web server responded with OCSP data. We grep this particular section and display it.

echo QUIT | openssl s_client -connect n0where.net:443 -status 2> /dev/null | grep -A 17 'OCSP response:' | grep -B 17 'Next Update'

Replace n0where.net with your domain name. If OCSP stapling is working properly the following output is displayed.

OCSP response:
======================================
OCSP Response Data:
    OCSP Response Status: successful (0x0)
    Response Type: Basic OCSP Response
    Version: 1 (0x0)
    Responder Id: 4C58CB25F0414F52F428C881439BA6A8A0E692E5
    Produced At: Nov  17 08:47:00 2014 GMT
    Responses:
    Certificate ID:
      Hash Algorithm: sha1
      Issuer Name Hash: B8A299F09D061DD5C1588F76CC89FF57092B94DD
      Issuer Key Hash: 4C58CB25F0414F52F428C881439BA6A8A0E692E5
      Serial Number: 0161FF00CCBFF6C07D2D3BB4D8340A23
    Cert Status: good
    This Update: Nov  17 08:45:00 2014 GMT
    Next Update: Nov 20 09:00:00 2014 GMT

No output is displayed if OCSP stapling is not working.