Secure servers with TLS

Setting up and managing ACME-IDs

For certificates issued by GÉANT TCS on our behalf, validation is performed using External Account Binding (EAB).

To use ACME with EAB, an ACME ID is required. Each ACME ID is named after the FQDN of a server. An ACME ID can be used to manage certificates for the FQDN giving it its name and for any number of other FQDNs. The other FQDNs for which an ACME ID can manage certificates can be set in the IT portal and changed at any time.

An FQDN can also be registered with several ACME IDs, but only one ACME ID can bear the FQDN as its name. If the FQDN giving the name is removed from the central computer database, the ACME ID also becomes invalid.

Those who are responsible for operating a server can set up and manage ACME IDs with the FQDN of the server in the IT portal. Existing ACME IDs can be edited by all persons responsible for the respective FQDN.

It is recommended to set up only one common ACME ID for groups of FQDNs that are managed by the same group of people.

The administration of ACME IDs can be found in the IT portal under “Digital ID (certificate)”.

Please see to the detailed instructions in the IT portal.

When you set up an ACME ID, you receive an EAB key ID, an EAB HMAC key, and the URL of the certificate server. You must then instruct your ACME-enabled software to register with the certificate server using these key data.

During this registration, key material is exchanged between your ACME-enabled software and the certificate server, which the software can then use to request and collect the desired certificates.

The following guide describes the necessary steps with the CertBot software. This software can manage certificates from several certification authorities at the same time, e.g. from our partner GÉANT TCS and from Let's Encrypt.

Brief information on alternatives can be found at the bottom.

Starting the CertBot daemon

The CertBot software must be installed and must be called up automatically at regular intervals so that it can renew certificates that are about to expire. This depends on your operating system variant and could look something like this:

yum install certbot

systemctl enable certbot-renew.timer
systemctl start certbot-renew.timer

It is not necessary to make the CertBot service fail-safe. As a rule, certificates are renewed 30 days before they expire. If the service fails, you therefore have four weeks to discover and rectify the problem.

Registration of an ACME ID set up in the IT portal

To register the local CertBot service with the ACME service, the following command must be entered once. This contains the URL of the certificate server, EAB key ID and EAB HMAC key for the external account binding, and the email address to be informed in case of problems:

certbot register --server 'server-url' --eab-kid 'key-id' --eab-hmac-key 'hmac-key' --email 'mailaddress' --agree-tos --no-eff-email

With:

• server-url = the URL of the certificate server displayed by the IT portal

• key-id = the (shorter) EAB key ID displayed by the IT portal

• hmac-key = the (longer) EAB HMAC key displayed by the IT portal

• mailaddress = the email address to which the CertBot sends its information

If an ACME ID is to be used on multiple servers, the CertBot directory /etc/letsencrypt can be copied to the other servers.

Requesting an initial certificate for one or more FQDNs

To actually request a certificate via ACME, collect it, install it, restart the certified service and trigger the regular automatic repetition of the entire process, the following command is used in the general case:

certbot certonly --standalone --agree-tos --server 'server-url' --cert-name 'certname' --domain fqdn1,fqdn2,... --email='mailaddress' --deploy-hook '/path/to/script arg ...'

With:

• server-url = the URL of the certificate server displayed by the IT portal

• certname = the freely selectable file name (without path) of the certificate (for example, the main FQDN of the server for which the certificate is intended)

• fqdn1,fqdn2,... = the fully qualified computer names (FQDNs) under which the service appears; these FQDNs must have been specified in the IT portal for the ACME ID; the FQDN specified first is also included in the subject of the certificate, the others only in the subject alternative names

• mailaddress = the e-mail address to which the CertBot sends its information for this server

• /path/to/script arg ... = the absolute path of a (possibly self-written) command, optionally with arguments, which is called after each retrieval of new certificates and which

  • in the best case copies the files from the directory /etc/letsencrypt/live/certname/ to the right location and then restarts the server

  • or otherwise at least informs someone that new certificates are ready.

  This script can look in the environment variables RENEWED_LINEAGE and RENEWED_DOMAINS to see for which certificate and which domains it was called. You can therefore use the same script for different servers.

  A shell script for sending an e-mail that is suitable for the option --deploy-hook '/usr/local/bin/newcert mailaddress' and can be used for any number of servers would be:

  #!/bin/sh
echo "See $RENEWED_LINEAGE" |
/usr/bin/mailx -s "New certificate for $RENEWED_DOMAINS" "$1"

This command can be modified in many ways and adapted to your own circumstances. Please refer to the description of the version of the CertBot software available to you.

In particular, there are plugins for Apache and Nginx that allow automatic installation in local web servers even without a deploy hook script.

Requesting certificates with existing CSR files

If you operate servers that cannot import private keys, but insist on generating key pairs and certificate request files (CSR files) themselves, you can additionally specify the --csr /path/to/csr/file option in the certbot certonly ... command above.

This CSR file is then also used for subsequent certificate renewals.

However, you should then ensure that you renew the key pair and the CSR file every few years.

Force certificate renewal

The following command is used to update immediately prematurely all certificates managed by the CertBot service:

certbot renew --noninteractive --no-random-sleep-on-renew --force-renewal

Using with private IP addresses

In order to reach the certification server, the proxy server should be set before issuing certbot commands:

http_proxy=http://wwwproxy.uni-muenster.de:3128
https_proxy=http://wwwproxy.uni-muenster.de:3128
export http_proxy https_proxy

For the timer-controlled automatic certificate retrieval to work, the following lines should be added to the system configuration file /etc/sysconfig/certbot:

http_proxy=http://wwwproxy.uni-muenster.de:3128
https_proxy=http://wwwproxy.uni-muenster.de:3128

Multiple certificates with one CertBot

To manage multiple certificates with the same ACME ID, select a different certname and the desired FQDNs fqdn1,fqdn2,... for each of these certificates.

The ACME ID must be enabled in the IT portal for all FQDNs in all certificates, but of course it is not necessary to include all FQDNs in every certificate.

For systems that span multiple servers (clusters, high availability, etc.) but are all administered by the same group of people, you can distribute the /etc/letsencrypt directory tree to the other servers after registration. Then each participating server can get its certificates under the same ACME ID.

If you also operate servers under third-party domains: The same CertBot can even be used to obtain certificates from other certification authorities such as Let's Encrypt.

However, as soon as different groups of people have administrative access to the various servers, separate ACME IDs and separate CertBots should definitely be used.

Access permissions to the certificate files

CertBot stores the certificate files, in particular the private key, with access rights only for the root user (0700). If applications that do not run as root but under a different ID are to access the files in /etc/letsencrypt/live/ directly, the corresponding read permissions must be set. This must not be done with chown, chgrp or chmod, as this will be changed or reported by CertBot.

setfacl -R -m u:username:rX /etc/letsencrypt/{live,archive}/certname
setfacl -m u:username:rX /etc/letsencrypt/{live,archive}
setfacl -m d:u:username:rX /etc/letsencrypt/{live,archive}/certname

With:

• username = the username under which the server runs

• certname = the freely choosable file name (without path) of the certificate

You should definitely check if the identifier can actually read the generated private keys, for example with:

getfacl /etc/letsencrypt/archive/certname/priv*

If necessary, the ACL mask in above commands must be adjusted.

Set up or change deploy-hook later on

If you specify the --deploy-hook option when retrieving a certificate for the first time (certbot certonly --standalone ...), this command/script will be called after every certificate exchange. This is an easy way to restart services automatically after a certificate change.

If this was not wanted or forgotten, there are various alternatives to realize the service restarts with CertBot:

• add the line
  renew_hook = /path/to/script arg ...
  to /etc/letsencrypt/renewal/certname.conf under [renewalparams]

• simply repeat the certificate retrieval with --deploy-hook; this will then generate this line, but will also renew the certificate at the same time

• call certbot renew --deploy-hook ... but this will affect all certificates and will result in this entry for all certificates

• place an appropriate script under /etc/letsencrypt/renewal-hooks/deploy, this also affects all certificates

The latter is especially useful if you want to do more than just issue a simple command. However, --deploy-hook also accepts paths to scripts written by yourself.

If you want even more control or want to use even more complex start/stop scripts, you can also work with --pre-hook and --post-hook or with scripts in /etc/letsencrypt/renewal-hooks/pre and /etc/letsencrypt/renewal-hooks/post respectively, the latter act on all certificates.

Alternative to Certbot: acme.sh

On the server you then set up a dedicated account (name at will), in whose HOME directory acme.sh is then installed and used as follows:

adduser acmeuser

su - acmeuser

git clone https://github.com/acmesh-official/acme.sh.git

cd ./acme.sh

./acme.sh --install -m e-mail-adresse

exit

To read the changed rc files, the account is logged in again, the default CA server is set, the EAB account is registered (by-product ACCOUNT_THUMBPRINT) and a certificate is requested:

su - acmeuser

acme.sh --set-default-ca --server 'server-url'

acme.sh --register-account --server 'server-url' --eab-kid 'key-id' --eab-hmac-key 'hmac-key'

acme.sh --issue -d fqdn1 --stateless

By default, the certificate files are then stored in the subfolder ~/.acme.sh/fqdn1/ and must still be copied to the location required by the application. As acmeuser you can use crontab -l to see the renewal entry already set up by acme.sh and adjust it or add a deployment.

Configuration of the digital ID

The Web server Apache needs as PEM files:

1. the unencrypted private key in /path/to/key.pem

2. the server certificate in /path/to/cert.pem

3. the intermediate CA certificates in /path/to/chain.pem

You can give three different files or a single file containg keys and certificates in this order. Names and paths can be chosen freely.

The CertBot software described above provides these three files in the directory /etc/letsencrypt/live/certname/ under the names privkey.pem, cert.pem, and chain.pem.

With the configuration statements below can be indicated which files are to be used:

SSLCertificateKeyFile   /path/to/key.pem
SSLCertificateFile      /path/to/cert.pem
SSLCertificateChainFile /path/to/chain.pem

If you want to use the files provided by the CertBot software directly, please refer to the section “Access permissions to the certificate files” above.

Urgently recommended further settings

Deactivate insecure SSL/TLS protocols:

SSLProtocol all -SSLv2 -SSLv3 -TLSv1 -TLSv1.1

Deactivate insecure encryption methods and prefer the best methods^[1][3]:

SSLHonorCipherOrder on
SSLCipherSuite "ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-RSA-AES128-SHA256"

or, even stricter^[2][3], according to the recommendation of the DFN-PKI team:

SSLHonorCipherOrder on
SSLCipherSuite "ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256"

The above line SSLCipherSuite "..." only works for connections with TLS 1.2 (and older if not deactivated).
Ciphers for connections with TLS 1.3 could be customized with an additional line SSLCipherSuite TLSv1.3 "..." if necessary.

Activate OCSP stapling (please use the same path as with SSLSessionCache):

SSLUseStapling on
SSLStaplingReturnResponderErrors off
SSLStaplingCache shmcb:/..../..../sslstaplingcache(512000)

Footnotes

[1] ↑ As a practical test over several days in 2021 in the university's web server park has shown, all clients that support TLS 1.2 can handle at least one of these encryption methods, so nobody is locked out.

[2] ↑ As a practical test over several days in 2021 in the university's web server park has shown, this tightening of the rules affects only 0,7 % of all accesses, all from outdated systems with known security problems.

[3] ↑ Apache uses the OpenSSL writing of the names. The original names are:
TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
TLS_DHE_RSA_WITH_AES_256_GCM_SHA384
TLS_DHE_RSA_WITH_AES_128_GCM_SHA256
TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384
TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256
TLS_DHE_RSA_WITH_AES_256_CBC_SHA256
TLS_DHE_RSA_WITH_AES_128_CBC_SHA256