Free SSL certificates with Certbot in AWS Lambda
Thanks to Certbot and to Electronic Frontier Foundation it is possible to provide a totally free SSL certificate to your website. Certbot is a command line tool to request a valid SSL certificate for your domain, following a process to verify the ownership. The tool can also deal with web server certificate installation and many other tasks (plugins available). This post is a guide about how to automatically request and renew your free SSL certificates with Certbot in AWS Lambda.
Why use Certbot in AWS Lambda?
I deal with several web applications using CloudFront for content distribution, associated with a source S3 bucket. So I decided to create a simple Lambda function that deals with obtaining SSL certificates with Certbot and periodically verifying their expiration date. If necessary, it automatically renews and imports the new certificate onto AWS Certificate Manager.
Result? No more expired SSL certificates! The automation of the process is particularly important considering the short life (90 days) of the certificates issued by Let’s Encrypt CA.
Solution Overview
The heart of the solution is obviously the Lambda function, periodically invoked by a CloudWatch event. The function manages a list of domains (specified in the DOMAINS_LIST environment variable – comma separated values) and for each of them:
- checks the presence of the certificate in ACM
- if not present, a new certificate is requested using Certbot. In order to correctly complete the domain ownership verification process, the function copies the validation token into S3 bucket hosting domain static resources
- for existing certificates, the expiration date is verified and, if necessary, renew process is done
At the end of the process, the configuration, certificates and private keys are stored in a private S3 bucket, to be reusable at the next run.
Using Certbot in AWS Lambda
Certbot is written in Python and can be easily used to automate the certificate request, renewal and revocation processes. However, using it in an AWS Lambda environment requires an additional preparation step, so that all the necessary packages and dependencies are correctly installed. This fantastic guide explains in detail how to use an EC2 instance for this purpose. I have prepared the package containing version 1.3.0 of Certbot which is available in the repository related to this post.
The Certbot configuration is located in the / tmp directory of the lambda instance and removed for security reasons at the end of the execution, because it includes the private keys of the certificates.
To proceed with the renewal operations, however, it is necessary to preserve the configuration of Certbot. The directory tree containing the configuration is compressed in a zip file and copied into the S3 bucket set by the environment variable CERTBOT_BUCKET. At the next execution, the configuration is restored from the zip file in the / tmp directory of the instance.
The dark side of the Symlinks
Certbot checks that its configuration tree is valid. It checks, among other things, the presence of symbolic links in the live directory for each domain.
The backup and restore process of the configuration tree in zip format removes these links (replaced by the actual files). To restore the initial situation, this method is used.
Pass the verification challenge with AWS S3
To request a certificate, you must pass a challenge, which proves ownership of the relevant domain. In order for the Lambda function to correctly manage this challenge, it is required:
- an S3 bucket named with domain name
- S3 bucket is already configured as source of CloudFront distribution for the domain or, alternatively, static website hosting (HTTP) of S3 bucket is – temporarily – active
- correct DNS configuration for the domain (CNAME to CloudFront / S3)
- IAM role to allows PutObject and DeleteObject operations on S3 bucket
The challenge consists to create a specific file containing a token provided by Certbot. This file must be placed temporarily on the website in order to be verified by Let’s Encrypt’s servers. At the end of the verification process the file is removed. These operations are performed by two Python scripts ( auth-hook.py and cleanup-hook.py ).
Access to the domain S3 bucket is required only to pass the challenge and to obtain the first SSL certificate. It will no longer be required during subsequent renewals.
Import the certificate to ACM
Once the certificate is obtained, the lambda function takes care of importing it into ACM. If a certificate already exists for the same domain, it is replaced by specifying the relative ARN during the import.
Important note: to be used in CloudFront, the certificate must be imported into the US East region (N. Virginia). For convenience, I built the entire stack in this AWS Region.
Scheduling with CloudWatch
CloudWatch is used to run the lambda function once a day. Among the lambda configuration parameters, an environment variable (CERTS_RENEW_DAYS_BEFORE_EXPIRATION ) determines how many days before the deadline renew the single certificate. The expiration date is obtained by ACM. No unnecessary renewal are attempted so lambda execution can be scheduled on a daily basis without worries.
The renewal through CertBot is forced , guaranteeing the obtaining of a new certificate 30 (default value) days before the expiration.
Deployment via CloudFormation
In the repository of this project the CloudFormation template is available for creating the stack. This includes the function, its role, the CloudWatch event and its permissions.
To deploy, the first step after cloning repository, is to build the function.
make lambda-build
To create the stack it is necessary to indicate the S3 bucket to be used for storing the Certbot configuration.
make BUCKET = your_bucket_name create-stack
Provided bucket is also used to temporarily store the sources of the lambda function for deployment.
Once the stack is created, it is necessary to set the environment variables DOMAINS_LIST with the list of domains to be managed separated by commas and DOMAINS_EMAIL with the email address to be use when requesting certificates. For each new domain it is necessary to provide the correct access policy to the relative S3 bucket that hosts the static resources.
Conclusions
Getting free SSL certificates for your projects is really useful; automating its management through this project has given me the opportunity to forget about it and live happily.
I recommend carrying out your own tests using the staging environment made available by Certbot. The CERTBOT_ENV environment variable allows you to define whether to use the production endpoint or the staging one.
Pay attention to ACM quota : the number of certificates that can be imported into ACM is limited in time (quantity in the last year). During my tests I ran into a very low limit: 20! As reported in this post you must contact the AWS support for removal.
Error: you have reached your limit of 20 certificates in the last year. Further developments? Lot of! The requirement to have an S3 bucket with the same domain name can, for example, be overcome by providing for a more advanced configuration, perhaps stored in a DB. Feel free to improve the original source code and let me know!
Did we have fun? See you next time!