Deploying EFF's Certbot in AWS Lambda26 Jan 2018 | 9 minute read
This post describes the steps needed to deploy Certbot (a well-maintained LetsEncrypt/ACME client) inside AWS Lambda. The setup used below is now powering 100% automated TLS certificate renewals for this website - the lambda runs once a day and if there’s less than 30 days remaining on my existing cert it will provision a new one and import it to be served by my CDN.
The post is broken down into 3 sections:
- building a self-contained, deployable zip file that includes certbot and its dependencies (the bulk of the work)
- creating an IAM role for the lambda function that gives it the necessary permissions to execute
- creating a CloudWatch timer that triggers the lambda function once a day
Building a self-contained certbot zip file
Certbot is written in python and supports both python 2 & 3. We’re going to use Lambda’s python 3.6.1 runtime, and to make sure all our packages and dependencies work in the Lambda environment we’ll perform all the installation steps in an environment identical to the Lambda one. Amazon’s documentation states:
The underlying AWS Lambda execution environment is based on the following:
- Public Amazon Linux AMI version (AMI name: amzn-ami-hvm-2017.03.1.20170812-x86_64-gp2) which can be accessed here.
So we’ll need to bring up an EC2 instance like that.
Step 1: Launch an EC2 instance with the
amzn-ami-hvm-2017.03.1.20170812-x86_64-gp2 AMI. You can use the cheapest instance with all the default settings - it will not require any IAM permissions or security group configuration.
Step 2: SSH onto the instance and install python3 (it’s not installed by default):
Step 3: Install
virtualenv to isolate all the python dependencies, install certbot, and zip it up:
In addition to certbot and certbot-dns-route53 (a plugin that handles the DNS challenge during cert provisioning by updating your Route53 settings), I’ve installed raven (an exception capturing client), but if you don’t need that then you can remove it.
When certbot runs it writes the generated private key and certificate to the filesystem. We’ll use little bit of glue code in a
main.py driver to take those files and automatically import them into Amazon Certificate Manager (ACM). It loops over all ACM certificates and checks if each certificate has less than 30 days until expiration. If so it triggers certbot to provision a new key & cert, then reads the generated files and imports them into ACM, and finally notifies me via SNS. You can view the source here.
After adding our glue code to the zip file we’ll have a working, self-contained package.
Step 4: Download the zip and add the glue code:
Now the zip file can be deployed, but it still needs permissions to run.
Creating an IAM role with the correct permissions
For permissions, create an IAM role with a trust relationship for
lambda.amazonaws.com (this is the principal that lambda functions assume when they execute). Then attach the inline policy below - it boils down to 4 categories:
- CloudWatch permissions that all lambda functions need to log and emit metrics
- Route53 permissions for certbot to list and update your DNS settings
- ACM permissions for the glue code to list and import certificates
- SNS publish permission to send notifications when a new certificate is imported
HOSTED_ZONE_ID are replaced with the relevant values in your environment.
At this point you should be able to deploy the lambda function using this IAM role and run it manually. Make sure to increase the timeout of the function from the default 3 seconds - it usually completes in about 60 seconds but my configuration uses 300 (the max allowed) just for good measure.
Creating a CloudWatch timer
The last step is to create a daily trigger for the function. To do this we can create a CloudWatch rule with the desired
schedule_expression (when should it run), and then create a rule target specifying the lambda function’s ARN.
This entire website is powered by Terraform so I’ve pasted my terraform config for it below, but it’s also very straightforward to do it from either the AWS console or cli.
I haven’t run into any problems at all with this setup, but if something does break then I should know about it thanks to:
- LetsMonitor free certificate monitoring: they’ll email me if my certificate has 7 days left until expiration
- Sentry free exception tracking: any runtime exceptions in the lambda function get captured and emailed to me, with full stacktrace and context
- Some CloudWatch invocation and error metrics that notify me via SNS to my email
When a new certificate is generated, I get an email about it via SNS:
Now you too can have 100% automated TLS certificate renewals!
Thoughts? Comments? Let me know on twitter!