Name: aws-lambda-manage-rds-connections
Owner: AWS Samples
Description: Sample code for dynamically managing RDS/RDBMS connections across a fleet of Lambda functions
Created: 2017-10-28 05:39:39.0
Updated: 2018-01-11 10:13:58.0
Pushed: 2017-12-03 05:56:25.0
Homepage: null
Size: 254
Language: Python
GitHub Committers
User | Most Recent Commit | # Commits |
---|
Other Committers
User | Most Recent Commit | # Commits |
---|
A connections management solution when using Lambda with RDS/RDBMS solutions.
There are some inherent challenges with using AWS Lambda with RDS/RDBMS solutions, associated with connections management. The Lambda container model manages the containers lifecycle, which host lambda function invocations. Some of the features of this container model are:
The above features mean using traditional connection pooling with Lambda, to access RDS/RDBMS solutions, doesn't make too much sense. If the connection pool is initialized in the global section, with multiple connections, only one connection (per container) will actually be used and rest will just count towards the DB connections quota, unused. Also, there is no hook to tap into and destroy these connection pools, once the container reaches end of life.
As a result, the best practice is to just use one connection per lambda container, initialized in the global section of the lambda function and relying on the DB server TTL to reclaim it.
Lambda concurrency limits also help mitigate some of the challenges, but some still remain:
Dynamic Connections Management tries to address these challenges.
I have chosen python as the launguage of choice and will be using MySQL RDS as a RDBMS solution.
This repository contains the sample code for test and helper Lambda functions, DynamoDB table, MySQL RDS instance as well as a SAM template to deploy the resources in an AWS region of your choice. It also leverages CloudFormation custom resources to prepare the MySQL and DynamoDb instance with test data.
The solution consists of maintaining a row in a DynamoDB table, which keeps track of the 'Maximum allowed connections' and 'connections in use' for a given DB. A helper lambda function is used to manipulate this count. This helper lambda function is called by the parent lambda, which wants to talk to the DB in question. The parent lambda function calls the helper once when it opens the connection and once when it closes the connection. Depending on the response from the helper lambda function (connections are available or not), the parent lambda function decides its course of action. The helper lambda function also publishes 2 metrics to Cloudwatch: 'Remaining Connections' and 'No Connections Left Error', which can then be used to create an alarm and do something interesting, like backing off the load on the DB or providing an alternate source for querying the same data.
Below is a snapshot of a test run, blue shows the 'Remaining Connections' (left Y Axis) and orange shows the 'No Connection Left Error' (right Y axis).
You can now create alarms on these metrics and use them to do something interesting.
This code depends on a bunch of libraries (not included in this distribution) which you will have to install yourself. You will be needing AWS CLI as well. The code comes with a SAM template, which you can use to deploy the entire solution.
Download the contents of this repository on your local machine (say: project-directory)
The solution is implemented in python, so make sure you have a working python environment on your local machine.
Open a command prompt, navigate to the project directory. Navigate to the /code sub directory and install the following libraries:
install pymysql --target .
Create a S3 bucket for deployment (note: use the same region throughout the following steps, I have used us-west-2, you can replace it with the region of your choice. Refer to the region table for service availability.)
s3 mb s3://rdslambdademo2017-us-west-2 --region us-west-2
We shall be using the LambdaRDS_Demo.yaml file as our SAM template.
Package the contents and prepare deployment package using the following command
cloudformation package --template-file LambdaRDS_Demo.yaml --output-template-file LambdaRDS_Demo_output.yaml --s3-bucket rdslambdademo2017-us-west-2 --region us-west-2
Replace the placeholders in the below command with username, password and dbname, and deploy the package:
cloudformation deploy --template-file LambdaRDS_Demo_output.yaml --stack-name RDSLambdaDemoStack --capabilities CAPABILITY_IAM --parameter-overrides RDSUserName=<username> RDSPassword=<password> RDSDBName=<dbname> --region us-west-2
If you want to make changes to the Lambda functions, you can do so on your local machine and redeploy them using the steps 6 and 7 above. The package and deploy commands take care of zipping up the new Lambda files (along with the dependencies) and uploading them to AWS for execution.
Following are the outputs from the SAM template
The TestHarness_Input.json file contains sample invocations that you can use to trigger the load test. Navigate to your Lambda consolse, look for the LambdaRDS_TestHarness function and create a test event. You can modify the test iterations count to your choice.
Please note, the LambdaRDS_Test function has a 'sleep' function to simulate some random querying times for the query. You should comment this out for production deployments.
Once the test finishes, you can navigate to your CloudWatch console and look for 'RDSLambda' metrics under the 'Custom' namespace. You can then plot these metrics and interact with them.
This library is licensed under the Apache 2.0 License.