Name: amazonmq-invoke-aws-lambda
Owner: AWS Samples
Description: Demonstrates an approach to invoking AWS Lambda from messages in an Amazon MQ queue.
Created: 2018-01-13 06:07:18.0
Updated: 2018-02-01 04:50:05.0
Pushed: 2018-01-29 03:28:12.0
Homepage: null
Size: 63
Language: JavaScript
GitHub Committers
User | Most Recent Commit | # Commits |
---|
Other Committers
User | Most Recent Commit | # Commits |
---|
This project is an example of one approach to invoking AWS Lambda from Queues and Topics managed by Amazon MQ brokers. This and other similar patterns can be useful in integrating legacy systems that leverage common APIs such as JMS with serverless architectures.
Our solution makes use of AWS CloudWatch Events to trigger a Lambda function that polls for messages on a queue. Messages are then sent to a second Lambda function that writes the message to an Amazon DynamoDB table. Further details can be found in our accompanying blog post.
To get started, clone this repository. The repository contains a CloudFormation template and source code to deploy and run the sample.
To run the sample, you will need to:
We will deploy the solution in two parts, first a new Amazon MQ broker, then two sample Lambda functions and DynamoDB table.
Launching an Amazon MQ broker is easy via the Amazon MQ Console or AWS CLI. For details, see Creating and Configuring an Amazon MQ Broker.
Once the broker is ready, you will need to capture its Security Group ID(s) (e.g. “sg-12345678”). This information is available from the Amazon MQ Console or the AWS CLI as follows:
s mq describe-broker --broker-id <MY_BROKER_ID> \
--query 'SecurityGroups'
We will use the AWS Serverless Application Model to manage deployment of our EC2 instance, Lambda function, and other AWS resources:
From the command line, create a new S3 Bucket from which to deploy our source code:
s s3 mb s3://<MY_BUCKET_NAME>
Install Lambda dependencies:
subscriber && npm install
../worker && npm install
..
Using the Serverless Application Model, package the source code:
s cloudformation package --template-file template.yaml \
--s3-bucket <MY_BUCKET_NAME> \
--output-template-file packaged.yaml
Next, deploy the CloudFormation stack, including the following parameters:
AmazonMQHost
- your broker endpoint, e.g. AmazonMQLogin
- username provided when creating your broker (e.g. master)AmazonMQPassword
- password provided when creating your brokerAmazonMQQueueName
- name of your broker's queue, we use SAMPLE_QUEUE
hereAmazonMQSecurityGroupId
- security group id, as captured above (e.g. sg-12345678)s cloudformation deploy --template-file packaged.yaml \
--stack-name aws-amazonmq-sample \
--capabilities CAPABILITY_IAM \
--parameter-overrides AmazonMQHost=<AMAZONMQ_BROKER_ENDPOINT> \
AmazonMQLogin=<AMAZONMQ_USERNAME> \
AmazonMQPassword=<AMAZONMQ_PASSWORD> \
AmazonMQQueueName=SAMPLE_QUEUE \
AmazonMQSecurityGroupId=<SECURITY_GROUP_ID>
It will take a few minutes for CloudFormation to finish deploying. Once status is CREATE_COMPLETE
, move on to testing the integration.
Once the CloudFormation stack is complete, we can send a test message using the ActiveMQ console.
Open the ActiveMQ Management Console, available at https://
Click the link “Manage ActiveMQ broker” and enter the username and password from when you created the broker.
In the menu just beneath the ActiveMQ logo, click the link at the far right, “Send”.
Modify two fields:
Click the Send button. Explore other aspects of the ActiveMQ console, particularly the Subscribers section, you should see one subscriber, your Flume instance.
When ready, open the AWS Console and navigate to DynamoDB.
Select “Tables” in the menu at left and then pick the table created by CloudFormation (name listed in CloudFormation Outputs).
Select the “Items” tab and view items below. You should see an entry that contains the message from above.
Finally, we can clean up the environment using CloudFormation:
s cloudformation delete-stack --stack-name aws-amazonmq-sample
As your Amazon MQ broker is not managed by CloudFormation, you will need to delete via the Console or CLI, if desired.
This sample code is made available under a modified MIT license. See the LICENSE file.