Name: aws-codepipeline-stepfunctions
Owner: Amazon Web Services - Labs
Owner: AWS Samples
Description: This AWS Labs project shows how to integrate AWS CodePipeline and AWS Step Functions state machines. The integration enables developers to build much simpler CodePipeline actions that perform a single task and to delegate the complexity of dealing with workflow-driven behavior associated with that task to a proper state machine engine. As such, developers will be able to build more intuitive pipelines and still being able to visualize and troubleshoot their pipeline actions in detail by examining the state machine execution logs.
Created: 2017-09-29 16:50:06.0
Updated: 2017-12-19 19:18:15.0
Pushed: 2017-11-21 18:25:40.0
Size: 347
Language: JavaScript
GitHub Committers
| User | Most Recent Commit | # Commits |
|---|
Other Committers
| User | Most Recent Commit | # Commits |
|---|
Welcome to the aws-codepipeline-stepfunctions project on AWS Labs!
How about delegating complex CodePipeline tasks to a proper state machine and keeping your pipeline clean and easy to understand?
This project shows you how to integrate AWS CodePipeline and AWS Step Functions state machines. The integration enables developers to build much simpler CodePipeline actions that perform a single task and to delegate the complexity of dealing with workflow-driven behavior associated with that task to a proper state machine engine. As such, developers will be able to build more intuitive pipelines and still being able to visualize and troubleshoot their pipeline actions in detail by examining the state machine execution logs.
app/ - contains a CloudFormation template that represents the application being deployed via CodePipeline/Step Functions.
pipeline/codepipeline - CodePipeline resources
pipeline/statemachines - Step Functions resources including the Deploy state machine
pipeline/local_modules - required local npm modules
pipeline/docs - project's documentation artifacts
Please install the following software in your local workstation before proceeding:
AWS Command Line Interface (version 1.11.170 or greater)
Configure your AWS credentials (access keys, AWS region, etc). At a minimum your credentials should allow you to create and manipulate resources associated withe the following AWS services: Amazon S3, AWS CodeCommit, AWS CodePipeline, AWS Lambda, AWS StepFunctions, and AWS IAM.
configure
If you plan to develop and contribute to the project it's a good idea to use SAM Local:
From the project's root directory, type:
d u+x **/*.sh
From the project's root directory, type:
pipeline/
stall_dependencies.sh
Clone this project in your local workstation (git clone …)
If you wish to use your default AWS profile skip this step, otherwise open script file config.sh and enter your profile using variable AWS_PROFILE.
ipeline/config.sh
Important: Make sure the AWS profile chosen uses an AWS region that supports AWS Step Functions (check here).
pipeline/
bootstrap.sh
The bootstrap script will create all necessary resources including:
Once the bootstrap script completes, open the AWS Console and check the various resources created for you including a CodeCommit repository, a CodePipeline pipeline, a Step Functions state machine, three S3 buckets, and IAM roles. Note that the CodePipeline pipeline is in a failing state since the CodeCommit repository does not have any code. The next step will be to push code to the CodeCommit repository which will cause the pipeline to start processing the changes.
Let's add code to the repository.
Using the AWS Console, navigate to the CodeCommit repository created for you
Click on “Connect” or follow these instructions to set up credentials to your IAM user to allow you to clone the CodeCommit repository (via SSH or HTTPS).
Back in your local workstation. From the project's root directory, type:
app/
t init && git add --all
t commit -m "initial commit"
t remote add codecommit [YOUR-CODECOMMIT-REPO-CLONE-URL]
t push codecommit master
This will push code into the CodeCommit repository and trigger the CodePipeline pipeline after a few seconds.
Open the CodePipeline Console and follow the execution of the pipeline. The Source pipeline action will simply load the source code from CodeCommit into an S3 bucket. The Deploy action invokes the StateMachineTriggerLambda Lambda function which, in turn, fetches the state machine input parameters from a file in S3 and triggers the state machine (see figure above for further details). The state machine starts executing while the StateMachineTriggerLambda Lambda sends a continuation token to CodePipeline and terminates. Seconds later, the StateMachineTriggerLambda Lambda is invoked again by CodePipeline. The Lambda will check whether the state machine execution has completed and, if that's the case, it will notify the pipeline that the pipeline action succeeded. If otherwise the state machine has failed, the Lambda will send a failure response to the pipeline action interrupting the pipeline execution. The StateMachineTriggerLambda Lambda fully decouples the pipeline from the state machine.
For further details please read our AWS DevOps Blog post
Here are a few details you need to know if you wish to contribute to this project.
Please use AWS SAM Local to run and debug Lambda functions locally. It is really useful and saves a lot of time. A script named run_local.sh is available under pipeline/codepipeline and pipeline/statemachine/deploy for convenience.
In order to run Lambda functions locally, SAM local requires that test events are passed as input to the Lambda functions. Sample test events are provided under test_events/. Important: the provided test events might contain account-specific parameters that need to be adjusted.
Here is an example of how to invoke the CreateStackStateMachineTask Lambda function:
pipeline/statemachines/deploy
run_local.sh CreateStackStateMachineTask create_stack
Note that template state_machine_template.yaml and test event file test_events/event_create_stack.json are referenced within the script.
If you plan to make changes to local modules (under _pipeline/local_modules) keep in mind that this requires updating the modules that depend on these local modules.
This can be accomplished by running these scrips:
ack changes made to local modules
pipeline/local_modules
ack_modules.sh
pdate codepipeline local modules references
pipeline/codepipeline
nstall_modules.sh
pdate statemachines/deploy local module references
pipeline/statemachines/deploy
nstall_modules.sh
Have fun and contribute!