Clever/sfncli

Name: sfncli

Owner: Clever

Description: Utility to create AWS Step Function activities out of command line programs

Created: 2017-09-17 01:46:10.0

Updated: 2018-05-24 00:08:48.0

Pushed: 2018-05-24 00:08:47.0

Homepage: null

Size: 74

Language: Go

GitHub Committers

User	Most Recent Commit	# Commits

Other Committers

User	Email	Most Recent Commit	# Commits

README

sfncli

Utility to create AWS Step Function (SFN) activities out of command line programs.

Usage

ncli -h
e of sfncli:
ctivityname string
    The activity name to register with AWS Step Functions. $VAR and ${VAR} env variables are expanded.
md string
    The command to run to process activity tasks.
egion string
    The AWS region to send Step Function API calls. Defaults to AWS_REGION.
loudwatchregion string
  The AWS region to send metric data. Defaults to the value of region.
ersion
    Print the version and exit.
orkername string
    The worker name to send to AWS Step Functions when processing a task. Environment variables are expanded. The magic string MAGIC_ECS_TASK_ARN will be expanded to the ECS task ARN via the metadata service.
orkdirectory string
    A directory path that is passed to the `cmd` using an env var `WORK_DIR`. For each activity task a new directory is created in `workdirectory` and it is cleaned up after the activity task exits. Defaults to "", does not create directory or set `WORK_DIR`

Example:

li -activityname sleep-100 -region us-west-2 --cloudwatchregion us-west-1 -workername sleep-worker -cmd sleep 100

High-level logic

• On startup, call CreateActivity to register an Activity with Step Functions.
• Begin polling GetActivityTask for tasks.
• Get a task. Take the JSON input for the task and
• if it's a JSON object, use this as the last arg to the cmd passed to sfncli.
• if it's anything else (e.g. JSON array), an error is thrown.
• if the task input an _EXECUTION_NAME key, it is added to the environment of the cmd as _EXECUTION_NAME.
• if workdirectory is set, create a sub-directory and add it to the environment of the cmd as WORK_DIR.
• Start SendTaskHeartbeat loop.
• When the command exits:
• Call SendTaskFailure if it exited nonzero, was killed, or sfncli received SIGTERM.
• Call SendTaskSuccess otherwise. Parse the last line of the stdout of the command as the output for the task (it must be JSON).
• If workdirectory was set then cleanup WORK_DIR/sub-directory-for-task

Error names

Error names in SFN state machines are useful for debugging and setting up branching/retry logic in state machine definitions. sfncli will report the following error names if it encounters errors it can identify:

• sfncli.TaskInputNotJSON: input to the task was not JSON
• sfncli.CommandNotFound: the command passed to sfncli was not found
• sfncli.CommandKilled: the command process received SIGKILL
• sfncli.CommandExitedNonzero: the command process exited with a nonzero exit code
• sfncli.TaskOutputNotJSON: the task output (last line of command's stdout) was not JSON
• sfncli.CommandTerminated: sfncli or the command received SIGTERM
• sfncli.Unknown: unexpected / unclassified errors

Additionally, the cmd can report a custom a custom error name in its output by including an error_name key. sfncli will look for this key if the command exits with a nonzero exit code, and if it exists it will report this error name instead of sfncli.CommandExitedNonzero.

Local testing

Start up a test activity that runs echo on the work it receives.

un cmd/sfncli/*.go -region us-west-2 -activityname test-activity -cmd echo

Create a new state machine that uses this activity for one of its states (this requires you to create a role for use with Step Functions):

--region us-west-2 stepfunctions create-state-machine --name test-state-machine --role-arn arn:aws:iam::589690932525:role/raf-test-step-functions --definition '{
"Comment": "Testing out step functions",
"StartAt": "foo",
"Version": "1.0",
"TimeoutSeconds": 60,
"States": {
    "foo": {
        "Resource": "arn:aws:states:us-west-2:589690932525:activity:test-activity",
        "Type": "Task",
        "End": true
    }
}

Note that you will need to replace the Resource above to reflect the correct ARN with your AWS account ID.

Start an execution of the state machine (again replacing the ARN below with the correct account ID):

--region us-west-2 stepfunctions start-execution --state-machine-arn arn:aws:states:us-west-2:589690932525:stateMachine:test-state-machine  --input '{"hello": "world"}'

You should see echo run with the argument {"hello": "world"}.