Evaluate Weigh the pros and cons of technologies, products and projects you are considering.

AWS Step Functions visualizes distributed app workflows

With Step Functions, developers can visualize the complex, multistep tasks that their cloud applications perform. Use this example to create a state machine and get started.

AWS Lambda is a popular service for running code in response to user requests -- however, not all requests are...

created equal.

In general, single-step operations work well with Lambda, but some operations require multiple steps. Imagine there's a service that monitors the AWS blog to find stories about AWS Step Functions. This monitoring task could include a few different steps for AWS Lambda to perform: read the AWS blog's RSS feed to look for new stories, see if any new story mentions Step Functions and, if so, send alerts via a Simple Notification Service topic.

You could make these steps three separate Lambda functions, with each triggering the next, but AWS Step Functions makes it much easier to tie these steps together. With AWS Step Functions, developers gain the advantage of an AWS-managed state machine, a special system that can track the progress of each task and identify any issues.

Let's walk through each step of the blog post example and how AWS Step Functions could play a role.

Create the state machine

First, you need to create the actual state machine in the AWS Step Functions console. To do this, you choose Create a state machine and then select from the sample blueprints, which show how the Amazon States Language works for different states. In this example, you would enter the code below:

{

  "Comment": "Read an RSS Feed, alert if there is new content matching a specific pattern",

  "StartAt": "ReadFeed",

  "States": {            

    "ReadFeed": {

      "Type": "Task",

      "Resource": "arn:aws:lambda:REGION:ACCOUNT_ID:function:lambdaGetNewStoriesForFeed",

      "Next": "CheckForNewStory"

    },

    "CheckForNewStory": {

      "Type" : "Choice",

      "Choices": [

        {

          "Variable": "$.newStoryCount",

          "NumericGreaterThan": 0,

          "Next": "CategorizeNewStories"

        }

      ],

      "Default": "EndState"

    },

 

    "CategorizeNewStories": {

      "Type" : "Task",

      "Resource": "arn:aws:lambda:REGION:ACCOUNT_ID:function:lambdaCategorizeNewStories",

      "Next": "ChoosePeopleToNotify"

    },

 

    "ChoosePeopleToNotify": {

      "Type" : "Choice",

      "Choices": [

        {

          "Variable": "$.category",

          "StringEquals": "Step Functions",

          "Next": "AlertStepFunctions"

        },

        {

          "Variable": "$.category",

          "StringEquals": "Lambda",

          "Next": "AlertLambda"

        },

        {

          "Variable": "$.category",

          "StringEquals": "Serverless",

          "Next": "AlertLambda"

        }

      ],

      "Default": "EndState"

    },

    "AlertStepFunctions": {

      "Type" : "Task",

      "Resource": "arn:aws:lambda:REGION:ACCOUNT_ID:function:lambdaAlertStepFunctionFollowers",

      "Next": "EndState"

    },

    "AlertLambda": {

      "Type" : "Task",

      "Resource": "arn:aws:lambda:REGION:ACCOUNT_ID:function:lambdaAlertLambdaFollowers",

      "Next": "EndState"

    },

 

    "EndState": {

      "Type": "Wait",

      "Seconds" : 300,

      "Next": "ReadFeed"

    }

  }

}

This creates an AWS state machine that follows the visual workflow shown below. This is a recursive state machine that runs indefinitely when started, so you only need to start one instance of it to continuously monitor the feed every five minutes.

The AWS Step Functions state machine follows a simple workflow.
The AWS state machine manages several tasks.

Let's go over each step we've created:

  • ReadFeed: The initial state, which runs the Lambda function lambdaGetNewStoriesForFeed. This should read the RSS feed for the AWS blog and return an object that includes stories and a value called newStoryCount, which indicates how many new stories were detected. This state transitions to CheckForNewStory.
  • CheckForNewStory: A choice state, which receives the output from the ReadFeed state. If no new stories were found, the state machine transitions to EndState. If it finds any new stories, it transitions to CategorizeNewStories.
  • CategorizeNewStories: A state that triggers when CheckForNewStory finds at least one new story. It executes a Lambda function called lambdaCategorizeNewStories and then executes the ChoosePeopleToNotify state.
  • ChoosePeopleToNotify: A state that looks at the category returned from the CategorizeNewStories state. If the category is Step Functions, it executes the AlertStepFunctions state. If it's either Lambda or Serverless, it executes the AlertLambda state. If none of those categories match, it executes the EndState state.
  • AlertStepFunctions and AlertLambda: States that execute Lambda functions to alert the appropriate person and then transition to EndState.
  • EndState: A wait state. It waits five minutes before executing the ReadFeed state to start the process over again.

Add the Lambda functions

The AWS Step Functions service is just one part of this process. While the AWS state machine connects the different states and passes the output from one Lambda function to another, the actual Lambda functions do the heavy lifting.

You might choose to use a platform such as Serverless Application Model, Apex or Lambda to deploy each of the individual Lambda functions. For example, the function lambdaGetNewStoriesForFeed needs to read the RSS feed, parse it into individual stories and then see if any of those stories are new. If they are new, it needs to then mark those stories that have been processed so they don't trigger a new story event later. Amazon DynamoDB is a perfect place to store this type of information.

The output of this function needs to be a valid JSON object, such as:

{

  "stories": [

    {

      "title": "Step Functions in-depth",

      "text": "…",

      "categories": [ "Serverless", "Lambda", "Step Functions" ]

    }

  ],

  "newStoryCount": 1

}

You can use whatever method you want to actually check for these new stories. Each step is completely independent, which means teams can split responsibilities, write steps in different languages and update any individual step without affecting the rest of the state machine.

Include error conditions

Step Functions is great for handling error conditions. For example, since our state machine needs to read an RSS feed from the AWS blog, what happens if that feed is down or broken? You could add a check on the CheckForNewStory state, which triggers an error state to alert developers. But there might be errors that you don't plan for, which is why it's always a good idea to have a generic error state that can alert developers to problems in your state machine or Lambda functions:

  "Catch": [

    {

      "ErrorEquals": [ "States.ALL" ],

      "Next": "AlertDevelopers"

    }

You can also add a retry block to a specific state to incorporate retry logic. This enables you to retry specific errors, such as a provisioned throughput exception or a temporary error with another Amazon service. You can specify a maximum number of attempts and a back-off policy, which determines how long to wait between attempts. This means, if a temporary issue occurs, the state machine can wait instead of forcing Lambda code to wait, which costs money.

The Step Functions service is good for more than just retry and error handling. It has numerous uses, ranging from the simple example above to more complex decision trees that define responses to give to a customer. In some cases, a Step Function is more cost-effective than a Lambda function, as Step Functions doesn't charge for resources while idling.

This was last published in November 2017

Dig Deeper on AWS Lambda

Join the conversation

1 comment

Send me notifications when other members comment.

By submitting you agree to receive email from TechTarget and its partners. If you reside outside of the United States, you consent to having your personal data transferred to and processed in the United States. Privacy

Please create a username to comment.

How do you use AWS state machines?
Cancel

-ADS BY GOOGLE

SearchCloudApplications

TheServerSide.com

SearchSoftwareQuality

SearchCloudComputing

Close