Triggering AWS Step Functions by AWS API Gateway Calls

by Emre Yilmaz
May 8, 2018

AWS • Serverless • AWS Step Functions • AWS API Gateway • AWS IAM
Istanbul

If you recall, in a previous blog post I discussed AWS Step Functions as a great service for orchestrating your AWS Lambda functions by giving an example. Today, I will show you how you can integrate your API Gateway resource methods with your AWS Step Functions state machines so that you can trigger your state machines with API calls.

Actually the benefits are obvious. Instead of serving our Lambda functions as separate endpoints and orchestrating which one to call and when in our front end; we can use a single endpoint and pass all logic to the back end. Then our back end, in this case our state machine, will execute the functions in sequential or parallel. This method especially suits well to cases when all these function calls are necessary to complete a single business action.

IAM execution role

Our API Gateway endpoint will trigger our state machine as an AWS Service integration. Therefore, we need to create an execution role that has necessary permissions to trigger the state machine and put logs to AWS CloudWatch for debugging.

I will describe sections of the IAM roles separately.

Trust relationsips

Trust relastionships in an IAM role define which services can assume the role. Hence, API Gateway will trigger the state machine and it will be the service to assume the role.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": "apigateway.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}

Permissions

Permissions in an IAM role define access privileges that an IAM role has.

Attach and define the policies below to your IAM role:

AmazonAPIGatewayPushToCloudWatchLogs - arn:aws:iam::aws:policy/service-role/AmazonAPIGatewayPushToCloudWatchLogs. This AWS managed policy will provide permissions to push logs to AWS CloudWatch.
An inline policy to grant permission to execute a state machine:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "ExecuteStateMachine",
            "Effect": "Allow",
            "Action": "states:StartExecution",
            "Resource": [
              "arn-of-the-state-machine"
            ]
        }
    ]
}

Please replace arn of your state machine accordingly.

API endpoint integration request

I assume that you have an API, a resource and an HTTP method on API Gateway that will trigger your step function. So, I continue with what you need to do to establish API Gateway resource method and AWS step function integration.

Integration requests are the sections in an API that configures necessary integrations to call a service or pass the request to another endpoint. For example, your endpoint method can pass everyting to another url or trigger an AWS Lambda function or trigger an AWS service as we will do in this example.

Now, let’s continue with setting up the Integration Request of your API method.

Select AWS Service as the Integration type
Set your AWS region, such as eu-central-1
Select Step Functions as AWS Service
As Exection role, use the arn of the IAM role you created in the previous section.
In AWS Subdomainsection define HTTP method as POST and StartExecutionas action.
Content Handling should be Passthrough

The things above are main properties of your integration request. But, where will we define which state machine this method will trigger? We will use Body mapping templates for this purpose.

Body Mapping Template

Body Mapping Templates section in an Integration Request defines which data will be passed to the AWS service. You can pass everything in the request to the service or define a template to pass certain parameters for the use of the service. We will use the latter and do as below.

First, we need to select When there are no templates defined (recommended) as Request body passthrough. This means that we will define a template for application/json and use it if Content-Type header is also application/json. If we do not define any template, everything will be passthrough to the service. If we define a template, this header needs to match the template or the request will be rejected. This avoids unintentional passthroughs and allows us to control them using templates.
Add a mapping template for application/json which will have these contents:

{
   "input": "{ \"id\": \"$input.params('id')\", \"user_id\": \"$input.path('$').user_id\" }",
   "stateMachineArn": "arn-of-your-state-machine"
}

Now let’s explain attributes of this JSON.

input attribute

Here, input attribute is a string containing a JSON that we use to pass inputs to the state machine. The value is custom and depends on the attributes you need for your state machine. If you can recall our first post our state machine needs two inputs: id for the recipe that will be moved and user_id for the final user that will own the recipe after execution. You can have different inputs for different purposes. Here, the structure is important. You should escape quotes (“) as the value should be a string containing a valid JSON.

Our endpoint in this example has this path structure: \recipes\:id\move. Therefore, we use $input.params('id) to get id from the path. We provide user_id in the request body during the call, so we use $input.path('$').user_id for accessing it. You can learn more about Body Mapping Templates from AWS documentation.

stateMachineArn attribute

This is where we pass which state machine the service should trigger. Simply, pass ARN of your state machine as value.

How about other sections?

The rest of the definitions on API Gateway resource method has same properties as triggering an AWS Lambda function. You can define a Cognito authorizer in Method Request section for authorization and/or define HTTP responses for Integration Response and Method Response sections.

Conclusion

AWS Step Functions and state machines are practical resources to orchestrate our serverless microservices. As in our example, with a single API call we can trigger multiple AWS Lambda function executions at once: Firstly, a function to update the recipe in Amazon RDS database and then trigger 3 parallel Amazon Elasticsearch indexing function executions for the recipe, the previous user and the new owner. The beauty is that we do not need to orchestrate these in our front end and can keep it light and business focused. We delegate it to the back end and it runs all necessary functions in sequential or parallel.

Thanks for reading!