Having a Ruby on Rails background, I got excited to try Ruby to code my AWS Lambda functions when its support for AWS Lambda was announced at Re:Invent 2018. Finally, to try Ruby on AWS Lambda, I developed a simple API using AWS Serverless Application Model (SAM) to access an Amazon RDS MySQL database. I wanted to compare it with my experiences of doing this with Python and Node.js.
Although it may seem simple, installing
mysql2 gem as a dependency proved itself to be challenging than others, because it has native extensions and depends on the environment you use
bundler. As always, Docker simplifies our job by providing a similar environment that our AWS Lambda function runs inside.
In this post, I will make an introduction to AWS SAM CLI and using Ruby for AWS Lambda functions. Let’s begin!
Introduction to AWS Serverless Application Model and SAM CLI
In this example, I will use AWS Serverless Application Model (SAM) and SAM CLI to setup the project and automate the deployment of our API and AWS Lambda functions.
If you developed AWS Lambda functions in the past, you may heard about the popular Serverless Framework. It is a third party framework written in Node.js to simplify the creation and management of serverless applications not only on AWS, but also on other platforms like Google Cloud and Azure. It has cool features and we may discuss them in the future as well.
AWS Serverless Application Model and SAM CLI is a similar tool to create and manage serverless applications specifically on AWS, but it is developed and maintained by AWS. It is an extension of AWS CloudFormation. So if you know CloudFormation well enough, it becomes easier to get used to SAM. AWS also launched SAM CLI as a CLI tool to make the project setup easier.
Creating a project with SAM CLI for Ruby 2.5 AWS Lambda environment
Installing SAM CLI
I will not dive into the details of installing SAM CLI here. If you are using Mac OSX and python 3 was installed in your machine, it is simple to install it using
For other environments, please go to SAM reference. You can find it in
References section at the end of this post.
By the way, you will also need to install Docker for building native extensions and testing your serverless applications using SAM in the future.
Example Project Database
In this project, I created a MySQL database on an Amazon RDS instance during the launch. Then I created an
items table inside it having only
description columns. It can be something below.
Then, I inserted 3 rows. I used some fruit names as items. You can insert anything you like. It is a simple database.
Initializing the Project
To build an AWS SAM project we need a template file, a project folder for our AWS Lambda function and other intermediary files. Luckily, SAM CLI provides a command to initialize our folder.
sam init command initializes the AWS Lambda environment in Node.js. So we need to provide
ruby2.5 as its runtime and a name for our project. This will create the folder structure below.
hello_world folder contains our AWS Lambda function and a basic example inside it. I will name it to
items to make it look like a real project.
app.rb inside this folder is the Ruby file that contains our AWS Lambda function code. I will keep it as it is, but it can also be named differently.
We need to get into
template.yamlis the main template we will use to define the resources and deploy our project.
testsfolder contains some unit tests.
Gemfilein the root folder are also used in tests. But, I will not cover unit testing in this post.
When your AWS Lambda functions need to access Amazon RDS, they have to be run inside a VPC and the security group attached to your database instance should allow access to the security group attached to your AWS Lambda functions. In a previous post in this blog, I shared how to do this. Please read that post for more information about how to run an AWS Lambda function in a VPC to access an Amazon RDS instance.
template.yaml file is actually a CloudFormation template file containing a
Transform section to integrate it with SAM.
Transform section defines the version of the SAM used in this template. Hence, we can add parameters to increase the reusability of it. The final version of our template becomes as below.
As you see, I added parameters for Lambda security group and subnets as well as our DB endpoint, port and credentials. So we can re-use this template to create multiple environments from it.
In this template, our only resource is
ItemsListFunction. But SAM will also create an API Gateway API and an IAM role from its definition.
Now let me explain
- Its type is
AWS::Serverless::Functionmaking it an AWS Lambda function.
- We provide the folder containing the Lambda code using
CodeUri. Please remember that I changed the
hello_worldfolder name to
- When using SAM, we do not need to create an IAM role separately. We can provide the policy template to it and it creates the role itself. This is why we use a
Policiesproperty. I am using a managed policy template called
VPCAccessPolicy. It simplifies the creation of IAM policies to provide AWS Lambda VPC access. For more examples of these, you can visit SAM’s policy templates documentation at GitHub
- We defined environment variables that will be defined in our function using
VpcConfigproperty allows us to define the security group attached to this function and the subnets it will be run inside.
In our template,
Globals section is used to define shared properties in all AWS Lambda functions unless they overwrite those properties. By the way, I also edited
Outputs of the template to adapt it to our name change.
Events property, we define how this function will be triggered. There are multiple methods like S3 events, SNS topic as well as API Gateway endpoint as we do here. This definition will create an
ItemsApi along with the function on API Gateway containing an
items resource and a
GET method inside it triggering our Lambda function. The integration method will be
By the way, we can create API separately and control its definition using a separate
Swagger file as I also do in most of my projects. But there is no need to do over-engineering in this example, so let’s keep it simple. Maybe I can cover it in another blog post.
items/app.rb - Lambda function’s Ruby code
If you noticed in the
Handler property has its value as
app.lambda_handler. This means that our Lambda code will be in an
app.rb file and that file must have an
lambda_handler function which will be run as our AWS Lambda function.
The final code becomes as below.
As in a Rails application, we will use
mysql2 gem to access our MySQL database. We need to place db connection code outside the
lambda_handler, so it can be run only when the function initialized first and all subsequent function runs can reuse the existing connection.
$client global variable allows us to achieve this.
Our query is simple. It returns all records of the
items table. We need to convert the query result to an array to be able to convert it to a proper json while returning the result.
As in Python and Node.js APIs which is configured as lambda_proxy on AWS API Gateway, we need to return a
body in our JSON response. As you may understand,
StatusCode is the HTTP response code and
body is the response body containing our records in JSON.
Packaging the AWS Lambda function
Until now, it was simple. We have a simple Ruby code accessing a MySQL database and a template file containing the definitions. But packaging this function is the trickiest part.
mysql2 gem has native extensions, so it must be built in the same environment that will run our AWS Lambda function. We will do this by running a Docker container built from AWS Lambda Ruby 2.5 runtime image.
Unless we use AWS Lambda layers, we need to package all libraries along with our Lambda function. So we will start with including
Gemfile in the root folder is for deploying the function locally, for testing. Please be sure to change the
Gemfile under the function’s folder.
The contents of
items/Gemfile becomes as below.
In this example, I do not neet
httparty gem, so I intentionally removed it from the file.
Then, we need to update our
bundler and lock the gem file locally using
bundle install inside this folder.
mysql2 gem using Docker
All external dependencies of our function should be under
vendor/bundle folder inside the function folder or in other words,
CodeUri value of the function in
template.yaml. It is the
items folder in our example.
mysql2 gem did not contain native extensions we could use
sam build command or
bundle install --deployment commands to install the gem and package with our function. But if you tried these, you may have already saw that your function executions fail because of this gem.
Therefore, we need to install this gem in a similar environment that our AWS Lambda function will be run into. To do this, we will create a Docker container using the Docker image for AWS Lambda Ruby 2.5 runtime. But we need to do this inside the function folder,
This will mount the current directory (
/var/task folder in our container which will also be the entry point for AWS Lambda and allow us the connect to it using
Now let’s proceed with building this gem. Inside the container, firstly we will install
Then we will update the
bundler inside the container to make it consistent with the bundler we used locally to lock the Gemfile. Then we will run
bundle install with
--deployment option to install the gem inside
vendor/bundle in the current folder.
Alternatively, we could also run the commands until now while starting your container as below.
After running bundler, you will see that
vendor/bundle folders were created in
items folder. This is where AWS Lambda searches dependencies of your Lambda function in a Ruby 2.5 environment.
However, this is not enough if you stop here and deploy the function. Because, AWS Lambda will also search for
libmysqlclient.so library inside the
lib folder to be able to use
mysql2 gem. So we will create a
lib folder in the current directory and copy
libmysqlclient.so.18 file in our container to there. This will also create a
lib folder in our local machine inside function folder as we mounted it as
/var/task while running the container.
Now we can exit the Docker container and package our function and deploy it using SAM CLI.
Packaging and deploying with SAM
Next, we will package our serverless application using SAM. It will prepare a zip file containing the necessary files inside the
items folder and also create a new template file as output which will be ready for deployment. Then we will deploy the package using
sam deploy command.
As we have parameters in our
template.yaml, I find it feasible to create a
deploy.sh bash script under the project folder and execute it to simplify the process.
The contents of
myApi/deply.sh is below.
Of course, you should edit
SAM_BUCKET variables with your own values as explained in this post.
SAM_BUCKET corresponds to
--s3-bucket option in
sam package command. It defines the S3 bucket name that your AWS Lambda artifact will be uploaded after packaging. You will see the references to the S3 url of these artifacts in
packaged-serverless.yaml file created as output of
sam package command. Then let’s explain both commands.
sam packagecommand takes a template file from its
--template-fileoption and produces an output file defined in
--output-template-fileoption which is
packaged.serverless.yamlin our example. It also uploads the Lambda artifacts to the S3 bucket defined in
sam deploycommand takes the output template file produced by
sam packagecommand from its
--template-fileoption and creates a CloudFormation stack. As you may understand, the inputs are same as
aws cloudformation deploycommand.
--stack-nameoption defines the name of the stack,
--parameter-overridesis used to provide the parameter values and
--capabilities CAPABILITY_IAMallows the call to be able to create an IAM role for the AWS Lambda function.
Making HTTP Calls
After stack creation finishes successfully, you can make a call to your API endpoint using
This call will return all the items as an array of items in JSON.
Although installing and packaging
mysql2 gem is more difficult when compared to installing
pymysql library for Python AWS Lambda functions with
pip, I am happy to be able to use Ruby for my AWS Lambda functions.
Normally, using DynamoDB is the way to go when you are creating a serverless application from scratch. I plan to try
aws-record gem to access a DynamoDB table soon. But if you have existing data on an Amazon RDS MySQL database, you can use Ruby for your AWS Lambda functions and create an API using AWS Serverless Application Model as well.
If you enjoyed this post and would like to follow my new posts, you can follow me on Twitter and LinkedIn.
Thanks for reading!
- What Is the AWS Serverless Application Model (AWS SAM)? | AWS Reference
- Installing the AWS SAM CLI | AWS SAM Reference
- AWS SAM CLI Command Reference | AWS SAM Reference
- AWS SAM Specification | GitHub
- https://docs.aws.amazon.com/lambda/latest/dg/lambda-ruby.html | AWS Lambda documentation
- Building Gems with Native Binary Packages in AWS Lambda | Blackninja Dojo Blog