Configuring Rails Logging for Docker on Amazon ECS & Fargate

by Emre Yilmaz
Nov 14, 2018

AWS • DevOps • Ruby on Rails • Amazon ECS • AWS Fargate • Amazon CloudWatch
Istanbul

When you dockerize a Ruby on Rails application on AWS, it is essential to configure logging correctly to monitor application health. There are some tweaks to achieve this and I will briefly describe the process in this blog post.

Firstly, let’s make a brief introduction to Docker along with its deployment options on AWS.

About Docker, Amazon Elastic Container Service and AWS Fargate

From its launch nearly 5 and a half years ago, we witnessed the growing popularity of Docker and how it almost became a standard in DevOps. In simple words, Docker allows us to create immutable containers for our code including a virtualized OS, all needed packages and an entry command that is executed during container launch. It solves problems such as compatibility issues between development and production; if your container runs in your machine it should run in production, too. Also, when you update a code, you build a new container image version, remove containers running older version and create new ones with the latest version. It is like replacing the broken parts of your car with an original,brand-new version instead of trying to fix them. Docker became the default architecture in deploying microservices as well.

As continuously updating its platform, AWS could not be indifferent to the rise of Docker and launched new managed services to orchestrate Docker containers. Let’s talk briefly about them.

Amazon Elastic Container Service (ECS): While using ECS, you provision container instances, mostly with an autoscaling architecture. Then you define your Docker containers as task and service definitions and ECS places them in the cluster you provisioned. In this architecture, you both manage instances and your Docker containers.
AWS Fargate for ECS: You can think, Fargate an enhanced version of Amazon ECS in terms of maintaining the infrastructure. You simply don’t do this anymore. You only define your containers as task and service definitions and Fargate runs and manages them as you require. If you are new to AWS, it might be simpler than ECS for you.

AWS Fargate can be a bit more costly if you have a service which has a steady traffic. Because you can make use of reserved instances when you use Amazon ECS in instance mode. If your traffic has ups and downs during the day, AWS Fargate would be more cost-effective as Fargate will only scale the number of containers according to need.

Amazon ECS for Kubernetes (EKS): Kubernetes is very popular among Docker community and most people create a Kubernetes infrastructure on AWS. This service simplifies this process. It is simply the Amazon ECS version to deploy Kubernetes infrastructure using specialized AMIs for this purpose and define your containers as pods, services. You can use all tools you use in using your current Kubernetes infrastructure. Without Kubernetes, Amazon ECS uses the daemon developed by AWS.
AWS Fargate for EKS: I remember that this service was announced as “in progress” last year in Re:Invent 2017. I expect it to be announced in this year’s Re:Invent conference a few weeks later. As you might guess, this will be the EKS version of Fargate.

By using one of these services, you benefit from integration with other AWS services out of the box. For example, you can use IAM roles for your tasks as you use in your EC2 instances. For logging, you can configure AWS CloudWatch Logs integration with a few definition and use CloudWatch as your centralized logging service along with its advantages.

You have other options on AWS for deploying Docker containers. For example, you can install Docker or Docker Swarm on EC2 instances and maintain the architecture yourself. But, I will not recommend this; because instead, you can focus on developing your containers by using managed services. Alternatively, you can use Elastic Beanstalk for deploying Docker containers, but you will have less options when compared to ECS.

Let’s continue with updating our Rails application’s logging configuration to allign it with Docker logging.

How to configure Ruby on Rails logging for Docker

To configure Ruby on Rails application logging correctly in a Docker container, we need to understand how Docker configures logging. In Unix and Linux systems, when you run a command in your terminal it opens three IO streams: STDIN for inputs, STDOUT for outputs and STDERR for errors. By default, when you use “docker logs” command on a container or configure Docker logging driver as we will do in this post to send logs to a centralized service, Docker uses STDOUT and STDERR streams. Hence, we need to configure our Rails command to log to one of these streams. As our Rails logs will contain also regular INFO logs, STDOUT will be the proper choice.

I will show you how to do this only for production environment. Because, I configured my Docker execute command to run my Ruby on Rails application in production. You can change the configuration as you need.

In our Rails project we change the logging driver in config/environments/production.rb as below:

config.logger = Logger.new(STDOUT)

Then, you will need to build your Docker image and upload it to a Docker repository. I use Amazon Elastic Container Registry (ECR) for this purpose and AWS CodePipeline to automate this process. We might discuss this in another blog post in the future.

How to configure Amazon ECS and AWS Fargate for logging to Amazon CloudWatch Logs

The steps for Amazon ECS and AWS Fargate task definitions to configure Docker logging driver to send logs to Amazon CloudWatch Logs are nearly same. I will show you how to do this using AWS CloudFormation to create your task definition.

Let’s assume that I have only one container definition in my task. For Amazon ECS this becomes:

  TaskDefinition:
    Type: AWS::ECS::TaskDefinition
    Properties:
      ExecutionRoleArn: !Sub "arn:aws:iam::${AWS::AccountId}:role/ecsTaskExecutionRole"
      TaskRoleArn: !GetAtt ECSTaskIAMRole.Arn
      Family: "MyTaskFamily"
      ContainerDefinitions:
        -
          Name: "web-service-container"
          Essential: "true"
          Image: !Sub "${AWS::AccountId}.dkr.ecr.${AWS::Region}.amazonaws.com/${ImageName}:latest"
          Memory: !Ref MemoryLimit
          PortMappings:
            -
              ContainerPort: "3000"
              Protocol: tcp
          LogConfiguration:
            LogDriver: awslogs
            Options:
                awslogs-group: !Ref AWS::StackName
                awslogs-region: !Ref AWS::Region

You can see LogConfiguration under ContainerDefinitions attribute. This is where the log configuration is done. In Docker documentation, you can find awslogs as a provided logging driver with the same options. Therefore, we actually use Docker’s built in mechanism.

For AWS Fargate, the same task definition becomes:

  TaskDefinition:
    Type: AWS::ECS::TaskDefinition
    Properties:
      Cpu: !Ref ContainerCpu
      Memory: !Ref ContainerMemory
      ExecutionRoleArn: !Sub "arn:aws:iam::${AWS::AccountId}:role/ecsTaskExecutionRole"
      TaskRoleArn: !GetAtt ECSTaskIAMRole.Arn
      Family: "MyFargateWebServiceFamily"
      NetworkMode: awsvpc
      RequiresCompatibilities:
        - FARGATE
      ContainerDefinitions:
        -
          Name: "web-service-container"
          Essential: "true"
          Image: !Sub "${AWS::AccountId}.dkr.ecr.${AWS::Region}.amazonaws.com/${ImageName}:latest"
          Cpu: !Ref ContainerCpu
          Memory: !Ref ContainerMemory
          PortMappings:
            -
              ContainerPort: "3000"
              Protocol: tcp
          LogConfiguration:
            LogDriver: awslogs
            Options:
                awslogs-group: !Ref AWS::StackName
                awslogs-region: !Ref AWS::Region
                awslogs-stream-prefix: MyFargateWebServiceLogPrefix

Some of the differences such as Cpu and Memory attributes are related to differences between ECS and Fargate. To keep this post on the subject, let’s concentrate on LogConfiguration. As you can see we added awslogs-stream-prefix option which is mandatory for Fargate. This option allows us to associate the log stream with a prefix. The log stream will take this format:

prefix-name/container-name/ecs-task-id

or in our example:

MyFargateWebServiceLogPrefix/web-service-container/58052bc3-7424-4725-a34f-403a40a90582

Where “58052bc3-7424-4725-a34f-403a40a90582” is the custom task id assigned to our container when started by AWS Fargate.

There is one additional thing to add. In both task definitions you can see TaskRoleArn where we defined the IAM role to be attached to our containers. This is the beauty of using Amazon ECS or Fargate, we can manage permisions/credentials while accessing other AWS services as we do in regular EC2 instances. Your Task role should contain the policy below for your container to be able to send logs to CloudWatch Logs:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Action": [
                "logs:CreateLogGroup",
                "logs:CreateLogStream",
                "logs:PutLogEvents",
                "logs:DescribeLogStreams"
            ],
            "Resource": "arn:aws:logs:*:*:*",
            "Effect": "Allow"
        }
    ]
}

Conclusion

In this post, we made an introduction to Docker and its deployment options on AWS. Then, we explained how to configure a Ruby on Rails application to write logs to STDOUT to allow its Docker container to catch them. Finally, we discussed how we can implement Amazon ECS and AWS Fargate task definition configuration for sending container logs to Amazon CloudWatch Logs with a CloudFormation resource example.

Although there are independent parts in configuration, they are related in terms of achieving the end goal: Sending Ruby on Rails application logs to Amazon CloudWatch Logs as we do in regular EC2 instances.

I hope you enjoyed the post.

Thanks for reading!