When it comes to building a reliable and scalable cloud infrastructure, efficient communication between your cloud components is essential. There are many AWS services to help you build a reliable communication architecture for your solution. So, in this post, we will discuss and compare two old and popular AWS messaging services, Amazon Simple Notification Service (SNS) and Amazon Simple Queue Service (SQS). These services allow you to easily exchange messages in a distributed system.
Amazon SNS and Amazon SQS are fully managed and serverless AWS services that offer automated communication. While they may sound similar, they each have unique features that set them apart. In the following sections, we will delve into these features, providing a comprehensive understanding of each service. Let’s start with Amazon SNS.
Amazon SNS is push-based!
The delivery method is the first and most notable difference between Amazon SNS and Amazon SQS. SNS uses an instant ‘push’ mechanism. This means messages are automatically delivered to the subscribed consumers, and you don’t need to check for or ‘poll’ updates. It sends copies of the same message to multiple consumers via topics.
Amazon SNS supports both application-to-application (A2A) communication (with AWS Lambda, AWS SQS, etc.) and application-to-person (A2P) communication (with email, SMS, etc.). Using Amazon SNS, you can build apps that receive push notifications or notify humans to react to system events.
For example, you can subscribe an AWS Lambda function to an SNS topic, which can do custom processing when receiving a message. You can also subscribe an email address to the same topic to be notified about this event. SNS also allows you to send SMS or mobile push notifications via platform-specific services, such as Apple Push Notification Serves for messages to iOS and macOS apps.
What is an SNS topic?
On Amazon SNS, messaging revolves around three main components: the SNS topic, the publisher, and the subscriber. The SNS topic serves as a logical access point where you can group multiple endpoints such as HTTPS, an email address, AWS Lambda, or an Amazon SQS queue. When you name an SNS topic, the service understands which endpoints you want to send messages to. Consumers who wish to receive notifications must subscribe to relevant topics. Multiple consumers can subscribe to a single topic, and multiple publishers can send messages. When a publisher sends a message to a topic, all subscribed clients are immediately notified. However, the publisher and the subscriber must have the necessary IAM permissions.
There are two types of SNS topics: Standard and FIFO.
Amazon SNS offers two different topic types: standard topics and FIFO (first in, first out) topics. When using standard topics, messages can be targeted to many endpoints, such as an email address, text messaging (SMS), AWS Lambda, etc. However, messages are not guaranteed to be delivered in the order received by Amazon SNS. Standard SNS topics perform at least once delivery. So, your endpoints must be ready for duplicate messages.
With FIFO topics, you can guarantee strict message ordering in the order received by your topic. But you can only subscribe SQS queues to a FIFO SNS topic and not other endpoints. The messages sent to a FIFO topic are delivered to subscribed SQS queues exactly once.
Let’s leave comparing SNS topic types to another blog post. You can easily set your SNS topic type while creating a topic on your AWS console:
Amazon SQS uses a pull mechanism!
We mentioned the main features of Amazon SNS above. Now, let’s have a look at Amazon SQS.
Unlike Amazon SNS, Amazon SQS uses a ‘polling’ model, which means messages are not automatically pushed. Instead, they are stored in a queue until the consumer pulls and processes messages from the queue.
Amazon SQS is especially suitable for processing large numbers of messages since it allows for parallel processing with multiple queues. It is generally used to decouple distributed systems, microservices, etc. This enables asynchronous communication between different components. So, using SQS queues can boost your user experience as multiple workloads can be independently processed simultaneously.
For example, I used SQS queues in the past to queue image-processing jobs and indexing jobs for Elasticsearch (Amazon OpenSearch now). In each use case, I implemented AWS Lambda functions to process the images or trigger the indexing operation on the Elasticsearch domain. It was a Rails application, and I replaced Sidekiq workers on EC2 instances with this solution. Not only I decoupled my app and job servers, but I also migrated the background jobs to a cost-effective, reliable and scalable serverless architecture.
The diagram below displays how messages are delivered to consumers through Amazon SQS:
Besides, Amazon SQS has the advantage of persisting of your messages. Messages can stay in the queue for up to 14 days, providing better reliability than Amazon SNS. Also, unlike Amazon SNS, only one consumer can process messages at a time, and it only supports A2A communication. You can easily customize the settings of an Amazon SQS queue, such as message retention period, maximum message size, etc.
Variants of SQS Queue Types
Like Amazon SNS, you can use standard and FIFO queues in Amazon SQS. When you use standard queues, messages will be delivered in no guaranteed order. If you need message ordering, you can create FIFO (first in, first out) queues. They enable the messages to be delivered in the same order they are sent to the queue.
You can use delay queues to postpone the delivery of your messages for a specific time. Besides, Amazon SQS offers dead letter queues to route the messages that can’t be processed due to a failure to be reprocessed later. These features increases reliability of your applications.
Which solution would be best for your need?
We analyzed each service. But how can you decide which one suits your use case best? Let’s focus on the main factors you should take into consideration:
1) First and foremost, Amazon SNS and Amazon SQS serve different purposes and operate in distinct ways.
Amazon SNS excels at sending multiple copies of messages to numerous subscribers, making it ideal for notifications, updates, reminders, and alerts. For instance, you can use Amazon SNS to inform subscribers about marketing campaigns or notifying your personnel on CloudWatch alarms.
On the other hand, Amazon SQS is designed to decouple distributed systems, making it the preferred choice for applications that require asynchronous communication between various components.
2) On Amazon SQS, only one consumer can process a message. Also, Amazon SQS allows for batch processing; sending ten messages in one batch is possible. However, messages cannot be sent in batches with Amazon SNS; one message is forwarded to multiple consumers.
3) If you want to be sure your message is delivered to your consumer, you should choose Amazon SQS. Exact delivery is ensured with Amazon SQS in most cases. However, with Amazon SNS, there is no certainty that your message will reach a consumer if the consumer is unavailable for some reason. It can be deleted without consumer receiving the message. So, if reliability is your priority, then you should go with Amazon SQS, which provides message persistency and features like dead-letter queues.
4) If latency is an issue, consider using Amazon SNS. Although Amazon SQS offers better reliability and persistence, its performance speed is slightly lower than Amazon SNS. The reason is that messages need to be polled in Amazon SQS, which takes extra time, while they are immediately pushed with Amazon SNS.
Combination of SNS and SQS: The Fanout Pattern
Despite their differences, Amazon SNS and Amazon SQS can complement each other effectively. When used in a fanout pattern, they can significantly enhance performance. This approach leverages SQS’s asynchronous message processing capability, leading to a more efficient system.
In the fanout pattern, a message published to an SNS topic is sent to many SQS queues simultaneously. To fan out messages to SQS queues, you must first create a standard SNS topic. Then, subscribe the SQS queues to that SNS topic. After that, the message will be delivered to all SQS queues subscribed to the topic. This enables parallel message processing.
Let’s clarify this more with a typical example. Let’s say you use Amazon SNS and Amazon SQS together, and suppose you have an application that allows users to upload photos. When a user uploads an image, a message is published to an SNS topic, which will then trigger multiple SQS queues for various tasks. This means that each process, such as image recognition or thumbnail creation, can be performed in parallel, saving time and improving user experience. Your users won’t have to wait for each process to be completed because they will all be handled in the background. This can be a more efficient and user-friendly solution in such cases.
Conclusion
In this post, we discussed the key features of AWS’s two popular services: Amazon SNS and Amazon SQS. We described their different use cases and functions.
To sum up, Amazon SNS is a push notification service usually used to send quick notifications to multiple subscribers. On the other hand, Amazon SQS is a reliable queue-type messaging solution with a pull mechanism. Moreover, in some cases, combining them can be more efficient.
I hope this post helps you understand better the main characteristics of Amazon SNS and Amazon SQS. Thank you for reading! I look forward to seeing you in our other posts.
References
