Introduction to Amazon SQS

Topics

This introduction to Amazon SQS is intended to give you a detailed summary of this web service. After reading this section, you should have a good idea of what SQS offers and how it can fit in with your business.

Overview of Amazon SQS

Amazon SQS is a distributed message queue system that provides a means for web service applications to quickly and reliably queue messages generated by one component to be consumed by another component. Using Amazon SQS, you can decouple the components of an application so they run independently, with Amazon SQS easing message management between components. Any component of a distributed application can store any type of data in a fail-safe queue. Any other component can then later receive the data programmatically using the SQS API.

The queue acts as a buffer between the component producing and saving data, and the component receiving the data for processing. This means the queue resolves issues that arise if the producer is producing work faster than the consumer can process it, or if the producer or consumer are only intermittently connected to the network.

SQS ensures delivery of each message at least once, and supports multiple readers and writers interacting with the same queue. A single queue can be used simultaneously by many distributed application components, with no need for those components to coordinate with each other to share the queue.

Amazon SQS is engineered to always be available and deliver messages. One of the resulting tradeoffs is that SQS does not guarantee first in, first out delivery of messages. For many distributed applications, each message can stand on its own, and as long as all messages are delivered, the order is not important. If your system requires that order be preserved, you can place sequencing information in each message so you can reorder the messages when the queue returns them.

Be sure to read Properties of Distributed Queues, which will help you understand how to design an application that works correctly with Amazon SQS.

Features

Amazon SQS provides the following major features:

Redundant infrastructure—Guarantees delivery of your messages at least once, highly concurrent access to messages, and high availability for sending and retrieving messages.

Management of multiple writers and readers—Multiple parts of your system can send or receive messages at the same time. SQS locks a message while it is being processed, keeping other parts of your system from processing the message simultaneously.

Configurable settings per queue—All of your queues don't have to be exactly alike. For example, one queue can be optimized for small messages and another for large messages.

Large messages—Maximum message size of 256 KB. For even larger messages, you can store the contents of the message using Amazon Simple Storage Service (Amazon S3) and use Amazon SQS to hold a pointer to the Amazon S3 object.
Unlimited queues and messages—You can have as many queues and messages in the Amazon SQS system as you want.
Access control—You can control who can send messages to a queue, and who can receive messages from a queue.

Architectural Overview

There are three main actors in the overall system:

The components of your distributed system
Queues
Messages in the queues

In the diagram below, your system has several components that send messages to the queue and receive messages from the queue. The diagram shows that a single queue, which has its messages (labeled A-E), is redundantly saved across multiple SQS servers.