Introduction to Amazon SQS

Topics

This introduction to Amazon SQS is intended to give you a high-level overview of this web service. After reading this section, you should understand the basics you need to work through the examples in this guide.

Overview of Amazon SQS

A queue is a temporary repository for messages that are awaiting processing. Amazon SQS is a distributed queue system that enables web service applications to quickly and reliably queue messages that one component in the application generates to be consumed by another component. Using Amazon SQS, you can decouple the components of an application so they run independently, with Amazon SQS easing message management between components. Any component of a distributed application can store any type of data in a fail-safe queue. Any other component can then later receive the data programmatically using the SQS API.

The queue acts as a buffer between the component producing and saving data, and the component receiving the data for processing. This means the queue resolves issues that arise if the producer is producing work faster than the consumer can process it, or if the producer or consumer are only intermittently connected to the network.

SQS ensures delivery of each message at least once, and supports multiple readers and writers interacting with the same queue. A single queue can be used simultaneously by many distributed application components, with no need for those components to coordinate with each other to share the queue.

Amazon SQS is engineered to always be available and deliver messages. One of the resulting tradeoffs is that SQS does not guarantee first in, first out delivery of messages. For many distributed applications, each message can stand on its own, and as long as all messages are delivered, the order is not important. If your system requires that order be preserved, you can place sequencing information in each message, so that you can reorder the messages when the queue returns them.

Features

Amazon SQS provides the following major features:

  • Redundant infrastructure—Guarantees delivery of your messages at least once, highly concurrent access to messages, and high availability for sending and retrieving messages

  • Multiple writers and readers—Multiple parts of your system can send or receive messages at the same time

    SQS locks the message during processing, keeping other parts of your system from processing the message simultaneously.

  • Configurable settings per queue—All of your queues don't have to be exactly alike

    For example, one queue can be optimized for messages that require a longer processing time than others.

  • Variable message size—Your messages can be up to 8 KB in size

    For even larger messages, you can store the contents of the message using the Amazon Simple Storage Service (Amazon S3) or Amazon SimpleDB and use Amazon SQS to hold a pointer to the Amazon S3 or Amazon SDB object. Alternately, you can split the larger message into smaller ones.

    For more information about the services, go to the Amazon S3 detail page and the Amazon SimpleDB detail page.

  • Unlimited queues and messages—You can have as many queues and messages in the Amazon SQS system as you want

Message Lifecycle

The following diagram and steps describe the lifecycle of an Amazon SQS message, called Message A, from creation to deletion. Assume that a queue already exists.

Message Lifecycle

Message Lifecycle

Component 1 sends Message A to a queue and the message is redundantly distributed across the SQS servers.

When Component 2 is ready to process a message, it retrieves messages from the queue, and Message A is returned. While Message A is being processed, it remains in the queue and is not returned to subsequent receive requests for the duration of the visibility timeout.

Component 2 deletes Message A from the queue to avoid the message being received and processed again once the visibility timeout expires.

[Note]Note

SQS automatically deletes messages that have been in a queue for more than 4 days.

About the Samples

You can see from the preceding discussion that, in general, your system establishes a queue, confirms it's ready to use, and then starts using it. During use, the various components of your system continually send, receive, and delete messages. The sample libraries available with this guide cover all possible operations available with SQS. However, the examples in this guide focus specifically on the core queue operations:

  • Creating a queue

  • Listing your queues

  • Sending a message to a queue

  • Retrieving messages from a queue

  • Deleting a message from a queue

For more information about the samples, see Preparing the Samples.

For information about the other operations you can perform with SQS, refer to the Amazon Simple Queue Service Developer Guide.