Did this page help you? Yes No Tell us about it...

How to Generate a Signature for a Query Request in Amazon EMR

Web service requests are sent across the Internet and thus are vulnerable to tampering. To check that the request has not been altered, Amazon Elastic MapReduce (Amazon EMR) calculates the signature to determine if any of the parameters or parameter values were changed en route. Amazon EMR requires a signature as part of every request.

The following topics describe the steps needed to calculate a signature using the AWS signature version 2.

Format the Query Request

Before you can sign the Query request, you must put the request into a completely unambiguous format. This is needed because there are different—and yet correct—ways to format a Query request, but the different variations would result in different HMAC signatures. Putting the request into an unambiguous, canonical, format before signing it ensures that your application and Amazon EMR will calculate the same signature for a given request.

The unambigous string to sign is built up by concatenating the Query request components together as follows. As an example, let's generate the string to sign for the following call to DescribeJobFlows.

https://elasticmapreduce.amazonaws.com?Action=DescribeJobFlows
&Version=2009-03-31
&AWSAccessKeyId=AKIAIOSFODNN7EXAMPLE
&SignatureVersion=2
&SignatureMethod=HmacSHA256
&Timestamp=2011-10-03T15%3A19%3A30

To create the string to sign (signature version 2)

Start with the request method (either GET or POST), followed by a newline character. (In the following, for human readability, the newline character is represented as \n.)
```
GET\n					
				
```
Add the HTTP host header in lowercase, followed by a newline character. The port information is omitted if it is the standard port for the protocol (port 80 for HTTP and port 443 for HTTPS), but included if it is a non-standard port.
```
elasticmapreduce.amazonaws.com\n
				
```
Add the URL-encoded version of the absolute path component of the URI (this is everything between the HTTP host header to the question mark character (?) that begins the query string parameters) followed by a newline character. If the absolute path is empty, use a forward slash (/).
```
/\n
				
```

Add the query string components (the name-value pairs, not including the initial question mark (?) as UTF-8 characters which are URL encoded per RFC 3986 (hexadecimal characters must be uppercased) and sorted using lexicographic byte ordering. Separate parameter names from their values with the equal sign character (=) (ASCII character 61), even if the value is empty. Separate pairs of parameter and values with the ampersand character (&) (ASCII code 38). All reserved characters must be escaped. All unreserved characters must not be escaped. Concatenate the parameters and their values to make one long string with no spaces between them. Spaces within a parameter value, are allowed, but must be URL encoded as %20. In the concatenated string, period characters (.) are not escaped. RFC 3986 considers the period character an unreserved character, and thus it is not URL encoded.

	Note
RFC 3986 does not specify what happens with ASCII control characters, extended UTF-8 characters, and other characters reserved by RFC 1738. Since any values may be passed into a string value, these other characters should be percent encoded as %XY where X and Y are uppercase hex characters. Extended UTF-8 characters take the form %XY%ZA... (this handles multi-bytes). The space character should be represented as '%20'. Spaces should not be encoded as the plus sign (+) as this will cause as error.

Note

RFC 3986 does not specify what happens with ASCII control characters, extended UTF-8 characters, and other characters reserved by RFC 1738. Since any values may be passed into a string value, these other characters should be percent encoded as %XY where X and Y are uppercase hex characters. Extended UTF-8 characters take the form %XY%ZA... (this handles multi-bytes). The space character should be represented as '%20'. Spaces should not be encoded as the plus sign (+) as this will cause as error.

The following example shows the query string components of a call to DescribeJobFlows, processed as described above.

AWSAccessKeyId=AKIAIOSFODNN7EXAMPLE&Action=DescribeJobFlows&SignatureMethod=HmacSHA256&SignatureVersion=2&Timestamp=2011-10-03T15%3A19%3A30&Version=2009-03-31

The string to sign for the call to DescribeJobFlows takes the following form:

GET\n
elasticmapreduce.amazonaws.com%0A
/\n
AWSAccessKeyId=AKIAIOSFODNN7EXAMPLE&Action=DescribeJobFlows&SignatureMethod=HmacSHA256&SignatureVersion=2&Timestamp=2011-10-03T15%3A19%3A30&Version=2009-03-31

Calculating the Signature

After you've created the canonical string as described in Format the Query Request, you calculate the signature by creating a hash-based message authentication code (HMAC) using either the HMAC-SHA1 or HMAC-SHA256 protocols.

You then add the value returned to the Query request as a signature parameter, as shown below. You can then use the signed request in an HTTP or HTTPS call. Amazon EMR will then return the results of the call formatted as a response. For more information about the inputs and outputs of the Amazon EMR API calls, go to the Amazon Elastic MapReduce Developer Guide.

https://elasticmapreduce.amazonaws.com?AWSAccessKeyId=AKIAIOSFODNN7EXAMPLE&Action=DescribeJobFlows&SignatureMethod=HmacSHA256&SignatureVersion=2&Timestamp=2011-10-03T15%3A19%3A30&Version=2009-03-31&Signature=lptk88A0LEP2KfwM3ima33DUjY0e%2FyfF7YfitJ%2FQw6I%3D

The AWS SDKs offer functions to generate Query request signatures. To see an example using the AWS SDK for Java, go to Using the Java SDK to Sign a Query Request. For an example using AWS Ruby Gems, go to Generating a Query Request Using AWS Ruby Gems.

Troubleshooting Request Signatures in Amazon EMR

This section describes some error codes you might see when you are initially developing code to generate the signature to sign Query requests.

SignatureDoesNotMatch Signing Error in Amazon EMR

The following error response is returned when Amazon EMR attempts to validate the request signature by recalculating the signature value and generates a value that does not match the signature you appended to the request. This can occur because the request was altered between the time you sent it and the time it reached the Amazon EMR endpoint (this is the case the signature is designed to detect) or because the signature was calculated improperly. A common cause of the error message below is not properly creating the string to sign, such as forgetting to URL encode characters such as the colon (:) and the forward slash (/) in Amazon S3 bucket names.

			
<ErrorResponse xmlns="http://elasticmapreduce.amazonaws.com/doc/2009-03-31">
  <Error>
    <Type>Sender</Type>
    <Code>SignatureDoesNotMatch</Code>
    <Message>The request signature we calculated does not match the signature you provided. 
    Check your AWS Secret Access Key and signing method. 
    Consult the service documentation for details.</Message>
  </Error>
  <RequestId>7589637b-e4b0-11e0-95d9-639f87241c66</RequestId>
</ErrorResponse>

IncompleteSignature Signing Error in Amazon EMR

The following error indicates that signature is missing information or has been improperly formed.

<ErrorResponse xmlns="http://elasticmapreduce.amazonaws.com/doc/2009-03-31">
  <Error>
    <Type>Sender</Type>
    <Code>IncompleteSignature</Code>
    <Message>Request must contain a signature that conforms to AWS standards</Message>
  </Error>
  <RequestId>7146d0dd-e48e-11e0-a276-bd10ea0cbb74</RequestId>
</ErrorResponse>

Document Conventions
Terms of Use

Did this page help you? Yes No Tell us about it...