Did this page help you? Yes No Tell us about it...

View Job Flow Details

This section describes the methods used to view the details of Amazon Elastic MapReduce (Amazon EMR) job flows. You can view job flows in any state.

Example Using the Amazon EMR console

After you start a job flow, you can monitor its status and retrieve extended information about its execution. This section explains how to view the details of a job flow using the Amazon EMR console.

To view the details of a job flow

Sign in to the AWS Management Console and open the Amazon Elastic MapReduce console at https://console.aws.amazon.com/elasticmapreduce/.
Select the job flow you want to view.
The Job Flow pane appears, providing detailed information about the selected job flow.

Example using the CLI

To view job flow details from the CLI, use the --list parameter to list job flows. This section presents some of these variations.

To list job flows created in the last two days

Use the --list parameter with no additional arguments to display job flows created during the last two days as follows:
If you are using... Enter the following...
Linux or UNIX
& ./elastic-mapreduce --list
Microsoft Windows
c:\ruby elastic-mapreduce --list

If you are using...	Enter the following...
Linux or UNIX	& ./elastic-mapreduce --list
Microsoft Windows	c:\ruby elastic-mapreduce --list

The response is similar to the following:

j-1YE2DN7RXJBWU   FAILED      Example Job Flow
                  CANCELLED   Custom Jar
j-3GJ4FRRNKGY97   COMPLETED   ec2-67-202-3-73.compute-1.amazonaws.com   Example job flow
j-5XXFIQS8PFNW    COMPLETED   ec2-67-202-51-30.compute-1.amazonaws.com  demo 3/24 s1
                  COMPLETED   Custom Jar

The example response shows that three job flows were created in the last two days. The indented lines are the steps of the job flow. The information for a job flow is in the following order: the job flow ID, the job flow state, the DNS name of the master node, and the job flow name. The information for a job flow step is in the following order: step state, and step name.

If no job flows were created in the previous two days, this command produces no output.

To list active job flows

Use the --list and --active parameters as follows:
If you are using... Enter the following...
Linux or UNIX
& ./elastic-mapreduce --list --active
Microsoft Windows
c:\ruby elastic-mapreduce --list --active

If you are using...	Enter the following...
Linux or UNIX	& ./elastic-mapreduce --list --active
Microsoft Windows	c:\ruby elastic-mapreduce --list --active

The response lists job flows that are in the state of STARTING, RUNNING, or SHUTTING_DOWN.

To list only running or terminated job flows

Use the --state parameter as follows:

If you are using...	Enter the following...
Linux or UNIX	& ./elastic-mapreduce --list --state RUNNING --state TERMINATED
Microsoft Windows	c:\ruby elastic-mapreduce --list --state RUNNING --state TERMINATED

The response lists job flows that are running or terminated.

You can get information about a job flow using the --describe parameter and specifying a job flow ID.

To retrieve information about a job flow

Use the --describe parameter with a valid job flow ID.

If you are using...	Enter the following...
Linux or UNIX	& ./elastic-mapreduce --describe --jobflow `JobFlowID`
Microsoft Windows	c:\ruby elastic-mapreduce --describe --jobflow `JobFlowID`

The response looks similar to the following:

{
  "JobFlows": [
    {
      "Name": "Development Job Flow (requires manual termination)",
      "LogUri": "s3n:\/\/AKIAIOSFODNN7EXAMPLE\/FileName\/",
      "ExecutionStatusDetail": {
        "StartDateTime": null,
        "EndDateTime": null,
        "LastStateChangeReason": "Starting instances",
        "CreationDateTime": DateTimeStamp,
        "State": "STARTING",
        "ReadyDateTime": null
      },
      "Steps": [],
      "Instances": {
        "MasterInstanceId": null,
        "Ec2KeyName": "KeyName",
        "NormalizedInstanceHours": 0,
        "InstanceCount": 5,
        "Placement": {
          "AvailabilityZone": "us-east-1a"
        },
        "SlaveInstanceType": "m1.small",
        "HadoopVersion": "0.20",
        "MasterPublicDnsName": null,
        "KeepJobFlowAliveWhenNoSteps": true,
        "InstanceGroups": [
          {
            "StartDateTime": null,
            "SpotPrice": null,
            "Name": "Master Instance Group",
            "InstanceRole": "MASTER",
            "EndDateTime": null,
            "LastStateChangeReason": "",
            "CreationDateTime": DateTimeStamp,
            "LaunchGroup": null,
            "InstanceGroupId": "InstanceGroupID",
            "State": "PROVISIONING",
            "Market": "ON_DEMAND",
            "ReadyDateTime": null,
            "InstanceType": "m1.small",
            "InstanceRunningCount": 0,
            "InstanceRequestCount": 1
          },
          {
            "StartDateTime": null,
            "SpotPrice": null,
            "Name": "Task Instance Group",
            "InstanceRole": "TASK",
            "EndDateTime": null,
            "LastStateChangeReason": "",
            "CreationDateTime": DateTimeStamp,
            "LaunchGroup": null,
            "InstanceGroupId": "InstanceGroupID",
            "State": "PROVISIONING",
            "Market": "ON_DEMAND",
            "ReadyDateTime": null,
            "InstanceType": "m1.small",
            "InstanceRunningCount": 0,
            "InstanceRequestCount": 2
          },
          {
            "StartDateTime": null,
            "SpotPrice": null,
            "Name": "Core Instance Group",
            "InstanceRole": "CORE",
            "EndDateTime": null,
            "LastStateChangeReason": "",
            "CreationDateTime": DateTimeStamp,
            "LaunchGroup": null,
            "InstanceGroupId": "InstanceGroupID",
            "State": "PROVISIONING",
            "Market": "ON_DEMAND",
            "ReadyDateTime": null,
            "InstanceType": "m1.small",
            "InstanceRunningCount": 0,
            "InstanceRequestCount": 2
          }
        ],
        "MasterInstanceType": "m1.small"
      },
      "BootstrapActions": [],
      "JobFlowId": "JobFlowID"
    }
  ]
}

Example using the API

The DescribeJobFlows operation in the Amazon EMR API returns details about specified job flows. You specify a job flow by the job flow ID, creation date, or state. Amazon EMR returns descriptions of job flows that are up to two months old. Specifying an older date returns an error. If you do not specify a CreatedAfter value, Amazon EMR uses the default of two months.

To return information about a job flow identified by its job flow ID

Issue a request similar to the following, replacing the <JobFlowID>, <AccessKeyID>, and Signature with the values required for your job flow.
```
https://elasticmapreduce.amazonaws.com?
JobFlowIds.member.1=JobFlowID&
Operation=DescribeJobFlows&
AWSAccessKeyId=AccessKeyID&
SignatureVersion=2&
SignatureMethod=HmacSHA256&
Timestamp=2009-01-28T21%3A49%3A59.000Z&
Signature=CalculatedValue
```
For more information about the input parameters unique to DescribeJobFlows, go to DescribeJobFlows.

To return information about a job flows in a specific state

Issue a request similar to the following, replacing the <AccessKeyID> and Signature with the values required for your job flows.

https://elasticmapreduce.amazonaws.com?
JobFlowStates=COMPLETED&
Operation=DescribeJobFlows&
AWSAccessKeyId=AccessKeyID&
SignatureVersion=2&
SignatureMethod=HmacSHA256&
Timestamp=2009-01-28T21%3A49%3A59.000Z&
Signature=CalculatedValue

For more information about the input parameters unique to DescribeJobFlows, go to DescribeJobFlows.

Document Conventions
Terms of Use

Did this page help you? Yes No Tell us about it...