Class that makes it easy to define Hadoop Streaming steps.

See also: Hadoop Streaming

CopyC#
AWSCredentials credentials = new BasicAWSCredentials(accessKey, secretKey);
AmazonElasticMapReduce emr = new AmazonElasticMapReduceClient(credentials);

HadoopJarStepConfig config = new StreamingStep()
    .WithInputs("s3://elasticmapreduce/samples/wordcount/input")
    .WithOutput("s3://my-bucket/output/")
    .WithMapper("s3://elasticmapreduce/samples/wordcount/wordSplitter.py")
    .WithReducer("aggregate")
    .ToHadoopJarStepConfig();

StepConfig wordCount = new StepConfig()
    .WithName("Word Count")
    .WithActionOnFailure("TERMINATE_JOB_FLOW")
    .WithHadoopJarStep(config);

RunJobFlowRequest request = new RunJobFlowRequest()
    .WithName("Word Count")
    .WithSteps(wordCount)
    .WithLogUri("s3://log-bucket/")
    .WithInstances(new JobFlowInstancesConfig()
        .WithEc2KeyName("keypairt")
        .WithHadoopVersion("0.20")
        .WithInstanceCount(5)
        .WithKeepJobFlowAliveWhenNoSteps(true)
        .WithMasterInstanceType("m1.small")
        .WithSlaveInstanceType("m1.small"));

RunJobFlowResult result = emr.RunJobFlow(request).RunJobFlowResult;

Namespace: Amazon.ElasticMapReduce.Model
Assembly: AWSSDK (in AWSSDK.dll) Version: 1.4.10.0 (1.4.10.0)

Syntax

         
 C#  Visual Basic  Visual C++ 
public class StreamingStep
Public Class StreamingStep
public ref class StreamingStep

Members

            
 All Members  Constructors   Properties   Methods  
 Public

 Protected
 Instance

 Static 
 Declared

 Inherited
 XNA Framework Only 

 .NET Compact Framework Only 

 MemberDescription
StreamingStep()()()()
Creates a new default StreamingStep.
Equals(Object)
Determines whether the specified Object is equal to the current Object.
(Inherited from Object.)
GetHashCode()()()()
Serves as a hash function for a particular type.
(Inherited from Object.)
GetType()()()()
Gets the Type of the current instance.
(Inherited from Object.)
HadoopConfig
Gets and sets the hadoopConfig.
Inputs
Gets and sets list of step input paths.
Mapper
Gets and sets the mapper.
Output
Gets and sets the output path.
Reducer
Gets and sets the reducer.
ToHadoopJarStepConfig()()()()
Creates the final HadoopJarStepConfig once you are done configuring the step. You can use this as you would any other HadoopJarStepConfig.
ToString()()()()
Returns a String that represents the current Object.
(Inherited from Object.)
WithHadoopConfig(String, String)
Add a Hadoop config override (-D value)
WithInputs(array<String>[]()[][])
Add more input paths to this step.
WithMapper(String)
Set the mapper.
WithOutput(String)
Set the output path for this step.
WithReducer(String)
Set the reducer.

Inheritance Hierarchy

System..::..Object
  Amazon.ElasticMapReduce.Model..::..StreamingStep

See Also