Amazon Elastic MapReduce
Developer Guide (API Version 2009-11-30)
Print this pageEmail this pageGo to the ForumsView the PDFShare this page on TwitterShare this page on FacebookBookmark this page on DeliciousSubmit this page to RedditSubmit this page to DiggDid this page help you?  Yes  No   Tell us about it...

HDFS Configuration (AMI 2.0)

The following table describes the default Hadoop Distributed File System (HDFS) parameters and their settings.

ParameterDefinitionDefault Value
dfs.block.sizeThe size of HDFS blocks. When operating on data stored in HDFS, the split size is generally the size of an HDFS block. Larger numbers provide less task granularity, but also put less strain on the cluster NameNode. 134217728 (128 MB)
dfs.replication This determines how many copies of each block to store for durability. For small clusters we set this to 2 because the cluster is small and easy to restart in case of data loss. You can change the setting to 1, 2, or 3 as your needs dictate.

1 for clusters < four nodes

2 for clusters < ten nodes

3 for all other clusters