Manifest File Parameters

Topics

The manifest file includes information related to processing your data and returning your storage device. This section explains the parameters in a manifest file.

Parameter Description Required

accessKeyId

Identifies the AWS account responsible for loading the objects. This value is the sender's Access Key ID. For more information about the Access Key ID, see Viewing Your AWS Identifiers.

The following is an example of accessKeyId.

accessKeyId: 02236Q3V0WHVSRW0EXG2

Type: String

Default: None

Yes

acl

Specifies the access control list (ACL) to apply to each object loaded to Amazon S3, for example, acl: public-read. The default value gives full permissions only the person who uploaded the object to the bucket.

Type: String

Default: private

Valid Values: private. public-read | public-read-write | authenticated-read

bucket

Specifies the Amazon S3 bucket where AWS Import/Export loads your data. You must create a bucket before we can load data into it; we do not create the bucket for you. The account doing the load must have write permissions on this bucket. The following is an example of a bucket name.

bucket: data-load-bucket

Type: String

Default: None

Constraints: Must be an existing Amazon S3 bucket name

Yes

contentDisposition

Specifies the HTTP Content-Disposition header to apply to all loaded files. For more details, go to RFC 2616 section 19. The following is an example of contentDisposition.

contentDisposition: attachment

	Note
	This example causes browsers to prompt to save the S3 object rather than displaying the object directly in the browser.

Type: String

Default: None

contentTypes

Contains a map of case-sensitive strings that are key/value pairs that set the Content-Type header. Each key/value pair contains a key that is the file extension (for example, jpg) and a corresponding value for the Content-Type (for example, image/jpeg). These values take precedence over the default file extension to Content-Type mapping. For more details on the default mappings, see File Extension to Mime Types.

The following is an example.

contentTypes:
 jpg: image/jpeg
 JPG: image/jpeg                   
 gif: image/gif
 GIF: image/gif
 mp3: audio/mpeg 
 txt: text/plain
 zip: application/zip

Type: Array of strings

Default: None

deviceId

This ID is your storage device's serial number or other unique identifier. If your device does not have a unique number, please place a sticker with a unique identifier onto the device and then enter that identifier as the deviceId. The following ID is an example.

deviceId: 49382

Type: String

Default: None

Yes

eraseDevice

After completing the data load all writable blocks on your storage device are overwritten with zeroes. You will have to repartition and format your drive to use it. The following is an example entry.

eraseDevice: No

Type: String

Default: None

Valid Values: Yes | No

Yes

ignore

Specifies directories, files, or file types that we should not load from your storage device. For more information about ignore, see Ignore.

Type: Array of strings

Default: We load all files except the SIGNATURE file

Constraints: The dash followed by a space, “- “, is YAML syntax and must precede each filter

logPrefix

Specifies a string to insert between the bucket name and log report name to prevent name collisions. For more information, see logPrefix.

manifestVersion

Identifies the version of the manifest specification used to write the data file. The following is an example.

manifestVersion: 1.1

Type: String

Default: None

Valid Values: 1.1

Yes

name

street1

city

stateOrProvince

postalCode

phoneNumber

country

All of these parameters are subparameters of returnAddress. Use these parameters to specify where we should return your storage device. For an example, see How to Create a Manifest File.

Type: String

Default: None

Constraints: Your return address must be a physical street address. UPS does not deliver to post boxes.

Yes

prefix

Specifies a string to insert between the bucket name and full path of each file located on your storage device. For more information, see Prefix.

Type: String

Default: None

returnAddress

The address we return your storage device to after the job completes. This parameter has a number of subparameters, including name, street1, street2, city, stateOrProvince, postalCode, phoneNumber, and country. For an example, see Creating a Manifest File.

Type: String

Default: None

Condition: Your return address must be a physical street address. UPS does not deliver to post boxes.

Yes

street2

A subparameter of returnAddress, which you use to specify where we should return your storage device. For an example, see returnAddress.

Type: String

Default: None

ignore

In the ignore parameter you specify directories, files, or file types on your storage device that you do not want us to load to AWS. To specify naming patterns, you use standard Java regular expressions. The following are example entries.

ignore:
 -  .*~$ 
 -  .*\.swp$

The first example ignores any filenames where the suffix ends with a tilda. The second example ignores any filenames where the suffix ends in .swp.

If you want to exclude all files in the root directory backup, you’d use the following ignore filter:

ignore: - ^backup/

When specifying a path that includes a file separator, for example, images/wookies/chewie.jpg, make sure to use a forward slash, “/”, and not a back slash.

ignore: 
  - ^images/wookies/

If you want to exclude all files with the .psd extension, you’d use the following ignore filter:

ignore:  
  - \.psd$ 
  - \.PSD$

The log report includes all ignored files, including the SIGNATURE file.

For more information about Java regular expressions, go to http://java.sun.com/j2se/1.4.2/docs/api/java/util/regex/Pattern.html.

logPrefix

logPrefix is a string AWS Import/Export inserts between the bucket name and log report name to prevent name collisions. The log file name always ends with the phrase import-log- followed by your JobId. For example, if the JobId is 53TX4, the log file name would end in import-log-53TX4.

If logPrefix is not set, a job loaded to mybucket with the jobId of 53TX4 saves the log file to http://mybucket .s3.amazonaws.com/load-log-53TX4. If you set logPrefix to logs/, the log file location would be http://s3.amazonaws.com/mybucket/logs/import-log-53TX4.

The following is an example of logPrefix.

logPrefix: my-log-reports/

	Note
	We do not include a forward slash (/) automatically. If you don't include the slash at the end of the value for `logPrefix`, the value is concatenated to the log file name. For example, if your `logPrefix` is `taxonomy` and your root directory contains the file `phylem.jpg`, your key would become `taxonomyphylum.jpg` instead of `taxonomy/phylum.jpg`.

logPrefix + import-log-JOBID cannot be longer than 1024 bytes. If it is, AWS Import/Export returns an InvalidManifestField error from the CREATE JOB command.

prefix

prefix is a string that AWS Import/Export inserts between the bucket name and full path of each file located on your storage device. The prefix is like a directory structure in which you put all of the data from one job into one directory. In this way, you can separate data by job in one bucket. For example, if your Amazon S3 bucket name is my-bucket, and you set prefix to my-prefix/, and the file on your storage device. is /jpgs/foo.jpg, then foo.jpg would be loaded to http://s3.amazonaws.com/my-bucket/my-prefix/jpgs/foo.jpg. If the prefix is not specified, foo.jpg would be loaded to http://s3.amazonaws.com/my-bucket/jpgs/foo.jpg.

The following is an example.

prefix: my-prefix/

Note that we do not include a forward slash (/) automatically. If you don't include the slash at the end of the value for prefix, the value is concatenated to the file name. For example if your prefix is images and your root directory contains the file wookie.jpg, your key would become imageswookie.jpg instead of images/wookie.jpg.

Type: String

Default: "" (empty string)

Constraint: A key is a sequence of unicode characters whose UTF-8 encoding is at most 1024 bytes long. If the logPrefix + import-log-JOBID is longer than 1024 bytes, AWS Import/Export returns an InvalidManifestField error from the CREATE JOB command.