Amazon Mechanical Turk
Developer Guide (API Version 2012-03-25)
Print this pageEmail this pageGo to the ForumsView the PDFShare this page on TwitterShare this page on FacebookBookmark this page on DeliciousSubmit this page to RedditSubmit this page to DiggDid this page help you?  Yes  No   Tell us about it...

Working With HITs

Creating HITs

A Human Intelligence Task, or HIT, is a question your application asks, and a Worker answers. Your application submits a HIT using the CreateHITOperation operation.

A HIT includes:

  • A title

  • A description

  • Keywords, used to help Workers find the HITs with a search

  • The amount of the reward

  • An amount of time in which the Worker must complete the HIT

  • An amount of time after which the HIT will no longer be available to Workers

  • The number of Workers needed to submit results for the HIT before the HIT is considered complete

  • Qualification requirements

  • All of the information required to answer the question

Once created, the HIT becomes browsable and searchable on the Amazon Mechanical Turk web site, and can be accepted and completed by a Worker whose Qualifications match the HIT's Qualification requirements.

[Note]Note

HITs, Hit types, and Qualifications types are available until they are explicitly disposed (using either the DisposeHIT or DisposeQualificationType operations), or until they have been inactive for 120 days. HITs and their related assignment data are disposed 120 days after you approve or reject the assignments. HIT types and Qualification types are disposed 120 days after their last use. If you require access to HIT data for longer than 120 days, we recommend you keep a local copy.

The Title, Description, and Keywords

A HIT has a Title, a Description, and Keywords that generally describe the HIT. These values show at a glance what kind of work is involved in the HIT.

The Amazon Mechanical Turk website includes a search engine for performing keyword searches for HITs. Search terms can match against a HIT's title, description or keywords.

Using International Characters

If a HIT uses international characters in the HIT title, description, or if the results contain international characters, you have to configure Excel to display the international characters.

The Reward

A HIT can have a Reward, an amount of money paid to the Worker once you approve the results the Worker submitted. This amount is transferred from your account to the Worker's account automatically once you approve the results.

The question is how much should you offer as an award? If you offer to high a reward, you will find that your results will be costly. If you set the reward too low, you run the danger of no one working on your HITs. To set your reward, look for HITs that are similar in terms of the amount and kind of work required of the Worker.

[Tip]Tip

A minimum HIT listing fee applies to each HIT, even if the reward is zero.

Deadlines and Expirations

Once a Worker has chosen a HIT to work on, Amazon Mechanical Turk starts a timer to keep track of how long the Worker has been holding on to the HIT. If the amount of time exceeds the AssignmentDurationInSeconds of the HIT, Amazon Mechanical Turk declares that the Worker has "abandoned" the HIT, cancels the Worker's assignment, and makes the HIT available for other Workers to accept. When a new Worker chooses the HIT, the timer starts over.

Amazon Mechanical Turk also keeps track of how long the HIT has been in the system, from the moment the HIT is created. If the HIT's lifetime exceeds the LifetimeInSeconds, the HIT is declared completed, whether or not all of the requested answers have been submitted by Workers. The HIT is removed from the Amazon Mechanical Turk website, and is no longer available for Workers to find and complete. For more information about the life cycle of a HIT, see Creating and Managing Assignments.

Asking Workers to Upload Files

One type of HIT question can prompt the Worker to upload a file. You specify minimum and maximum sizes for the file, and Amazon Mechanical Turk will ensure that the Worker uploads a file within the specified size range.

The results for the HIT will include the actual size of the file the Worker uploaded. When your application is ready to retrieve the file, it calls GetFileUploadURL with the assignment ID and the question identifier. The operation returns a temporary URL that your application can use to download the file. The URL will only work for 60 seconds after GetFileUploadURL is called.

[Tip]Tip

The 60-second expiration of the temporary URL returned by the GetFileUploadURL operation ensures that only your application can access the data, while allowing your application to retrieve the file using a direct HTTP connection to the URL. The time limit only applies to initiating the download; the download itself will take as long as is necessary to retrieve the complete file. If you need to initiate the download after the URL has expired, you can call GetFileUploadURL at any time to get a new temporary URL.

Using Your Website to Host Questions

If you need more control over the display or logic of how HITs are presented to users than is provided by the Amazon Mechanical Turk content types, you can create a HIT whose question is hosted on your own website.

An "external question" HIT appears to the Worker as a HIT whose question form is a web browser frame. The Worker's browser loads the contents of the frame directly from a URL you provide when you create the HIT. This gives you complete control over what appears in the frame, and how the Worker interacts with the question.

The Worker submits results for your HIT using the form on your website. Your form then submits this data back to Amazon Amazon Mechanical Turk, where it is stored with the HIT results. Amazon Mechanical Turk then advances the user's browser to another HIT. The result is similar to submitting any other kind of HIT.

[Note]Note

Setting up a HIT with an external question requires a web server capable of functioning under very high load. Similar to images, Java applets and Flash applications, failure to serve files may prevent the Worker from seeing the data required to complete the task. However, unlike images and applets, if the web server fails to function for an external HIT, the Worker may not be able to submit results to Amazon Amazon Mechanical Turk at all.

An external HIT can be as simple as a web form in an HTML file. As such, the web server hosting the external HIT content does not need sophisticated server functionality. For example, you can create a sophisticated web form for an external HIT using HTML and JavaScript, then host the HTML file at Amazon S3.

For information about created HITs with external questions, see ExternalQuestion.

The Requester Annotation

Your application can include a RequesterAnnotation for each HIT, a value visible only to you. You can use this value to associate the HIT data with an identifier internal to your application. The Requester annotation is returned with other HIT data, such as from a call to GetHIT.