Introduction to Amazon Mechanical Turk

Amazon Mechanical Turk works some ways similar to a job board. Requesters advertise jobs they are willing to pay people to do. Workers look at the jobs available from all of the Requesters and choose to work on the jobs that interest them and the ones they qualify for. Requesters review submitted work either manually or programmatically and agree to pay for the work or not.

What makes Amazon Mechanical Turk special is that the job advertising and job completion happens over the Internet so the workforce is international and numbers in the hundreds of thousands. The workforce scales with the Requester's needs from none to hundreds, as specified by the Requester.

Following are the major advantages of Amazon Mechanical Turk.

A Requester is a person (or company or organization) who asks questions to Amazon Mechanical Turk. As a Requester, you use a software application to interact with the Amazon Mechanical Turk Service to submit questions, retrieve answers, and perform other automated tasks. You can use the Requester Console (http://requester.mturk.amazon.com/) to check the status of your questions, and manage your account.

To Workers, you are known as the creator of your HITs, and as the creator and maintainer of your Qualification types. Workers see your name, as specified with your Amazon.com account, on the Amazon Mechanical Turk web site.

You perform actions with the Amazon Mechanical Turk Service by using an AWS Access Key ID and AWS Secret Key to cryptographically sign each request. To obtain an AWS Access Key ID and AWS Secret Key, go to http://aws.amazon.com/mturk and sign in with your Amazon.com account e-mail address and password.

A Worker is a person who answers questions for Amazon Mechanical Turk. A Worker uses the Amazon Mechanical Turk web site (http://mturk.amazon.com/) to find questions, submit answers, and manage his or her account.

To Requesters, a Worker is known as the submitter of a HIT assignment, and as a user requesting a Qualification. You see the Worker's account ID (an alphanumeric string assigned by the system) included with assignment data and Qualification requests.

Qualifications represent the Worker's reputation and abilities. A Worker's Qualifications are matched against a HIT's Qualification requirements to allow or disallow the Worker to accept the HIT. A Worker's Qualifications cannot be accessed directly by other users.

Each question your application asks is a Human Intelligence Task, or HIT. A HIT contains all of the information a Worker needs to answer the question, including information about how the question is shown to the Worker and what kinds of answers would be considered valid.

Each HIT has a reward, an amount of money you pay to the Worker that successfully completes the HIT.

You can request that more than one Worker ought to complete a HIT by specifying a MaxAssignments property for the HIT. For more information, see Creating and Managing Assignments.

When a Worker finds a HIT to complete, the Worker accepts the HIT. Amazon Mechanical Turk creates an assignment to track the completion of the task and store the answer the Worker submits.

Amazon Mechanical Turk reserves the assignment while the Worker is actively working on it, so no other Worker can accept it or submit results. If the Worker fails to complete the assignment before the deadline you specified (the Worker abandons the HIT), or if the Worker chooses not to complete it after accepting it (the Worker returns the HIT), the assignment is once again made available for other Workers to accept.

A HIT can have multiple assignments. This is useful for gathering multiple answers to a single question for comparison, or for collecting multiple opinions. A Worker can only accept a HIT once, so a HIT with multiple assignments is guaranteed to be performed by multiple Workers.

You can specify the maximum number of assignments that any Worker can accept for your HITs. You can set two types of limits:

For more information, see Creating and Managing Assignments.

Once a HIT has all of the answers that were requested, or an expiration date you specified has passed, your application retrieves the assignments with the answer data. If an assignment's answer satisfies the question, you approve the assignment. You may reject the assignment if the HIT was not completed successfully.

Amazon Mechanical Turk automatically processes payment of the reward to the Worker once the assignment is approved. The reward is transferred from your Amazon.com account to the Worker's Amazon.com account. You can deposit or withdraw funds from your Amazon Mechanical Turk account at any time using the Requester web site (http://requester.mturk.amazon.com/).

You can manage which Workers can accept a particular HIT using Qualifications. A Qualification is an attribute assigned by you to a Worker. It includes a name and a number value. A HIT can include Qualification requirements that a Worker must meet before they are allowed to accept the HIT. Each QualificationRequirement describes an expression that a score or metric about the Worker must match for the Worker to be considered "qualified" to complete the HIT. For more information, see Creating and Managing Qualifications.

You create a Qualification type to represent a Worker's skill or ability. A Worker discovers your Qualification type either by browsing HITs that require it, or by browsing Qualification types directly. The Worker requests a Qualification of the type, and you grant the request with a value.

A Qualification type may include a Qualification test. A Qualification test is a set of questions, similar to a HIT, that the Worker must answer to request the Qualification. You can grant the request manually by evaluating the Worker's test answers, or you can include an answer key for the test when they create the Qualification type. For Qualification types with a test and an answer key, Amazon Mechanical Turk processes Qualification requests automatically, and sets Qualification values as specified by the answer key.

Amazon Mechanical Turk provides several system Qualifications that represent a Worker's account history. The values are updated continuously as the Worker uses the system. A HIT may include Qualification requirements based on these system Qualifications.

For more information, see Creating and Managing Qualifications.

The Question field of a HIT describes what is being asked of the Worker. It includes any information required to answer the question, such as text or images, as well as a description of the range possible answers.

Tip

The Amazon Mechanical Turk Service passes questions and answers between your Requester application and Workers. Workers read questions and provide answers using the Amazon Mechanical Turk web site. The format of this data is device-independent, so future Worker interfaces to Amazon Mechanical Turk can be built on platforms with varying capabilities. Be aware that the Worker interface is not guaranteed to display your questions in a particular way, nor is it guaranteed to return answers within the ranges you specify in the question form. Amazon Mechanical Turk only ensures that the question and answer data conform to the appropriate schemas.

You can include several different kinds of data in a HIT question:

The question form specification may include multiple fields, or "questions." A question can have the Worker select zero, one or more options from a list (true/false, multiple choice), or it can have the Worker type in text or a number. A field can also request that the Worker upload a file.

The question form specification may suggest the style of a field, guiding how a question may appear to the Worker. Multiple choice questions may appear as radio buttons, checkboxes, or a dropdown list, among others. The suggested style is not guaranteed, since Amazon Mechanical Turk may adjust the appearance to fit the device the Worker is using to see the question.

The specification may also suggest ranges of possible answers for the question. It is up to the device presenting the question to the Worker to validate the Worker's answers, so the results in the assignment are not guaranteed to meet these criteria. Your application should always validate the answers it receives.

	Tip
	For more information about the question and answer specification format, see QuestionForm Data Structure.