In 1769, Hungarian nobleman Wolfgang von Kempelen astonished Europe by building a mechanical chess-playing automaton that defeated nearly every opponent it faced. A life-sized wooden mannequin, adorned with a fur-trimmed robe and a turban, Kempelen’s “Turk” was seated behind a cabinet and toured Europe confounding such brilliant challengers as Benjamin Franklin and Napoleon Bonaparte. To persuade skeptical audiences, Kempelen would slide open the cabinet’s doors to reveal the intricate set of gears, cogs and springs that powered his invention. He convinced them that he had built a machine that made decisions using artificial intelligence. What they did not know was the secret behind the Mechanical Turk: a chess master cleverly concealed inside.
Today, we build complex software applications based on the things computers do well, such as storing and retrieving large amounts of information or rapidly performing calculations. However, humans still significantly outperform the most powerful computers at completing such simple tasks as identifying objects in photographs – something children can do even before they learn to speak.
When we think of interfaces between human beings and computers, we usually assume that the human being is the one requesting that a task be completed, and the computer is completing the task and providing the results. What if this process were reversed and a computer program could ask a human being to perform a task and return the results? What if it could coordinate many human beings to perform a task?
Amazon Mechanical Turk provides a web services API for computers to integrate “artificial, artificial intelligence” directly into their processing by making requests of humans. Developers use the Amazon Mechanical Turk web services API to submit tasks to the Amazon Mechanical Turk web site, approve completed tasks, and incorporate the answers into their software applications. To the application, the transaction looks very much like any remote procedure call – the application sends the request, and the service returns the results. In reality, a network of humans fuels this artificial, artificial intelligence by coming to the web site, searching for and completing tasks, and receiving payment for their work.
Your application interacts with the Amazon Mechanical Turk web service to ask questions to be answered by human beings. In Mechanical Turk terms, you are the Requester of information, and your application is the Requester application. The Workers, the human beings providing the answers, find and answer your questions using the Mechanical Turk web site (http://mturk.amazon.com/).
Each question your application asks is a Human Intelligence Task, or HIT. A HIT contains all of the information a Worker needs to answer the question, including information about how the question is shown to the Worker and what kinds of answers would be considered valid.
Each HIT has a reward, an amount of money you pay to the Worker that successfully completes the HIT.
When a Worker finds a HIT to complete, the Worker accepts the HIT. Mechanical Turk creates an assignment to track the completion of the task and store the answer the Worker submits.
Mechanical Turk reserves the HIT while the Worker is actively working on it, so no other Worker can accept it or submit results. If the Worker fails to complete the HIT before the deadline you specified (the Worker abandons the HIT), or if the Worker chooses not to complete it after accepting it (the Worker returns the HIT), the HIT is once again made available for other Workers to accept.
Once a HIT has all of the answers that were requested, or an expiration date you specified has passed, your application retrieves the assignments with the answer data. If an assignment's answer satisfies the question, you approve the assignment. You may reject the assignment if the HIT was not completed successfully.
Mechanical Turk automatically processes payment of the reward to the Worker once the assignment is approved. The reward is transferred from your Amazon.com account to the Worker's Amazon.com account. You can deposit or withdraw funds from your Mechanical Turk account at any time using the Requester web site (http://requester.mturk.amazon.com/).
You can manage which Workers can accept a particular HIT using Qualifications. A Qualification is an attribute assigned by you to a Worker. It includes a name and a number value. A HIT can include Qualification requirements that a Worker must meet before they are allowed to accept the HIT.
You create a Qualifiation type to represent a Worker's skill or ability. A Worker discovers your Qualification type either by browsing HITs that require it, or by browsing Qualifications directly. The Worker requests a Qualification of the type, and you grant the request with a value.
A Qualification type may include a Qualification test. A Qualification test is a set of questions, similar to a HIT, that the Worker must answer to request the Qualification. You can grant the request manually by evaluating the Worker's test answers, or you can include an answer key for the test when they create the Qualification type. For Qualification types with a test and an answer key, Mechanical Turk processes Qualification requests automatically, and sets Qualification values as specified by the answer key.
Mechanical Turk provides several system Qualifications that represent a Worker's account history. The values are updated continuously as the Worker uses the system. A HIT may include Qualification requirements based on these system Qualifications.