A Qualification is a property of a Worker that represents a Worker's skill, ability or reputation. A HIT can have Qualification requirements that a Worker's Qualifications must meet before the Worker is allowed to accept the HIT.
A Qualification is assigned to a Worker either by a Requester or by the Mechanical Turk system. The creator and maintainer of a Qualification type is known as the "author" of the type.
A Qualification is assigned to a Worker with a Qualification type and a number value. A Qualification requirement specifies the type, and a rule that determines if the value meets the requirement. A Worker can only accept a HIT if all of the HIT's Qualification requirements are met by the Worker's Qualifications.
You can create a new Qualification type for use with your HITs by calling the CreateQualificationType operation. Once a Qualification type has been created, it is available for use in a Qualiciation requirement, and can be searched or browsed at the Mechanical Turk web site.
Tip:
Any Requester can use any Qualification type for a Qualification requirement, regardless of who created the type. However, only the author the Qualification type can grant requests for the type and assign values.A Worker discovers a Qualification type by browsing HITs that require a Qualification of that type, or by searching or browsing the Qualification types directly. The Worker can view a description of the type, and request a Qualfication of the type from the type's author.
By default, a Worker can only request a Qualification for a given type once. The author of the type can allow a Worker to re-request a Qualification by specifying a value for the RetryDelayInSeconds parameter when creating the type. If set, the Worker must wait until the delay has elapsed after the first request before requesting the Qualification again.
Your can retrieve pending requests for your types using the GetQualificationRequests operation. You grant the Qualification, with a number value, using the GrantQualification operation.
Note:
There is no way to "reject" a Qualification request. To prevent a Worker who has requested a Qualification from completing a HIT that requires that Qualification, grant the request with a value that does not meet the HIT's Qualification requirement.
Note:
A Qualification can only be given to a Worker by granting a Qualification request. There is no way to assign a Qualification to a Worker without the Worker first requesting the Qualification.
A Qualification type may include a Qualification test, a set of questions similar to a HIT that a Worker must answer when requesting the Qualification. The test answers are returned with the request from a call to GetQualificationRequests. The author of the type can use the answers to determine the value to assign when granting the request.
The Qualification type may also include an answer key for the test, with answers and score values for each question in the test. In addition to score values for each matching answer, the answer key can specify how the Qualification value is calculated from the sum of the scores. For example, the Qualification value may be a percentage of correct answers.
Note:
If a Qualification test has an answer key, the test may only contain multiple choice questions. An answer key cannot grade questions with free-text answers.Tip:
For more information on answer keys, see the AnswerKey data structure.After you have granted a Qualification to a Worker and the Worker has completed a HIT for you, you can update the Worker's Qualification score with the UpdateQualificationScore operation. The operation requires the Worker's user ID, which is included in the assignment data the Worker submits for your HITs. You can use this operation to revoke a qualifying score if the Worker is not performing to expectations, or promote a score if the Worker has earned it with good results.
You can query a user's Qualification score for a type you created using the GetQualificationScore operation.
Note:
If your Qualification type has a Qualification test, an answer key, and allows test retakes, a Worker who has been granted the Qualification can take the test again to get a new score, even if you have updated the score since it was first granted. To query a user's current score, use the GetQualificationScore operation. You can configure the ability to retake a Qualification test when you create the Qualification type.
Once you create a Qualification type, you are responsible for granting Qualification requests for the type (or having requests granted automatically using a Qualification test and answer key). You can temporarily deactivate a Qualification type using the UpdateQualificationType operation. The type can be activated again with the same operation.
An inactive Qualification type cannot be requested by a Worker. It does not appear on the Mechanical Turk web site, and does not appear in search results. While the type is inactive, a HIT that requires a Qualification of the type cannot be created. Workers with Qualifications of the type will continue to have those Qualifications, and will continue to qualify for existing HITs that require the Qualification.
Mechanical Turk maintains a special set of Qualifications that represent a Worker's history and reputation. The system assigns these Qualifications to every user, and continuously updates their values as Workers use the system. HITs may have Qualification requirements based on system Qualifications.
Tip:
For a list of system Qualifications, see the QualificationRequirement data structure.The SearchQualificationTypes operation returns Qualification types whose names or descriptions match a search query. The results are similar to what is returned when performing such a search on the Mechanical Turk web site or Requester Console, except that system Qualifications may also be included in results.