Qualifications

A Qualification is a property of a Worker that represents a Worker's skill, ability or reputation. A HIT can have Qualification requirements that a Worker's Qualifications must meet before the Worker is allowed to accept the HIT. A requirement can also state that a Worker must meet the requirement to see the HIT's question data when previewing the HIT.

A Qualification is assigned to a Worker either by a Requester or by the Mechanical Turk system. The creator and maintainer of a Qualification type is known as the "author" of the type.

A Qualification is assigned to a Worker with a Qualification type and a number value. A Qualification requirement specifies the type, and a rule that determines if the value meets the requirement. A Worker can only accept a HIT if all of the HIT's Qualification requirements are met by the Worker's Qualifications.

You can create a new Qualification type for use with your HITs by calling the CreateQualificationType operation. Once a Qualification type has been created, it is available for use in a Qualiciation requirement, and can be searched or browsed at the Mechanical Turk web site.

A Worker discovers a Qualification type by browsing HITs that require a Qualification of that type, or by searching or browsing the Qualification types directly. The Worker can view a description of the type, and request a Qualfication of the type from the type's author.

By default, a Worker can only request a Qualification for a given type once. The author of the type can allow a Worker to re-request a Qualification by specifying a value for the RetryDelayInSeconds parameter when creating the type. If set, the Worker must wait until the delay has elapsed after the first request before requesting the Qualification again.

Your application can retrieve pending requests for your Qualification types using the GetQualificationRequests operation. To grant the request and (optionally) assign the Qualification's value, call the GrantQualification operation. To reject the request, call the RejectQualificationRequest operation.

A Qualification type may include a Qualification test, a set of questions similar to a HIT that a Worker must answer when requesting the Qualification. The test answers are returned with the request from a call to GetQualificationRequests. The author of the type can use the answers to determine the value to assign when granting the request.

The Qualification type may also include an answer key for the test, with answers and score values for each question in the test. In addition to score values for each matching answer, the answer key can specify how the Qualification value is calculated from the sum of the scores. For example, the Qualification value may be a percentage of correct answers.

After you have granted a Qualification to a Worker and the Worker has completed a HIT for you, you can update the Worker's Qualification score with the UpdateQualificationScore operation. The operation requires the Worker's user ID, which is included in the assignment data the Worker submits for your HITs. You can use this operation to revoke a qualifying score if the Worker is not performing to expectations, or promote a score if the Worker has earned it with good results.

You can query a user's Qualification score for a type you created using the GetQualificationScore operation.

You can revoke the Qualification at any time by calling the RevokeQualification operation. A revoked Qualification behaves as if the Qualification were never granted: The Worker cannot qualify for HITs that require the Qualification be present. Also, calling the GetQualificationScore operation on a revoked Qualification will return an error.

A Worker may be able to request that a revoked Qualification be granted again if allowed by the Qualification type's retry policy. If the type does not have a retry delay, the Worker will not be able to request the Qualification a second time.

You can query all of the Qualifications you have granted for a type that you created using the GetQualificationsForQualificationType operation. The operation returns the Worker ID and current Qualification score for every Qualification of the type that you have granted. You can request either all granted (active) Qualifications of the given type, or all revoked Qualifications of the given type.

You can query all of your HITs that use a given Qualification type in a Qualification requirement using the GetHITsForQualificationType operation.

It is sometimes useful to make modifications to a Qualification type's test, to improve the accuracy of the resulting Qualification scores, or to replace old test questions with new ones. You can replace the test for a Qualification type using the UpdateQualificationType operation.

The UpdateQualificationType operation can also be used to replace an automatically graded test (with an answer key) with a manually graded one (without an answer key), and vice versa. It can also be used to change the test duration, the amount of time the Worker has to complete the test.

Once you create a Qualification type, you are responsible for granting Qualification requests for the type (or having requests granted automatically using a Qualification test and answer key). You can temporarily deactivate a Qualification type by using the UpdateQualificationType operation to update the status of the type. The type can be activated again with the same operation.

An inactive Qualification type cannot be requested by a Worker. It does not appear on the Mechanical Turk web site, and does not appear in search results. While the type is inactive, a HIT that requires a Qualification of the type cannot be created. Workers with Qualifications of the type will continue to have those Qualifications, and will continue to qualify for existing HITs that require the Qualification.

Mechanical Turk maintains a special set of Qualifications that represent a Worker's history and reputation. The system assigns these Qualifications to every user, and continuously updates their values as Workers use the system. HITs may have Qualification requirements based on system Qualifications.

HITs may also have a Qualification requirement based on the Worker's location, according to the mailing address given to Mechanical Turk by the Worker. This allows HITs to be restricted to Workers that reside in specific countries, such as for tasks that can only be performed in certain locations, or for HITs that need to be performed by Workers that reside in the same country as the Requester for tax purposes.

The SearchQualificationTypes operation returns Qualification types whose names or descriptions match a search query. The results are similar to what is returned when performing such a search on the Mechanical Turk web site or Requester Console, except that system Qualifications may also be included in results.