Creating and Managing Qualifications

A Qualification is a property of a Worker that represents a Worker's skill, ability or reputation. You can use Qualifications to control which Workers can perform your HITs. A HIT can have Qualification requirements that a Worker's Qualifications must meet before the Worker is allowed to accept the HIT. A requirement can also state that a Worker must meet the requirement to see the HIT's question data when previewing the HIT.

You can create and maintain your own Qualifications using the web service API. You create a Qualification type that describes what Qualifications of that type are for, and how Qualifications will be assigned.

A Qualification can have an optional value, such as a number. A HIT's Qualification requirement can specify a condition that the value must meet for the Qualification requirement to be met. A Worker can only accept a HIT if all of the HIT's Qualification requirements are met by the Worker's Qualifications.

You can use any Qualification type as the basis for your HIT's Qualification requirements, even if you did not create the type. For example, the Amazon Mechanical Turk system maintains a set of Qualification types that represent a Worker's account history, such as how many HITs the Worker has submitted in the lifetime of her account. You can create HIT requirements based on these statistics using the corresponding Qualification types.

You can create a new Qualification type by calling CreateQualificationType. Once a Qualification type has been created, it is available for use in a Qualification requirement, and can be searched or browsed at the Amazon Mechanical Turk website.

	Tip
	Any Requester can use any Qualification type for a Qualification requirement, regardless of who created the type. However, only the creator of the Qualification type can grant requests for the type and assign values.

A Worker discovers a Qualification type by browsing HITs that require a Qualification of that type, or by searching or browsing the Qualification types directly. The Worker can view a description of the type, and request a Qualification of the type from the type's creator.

By default, a Worker can only request a Qualification for a given type once. The creator of the type can allow a Worker to re-request a Qualification by specifying a value for the RetryDelayInSeconds parameter when creating the type. If set, the Worker must wait until the delay has elapsed after the first request before requesting the Qualification again.

A Qualification type may include an optional Qualification test, a form similar to a HIT that the Worker must complete to request the Qualification. The data entered into this form can be collected by your application, used to determine whether or not the Qualification request should be granted, or used to determine the Qualification's score.

Qualification requests can be granted in three different ways:

	Tip
	If you have a prior relationship with a Worker and know the Worker's ID, you can assign Qualifications directly to the Worker without the Worker first making a request. See the following discussion.

Your application can retrieve pending requests for your Qualification types using GetQualificationRequests. To grant the request and (optionally) assign the Qualification's value, call GrantQualification. To reject the request, call RejectQualificationRequest.

A Qualification type may include a Qualification test, a set of questions similar to a HIT that a Worker must answer when requesting the Qualification. The test answers are returned with the request from a call to GetQualificationRequests. You can use the answers to determine the value to assign when granting the request.

The Qualification type may also include an answer key for the test, with answers and score values for each question in the test. In addition to score values for each matching answer, the answer key can specify how the Qualification value is calculated from the sum of the scores. For example, the Qualification value may be a percentage of correct answers.

	Note
	If a Qualification test has an answer key, the test may only contain multiple choice questions. An answer key cannot grade questions with free-text answers.

	Tip
	For more information on answer keys, see AnswerKey Data Structure.

If you have a prior relationship with a Worker, such as if the Worker has submitted results for your HITs in the past, and you know the Worker's ID, you can assign a Qualification directly to the Worker without the Worker first making a request. AssignQualification gives a Worker a Qualification, and can send an email message to the Worker saying that the Qualification was assigned. You can also tell Amazon Mechanical Turk to not send an email message, and just assign the Qualification.

AssignQualification is useful for managing Qualifications for people that already do work for you on a regular basis. If you wish to establish new Qualifications for Workers that have never done work for you in the past, you can use HITs without Qualification requirements, or requestable Qualifications, to establish a work history.

After you have granted a Qualification to a Worker and the Worker has completed a HIT for you, you can update the Worker's Qualification score with UpdateQualificationScoreO. The operation requires the Worker's user ID, which is included in the assignment data the Worker submits for your HITs. You can use this operation to revoke a qualifying score if the Worker is not performing to expectations, or promote a score if the Worker has earned it with good results.

You can query a user's Qualification score for a type you created using GetQualificationScore.

Note

If your Qualification type has a Qualification test, an answer key, and allows test retakes, a Worker who has been granted the Qualification can take the test again to get a new score, even if you have updated the score since it was first granted. To query a user's current score, use the GetQualificationScore. You can configure the ability to retake a Qualification test when you create the Qualification type.

You can revoke the Qualification at any time by calling RevokeQualification. A revoked Qualification behaves as if the Qualification were never granted: The Worker cannot qualify for HITs that require the Qualification be present. Also, calling the GetQualificationScore operation on a revoked Qualification will return an error.

A Worker may be able to request that a revoked Qualification be granted again if allowed by the Qualification type's retry policy. If the type does not have a retry delay, the Worker will not be able to request the Qualification a second time.

You can query all of the Qualifications you have granted for a type that you created using GetQualificationsForQualificationType. The operation returns the Worker ID and current Qualification score for every Qualification of the type that you have granted. You can request either all granted (active) Qualifications of the given type, or all revoked Qualifications of the given type.

You can query all of your HITs that use a given Qualification type in a Qualification requirement using GetHITsForQualificationType.

	Note
	You can only query your own HITs with GetHITsForQualificationType. The operation will not return HITs created by other Requesters that use a Qualification type. You can query for HITs using any Qualification type, not only Qualification types that you created.

It is sometimes useful to make modifications to a Qualification type's test, to improve the accuracy of the resulting Qualification scores, or to replace old test questions with new ones. You can replace the test for a Qualification type using UpdateQualificationType.

UpdateQualificationType can also be used to replace an automatically graded test (with an answer key) with a manually graded one (without an answer key), and vice versa. It can also be used to change the test duration, the amount of time the Worker has to complete the test.

Once you create a Qualification type, you are responsible for granting Qualification requests for the type (or having requests granted automatically using a Qualification test and answer key). You can temporarily deactivate a Qualification type by using UpdateQualificationType to update the status of the type. The type can be activated again with the same operation.

An inactive Qualification type cannot be requested by a Worker. It does not appear on the Amazon Mechanical Turk website, and does not appear in search results. While the type is inactive, a HIT that requires a Qualification of the type cannot be created. Workers with Qualifications of the type will continue to have those Qualifications, and will continue to qualify for existing HITs that require the Qualification.

You can permanently remove a Qualification type by using DisposeQualificationType. Once a Qualification type has been disposed, you cannot use it to create HITs or HIT types. It does not appear on the website and does not appear in search results. You can reuse the old Qualification type name and parameters to create a new Qualification type.

HITs that require a disposed Qualification type cannot be extended. If there are active HITs related to a disposed Qualification type, the HITs remain active and Workers who have qualified for that type can continue to work on those HITs. However, HIT types that use a disposed Qualification type can no longer be used to create new HITs.

Workers who have the Qualification of the disposed type keep those Qualifications, and will continue to be qualified for existing HITs that require the Qualification. If a Worker has a request pending to receive a Qualification for a disposed type, Amazon Amazon Mechanical Turk rejects the request.

Amazon Mechanical Turk provides a special set of Worker Qualifications available to all Requesters. System-assigned qualifications include approval rate and location. The system assigns these Qualifications to every Worker and updates their values as Workers use the system. For a list of system Qualifications and more information about locale-based Qualification requirements, go to the QualificationRequirement Data Structure, in the Amazon Mechanical Turk API Reference.

Mechanical Turk Masters

Mechanical Turk also provides Requesters access to Mechanical Turk Masters. Masters have demonstrated accuracy in completing HITs in the Mechanical Turk marketplace. There are two types of Master Qualifications: Photo Moderation Masters and Categorization Masters.

SearchQualificationTypes returns Qualification types whose names or descriptions match a search query. The results are similar to what is returned when performing such a search on the Amazon Mechanical Turk website or Requester Console, except that system Qualifications may also be included in results.