Comparing Amazon's Mechanical Turk Platform to Conventional Data Collection Methods in the Health and Medical Research Literature.

Abstract

The goal of this article is to conduct an assessment of the peer-reviewed primary literature with study objectives to analyze Amazon.com 's Mechanical Turk (MTurk) as a research tool in a health services research and medical context. Searches of Google Scholar and PubMed databases were conducted in February 2017. We screened article titles and abstracts to identify relevant articles that compare data from MTurk samples in a health and medical context to another sample, expert opinion, or other gold standard. Full-text manuscript reviews were conducted for the 35 articles that met the study criteria. The vast majority of the studies supported the use of MTurk for a variety of academic purposes. The literature overwhelmingly concludes that MTurk is an efficient, reliable, cost-effective tool for generating sample responses that are largely comparable to those collected via more conventional means. Caveats include survey responses may not be generalizable to the US population.