Making it real: Conducting an Online Experiment Replication

Abstract

Online experimentation is a crucial part of a new frontier of the social sciences. This class exercise and assignments give students hands-on experience with online experimentation. The exercise and assignment challenges students to replicate simple online experiments using Qualtrics and Amazon Mechanical Turk (mTurk). Rather than designing an experiment...

Download this resource to see full details. Download this resource to see full details.

Details

Subject Area(s):

Qualitative Methodology, Quantitative Methodology, Teaching and Learning in Sociology

Resource Type(s):

Assignment, Class Activity

Class Level(s):

College 300, College 400

Class Size(s):

Any

Usage Notes

I used this assignment with undergraduate students in a private four-year college, but instructors can also use it in other settings.

The technical load of this assignment is considerable. If your students are not familiar with Qualtrics and mTurk, learning to use the different functions of these web services can be challenging. If your...

Learning Goals and Assessments

Learning Goal(s):

1. Students will use Qualtrics to design a simple online survey experiment.
2. Students will outline and explain the different possible results of the survey experiment.
3. Students will design, program, and execute an online experiment replication.
4. Students will compare the experiment’s result to the original study.
5. Students evaluate their data and interpret the experimental results.
6. Students will assess the validity of AMT and Qualtrics as an interface for conducting experiments on human subjects.

Goal Assessment(s):

1. Students correctly set up and program the experiment (review downloaded Qualtrics programming).
2. Students successfully recruit and screen participants (review report).
3. Students successfully execute the experiment (review report).
4. Students critically interpret the results. The evaluation must consider both the experiment and the online interface of the replication (review report).