Link Search Menu Expand Document
Running web-based experiments

Setup

Table of contents

  1. Python
  2. Git and GitHub
  3. Platform-specific tools
    1. Mechanical Turk and Submiterator
    2. Prolific and Proliferate
  4. RStudio
  5. Text editor
  6. OSF
  7. Checklist

Python

We will not directly program in Python, but some of the tools we use depend on it, so make sure you have installed Python 3. We recommend using the Anaconda Python distribution.

To test whether you have Python 3 installed, open the Termninal (on MacOS or on Linux) or the Anaconda Prompt (Windows) and type

python -V

and run the command with the return key.

This should print the python version.

Git and GitHub

We will use GitHub to manage experiment files and host the experiment. To interact with GitHub, you need to make sure that you have git installed.

To test whether you have git installed, open the Termninal (on MacOS or on Linux) or the Anaconda Prompt (Windows) and run

git --version

If you get an error message, make sure to install Git:

  1. Install git. You can read the rest of the linked tutorial to learn more about git.
  2. First time Git setup: The main thing is to configure user.name and user.email

If you don’t already have a GitHub account, sign up for one.

Platform-specific tools

Depending on whether you want to set up an experiment on Prolific Academic or Amazon’s Mechanical Turk, follow one or the other set of the following instructions.

Mechanical Turk and Submiterator

Amazon Mechanical Turk is a crowdsourcing platform well suited for running web-based experiments. We use Supersubmiterator to post experiments through Mechanical Turk and to download the data after participants complete the experiment.

  1. Follow these instructions to sign up for an Amazon Web Services (AWS) account and a requester account and to create an access key/secret pair on Amazon Mechanical Turk.
  2. Clone Supersubmiterator and follow the setup instructions to install the boto3 and xmltodict packages and your MTurk credentials (access key/secret pair).]

Prolific and Proliferate

We will test experiments on Prolific Academic, a platform for recruiting and paying participants. For collecting data, we will use Proliferate.

  1. Sign up for a Prolific account (if you don’t already have one).
  2. Sign up for a Proliferate account.
  3. Install the proliferate command line tool.

RStudio

For analyzing the data we will use R and the tidyverse library.

  1. Install R
  2. Install RStudio
  3. Install the tidyverse package: Lauch RStudio and run in the console: install.packages("tidyverse")
    (Note that quotes are needed when you install packages)
  4. You need to load the package before you can use it: library(tidyverse)
    (Note that there are NO quotes when you load packages)
    If you see a list of attached packages (including ggplot2, dplyr etc) then everything is good. (Don’t worry about the conflicts printed afterwards.)

Text editor

We will edit the code for some experiments and it is generally much easier to edit code with a good editor that supports syntax highlighting of your code. In case you don’t have an editor installed, Atom is a good free editor.

OSF

We will cover the theoretical background of pre-registration and pre-register the experiment that you’ll test on the Open Science Foundation (OSF) platform.

  1. Sign up for an OSF account (if you don’t already have one).

Checklist

Here is a recap of all the things that should be set up before the course:

  • Python 3 is installed.
  • Git is installed
  • You have a GitHub account
  • You have a Prolific account
  • You have a Proliferate account
  • The Proliferate command-line utility is installed
  • R is installed
  • RStudio is installed
  • A text editor is installed
  • You have an OSF account.