Stroop Task (Web adaptation)

Note: This tutorial has been adapted from our paper.

In this tutorial we describe how to build a Stroop (MacLeod, 1991; Stroop, 1935) task using the QRTEngine. The task will consist of one type of trial containing three screens: a fixation cross displayed for 1000 ms; a blank screen displayed for 500 ms, and a screen showing a color word with congruent and incongruent ink colors that is displayed until a response is made. Participants are instructed to respond to the ink color of the word by typing the first letter of the corresponding color (e.g. “r” if the ink color is red). In order to build this task, one only needs a computer with Internet connection, and a Qualtrics account.

Step 1: Setting up the survey

The Stroop RT task will be part of a survey, so the first step is to create a new survey. After creating a new survey, the QRTEngine needs to be included. To do this, go to Look & Feel > Advanced and paste the QRTEngine JavaScript code (that is freely available on in the ‘Header’ field (please do NOT click edit, just paste the code directly in the header field). Click Save. The QRTEngine is now available for use in the survey.

Before the QRTEngine is able to work, one needs to set a number of embedded data fields. These are essentially global variables that the QRTEngine needs to be able to use when the survey is running. Click Survey Flow > Add a New Element Here > Embedded Data and add the fields as displayed in Figure 1A, then click Save.

A final step in setting up the survey is selecting the right layout. Although aspects of the layout can be manually manipulated using CSS (An Internet standard that allows for extensive visual formatting of webpages), one can typically use the standard Qualtrics ‘Minimal’ layout. This layout uses a plain white background with black text and can be found under the Look & Feel tab. After setting the layout, the QRTEngine is included successfully in the survey.

Figure 1. A) Screenshot of what the embedded data overview should look like for the Stroop RT task. B) A screenshot of the Loop & Merge list. In the Stroop RT task 96 trials will be displayed, 4 columns are needed to define the variable content for each trial. C) A screenshot of the question block along with the JavaScript for each question. Each question represents a screen in the task. D) A schematic overview of a trial in the Stroop RT task.

Step 2: Setting up the list of trials

On the ‘Edit Survey’ page, an empty question block is presented. This question block will be used to present trials from the Stroop task. We start by defining the list of trials to be displayed using the Qualtrics Loop & Merge (L&M) functionality. L&M essentially loops a question block, providing different properties (in our case, colors, color words and correct responses) for each trial. To turn on the L&M functionality for a question block: click on Block options > Loop & Merge > Turn On Loop & Merge. Here, the list of trials can be created by adding as many columns and rows as necessary (it is often helpful to paste a list from a spreadsheet editor like Excel). An example of a list for the Stroop task is shown in Figure 1B. The first column defines the word that will be displayed. The second column defines the ink color in which the word will be displayed. The third column defines the correct response key. The fourth column indicates whether the ink color is congruent with the word that is shown. This allows us to store this variable for later data analysis using JavaScript snippets (see below). Finally, we check the option Randomize Loop Order to ensure that the trials are presented in random order.

Note: In our Task folder on Google Docs we have a Stroop Task available, this was the Stroop Task we ran for our paper. One has to take care when copy-pasting that trialdata into the Loop & Merge spreadsheet, because column 3 (Congruency) and column 4 (Correct Response) are switched around when compared to this Stroop Task’s L&M data.

Step 3: Setting up the trial

Next, we add the screens showing the content of a trial. In the question block, click on Create a New Item, a new question of the type ‘Multiple Choice’ is inserted. The new question contains three bullets, these will not be displayed or used, and can be removed (This can be done by changes the Question Type to ‘Descriptive Text’). Add more questions until there are four questions. Select the last question and change the Item Type to ‘Text Entry’, the Text Type to ‘Form’ and set the number of Form Fields to two. The QRTEngine will use these fields for data storage. The form fields will not be visible when running the experiment. For the first and the last questions, add a ‘+’ sign that will function as a fixation cross. For the second question, add a space ‘ ‘, since a fully empty text field defaults back to the Qualtrics text ‘Click to write the text’, the space will keep it from reverting, while still showing a blank screen.

To present the stimulus, use the HTML View mode to insert the HTML code displayed in Figure 1C in the right column of question 3. Note that the values from the first two columns of the L&M list are used for the stimulus (in Q3) using the “Piping text” feature. Accordingly, the code ${lm://Field/2}  will insert the color from the second column, whereas the code ${lm://Field/1}  will insert the word from the first. The code below can be copy-pasted to the HTML view of the stimulus:

Note: The line numbers shouldn’t be copied, so make sure to toggle plain text (one of the buttons in the top menu of the code box) whenever code copy is expected.

Finally, change the question tag of the last question to ‘ExitQ’. As long as the survey question tags (in the figure: Q1; Q2: Q3; and ExitQ) are unique in the survey the exact naming is arbitrary. However, it is important to correctly refer (in the JS snippet) to the tag that belongs to the final question as will be explained later.

The question block now should look like the one depicted in Figure 1C. Note that each trial consists of a fixation cross, a blank screen, and a screen showing the color word stimulus. However, while we want to display only three screens, we have added four questions. This is to allow for accurate timing across trials: the engine assumes that the stimulus shown during the inter-trial interval is available both at the first and the last position of the question block.

Step 4: Adding the timing

As a final step, some JavaScript code is added to each question to set the timing features of the trial. Select the question, then click the gear-icon on the left of the question and click ‘Add JavaScript’. (Note: if you do not see this option, you likely do not have a full Qualtrics account; Javascript is no longer supported for trial accounts)

For each question, replace the standard JavaScript Qualtrics has added with the code that is displayed in Figure 1C. In the code, the lines containing setConfig set the configurations for the screens, like the duration of the ‘BlankInterval’ screen and the allowable responses of the ‘StroopItem’ screen. Because the content of the L&M list is not stored automatically, the lines containing setTrialData determine which values from the L&M list will be saved for later data analysis.

Note that the JavaScript snippet for the first question also contains the following line:

The value mentioned here should correspond to the name of the question tag of the final question of the current question block (“ExitQ” in our example). Please find copyable versions of the in the blocks below:

Note: Please make sure to overwrite every line of code already present in the JavaScript box (Qualtrics adds a few lines by default, overwrite them!!)





Step 5: Run the survey

If all the elements and code-snippets have been added the Stroop RT task should work properly. To test the survey, press the ‘Preview Survey’ button and run the task. To start distributing the survey (for instance, via AMT), one can simply click the Launch Survey button or the Distribute Survey tab.

Importantly, this RT task now does not include instruction screens and other tasks. Those can be added easily using new question blocks. Note that when the engine is included in a survey, all content including the ‘next’ button is hidden automatically for all questions. To unhide this content, add the following JavaScript

 Step 6: Exporting and analyzing participant data

The data that has been collected using the survey can be found under the ‘View Results’ tab. Because Qualtrics uses a standard wide format (each participant’s data is stored in a single row), all trial information is stored using JSON representation. A data parser has been published (freely available on that allows to convert and export a Qualtrics QRTEngine CSV file to a long format (each row presents data of a single trial). The QRTEParser is a Java program that can be used on different platforms (e.g., Windows, OSX, and Linux). Standard packages such as SPSS and Excel can then be used to read the resulting CSV data file and perform data aggregation and statistics. For a guide on how to use the parser, please look here – QRTEngine Parser Tutorial


2 thoughts on “Stroop Task (Web adaptation)

  1. I can get it to run but not collect any data other than which items were displayed.
    How do I get it to collect the response given for each stimuli and the reaction of that response?
    Any help gratefully received…

    1. Hi Andrew,

      Have you added all the Embedded Data Fields to the Survey Flow (The fields shown in Figure 1A?). If so, it makes debugging slightly harder. Please contact us at and send the faulty .qsf, so we can try and find out what’s wrong and update the tutorial accordingly.



Comments are closed.