Building an Experiment from Scratch

Although the Stroop Task shows you how to set-up a Survey from scratch, it is intermixed with how to set-up the Stroop Task itself and also fairly short on guidance. This tutorial is intended to show the basic steps necessary for getting the QRTEngine working in a Survey. Including the steps required to get the data-file to come out correctly.

Step 1. Create a new Survey

At the top of page the you land after you log-in, click the ‘Create Survey..’ button.createsurvey

Following that, click on ‘Quick Survey Builder’ (in the background), choose a name and click ‘Create Survey’.



Congratulations! You have now created your Survey.

Step 2. Adding the QRTEngine

First off, download the latest version of the QRTEngine. In your Qualtrics Survey Editor, click on Look & Feel in the top left of the screen:



In this screen we have to do two things. First – go to the ‘advanced’ tab, and add the contents of the QRTEngine.js file you just downloaded (hopefully) to the ‘header’ textbox. Like so:


Important: please do NOT click the edit link, just paste the code directly in the header field.

Step 3. Change the Theme

The second thing we have to do in the Look & Feel window is the change the look to plain jane. This is a standard Qualtrics theme, that doesn’t have any visible Qualtrics styling, thus is perfectly suited for behavioral experiments. In the image below, you can find it in the bottom row, second to the right.plainjane

Step 4. Add Embedded Data fields

Although your experiment should run fine without this step, the data itself won’t be stored correctly if we don’t define these Embedded Data (ED) fields. These ED fields are required for the parser, so that it knows which fields to handle in what way. If these ED fields aren’t defined in the Survey Flow, then Qualtrics doesn’t export them into the resulting data file, does making it impossible for the parser to complete the file conversion. First, click on ‘Survey Flow’ in the top menu:


Now, add a new element by clicking the ‘Add a new element here’ link, which will make the Survey Flow look like this:


Click on ‘Embedded Data’ and add the following fields:

  • QRTE_blockData
  • QRTE_idData
  • QRTE_exitQuestions
  • QRTE_columns

And then drag the ED block to the top of the Survey Flow, like so:



Make sure to leave the Values empty, the QRTEngine will fill these automatically as the trial is ran.

Step 5. Adding a QRTEngine-powered block

Next up, we need to add at least three questions to an empty block. We will simply use the block Qualtrics provides us upon the creation of a new survey. Add three questions to the block, by clicking the green ‘Create new Item’ button three times.


Set the question type of the first two questions (‘Q2′ and ‘Q3′ in this case) to ‘Descriptive Text’ (Click on a Question to give it ‘focus’ and then on the green ‘Change Type’ button to the right). descriptivetext

Set the last question to ‘Text Entry’ and change it to a ‘Form’ with 2 answers options.


The questions should now look like this:


Step 6. Add Correct JavaScript to Each Question

The next step is important, as adding the JavaScript will allow the QRTEngine to know that these questions are to be handled by it, rather than left alone. We have added three questions to the survey, so that we can show how to add the three question types: ‘Init’, ‘Stimulus’ and ‘Exit’. This is a strict ordering, Init cannot follow Exit or Stimulus, Exit can’t precede Stimulus or Init, nor can Stimulus questions be after Exit and before Init. We have changed the tags in the previous image to make referral easier.

First off, we’ll add JavaScript to the Init question. Click on the ‘Gear’ icon below ‘Init’ and then on ‘Add JavaScript’. (Note: if you do not see this option, you likely do not have a full Qualtrics account; Javascript is no longer supported for trial accounts)


Copy and paste this JavaScript into the box you just opened, make sure to replace everything, even the JavaScript that Qualtrics adds to the question by default. The marked lined need to be edited for the Survey to work correctly. First off, the exitItemTag needs to be set to the tag of the Exit Question (which conveniently is ‘Exit’ in this case, so fill that in!). Secondly, the block needs to be given a unique id (across the survey), so change line 15 so that it contains your own unique id.

Now add this JavaScript to the ‘Stimulus’ question, and any other stimulus questions you might add.

We have set the Stimulus id to ‘Stimulus’, however this should be unique across all stimuli added to a trial block.

Finally, add this to the Exit question:

Nothing else needs to be added. Some fairly complex processing will happen after this Question is called, but this is done behind the scenes, the process doesn’t require any input from the user.

Step 7. Change the stimuli

Now it’s time to change the stimuli. Clicking on the text of the questions, you can start editing it. Please look at this tutorial – Creating Stimuli – for tips on how to create your own stimuli. It is advisable that you keep the Exit and Init stimuli the same, the combined stimuli will serve as a ‘Intertrial Interval’ stimulus, and be displayed for the requested amount of time (which is set in the ‘Init’ question JavaScript (at the bottom).

Step 8. Add L&M data to dynamically build your trials

In order to build a looping structure into your experiment that will change the stimuli based on some data you provide, we will use the Loop & Merge feature native to Qualtrics. Click on Block Options in the top right corner of the block, and then on Loop & Merge:


Then click on ‘Turn on Loop & Merge’ at the top:


And then add your spreadsheet to the L&M spreadsheet that has just been created. The block will, by default, be repeated for as many times as there are rows in your spreadsheet, and the data from each column can be input into your stimuli by entering ${lm://Field/1} for column 1, ${lm://Field/2} for column 2, etc.

Step 8b. Make copies of your block (optional)

In case you want to present the same block multiple times (e.g. separated by ‘break’ screens) you can copy the block after you are sure it is perfect and complete (click on Block Options > Copy Block …).


Give the new block a logical name (e.g. ‘TestBlock’) and press the ‘Copy’ button.

Now go the newly created block and make the following information unique. The parser requires you to do this, otherwise some information will not be stored and/or parsed correctly, so please be careful!

1. The tag of the Exit item

E.g. name it ‘ExitTestBlock’

2. The reference to the Exit tag

Update the JavaScript in the Init question of the new block to match the new name of the Exit tag, e.g.:

3. The name of the blockId

Update the JavaScript in the Init question of the new block to give this block a new blockId, e.g.

Note that you do not necessarily need to change the ids of the Stimulus questions. It is actually preferred to not change their names; after parsing your data you can then simply look at the same column in your data file and filter the relevant data based on the relevant blockId (which is a separate column in your parsed data file).

Finally, you might want to move your block to the correct position in your survey. The easiest way to do this is via the Survey Flow view. Simply hover Move and drag the block with your mouse. Do not use the Duplicate option here because it will create another reference to the same instance and the data is not stored only once.

Step 9. Test & Run the experiment!

We can now run the experiment. Click on ‘Preview Survey’ to view it without releasing it to the public. Please do note that you might have slight ‘vertical jumps’ due to the Qualtrics PreviewBanner being there.

Please do not forget to test whether your data is parsed correctly by the QRTE parser before you publish the experiment!