Building a Complex Experiment

Previous tutorials have explained much about the techniques with which we can build different features into our experiments using the QRTEngine. In this tutorial we integrate all these different techniques into a single, relatively complex, experiment that will hopefully take your experiment-building prowess to the next level. Before we start, it is advisable that you have read all the preceding tutorials on Creating Stimuli, Conditional Stimuli, Counterbalancing and Multiple Stimuli. We will be starting where the Building an Experiment from Scratch tutorial left off.

The Task

In this tutorial we will build a so-called Multi-Source Interference Task, or MSIT (Bush, et. al., 2003). The MSIT is a combination of a Simon task, introducing a spatial dimension, and a Flanker Task, introducing a flanker dimension on the task difficulty.  MSIT stimuli consist of three numbers, two of those numbers are flankers, which can be 0, 1, 2 or 3 and one of those is the ‘target’, which can be 1, 2 or 3. Participants are expected to respond to the ‘identity’ of the target, so if the target is 1, the participant has to press the ‘1’ key on his keyboard. Same goes for 2 and 3. For example, a valid MSIT stimulus looks like this:


like this:


Or like this:


In the first example, the target is ‘1’ on the first position, which makes it Simon congruent, however the flankers are ‘2’, which in a different stimulus could be a valid response, thus the flankers are ‘distracting’. This is not the case in the second example, where the flankers are neutral, however the target ‘3’ is on the second position, which makes it Simon incongruent. The third example combines all these incongruencies, having the target ‘2’ on the first position, with three as a flanker.

An optional part of the task is giving feedback. This can either be done by simply showing ‘correct’ or ‘incorrect’ based on the accuracy of the respondent, however a more interesting approach would be to color the target either green and red for a correct and incorrect response respectively. Next to that, depending on the trial, we will also color the target blue, for neutral feedback (which just shows the correct answer, rather than whether the participant responded correctly).

Note: Arguably, an even harder stimulus would be where the flanker’s identity is equal to the target’s location, for example, in 211. 

Before we get started on building the task, we would like to note that you should, in theory, have enough information now to get started on building the task yourself. We advice you to at least try your hand at making it yourself, and then checking back into this tutorial once you get stuck, or to compare your solution with ours. Good luck!

Building the task

As mentioned before, we’re basing our task off the basic experiment we built in the Building an Experiment from Scratch tutorial. This process will consist of several steps: Setting up the Loop & Merge spreadsheet, setting up the stimuli, setting up the correct JavaScript (although timings are a bit arbitrary here) and setting up the feedback function. We will be taking you through all of these steps, one by one. So please stick around!


Step 1. Setting up the Spreadsheet

One of the big advantages of using the QRTEngine, and specifically Loop & Merge  (L&M) blocks to control your trial flow, is that you can set-up your trial data in your favourite Spreadsheet program, and then simply  copy & paste the whole spreadsheet to the L&M window.  For ease of sharing, I will be using Google Spreadsheet, which you can check out here.

First off, we will get the bare basics started. We will need to have a column for both the flanker and the target, where the target can double as the identity of the correct response. Next to that, we will need to have the position of the target, which we will store in a third column. Flankers can be 0,1,2,3; targets can be 1,2,3 and target positions can be 1,2 and 3. This results in 36 unique combinations (4 x 3 x 3), however not all combinations are valid. The flanker cannot be the same as the target, so removing those trials leaves us with 27 valid combinations of flanker, target and position. As can be seen in the spreadsheet.

Now, from here on out there are two separate ways in which we could continue. The above flanker identity, target identity and target position are the bare essentials for setting up a trial, everything else can be computed from it (simon/flanker congruency and the specific look of the stimuli). With this information, you could build the Stimuli using JavaScript, by simply targeting the right elements, by Piped Text directly into the questions, but here I’d like to show a new, pretty cool technique making use of CSS to add content to pseudo-elements.

Step 2. Building our Stimuli

We have already set-up the JavaScript snippets during the ‘From scratch’ tutorial, so we will just need to fill in the details. As far as the flow of the trial goes, our Inter-trial interval (ITI) will be 1.5 seconds, our Cue will last 2.5 seconds, terminating upon an allowable button press and our feedback will last 1 second. We based these numbers on absolutely nothing but how nice the MSIT should feel to run.

JavaScript Snippet set-up

So, compared to the From Scratch experiment, we will need to add one Stimulus question, which we will name Feedback. We will rename the ‘Stimulus’ question to ‘Cue’, to make it more explicit which question it is.

As Init questions are used to set-up our trial configurations, we will need to add the following to the Init question’s JavaScript, to set up our task properly:

  • Set the Block ID (we’ll name it MSIT)
  • Set the Exit Item Tag (which is ‘Exit’)
  • Set duration of ITI to 1.5 sec.
  • Set duration of Cue to 2.5 sec.
  • Set Allowable keys of Cue to ‘123’.
  • Set Correct response to whatever is in column 2 of the Loop & Merge.
  • Set Cue to TERMINATE upon an allowable key press
  • Set Duration of feedback to 1 sec.

Which should result in the following JavaScript:

The Cue and Feedback (Both correct and incorrect)JavaScript snippets should be set to the correct id, and will respectively be:


Nothing will change about the Exit question JS.

HTML snippet set-up

As far as the looks of stimuli are concerned, we are sort of left up to arbitrary choices again. We generally pick a font-size of 32 pixels for our stimuli (because they’re purely text based) and a + as an Intertrial stimulus and also text-align them in the center. This can all be neatly done using the rich content editor qualtrics provides. So we get the resulting HTML for The Init and Exit questions (They are the same):

The HTML for the Cue and Feedback can be created using the Rich Content Editor as well, but, in order to gain a bit more flexibility, we need to edit the generated HTML slightly, giving each number in the stimulus its own <span> with their own unique id’s and classes. This will be useful for both our JS and piped text based stimuli.

This is what the Rich Content Editor makes for us:

Whereas this is what we created, base on the above:

The same idea was applied to the feedback stimuli:



Each span has a unique id relating to which type of stimulus they belong to. Next to that, each stimulus is an MSIT stimulus, thus we added a class for each character in the stimulus telling us whether it’s the first, second or third character of the stimulus. This will be useful for our CSS-based solution.

Building the Stimuli

As you can probably see, we added a bunch of spans to represent the location of the different components the stimuli are built out of., but they are empty. This won’t do, so we’ll need to change these elements to represent the correct numbers. The best place to do this is also during the Initialisation phase (we can count this as part of the initialisation!). So we’ll add the required CSS to the Init question!

So what are CSS pseudo-elements we mentioned earlier and why are they so cool? Generally the separation between HTML and CSS is that HTML is meant for content (and marking it up) and CSS for styling that content. However, the borders become sort of fuzzy when one starts looking at icons and very standard text. Icons are images, and images are content, however these images are generally very standard, and used to style up certain text (think of an E-mail adress getting an e-mail icon, a save button getting a floppy disk icon). So, the case can be made for CSS to be able to handle this sort of thing, rather than having to ‘contaminate’ your HTML with this ‘styling’.

Pseudo-elements are basically the implementation of this idea, it allows you to add stuff ‘before’ and ‘after’ elements. Stuff such as ‘content’, which can be anything you can do with CSS, but also text. See – for more information on the subject.

So, the way we’ll use it, is sort of abuse of the functionality, as the stimuli definitely are in fact content, but it allows us to very neatly use CSS to generate the stimuli that we need. First off, note that we defined the class – MSIT-1, MSIT-2 and MSIT-3, for the first, second and third characters of each stimulus respectively. If we add the following HTML (well, CSS) to our Init question, we can set the content of these classes to be X.

You should immediately see the effects in your editor as well. Every stimulus becomes X. Now, we can make use of the Piped Text features of Qualtrics to set this to be the flanker! See the following CSS. (If you haven’t already, now would be a good time to enable Loop & Merge on your block and copy the columns & rows from the Google Docs spreadsheet to your L&M spreadsheet).

Out comes our next trick, the first line of any CSS statement is called the selector. Our styling above selects all elements that are of the class MSIT-1, MSIT-2 OR MSIT-3. However, using Qualtrics Piped Text (again) we can make a CSS statement that only applies to the element that corresponds with the location of our target, and then set the content of the before pseudo-element to that content! What in theory should work:

The !important means that that rule will raise the priority of that rule, overwriting any existing rule (such as the one that sets the character to be the flanker). This doesn’t work however, because, in the editor, the CSS will be evaluated as is, and for reasons we don’t understand, in the editor every single element is selected and then gets the ${lm://Field/2} text added to it. Although it’s funny to see, it’s not very productive. For that reason, we need to resort to a slightly different selector that does the same thing.

Span is the tag that we put these things in, and class is an attribute of that element (a special one though, that allows CSS selectors to work over them). So instead of use the . syntax to select the classes, we can use the attribute selector syntax to do the same, which doesn’t suffer from the problems introduced by the previous example CSS.

Next up, we need to set the correctly colours (red and green) for the incorrect and correct feedback stimuli. Note that due to our tendency to select classes, every stimulus is correctly set, and not just the cue. We have added unique id’s to each span, so we can select these in CSS as well, using the same trick as we did before for the class selection:

So, our full HTML for the Init question will become:

This is the neatest way we have found to build the stimuli, there are several other possibilities, such as directly piping the text to the stimuli, rather than through the CSS, but this results in a much bulkier L&M spreadsheet.

Step 3. Setting the conditionals

Last, but not least, we need to set the conditionals. This is a rather trivial step if you have followed the Conditional Stimuli guide, as we simply need to check whether the response was correct or not, or rather whether the accuracy was 1 or 0 for the correct and incorrect feedback questions.

So the final JavaScript for the Feedback questions become:

For correct and:

For incorrect.

Run the experiment

If you have followed all the above steps correctly, you should be able to run your experiment now and have a working MSIT task. Our implementation of the task can be downloaded here – MSIT Task. Both the Trial Data and the .QSF files are there. This task concludes the series of Basic tutorials we have created, at least for now. Please do contact us at if you have any questions or would like to see a tutorial about any other aspect the Engine. Thanks for reading!