IMS Question and Test Interoperability (QTI) is the most widely used and adopted standard for marking up quizzes, assessments, tests and exams. At its core, QTI is basically a zip file containing quiz data in the form of XML and its associated multimedia content.

Most major Learning Management Systems (LMS) like Canvas, Brightspace, Blackboard, Schoology, Sakai and NEO LMS supports QTI either in its importing or exporting or both. It is arguably the most common format used to exchange and store quiz data.

There are 3 major ‘flavours’ of QTI - QTI 1.x, 2.x, and the most recently release 3. Most modern LMS supports both QTI 1.2 and 2.1, with older LMS like Sakai only supporting QTI 1.2. Publishers like Wiley, Pearson, and Cognero also offer exporting of their assessment question bank in QTI format too. If you are intending to support QTI, we would recommend that you start with QTI 2.1 as most major players within the industry are on this. At the time of writing, there’s no known LMS or publishers that supports QTI 3.0 as the standard was only finalised on September 2020 and publicly released on March 2022.

Finally, despite QTI’s prevalence in the edutech industry, there has been a shockingly lack of good articles talking about how QTI works in an easy-to-understand way. So, in this article, we will talk about how QTI 2.1 works and how you might go about implementing it on your own learning platform.
About us
GETMARKED offers a Quiz Import and Export API that helps Edtech startups and companies create and read any questions bank files (e.g. QTI, Moodle XML, Common Cartridge, Docx, etc.) so that you do not have to deal with the many QTI variants out there as every platform implements QTI differently. We provide a single API to meet all your quiz importing and exporting needs.




QTI Assessment Item

Understanding QTI starts from understanding the Assessment Item. The Assessment Item is the smallest building block of an Assessment, with an Assessment being a collection of those Assessment Items. An Assessment Item (we will be referring to it as item from this point onwards) can contain any arbitrary content like images, video, audio, text, or html. It can also optionally contain information on how to grade and process the results and students responses. In most real-life cases, items are structured simply even though the standard can accommodate complex representations.

Here’s the structure of an Assessment Item. 
    <assessmentItem xmlns="http://www.imsglobal.org/xsd/imsqti_v2p1" ...>
        <responseDeclaration ...>
            <!-- Information about the answers to the questions inside -->
        </responseDeclaration>

        <outcomeDeclaration ...>
            <!-- Internal variables for computing scores, controlling feedback, etc. -->
        </outcomeDeclaration>

        <itemBody ...>
           <!-- Text and the interactive parts (questions) -->
        </itemBody>

        <responseProcessing ...>
           <!-- How to handle the user's response -->
        </responseProcessing>

        <modalFeedback ...>
           <!-- Optional feedback messages to the user -->
        </modalFeedback>
    </assessmentItem>

  • <responseDeclaration> declares the correct answer(s) to the ‘question’. Within a single <AssessmentItem> there may be more than one ‘question’. Every ‘question’ will require a corresponding <responseDeclaration>.
  • <outcomeDeclaration> creates internal variables that can be used to calculate scores or specify dynamic behaviours like when to show feedback (perhaps you want to show different feedback for different student’s response or grade).
  • <itemBody> is where the actual question content lives. This also will be what is visible to the students. Generally, you can place any valid html (XHTML really). While standard does not prescribe that you must render it with a browser, which leaves a lot of room for interpretation, most learning platform do.
  • <responseProcessing> contains the actions to perform when the student is finished with the item. For example, how to grade a the student’s response. Whether to show feedback or not. Its content is a simple but verbose programming language written in XML to which you can perform computations, set <outcomeDeclaration> variables, and more.
  • <modalFeedback> defines the item's feedback to the student. You may have any number of customised feedback that can be reflected to the students based on the conditions you set in <responseProcessing>. Generally, only 2 kinds are used, correct and incorrect so as to simplify the creation of <responseProcessing>.


Interaction Types
Each <itemBody> can contain zero or more interactions. The best way to understand what is an interaction is to see it as a ‘question’ or a field that requires a student to response to.

Take for example a MCQ, Which only has one interaction. 
                    

    <itemBody>
       <choiceInteraction responseIdentifier="RESPONSE" shuffle="true" maxChoices="1">
          <prompt>What is the top color of the Dutch flag?</prompt>
          <simpleChoice identifier="A">Red</simpleChoice>
          <simpleChoice identifier="B">Orange</simpleChoice>
          <simpleChoice identifier="C">Black</simpleChoice>
          <simpleChoice identifier="D">White</simpleChoice>
       </choiceInteraction>
    </itemBody>
Versus a cloze (fill-in-the-blank) passage, which has two interactions. 
                    

<itemBody>
   <blockquote>
    The sky is <textEntryInteraction responseIdentifier=”RESPONSE_1” expectedLength=”15” />
    in colour. The same colour as the
    <textEntryInteraction responseIdentifier=”RESPONSE_2” expectedLength=”15” />
   </blockquote>
</itemBody>
Each <itemBody> can contain zero or more interactions. Items with two or more interactions are called composite items. Composite items may contain multiple instances of the same type of interaction or have a mixture of interaction types.

Note that every interaction would have a 'responseIdentifier' that binds to a <responseDeclaration> with the same identifier. This binding will associate the relevant correct answer to each of the interaction field. With this association you will then be able to use <responseProcessing> to grade the responses.

The sample interactions that is shown here is only the tip of the iceberg of what QTI supports. QTI v2.1 defines a total of 20 interaction types. You can find a full set of examples in the IMS Question and Test Interoperability Implementation Guide. While you determine which types to use, note that not all QTI processing engines support all interaction types.

Lastly, within the QTI standard there is a concept of <prompt> and <stimulus>, which always appears inside the <itemBody>. Prompt can be thought of as the exact specific question text or instruction to the student. While stimulus is shared content between items and typically sits outside of interactions (prompt usualy is inside). The important thing to note is that it’s not a requirement to have <prompt> nor <stimulus> inside <itemBody>. Most learning platform don’t seem to treat contents in <prompt> or <stimulus> any differently than say <p> or <div>.



QTI Data Model

To udnerstand QTI's response and result processing, it is critical to understand the possible data types answers could take, as the huge variety of data types makes <responseProcessing> seemingly complicated.

When a student completes doing a quiz (item), their responses need to be graded. Sometimes, the responses is represented simply: A multiple-choice, single-answer question just returns the identifier of the given answer, and a numeric slider interaction returns a number. Most interaction types, however, are more complicated. For instance:
  • Multi-response question returns a set of answer identifiers (e.g. "The identifiers of the answers checked were A, C, and D.")
  • Ordering things returns an ordered set of answer identifiers (e.g. "The identifiers of the sequence created were C, A, and B.").
  • Matching questions return a set of identifier pairs (e.g. "The words were associated as (A, C), (B, D), and (E, F).").
  • Positioning objects on a map returns a set of points (e.g. "The objects were placed at locations (102, 10), (125, 653).").

Here’s sample response declaration of an interaction that uses the data type that is a set of points. 
                    

    <responseDeclaration identifier="RESPONSE" baseType="point" cardinality="multiple">
       <correctResponse>
          <value>118 184</value>
          <value>150 235</value>
          <value>96 114</value>
       </correctResponse>
    </responseDeclaration>
QTI data types are defined by the baseType and cardinality attributes in <responseDeclaration> and <outcomeDeclaration> elements
  • The baseType attribute defines the basic data type:
    • boolean
    • directedPair
    • duration
    • file
    • float
    • identifier
    • integer
    • pair
    • point
    • string
    • uri
  • The cardinality attribute defines the number of baseType elements and whether the set is ordered. Values are:
    • multiple
    • ordered
    • record (a special, rather complicated type not discussed in this article)
    • single

No learning platforms supports all interaction types. Usually the complicated ones are not implemented.



Response Processing
<responseProcessing> takes care of what happens after the user completes attempting the item. The QTI processor must perform tasks such as deciding whether the answer is correct, computing a score, and providing feedback. The <responseDeclaration>, <outcomeDeclaration>, and <responseProcessing> sections all have a role in this process.



Response Declarations
The identifier of a <responseDeclaration> element always binds it to an interaction in the item's body. Here’s a sample. 
                    

<responseDeclaration identifier="QUESTION1" cardinality="single"
                     baseType="identifier">
   <correctResponse>
      <value>B</value>
   </correctResponse>
</responseDeclaration>
<itemBody>
   <choiceInteraction responseIdentifier="QUESTION1" shuffle="false" maxChoices="1">
      <prompt>Is a goldfish a mammal?</prompt>
      <simpleChoice identifier="A">Yes</simpleChoice>
      <simpleChoice identifier="B">No</simpleChoice>
   </choiceInteraction>
</itemBody>
The identifier can be any string, though its customary to use some “RESPONSE” if there's only one interaction. This will allow you to use predefined templates for <responseProcessing>, which we will talk about later. Also, every interaction you have always requires a corresponding responseDeclaration that is bound to it.

A <responseDeclaration> is an object (or variable) that has three "methods" that you can call in your computations in the <responseProcessing> section of the item:
  • Get the learner's response to the question.
  • Get the correct (optimal) response to the question.
  • Map different answers to points so that an answer can have more nuances than plain right or wrong.

Here's an example of a <responseDeclaration> being used to grade student's responses and assigning variable score. 
                    

<responseDeclaration identifier="QUESTION1" cardinality="multiple"
                     baseType="identifier">
   <correctResponse>
      <value>A</value>
      <value>D</value>
   </correctResponse>
   <mapping defaultValue="0" lowerBound="0" upperBound="1">
      <mapEntry mapKey="A" mappedValue="0.5"/>
      <mapEntry mapKey="B" mappedValue="-0.5"/>
      <mapEntry mapKey="C" mappedValue="-0.5"/>
      <mapEntry mapKey="D" mappedValue="0.5"/>
   </mapping>
</responseDeclaration>
<itemBody>
   <choiceInteraction responseIdentifier="QUESTION1" shuffle="false" maxChoices="0">
      <prompt>Which are colors?</prompt>
      <simpleChoice identifier="A">Red</simpleChoice>
      <simpleChoice identifier="B">Small</simpleChoice>
      <simpleChoice identifier="C">Soft</simpleChoice>
      <simpleChoice identifier="D">Purple</simpleChoice>
   </choiceInteraction>
</itemBody>
So, if the student provided the answers A (red), B (small), and D (purple), the score would be calculated according to the <mapping> as (A) 0.5 + (B) -0.5 + (D) 0.5 = 0.5.

You might wonder why this scoring done inside a <responseDeclaration>, when throughout the article, we stated that <responseProcessing> was the one that does the grading. This is because QTI wants to make it easy to create simple <assessmentItem> to reduce adoption barrier while <responseProcessing> is meant for more complex cases. Some learning platform even ignores the response processing and only looks at the defined correct responses. The same reasoning applies for the existence of predefined response processing templates.



Outcome Declarations
An <outcomeDeclaration> is also a variable, like <responseDeclaration> and its simple to declare. Here's an example. 
                    

<outcomeDeclaration identifier="SCORE" cardinality="single" baseType="float">
   <defaultValue>
      <value baseType="float">0.5</value>
   </defaultValue>
</outcomeDeclaration>
This markup defines a variable called SCORE, with a single float in it and a default value of 0.5. Providing a default is optional. If you don't, numeric variables are initialized to 0.0 and non-numeric variables to NULL.

Note: SCORE is a reserved name. It is the total numeric score representing the student's grade on the specific assessment item.

You can add more intelligence to an <outcomeDeclaration> by providing a structure called <matchTable> or <interpolationTable>. I haven't seen them used, and most learning platform probably don't support them, so I won't explain them here.

If you are using feedback, you would declare it here like so: 
                    

<outcomeDeclaration identifier="FEEDBACK" cardinality="string" />


Response Processing
The <responseProcessing> section of an item contains instructions for the learning platform to know what to do after the student has provided an answer. The instructions can be in two formats:
  • A program or script in a simple, XML-based programming language
  • A reference to a predefined template that is also a script in the response processing programming language.
Let's take a look at how the scripting language works. 
                    

<responseDeclaration identifier="RESPONSE_01" cardinality="single"
                     baseType="string">
   <correctResponse>
      <value>white</value>
   </correctResponse>
</responseDeclaration>
<responseDeclaration identifier="RESPONSE_02" cardinality="single"
                     baseType="string">
   <correctResponse>
      <value>green</value>
   </correctResponse>
</responseDeclaration>

<outcomeDeclaration identifier="SCORE" cardinality="single" baseType="float"/>

<itemBody>
   <p>Fill in the right colors:</p>
   <p>Snow is
      <textEntryInteraction responseIdentifier="RESPONSE_01" expectedLength="5"/>
      and grass is
      <textEntryInteraction responseIdentifier="RESPONSE_02" expectedLength="5"/>.
   </p>
</itemBody>

<responseProcessing>
   <responseCondition>
      <responseIf>
         <and>
            <match>
               <variable identifier="RESPONSE_01"/>
               <correct identifier="RESPONSE_01"/>
            </match>
            <match>
               <variable identifier="RESPONSE_02"/>
               <correct identifier="RESPONSE_02"/>
            </match>
         </and>
         <setOutcomeValue identifier="SCORE">
            <baseValue baseType="integer">1.0</baseValue>
         </setOutcomeValue>
      </responseIf>
      <responseElse>
         <setOutcomeValue identifier="SCORE">
            <baseValue baseType="integer">0.0</baseValue>
         </setOutcomeValue>
      </responseElse>
   </responseCondition>
</responseProcessing>
This response processing examines the values of the learner's two responses to the defined correct (optimal) responses. This is done in the <match> elements. Both must be correct, which is handled by the surrounding <and> element. If they are, the SCORE <outcomeDeclaration> is set to 1.0; otherwise, it is set to 0.0.

Here's another somewhat more complicated example: 
                    

<responseDeclaration identifier="RESPONSE" cardinality="multiple"
                     baseType="identifier">
   <correctResponse>
      <value>A</value>
      <value>C</value>
   </correctResponse>
   <mapping lowerBound="0.0" upperBound="1.0" defaultValue="0.0">
      <mapEntry mapKey="A" mappedValue="0.5"/>
      <mapEntry mapKey="B" mappedValue="-0.25"/>
      <mapEntry mapKey="C" mappedValue="0.5"/>
      <mapEntry mapKey="D" mappedValue="-0.25"/>
   </mapping>
</responseDeclaration>

<outcomeDeclaration identifier="SCORE" cardinality="single" baseType="float"/>
<outcomeDeclaration identifier="FEEDBACK" cardinality="single" baseType="identifier"/>
<outcomeDeclaration identifier="FEEDBACK_TRESHOLD" cardinality="single"
                    baseType="float">
   <defaultValue>
      <value baseType="float">0.75</value>
   </defaultValue>
</outcomeDeclaration>

<itemBody>
   <choiceInteraction responseIdentifier="RESPONSE" shuffle="true" maxChoices="0">
      <prompt>Which countries are south of the USA?</prompt>
      <simpleChoice identifier="A">Mexico</simpleChoice>
      <simpleChoice identifier="B">Canada</simpleChoice>
      <simpleChoice identifier="C">Brazil</simpleChoice>
      <simpleChoice identifier="D">Norway</simpleChoice>
   </choiceInteraction>
</itemBody>

<responseProcessing>
   <responseCondition>
      <responseIf>
         <isNull>
            <variable identifier="RESPONSE"/>
         </isNull>
         <setOutcomeValue identifier="SCORE">
            <baseValue baseType="float">0.0</baseValue>
         </setOutcomeValue>
         <setOutcomeValue identifier="FEEDBACK">
            <baseValue baseType="identifier">FAILURE</baseValue>
         </setOutcomeValue>
      </responseIf>
      <responseElse>
         <setOutcomeValue identifier="SCORE">
            <mapResponse identifier="RESPONSE"/>
         </setOutcomeValue>
         <responseCondition>
            <responseIf>
               <gte>
                  <variable identifier="SCORE"/>
                  <variable identifier="FEEDBACK_TRESHOLD"/>
               </gte>
               <setOutcomeValue identifier="FEEDBACK">
                  <baseValue baseType="identifier">ANSWER_CORRECT</baseValue>
               </setOutcomeValue>
            </responseIf>
            <responseElse>
               <setOutcomeValue identifier="FEEDBACK">
                  <baseValue baseType="identifier">FAILURE</baseValue>
               </setOutcomeValue>
            </responseElse>
         </responseCondition>
      </responseElse>
   </responseCondition>
</responseProcessing>
In this example, as the grading logic was moved to <mapping> and <responseProcessing> only need to set the score to SCORE from <mapResponse>. This response processing also sets an <outcomeDeclaration> called FEEDBACK to either FAILURE or ANSWER_CORRECT. You can use this declaration to provide a suitable feedback to the user based on any condition you might want (typically based on SCORE).

In a nutshell, response processing followed these steps in the sample:
  • Determine whether an answer is present (maybe the user clicked Continue without providing an answer). If so, the <responseDeclaration> value is NULL (checked by the <isNull> element), and SCORE and FEEDBACK are set to the right values.
  • Set the SCORE to the value provided by the <mapping> from the <responseDeclaration> (using the <mapResponse> element).
  • Compare the SCORE to a fixed-value set by the FEEDBACK_TRESHOLD <outcomeDeclaration> (in the <gte> [greater than or equal] element) and sets the FEEDBACK accordingly.

Besides declaring your own variables through outcome declaration, there also exist some predefined variables in the response processing:
  • duration. The time the user spent answering the question in seconds. Use this variable, for instance, to lower the score when it took the user too long to provide an answer.
  • nunAttempts and completionStatus. These variables are used in adaptive processing which we wouldn't cover here.

While there might seem like a lot of things going on, writing response processing is really not complicated, only verbose. A full definition of the language can be found in QTI document, IMS Question and Test Interoperability Assessment Test, Section, and Item Information Model.



Predefined Response Processing Templates
As I already touched on in Response declarations, the QTI standard has built-in options for simple response processing. For instance, by using the correct response in a <responseDeclaration>, it can determine whether a given answer is right or wrong without doing any scripted response processing at all. This approach allows simple content and accompanying rendering engines without violation of the standard. Another simplifying feature is the response processing templates. QTI predefines three templates for basic response processing. The templates are identified by fixed URIs—for instance: 
                    

<responseProcessing
   template="http://www.imsglobal.org/question/qti_v2p0/rptemplates/match_correct"/>
Actual scripts are coupled to these URIs. A rendering engine can handle them as scripts should. To help rendering engines find the right template, you can provide a location such as: 
                    

<responseProcessing template="..." templateLocation="/rptemplates/match_correct.xml/>
You can use the predefined templates if:
  • The action includes only one interaction
  • The <responseDeclaration> for this action is called RESPONSE
  • The action includes an <outcomeDeclaration> called SCORE of type float

The predefined templates are:
  • Match_Correct: Set the score to 0.0 if the answer is absent or wrong and to1.0 if it is right. The URI is: http://www.imsglobal.org/question/qti_v2p0/rptemplates/match_correct
  • Map_Response: Set the score to 0.0 if there was no answer; otherwise, use the mapping inside the RESPONSE <responseDeclaration>. The URI is: http://www.imsglobal.org/question/qti_v2p1/rptemplates/map_response
  • Map_Response_Point: This is the same as Map_Response but for graphic interaction types. It uses area mapping instead of identifier mapping. The URI is: http://www.imsglobal.org/question/qti_v2p1/rptemplates/map_response_point




Conclusion
Hopefully this guide was useful in helping you understand how the QTI standard works. In most cases, it should be fairly straightforward to create and read it. The only complicated portion is in response processing, where you might need to implement some kind of intrepreter. If you don't want to or don't have to implement the full QTI response and response processing, you can still stay within the standard and fall back on comparing the given correct responses or use the built-in response processing templates.

Lastly, if you want to test if the QTI file you have created is compliant, there are some websites where you can do this. You can create an account on Canvas LMS or use Onyx Editor. Both options are free to use. All the best!




About Us
GETMARKED Digitaliser API helps you convert any QTI zip file or docx quiz document into a simple structured JSON question bank that you can integrate with in a single afternoon. If you are a developer and you are frustrated by the plethora of QTI standards out there, you can use our API to handle all the quiz importing for you. One API to import from anywhere.