Multiple choice inputs

The whole point of STACK is not to use multiple-choice questions, but instead to have the student enter an algebraic expression! That said there are occasions where it is very useful, if not necessary, to use multiple-choice questions in their various forms. STACK's use of a CAS is then very helpful to generate random variants of multiple-choice questions based on the mathematical values.

This can also be one input in a multi-part randomly generated question. E.g. you might say "which method do you need to integrate $\sin(x)\cos(x)$?" and give students the choice of (i) trig functions first, (ii) parts, (iii) substitution, (iv) replace with complex exponentials. (Yes, this is a joke: all these methods can be made to work here!) Another algebraic input can then be used for the answer.

If you are new to STACK please note that in STACK MCQs are not the place to start learning how to author questions. Please look at the authoring quick-start guide.

Please read the section on inputs first.

Multiple choice input types return a CAS object which is then assessed by the potential response tree. For this reason, these inputs do not provide "feedback" fields for each possible answer, as does the Moodle multiple choice input type.

• Checkbox inputs return a list of the CAS objects selected;
• Dropdown inputs and radio inputs return a single expression, which is the CAS object selected.

The goal of these input types is to provide modest facilities for MCQ. An early design decision was to restrict each of the possible answers to be a CAS expression. In particular, we decided NOT to make each possible answer CASText. Adopting CASText would have provided more flexibility but would have significantly increased the complexity of the internal code. If these features are extensively used we will consider a different input type.

Example questions are in the stack library under Features\MCQ_.....

Model answer

This input type uses the "model answer" both to input the teacher's answer and the other options. In this respect, this input type is unique, and the "model answer" field does not contain just the teacher's model answer. Constructing a correctly formed model answer is complex, and so this input type should be considered "advanced". New users are advised to gain confidence writing questions with algebraic inputs first, and gain experience in using Maxima lists.

The "model answer" must be supplied in a particular form as a list of lists [[value, correct(, display)], ... ].

• value is the value of the teacher's answer.
• correct must be either true or false. If it is not true then it will be considered to be false!
• (optional) display is another CAS expression to be displayed in place of value. This can be a string value here, but it will be passed through the CAS if you choose the LaTeX display option below. display is only used in constructing the question. STACK will take value as the student's answer internally, regardless of what is set here.

For example

 ta:[[diff(p,x),true],[p,false],[int(p,x),false]]

At least one of the choices must be considered correct. However, the true and false values are only used to construct the "teacher's correct answer". You must still use a potential response tree to assess the student's answer as normal.

Note the value expression must be constructed with noun functions where a noun alternative exists. E.g. use %union() (the noun form) not union(). STACK has to match up what the student selects, and this is an expression they might well type into an algebraic input. In this situation functions in students' expressions are turned to noun forms. Hence the teacher must also use noun forms in setting up the MCQ input options.

STACK provides some helper functions

1. mcq_correct(ta) takes the "model answer" list and returns a list of values for which correct is true.
2. mcq_incorrect(ta) takes the "model answer" list and returns a list of values for which correct is false.

Note, that the optional display field is only used when constructing the choices seen by the student when displaying the question. The student's answer will be the value, and this value is normally displayed to the student using the validation feedback, i.e. "Your last answer was interpreted as...". A fundamental design principal of STACK is that the student's answer should be a mathematical expression, and this input type is no exception.

In situations where there is a significant difference between the optional display and the value which would be confusing, the only current option is to turn off validation feedback. After all, this should not be needed anyway with this input type. In the example above when a student is asked to choose the right method the value could be an integer and the display is some kind of string.

An example which includes the display option is

tacp:[[A, false, "A. Direct proof"],  [B, false, "B. Definition-chasing"], [C, false, "C. If and only if"], [D, false, "D. Exhaustive cases"], [E, false, "E. Induction"], [F, false, "F. Contrapositive"], [G, true, "G. Contradiction"]];

Note in this example the value of the student's answer will be a letter which is literally a Maxima variable name. In this situation you can't really randomize the letters used easily. (Not impossible with some cunning code, but a lot of work....)

If you choose to use an integer, and randomly shuffle the answers then the validation feedback would be confusing, since an integer (which might be shuffled) has no correspondence to the choices selected. This behaviour is a design decision and not a bug! It may change in the future if there is sufficient demand, but it requires a significant change in STACK's internals to have parallel "real answer" and "indicated answer". Such a change might have other unintended and confusing consequences.

Normally we don't permit duplicate values in the values of the teacher's answer. If the input type receives duplicate values STACK will throw an error. This probably arises from poor randomization. However it may be needed. If duplicate entries are permitted use the display option to create unique value keys with the same display. This behaviour is a design decision may change in the future.

When STACK displays the "teacher's answer", e.g. after a quiz is due, this will be constructed from the display fields corresponding to those elements for which correct is true. I.e. the "teacher's answer" will be a list of things which the student could actually select. Whether the student is able to select more than one, or if more than one is actually included, is not checked. The teacher must indicate at least one choice as true.

If you need "none of these" you must include this as an explicit option, and not rely on the student not checking any boxes in the checkbox type. Indeed, it would be impossible to distinguish the active selection of "none of these" from a passive failure to respond to the question.

Model answer: nouns

If one of the responses is $x=1 \text{ or } x=2$ then use nounor which is commutative and associative. Do not use or which always simplifies its arguments. In this example x=1 or x=2 evaluates to false.

Functions diff and int will evaluate, so you don't have displayed calculus operation. If one of the responses is a displayed derivative or integral then construct your answer with Maxima's inert forms 'diff and 'int.

Model answer: LaTeX display in dropdowns

HTML dropdowns cannot display LaTeX within the options. This is a restriction of HTML/MathJax (not of STACK). You can use HTML-entities within a string field. For example

ta1:[[0,false,"n/a"],[1,true,"≥"],[2,false,"≤"],[3,false,"="],[4,false,"?"]];

Note here that an integer will returned internally.

Similarly, you can include logical symbols. For example

ta1:[[0, false, "⇒"], [1, true, "⇐"], [2, false, "⇔"]];

will give a choice, e.g. a dropdown, from ⇒, ⇐ and ⇔ and an integer will returned internally.

ta1:[[0, false, "∀"], [1, true, "∃"]];

will give a choice, e.g. a dropdown, from ∀ and ∃.

ta1:[[0, true, "c ∈"], [1, false, "c ∉"]];

will give a choice, e.g. a dropdown, from "c ∈" and "c ∉".

ta1:[[N, false, "ℕ"], [Z, true, "ℤ"], [Q, false, "ℚ"], [R, false, "ℝ"], [C, false, "ℂ"]];

will give a choice between sets of numbers ℕ, ℤ, ℚ, ℝ, and ℂ.

Example: degree of a polynomial

Here is an example where the teacher would like the student to state the degree of a polynomial using the adjectives, rather than a number.

pol:sum(rand_with_prohib(-9,9,[0])*x^k,k,0,1+rand(5));
deg:hipow(pol,x);
/* Use strings, and not keywords. */
l1:["constant", "linear", "quadratic", "cubic", "quartic", "quintic"];
/* Create a matching list of true/false values as to whether each option is correct. */
a1:maplist(lambda([ex], is(ex=deg)), makelist(k,k,0,length(l1)));
/* Basic answer list for MCQ in the correct format. This returns the string to the PRT. */
ta1:zip_with("[",l1,a1);
/* Since lists index at one, you need this for the correct answer! */
tac:l1[deg+1];
/* Returns the degree as the student's answer, not the word. */
ta2:maplist(flatten, zip_with("[",makelist(k,k,0,length(l1)), zip_with("[",a1,l1)));
/* Feedback can turn this into a word using indexing, e.g. l1[ans1+1] in the PRT. */

Either ta1 or ta2 can be used with the MCQ inputs, in this case dropdown probably makes most sense and checkbox least sense! With the ta2 option you probably want to hide the validation feedback.

Internals

The dropdown and radio inputs return the value, but the checkbox type returns the student's answer as Maxima list, even if they have only chosen one option.

The function mcq_correct returns a list, so for dropdown and radio you might need to take the first element when using this function to extract values from the list of teacher's answers.

If, when authoring a question, you switch from radio/dropdown to checkboxes or back, you will probably break a PRT because of mismatched types.

For the select and radio types the first option on the list will always be "Not answered". This enables a student to retract an answer and return a "blank" response.

For the checkbox type there is a fundamental ambiguity between a blank response and actively not selecting any of the provided choices, which indicates "none of the others". Internally STACK has a number of "states" for a student's answer, including blank, valid, invalid, score etc. A student who has not answered will be considered blank, which is different from invalid. Potential response trees which rely on this input type will not activate until the state is score.

To enable a student to indicate "none of the others", the teacher must add this as an explicit option in all MCQ types. If you add an option "none of the others" then it will return the value of that selection: you could give this the value of null, for example, which is a Maxima atom. We did not add support for a special internal "none of the others" because the teacher still needs to indicate whether this is the true or false answer to the question. To support randomisation, this needs to be done as an option in the teacher's answer list.

The radio and dropdown types always add a "not answered" option as the first option. This allows a student to retract their choice, otherwise they will be unable to "uncheck" a radio button, which will be stored, validated and possibly assessed (to their potential detriment). If you want to remove this then use the extra option nonotanswered, but keep in mind the possible effect when using the penalty scheme.

If one of the items in the teacher's answer list is is the special variable name notanswered, and then default message (Clear my choice) will be replaced by the display value. If no display value is given (and it is optional) then the original message will remain. notanswered will not appear in the list of valid choices for a user and value for this input is ingored.

Extra options

These input types make use of the "Extra options" field of the input type to pass in options. These options are not case sensitive. This must be a comma-separated list of values as follows, but currently the only option is to control the display of mathematical expressions.

The way the items are displayed can be controlled by the following options.

• LaTeX The default option is to use LaTeX to display the options, using an inline maths environment \(...\). This is probably better for radio and checkboxes. It sometimes works in dropdowns, but not always and we need to test this in a wider variety of browsers.
• LaTeXdisplay use LaTeX to display the options, using the display maths environment \[...\].
• LaTeXinline use LaTeX to display the options, using the inline maths environment \(...\).
• LaTeXdisplaystyle use LaTeX to display the options, using the inline maths environment and the displaystyle option \(\displaystyle...\).
• casstring does not use the LaTeX value, but just prints the casstring value in <code>...</code> tags.
• nonotanswered removes the ``Not answered'' option from radio and dropdown. This is not recommended as it means a student has no opportunity to "uncheck" a radio button once selected. They may wish not to answer, rather than save an incorrect answer.

Randomly shuffling the options

To randomly shuffle the options create the list in the question variables and use the Maxima command random_permutation in the question variables.

For example, the question variables might look like the following.

/* Create a list of potential answers. */
p:sin(2*x);
ta:[[diff(p,x),true],[p,false],[int(p,x),false],[cos(2*x)+c,false]];
/* The actual correct answer.    */
tac:diff(p,x)
/* Randomly shuffle the list "ta". */
ta:random_permutation(ta);
/* Add in a "None of these" to the end of the list.  The Maxima value is the atom null. */
tao:[null, false, "None of these"];
ta:append(ta,[tao]);

These commands ensure (1) the substantive options are in a random order, and (2) that the None of these always comes at the end of the list. Note, the value for the None of these is the CAS atom null. In Maxima null has no special significance but it is a useful atom to use in this situation.

As the Question Note, you might like to consider just taking the first item from each list, for example:

{@maplist(first,ta)@}.  The correct answer is {@tac@}.

This note stores both the correct answer and the order shown to the student without the clutter of the true/false values or the optional display strings. Remember, random variants of a question are considered to be the same if and only if the question note is the same, so the random order must be part of the question note if you shuffle the options.

Constructing MCQ arrays in Maxima

It is not easy to construct MCQ arrays in Maxima. This section contains some tips for creating them, using Maxima's lambda command. Below is an example of a correctly constructed teacher's answer.

ta:[[x^2-1,true],[x^2+1,false],[(x-1)*(x+1),true],[(x-i)*(x+i),false]]

To create a list of correct answers you could use the function mcq_correct(ta). This essentially consists of the following code.

maplist(first, sublist(ta, lambda([ex], second(ex))));

The function sublist "filters" out those entries of ta for which the second element of the list is true. We then "map" first onto these entries to pull out the value. It is relatively simple to modify this code to extract the incorrect entries, the displayed forms of the correct entries etc.

To go in the other direction, the first list ta1 is considered "correct" and the second ta2 is considered incorrect.

ta1:[x^2-1,(x-1)*(x+1)];
ta1:maplist(lambda([ex],[ex, true]), ta1);
ta2:[x^2+1,(x-i)*(x+i)];
ta2:maplist(lambda([ex],[ex, false]), ta2);
ta:append(ta1,ta2);
/* If you want to shuffle the responses then use the next line. */
ta:random_permutation(ta);

Also, you can use STACK's rand_selection(L, n) to select $n$ different elements from the list $L$. Say you have the following list of wrong answers and you want to take only 3 out of 5.

ta2:[x^2,w^2,w^6,z^4,2*z^4];
ta2:rand_selection(ta2, 3);
/* Then, as before. */
ta2:maplist(lambda([ex],[ex, false]), ta2);

Another way to create an MCQ answer list is to have Maxima decide which of the answers are true. For example, in this question the student has to choose which of the answers are integers.

L:[1,4/2,3.0,2.7,1/4,%pi,10028];
/* Map the appropriate predicate to the list to create the true/false list. */
A:maplist(integerp,L);
/* Note the use of zip_with together with the list constructing function "[". */
ta:zip_with("[",L,A);
/* If you want to shuffle the responses then use the next line. */
ta:random_permutation(ta);

MCQ helper functions

STACK has helper functions to create MCQ arrays in Maxima.

[ta, variant] = multiselqn(corbase, numcor, wrongbase, numwrong)

This function takes two lists corbase and wrongbase and two integers numcor and numwrong. It randomly selects numcor from corbase, and numwrong from wrongbase and then creates the MCQ list ta with these selections, and an answernote variant.

The function returns a list with two arguments. The first argument of the list is the MCQ array, the second is just the list of answers which is useful for the answer note. Note, this function does use random_permutation internally to randomly order the random selections.

For example, the following generates random expressions for an MCQ calculus question. Note the use of ev(...) later to evaluate the derivative.

trg:rand([sin(p), cos(p)]);
dtrg:diff(trg, p);
wrongbase:[a*trg, 2*a*x*trg, -2*a*x*trg, ev(dtrg, p=2*a*x), 2*a*x*ev(dtrg, p=2*a*x)];
p:a*x^2+b;
wrongbase:ev(wrongbase); /* Now we have a value for p, the extra evaluation will use it. */
ans:diff(ev(trg), x);
multisel:multiselqn([ans], 1, wrongbase, 3);
ta:multisel[1];
variant:multisel[2];

In the above example there is only one correct answer, so we just select 1 from [ans]. This is fine, and we then choose three randomly generated wrong answers.

This returns (for example) the values

ta = [[-2*a*x*cos(a*x^2+b),false],[-sin(2*a*x),false],[a*cos(a*x^2+b),false],[-2*a*x*sin(a*x^2+b),true]];
variant = [-2*a*x*cos(a*x^2+b),-sin(2*a*x),a*cos(a*x^2+b),-2*a*x*sin(a*x^2+b)];

The following function does a similar job when we have MCQ display strings.

[ta, variant] = multiselqndisplay(corbase, numcor, wrongbase, numwrong)

For example, here the return values could be

ta = [[3,false,2*a*x*sin(a*x^2+b)],[2,false,a*sin(a*x^2+b)],[5,false,cos(2*a*x)],[1,true,2*a*x*cos(a*x^2+b)]]
variant = [3,2,5,1]

The function multiselqndisplay automatically assigns numbers $1,\cdots, k$ to the corbase entries, and then $k+1,\cdots, n$ to the wrongbase entries so that the numbers returned by the input type uniquely map to the entries in the two lists regardless of which random variant is generated. These numbers are useful internally, but not for students.

The function multiselqnalpha automatically selects the appropriate number of correct and incorrect entries, permutes the list and then assigns strings "(a)", "(b)", "(c)" etc. in the correct order. The student's answer will thus be a list of strings (somewhat subverting the whole purpose of STACK in returning a genuine mathematical expression, but this has its place...).

[ta, variant] = multiselqnalpha(corbase, numcor, wrongbase, numwrong, [dispstyle])

For example, here the return values (with inline maths) could be

ta = [["(a)",false,"<b>(a)</b> \\(-2\\,a\\,x\\,\\sin \\left( a\\,x^2+b \\right)\\)"],["(b)",true,"<b>(b)</b> \\(2\\,a\\,x\\,\\cos \\left( a\\,x^2+b \\right)\\)"],["(c)",false,"<b>(c)</b> \\(\\cos \\left( 2\\,a\\,x \\right)\\)"],["(d)",false,"<b>(d)</b> \\(2\\,a\\,x\\,\\cos \\left( 2\\,a\\,x \\right)\\)"]]
variant = [-2*a*x*sin(a*x^2+b),2*a*x*cos(a*x^2+b),cos(2*a*x),2*a*x*cos(2*a*x)]

Notes

1. The fifth optional argument to multiselqnalpha controls the display style of the LaTeX. Use "i" (a string containing i) for inline, "d" for displayed and "id" for inline "displaystyle" (which is the default).
2. Note that the "student's" answer will now be a string such as "(a)". When you construct the PRT you will not need to check against these strings, not the original CAS expressions.
3. The variant retains the CAS statements, in order.
4. Do not use multiselqnalpha in conjunction with the shuffle option. There is no need as the selection are permuted, and it messes up the order of the choices for the student.

Dealing with strings in MCQ

A likely situation is that a teacher wants to include a language string as one of the options for a student's answer in a multiple-choice question.

Recall: A fundamental design principal of STACK is that the student's answer should be a mathematical expression which can be manipulated by the CAS as a valid expression. Students are very limited in the keywords they are permitted to use in an input type. It is very likely that strings will contain keywords forbidden in student expressions.

One option to overcome this is to do something like this as one option in the teacher's response:

[C, false, "(C) None of the other options"]

The optional display part of this input is displayed to the student. Their answer is the (valid) CAS atom C which the PRT will deal with appropriately.

To construct appropriate arrays use the multiselqnalpha function.

The quotation marks will be removed from strings, and the strings will not be wrapped <code>...</code> tags or LaTeX mathematics environments.

Question authors should consider using the Moodle MCQ question type in addition to these facilities for purely text-based answers.

You must protect characters within strings. E.g. in

[A, true, "A. There exists \\(M\\) such that \\( |a_n| < M \\)."]

We have protected the backslash, and the inequality.

The language strings are not CAStext, they are simply raw strings. It is possible to construct strings which include variable values using the stack_disp function.

[oc(-inf,a), false, sconcat("The half interval: ", stack_disp(oc(-inf,a),"i"))]

The argument "i" here displays the expression "inline", other options are "" (you are responsible for maths environments), "d" (displayed), and "di" (inline but using displaystyle). If you construct strings in this way the display of any equations will not respect the display options in the particular input since variables are typically defined in the question variables and the input options are not available at that point in the code base.

For checkbox and radio inputs you could add modest HTML style to your strings. E.g.

ta:[[A, false, "(A) This is <b>bold</b>"], [B, true, "(B) this is <em>emphasis</em>"]];

Dropdown lists are created with the HTML <select> and <option> elements. There is no support for HTML (or LaTeX) within dropdown lists. You can use HTML-entities within a string field: see the separate documentation on this.

Inline CASText as MCQ labels

Since 4.4 it has been possible to write more complex labels using inline CASText. Inline CASText is basically a static string value wrapped in special function call and how one would use it is as follows:

/* The old way of constructing a string: */
[oc(-inf,a), false, sconcat("The half interval: ", stack_disp(oc(-inf,a),"i"))]
/* Same using a inline CASText */
[oc(-inf,a), false, castext("The half interval: {@oc(-inf,a)@}")]

You may write normal CASText syntax inside that string and it should behave exactly like it does in question-text or PRT feedback etc.. The only limitation at this time is that the list that includes these labels must be defined in the question-variables, you may not write inline CASText directly to the model answer field of the input. The castext()-function is not a real CAS-function it is converted to more complex logic at compile time and therefore it must receive a static string as its argument.

The most obvious use case for inline CASText is to provide localisation inside MCQ labels in situations where the mlang2-filter is not an option:

[true, true, castext("[[lang code='en']]Yes[[/lang]][[lang code='fi']]Kyllä[[/lang]]")]

Dealing with plots in MCQ

It is possible to use plots as the options in a STACK MCQ.

Recall again the MCQ are limited to legitimate CAS objects. The plot command returns a string which is the URL of the dynamically generated image on the server. The "value" of this can't be assessed by the potential response trees. For this reason you must use the display option with plots and must only put the plot command in the display option. (Otherwise STACK will throw an error: this behaviour could be improved). For example, to create a correct answer consisting of three plots consider the following in the question variables.

p1:plot(x,[x,-2,2],[y,-3,3])
p2:plot(x^2,[x,-2,2],[y,-3,3])
p3:plot(x^3,[x,-2,2],[y,-3,3])
ta:[[1,true,p1],[2,false,p2],[3,false,p3]]

The actual CAS value of the answer returned will be the respective integer selected (radio or dropdown) or list of integers (checkbox). The PRT can then be used to check the value of the integer (or list) as normal.

For this reason you will probably want to switch off the validation feedback ``your last answer was...".

Using a PRT is slight overkill, but it maintains the consistent internal design.

Dealing with external images in MCQ

It is also possible to embed the URL of an externally hosted image as the "display" field of an MCQ. The string is not checked, and is also passed through the CAS. This feature is fragile to being rejected as an invalid CAS object, and so is not recommended. (This could also be improved...)

For example, the question variables could be something like

i1:"<img src='http://www.maths.ed.ac.uk/~csangwin/Pics/z1.jpg' />"
i2:"<img src='http://www.maths.ed.ac.uk/~csangwin/Pics/z2.jpg' />"
i3:"<img src='http://www.maths.ed.ac.uk/~csangwin/Pics/z3.jpg' />"
ta:[[1,true,i1],[2,false,i2],[3,false,i3]]

Partial Marking for MCQs

Due to the wide variety of partially correct options in a typical MCQ, the usual method of creating a node in a PRT for each partially correct option and then traversing the tree is both brittle and unwieldy. To get around this, you can use the fact that the Score in a PRT can be defined by a Maxima value between 0 and 1 to dynamically generate the score.

The following example (written in the feedback variables of a PRT with student solution called ans1 and the MCQ called MCQ) checks if the student has checked any incorrect solutions, and if they have assigns them 0. If they haven't, it assigns them the fraction they got correct:

 incorrectInputs : sublist(ans1, lambda([x], member(x, mcq_incorrect(MCQ))));
 mark : if emptyp(incorrectInputs) then length(ans1) / length(mcq_correct(MCQ)) else 0;

In this example, the output value mark can be placed into the Score of the false route in the MCQ PRT to get the desired marking.

Writing question tests

Quality control of questions is important. See the notes on testing questions.

When entering test cases the question author must type in the CAS expression they expect to be the value of the student's answer (NOT the optional display field!). For example, if the teacher's answer (to a checkbox) question is the following.

 ta:[[x^2-1,true],[x^2+1,false],[(x-1)*(x+1),true],[(x-i)*(x+i),false]]

Then the following test case contains all the "true" answers.

 [x^2-1,(x-1)*(x+1)]

There is currently minimal checking that the string entered by the teacher corresponds to a valid choice in the input type. If your test case returns a blank result this is probably the problem.