Skip to content

Deploying

Questions in STACK can contain randomly generated elements. A student will be given a random variant of a question generated by a pseudo-random integer seed. The seed is stored as part of the attempt, indexing which variant a student was given.

There is (potentially) a many-to-one mapping of seeds onto variants. Therefore, STACK considers question variants the same if and only if the (teacher-generated) question note is the same. Teachers must create meaningful question notes.

If questions are randomly generated on the fly then errors can, and do result! E.g. impossible random variants. Instead, the teacher is strongly advised to pre-generate, test, and "deploy" variants of a question. When a student attempts the question they will be given a random selection from the deployed variants. The pre-generated variant can be tested in advance.

  1. STACK runs all the question tests on each deployed variant. By using question tests it is unlikely a student will be given a random variant which does not work correctly (within the test coverage).
  2. The teacher can exercise judgement as to whether each deployed variant appears to be of equal difficulty. The teacher can easily delete variants they do not like. Experience suggests there are unanticipated consequences of randomly generating questions. Really, only statistics generated by real use can establish fairness amongst variants.
  3. The pre-generation combined with a cache helps minimise server load during the start of a large class which aids robustness of the whole experience. This helps STACK to optimize its use of the CAS for efficiency and reliability via a dynamic cache.
  4. Teachers are discouraged from creating variants with the same question note. This is to help generate meaningful statistics, grouped by note.

Notes:

  1. If an author does not deploy any variants (not advised!) then the student gets any random variant.
  2. STACK automatically detects randomization. Questions that don't use randomization cannot be deployed explicitly.

Once a quiz is underway it is still possible to edit a question, and to re-grade students' attempts. This is useful in rare cases where there is a mistake, you want to improve the worked solution, you would like to add better feedback/partial credit for a particular etc. However, do not change anything related to the random generation of questions! Results are unpredictable, and may well result in a situation when the modified question is different to that answered by students taking the test prior to modifications.

How to deploy question variants

The deployment, and variant management, interface can be found on the STACK question dashboard, under the "Variants" tab.

Typically users want to "Deploy # of variants:". The system will give up if too many duplicate question notes are generated, tests fail (or runtime errors are generated).

When you deploy new variants STACK will run all the question tests. If a test fails, the generation process will stop with an error message, showing the failing test. You do have the option to deploy questions which fail, but that rather defeats the purpose of testing and deploying!

The deployment management also allows specific variants to be dropped.

You can also return to the question preview window and try a specific deployed variant.

Systematic and matched variants

It is possible to systematically deploy variants with particular seeds, either a range of values or a list of seed values.

  1. It is possible to deploy all variants systematically.
  2. It is possible to match variants between separate questions.

These are advanced features.