Skip to content

Bulk testing STACK questions on your site

You can bulk test all question tests on all variants of all STACK questions by using the bulk-test script. There are two ways to bulk test.

  1. Bulk test all questions in a course or category. To do this, access STACK's "adminui" page (or the plugin setting page) [...]/question/type/stack/adminui/index.php
  2. Bulk test all questions in a particular quiz. Follow the link from the question dashboard to see quizzes in which that question is used.

To make use of the bulk test users require the capability qtype/stack:usediagnostictools via Moodle's capability system.

Bulk testing does the following.

  1. Check the STACK question for features which are no longer supported, such as the "mul" input option or the RegExp answer test, and suggest an alternative where there is one. A few checks only apply to questions older than a given version, so they use the question's stackversion number: the version of the STACK plug-in last used to edit the question (see the STACK version stamp).
  2. Run all question tests on all variants of each STACK question.
  3. Catch any run-time errors generated in the above.
  4. List questions with deficiencies:
  5. No question tests
  6. No deployed variants
  7. Missing general feedback (worked solution)

A report is generated with links to the question dashboard for each question.

Bulk test all STACK questions

Moodle site-admins can bulk test all STACK questions on a site. This is typically slow, and so is not available to general users. Indeed, we recommend this is done using the command line.

[...]/question/type/stack/cli/bulkrestall.php

On Linux consider using "screen" to run this bulk test in the background. Typically this process is done as part of upgrade acceptance testing.

Site admins will find a link "Run all the question tests for all the questions in the system (slow, admin only)" at the bottom of the bulk-test web index page. However, running all the tests on a large site is typically too slow.

Adding [[todo]] blocks

Running question tests is slow, and not all users have the capability qtype/stack:usediagnostictools. For this reason, it is useful to be able to "tag" questions requiring attention with [[todo]] blocks.

You can add in [[todo]] blocks to the question description listing problems as a one-off process.

  • For the web page run the bulk-test on the chosen context, then run the page again with the option &addtags=true in the URL. There is no web-form interface for this (advanced) option.
  • For the CLI use the flag --addtags.

The purpose of adding [[todo]] blocks is (1) to find questions more easily without running the whole bulk test again, and (2) to enable people without the capability qtype/stack:usediagnostictools to find questions which we know need work.

The [[todo]] blocks are added for broken questions, problems with upgrade, runtime errors, missing seeds, missing tests and failing question tests. The [[todo]] blocks are not added for empty worked solutions.

These [[todo]] blocks are only added once, but it does change the DB entry for each question. Use with care.

Any teacher with Moodle capability moodle/question:editall for a given context can find all questions with any [[todo]] blocks from STACK's "adminui" page.

[...]/question/type/stack/adminui/index.php

Bulk test materials on other sites

It is possible to bulk test materials on other sites.