Skip to content

Reporting

Reviewing students' answers closes the learning cycle for teachers by allowing them to understand what students are doing. Basic reporting is undertaken through the Moodle quiz. For example, lists of scores, etc. are available there. The Moodle quiz also calculates basic statistics based on the numerical information.

To review work in more detail we need to use two important parts of the question. Please read the following two entries before continuing with this article.

Moodle quiz features

Most of the day to day reporting is done through the Moodle quiz features.

For detailed reporting go to

Quiz administration -> Results -> Reporting

The "response" column shows the raw Maxima input value entered by the student and the validity status, e.g. "blank" (not reported), "invalid", "valid", "score". The "response" column also shows the value of the answer note for each potential response tree.

An example note is shown below. In this question a student has answered with (x-4)^6+c for the input ans1. The potential response tree prt1 has executed and the note is ATInt_generic. | 1-0-F.

ans1: (x-4)^6+c [score]; prt1: ATInt_generic. | 1-0-F

Notes:

  1. If the potential response tree does not execute it will be reported as prt1: # to indicate it exists but was not used.
  2. Reporting is stored in the database, so you may need to "regrade" attempts if you update the answer notes.
  3. You can download the responses for offline analysis. Split over ; to separate inputs and separate PRT answer notes. Split the individual answer note over | to get the route through the tree, and the notes added by the answer tests.

Individual STACK item analysis

STACK has a bespoke basic reporting mechanism for analysing responses to questions. To access the Basic question use report you need to navigate to the STACK question dashboard and choose "Analyze responses". The report currently contains all attempts at the question, regardless of how many different quizzes the question appears in.

The report makes use of the response summary. (For developers, this is generated by qtype_stack_question::summarise_response in question.php.)

The response summary is a string consisting of the following information.

  • The "seed" for the question used, enabling attempts to be grouped by question variant.
  • The Maxima string value of each input, and its status (blank, invalid, valid, score).
  • The answernote generated by each potential response tree. ! denotes a blank note, i.e. the PRT did not execute.

Here is a raw response summary.

 Seed: 254530683; ans1: 15180 [score]; ans2: 24360 [score]; prt1: # = 1 | prt1-1-T; prt2: # = 0 | prt2-1-F

Here we see the following.

  • The report can be split over ; into separate fields. Within the PRTs the data can be split over |.
  • The seed used in this example was 254530683.
  • There are two inputs, each at the "score" state.
  • ans1: 15180 [score]
  • ans2: 24360 [score]
  • There are two potential response trees
  • prt1: # = 1 | prt1-1-T The prt1 was activated, the score # returned was 1 and we see the answer note.
  • prt2: # = 0 | prt2-1-F The prt2 was activated, the score # returned was 0 and we see the answer note.

The response summary contains a lot of information in a condensed way, and can only be understood with an appreciation of the question structure itself in STACK (inputs, prts, variants) and the individual question.

The report splits this information into the following, text-based, reports.

  • Frequency of answer notes, for each PRT, regardless of which variant was used. Useful to know the route through the tree.
  • Frequency of answer notes, for each PRT, split by |, regardless of which variant was used. Useful to understand the outcome at each node.
  • Raw inputs, regardless of which variant was used.
  • Raw inputs and PRT answer notes by variant.
  • Raw data.

This feature is new in STACK 4.3. Comments and suggestions for improvements are very welcome.

Note, the question summary is stored in a database and will not be updated if you subsequently edit the question. If you wish to improve your question you will need to do a quiz "regrade" to update the basic question use report.

Site-wide usage

Moodle Admins can get stats on site-wide STACK usage.