Blind quality monitoring (BQM) is the technique of monitoring Marker consistency by randomly presenting a Marker with a candidate response that was standardised during S1, and then monitoring any error/discrepancy.

You can view how a Marker is tracking against the monitoring tolerances and compare these with the team average. Markers will be unaware that they are being monitored.

BQM can use either random allocation or stratified monitoring.

Random allocation

1. Select the Edit icon next to the Blind Quality Monitoring expandable section.
   
   

2. Complete the following:

   1. Select the Use monitoring scripts for blind quality monitoring option.
   2. Select Recycle monitoring scripts (optional) if you would like to reuse the monitoring scripts in future marking.
   3. Select the Randomly allocated option.
   4. Enter a percentage number in the Sample rate field. This should be set to the number of monitoring scripts divided by the number of responses markers are expected to mark per question x 100.
   

3. For each question, complete the following:

   1. Enter a number in the RMS Tolerance field. RMS value is the root mean square tolerance allowed – this is an objective measure of the average deviation from the correct score.
   2. Enter a number in the Absolute Tolerance fields for both above (+) and below (-) the correct score.
      

      Tip

      The values entered here will automatically populate across the grid, but you can change them if required.

   
4. Select the notification settings to determine how to handle Markers giving scores that exceed the tolerance levels. There are two options:
   

   Option	Description
   Send the notification to SM’s or KP’s	This sends a notification to Senior Markers or key personnel alerting them to a deviation outside of the tolerance level set.
   Suspend the marker	This suspends the Marker from marking.

   A tolerance level will need to be set in the No. of out-of-tolerance scripts before action is taken field. When the Marker exceeds this tolerance (eg. 2 scripts) the system will take the specified action.

   
5. Select the Save button to save the changes.

Stratified monitoring

Stratified monitoring is an extension of BQM and allows you to break the sequence of the monitoring scripts into stages and blocks. The placement of specific monitoring scripts can be set in different stages. You have an option to manually allocate monitoring scripts to your preferred location.

To set stratified monitoring:

1. Select the Edit icon next to the Blind Quality Monitoring expandable section.
   
   

2. Complete the following:

   1. Select the Use monitoring scripts for blind quality monitoring option.
   2. Select Recycle monitoring scripts (optional) if you would like to reuse the monitoring scripts in future marking.
   3. Select the Stratified monitoring option.
   
3. Configure the General area first by completing the following for each question:
   
   1. Enter a number in the RMS Tolerance field. RMS value is the root mean square tolerance allowed – this is an objective measure of the average deviation from the correct score.
   2. Enter a number in the Absolute Tolerance fields for both above (+) and below (-) the correct score.
      

      Tip

      The values entered here will automatically populate across the grid, but you can change them if required.

   3. Select the notification settings to determine how to handle Markers giving scores that exceed the tolerance levels. There are two options:
      

      Option	Description
      Send the notification to SM’s or KP’s	This sends a notification to Senior Markers or key personnel alerting them to a deviation outside of the tolerance level set.
      Suspend the marker	This suspends the Marker from marking.

      A tolerance level will need to be set in the No. of out-of-tolerance scripts before action is taken field. When the Marker exceeds this tolerance (eg. 2 scripts) the system will take the specified action.

   
4. Select the Next button to move to the next area – Stages & Blocks. In this next area you will determine the number of stages and blocks as the basis for generating the stratified sequence.
5. Configure the Stages & Blocks area by completing the following:
   
   1. Enter a number in the No. of Blocks field.
   2. Enter a number in the No. of Live Scripts (in each BLOCK) field.
   3. Enter a number in the No. of Monitoring Scripts (in each BLOCK) field.
   4. Select the Add Stage button (optional) if you would like to add another stage.
   5. Select the Copy button (optional) to copy the stages created in the first question to all other questions or select each question and repeat these steps.
      

      Tip

      The values entered here will automatically populate across the grid, but you can change them if required.

   
6. Select the Next button to move to the next area – Stratified Sequence. In this next area you can view the stages and blocks you configured in the previous step. You can now configure the stratified sequence, which can be random, manual or a combination of the two.
   

   Important

   Standardisation 1 must be completed before you can configure these settings.

7. Configure the Stratified Sequence area by completing the following:

   1. Select the drop-down arrow of the Generate All button and choose one or more options (optional) as follows:

      Option	Description
      Allocate monitoring scripts	This will allocate specific scripts to the cells.
      Repeat sequence for all items	If this option is selected, you won’t need to generate scripts for each question.

   2. Select the Generate All button. Scripts will be allocated as can be seen in the following image.
   3. Select Generate (optional). If you selected General All in the previous step, you won’t need to do this.
   4. Select Copy (optional) to copy this to the other questions. If you had Repeat sequence for all items selected in step a, you won’t need to do this.
   

   Important

   You can hover over the cells to view more information.

   

Manage manually allocated scripts

For manually allocated scripts, there are four actions available:

• Change the position of this monitoring script
• Allocate a random monitoring script at this position
• View the monitoring script allocated at this position
• Remove this monitoring script allocation

The first step is the same for all four actions. That is, select a manually allocated monitoring script.

Change the position of this monitoring script

Select the Change the position of this monitoring script option, then select the Continue button.

Select the required values in the Stage, Block and Index fields, then select the Save button.

The script displays in the new position.

Allocate a random monitoring script at this position

Select the Allocate a random monitoring script at this position option, then select the Continue button.

The cell will change from a manually allocated monitoring script to a randomly allocated monitoring script.

View the monitoring script allocated at this position

Select the View the monitoring script allocated at this position option, then select the Continue button.

The script will appear in a popup, where you can view the standardisation scores.

Remove this monitoring script allocation

Select the Remove the monitoring script allocation option, then select the Continue button.

The cell will change from a manually allocated monitoring script to a live script.

Manage randomly allocated scripts

There are three actions available for randomly allocated scripts:

• Change the position of this monitoring script
• Allocate a specific monitoring script at this position
• Remove this monitoring script allocation

The first step is the same for all three actions. That is, select a randomly allocated monitoring script.

Change the position of this monitoring script

Select the Change the position of this monitoring script option, then select the Continue button.

Select the required values in the Stage, Block and Index fields, then select the Save button.

The script displays in the new position.

Allocate a specific monitoring script at this position

Select the Allocate a random monitoring script at this position option, then select the Continue button.

Use the arrows to scroll through the scripts until you identify the required one (if applicable), then select the Allocate button.

The cell will change from a randomly allocated monitoring script to a manually allocated monitoring script.

Remove this monitoring script allocation

Select the Remove this monitoring script allocation option, then select the Continue button.

The cell will change from a randomly allocated monitoring script to a live script.

Manage live scripts

There are two actions available for live scripts:

• Allocate a random monitoring script at this position
• Allocate a specific monitoring script at this position

The first step is the same for both actions. That is, select a live script.