Some assignments involve multiple assessors, for example, in the following situations:
- Grading according to the four-eye principle (e.g., graduation). Here the first and second assessor first assess independently and then come to a joint evaluation.
- The grading criteria of one rubric are distributed among multiple assessors (e.g., by different areas of expertise)
Read how to set up an Assignment involving multiple assessors here.
Grading an Assignment using the four-eye principle
When grading according to the four-eye principle, two assessors must have the opportunity to each complete their own assessment first and then complete a joint assessment. The individually completed assessments of the first and second assessor are not shown to the student. Only the joint assessment can be viewed by the student after publication.
There are two ways to grade according to this four-eye principle:
- With the multiple assessor assessment options. In this, each assessor gets their own tab containing the individual assessment rubric and there is one joint tab.
- Using three rubrics linked to one Assignment (first assessor, second assessor, joint assessment).
To determine how the assignment is set up you click (as in grading) on Grades. There, in the Assignment column for the student you want to grade, click on the icon indicating that work has been submitted.
In the case of the first option, you will see that the names of the assessors directly above the rubrics (with the 'Aggregated' tab last).
In the case of the second option, there are three rubrics associated with the Assignment:
Grading according to the four-eye principle through the multiple assessor grading options
First, make sure the assignment is set up properly. After the work is turned in, you can grade it as follows:
- Navigate to Grades and there, in the Assignment column under the student you want to grade, click on the icon indicating that work has been submitted.
- On the right side you will now see a separate tab at the top for each assessor.
- On the far right you will see the 'Aggregate' tab where the aggregate assessment can be entered later.
- Complete the evaluation in the usual way and click Ready to Aggregate when you are done. If necessary, you can still make adjustments afterwards and update your evaluation via 'Update evaluation'.
- Return at the top left via Back to Submissions or click Next at the top or bottom right to assess the next student.
Once all assessors have filled-out their own rubric, the 'Aggregate' tab can be filled-out.
- Navigate again to Grades and there, in the Assignment column by the student you wish to grade, click on the icon indicating that work has been submitted. You are now automatically taken to the Aggregate tab. This otherwise looks the same as the other tabs.
- Enter the aggregate assessment here and click Save Draft.
Depending on the agreements made within the academy/program about who publishes the grades, the person who may publish the grade will click Publish.
If you accidentally clicked Publish or want to retract the publication of the grade for some other reason, click Retract. The student will have received notification of the publication, but will not be able to retrieve the grade until you click Publish again.
Grade according to the four-eye principle with three linked rubrics
First, make sure the assignment is set up properly. After the work is turned in, you can grade it as follows:
- Navigate to Grades and there, in the Assignment column for the student you want to grade, click on the icon indicating that work has been submitted.
- On the right side, you will see that there are three rubrics linked: one for the first assessor, one for the second assessor and one for the joint assessment.
If the rubrics for the first and second assessors are properly set then they remain hidden from students.
- Under 'Grading Rubric' (directly below the three rubrics), the rubric for joint grading is selected. That means this rubric determines the student's grade.
For grading, you have the usual two options:- You click on the little arrow behind the rubric for the first or second assessor (depending on your role) to open it.
- You click on the little square with the arrow located at the top right of the rubrics to open the rubrics in a separate window. In it, all three rubrics are displayed one below the other.
- Individually complete the rubric.
If you opened the rubric in a separate window you will see all three rubrics listed below each other. You fill in only the individual rubric that fits your role (first or second assessor). Once this has been completed, click Close at the bottom to close the window. You will see that your changes have been adopted when you click the arrow behind the appropriate rubric. - Click Save Draft to save your assessment.
If you accidentally click Publish, then click Retract to immediately withdraw the published grade. The student will have received notification of the publication but will not be able to retrieve the assessment until you click Publish again.
Once both assessors have each completed their own rubric, they can complete the joint rubric together in the same way.
- Again go to Course Tools > Assignments, click on the Assignment you want to assess and click Go To Evaluation by the student you want to grade.
- Fill out the joint rubric.
Again, this may be possible by opening the rubric in a separate window. In that window, the three linked rubrics are below each other, so be sure to fill in the joint rubric now. - Click Save Draft to save the evaluation.
- Depending on the agreements made within the academy/program about who publishes the grades, the person who can publish the grade clicks Publish.
Who is allowed to do this is set at the Assignment.
If you accidentally clicked Publish or want to retract the publication of the grade for some other reason, click Retract. The student will have received notification of the publication but will not be able to retrieve the assessment until you click Publish again.
Using multiple assessors to assess different criteria from the same rubric
For some Assignments, some criteria of the rubric are graded by one assessor and some criteria are graded by one or more other assessors.
This can be done in Brightspace in two ways:
- Through advanced options to simultaneously grade the same rubric (Currently contains a bug - this is explained below).
- Through coordinating who grades which students when, thus preventing both assessors from grading the same student at the same time (because then either evaluation will be lost entirely)
In the first case you will see names of assessors above the rubric (2) and in the second case you will not.
These may be more than two assessors, even if only two assessors end up completing the rubric. So not every assessor shown is required to complete the rubric.
Note that there is a difference with grading an assignment according to the four-eye principle.
With multiple assessors simultaneously grading different criteria of the same rubric
First, make sure the assignment is set up properly. After the work is turned in, you can grade it as follows:
- Go to Course Tools > Assignment and click on the assignment you want to grade.
- For the student you want to grade, click Go to Evaluation.
You now see who can grade. Only the assessors whose names are shown here can grade this Assignment. If an assessor is missing, they must be added to the Assignment first.
- Complete the rubric in the usual way.
It is not set who can grade which criteria. So please coordinate this with the other assessors.
- Click Save Draft to save your evaluation.
If you accidentally click Publish, then click Retract to immediately withdraw the published grade. The student will have received notification of the publication but will not be able to retrieve the assessment until you click Publish again. - Once all assessors have completed their criteria, the rubric can be published by clicking Publish.
Note! Currently, there is a bug in the first option that causes more sensitivity to error: Namely, if you actually grade the same student with another assessor at the same time, the total score only includes the score of the assessor who last saved the rubric that was filled-in. This is not automatically corrected when reopening, saving or publishing. All feedback and scores per criterion from both assessors are recorded though.
There is a workaround: If you open the filled-in rubric again, uncheck one of the criteria scores and then check it again, the total score is corrected. However, this can easily be forgotten and then the wrong score is published to the student. If the student or the assessor discovers the error AFTER publication it can still be corrected in the same way and then clicking Update.
To avoid errors as much as possible you could also choose for now, to coordinate who checks which students and when.
With multiple assessors, grade different criteria of the same rubric in succession
In an Assignment with multiple assessors, are you going to grade different criteria from the same rubric and you see no assessors above the rubric in the grading window? Then you can assess the same student only AFTER each other.
Agree among the other assessors who is grading which students when to avoid the possibility that you and another assessor are simultaneously completing the same rubric for the same student.
This is because if you are working on the same assignment and rubric for the same student at the same time, only the last saved evaluation and feedback will be saved. Also, there is no warning that another assessor has already opened the rubric.
Setting up the assignment and grading is done in the usual way.
After one teacher has graded, they save this completed rubric as a Draft and then the next can proceed with it. How you inform each other about this you have to decide among yourselves. There is no function for this in Brightspace.
When all criteria have been graded the result can be published by clicking Publish.
If you accidentally click Publish, then click Retract to immediately withdraw the published grade. The student will have received notification of the publication but will not be able to retrieve the assessment until you click Publish again.