CSC/ECE 517 Fall 2020 - E2078. Improve self-review Link peer review & self-review to derive grades

Project Overview

In Expertiza, it is currently possible to check the “Allow self-review” box on the Review Strategy tab of assignment creation, and then an author will be asked to review his/her own submission in addition to the submissions of others. But as currently implemented, nothing is done with the scores on these self-reviews.

There has been a previous attempt at solving this problem, but there were several issues with that implementation:

• The formula for weighting self-reviews is not modular. It needs to be, since different instructors may want to use different formulas, so several should be supported.
• There are not enough comments in the code.
• It seems to work for only one round of review.

View documentation for previous implementation here.

Objectives

Our objectives for this project are the following:

• Display the self-review score with peer-review scores for the logged in user
• Implement a way to achieve a composite score with the combination of the self-review score and peer-review scores
• Implement a requirement for the logged in user to self-review before viewing peer-reviews
• Assure that we overcome the issues outlined for the previous implementation of this project

Team

Courtney Ripoll (ctripoll)

Jonathan Nguyen (jhnguye4)

Justin Kirschner (jkirsch)

Tasks

Display Self-Review Scores w/ Peer-Reviews

It should be possible to see self-review scores juxtaposed with peer-review scores. Design a way to show them in the regular "View Scores" page and the alternate (heat-map) view. They should be shown amidst the other reviews, but in a way that highlights them as being a different kind of review.

Design Plan

In the current implementation of Expertiza, students can view a compilation of all peer review scores for each review question and an average of those peer-reviews. For our project, we plan to add the self-review score alongside the peer-review scores for each review question. In the wire-frame below, note that for each criterion there is a column for each peer-review score and a single column for the self-review score. The avg column then takes an average of all review scores (peer-review and self-review). Additionally, the composite score is displayed on the page under the average peer review score. How we plan to derive a composite score is explained in detail in the follow section Derive Composite Score.

In the alternate view, our plan does not alter the current interface too much. The only addition we plan to implement is an additional column describing the composite score a student has received for an assignment. Likewise, there is now an additional doughnut chart providing a visual of the composite score (green) alongside the final review score (yellow).

Derive Composite Score

Implement a way to combine self-review and peer-review scores to derive a composite score. The basic idea is that the authors get more points as their self-reviews get closer to the scores given by the peer reviewers. So the function should take the scores given by peers to a particular rubric criterion and the score given by the user. The result of the formula should be displayed in a conspicuous page on the score view.

Implementation Plan

Implement Requirement to Review Self before Viewing Peer Reviews

There would be no challenge in giving the same self-review scores as the peer reviewers gave if the authors could see peer-review scores before they submitted their self-reviews. The user should be required to submit their self-evaluation(s) before seeing the results of their peer evaluations.

Implementation Plan

Relevant Links

Our repository: https://github.com/jhnguye4/expertiza/tree/beta

Pull request: Does not exist yet

Video demo: Does not exist yet

References

Scoring models for peer assessment in team-based learning projects

E1984 Pull Request