CSC/ECE 517 Fall 2021 - E2148. Completion/Progress View
Problem Statement
In Expertiza, peer reviews are used as a metric to evaluate someone’s project. Once someone has peer-reviewed a project, the authors of the project can also provide feedback for this review, called “author feedback.” While grading peer reviews, it would be nice for the instructors to include the author's feedback, since it shows how helpful the peer review actually was to the author of the project. Currently, however, the instructor has no easy way of seeing the author-feedback scores, so it would be far too much trouble to include them in grades for reviewing. The aim of this project is to build this into the system.
Breaking down the problem statement into 3 subproblems:
1. A toggle button to view/hide average authors scores
The review report is already pretty crowded. The addition of a new column for average authors' feedback scores will make things clumsy. And often there is no author feedback, so it will be mostly wasted space. So, we are updating the UI in such a way that it's up to the instructor to view or hide the author's feedback. So, a toggle button at the top of the table will be added to view or hide the average author's feedback. When average authors' feedback is viewed the reviewer column will be squeezed, by using wrapping, in order to allocate space to the average authors' feedback column.
Old UI
LOFI UI sketch
2. Author's summary link
Right now, if the average author's feedback is too low, there is no way for the instructor to take a look at the author's comments. It's good if we can maintain an author's summary page where the instructor can take a look at the author's comments in order to come to a conclusion about the low average author's feedback. Sometimes even the reviewer's comments can be attacking the author's work. So if the author expresses such kinds of issues in the author's feedback instructor will be aware of how legitimate and polite the reviewer was if we can expose the author's comments somewhere in the system. The plan is to add this link in the reviews done column right after the summary link.
LOFI UI sketch
3. Austor's summary page
As discussed in the 2nd point, a new author's summary link will be provided on the review report page which will redirect the instructor to a new author's summary page. The author's summary page will have all the author's comments for a student who has reviewed the project. Each team author's scores will be rendered one by one.
LOFI UI sketch
Where to find review report UI?
When logged in as an instructor, click on Manage -> Assignments. You'll be able to see the list of all assignments available. For a particular assignment, click on the view reports icon (search image on the head of a person) which is just beside the view scores icon (start icon with search image). Then select Review report from the drop-down and click on the view button.
http://152.7.98.254:8084/reports/response_report?id=834 - temporary link to navigate to the review report UI page. Select Review report from the drop-down and click on the view button.
Scope
Though the project mentioned 5 points that need to be done as part of this project, there are a few features that are already built into the system.
Integration of review performance – Something that combines the author feedback data with the review data so instructors can grade reviewers based on:
1. # of reviews completed.
2. Length of reviews.
3. [Summary of reviews].
4. Whether reviewers added a file or link to their review.
5. The average ratings they received from the authors.
1. # of reviews completed. - This is already present in the current UI. Reviews Done column in review report page gives the number of reviews completed.
2. Length of the reviews. - This is already present in the current UI, but in the form of metrics. The metrics column on the review report page gives a visual representation of what's the average length of reviews given by the reviewer Vs what's the average length of reviews given by the class.
3. Summary of reviews. - In the reviews done column a summary link is available which will navigate the instructor to the summary page where the instructor can look at the summary of comments given by the reviewer.
The scope of the present project is just to focus on the last 2 points, i.e., the Instructor should be able to know whether reviewers added a file or link to their review and The average ratings reviewer received from the authors.
Design
Testing
UI testing will be done manually. We will use Rspec testing for the author feedback score functions used for computing author scores and averages.