CSC/ECE 517 Spring 2018- Project E1815: Improvements to review grader

Introduction

This project aims to enhance the review report and summary page, and display more information necessary for grading. The goal is to help the instructors by making the process of grading reviews easier and more accurate.

Problem Statement

Issue 1: Visualization of Metrics in Review Reports

There are some issues with the current review report page with regards to the Metrics column.

• Metrics are displayed as plain text.
• No comparison on how this student compares with other students.
• If more metrics are added, showing multiple lines of text for each metric can make the column cluttered and difficult to read.

Showing a bar chart for the metric rather than just text would improve the user experience. The bar chart can also depict the average for all students for this metric, so the grader can see how this student compares with other students. Also, if several metrics are shown in the column, the bar chart can be made resizable to accommodate multiple bars.

Issue 2: Updating Review Summary view

To view this page you click the summary in the row for any student.

This page summarizes the peer Reviews given to the Author. They are basically separated into three columns i.e reviewee, score and comments.

1. The first issue in this view is that there is no header saying what course, assignment, or student this relates to.
2. Secondly, The field that lists the team members is wider than the rest of the columns combined. The comments column needs more space it will contain more information
3. Finally, For questions answered with check-boxes, comments are not necessary, but still a blank cell is displayed.

Issue 3: Adding color schemes to identify updates to reviews as per submissions

Currently, in review report, each team reviewed is color coded as, red indicating that the review is not yet completed and blue indicating that the review grade is not assigned or updated. So if an instructor sees a name in red, the student should not be given credit for the review. If text is in blue, then the student has not been graded for the review, so the instructor should grade it now. But there is caveat here. If the reviewer has reviewed the work but the author did not resubmit, the reviewer has nothing new to review in the latest round and should not be downgraded for not re-reviewing the work. This issue needs to be addressed.

This is what the current color scheme looks like. The text in blue indicates that the student has not been graded, while the text in red indicates the student should not be given credits.

Files Changed

• app/assets/stylesheets/redbox.scss
• app/helpers/review_mapping_helper.rb
• app/views/review_mapping/_review_report.html.erb
• app/views/popup/view_review_scores_popup.html.erb
• config/application.rb

Solutions Implemented

Issue 1: Visualization of Metrics in Review Reports

We replaced the text given in the Metrics column with a bar chart, as shown below. The bar chart depicts the overall average of all students for this metric, the average of this student for this metric (based on each round) and the metric values for each round for this student.

Code Changes

• File: app/helpers/review_mapping_helper.rb

The following methods were added to calculate average for this metric for all students and for calculating maximum value for this metric. The maximum is needed for constructing bar chart. The second method was added to construct the x-axis labels in the format "0|50|100|...|max" for the bar chart.

• File: app/helpers/review_mapping_helper.rb

The following method constructs the bar chart.

• File: app/views/review_mapping/_review_report.html.erb

The following code is added to get the average metric, maximum value of metric and x-axis value from controller.

• File: app/views/review_mapping/_review_report.html.erb

The following code includes the bar chart in the review report page.

Screenshots

Issue 2: Updating Review Summary view

We will start of by adding a title to the review summary, this summary will have the information about the reviewer. we will span the question text to all three columns instead of it being one, also the width of the first column containing the reviewee information will be reduced appropriately. Checkbox questions could be displayed more compactly by showing the question on the left, and then having separate column rather a than separate row for all the reviews. If the reviewer had checked a box for a particular review, the box would be checked, otherwise not.

Issue 3

To address the caveat mentioned above, we could use a different color, perhaps green, to distinguish reviewers who have nothing new to review as the authors have not made any changes. This is what we are planning to implement.

alt text

Test Plan

Issue 1: Visualization of Metrics in Review Reports

1) Login as an instructor

2) Navigate to review grader by doing this (Manage > Assignments)

3) Click on the “View review report” icon (the one with the spyglass and two people)

4) Check if the visualizations are appearing in the review report.

Issue 2: Updates and Review Summary View

1) Login as an instructor

2) Navigate to review grader by doing this (Manage > Assignments)

3) Click on the “View review report” icon (the one with the spyglass and two people)

4) In the review report click on the “summary” in the row for any student.

5) Check if the changes are appearing in the page as mentioned in the proposed solution.

Issue 3: Adding Color Schemes to Identify Updates to Review As Per Submission

1) Login as an instructor

2) Navigate to review grader by doing this (Manage > Assignments)

3) Click on the “View review report” icon (the one with the spyglass and two people)

References

Expertiza

• https://expertiza.ncsu.edu/

Expertiza Github

• https://github.com/expertiza/expertiza

Expertiza Documentation

• http://wiki.expertiza.ncsu.edu/index.php/Expertiza_documentation