CSC/ECE 517 Spring 2018- Project E1815: Improvements to review grader

Introduction

This project aims to enhance the review report and summary page, and display more information necessary for grading. The goal is to help the instructors by making the process of grading reviews easier and more accurate.

Problem Statement

Issue 1: Visualization of Metrics in Review Reports

There are some issues with the current review report page with regards to the Metrics column.

• Metrics are displayed as plain text.
• No comparison on how this student compares with other students.
• If more metrics are added, showing multiple lines of text for each metric can make the column cluttered and difficult to read.

Showing a bar chart for the metric rather than just text would improve the user experience. The bar chart can also depict the average for all students for this metric, so the grader can see how this student compares with other students. Also, if several metrics are shown in the column, the bar chart can be made resizable to accommodate multiple bars.

Issue 2: Updating Review Summary view

To view this page you click the summary in the row for any student.

This page summarizes the peer Reviews given to the Author. They are basically separated into three columns i.e reviewee, score and comments.

1. The first issue in this view is that there is no header saying what course, assignment, or student this relates to.
2. Secondly, The field that lists the team members is wider than the rest of the columns combined. The comments column needs more space it will contain more information
3. Finally, For questions answered with check-boxes, comments are not necessary, but still a blank cell is displayed.

Issue 3: Adding color schemes to identify updates to reviews as per submissions

Currently, in review report, each team reviewed is color coded as, red indicating that the review is not yet completed and blue indicating that the review grade is not assigned or updated. So if an instructor sees a name in red, the student should not be given credit for the review. If text is in blue, then the student has not been graded for the review, so the instructor should grade it now. But there is caveat here. If the reviewer has reviewed the work but the author did not resubmit, the reviewer has nothing new to review in the latest round and should not be downgraded for not re-reviewing the work. This issue needs to be addressed.

This is what the current color scheme looks like. The text in blue indicates that the student has not been graded, while the text in red indicates the student should not be given credits.

Files Changed

• app/assets/stylesheets/redbox.scss
• app/helpers/review_mapping_helper.rb
• app/views/review_mapping/_review_report.html.erb
• app/views/popup/view_review_scores_popup.html.erb
• config/application.rb

Solutions Implemented

Issue 1: Visualization of Metrics in Review Reports

We replaced the text given in the Metrics column with a bar chart, as shown below. The bar chart depicts the overall average of all students for this metric, the average of this student for this metric (based on each round) and the metric values for each round for this student.

Code Changes

• File: app/helpers/review_mapping_helper.rb

The following methods were added to calculate average for this metric for all students and for calculating maximum value for this metric. The maximum is needed for constructing bar chart. The second method was added to construct the x-axis labels in the format "0|50|100|...|max" for the bar chart.

• File: app/helpers/review_mapping_helper.rb

The following method constructs the bar chart.

• File: app/views/review_mapping/_review_report.html.erb

The following code is added to get the average metric, maximum value of metric and x-axis value from controller.

• File: app/views/review_mapping/_review_report.html.erb

The following code includes the bar chart in the review report page.

Screenshots

Issue 2: Updating Review Summary view

We have spanned the question text to all three columns instead of it being one, also the width of the first column containing the reviewee information is reduced appropriately. Checkbox Questions are displayed more compactly by including Question to the left and automatically adjusting the width of the reviews to right. A green tick is added if the reviewer selected the checkbox and a red cross mark is added if the reviewer did not select the checkbox.

The file modified is app/views/popup/view_review_scores_popup.html.erb

In the code, we identified whether a question is Checkbox question or not and changed the row to include the question on the left and auto adjusting the response from the reviewer's on right.

The page looks as follows

Issue 3

To address the caveat mentioned above, we could use a different color, perhaps green, to distinguish reviewers who have nothing new to review as the authors have not made any changes. This is what we are planning to implement.

For doing this, we are getting the last modified date/time of the wiki links and comparing it with the last review submission deadline. If changes the last modified date/time of the wiki links is before the last review submission deadline, there is nothing new to review and hence green should be used instead of blue to display the author's name. To ensure this check we have added the following function to the review_mapping_helper.rb file,

def is_submission_updated_for_last_round(response_map)

   last_round = 0
   assignment_deadlines = DueDate.where(parent_id: response_map.reviewed_object_id)
   number_of_assignemnt_rounds = @assignment.num_review_rounds
   current_time = Time.new
   r = number_of_assignemnt_rounds..1
   due_time = DueDate.new
   (r.first).downto(r.last).each do |round|
     due_time = assignment_deadlines.where(round: round, deadline_type_id: 1).try(:first).try(:due_at)
     if(current_time > due_time)
       last_round = round
       break
     end
   end
   if(last_round > 1)
     previous_round = last_round-1
     prev_due_time=assignment_deadlines.where(round: previous_round, deadline_type_id: 1).try(:first).try(:due_at)
     team = Team.find(response_map.reviewee_id)
     hyperlinks_count = team.hyperlinks.size
     testable_links_count = 0
     if !team.hyperlinks.empty?
       #flag is false when none of the wiki links are updated
       flag = false
       team.hyperlinks.each do |link|
         if ((link.include? "wiki") && !flag)
           testable_links_count = testable_links_count + 1
           last_modified = open(link) do |f|
             if(!(f.last_modified < prev_due_time))
               flag= true
               break
           end
         end
       end
     end
     end
     if(flag == false && hyperlinks_count == testable_links_count)
       return false
     else
       return true
     end
   else
     return true
   end
 end

Test Plan

Issue 1: Visualization of Metrics in Review Reports

1) Login as an instructor

2) Navigate to review grader by doing this (Manage > Assignments)

3) Click on the “View review report” icon (the one with the spyglass and two people)

4) Check if the visualizations are appearing in the review report.

Issue 2: Updates and Review Summary View

1) Login as an instructor

2) Navigate to review grader by doing this (Manage > Assignments)

3) Click on the “View review report” icon (the one with the spyglass and two people)

4) In the review report click on the “summary” in the row for any student.

5) Check if the changes are appearing in the page as mentioned in the proposed solution.

Issue 3: Adding Color Schemes to Identify Updates to Review As Per Submission

1) Login as an instructor

2) Navigate to review grader by doing this (Manage > Assignments)

3) Click on the “View review report” icon (the one with the spyglass and two people)

References

Expertiza

• https://expertiza.ncsu.edu/

Expertiza Github

• https://github.com/expertiza/expertiza

Expertiza Documentation

• http://wiki.expertiza.ncsu.edu/index.php/Expertiza_documentation