CSC/ECE 517 Spring 2020/E2020 Let course staff as well as students do reviews

Introduction

About Expertiza

Expertiza is an open-source project that is built using the Ruby on Rails framework. At its core, it is a highly versatile tool that academic institutions can use to craft learning objectives centered around team projects, peer reviews, and highly-customizable assignments.

Purpose and Problem

The purpose of this project is to allow instructors or teaching assistants who are participants of an assignment to be able to review (before the last due date of the specific assignment) any team that has submitted the assignment. Beyond this general endeavor, there is also the condition that team members of the assignment should be able to view reviews done by the course staff that can be identified in an aesthetically pleasing manner. As an added feature, reviewers should see how their reviews differ from those done by the course staff.

There are two main issues (and a third as an extra feature):

1. Allow the instructor and teaching assistants to review teams that have submitted assignments.

2. Allow authors to visually identify reviews done by the course staff.

3. Allow the reviewers to see how their reviews differ from those done by the course staff.

Design

Proposed Solution

Problem 1:

• Check will be done to verify whether the course staff is a participant. Staff can only review if they are a participant of the assignment.
• In the list_submissions.html.erb file, have a "Perform Review" link present if the due date of the assignment has not passed yet.
• If a review was already saved and the final review due date of the assignment has not passed, then a link to "Edit Review" will be present. If a review was already submitted and the due date has not passed, then a link to "View Review" will be present.
• The already present "Assign Grade" link will only appear after the final review due date of the assignment has passed.
• Clicking on the "Perform Review", "Edit Review", and "View Review" links will take the instructor/TA to the pages governed by response_controller#new, response_controller#edit, and response_controller#view respectively.

Problem 2:

• A star icon will be used to distinguish whether a review was done by a course staff and will appear below the 'Review #' heading.
• Add helper class grades_helper.rb to handle check to determine whether review was done by course staff based on the boolean flag attached to a response.
• Change will be made in view_team.html.erb such that after verifying that a response was done by a course staff via identity_helper.rb, a star is displayed below the 'Review #' heading.

Problem 3:

• Add the Criterion review type by expanding the current conditional statements targeting Checkbox to include Criterion in show_calibration_results_for_student.html.erb.

Relevant Files

• views/assignments/list_submissions.html.erb
• views/reports/calibration_report.html.erb
• views/response/show_calibration_results_for_student.html.erb
• views/response/response.html.erb
• views/grades/view_team.html.erb
• views/grades/view_my_scores.html.erb
• controllers/response_controller.rb
• app/helpers/grades_helper.rb

Flow Chart

This flow chart demonstrates the desired functionality. While students must submit reviews in the review stage, instructors/TA's can submit reviews any time before the final deadline. In order to submit reviews, the instructor/TA must be added as a participant to the assignment. Then, they will go to "View Submissions" on the Manage Assignments page. If the final deadline for the assignment has passed, they will see an "Assign Grade" link next to each submission. Otherwise, they will see a "Perform Review" link if a review has not been submitted yet or an "Edit Review" link if a review has been submitted already. Students can view only the reviews they have performed and the reviews of their work, while instructors/TA's can see all reviews for the assignment. Reviews done by course staff will be marked with a star to distinguish them from students' reviews.

Database Design

This diagram represents the relationship between different models involved with our proposed solution. We do not anticipate any changes to the database schema.

A model that's central to our proposed solution is the AssignmentQuestionnaire model defined in assignment_questionnaire.rb. It holds the user ID of the User (user.rb) who reviewed the assignment, from which we will determine the user's role: Instructor, TA, or student. It also contains the ID of the Assignment (assignment.rb) being reviewed. Additionally, the Questionnaire (questionnaire.rb) can be located by its ID defined in AssignmentQuestionnaire and consists of the Questions (question.rb) users use to review the assignment.

Implementation Details

Problem 2: Allow authors to visually identify reviews done by the course staff

Changed files:

app/helpers/grades_helper.rb

   # used to determine whether a review was done by a course staff
   # course staff is defined as an Instructor or Teaching Assistant
   def review_done_by_course_staff?(review)
     if review.nil?
       false
     else
       role = Role.find(User.find(Participant.find(ResponseMap.find(Response.find(review.id).map_id).reviewer_id).user_id).role_id).name
       role == ("Instructor" || "Teaching Assistant") ? true : false
     end
   end

app/views/grades/view_team.html.erb

    <!-- instructors (or higher level users) see reviewer name, students see anonymized string -->
    <!-- instructor reviews labeled with a black star under the Review heading -->
    <% if (['Student'].include? @current_role_name) && (@assignment.is_anonymous) %>
        <% review_name = "Review " + i.to_s  %>
    <% else %>
        <% review_name = User.find(Participant.find(ResponseMap.find(Response.find(review.id).map_id).reviewer_id).user_id).fullname(session[:ip]).to_s %>
    <% end %>
    <th class="sorter-false"> <a target="_blank" href="../response/view?id=<%= review.id.to_s %>"  data-toggle="tooltip" data-placement="right" title="Click to see details"><%= review_name %></a>
    <% if review_done_by_course_staff?(review) %>
        <br><span class="glyphicon glyphicon-star" style="margin:auto; display:table"></span>
    <% end %>

Pull request can be found here

Testing Plan

Testing from the UI

• Add an instructor as a participant in the assignment.

1. After logging in as the instructor, go to Manage > Assignments.
2. Find the test assignment and click the icon to add participants.
3. Add the instructor as a participant on the assignment.

• Instructor should be able to add a review on an assignment.

1. After logging in as the instructor, go to Manage > Assignments.
2. Find the test assignment and click on the View Submissions icon.
3. Verify a list of submissions appears.
4. If the last due date for the assignment has not passed, the “Perform review” link should be visible.
5. Click on the “Perform review” link.
6. Verify submitted work for team is visible.
7. Complete review and click save.

• When an author view their reviews, reviews done by instructors should present a star.

1. After logging in as the student, go to Assignments.
2. Click on the test assignment
3. Click "Your scores"
4. If a review was done by an instructor, verify the star is present.

Automated Testing with RSpec

We plan on adding comprehensive RSpec tests for all controller and model classes that we change as well as automated UI tests for features we added. Some of the planned test cases include:

• Verify that instructors can be added as participants for assignments.
• Verify that assignment reviews can be submitted by users with an Instructor, TA, or Student role assigned.
• Verify if an assignment's deadline is passed, instructors can see a "Assign Grade" link to assign a grade.
• Verify if an assignment's deadline has not passed, instructors can see a "Perform Review" link to review work if they are a participant.
• Verify a star icon appears next to only reviews done by instructors/TAs in the responses view.

Team Information

Danielle Hancock

Eric Peden

Hosung Hwang

Mentor: Abhirav Kariya

Resources

Forked Repo: https://github.com/epeden/expertiza

Expertiza Pull Request: https://github.com/expertiza/expertiza/pull/1729

Demo Video: https://youtu.be/mDsc5n1NZ9E