CSC/ECE 517 Spring 2020/E2020 Let course staff as well as students do reviews

Introduction

About Expertiza

Expertiza is an open-source project that is built using the Ruby on Rails framework. At its core, it is a highly versatile tool that academic institutions can use to craft learning objectives centered around team projects, peer reviews, and highly-customizable assignments.

Purpose and Problem

The purpose of this project is to allow instructors or teaching assistants who are participants of an assignment to be able to review (before the last due date of the specific assignment) any team that has submitted the assignment. Beyond this general endeavor, there is also the condition that team members of the assignment should be able to view reviews done by the course staff that can be identified in an aesthetically pleasing manner. As an added feature, reviewers should see how their reviews differ from those done by the course staff.

There are two main issues:

1. Allow the instructor and teaching assistants to review teams that have submitted assignments.

2. Allow authors to visually identify reviews done by the course staff.

Implementation Details

Problem 1: Allow the instructor and teaching assistants to review teams that have submitted assignments

• Check will be done to verify whether the course staff is a participant. Staff can only review if they are a participant of the assignment.
• In the list_submissions.html.erb file, a "Perform Review" link is present if the final review due date of the assignment has not passed yet. If a review was already saved and the final review due date of the assignment has not passed, then a link to "Edit Review" will be present. If a review was already submitted and the due date has not passed, then a link to "View Review" will be present.

The instructor/TA will see a "Perform Review" link if the final review due date has passed.

• Clicking on the "Perform Review", "Edit Review", and "View Review" links will take the instructor/TA to the pages governed by response_controller#new, response_controller#edit, and response_controller#view respectively.
• The already present "Assign Grade" link will only appear after the final review due date of the assignment has passed, if there are no review due dates, or if the user is not a participant of the assignment.

The instructor/TA will see a "Perform Review", "Edit Review", or "View Review" link if the final review due date has not passed.

• View changes

app/views/assignments/list_submissions.html.erb

       <% unless participants.empty? %>
             <!-- If no due dates for assignment, allow assigning grades. -->
            <% if not @assignment.find_due_dates('review').nil? %>
             <!-- If due date has passed or user not a participant, Assign Grade. Otherwise, allow reviews. -->
             <% if Time.now > @assignment.find_due_dates('review').sort_by{|due_date| due_date.due_at}[-1].due_at or not @participant %>
               <%= link_to 'Assign Grade', { controller: 'grades', action: 'view_team', id: participants.first.id}, target: '_blank' %>
             <% else %>
               <% review = @review_maps.select{|m| m.reviewee_id == team.id} %>
               <!-- If a review has not yet been performed, allow review. Otherwise, allow view/edit actions. -->
               <% if review.size == 0 %>
                 <%= link_to 'Perform Review', {:controller => 'review_mapping', :action => 'add_staff_as_reviewer', :id => @assignment.id, :contributor_id => team.id, :user_id => current_user.id} %>
               <% else %>
                 <% r =  Response.where(["map_id IN (?)", review.last.id]).first %>
                 <% if r %>
                   <!-- Reviews cannot be edited after being submitted. -->
                   <% if r.is_submitted %>
                     <%= link_to 'View Review', {:controller => 'response', :action => 'view', :id => r.id, :return => "instructor_review"} %>
                   <% else %>
                     <%= link_to 'Edit Review', {:controller => 'response', :action => 'edit', :id => r.id, :return => "instructor_review"} %>
                   <% end %>
                 <% end %>
               <% end %>
             <% end %>
           <% else %>
             <%= link_to 'Assign Grade', { controller: 'grades', action: 'view_team', id: participants.first.id}, target: '_blank' %>
           <% end %>
         <% end %>

• A method was added to review_mapping_controller.rb that creates a new ReviewResponseMap for an instructor, TA, or Admin for the assignment specified by the id in the params. This will be called when the staff user clicks "PerformReview". It will then redirect to response_controller#new.

app/controllers/review_mapping_controller.rb

 # Enable a staff user (Instructor, Teaching Assistant, Admin) to review all submissions for an assignment when they are a participant.
 # Will redirect to the list_submissions view.
 def add_staff_as_reviewer
   assignment = Assignment.find(params[:id])
   user = User.from_params(params)
   # Get registration url for user in case the user is not a participant in the assignment.
   regurl = url_for id: assignment.id,
                    user_id: user.id,
                    contributor_id: params[:contributor_id]
   reviewer = get_reviewer(user, assignment, regurl)
   # Create response map for the participant so they can begin review.
   map = ReviewResponseMap.where(reviewee_id: params[:contributor_id], reviewer_id: reviewer.id, reviewed_object_id: assignment.id).first
   if map.nil?
     map = ReviewResponseMap.create(reviewee_id: params[:contributor_id], reviewer_id: reviewer.id, reviewed_object_id: assignment.id)
   end
   redirect_to :controller => 'response', :action => 'new', :id => map.id, :contributor_id => params[:contributor_id], :return => "instructor_review"
 end

• A check is now performed to determine if the current user is a participant in the assignment. If so, we'll check if they have any existing reviews and those will be available to the view.

app/controllers/assignments_controller.rb

 def list_submissions
   @assignment = Assignment.find(params[:id])
   @teams = Team.where(parent_id: params[:id])
   @participant = AssignmentParticipant.where(user_id: current_user.id, parent_id: @assignment.id).first
   # If the current user is not a participant in the assignment, they cannot review submissions.
   if not @participant.nil?
     @review_maps = ReviewResponseMap.where(reviewer_id: @participant.id, reviewed_object_id: @assignment.id)
   else
     @review_maps = []
   end
 end

Problem 2: Allow authors to visually identify reviews done by the course staff

• A star icon will be used to distinguish whether a review was done by a course staff and will appear below the 'Review #' heading.
• Add helper class grades_helper.rb to handle check to determine whether review was done by course staff based on the boolean flag attached to a response.
• Change will be made in view_team.html.erb such that after verifying that a response was done by a course staff via identity_helper.rb, a star is displayed below the 'Review #' heading.

Note: Previously, a review that was done by a course staff would not be visually recognizable by an author of the particular reviewed assignment. With the following changes, however, authors will be able to recognize reviews done under the "Summary Report for assignment" page, as reviews done by course staff will have a black star located immediately below the "Review #" header. A boolean check is done in the review_done_by_course_staff? method and used in view_team.html.erb to display a black star below a particular review if and only if review_done_by_course_staff? outputs true.

Changed files:

app/helpers/grades_helper.rb

   # used to determine whether a review was done by a course staff
   # course staff is defined as an Instructor or Teaching Assistant
   def review_done_by_course_staff?(review)
     if review.nil?
       false
     else
       role = Role.find(User.find(Participant.find(ResponseMap.find(Response.find(review.id).map_id).reviewer_id).user_id).role_id).name
       role == ("Instructor" || "Teaching Assistant") ? true : false
     end
   end

app/views/grades/view_team.html.erb

    <!-- instructors (or higher level users) see reviewer name, students see anonymized string -->
    <!-- instructor reviews labeled with a black star under the Review heading -->
    <% if (['Student'].include? @current_role_name) && (@assignment.is_anonymous) %>
        <% review_name = "Review " + i.to_s  %>
    <% else %>
        <% review_name = User.find(Participant.find(ResponseMap.find(Response.find(review.id).map_id).reviewer_id).user_id).fullname(session[:ip]).to_s %>
    <% end %>
    <th class="sorter-false"> <a target="_blank" href="../response/view?id=<%= review.id.to_s %>"  data-toggle="tooltip" data-placement="right" title="Click to see details"><%= review_name %></a>
    <% if review_done_by_course_staff?(review) %>
        <br><span class="glyphicon glyphicon-star" style="margin:auto; display:table"></span>
    <% end %>

Pull request can be found here

• A staff user (Instructor, TA, Admin) will see the star for reviews done by another staff member as well as the names of all the reviewers.

• A student will still see the star for reviews done by a staff member but all names are anonymous.

All Modified Files

• app/controllers/assignments_controller.rb
• app/controllers/response_controller.rb
• app/controllers/review_mapping_controller.rb
• app/helpers/grades_helper.rb
• app/views/assignments/list_submissions.html.erb
• app/views/grades/view_team.html.erb
• app/views/response/response.html.erb
• config/routes.rb

Flow Chart

This flow chart demonstrates the desired functionality. While students must submit reviews in the review stage, instructors/TA's can submit reviews any time before the final deadline. In order to submit reviews, the instructor/TA must be added as a participant to the assignment. Then, they will go to "View Submissions" on the Manage Assignments page. If the final deadline for the assignment has passed, they will see an "Assign Grade" link next to each submission. Otherwise, they will see a "Perform Review" link if a review has not been submitted yet or an "Edit Review" link if a review has been submitted already. Students can view only the reviews they have performed and the reviews of their work, while instructors/TA's can see all reviews for the assignment. Reviews done by course staff will be marked with a star to distinguish them from students' reviews.

Database Design

This diagram represents the relationship between different models involved with our proposed solution. We do not anticipate any changes to the database schema.

A model that's central to our proposed solution is the AssignmentQuestionnaire model defined in assignment_questionnaire.rb. It holds the user ID of the User (user.rb) who reviewed the assignment, from which we will determine the user's role: Instructor, TA, or student. It also contains the ID of the Assignment (assignment.rb) being reviewed. Additionally, the Questionnaire (questionnaire.rb) can be located by its ID defined in AssignmentQuestionnaire and consists of the Questions (question.rb) users use to review the assignment.

Testing

Testing from the UI

Manual UI verification of our added features can be found in the demo video in the Resources section. More details on the UI testing can be found in the sections below.

Automated Testing with RSpec

Comprehensive RSpec tests were added for all controller and model classes that we changed as well as automated UI tests for features we added. Test cases include:

• Verify that instructors can be added as participants for assignments.
• Verify that assignment reviews can be submitted by users with an Instructor, TA, or Student role assigned.
• Verify if a instructor is not a participant, an "Assign Grade" is visible.
• Verify if an assignment's deadline is passed and the instructor is a participant, an "Assign Grade" is visible.
• Verify if an assignment's deadline is not passed and the instructor is a participant, a "Perform Review" link is visible.
• Verify if a review has already been submitted by an instructor, a new ReviewResponseMap will not be created for the instructor when they load the list_submissions view.
• Verify a star icon appears next to only reviews done by instructors/TAs in the responses view. This should be visible to both staff and students.

All modified RSpec test files:

• spec/controllers/assignments_controller_spec.rb
• spec/controllers/review_mapping_controller_spec.rb
• spec/features/assignment_list_submissions_spec.rb
• spec/helpers/grades_helper_spec.rb

Problem 1: Allow the instructor and teaching assistants to review teams that have submitted assignments

Manual UI verification of our added features can be found in the demo video in the Resources section. These cases are included:

• An "Assign Grade" link is visible in the submissions list if the instructor is not a participant. Instructor can be added as participant

1. After logging in as the instructor, go to Manage > Assignments.
2. Find the test assignment and click on the icon to view submissions.
3. An "Assign Grade" link should be visible for the submissions.
4. Navigate back to the test assignment and click the icon to add participants.
5. Add the instructor as a participant on the assignment.
6. Click on the icon to view submissions. A "Perform Review" link should be present for the submissions.

• Instructor should be able to add a review on an assignment if they are a participant and the review deadline has not passed.

1. After logging in as the instructor, go to Manage > Assignments.
2. Find the test assignment and click on the Edit icon. Click on Due Dates and confirm the review due date has not passed.
3. Click on the View Submissions icon.
4. Verify a list of submissions appears. The “Perform review” link should be visible.
5. Click on the “Perform review” link.
6. Verify submitted work for team is visible.
7. Complete review and click save.

RSpec Testing

spec/controllers/review_mapping_controller_spec.rb

 describe '#add_staff_as_reviewer' do
   context 'when a review map does not exist for the user' do
     it 'creates a new ReviewResponseMap for the user' do
       allow(Assignment).to receive(:find).with(any_args).and_return(user)
       allow(User).to receive(:from_params).with(any_args).and_return(assignment)
       allow_any_instance_of(ReviewMappingController).to receive(:get_reviewer)
         .with(assignment, user, "http://test.host/review_mapping/add_staff_as_reviewer?contributor_id=1&id=3&user_id=1")
         .and_return(double('AssignmentParticipant', id: 1, name: 'no one'))
       allow(ReviewResponseMap).to receive(:where)
         .with(reviewed_object_id: 3, reviewer_id: 1, reviewee_id: '1').and_return([])
       allow(ReviewResponseMap).to receive(:create)
         .with(reviewed_object_id: 3, reviewer_id: 1, reviewee_id: '1').and_return(review_response_map)
       post :add_staff_as_reviewer, :id => 3, :contributor_id => 1, :user_id => 1
       expect(response).to redirect_to '/response/new?contributor_id=1&id=1&return=instructor_review'  #:controller => 'response', :action => 'new', :id => 1, :contributor_id => 1, :return => "instructor_review"
     end
   end
   context 'when a review map exists for the user' do
     it 'creates a new ReviewResponseMap for the user' do
       allow(Assignment).to receive(:find).with(any_args).and_return(user)
       allow(User).to receive(:from_params).with(any_args).and_return(assignment)
       allow_any_instance_of(ReviewMappingController).to receive(:get_reviewer)
                                                             .with(assignment, user, "http://test.host/review_mapping/add_staff_as_reviewer?contributor_id=1&id=3&user_id=1")
                                                             .and_return(double('AssignmentParticipant', id: 1, name: 'no one'))
       allow(ReviewResponseMap).to receive(:where)
                                       .with(reviewed_object_id: 3, reviewer_id: 1, reviewee_id: '1').and_return([review_response_map])
       post :add_staff_as_reviewer, :id => 3, :contributor_id => 1, :user_id => 1
       expect(response).to redirect_to '/response/new?contributor_id=1&id=1&return=instructor_review'  #:controller => 'response', :action => 'new', :id => 1, :contributor_id => 1, :return => "instructor_review"
     end
   end
 end

Problem 2: Allow authors to visually identify reviews done by the course staff

UI Testing

1. Login as an instructor
   • Username: instructor6
   • Password: password
2. As this test is an extension of the one conducted in Part 1, impersonate the student who the instructor performed a review for. Go to Manage -> Impersonate User and enter the student's user name.
3. Now impersonated as the chosen student, go to Assignments -> Project_Name -> Your Scores.
4. You should now be in the "Summary Report for assignment" page. Below whichever review that the instructor submitted, there will be a black star.

RSpec Testing

Note: Additional tests were created for methods in the same grades_helper helper class that were not written by this team in order to abide by the philosophy that you should always leave code in a better position that they were originally.

spec/helpers/grades_helper_spec.rb

 describe 'has_team_and_metareview' do
   before(:each) do
     @assignment = create(:assignment, max_team_size: 1)
   end
   it 'should correctly identify the assignment from an assignment id' do
     params[:id] = @assignment.id
     result = Assignment.find(params[:id])
     expect(result).to eq(@assignment)
   end
   it 'should correctly identify the assignment from a participant id' do
     participant = create(:participant, assignment: @assignment)
     params[:id] = participant.id
     result = Participant.find(params[:id]).parent_id
     expect(result).to eq(@assignment.id)
   end
   it 'should return 0 for an assignment without a team or a metareview deadline after a view action' do
     params[:action] = 'view'
     params[:id] = @assignment.id
     result = has_team_and_metareview?
     expect(result).to be == {has_team: false, has_metareview: false, true_num: 0}
   end
   it 'should return 1 for an assignment with a team but no metareview deadline after a view action' do
     @assignment.max_team_size = 2
     @assignment.save
     params[:action] = 'view'
     params[:id] = @assignment.id
     result = has_team_and_metareview?
     expect(result).to be == {has_team: true, has_metareview: false, true_num: 1}
   end
 end

 describe 'type_and_max' do
   it 'should fail when passing in nil row value' do
     expect { type_and_max(nil) }.to raise_exception(NoMethodError)
   end
 end

 describe 'underlined' do
   it 'should fail when passing in nil score value' do
     expect { underlined?(nil) }.to raise_exception(NoMethodError)
   end
 end

 describe 'retrieve_questions' do
   it 'should fail when passing in nil values for questionnaires and assignment_id' do
     expect { retrieve_questions(nil,nil) }.to raise_exception(NoMethodError)
   end
 end

 describe 'review_done_by_course_staff' do
   it 'should return false' do
     expect(review_done_by_course_staff?(nil)).to eq(false)
   end
 end

Code Coverage

Coverage increased (+11.01%) to 41.493%

Team Information

Danielle Hancock

Eric Peden

Hosung Hwang

Mentor: Abhirav Kariya

Resources

Forked Repo: https://github.com/epeden/expertiza

Expertiza Pull Request: https://github.com/expertiza/expertiza/pull/1729

Demo Video: https://youtu.be/mDsc5n1NZ9E