CSC/ECE 517 Spring 2018- Project E1824: Let course staff as well as students do reviews

From Expertiza_Wiki
Jump to navigation Jump to search

Introduction

The way Expertiza is set up right now is that only peers can review your work. However, there are cases when the course staff (Instructor/ TAs) would want to submit reviews as well. This project aims to implement this feature by allowing course staff to review the project on the same metrics as other students who review the project.


Problem Statement

Peer review is a great way for students to learn about how well they have developed their application. However there are some problems with this:

  • Sometimes, the peer reviews may not be thorough and the team/person's work reviewed might not reflect the actual status of the development.
  • The reviewer might not know how well they are reviewing the peers work. They might not entirely know as to what tone to use or what suggestions to put forward.

By letting course staff perform reviews as well, the reviewer and the reviewee are both benefited, which can improve the overall learning experience.


Current scenario

This is how some of the pages we are concerned with, currently look.


Submissions page for an assignment in Instructor View


A typical Scores table in a Student View


Proposed Solutions and Implementation

We performed the following changes to let staff perform reviews as well:

Step 1: Add a way for the instructors to perform a review.

To do this, we added links for performing/viewing/editing/updating a review in the assignment submissions view. When the instructor/TA reviews a work for the first time, he is added as a participant and a review response mapping is created.

Files edited:

  • View: app/views/assignments/list_submissions.html.erb - To add links in the instructor view
  • Controller: app/controllers/review_mapping_controller.rb - Method: add_instructor_as_reviewer

Code snippet from the controller:

  def add_instructor_as_reviewer
    assignment_team = AssignmentTeam.find(params[:team_id])
    reviewer = AssignmentParticipant.where(user_id: params[:reviewer_id], parent_id: assignment_team.parent_id).first
    if reviewer.nil?
      reviewer = AssignmentParticipant.create(parent_id: assignment_team.parent_id, user_id: session[:user].id, can_submit: false, can_review: true,
                                              can_take_quiz: false, handle: 'handle')

    end
    @review_map_id=ReviewResponseMap.where(reviewee_id: assignment_team.id, reviewer_id: reviewer.id,
                             reviewed_object_id: params[:assignment_id]).first
    @review_map_id=assignment_team.assign_reviewer(reviewer) if @review_map_id.nil?
    redirect_to controller: 'response', action: 'new', id: @review_map_id.map_id
  end



In the second case, we can see a 'Perform review' link. This is the initial state when instructor review has not been added to the submission. Once the instructor adds a review and submits it, we can see the 'View review'. In case he just saves the review, an 'Edit review' link will appear. If the review deadline of one round is passed, the 'Edit review' link becomes 'Update review' link. This follows in line with what the student sees while performing a review. However, the course staff will be allowed to perform a review even if the deadline for reviewing has passed.


Step 2: Add the instructor review in Your Scores table in case he has reviewed your work. Provide a highlight/way to make it look distinct.


Files edited:

  • Assets: app/assets/stylesheets/grades.scss - To make an instructor performed review distinct from other reviews
  • View: app/views/grades/view_team.html.erb - Modified the "Others work" page to include the instructor review as well as ensure the score is not used.
  • Models:
    • app/models/answer.rb - Method: compute_scores | Modification in logic to ensure instructor review scores are not counted in the overall score of the student.
    • app/models/vm_question_response_score_cell.rb - Included a new variable called 'is_instructor_review' to identify an instructor review
    • app/models/vm_question_response_row.rb - Change in average_score_for_row method to not include an instructor review score
    • app/models/vm_question_response.rb - Modified the add_answer method to include the is_instructor_review to each row_score object

Code snippet from the add_answer method:

  participant= Participant.find(ResponseMap.find(Response.find(response_id).map_id).reviewer_id)
  user_id = participant.user_id
  course =  Course.find(@assignment.course_id)
  row.score_row.push(VmQuestionResponseScoreCell.new(answer.answer, color_code, answer.comments,
                     course.is_ta_or_instructor?(user_id), vm_tag_prompts))



A student should be able to make out if an instructor has reviewed his/his team's work. In case the instructor performs a review on the team's work, it will highlight the instructor's review with a golden border as seen in the round two score table along with explicitly stating that it is an 'Instructor review'. The average peer review score for the team has been modified to exclude the instructor's scores. Same has been taken into account for the average score calculated for each row.


Additional Files modified:

  • Controllers:
    • app/controllers/response_controller.rb - Change in re-directions in case of an instructor taking him back to the submissions page.
  • Models:
    • app/models/assignment.rb - Method: number_of_current_round_for_instructor | To allow an instructor to review an assignment even after the due date
    • app/models/course.rb - Method: is_ta_or_instructor? | To identify whether the user is a TA or an instructor of the current course

Code snippet from the model:

  def is_ta_or_instructor?(user_id)
    instructors = TaMapping.where(course_id: self.id).collect { |x| x.ta_id}
    instructors.push(self.instructor_id)
    return user_id.in?(instructors)
  end

Test Plan

Features testing: Many of our changes are reflected on views (user interface). Following is the list of tests that are conducted with rspec/capybara.

  1. it "should let instructor/TAs perform a review for the submission of latest finished round if instructor/TAs has not started a review for the submission yet"
  2. it "should let instructor/TAs save a review"
  3. it "should let instructor/TAs review a previously saved review"
  4. it "should let instructor/TAs edit a previously saved review"
  5. it "should allow instructor/TAs perform a review after the deadline for reviewing has passed"
describe "instructor review testing" do
  before(:each) do
    create(:assignment, name: "TestAssignment", directory_path: 'test_assignment')
    create_list(:participant, 3)
    create(:assignment_node)
    create(:deadline_type, name: "submission")
    create(:deadline_type, name: "review")
    create(:deadline_type, name: "metareview")
    create(:deadline_type, name: "drop_topic")
    create(:deadline_type, name: "signup")
    create(:deadline_type, name: "team_formation")
    create(:deadline_right)
    create(:deadline_right, name: 'Late')
    create(:deadline_right, name: 'OK')
    create(:assignment_due_date, deadline_type: DeadlineType.where(name: 'review').first, due_at: Time.now.in_time_zone + 1.day)
    create(:topic)
    create(:topic, topic_name: "TestReview")
    create(:team_user, user: User.where(role_id: 2).first)
    create(:team_user, user: User.where(role_id: 2).second)
    create(:assignment_team)
    create(:team_user, user: User.where(role_id: 2).third, team: AssignmentTeam.second)
    create(:signed_up_team)
    create(:signed_up_team, team_id: 2, topic: SignUpTopic.second)
    create(:assignment_questionnaire)
    create(:question)
    create(:review_response_map, reviewer_id: User.find_by(name: "instructor6").id, reviewee: AssignmentTeam.second)
  end

  it "lets instructor perform a review and saves" do
    assignment = Assignment.first
    login_as("instructor6")
    visit "/assignments/list_submissions?id=#{assignment.id}"
    expect(page).to have_content 'Perform review'
    all(:link, 'Perform review')[1].click
    fill_in "responses[0][comment]", with: "DRY"
    select 5, from: "responses[0][score]"
    click_button "Save Review"
    expect(page).to have_content "Your response was successfully saved."
  end

  it "lets instructor view a saved review" do
    assignment = Assignment.first
    login_as("instructor6")
    visit "/assignments/list_submissions?id=#{assignment.id}"
    all(:link, 'Perform review')[1].click
    fill_in "responses[0][comment]", with: "Hello world"
    click_button "Save Review"
    visit "/assignments/list_submissions?id=#{assignment.id}"
    expect(page).to have_content 'View review'
    click_link "View review"
    expect(page).to have_content "show review"
  end

  it "lets instructor edit a saved review and saves" do
    assignment = Assignment.first
    login_as("instructor6")
    visit "/assignments/list_submissions?id=#{assignment.id}"
    all(:link, 'Perform review')[1].click
    fill_in "responses[0][comment]", with: "Good job"
    click_button "Save Review"
    visit "/assignments/list_submissions?id=#{assignment.id}"
    expect(page).to have_content 'Edit review'
    click_link "Edit review"
    expect(page).to have_text "Good job"
    fill_in "review[comments]", with: "Excellent work"
    click_button "Save Review"
    expect(page.current_path).to eq "/assignments/list_submissions"
  end

  it "allows instructor perform a review and saves after deadline for reviewing has passed" do
    assignment = Assignment.first
    due_date = AssignmentDueDate.first
    due_date.due_at = Time.now.in_time_zone - 1.day
    login_as("instructor6")
    visit "/assignments/list_submissions?id=#{assignment.id}"
    expect(page).to have_content 'Perform review'
    all(:link, 'Perform review')[1].click
    fill_in "responses[0][comment]", with: "DRY"
    select 5, from: "responses[0][score]"
    click_button "Save Review"
    expect(page).to have_content "Your response was successfully saved."
  end
end

Controllers testing: review_mapping_controller#add_instructor_as_reviewer

  describe '#add_instructor_as_reviewer' do
    context 'when instructor is not a participant for the assignment and review_response_map has not been created' do
      it 'adds instructor as a participant for the assignment and creates review_response_map and redirects to responses#new' do
        allow(AssignmentTeam).to receive(:find).with('1').and_return(team2)
        allow(AssignmentParticipant).to receive_message_chain(:where, :first)
          .with(user_id: 1, parent_id: '1').with(no_args).and_return(nil)
        allow(AssignmentParticipant).to receive(:create)
          .with(parent_id: '1', user_id: 1, can_submit: false, can_review: true, can_take_quiz: false, handle: 'handle').and_return(participant)
        allow(ReviewResponseMap).to receive_message_chain(:where, :first)
          .with(reviewee_id: '1', reviewer_id: 1, reviewed_object_id: '1').with(no_args).and_return(nil)
        allow(team2).to receive(:assign_reviewer)
          .with(participant).and_return(review_response_map)
        params = {id: 1, team_id: 1, assignment_id: 1}
        session = {user: build(:instructor, id: 1)}
        get :add_instructor_as_reviewer, params, session
        expect(response).to redirect_to '/response/new?id=1'
      end
    end

    context 'when instructor is already a participant for the assignment and review_response_map has been created' do
      it 'does not need to add instructor as a participant for the assignment or create review_repsonse_map and redirects to response#new' do
        allow(AssignmentTeam).to receive(:find).with('1').and_return(team2)
        allow(AssignmentParticipant).to receive_message_chain(:where, :first)
          .with(user_id: 1, parent_id: '1').with(no_args).and_return(participant)
        allow(ReviewResponseMap).to receive_message_chain(:where, :first)
          .with(reviewee_id: '1', reviewer_id: 1, reviewed_object_id: '1').with(no_args).and_return(review_response_map)
        params = {id: 1, team_id: 1, assignment_id: 1}
        session = {user: build(:instructor, id: 1)}
        get :add_instructor_as_reviewer, params, session
        expect(response).to redirect_to '/response/new?id=1'
      end
    end
  end

More testing:

  1. it "should let instructor/TAs update a review for the submission of latest finished round if instructor/TAs performed and saved a review for the previous round"
  2. it "should show "Instructor review" when a student views feedback on his/her submission if an instructor/TA has reviewed his/his team's work"
  3. it "should exclude the instructor's/TA's review score when calculating the average peer review score for the team"
  4. it "should exclude the instructor's/TA's review score when calculating the average score for each question in the review"

External Links

Team Members

  • Ashis Sahoo
  • Satvik Shetty
  • Xiaohui Ellis
  • Kushal Nawalakha


References

Expertiza

Expertiza Github

Expertiza Documentation