CSC/ECE 517 Spring 2016 / Expertiza Self-Review Feature

This page will discuss the newly added peer self review feature that was added to the Expertiza project.

Background

Developed by both students and faculty at North Carolina State University using Ruby on Rails, Expertiza is a educational based web application that allows to students to peer review other student's work. In Expertiza, instructors are in charge of courses and within each course, they may create a series of assignments for students to contribute to, either as a member of a team or individually. Students or Teams may submit their assignments to the system through multiple means including file uploads and embedded links. Then assignments may become available to peer review by other members of the course, which provides a way to crowd-source grading on the work.

Problem Statement

Expertiza previously contained a self-review feature where student's were able to review their own work after submitting their assignments but it was removed due to the inconsistency with the other types of reviews. The task was to re-implement the feature in a way that ensured the overall design would remain intact. The self-review feature should allow students to review their own submission using the same rubric that was provided to other peer reviewers of the assignment. By reviewing their own work, students will be able to assess the artifact through the rubric and may be inspired to make improvements between rounds of peer-reviews.

What Needs Done:

The following tasks were given to be completed by the team:

• The self-review feature should use the same questionnaire_id as a peer-review. That way developers may call "assignment.get_review_questionnaire_id(round)" to get a questionnaire for the self-review.
• A method called "self_review_response_map?" should be added to review_response_map.rb. The method will return true if the response_map is a self_response map. In order to determine the result the method shall compare the reviewer_id and reviewee_id in the response_map record and if the participant is a member of the reviewee team, true will be returned.
• Student shall do a self-review in the "your work" section. There should be a button called "Review my own work" (or "our own work" if it is a team assignment). When the button is clicked a new review_response_map record is created. Within the map, the reviewer_id is the participant id and the reviewee_id is the team_id.
• If the response_map record for self-review exists, a hyperlink with the text "begin" should appear. A student may click the link and fill in the questionnaire just like how they would for a regular peer review.
• If the self-review response_map exists and there has already been a response, two links entitled "view" and "edit" will appear that will function similarly to peer reviews.
• If the self-review was completed in the last round and the current round uses a different questionnaire (using the vary_rubric_by_round feature), a link called "update" will appear, which brings students to a new page to fill in the new rubric for the new round.
• In grades_controller.view_my_scores, self-review responses shall be excluded.
• In grades_controller.view_team, self-review responses shall be excluded.
• In grades_controller.view, self-review responses shall be excluded.
• The method called "add_self_reviewer" in ReviewMappingController, shall allow instructors to create self-review response maps. This method needs to be made functional again.
• Automated tests must be created.

Files Modified in Project

The following files were modified in creating the new self-review feature

• review_response_map.rb
• review_mapping_controller.rb
• response_controller.rb
• _set_dynamic_review.html.erb
• _main.html.erb
• _review_strategy.html.erb
• response.html.erb
• routes.rb
• test_helper.rb
• submitted_content_controller_test.rb
• factories.rb
• test_helper.rb

The following files were created:

• self_review_response_map.rb
• _self_review.html.erb
• 20160321221146_add_columnselfreviewenabled.rb

The following test classes were created:

• self_review_response_maps.rb
• self_review_response_map_test.rb
• response_map_test.rb

The following fixtures were modified:

• response_maps.yml
• participants.yml
• assignments.yml
• questionnaires.yml
• roles.yml
• users.yml
• assignment_questionnaires.yml
• response_maps.yml
• responses.yml
• teams.yml
• teams_users.yml

Walkthrough

Below are walkthroughs that will guide users through the new functionality included in the update.

Instructor Perspective

When creating a new assignment, an instructor will have an option to enable self-reviews. In order to allow for self-reviews to be completed, the instructor must check the box while either creating or editing the assignment under the Review Strategy tab.

Student Perspective

If self-reviews are available on an assignment the following will be available for students to use.

1. During the beginning stages of an assignment under "your work" the User Interface will show a label called Self-Review and a button which reads "Review my/our own work" (depending on whether it is an individual or team assignment).
2. After selecting "Review my/our own work" the button will change to "Begin" (just like how a peer review works).
3. Once the user selects "Begin" they will be taken to a questionnaire to complete. The questionnaire is the same one that a peer reviewer would complete.
4. Similarly to a peer review, a user may either "Save" or "Submit" their self-review.
5. After a user saves a self-review, "View" and "Edit" links will appear. However, once a user submits their self-review, they will be unable to edit the peer review.
6. If there are multiple rounds of submissions and reviews, then in all following rounds an update link will appear to allow for students to update their self-review.
7. If late submissions are allowed (via Expertiza "submission allowed" flag) then a user may begin and submit self reviews even in other review stages.
8. If a self-review is not submitted throughout the duration of the assignment then "Work yet to be submitted" will be shown.
9. Scores from self-reviews will not be considered when calculating grades throughout the assignment.

Database modification

In order to implement the self-review feature a modification of the database was required to add the column, is_selfreview_enabled, to the assignments table. Below is the new database migration:

#E1600
class AddColumnselfreviewenabled < ActiveRecord::Migration
  def self.up
    add_column :assignments, :is_selfreview_enabled, :boolean
  end

  def self.down
    remove_column :assignments, :is_selfreview_enabled, :boolean
  end
end

Notable Code Changes

The largest change to the codebase to accommodate for the new self-review feature was the addition of a new type of Response Map called "SelfReviewResponseMap." The new model class is shown below:

#E1600
class SelfReviewResponseMap < ResponseMap
  belongs_to :reviewee, :class_name => 'Team', :foreign_key => 'reviewee_id'
  belongs_to :assignment, :class_name => 'Assignment', :foreign_key => 'reviewed_object_id'

  def questionnaire(round)
    if self.assignment.varying_rubrics_by_round?
      Questionnaire.find(self.assignment.get_review_questionnaire_id(round))
    else
      Questionnaire.find(self.assignment.get_review_questionnaire_id)
    end
  end

  def contributor
    Team.find_by_id(self.reviewee_id)
  end

  def get_title
    return "Self Review"
  end
end

In order to take full advantage of the new feature, instructors had to have the ability to turn it on or off as they see fit. As a result a checkbox to goggle the self-review feature was added to the assignment settings page in _general.html.erb. Below is some sample code demonstrating the addition of a checkbox to the assignment settings:

<!-- self review flag should be the last checkbox-->
<tr>
  <td style='padding:5px' id='assignment_is_selfreview_enabled'>
    <input name="assignment_form[assignment][is_selfreview_enabled]" type="hidden" value="false"/>
    <%= check_box_tag('assignment_form[assignment][is_selfreview_enabled]', 'true', @assignment_form.assignment.is_selfreview_enabled) %>
    <%= label_tag('assignment_form[assignment][is_selfreview_enabled]', 'Allow self reviews?') %>
  </td>
</tr>

Changes were also made to review_mapping_controller.rb in order to accommodate the new self-review feature. The following method was added to handle the creation of new self-review for team members.

#E1600
# Start self review if not started yet
def start_self_review
  assignment = Assignment.find(params[:assignment_id])
  team_id = TeamsUser.find_by_sql(["SELECT t.id as t_id FROM teams_users u, teams t WHERE u.team_id = t.id and t.parent_id = ? and user_id = ?", assignment.id, params[:reviewer_userid]])

  begin
    #ACS Removed the if condition(and corressponding else) which differentiate assignments as team and individual assignments
    # to treat all assignments as team assignments
    if SelfReviewResponseMap.where(['reviewee_id = ? and reviewer_id = ?', team_id[0].t_id, params[:reviewer_id]]).first.nil?
      SelfReviewResponseMap.create(:reviewee_id => team_id[0].t_id,
                               :reviewer_id => params[:reviewer_id],
                               :reviewed_object_id => assignment.id)
    else
      raise "Self review already assigned!"
    end
    redirect_to :controller => 'submitted_content', :action => 'edit', :id => params[:reviewer_id]
  rescue
    redirect_to :controller => 'submitted_content', :action => 'edit', :id => params[:reviewer_id], :msg => $!
  end
end

Testing

Tests have been added to confirm all of the new functionality, below are some test samples that were implemented with the changes:

test "method_questionnaire" do
  @questionnaire = questionnaires(:questionnaire0)
  @assignment = assignments(:assignment1)
  reviewrespmap = SelfReviewResponseMap.new
  reviewrespmap.assignment = @assignment
  reviewrespmap.questionnaire 1
  assert_equal reviewrespmap.assignment.questionnaires[0].type, "ReviewQuestionnaire"
end

test "method_get_title" do
  p = SelfReviewResponseMap.new
  assert_equal p.get_title, "Self Review"
end

test "method_contributor" do
  p = SelfReviewResponseMap.new
  p.reviewee = teams(:team0)
  team_id = p.contributor
  assert_equal p.reviewee.id, team_id.id
end

test "reviewer_reviewee_same_team_test" do
  p = SelfReviewResponseMap.new
  p.reviewee = teams(:team0)
  @participant = participants(:par1)
  team_id = TeamsUser.find_by_sql(["SELECT t.id as t_id FROM teams_users u, teams t WHERE u.team_id = t.id and t.parent_id = ? and user_id = ?", @participant.parent_id, @participant.user_id])
  assert_equal p.reviewee.id, team_id[0].t_id
end

References

1. Expertiza on GitHub
2. Expertiza Documentation
3. Expertiza Homepage
4. Feature Fork