CSC/ECE 517 Fall 2018/E1841 Issues Related to Rubrics

E1553 : Issues Related to Rubrics

Introdution

Expertiza is an Peer Review Web Application System. It allows multiple students to participate in various assignments posted by the Instructor and provides a platform to all the students to conduct a peer review on the work done by their peers. Expertiza is an opensource project written in Ruby on Rails and React.js. We as a team have targeted some specific issues related to this project and Have tried Our best to fix them.

Problem Statement

What it Does

In Expertiza, instructors (also admin, super admin and TAs) can create rubrics (they are called questionnaires in DB, there are different types like review rubric, teammate review rubric, etc. Each rubric may have one or many criteria (called questions in DB). For each criterion, it may have 0 to many suggestions.

Fixes Required

If an instructor has created an assignment with specific review rubric and reviews have been performed using the same rubric. After that if an instructor changes the rubric to more reasonable one, the reviews performed earlier won’t work. You need to fix this issue. An instructor should be warned about changing the rubric if there are outstanding reviews. Maybe it would be better to ask the instructor whether to delete previous reviews, & if so, tell the students to redo those reviews because the rubric has changed.

After an instructor has created an assignment and if an assignment is rubric-varying-by-round and then mistakenly changed to be not varying by round, and then corrected to be varying-by-round again, the formative rubric is applied to all rounds. Of course this is incorrect; the summative rubric should be re-enabled for the summative round. The problem is that when you make an assignment rubric-varying-by-round, the Javascript just assumes that the summative review is the same as the formative review. AJAX should be added to fetch the summative review ID from the db (it’s still there, because we assume the assignments_questionnaires table entry hasn’t been removed). You need to fix this issue.

Log in as an instructor into the expertiza system. If you try to select "Manage > Questionnaires > Review rubrics", it should pull up a list of review rubrics. But actually, it just takes you to the main Questionnaires page. The exact same error is happening for "Manage > Questionnaires > Author feedback", etc. You need to figure out what causes the error and fix it.

Current Functionality

Instructor able to create an Assignment but once participants have started reviews, changing the rubrics doesn't update the score.

Instructor able to populate an Assignment with multiple rubrics

Display of the Main tabs

Instructor able to navigate between tabs(Courses, Assignments and Questionnaires), but Manage menu selection doesn't work.

Solutions

Issue #1186

When one tries to select "Manage > Questionnaires > Review rubrics" , it just takes back to the Questionnaires Main page rather than displaying the Review rubrics page.

Solution Description

tree_display.jsx File

Path to file: /app/assets/javascripts/tree_display.jsx
A new React lifecycle function was added to the ContentTableRow class, viz. componentDidMount: function() . The function determines which sub-menu item was clicked in the Manage Instructor Content- Questionnaire menu and expands the corresponding rubric by updating the state of the table row.

 componentDidMount: function() {
      selectedMenuItem = document.getElementById("tree_display").getAttribute("data-menu-item");
      rubricArray = ["Review", "Metareview", "Author Feedback", "Teammate Review", "Course Survey", "Assignment Survey", "Global Survey"];
      selectedMenuItemIndex = rubricArray.indexOf(selectedMenuItem);

      if(selectedMenuItemIndex !== -1 && rubricArray[selectedMenuItemIndex] === this.props.name) {
          this.setState({
              expanded: true
          }, function() {
              this.props.rowClicked(this.props.id, true,this.props.newParams)
          })
      }
    },

tree_display_controller.rb File

Path to file: /app/controllers/tree_display_controller.rb
In this file, multiple functions were updated viz. goto_controller() and list() and goto_{rubric_name}, where rubric_name is one of the following: [questionnaires, review_rubrics, metareview_rubrics, teammatereview_rubrics, author_feedbacks, global_survey, surveys, course_surveys, bookmarkrating_rubrics, courses, assignments]

The goto_controller function now takes in a new parameter names "last_open_tab", which indicates which tab ought to be selected after a refresh (due to list action). This value is stored in a session variable. The goto_{rubric_name} functions pass in the value for last_open_tab corresponding to the tab to which they belong.

The list action was updated to store an instance variable that will be accessible in list.html.erb.

goto_controller():

The modification in goto_controller enables the function to keep a track of which tab was last opened by adding a new parameter viz. last_open_tab and setting its values to the integer value corresponding to the position of the tab on the web page . For example , if the Questionnaires tab is opened then a value of 3 will be assigned to the last_opened_tab parameter as the position of the Questionnaires tab is 3 on the web page.This function also handles the direction of the control from Questionnaires tab to the Review Rubrics Tab when Review Rubrics tab is clicked.

#Adding of the parameter 
def goto_controller(name_parameter, last_open_tab)
    node_object = TreeFolder.find_by(name: name_parameter)
    session[:root] = FolderNode.find_by(node_object_id: node_object.id).id
    session[:last_open_tab] = last_open_tab unless last_open_tab.nil?
    redirect_to controller: 'tree_display', action: 'list', current_controller: name_parameter
  end
#Setting of the Parameter
def goto_questionnaires
    goto_controller('Questionnaires', '3')
  end

list():

This function generates instance variable which contains the params variable current value and makes that value available to the list.html.erb file.

def list
    @current_controller = params[:current_controller]
    redirect_to controller: :content_pages, action: :view if current_user.nil?
    redirect_to controller: :student_task, action: :list if current_user.try(:student?)
  end

List.html.erb file

Path to file: /app/views/tree_display/list.html.erb This file has been modified so that the tree_display.jsx file is able to access the sub-menu item which is was clicked. The change involves added a data property called data-menu-item to the root div element with id "tree_display". The value is set equal to the instance variable set in tree_display_controller.rb.

#list.html.erb

<h1>Manage content</h1>

<%= link_to 'Manage Notifications', notifications_url %>
<div id="tree_display" params="#{@reactjsParams}" data-menu-item= '<%= "#{@current_controller}" %>'></div>

Issue #1096

Solution Description

This fix for this issues involves determining which, if any, of the rubric values were changed during the modification of an assignment, retrieving the corresponding responses, deleting those responses and sending mail to those reviewers. We also alert the instructor about the above happening and they may choose not to update the rubric.

assignments_controller.rb File

Path to file: /app/controllers/assignments_controller.rb

This file was updated to contain all the logic for handling review-response deletion and sending mails to appropriate reviewers. The following functions were updated: edit, update

• The edit function was updated to-

store the current(before edit) rubric values in session.

store the current assignment in session.

store the current action(edit) in a instance variable so that we can distinguish the creation and deletion views.

    # Store rubrics before user edits the assignments, so that we can determine if rubric was changed for any round.
    session[:rubrics_by_round] = rubrics_before_edit
    # For use in update action.
    session[:assignment] = @assignment
    # We use this variable to show alert warning for pending reviews, in _rubrics.html.erb.
    @current_action = "edit"

• The update function was updated to-

call the private handle_rubric_modification function.

     handle_rubric_modification

This following functions were added: (private)rubrics_before_edit, (private)handle_rubric_modification, private(rubric_modified_rounds), (private) get_responses_for_modified_rounds, (private) notify_reviewers_about_rubric_change, (private) reviewer_emails.

• The rubrics_before_edit function is used to store the values of rubrics on an assignment before the users tries to update them. The values are retrieved from the instance variable @assignment_questionnaires.

  # @return [Array]
  def rubrics_before_edit
    @assignment_questionnaires.each_with_object({}) do |questionnaire, rubric|
      current_round = questionnaire.used_in_round
      rubric[current_round.to_s] = questionnaire.questionnaire_id.to_s unless current_round.nil?
    end
  end

• The handle_rubric_modification method is used to handle rubric changes by an instructor to an assignment.

  def handle_rubric_modification
    responses = get_responses_for_modified_rounds(rubric_modified_rounds)
    notify_reviewers_about_rubric_change(responses)
    responses.each(&:destroy)
  end

• The rubric_modified_rounds method gets all rounds for which an instructor changed a rubric when updating an assignment.

  def rubric_modified_rounds
    params[:assignment_form][:assignment_questionnaire].each_with_object([]) do |questionnaire, rubric_modified_rounds|
      current_round = questionnaire["used_in_round"]
      if current_round != " " && session[:rubrics_by_round][current_round] != questionnaire["questionnaire_id"]
        rubric_modified_rounds << current_round
      end
    end
  end

• The get_responses_for_modified_rounds method retrieves all responses to reviews whose rubric was changed by an instructor when updating an assignment.

  # @param [Array] rubric_modified_rounds
  def get_responses_for_modified_rounds(rubric_modified_rounds)
    # Get responses for the current assignment.
    ResponseMap.where(reviewed_object_id: session[:assignment].id).each_with_object([]) do |review, responses|
      # For each review, get the responses that have been started(reviewer has clicked "begin") or finished.
      # Select only those responses which correspond to the rounds for which the rubric was updated.
      responses.concat(Response.where(map_id: review.id).select do |response|
        rubric_modified_rounds.include? response.round.to_s
      end)
    end
  end

• The notify_reviewers_about_rubric_change method is used to notify(mail) reviewers that their review responses have been deleted.

  # @param [Array] responses
  def notify_reviewers_about_rubric_change(responses)
    @reviewer_emails = reviewer_emails(responses)
    Mailer.notify_reviewers_on_review_reset(bcc: @reviewer_emails.uniq, assignment_name: session[:assignment].name).deliver_now unless @reviewer_emails.empty?
  end

• The reviewer_emails method retrieves reviewer emails from response objects.

  # @param [Array] responses
  def reviewer_emails(responses)
    responses.collect do |response|
      Participant.find(ResponseMap.where(id: response.map_id).first.reviewer_id).user.email
    end
  end

Solution Screenshots

Issue #1186

'Initial Page'- Assignments tab selected

The Questionnaires tab gets selected and Review rubric is auto-loaded.

Review Rubric Page

Meta Rubric Page

Issue #1186

'Initial Page'- Assignments tab selected - editing assignment

User alert

After Cancel in alert

After Ok in alert

Reviewer mailed

Review response deleted