CSC/ECE 517 Fall 2018/E1841 Issues Related to Rubrics

From Expertiza_Wiki
Revision as of 06:13, 6 November 2018 by Sbhoyar (talk | contribs)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search

E1553 : Issues Related to Rubrics

Introdution

Expertiza is an Peer Review Web Application System. It allows multiple students to participate in various assignments posted by the Instructor and provides a platform to all the students to conduct a peer review on the work done by their peers. Expertiza is an opensource project written in Ruby on Rails and React.js. We as a team have targeted some specific issues related to this project and Have tried Our best to fix them.

Problem Statement

What it Does

In Expertiza, instructors (also admin, super admin and TAs) can create rubrics (they are called questionnaires in DB, there are different types like review rubric, teammate review rubric, etc. Each rubric may have one or many criteria (called questions in DB). For each criterion, it may have 0 to many suggestions.

Fixes Required

  • If an instructor has created an assignment with specific review rubric and reviews have been performed using the same rubric. After that if an instructor changes the rubric to more reasonable one, the reviews performed earlier won’t work. You need to fix this issue. An instructor should be warned about changing the rubric if there are outstanding reviews. Maybe it would be better to ask the instructor whether to delete previous reviews, & if so, tell the students to redo those reviews because the rubric has changed.
  • After an instructor has created an assignment and if an assignment is rubric-varying-by-round and then mistakenly changed to be not varying by round, and then corrected to be varying-by-round again, the formative rubric is applied to all rounds. Of course this is incorrect; the summative rubric should be re-enabled for the summative round. The problem is that when you make an assignment rubric-varying-by-round, the Javascript just assumes that the summative review is the same as the formative review. AJAX should be added to fetch the summative review ID from the db (it’s still there, because we assume the assignments_questionnaires table entry hasn’t been removed). You need to fix this issue.
  • Log in as an instructor into the expertiza system. If you try to select "Manage > Questionnaires > Review rubrics", it should pull up a list of review rubrics. But actually, it just takes you to the main Questionnaires page. The exact same error is happening for "Manage > Questionnaires > Author feedback", etc. You need to figure out what causes the error and fix it.

    Current Functionality

  • Instructor able to create an Assignment but once participants have started reviews, changing the rubrics doesn't update the score.
  • Instructor able to populate an Assignment with multiple rubrics
  • Display of the Main tabs
  • Instructor able to navigate between tabs(Courses, Assignments and Questionnaires), but Manage menu selection doesn't work.

    Solutions

    Issue #1186

    When one tries to select "Manage > Questionnaires > Review rubrics" , it just takes back to the Questionnaires Main page rather than displaying the Review rubrics page.

    Solution Description
    tree_display.jsx File

    Path to file: /app/assets/javascripts/tree_display.jsx
    A new React lifecycle function was added to the ContentTableRow class, viz. componentDidMount: function() . The function determines which sub-menu item was clicked in the Manage Instructor Content- Questionnaire menu and expands the corresponding rubric by updating the state of the table row.

     componentDidMount: function() {
          selectedMenuItem = document.getElementById("tree_display").getAttribute("data-menu-item");
          rubricArray = ["Review", "Metareview", "Author Feedback", "Teammate Review", "Course Survey", "Assignment Survey", "Global Survey"];
          selectedMenuItemIndex = rubricArray.indexOf(selectedMenuItem);
    
          if(selectedMenuItemIndex !== -1 && rubricArray[selectedMenuItemIndex] === this.props.name) {
              this.setState({
                  expanded: true
              }, function() {
                  this.props.rowClicked(this.props.id, true,this.props.newParams)
              })
          }
        },
    
    tree_display_controller.rb File

    Path to file: /app/controllers/tree_display_controller.rb
    In this file, multiple functions were updated viz. goto_controller() and list() and goto_{rubric_name}, where rubric_name is one of the following: [questionnaires, review_rubrics, metareview_rubrics, teammatereview_rubrics, author_feedbacks, global_survey, surveys, course_surveys, bookmarkrating_rubrics, courses, assignments]

    The goto_controller function now takes in a new parameter names "last_open_tab", which indicates which tab ought to be selected after a refresh (due to list action). This value is stored in a session variable. The goto_{rubric_name} functions pass in the value for last_open_tab corresponding to the tab to which they belong.

    The list action was updated to store an instance variable that will be accessible in list.html.erb.

    goto_controller():
  • The modification in goto_controller enables the function to keep a track of which tab was last opened by adding a new parameter viz. last_open_tab and setting its values to the integer value corresponding to the position of the tab on the web page . For example , if the Questionnaires tab is opened then a value of 3 will be assigned to the last_opened_tab parameter as the position of the Questionnaires tab is 3 on the web page.This function also handles the direction of the control from Questionnaires tab to the Review Rubrics Tab when Review Rubrics tab is clicked.
    #Adding of the parameter 
    def goto_controller(name_parameter, last_open_tab)
        node_object = TreeFolder.find_by(name: name_parameter)
        session[:root] = FolderNode.find_by(node_object_id: node_object.id).id
        session[:last_open_tab] = last_open_tab unless last_open_tab.nil?
        redirect_to controller: 'tree_display', action: 'list', current_controller: name_parameter
      end
    #Setting of the Parameter
    def goto_questionnaires
        goto_controller('Questionnaires', '3')
      end
    
    list():
  • This function generates instance variable which contains the params variable current value and makes that value available to the list.html.erb file.
    def list
        @current_controller = params[:current_controller]
        redirect_to controller: :content_pages, action: :view if current_user.nil?
        redirect_to controller: :student_task, action: :list if current_user.try(:student?)
      end
    


    List.html.erb file

    Path to file: /app/views/tree_display/list.html.erb This file has been modified so that the tree_display.jsx file is able to access the sub-menu item which is was clicked. The change involves added a data property called data-menu-item to the root div element with id "tree_display". The value is set equal to the instance variable set in tree_display_controller.rb.

    #list.html.erb
    
    <h1>Manage content</h1>
    
    <%= link_to 'Manage Notifications', notifications_url %>
    <div id="tree_display" params="#{@reactjsParams}" data-menu-item= '<%= "#{@current_controller}" %>'></div>
    

    Issue #1096

    Solution Description

    This fix for this issues involves determining which, if any, of the rubric values were changed during the modification of an assignment, retrieving the corresponding responses, deleting those responses and sending mail to those reviewers. We also alert the instructor about the above happening and they may choose not to update the rubric.


    assignments_controller.rb File

    Path to file: /app/controllers/assignments_controller.rb

    This file was updated to contain all the logic for handling review-response deletion and sending mails to appropriate reviewers. The following functions were updated: edit, update

    • The edit function was updated to-
    store the current(before edit) rubric values in session.
    store the current assignment in session.
    store the current action(edit) in a instance variable so that we can distinguish the creation and deletion views.
    
        # Store rubrics before user edits the assignments, so that we can determine if rubric was changed for any round.
        session[:rubrics_by_round] = rubrics_before_edit
        # For use in update action.
        session[:assignment] = @assignment
        # We use this variable to show alert warning for pending reviews, in _rubrics.html.erb.
        @current_action = "edit"
    
    • The update function was updated to-
    call the private handle_rubric_modification function.
         handle_rubric_modification
    

    This following functions were added: (private)rubrics_before_edit, (private)handle_rubric_modification, private(rubric_modified_rounds), (private) get_responses_for_modified_rounds, (private) notify_reviewers_about_rubric_change, (private) reviewer_emails.

    • The rubrics_before_edit function is used to store the values of rubrics on an assignment before the users tries to update them. The values are retrieved from the instance variable @assignment_questionnaires.
      # @return [Array]
      def rubrics_before_edit
        @assignment_questionnaires.each_with_object({}) do |questionnaire, rubric|
          current_round = questionnaire.used_in_round
          rubric[current_round.to_s] = questionnaire.questionnaire_id.to_s unless current_round.nil?
        end
      end
    
    • The handle_rubric_modification method is used to handle rubric changes by an instructor to an assignment.
      def handle_rubric_modification
        responses = get_responses_for_modified_rounds(rubric_modified_rounds)
        notify_reviewers_about_rubric_change(responses)
        responses.each(&:destroy)
      end
    
    • The rubric_modified_rounds method gets all rounds for which an instructor changed a rubric when updating an assignment.
      def rubric_modified_rounds
        params[:assignment_form][:assignment_questionnaire].each_with_object([]) do |questionnaire, rubric_modified_rounds|
          current_round = questionnaire["used_in_round"]
          if current_round != " " && session[:rubrics_by_round][current_round] != questionnaire["questionnaire_id"]
            rubric_modified_rounds << current_round
          end
        end
      end
    
    • The get_responses_for_modified_rounds method retrieves all responses to reviews whose rubric was changed by an instructor when updating an assignment.
      # @param [Array] rubric_modified_rounds
      def get_responses_for_modified_rounds(rubric_modified_rounds)
        # Get responses for the current assignment.
        ResponseMap.where(reviewed_object_id: session[:assignment].id).each_with_object([]) do |review, responses|
          # For each review, get the responses that have been started(reviewer has clicked "begin") or finished.
          # Select only those responses which correspond to the rounds for which the rubric was updated.
          responses.concat(Response.where(map_id: review.id).select do |response|
            rubric_modified_rounds.include? response.round.to_s
          end)
        end
      end
    
    • The notify_reviewers_about_rubric_change method is used to notify(mail) reviewers that their review responses have been deleted.
      # @param [Array] responses
      def notify_reviewers_about_rubric_change(responses)
        @reviewer_emails = reviewer_emails(responses)
        Mailer.notify_reviewers_on_review_reset(bcc: @reviewer_emails.uniq, assignment_name: session[:assignment].name).deliver_now unless @reviewer_emails.empty?
      end
    
    • The reviewer_emails method retrieves reviewer emails from response objects.
      # @param [Array] responses
      def reviewer_emails(responses)
        responses.collect do |response|
          Participant.find(ResponseMap.where(id: response.map_id).first.reviewer_id).user.email
        end
      end
    

    Solution Screenshots

    Issue #1186

    "When clicked on Review Rubric"

    'Initial Page'- Assignments tab selected


    The Questionnaires tab gets selected and Review rubric is auto-loaded.

    intial Manage Page

    Review Rubric Page



    "when clicked on Meta Review Rubric"

    Meta Rubric Page


    Issue #1186

    "On edit assignment"

    'Initial Page'- Assignments tab selected - editing assignment



    "on selecting from drop-down"

    User alert



    "on clicking cancel in alert"

    After Cancel in alert


    "on clicking ok in alert"

    After Ok in alert


    "mail received"

    Reviewer mailed


    "review response deleted"

    Review response deleted