CSC/ECE 517 Fall 2021 - E2156. Issues related to meta-reviewing

From Expertiza_Wiki
Revision as of 19:30, 11 November 2021 by Admin (talk | contribs) (→‎Testing)
Jump to navigation Jump to search

Introduction

Expertiza Meta-Reviewing

In Expertiza, meta-review is a feature that enables a third party to evaluate the quality of feedback that a reviewer gives a reviewee. meta-review is an important feature that can help students become better reviewers for their peers by providing detailed feedback that is actionable and positively formulated. Unfortunately, this feature is broken and the following issues were identified. The task is to fix these issues so that this feature can be used again.

Previous Work

This assignment is a reimplementation of projects E2025 and E1997.

Previous implementations can be found at E2025 and E1997

Meta-Review Issues

Issue 1

Description: When a user requests a new meta-review, (s)he is unable to get 'Begin' link to start a new meta-review.

Issue 2

Description: When a user requests a new meta-review, the 'number of meta-reviews left' does not change and the button to request a new meta-review disappears.

Refer to image above

Issue 3

Description: 'Has Meta-Review Limit?' and 'Set Allowed/Required number of meta-reviews per Reviewer' should only be shown if the instructor has checked the 'Use meta-review deadline' in the Due Dates tab of edit assignment.

Issue 4

Description: In the database (assignment table), the default number of meta-reviews allowed and required should be null. When an instructor checks 'Has meta-review Limit' box no the review strategy tab, the UI should fill in 3 and 3 as the required and allowed number.

Proposed Solutions

Issue 1

During testing we found that the Begin link was working as expected. It should not link to the questionnaire if work has not been submitted and that work has not been reviewed as shown in the image in Issue 1. We believe this was an issue either resolved previously, or someone tried to test the function without creating and completing a review.

Issue 2

The current code is: 

 <h4> Number of Meta-Reviews left: <%= @assignment.num_metareviews_allowed - @num_metareviews_completed %></h4>

We believe this line should read the number of reviews you're allowed to request. This is an easy fix: 

 <h4> Number of Meta-Reviews left: <%= @assignment.num_metareviews_allowed - @num_metareviews_total %></h4>

Issue 3

Add an if statement in the _review_strategy.html.erb partial to the section that deals with meta reviews. The if statement will check if the 'Use meta-review deadline' box has been checked. This will make sure only the appropriate fields are shown.

This is the current code: 

 <tr id="assignment_review_topic_threshold_row">
    <td>
        <span id="meta_reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_metareviews_allowed.nil?%> class="form-inline">

        <%= label_tag('assignment_form[assignment][num_metareviews_allowed]', ' Set allowed number of meta-reviews per reviewer') %>
          <%if @assignment_form.assignment.num_metareviews_allowed && @assignment_form.assignment.num_metareviews_allowed >= 0 %>
                <%= text_field_tag('assignment_form[assignment][num_metareviews_allowed]', @assignment_form.assignment.num_metareviews_allowed ||= 3, size: 1, class: "form-control width-70", :style => 'margin-left :100px') %>
           <%else%>
          <%= text_field_tag('assignment_form[assignment][num_metareviews_allowed]',  3, size: 1, class: "form-control width-70", :style => 'margin-left :100px') %>
            <%end%>
          </br>
          <%= label_tag('assignment_form[assignment][num_metareviews_allowed]', 'Set required number of meta-reviews per reviewer') %>
          <%if @assignment_form.assignment.num_metareviews_allowed && @assignment_form.assignment.num_metareviews_allowed >= 0 %>
              <%= text_field_tag('assignment_form[assignment][num_metareviews_allowed]', @assignment_form.assignment.num_metareviews_allowed ||= 3, size: 1, class: "form-control width-70", :style => 'margin-left :90px') %>
        <%else%>
              <%= text_field_tag('assignment_form[assignment][num_metareviews_allowed]',  3, size: 1, class: "form-control width-70", :style => 'margin-left :90px') %>
        <%end%>
        </span>
    </td>
  </tr>

This is the changed code. 

<%if @metareview_allowed_checkbox == true%>
 <tr id="assignment_review_topic_threshold_row">
    <td>
        <span id="meta_reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_metareviews_allowed.nil?%> class="form-inline">

        <%= label_tag('assignment_form[assignment][num_metareviews_allowed]', ' Set allowed number of meta-reviews per reviewer') %>
          <%if @assignment_form.assignment.num_metareviews_allowed && @assignment_form.assignment.num_metareviews_allowed >= 0 %>
                <%= text_field_tag('assignment_form[assignment][num_metareviews_allowed]', @assignment_form.assignment.num_metareviews_allowed ||= 3, size: 1, class: "form-control width-70", :style => 'margin-left :100px') %>
           <%else%>
          <%= text_field_tag('assignment_form[assignment][num_metareviews_allowed]',  3, size: 1, class: "form-control width-70", :style => 'margin-left :100px') %>
            <%end%>
          </br>
          <%= label_tag('assignment_form[assignment][num_metareviews_allowed]', 'Set required number of meta-reviews per reviewer') %>
          <%if @assignment_form.assignment.num_metareviews_allowed && @assignment_form.assignment.num_metareviews_allowed >= 0 %>
              <%= text_field_tag('assignment_form[assignment][num_metareviews_allowed]', @assignment_form.assignment.num_metareviews_allowed ||= 3, size: 1, class: "form-control width-70", :style => 'margin-left :90px') %>
        <%else%>
              <%= text_field_tag('assignment_form[assignment][num_metareviews_allowed]',  3, size: 1, class: "form-control width-70", :style => 'margin-left :90px') %>
        <%end%>
        </span>
    </td>
  </tr>
<%end%>

Issue 4

In _review_strategy.html.erb, the code was hardwired to not save the selection provided by the user which resulted in 3 not being set on default. The issue and change is shown below.

Current Code: 

 <%= check_box_tag('meta_review', 'true', true, {:onChange => 'hasMetaReviewChanged()'}) %>

Changed Code: 

 <%= check_box_tag('meta_review', 'true', @metareview_allowed, {:onChange => 'hasMetaReviewChanged()'}) %>

We are also going to have to change the onclick javascript that is run. We need to add in the line: 

 <%@metareview_allowed_checkbox=true%>

Testing

Perform a Meta-Review as Student (Manual)

  1. Login to Expertiza as student
  2. Select an assignment
  3. Select "Others work"
  4. Click on the button "Request a new meta review" and a new review will be dynamically assigned for metareview.

Add Meta-Review option as Instructor (Manual)

  1. Login to Expertiza as instructor
  2. From Manage>Assignments, click on the Edit Assignment button for the appropriate assignment.
  3. On the Due Date tab, the instructor can select if they want to have meta-reviews done for the assignment, by clicking on the “Use meta-review deadline” checkbox and setting a meat-review deadline below.
  4. Under the “Review Strategy” tab of edit assignment, instructor can set Allowed and Required number of meta-reviews per reviewer by checking the "Has meta-review Limit?" checkbox

Automated Tests

  • Due to scope of this project and the need to have a automated test for a functionality that has many changes of breakage, we will be borrowing testing code done by E2132 - Add test cases to review_mapping_helper.rb (this code is also being used by E2168. Testing - Reputations).
  • To use the automated tests for Meta-Reviewing, please follow the documentation provided by E2168 regarding Assignments.
Retrieved from "https://wiki.expertiza.ncsu.edu/index.php?title=CSC/ECE_517_Fall_2021_-_E2156._Issues_related_to_meta-reviewing&oldid=141603"