CSC/ECE 517 Fall 2021 - E2156. Issues related to meta-reviewing

Introduction

Expertiza Meta-Reviewing

In Expertiza, meta-review is a feature that enables a third party to evaluate the quality of feedback that a reviewer gives a reviewee. meta-review is an important feature that can help students become better reviewers for their peers by providing detailed feedback that is actionable and positively formulated. Unfortunately, this feature is broken and the following issues were identified. The task is to fix these issues so that this feature can be used again.

Previous Work

This assignment is a reimplementation of projects E2025 and E1997.

Previous implementations can be found at E2025 and E1997

Meta-Review Issues

Issue 1

Description: When a user requests a new meta-review, (s)he is unable to get 'Begin' link to start a new meta-review.

Issue 2

Description: When a user requests a new meta-review, the 'number of meta-reviews left' does not change and the button to request a new meta-review disappears.

Refer to image above

Issue 3

Description: 'Has Meta-Review Limit?' and 'Set Allowed/Required number of meta-reviews per Reviewer' should only be shown if the instructor has checked the 'Use meta-review deadline' in the Due Dates tab of edit assignment.

Issue 4

Description: In the database (assignment table), the default number of meta-reviews allowed and required should be null. When an instructor checks 'Has meta-review Limit' box no the review strategy tab, the UI should fill in 3 and 3 as the required and allowed number.

Proposed Solution

Issue 1

In _responses.html.erb, we will modify the else if condition to pass only when an assignment is ready to be meta-reviewed. Else if to be modified is shown below:

<% elsif @assignment.get_current_stage(topic_id) != "Complete" && ((map.type.to_s == "MetareviewResponseMap" 
&& @assignment.metareview_allowed(topic_id)) || @assignment.can_review(topic_id)) && latest_submission%>

Issue 2

Decrement 'number of meta-reviews left' by one if the meta-review is assigned successfully. Only deactivate the button to request a new meta-review when 'number of meta-reviews left' is zero.

Issue 3

Add an if statement in the _review_strategy.html.erb partial to the section that deals with meta reviews. The if statement will check if the 'Use meta-review deadline' box has been checked. This will make sure only the appropriate fields are shown.

Issue 4

This has already been fixed in the previous work, although there are some magic numbers need to be fixed.

Testing

Perform a Meta-Review as Student (Manual)

1. Login to Expertiza as student
2. Select an assignment
3. Select "Others work"
4. Click on the button "Request a new meta review" and a new review will be dynamically assigned for metareview.

Add Meta-Review option as Instructor (Manual)

1. Login to Expertiza as instructor
2. From Manage>Assignments, click on the Edit Assignment button for the appropriate assignment.
3. On the Due Date tab, the instructor can select if they want to have meta-reviews done for the assignment, by clicking on the “Use meta-review deadline” checkbox and setting a meat-review deadline below.
4. Under the “Review Strategy” tab of edit assignment, instructor can set Allowed and Required number of meta-reviews per reviewer by checking the "Has meta-review Limit?" checkbox

Automated Tests

• Due to scope of this project and the need to have a automated test for a functionality that has many changes of breakage, we will be borrowing testing code done by E2168. Testing - Reputations.
• To use the automated tests for Meta-Reviewing, please follow the documentation provided by E2168 regarding Assignments.