CSC/ECE 517 Fall 2021 - E2156. Issues related to meta-reviewing

Introduction

Expertiza Background

Expertiza is a open source project developed on Ruby on Rails. This web application is maintained by the student and faculty at NC State. This application gives complete control to the instructor to maintain the assignments in their class. With multiple functionalities such as adding topics, creating groups, and peer reviews, Expertiza is a well developed application that can handle all types of assignments. To learn more about the full functionality Expertiza has to offer, visit the Expertiza wiki.

Expertiza Meta-Reviewing

In Expertiza, meta-review is a feature that enables a third party to evaluate the quality of feedback that a reviewer gives a reviewee. meta-review is an important feature that can help students become better reviewers for their peers by providing detailed feedback that is actionable and positively formulated. Unfortunately, this feature is broken and the following issues were identified. The task is to fix these issues so that this feature can be used again.

Previous Work

This assignment is a continuation of team E2025 and E1997.

Previous implementations can be found at E2025 and E1997

Meta-Review Issues

Issue 1

Description: When a user requests a new meta-review, (s)he is unable to get 'Begin' link to start a new meta-review.

Issue 2

Description: When a user requests a new meta-review, the 'number of meta-reviews left' does not change and the button to request a new meta-review disappears.

Issue 3

Description: 'Has Meta-Review Limit?' and 'Set Allowed/Required number of meta-reviews per Reviewer' should only be shown if the instructor has checked the 'Use meta-review deadline' in the Due Dates tab of edit assignment.

Issue 4

Description: In the database (assignment table), the default number of meta-reviews allowed and required should be null. When an instructor checks 'Has meta-review Limit' box no the review strategy tab, the UI should fill in 3 and 3 as the required and allowed number.

Proposed Solution

1. Insert a hyper-link on 'Begin' that links to the newly created meta-review form.

2. Decrement 'number of meta-reviews left' by one if the meta-review is assigned successfully. Only deactivate the button to request a new meta-review when 'number of meta-reviews left' is zero.

3. Add an if statement in the _review_strategy.html.erb partial to the section that deals with meta reviews. The if statement will check if the 'Use meta-review deadline' box has been checked. This will make sure only the appropriate fields are shown.

4. This has already been fixed in the previous work, although there are some magic numbers need to be fixed.

Testing

Perform a Meta-Review as Student

1. Login to Expertiza as student
2. Select an assignment
3. Select "Others work"
4. Click on the button "Request a new meta review" and a new review will be dynamically assigned for metareview.

Add Meta-Review option as Instructor

1. Login to Expertiza as instructor
2. From Manage>Assignments, click on the Edit Assignment button for the appropriate assignment.
3. On the Due Date tab, the instructor can select if they want to have meta-reviews done for the assignment, by clicking on the “Use meta-review deadline” checkbox and setting a meat-review deadline below.
4. Under the “Review Strategy” tab of edit assignment, instructor can set Allowed and Required number of meta-reviews per reviewer by checking the "Has meta-review Limit?" checkbox