CSC/ECE 517 Spring 2020 - E2025: Issues Related to Meta - Reviewing

Overview of Project

In Expertiza, meta-review is a feature that enables a third party to evaluate the quality of feedback that a reviewer gives a reviewee. meta-review is an important feature that can help students become better reviewers for their peers by providing detailed feedback that is actionable and positively formulated. Unfortunately, this feature is broken and the following issues were identified. The task is to fix these issues so that this feature can be used again.

• An assignment is deemed to have meta-reviews if and only if there is a meta-review deadline for this assignment in the `assignments_questionnaires` table.
• If there is no meta-review deadline, then don’t show the question about whether there is a max meta-review limit.
• In the DB the default number of meta-reviews allowed and required should be null.
• As soon as someone checks the Has meta-review Limit box on the Review Strategy tab, the UI should fill in 3 and 3 as the required and allowed number.
• Student pages should show the list of reviews that have been performed.

Previous Project

Details of the previous implementation can be found here: E1997

Existing Design

(From E1997)

Issue 1 - Removing Debugging Comments

Issue Description

It was noted in the previous team's implementation that multiple debugging comments were showing up during their video testing

Issue 2 - Request a new Review Button

Issue Description

When there are no valid metareviews to perform (either there are no reivews performed on the assignment, the only valid reviews concern your own work or own reviews, you have completed all allowed metareviews, or you have reviewed all valid reviews already) then the "Request a New Review" button still is visible and displays the following error (correctly) when pressed.

Reproducing Existing Issue

1. Log in as instructor6, click on mange...-> Assignments

2. Click on edit assignment, navigate to Due Dates and check "Use meta-review deadline" checkbox

3. Select "Yes" for all submission allowed, Review allowed and Meta-review allowed in all of the deadline types

4. Navigate back to Assignment Tree View and click on "Add participant"

5. Add couple of students(say student1 and student2) to the assignment

6. Impersonate/login as student1 and make a submission

7. Log in as instructor6 as previously described and change the submission due date to be passed so we are currently in the review stage

8. Impersonate/login as student2 and Request for a new review under others work, submit the review

9. Log in as instructor6 as previously described and change the review due date to be passed so we are currently in the meta-review stage

10. Impersonate/login back as student1 (the one with the submission) and Request for a new meta review

Issue 3 - Private Methods

Issue Description

Within the code for the meta-reviewing, it is noted that many statements could be moved to private methods to avoid conflating the purpose of the method with meta-reviewing specifics. We will use the code climate as a guide to reduce the block size and use existing or new private methods when possible to make the logic of the program easier to follow.

Issue 4 - Creating Tests

Issue Description

There are currently no spec files associated with meta-reviewing. We will need to create these files and test according to the test plan listed below.

Issue 5 - Defaulting Required and Allowed Meta-reviews to NULL in Database

Issue Description

Currently, the number of default and allowed meta-reviews is always being set to 3. Per the specs of the feature, it should default to NULL.

Reproducing Existing Issue

1. Log in as instructor6, click on mange...-> Assignments

2. Click on edit assignment, navigate to Due Dates and confirm that "Use meta-review deadline" checkbox is unchecked

3. Use rails c to access the DB

4. Observe that all assignments contain the incorrect db value

Issue 6 - Meta Review Limit Table is Not Functioning Correctly

Issue Description

The "has meta review limit" should determine whether a limit or requirement is placed on the student to complete meta reviews. If disabled, then they should be able to complete as many as they want. This is independent of the Meta Review STAGE that is enabled/disabled in the due dates tab. They should not impact each other and currently do

Reproducing Existing Issue

1. If the "Use meta-review deadline" is disabled, then the Meta Review Limit panel disappears

2. "Has meta-review limit?" is hidden on edit

3. Possibility the "Has meta-review limit?" not being saved on assignment creation?

4. If the "Use meta-review deadline" is toggled (disabled, enabled), then the "Has meta-review limit?" resets to disabled

Testing

Manual Testing

1. Log in as instructor6, click on mange...-> Assignments

2. Click on edit assignment, navigate to Due Dates and check "Use meta-review deadline" checkbox

3. Select "Yes" for all submission allowed, Review allowed and Meta-review allowed in all of the deadline types

4. Navigate back to Assignment Tree View and click on "Add participant"

5. Add couple of students(say student1, student2, and student3) to the assignment

6. Impersonate/login as student1 and make a submission

7. Log in as instructor6 as previously described and change the submission due date to be passed so we are currently in the review stage

8. Impersonate/login as student2 and Request for a new review under others work, submit the review

9. Log in as instructor6 as previously described and change the review due date to be passed so we are currently in the meta-review stage

10. Impersonate/login back as student3 and Request for a new meta review

11. Press "Begin" and complete the meta-review

12. When the review is complete, the "Begin" should disappear and be replaced with "View" and "Update"

Automated Test Plan

1. If I create an assignment, then the number of meta-reviews is set to null.

2. If 'Use metareview deadline' is not enabled in the due dates tab, then the meta-review fields in the review strategy tab are not visible.

3. If the metareview limit on the assignment is set to 3, then a student will see they need to submit 3 meta reviews

4. If the metareview deadline is enabled, the assignment cannot be created without a metareview rubric

5. If the assignment created is in a stage that does not have metareview enabled, a metareview cannot be created

6. If the assignment does not have any legal meta reviews, then the "Request a new metareview to perform" button is not visible

7. A student should not be able to request a metareview if they have reached the limit of their allowed reviews

8. A student should be able to request a metareview if they are above their required but below their allowed reviews

9. A student should not be able to request a metareview about their own work.

10. A student should not be able to request a metareview about themselves.

11. If there is a valid metareview to perform related to two other students, then the metareview should be able to be performed.

12. If a student has requested two metareviewes but have not submitted it, then they should not be able to request a new metareview.

13. If the metareview limit on the assignment is set to 1 then a student should not be able to request a second meta review