CSC/ECE 517 Spring 2020 - E2025: Issues Related to Meta - Reviewing

Overview of Project

In Expertiza, meta-review is a feature that enables a third party to evaluate the quality of feedback that a reviewer gives a reviewee. meta-review is an important feature that can help students become better reviewers for their peers by providing detailed feedback that is actionable and positively formulated. Unfortunately, this feature is broken and the following issues were identified. The task is to fix these issues so that this feature can be used again.

• An assignment is deemed to have meta-reviews if and only if there is a meta-review deadline for this assignment in the `assignments_questionnaires` table.
• If there is no meta-review deadline, then don’t show the question about whether there is a max meta-review limit.
• In the DB the default number of meta-reviews allowed and required should be null.
• As soon as someone checks the Has meta-review Limit box on the Review Strategy tab, the UI should fill in 3 and 3 as the required and allowed number.
• Student pages should show the list of reviews that have been performed.

Previous Project

Details of the previous implementation can be found here: E1997

Meta-Review Flow Design

Because our changes to the code is entirely logical, refactoring, and bug fixing, the existing design describes the interactions of the system that will continue after our involvement.

(From E1997)

Issue 1 - Removing Debugging Comments

Issue Description

It was noted in the previous team's implementation that multiple debugging comments were showing up during their video testing

Proposed Changes

Review MetaviewResponseMap.html.erb that deals with displaying the review and remove the debugging code.

Issue 2 - Request a new Review Button

Issue Description

When there are no valid metareviews to perform (either there are no reivews performed on the assignment, the only valid reviews concern your own work or own reviews, you have completed all allowed metareviews, or you have reviewed all valid reviews already) then the "Request a New Review" button still is visible and displays the following error (correctly) when pressed.

Reproducing Existing Issue

1. Log in as instructor6, click on mange...-> Assignments

2. Click on edit assignment, navigate to Due Dates and check "Use meta-review deadline" checkbox

3. Select "Yes" for all submission allowed, Review allowed and Meta-review allowed in all of the deadline types

4. Navigate back to Assignment Tree View and click on "Add participant"

5. Add couple of students(say student1 and student2) to the assignment

6. Impersonate/login as student1 and make a submission

7. Log in as instructor6 as previously described and change the submission due date to be passed so we are currently in the review stage

8. Impersonate/login as student2 and Request for a new review under others work, submit the review

9. Log in as instructor6 as previously described and change the review due date to be passed so we are currently in the meta-review stage

10. Impersonate/login back as student1 (the one with the submission) and Request for a new meta review

Proposed Changes

This issue deals with the view "list.html.erb" under /app/views/student_review. Here we will conditionally hide the button based on checks that are already done on the database side, but now can be used on the view side. These checks are located in assignment.rb which is the Model for Assignment.

Issue 3 - Private Methods

Issue Description

Within the code for the meta-reviewing, it is noted that many statements could be moved to private methods to avoid conflating the purpose of the method with meta-reviewing specifics. We will use the code climate as a guide to reduce the block size and use existing or new private methods when possible to make the logic of the program easier to follow.

Proposed Changes

On first inspection, this will mainly apply to the code that is located in the Response Controller (response_conotroller.rb) and the view for "list.html.erb" under app/views/student_review. We will also continue applying this to any changes that we make across the course of the project.

This also will apply to the scripts that are used in the _due_dates.html.erb and the _review_strategy.html.erb within the /app/app/views/assignments/edit folder. They should be extracted to somewhere else.

Issue 4 - Creating Tests

Issue Description

There are currently no spec files associated with meta-reviewing. We will need to create these files and test according to the test plan listed below.

Proposed Changes

We will add tests to the following files where necessary to match the testing for normal reviews:

assignments_controller_spec.rb

assignment_form_spec.rb

assignment_participant_spec.rb

reports_controller_spec.rb

response_controller_spec.rb

review_mapping_controller_spec.rb

review_mapping_helper_spec.rb

review_mapping_spec.rb

student_review_controller_spec.rb

student_task_controller_spec.rb

Issue 5 - Defaulting Required and Allowed Meta-reviews to NULL in Assignment Database

Issue Description

Currently, the number of default and allowed meta-reviews is always being set to 3. Per the specs of the feature, it should default to NULL.

Reproducing Existing Issue

1. Log in as instructor6, click on mange...-> Assignments

2. Click on edit assignment, navigate to Due Dates and confirm that "Use meta-review deadline" checkbox is unchecked

3. Use rails c to access the DB

4. Observe that all assignments contain the incorrect db value (3 for all) for Assignments in the db

Proposed Change

This will just require making a new migration in the Assignment database to set the default to NULL instead of 3, which we believe it is now on default. This may also bring up other issues in the code in the view by allowing assignments without metareview limits, but we are unable to see that at this time.

Issue 6 - Meta Review Limit is Not Functioning Correctly in Assignment Creation

Issue Description

The "has meta review limit" should determine whether a limit or requirement is placed on the student to complete meta reviews. If disabled, then they should be able to complete as many as they want. This is independent of the Meta Review STAGE that is enabled/disabled in the due dates tab. They should not impact each other and currently do within the assignment creation.

Reproducing Existing Issue

1. If the "Use meta-review deadline" is disabled, then the Meta Review Limit panel disappears

2. "Has meta-review limit?" is hidden on edit

3. Possibility the "Has meta-review limit?" not being saved on assignment creation?

4. If the "Use meta-review deadline" is toggled (disabled, enabled), then the "Has meta-review limit?" resets to disabled

Proposed Changes

The code in _due_dates.html.erb and the _review_strategy.html.erb within the /app/app/views/assignments/edit folder is lazily written and does not really adhere to the project specifications of how metareviews should work on assignment creation. These views need to be edited to work correctly and also respond to the changes in issue 5.

Testing

Manual Testing

1. Log in as instructor6, click on mange...-> Assignments

2. Click on edit assignment, navigate to Due Dates and check "Use meta-review deadline" checkbox

3. Select "Yes" for all submission allowed, Review allowed and Meta-review allowed in all of the deadline types

4. Navigate back to Assignment Tree View and click on "Add participant"

5. Add couple of students(say student1, student2, and student3) to the assignment

6. Impersonate/login as student1 and make a submission

7. Log in as instructor6 as previously described and change the submission due date to be passed so we are currently in the review stage

8. Impersonate/login as student2 and Request for a new review under others work, submit the review

9. Log in as instructor6 as previously described and change the review due date to be passed so we are currently in the meta-review stage

10. Impersonate/login back as student3 and Request for a new meta review

11. Press "Begin" and complete the meta-review

12. When the review is complete, the "Begin" should disappear and be replaced with "View" and "Edit"

Automated Test Plan

1. If I create an assignment, then the number of meta-reviews is set to null.

2. If 'Use metareview deadline' is not enabled in the due dates tab, then the meta-review fields in the review strategy tab are not visible.

3. If the metareview limit on the assignment is set to 3, then a student will see they need to submit 3 meta reviews

4. If the metareview deadline is enabled, the assignment cannot be created without a metareview rubric

5. If the assignment created is in a stage that does not have metareview enabled, a metareview cannot be created

6. If the assignment does not have any legal meta reviews, then the "Request a new metareview to perform" button is not visible

7. A student should not be able to request a metareview if they have reached the limit of their allowed reviews

8. A student should be able to request a metareview if they are above their required but below their allowed reviews

9. A student should not be able to request a metareview about their own work.

10. A student should not be able to request a metareview about themselves.

11. If there is a valid metareview to perform related to two other students, then the metareview should be able to be performed.

12. If a student has requested two metareviewes but have not submitted it, then they should not be able to request a new metareview.

13. If the metareview limit on the assignment is set to 1 then a student should not be able to request a second meta review

Coverage

Before implementation of the tests, coverage is 41.348%