CSC/ECE 517 Fall 2019 - E1997. Issues related to meta-reviewing

Overview of Project

In Expertiza, meta-review is a feature that enables a third party to evaluate the quality of feedback that a reviewer gives a reviewee. meta-review is an important feature that can help students become better reviewers for their peers by providing detailed feedback that is actionable and positively formulated. Unfortunately, this feature is broken and the following issues were identified.

Issue1

Issue Description

When a user requests a new meta-review, (s)he should be able to click on the Begin button, which should redirect him/her to the meta-review questionnaire and when the user returns back (to the page where he can view the reviews and meta-reviews, by clicking on Others’ work in his/her assignment), (s)he should be then able to see view and update links for the meta-reviews (s)he has performed.

But the issue is Begin and Edit are not working for the given scenario.

Test Plan

1. Log in as instructor6, click on Manage...-> Assignments

2. Click on edit assignment, navigate to Due Dates and check "Use meta-review deadline" checkbox

3. Select "Yes" for all of the following: Submission allowed, Review allowed and Meta-review allowed in all of the deadline types

4. Navigate back to Assignment Tree View and click on "Add participant"

5. Add couple of students(say student1 and student2) to the assignment

6. Impersonate/login as student1 and make a submission

7. Impersonate/login as student2 and Request for a new review under others work, submit the review

8. Impersonate/login back as student1 and Request for a new meta review

9. When Begin is clicked following will be displayed

10. A template must be added to fix the broken link

Files to be modified

response.html.erb

Issue2

Issue Description

The number of meta-reviews(refer to the snapshot below),should decrease when a user requests a meta-review.Also, the button to request a new meta-review should not disappear until the total number of reviews assigned has reached the allowed limit.

Test Plan

Log in as instructor6, click on mange...-> Assignments

Click on edit assignment, navigate to Due Dates and check "Use meta-review deadline" checkbox

Select "Yes" for all submission allowed, Review allowed and Meta-review allowed in all of the deadline types

Navigate back to Assignment Tree View and click on "Add participant"

Add couple of students(say student1 and student2) to the assignment

Impersonate/login as student1 and make a submission

Impersonate/login as student2 and Request for a new review under others work, submit the review

Impersonate/login back as student1 and Request for a new meta review

When Begin is clicked, the meta-review count should be decremented

Files to be modified

List.html.erb

Design strategy For Issue1 and Issue 2

The flow chart below describes the design we choose to implement the above two issues

Issue3

Background

1) From Manage>Assignments, click on the Edit Assignment button for the appropriate assignment.

2) On the Due Date tab, the instructor can select if (s)he wants to have meta-reviews done for the assignment, by clicking on the “Use meta-review deadline” checkbox and setting a meta-review deadline below.

3) Under the “Review Strategy” tab of edit assignment, the instructor can set Allowed and Required number of meta-reviews per reviewer by checking the Has meta-review Limit? checkbox.

Issue Description

For an assignment, when 'Use Metareview Deadline' is unchecked, it means meta-reviews are not enabled for that assignment. The checkbox is as below

As said, when this box is unchecked, it means there is no meta reviewing system for this assignment. The issue is, there are some meta review related fields that show up even if the above box is unchecked. Like in below image

Also, when a new issue is created, the default # of meta-reviews allowed and required should be null. As soon as someone checks the Has meta-review Limit box on the “Review Strategy” tab, the UI should fill in 3 and 3 as the required and allowed number. Of course, the user can change this number before submitting it.

Solution and Code Modification

As the checkbox and the content to be displayed on clicking the checkbox we used the concept of local storage and event listener.

we have made changes to two files:

1)/expertiza/app/views/assignments/edit/_due_dates.html.erb 2)/expertiza/app/views/assignments/edit/_review_strategy.html.erb

In the _due_dates.html.erb: There is an existing code where on clicking the checkbox 'Use metareview deadline', a function called 'removeOrAddMetareview()' is called. We made use of this function to call two new functions.

Test Plan

1. Ensure that when an assignment is created, the number of meta reviews is set to null. 2. Ensure that the meta review fields in the review strategy tab are visible only when 'Use metareview deadline' is enabled in Due dates tab.

Files to be modified

1. _review_strategy.html.erb Source file: https://github.com/expertiza/expertiza/blob/master/app/views/assignments/edit/_review_strategy.html.erb

2. _due_dates.html.erb Source file: https://github.com/expertiza/expertiza/blob/master/app/views/assignments/edit/_due_dates.html.erb

3. assignments_controller.rb Source file: https://github.com/expertiza/expertiza/blob/master/app/controllers/assignments_controller.rb