CSC/ECE 517 Spring 2020 - E2025: Issues Related to Meta - Reviewing

Overview of Project

In Expertiza, meta-review is a feature that enables a third party to evaluate the quality of feedback that a reviewer gives a reviewee. meta-review is an important feature that can help students become better reviewers for their peers by providing detailed feedback that is actionable and positively formulated. Unfortunately, this feature is broken and the following issues were identified. The task is to fix these issues so that this feature can be used again.

• An assignment is deemed to have meta-reviews if and only if there is a meta-review deadline for this assignment in the `assignments_questionnaires` table.
• If there is no meta-review deadline, then don’t show the question about whether there is a max meta-review limit.
• In the DB the default number of meta-reviews allowed and required should be null.
• As soon as someone checks the Has meta-review Limit box on the Review Strategy tab, the UI should fill in 3 and 3 as the required and allowed number.
• Student pages should show the list of reviews that have been performed.
• When a user requests a new meta-review, the field which says the number of meta-reviews left ( refer to above image “Numbers of Meta-reviews left:”) does not change and the button to request a new meta-review disappears.

Previous Work

This assignment is a repeat of E1997 which was assigned to students last semester, Fall 2019.

Details of the previous implementation can be found here: E1997

Our goals are to update the logic, refactor non-DRY code, and fix bugs.

Meta-Review Flow Design

This diagram describes the interactions of the meta-review system.(From E1997)

Issue 1 - Removing Debugging Comments

Issue Description

It was noted in the previous team's implementation that multiple debugging comments were showing up during their video testing.

Debug messages shown when viewing a completed meta review:

Proposed Changes

Review MetaviewResponseMap.html.erb that deals with displaying the review and remove the debugging code.

Issue 2 - Request a new Review Button

Issue Description

When there are no valid metareviews to perform (either there are no reivews performed on the assignment, the only valid reviews concern your own work or own reviews, you have completed all allowed metareviews, or you have reviewed all valid reviews already) then the "Request a New Review" button still is visible and displays the following error (correctly) when pressed.

Reproducing Existing Issue

1. Log in as instructor6, click on mange...-> Assignments

2. Click on edit assignment, navigate to Due Dates and check "Use meta-review deadline" checkbox

3. Select "Yes" for all submission allowed, Review allowed and Meta-review allowed in all of the deadline types

4. Navigate back to Assignment Tree View and click on "Add participant"

5. Add couple of students(say student1 and student2) to the assignment

6. Impersonate/login as student1 and make a submission

7. Log in as instructor6 as previously described and change the submission due date to be passed so we are currently in the review stage

8. Impersonate/login as student2 and Request for a new review under others work, submit the review

9. Log in as instructor6 as previously described and change the review due date to be passed so we are currently in the meta-review stage

10. Impersonate/login back as student1 (the one with the submission) and Request for a new meta review

Proposed Changes

This issue deals with the view "list.html.erb" under /app/views/student_review.

<% elsif @num_metareviews_in_progress == 0 %> 
   <%= render :partial => 'set_dynamic_metareview', :locals => {:assignment => @assignment} %>
<% end %>

Here we will conditionally hide the button based on checks that are already done on the database side, but now can be used on the view side. These checks are located in assignment.rb which is the Model for Assignment.

Issue 3 - Private Methods

Issue Description

Within the code for the meta-reviewing, it is noted that many statements could be moved to private methods to avoid conflating the purpose of the method with meta-reviewing specifics. We will use the code climate as a guide to reduce the block size and use existing or new private methods when possible to make the logic of the program easier to follow.

Proposed Changes

On first inspection, this will mainly apply to the code that is located in the Response Controller (response_conotroller.rb) and the view for "list.html.erb" under app/views/student_review.

For example: The "new" method has meta review checks that should probably be abstracted and placed as private methods.

We will also continue applying this to any changes that we make across the course of the project. This also will apply to the scripts that are used in the _due_dates.html.erb and the _review_strategy.html.erb within the /app/app/views/assignments/edit folder. They should be extracted to somewhere else.

Issue 4 - Creating Tests

Issue Description

There are currently no spec files associated with meta-reviewing. We will need to create these files and test according to the test plan listed below.

Proposed Changes

We will add tests to a new file called meta_review_spec from the user perspective and edit the assignment_creation_spec from the instructors perspective.

Issue 5 - Defaulting Required and Allowed Meta-reviews to NULL in Assignment Database

Issue Description

Currently, the number of required and allowed meta-reviews is always being set to 3 in the database. Per the specs of the feature, it should default the database values to NULL. When checkbox "Use meta-review deadline", in the "Due Dates" tab, is selected then the view should populate required and allowed meta-reviews to 3.

Reproducing Existing Issue

1. Log in as instructor6, click on mange...-> Assignments

2. Click on edit assignment, navigate to Due Dates and confirm that "Use meta-review deadline" checkbox is unchecked

3. Use rails c to access the DB

4. Observe that all assignments contain the incorrect db value (3 for all) for Assignments in the db

Proposed Change

This will just require making a new migration in the Assignment database to set the default to NULL instead of 3, which we believe it is now on default. This may also bring up other issues in the code in the view by allowing assignments without meta-review limits, but we are unable to see that at this time.

file: schema.rb

 create_table "assignments", force: :cascade do |t|
   ...
   t.boolean  "is_anonymous",                                                     default: true
   t.integer  "num_reviews_required",                               limit: 4,     default: 3
   t.integer  "num_metareviews_required",                           limit: 4,     default: NULL
   t.integer  "num_metareviews_allowed",                            limit: 4,     default: NULL
   t.integer  "num_reviews_allowed",                                limit: 4,     default: 3
   t.integer  "simicheck",                                          limit: 4,     default: -1
   ...
 end

file: _review_strategy.html.erb

This file has all the logic to populate the view with a default value once the "Use meta-review deadline" is checked. No modifications needed.

Issue 6 - Meta Review Limit is Not Functioning Correctly in Assignment Creation

Issue Description

The "has meta review limit" should determine whether a limit or requirement is placed on the student to complete meta reviews. If disabled, then they should be able to complete as many as they want. This is independent of the Meta Review STAGE that is enabled/disabled in the due dates tab. They should not impact each other and currently do within the assignment creation.

Reproducing Existing Issue

1. If the "Use meta-review deadline" is disabled, then the Meta Review Limit panel disappears

2. "Has meta-review limit?" is hidden on edit

3. Possibility the "Has meta-review limit?" not being saved on assignment creation?

4. If the "Use meta-review deadline" is toggled (disabled, enabled), then the "Has meta-review limit?" resets to disabled

Proposed Changes

The code in _due_dates.html.erb and the _review_strategy.html.erb within the /app/app/views/assignments/edit folder is lazily written and does not really adhere to the project specifications of how metareviews should work on assignment creation. These views need to be edited to work correctly and also respond to the changes in issue 5.

Testing

Manual Testing

1. Log in as super_administrator2. Password is "password". Click on mange...-> Assignments.

2. Create a new Assignment by clicking the "+" button. Give the assignment these fields: Name, Course and Submission Directory.

3. Go to "Due Dates" tab and check "Use meta-review deadline" and then set the due dates 1 into the FUTURE for each review round and meta-review. Leave everything else as default as shown below:

4. Go to the "Rubrics" tab and populate as shown below:

5. OPTIONAL: Go to "Review strategy page and select "Has meta-review limit?" and set the number of allowed and required meta-reviews.

6. Once done, "Create" the assignment.

7. Go back to Assignments. manage...-> Assignments.

8. Find the newly created assignment and click on "Add participant". Add the students below:

9. Go to manage...Impersonate User and impersonate "Student485"

10. Go to the newly created assignment and submit a link for the assignment.

11. Revert back to super_administrator2 and change the submission due date to a date in the PAST so we are currently in the review stage.

12. Impersonate student486 and request for a new review under others work, submit the review.

13. Revert back to super_administrator2 as previously described and change the review due date to be passed so we are currently in the meta-review stage.

14. Impersonate as student485 and Request for a new meta review.

15. Press "Begin" and complete the meta-review

16. When the review is complete, the "Begin" should disappear and be replaced with "View" and "Edit"

Automated Test Plan

1. If I create an assignment, then the number of meta-reviews is set to null.

2. If 'Use metareview deadline' is not enabled in the due dates tab, then the meta-review fields in the review strategy tab are not visible.

3. If the metareview limit on the assignment is set to 3, then a student will see they need to submit 3 meta reviews

4. If the metareview deadline is enabled, the assignment cannot be created without a metareview rubric

5. If the assignment created is in a stage that does not have metareview enabled, a metareview cannot be created

6. If the assignment does not have any legal meta reviews, then the "Request a new metareview to perform" button is not visible

7. A student should not be able to request a metareview if they have reached the limit of their allowed reviews

8. A student should be able to request a metareview if they are above their required but below their allowed reviews

9. A student should not be able to request a metareview about their own work.

10. A student should not be able to request a metareview about themselves.

11. If there is a valid metareview to perform related to two other students, then the metareview should be able to be performed.

12. If a student has requested two metareviewes but have not submitted it, then they should not be able to request a new metareview.

13. If the metareview limit on the assignment is set to 1 then a student should not be able to request a second meta review

Coverage

Current test coverage is 41.348%. Code coverage will increase based on the test plan above and the failing test case described below will be fixed.

Travis CI is currently failing because of:

 Failures:
 1) assignment function general tab should edit quiz number available to students
    Failure/Error:
      child_nodes.each do |node|
        initialize_fnode_update_children(params, node, tmp_res)
      end
    NoMethodError:
      undefined method `each' for "":String
    # ./app/controllers/tree_display_controller.rb:209:in `children_node_ng'
    # ./config/diagnostic.rb:11:in `call'
    # ------------------
    # --- Caused by: ---
    # Capybara::CapybaraError:
    #   Your application server raised an error - It has been raised in your test code because Capybara.raise_server_errors == true
    #   /home/travis/build/expertiza/expertiza/vendor/bundle/ruby/2.2.0/gems/capybara-2.17.0/lib/capybara/session.rb:145:in `raise_server_error!'