CSC/ECE 517 Spring 2022 - E2235. Issues related to meta-reviewing

From Expertiza_Wiki
Revision as of 17:16, 6 April 2022 by Pgarg5 (talk | contribs)
Jump to navigation Jump to search

Overview of Project

In Expertiza, meta-review is a feature that enables a third party to evaluate the quality of feedback that a reviewer gives a reviewee. meta-review is an important feature that can help students become better reviewers for their peers by providing detailed feedback that is actionable and positively formulated. Unfortunately, this feature is broken and the following issues were identified. The task is to fix these issues so that this feature can be used again.

  • When a user requests a new meta-review, (s)he is unable to get Begin (refer to above image) link to start a new meta-review. (Issue #198)
  • When a user requests a new meta-review, the field which says the number of meta-reviews left ( refer to above image “Numbers of Meta-reviews left:”) does not change and the button to request a new meta-review disappears.

Previous Work

This assignment is a repeat of E2025 which was assigned to students last semester, Spring 2020.

Details of the previous implementation can be found here: [CSC/ECE 517 Spring 2020 - E2025: Issues Related to Meta - Reviewing]

Our goals are to update the logic, refactor non-DRY code, and fix bugs.

#198 - Begin not working correctly

Issue Description

In beta branch begin is not working correctly. The page crashes sometime.


Changes

There was a mistake in app/views/response/_review.html.erb where "<br/><hr/><br/>" was being displayed under the review being meta-reviewed. This was due to being included as a string constant rather than html code.

There was an instance in app/views/response/view.html.erb where ruby code responsible for getting a review to be displayed was being interpreted and displayed, even though the response was being displayed as html in the line before. This resulted in the response details being listed in the UI erroneously.

#49 - "Request a new meta-review" button crashes when there are no available reviews

Issue Description

When there are no valid metareviews to perform (either there are no reivews performed on the assignment, the only valid reviews concern your own work or own reviews, you have completed all allowed metareviews, or you have reviewed all valid reviews already) then the "Request a New Review" button still is visible and displays the following error (correctly) when pressed.

Reproducing Existing Issue

1. Log in as instructor6, click on mange...-> Assignments

2. Click on edit assignment, navigate to Due Dates and check "Use meta-review deadline" checkbox

3. Select "Yes" for all submission allowed, Review allowed and Meta-review allowed in all of the deadline types

4. Navigate back to Assignment Tree View and click on "Add participant"

5. Add couple of students(say student1 and student2) to the assignment

6. Impersonate/login as student1 and make a submission

7. Log in as instructor6 as previously described and change the submission due date to be passed so we are currently in the review stage

8. Impersonate/login as student2 and Request for a new review under others work, submit the review

9. Log in as instructor6 as previously described and change the review due date to be passed so we are currently in the meta-review stage

10. Impersonate/login back as student1 (the one with the submission) and Request for a new meta review

Changes

This issue deals with the view "list.html.erb" under /app/views/student_review. The code below shows the conditional used to display the "Request new meta-review" button. 

 <% if @num_metareviews_in_progress >= Assignment.max_outstanding_reviews %>
   Note: You can't have more than 2 outstanding meta-reviews. You must complete one of your
   outstanding meta-reviews before selecting another.
 <% elsif @assignment.num_metareviews_allowed && @num_metareviews_total == @assignment.num_metareviews_allowed %>
   Note: You can not do more than <%=@assignment.num_metareviews_allowed%> meta-reviews according
   to assignment policy.
 <% elsif (!@assignment.metareview_available?(session[:user].id)) %>
   Meta-reviews cannot be performed at this time.
 <% else %>
   <%= render :partial => 'set_dynamic_metareview', :locals => {:assignment => @assignment} %>
 <% end %>

Here we conditionally hide the button based on whether or not the participant has reviews available to perform. These checks are located in assignment.rb which is the model for an Assignment. The following method was added to assignment.rb and returns true when the participant has a valid meta-review available. 

 # Returns true if there is a meta-review task available for the user
 def metareview_available?(user_id)
   metareviewer = AssignmentParticipant.where(user_id: user_id, parent_id: self.id).first
   begin
     self.response_map_to_metareview(metareviewer)
     true
   rescue 
     false
   end
 end

#27 - Defaulting Required and Allowed Meta-reviews to NULL in Assignment Database

Issue Description

Currently, the number of required and allowed meta-reviews is always being set to 3 in the database. Per the specs of the feature, it should default the database values to NULL. When checkbox "Use meta-review deadline", in the "Due Dates" tab, is selected then the view should populate required and allowed meta-reviews to 3.

Changes

We created a migration that defaulted the metareviews_allowed and metareviews_required in the Assignment table to nil instead of what the previous team had as 3.

Since the number of meta-reviews allowed and required are recorded using the values in the text fields, changes were made to app/views/assignment/edit/_review_strategy.html.erb to set the values and display them as expected.

When the checkbox for setting meta-review limits is unchecked, the values of number of meta-reviews allowed and required should be null. When the checkbox is checked, the values should be set to 3 and 3 by default, or to the values already defined before (in case the assignment was saved with the values set already). On page load, if the checkbox is enabled, then the inputs should be shown with the default or pre-set values, but if the checkbox is disabled then they should be hidden and set to null.

Changes had to be made to app/views/student_review/list.html.erb to account for the values being null.

Testing

Tests have been created that test if a metareview can have nil limits. In this case, no limit will be shown to the user. Before this change that was impossible

Issue 5 - Meta Review Limit is Not Functioning Correctly in Assignment Creation

Issue Description

The "has meta review limit" should determine whether a limit or requirement is placed on the student to complete meta reviews. If disabled, then they should be able to complete as many as they want. This is independent of the Meta Review STAGE that is enabled/disabled in the due dates tab. They should not impact each other and currently do within the assignment creation.

Reproducing Existing Issue

1. If the "Use meta-review deadline" is disabled, then the Meta Review Limit panel disappears

2. "Has meta-review limit?" is hidden on edit

3. Possibility the "Has meta-review limit?" not being saved on assignment creation?

4. If the "Use meta-review deadline" is toggled (disabled, enabled), then the "Has meta-review limit?" resets to disabled

Proposed Changes

The code in _due_dates.html.erb and the _review_strategy.html.erb within the /app/app/views/assignments/edit folder is lazily written and does not really adhere to the project specifications of how metareviews should work on assignment creation. These views need to be edited to work correctly and also respond to the changes in issue 5.

Changes

The changes were too long to screenshot, so here is the specific diff section:

https://github.com/expertiza/expertiza/pull/1715/files#diff-540fd2bd6c34b48213716b1dd713733a https://github.com/expertiza/expertiza/pull/1715/files#diff-80f39fa5f62c73ff8741b02d3b11656c

Discovered Bugs:

#45 - "Request a new meta-review" button bugs

Issue Description

The "Request new a meta-review" button should be hidden when:

  1. The number of meta-reviews in-progress is greater than the maximum allowed outstanding reviews.
  2. The number of reviews allowed is limited AND the total number of meta-reviews completed or in-progress is greater than the number allowed.
  3. There are no valid reviews available for the participant.

Changes

This issue deals with the view "list.html.erb" under /app/views/student_review. The code below shows the conditional used to display the "Request new meta-review" button. 

 <% if @num_metareviews_in_progress >= Assignment.max_outstanding_reviews %>
   Note: You can't have more than 2 outstanding meta-reviews. You must complete one of your
   outstanding meta-reviews before selecting another.
 <% elsif @assignment.num_metareviews_allowed && @num_metareviews_total == @assignment.num_metareviews_allowed %>
   Note: You can not do more than <%=@assignment.num_metareviews_allowed%> meta-reviews according
   to assignment policy.
 <% elsif (!@assignment.metareview_available?(session[:user].id)) %>
   Meta-reviews cannot be performed at this time.
 <% else %>
   <%= render :partial => 'set_dynamic_metareview', :locals => {:assignment => @assignment} %>
 <% end %>

Testing

Functional tests from a student perspective are in /spec/features/meta_review_spec.rb. The tests assume the allowed number of meta-reviews is 5 and the required number of meta-reviews is 3. Tests for this feature are explained in summary below:

  • For assignments that have a meta-review deadline...
    • with 0 reviewers. The user should NOT be able to see the 'Request a new meta-review to perform' button when there are no reviews available.
    • with 1 reviewer.
      • A student should NOT be able to request a meta-review on their own review.
      • When the meta-review limit on the assignment is set to 1 then a student should NOT be able to request a second meta-review.
      • The user should not be able to see the 'Request a new meta-review to perform' button when they have performed all valid reviews.
    • with 4 reviewers.
      • The user should be able to click the 'Begin' button and still see the 'Request a new meta-review to perform' button.
      • The user should NOT be able to see the 'Request a new meta-review to perform' button when they have reached the meta-review limit.
      • If the user requests two meta-reviews but has not submitted, then they should not be able to request a new meta-review.
      • The user should be able to request a meta-review if they are above their required but below the allowed number of meta-reviews.

#30 - “Numbers of Meta-reviews left:” does not update after a user requests a review.

Issue Description

Number of meta-reviews left should decrement when a user requests a meta-review. Current behavior only decremented when a review was submitted.

Changes

Previous behavior: 

Number of Meta-Reviews left: <%= @assignment.num_metareviews_allowed - @num_metareviews_completed %>

Fixed behavior: 

Number of meta-reviews left: <%= @assignment.num_metareviews_allowed - @num_metareviews_total %>

Testing

Functional tests from a student perspective are in /spec/features/meta_review_spec.rb. The tests assume the allowed number of meta-reviews is 5 and the required number of meta-reviews is 3. Tests for this feature are explained in summary below:

  • For assignments that have a meta-review deadline...
    • with 1 reviewer.
      • The user should see the number of meta-reviews left decrement after a new meta-review is requested.
      • The user should see the number of meta-reviews left decrement after a new meta-review is completed.

#43 - "Review Allowed" flag is required to be set on a due_date for a user to perform a meta-review.

Issue Description

When an assignment with a meta-review is created the round that meta-reviews will be performed in should have the "Meta-review allowed" option set to yes. This should be the only requirement for making meta-reviews available in that round. The problem with the current implementation is that meta-reviews can only be performed in rounds where the "Review allowed" option is set.

Changes

Code changes were applied to app/views/student_review/_responses.html.erb. Since this partial is shared by both reviews and meta-reviews in list.html.erb the title variable was used to determine if the begin option should be active or not.

Previous behavior: 

<% elsif @assignment.get_current_stage(topic_id) != "Complete" && @assignment.can_review(topic_id) %>

Current behavior: 

<% elsif @assignment.get_current_stage(topic_id) != "Complete" && 
       ((@assignment.can_review(topic_id) && title == "Review") ||
        (@assignment.metareview_allowed(topic_id) && title == "Meta-review")) %>

Testing

Functional tests from a student perspective are in /spec/features/meta_review_spec.rb. All meta-review tests make meta-reviews allowed only during the meta-review round. This means that if the simple meta-review test is passing then this feature is working as intended.

#50 - The submit button is not available for meta-reviews.

Issue Description

Users should see save and submit buttons as part of the meta-review form. These buttons should have the same behavior as the review form. The problem right now is there is not submit button for meta-reviews.

Changes

Code changes were made adding 'Metareview' to the conditional that determines if the submit button should be shown or not.

Previous behavior: 

<% if @title && ((@title.eql? 'Review') || (@title.eql? 'Self Review')) %>

New behavior: 

<% if @title && ((@title.eql? 'Review') || (@title.eql? 'Self Review') || (@title.eql? 'Metareview'))%>
  <%= submit_tag "Submit "+@title, :name => "Submit", :id => "Submit" %>
  <img src="/assets/info.png" title='- SAVE will only store the peer-review as draft and the peer-review will not be displayed to the reviewee. After clicking SUBMIT, the peer-review will be displayed to reviewee, and you will not be able to update it any more.'>
<% end %>

Testing

Functional tests from a student perspective are in /spec/features/meta_review_spec.rb. The tests assume the allowed number of meta-reviews is 5 and the required number of meta-reviews is 3. Tests for this feature are explained in summary below:

  • For assignments that have a meta-review deadline...
    • with 1 reviewer.
      • The user is able to SUBMIT a simple meta-review.

Testing

Manual Testing

1. Log in as super_administrator2. Password is "password". Click on mange...-> Assignments.

2. Create a new Assignment by clicking the "+" button. Give the assignment these fields: Name, Course and Submission Directory.

3. Go to "Due Dates" tab and check "Use meta-review deadline" and then set the due dates 1 into the FUTURE for each review round and meta-review. Leave everything else as default as shown below:

4. Go to the "Rubrics" tab and populate as shown below:

5. OPTIONAL: Go to "Review strategy page and select "Has meta-review limit?" and set the number of allowed and required meta-reviews.

6. Once done, "Create" the assignment.

7. Go back to Assignments. manage...-> Assignments.

8. Find the newly created assignment and click on "Add participant". Add the students below:

9. Go to manage...Impersonate User and impersonate "Student485"

10. Go to the newly created assignment and submit a link for the assignment.

11. Revert back to super_administrator2 and change the submission due date to a date in the PAST so we are currently in the review stage.

12. Impersonate student486 and request for a new review under others work, submit the review.

13. Revert back to super_administrator2 as previously described and change the review due date to be passed so we are currently in the meta-review stage.

14. Impersonate as student485 and Request for a new meta review.

15. Press "Begin" and complete the meta-review

16. When the review is complete, the "Begin" should disappear and be replaced with "View" and "Edit"

Automated Test Plan

Context 1: Meta-review tests that have a meta-review deadline with 0 reviewers.

1. User should NOT be able to see 'Request a new meta-review to perform' button when there are no reviews available. When there are reviews that don't belong to the meta-reviewer submitted, meta-reviews can be performed.

Context 2: Meta-review tests that have a meta-review deadline with 1 reviewers.

1. User is able to SAVE a simple meta-review.

2. User is able to go BACK during a simple meta-review since this functionality is available for reviews.

3. When the limit and required number of meta-reviews on the assignment are equal, then a student will see they need to submit exactly that number of meta-reviews.

4. When there are more meta-reviews allowed than required, then a student will see they need to submit between the two numbers of meta-reviews

5. When the meta-review limits for an assignment are unset then a student will see that the number of meta-reviews aren't limited

6. A student should not see a number of required or allowed meta-reviews when there are no limits on meta-reviews

7. A student should see the number of meta-reviews decrement after a review is requested

8. A student should see the number of meta-reviews decrement after they complete a review

9. A student should not be able to request a meta-review on their own reviews

10. If the meta-review limit on the assignment is set to 1 then a student should not be able to request a second meta review

11. User should not be able to see 'Request a new meta-review to perform' button when they have reviewed all valid reviews already

Context 3: Meta-review tests that have a meta-review deadline with 4 reviewers.

1. User should be able to request more than one meta-review before completing any

2. User should not be able to see 'Request a new meta-review to perform' button when they have reached the meta-review limit

3. User should not be able to request more than two meta-reviews without completing at least 1

4. A student should be able to request a meta-review if they are above their required but below their allowed meta-reviews

Coverage

Current test coverage is 41.348%. Code coverage will increase based on the test plan above and then if the failing test case described below would be fixed.

Travis CI is currently failing because of: 

 Failures:
 1) assignment function general tab should edit quiz number available to students
    Failure/Error:
      child_nodes.each do |node|
        initialize_fnode_update_children(params, node, tmp_res)
      end
    NoMethodError:
      undefined method `each' for "":String
    # ./app/controllers/tree_display_controller.rb:209:in `children_node_ng'
    # ./config/diagnostic.rb:11:in `call'
    # ------------------
    # --- Caused by: ---
    # Capybara::CapybaraError:
    #   Your application server raised an error - It has been raised in your test code because Capybara.raise_server_errors == true
    #   /home/travis/build/expertiza/expertiza/vendor/bundle/ruby/2.2.0/gems/capybara-2.17.0/lib/capybara/session.rb:145:in `raise_server_error!'
Retrieved from "https://wiki.expertiza.ncsu.edu/index.php?title=CSC/ECE_517_Spring_2022_-_E2235._Issues_related_to_meta-reviewing&oldid=144321"