CSC/ECE 517 Spring 2022 - E2235. Issues related to meta-reviewing: Difference between revisions

From Expertiza_Wiki
Jump to navigation Jump to search
No edit summary
 
(35 intermediate revisions by 4 users not shown)
Line 1: Line 1:
== '''Overview of Project''' ==
== '''Overview of Project''' ==


In Expertiza, meta-review is a feature that enables a third party to evaluate the quality of feedback that a reviewer gives a reviewee. meta-review is an important feature that can help students become better reviewers for their peers by providing detailed feedback that is actionable and positively formulated. Unfortunately, this feature is broken and the following issues were identified. The task is to fix these issues so that this feature can be used again.
In Expertiza, meta-review is a feature that enables a third party to evaluate the quality of feedback that a reviewer gives a reviewee. Meta-review is an important feature that can help students become better reviewers for their peers by providing detailed feedback that is actionable and positively formulated. Unfortunately, this feature is broken and the following issues were identified. The task is to fix these issues so that this feature can be used again.


* An assignment is deemed to have meta-reviews if and only if there is a meta-review deadline for this assignment in the `assignments_questionnaires` table.
* When a user requests a new meta-review, (s)he is unable to get Begin (refer to above image) link to start a new meta-review. (Issue #198)
* If there is no meta-review deadline, then don’t show the question about whether there is a max meta-review limit.
* In the DB the default number of meta-reviews allowed and required should be null.
* As soon as someone checks the Has meta-review Limit box on the Review Strategy tab, the UI should fill in 3 and 3 as the required and allowed number.
* Student pages should show the list of reviews that have been performed.
* When a user requests a new meta-review, the field which says the number of meta-reviews left ( refer to above image “Numbers of Meta-reviews left:”) does not change and the button to request a new meta-review disappears.
* When a user requests a new meta-review, the field which says the number of meta-reviews left ( refer to above image “Numbers of Meta-reviews left:”) does not change and the button to request a new meta-review disappears.


===Previous Work===
===Previous Work===


This assignment is a repeat of E1997 which was assigned to students last semester, Fall 2019.
This assignment is a repeat of E2025 which was assigned to students last semester, Spring 2020.


Details of the previous implementation can be found here:
Details of the previous implementation can be found here:
[http://https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2019_-_E1997._Issues_related_to_meta-reviewing E1997]
[CSC/ECE 517 Spring 2020 - E2025: Issues Related to Meta - Reviewing]


Our goals are to update the logic, refactor non-DRY code, and fix bugs.
Our goals are to update the logic, refactor non-DRY code, and fix bugs.


== '''[https://github.com/RPHolloway/expertiza/issues/32 #32 - Remove all debug comments in the UI]''' ==
== '''Approach taken to solve the issues''' ==
 
* We started by looking at issues happening for meta reviews in beta branch
===Issue Description===
* Next we tried to understand difference between meta-review and review.
 
* Further we tried to explore the possibility of combining meta-reviews and review to follow DRY principle in code.
It was noted in the previous team's implementation that multiple debugging comments were showing up during their video testing.
* But we observed that since meta review is advanced stage in review cycle and each meta-review is associated with some previous review and reviewee, there is no way to have single model and controller.
 
* So we scrapped the idea of changing the design at root level.
Debug messages shown when viewing a completed meta review:
* The best approach here to handle current meta-review issues is to fix them individually.
[[File:Debug_comments.png]]
* The details on how each issue will be approached and fixed is given below.
 
====* None of the issues need any change of workflow as per preliminary analysis, thus no UML diagram is added.====
===Changes===


There was a mistake in <code>app/views/response/_review.html.erb</code> where "<<nowiki/>br/><<nowiki/>hr/><<nowiki/>br/>" was being displayed under the review being meta-reviewed. This was due to being included as a string constant rather than html code.
== '''[https://github.com/expertiza/expertiza/issues/198 #198 - Begin link not working correctly]''' ==
 
[[File:E2025_debug_comments_response_review.PNG]]
 
There was an instance in <code>app/views/response/view.html.erb</code> where ruby code responsible for getting a review to be displayed was being interpreted and displayed, even though the response was being displayed as html in the line before. This resulted in the response details being listed in the UI erroneously.
 
[[File:E2025_debug_response_view.PNG]]
 
== '''[https://github.com/RPHolloway/expertiza/issues/49 #49 - "Request a new meta-review" button crashes when there are no available reviews]''' ==


===Issue Description===
===Issue Description===


When there are no valid metareviews to perform (either there are no reivews performed on the assignment, the only valid reviews concern your own work or own reviews, you have completed all allowed metareviews, or you have reviewed all valid reviews already) then the "Request a New Review" button still is visible and displays the following error (correctly) when pressed.
In beta branch begin is not working correctly. The page crashes sometime:
Observations:
* While selecting "Metareview" rubric in local db, the begin link crashed.
* Following the code we observed it was returning questionnaire list, where only object was expected.
* Fixing it, we were able to see page redirecting successfully.
[[File: Meta review rubric.png | 400px]]
[[File: Request disappear.jpeg | 400px]]
[[File: Meta begin crash.png | 400px]]
==== Plan:====
* We will be looking into this issue further, if the questionnaire was supposed to be the way it is?
* Further the loaded page doesn't have any portion of view for meta-review, instead it shows just plain old review. We will be checking the history of this change and try to find at what time partial view file was removed from code.
* We will be looking into the response_controller.rb, response_helper.rb, and metareview_response_map.rb files to fix the issue.


===Reproducing Existing Issue===


1. Log in as instructor6, click on mange...-> Assignments
==== Solution:====
* The issue was that when a user requests a meta-review, instead of getting questionnaire for meta-review we get the questionnaire for all the available rubrics for the given assignment.  
We changed this such that when a meta-review is requested, the response will only have a rubric quetionnaire for meta-review.


2. Click on edit assignment, navigate to Due Dates and check "Use meta-review deadline" checkbox
* The bug is found and fixed and can be checked here:
https://github.com/amundra02/expertiza/commit/f3f2343ce96a43e81eb7893a415f9077dbf7f32f


3. Select "Yes" for all submission allowed, Review allowed and Meta-review allowed in all of the deadline types


4. Navigate back to Assignment Tree View and click on "Add participant"


5. Add couple of students(say student1 and student2) to the assignment
== '''Submit button isn't available for meta-reviews''' ==


6. Impersonate/login as student1 and make a submission
=== Issue Description===
* The issue at present is there is no submit button for meta-reviews.
* Users should see save and submit buttons as part of the meta-review form similar to the behavior as the review form.
==== Plan:====
* We would be looking into the response.html.erb file to fix the issue.


7. Log in as instructor6 as previously described and change the submission due date to be passed so we are currently in the review stage
The bug is found and fixed and can be checked here:
https://github.com/amundra02/expertiza/commit/d0cd6b2c3816115360e798f8962a2b8f5e64db2a


8. Impersonate/login as student2 and Request for a new review under others work, submit the review


9. Log in as instructor6 as previously described and change the review due date to be passed so we are currently in the meta-review stage


10. Impersonate/login back as student1 (the one with the submission) and Request for a new meta review
== '''Hide meta-review deadline under review strategy when use meta review deadline checkbox is unchecked under due date ''' ==


[[File:E2025_outofreviews.png]]
=== Issue Description===
* The issue at present is that when the meta-review deadline checkbox under due-date has no impact on meta-review deadline under review strategy.
* Users should see the meta-review deadline under review strategy only when meta-review deadline checkbox is checked.
==== Plan:====
* We would be looking into the review_stragtegy.html.erb file to fix the issue.
==== Changes:====
* We have added inline style to the section in meta-review under review strategy to hide/show the section based on a ruby variable.  


===Changes===


This issue deals with the view "list.html.erb" under /app/views/student_review. The code below shows the conditional used to display the "Request new meta-review" button.


  <% if @num_metareviews_in_progress >= Assignment.max_outstanding_reviews %>
    <span>Note: You can't have more than 2 outstanding meta-reviews. You must complete one of your
    outstanding meta-reviews before selecting another.</span>
  <% elsif @assignment.num_metareviews_allowed && @num_metareviews_total == @assignment.num_metareviews_allowed %>
    <span>Note: You can not do more than <%=@assignment.num_metareviews_allowed%> meta-reviews according
    to assignment policy.</span>
  <% elsif (!@assignment.metareview_available?(session[:user].id)) %>
    <span>Meta-reviews cannot be performed at this time.</span>
  <% else %>
    <%= render :partial => 'set_dynamic_metareview', :locals => {:assignment => @assignment} %>
  <% end %>


Here we conditionally hide the button based on whether or not the participant has reviews available to perform. These checks are located in assignment.rb which is the model for an Assignment. The following method was added to assignment.rb and returns true when the participant has a valid meta-review available.


  # Returns true if there is a meta-review task available for the user
  def metareview_available?(user_id)
    metareviewer = AssignmentParticipant.where(user_id: user_id, parent_id: self.id).first
    begin
      self.response_map_to_metareview(metareviewer)
      true
    rescue
      false
    end
  end


== '''[https://github.com/RPHolloway/expertiza/issues/27 #27 - Defaulting Required and Allowed Meta-reviews to NULL in Assignment Database]''' ==


===Issue Description===


Currently, the number of required and allowed meta-reviews is always being set to 3 in the database. Per the specs of the feature, it should default the database values to NULL. When checkbox "Use meta-review deadline", in the "Due Dates" tab, is selected then the view should populate required and allowed meta-reviews to 3.


===Changes===
== '''[https://github.com/RPHolloway/expertiza/issues/32 #32 - Remove all debug comments in the UI]''' ==
 
We created a migration that defaulted the metareviews_allowed and metareviews_required in the Assignment table to nil instead of what the previous team had as 3.
 
[[File:E2025_13.png]]
 
Since the number of meta-reviews allowed and required are recorded using the values in the text fields, changes were made to <code>app/views/assignment/edit/_review_strategy.html.erb</code> to set the values and display them as expected.
 
When the checkbox for setting meta-review limits is unchecked, the values of number of meta-reviews allowed and required should be null. When the checkbox is checked, the values should be set to 3 and 3 by default, or to the values already defined before (in case the assignment was saved with the values set already). On page load, if the checkbox is enabled, then the inputs should be shown with the default or pre-set values, but if the checkbox is disabled then they should be hidden and set to null.
 
[[File:E2025_metareview_limit_1.PNG]]
 
[[File:E2025_metareview_limit_2.PNG]]
 
Changes had to be made to <code>app/views/student_review/list.html.erb</code> to account for the values being <code>null</code>.
 
[[File:E2025_metareview_limit_3.PNG]]
 
===Testing===
 
Tests have been created that test if a metareview can have nil limits. In this case, no limit will be shown to the user. Before this change that was impossible
 
== '''Issue 5 - Meta Review Limit is Not Functioning Correctly in Assignment Creation''' ==


===Issue Description===
===Issue Description===


The "has meta review limit" should determine whether a limit or requirement is placed on the student to complete meta reviews. If disabled, then they should be able to complete as many as they want. This is independent of the Meta Review STAGE that is enabled/disabled in the due dates tab. They should not impact each other and currently do within the assignment creation.
It was noted in the previous team's implementation that multiple debugging comments were showing up during their video testing.


===Reproducing Existing Issue===
Debug messages shown when viewing a completed meta review:
 
[[File:Debug_comments.png]]
1. If the "Use meta-review deadline" is disabled, then the Meta Review Limit panel disappears
 
2. "Has meta-review limit?" is hidden on edit
 
3. Possibility the "Has meta-review limit?" not being saved on assignment creation?
 
4. If the "Use meta-review deadline" is toggled (disabled, enabled), then the "Has meta-review limit?" resets to disabled
 
===Proposed Changes===
 
The code in _due_dates.html.erb and the _review_strategy.html.erb within the /app/app/views/assignments/edit folder is lazily written and does not really adhere to the project specifications of how metareviews should work on assignment creation. These views need to be edited to work correctly and also respond to the changes in issue 5.


===Changes===
===Changes===
There was an error tag in view, which is corrected:
https://github.com/amundra02/expertiza/commit/3e535992876291738334e2504bc5ef22bd8610d0


The changes were too long to screenshot, so here is the specific diff section:
== '''"Numbers of Meta-reviews left:” does not update after a user requests a review.''' ==
 
https://github.com/expertiza/expertiza/pull/1715/files#diff-540fd2bd6c34b48213716b1dd713733a
https://github.com/expertiza/expertiza/pull/1715/files#diff-80f39fa5f62c73ff8741b02d3b11656c
 
== '''Discovered Bugs:''' ==
 
== '''[https://github.com/RPHolloway/expertiza/issues/45 #45 - "Request a new meta-review" button bugs]''' ==
 
===Issue Description===
===Issue Description===
* Number of meta-reviews left should decrement when a user requests a meta-review. Current behavior only decremented when a review was submitted.
==== Plan:====
* This isn't an issue and is the expected behavior. The same logic is for reviews also. Number of left reviews should only be reduced once any review is submitted.


The "Request new a meta-review" button should be hidden when:
== '''"Request a new meta-review" button bugs''' ==
# The number of meta-reviews in-progress is greater than the maximum allowed outstanding reviews.
# The number of reviews allowed is limited AND the total number of meta-reviews completed or in-progress is greater than the number allowed.
# There are no valid reviews available for the participant.
 
===Changes===
 
This issue deals with the view "list.html.erb" under /app/views/student_review. The code below shows the conditional used to display the "Request new meta-review" button.
 
  <% if @num_metareviews_in_progress >= Assignment.max_outstanding_reviews %>
    <span>Note: You can't have more than 2 outstanding meta-reviews. You must complete one of your
    outstanding meta-reviews before selecting another.</span>
  <% elsif @assignment.num_metareviews_allowed && @num_metareviews_total == @assignment.num_metareviews_allowed %>
    <span>Note: You can not do more than <%=@assignment.num_metareviews_allowed%> meta-reviews according
    to assignment policy.</span>
  <% elsif (!@assignment.metareview_available?(session[:user].id)) %>
    <span>Meta-reviews cannot be performed at this time.</span>
  <% else %>
    <%= render :partial => 'set_dynamic_metareview', :locals => {:assignment => @assignment} %>
  <% end %>
 
===Testing===
Functional tests from a student perspective are in /spec/features/meta_review_spec.rb. The tests assume the allowed number of meta-reviews is 5 and the required number of meta-reviews is 3. Tests for this feature are explained in summary below:
* For assignments that have a meta-review deadline...
** with 0 reviewers. The user should NOT be able to see the 'Request a new meta-review to perform' button when there are no reviews available.
** with 1 reviewer.
*** A student should NOT be able to request a meta-review on their own review.
*** When the meta-review limit on the assignment is set to 1 then a student should NOT be able to request a second meta-review.
*** The user should not be able to see the 'Request a new meta-review to perform' button when they have performed all valid reviews.
** with 4 reviewers.
*** The user should be able to click the 'Begin' button and still see the 'Request a new meta-review to perform' button.
*** The user should NOT be able to see the 'Request a new meta-review to perform' button when they have reached the meta-review limit.
*** If the user requests two meta-reviews but has not submitted, then they should not be able to request a new meta-review.
*** The user should be able to request a meta-review if they are above their required but below the allowed number of meta-reviews.
 
== '''[https://github.com/RPHolloway/expertiza/issues/30 #30 - “Numbers of Meta-reviews left:” does not update after a user requests a review.]''' ==
===Issue Description===
===Issue Description===
Number of meta-reviews left should decrement when a user requests a meta-review. Current behavior only decremented when a review was submitted.
Currently after the "Request a new meta-review" button is clicked the button disappears and no new meta-review can be requested. The "Request new a meta-review" button should be hidden only when:
 
===Changes===
 
Previous behavior:
 
Number of Meta-Reviews left: <%= @assignment.num_metareviews_allowed - @num_metareviews_completed %>
 
Fixed behavior:
 
Number of meta-reviews left: <%= @assignment.num_metareviews_allowed - @num_metareviews_total %>


===Testing===
* The number of meta-reviews in-progress is greater than the maximum allowed outstanding reviews.
Functional tests from a student perspective are in /spec/features/meta_review_spec.rb. The tests assume the allowed number of meta-reviews is 5 and the required number of meta-reviews is 3. Tests for this feature are explained in summary below:
* The number of reviews allowed is limited AND the total number of meta-reviews completed or in-progress is greater than the number allowed.
* For assignments that have a meta-review deadline...
** with 1 reviewer.
*** The user should see the number of meta-reviews left decrement after a new meta-review is requested.
*** The user should see the number of meta-reviews left decrement after a new meta-review is completed.


== '''[https://github.com/RPHolloway/expertiza/issues/43 #43 - "Review Allowed" flag is required to be set on a due_date for a user to perform a meta-review.]''' ==
[[File: Request meta.png | 600px]]
===Issue Description===
[[File: Request button disappear.png | 600px]]
When an assignment with a meta-review is created the round that meta-reviews will be performed in should have the "Meta-review allowed" option set to yes. This should be the only requirement for making meta-reviews available in that round. The problem with the current implementation is that meta-reviews can only be performed in rounds where the "Review allowed" option is set.


[[File:E2025 Issue43.png]]
==== Plan:====
* This issue deals with the view "list.html.erb" under /app/views/student_review. The code below shows the conditional used to display the "Request new meta-review" button.
The code changes are done to fix this issue are availabel here: https://github.com/amundra02/expertiza/commit/d319dbf02100501c657c1257917c325607192a1f


===Changes===
[[File:Request meta review code.png|1200px]]
Code changes were applied to app/views/student_review/_responses.html.erb. Since this partial is shared by both reviews and meta-reviews in list.html.erb the title variable was used to determine if the begin option should be active or not.


Previous behavior:
<% elsif @assignment.get_current_stage(topic_id) != "Complete" && @assignment.can_review(topic_id) %>
Current behavior:
<% elsif @assignment.get_current_stage(topic_id) != "Complete" &&
        ((@assignment.can_review(topic_id) && title == "Review") ||
        (@assignment.metareview_allowed(topic_id) && title == "Meta-review")) %>


===Testing===
== '''Request for meta review is breaking the website when there are no review''' ==
Functional tests from a student perspective are in /spec/features/meta_review_spec.rb. All meta-review tests make meta-reviews allowed only during the meta-review round. This means that if the simple meta-review test is passing then this feature is working as intended.


== '''[https://github.com/RPHolloway/expertiza/issues/50 #50 - The submit button is not available for meta-reviews.]''' ==
=== Issue Description===
===Issue Description===
* When there are no review available the request for meta-review deadline is crashing the site.  
Users should see save and submit buttons as part of the meta-review form. These buttons should have the same behavior as the review form. The problem right now is there is not submit button for meta-reviews.
==== Plan:====
===Changes===
* Fixing it, we were able to show a small error message instead of allowing the site to crash
Code changes were made adding 'Metareview' to the conditional that determines if the submit button should be shown or not.
Code changes: https://github.com/amundra02/expertiza/commit/de9366941a7932af1cd9c0bf0c946f2ac05638c0


Previous behavior:
<% if @title && ((@title.eql? 'Review') || (@title.eql? 'Self Review')) %>
New behavior:
<% if @title && ((@title.eql? 'Review') || (@title.eql? 'Self Review') || (@title.eql? 'Metareview'))%>
  <%= submit_tag "Submit "+@title, :name => "Submit", :id => "Submit" %>
  <img src="/assets/info.png" title='- SAVE will only store the peer-review as draft and the peer-review will not be displayed to the reviewee. After clicking SUBMIT, the peer-review will be displayed to reviewee, and you will not be able to update it any more.'>
<% end %>


===Testing===
== '''Testing Plan''' ==
Functional tests from a student perspective are in /spec/features/meta_review_spec.rb. The tests assume the allowed number of meta-reviews is 5 and the required number of meta-reviews is 3. Tests for this feature are explained in summary below:
* For assignments that have a meta-review deadline...
** with 1 reviewer.
*** The user is able to SUBMIT a simple meta-review.
 
== '''Testing''' ==


=== Manual Testing ===
=== Manual Testing ===
Line 298: Line 173:


===Automated Test Plan===
===Automated Test Plan===
==== Context 1: Meta-review tests that have a meta-review deadline with 0 reviewers. ====
1. We will target to get all existing test passed, to make sure no existing functionality for other features break.
1. User should NOT be able to see 'Request a new meta-review to perform' button when there are no reviews available. When there are reviews that don't belong to the meta-reviewer submitted, meta-reviews can be performed.
 
==== Context 2: Meta-review tests that have a meta-review deadline with 1 reviewers. ====
1. User is able to SAVE a simple meta-review.
 
2. User is able to go BACK during a simple meta-review since this functionality is available for reviews.
 
3. When the limit and required number of meta-reviews on the assignment are equal, then a student will see they need to submit exactly that number of meta-reviews.
 
4. When there are more meta-reviews allowed than required, then a student will see they need to submit between the two numbers of meta-reviews
 
5. When the meta-review limits for an assignment are unset then a student will see that the number of meta-reviews aren't limited
 
6. A student should not see a number of required or allowed meta-reviews when there are no limits on meta-reviews
 
7. A student should see the number of meta-reviews decrement after a review is requested
 
8. A student should see the number of meta-reviews decrement after they complete a review
 
9. A student should not be able to request a meta-review on their own reviews
 
10. If the meta-review limit on the assignment is set to 1 then a student should not be able to request a second meta review
 
11. User should not be able to see 'Request a new meta-review to perform' button when they have reviewed all valid reviews already
 
==== Context 3: Meta-review tests that have a meta-review deadline with 4 reviewers. ====
1. User should be able to request more than one meta-review before completing any
 
2. User should not be able to see 'Request a new meta-review to perform' button when they have reached the meta-review limit


3. User should not be able to request more than two meta-reviews without completing at least 1
2. Also, we will be adding few test cases for the issues fixed in this project and hoping to increase coverage.


4. A student should be able to request a meta-review if they are above their required but below their allowed meta-reviews
=====New test case added:=====
Objective: To find the behavior of questionnaire function in metareview_response_map.rb when there is no questionnaire in assignment.
Expected behavior: It should return nil object, rather than some garbage object.


[[File:E2025_tests_passing.PNG]]
The test case added increased test coverage by small margin.


===Coverage===
Code changes: https://github.com/amundra02/expertiza/commit/bca6534e78f1edb46dc22e907c23af6bcbef952b
Current test coverage is 41.348%. Code coverage will increase based on the test plan above and then if the failing test case described below would be fixed.


Travis CI is currently failing because of:
== '''Important Links''' ==
  Failures:
# Video Link: https://drive.google.com/file/d/1ZLcVpftAaR9XpmlgMlVeWxfVl-PlsqJn/view?usp=sharing
  1) assignment function general tab should edit quiz number available to students
# Pull Request: https://github.com/expertiza/expertiza/pull/2397
    Failure/Error:
# Project Repo: https://github.com/amundra02/expertiza
      child_nodes.each do |node|
        initialize_fnode_update_children(params, node, tmp_res)
      end
    NoMethodError:
      undefined method `each' for "":String
    # ./app/controllers/tree_display_controller.rb:209:in `children_node_ng'
    # ./config/diagnostic.rb:11:in `call'
    # ------------------
    # --- Caused by: ---
    # Capybara::CapybaraError:
    #  Your application server raised an error - It has been raised in your test code because Capybara.raise_server_errors == true
    #  /home/travis/build/expertiza/expertiza/vendor/bundle/ruby/2.2.0/gems/capybara-2.17.0/lib/capybara/session.rb:145:in `raise_server_error!'

Latest revision as of 03:25, 2 May 2022

Overview of Project

In Expertiza, meta-review is a feature that enables a third party to evaluate the quality of feedback that a reviewer gives a reviewee. Meta-review is an important feature that can help students become better reviewers for their peers by providing detailed feedback that is actionable and positively formulated. Unfortunately, this feature is broken and the following issues were identified. The task is to fix these issues so that this feature can be used again.

  • When a user requests a new meta-review, (s)he is unable to get Begin (refer to above image) link to start a new meta-review. (Issue #198)
  • When a user requests a new meta-review, the field which says the number of meta-reviews left ( refer to above image “Numbers of Meta-reviews left:”) does not change and the button to request a new meta-review disappears.

Previous Work

This assignment is a repeat of E2025 which was assigned to students last semester, Spring 2020.

Details of the previous implementation can be found here: [CSC/ECE 517 Spring 2020 - E2025: Issues Related to Meta - Reviewing]

Our goals are to update the logic, refactor non-DRY code, and fix bugs.

Approach taken to solve the issues

  • We started by looking at issues happening for meta reviews in beta branch
  • Next we tried to understand difference between meta-review and review.
  • Further we tried to explore the possibility of combining meta-reviews and review to follow DRY principle in code.
  • But we observed that since meta review is advanced stage in review cycle and each meta-review is associated with some previous review and reviewee, there is no way to have single model and controller.
  • So we scrapped the idea of changing the design at root level.
  • The best approach here to handle current meta-review issues is to fix them individually.
  • The details on how each issue will be approached and fixed is given below.

* None of the issues need any change of workflow as per preliminary analysis, thus no UML diagram is added.

#198 - Begin link not working correctly

Issue Description

In beta branch begin is not working correctly. The page crashes sometime: Observations:

  • While selecting "Metareview" rubric in local db, the begin link crashed.
  • Following the code we observed it was returning questionnaire list, where only object was expected.
  • Fixing it, we were able to see page redirecting successfully.

Plan:

  • We will be looking into this issue further, if the questionnaire was supposed to be the way it is?
  • Further the loaded page doesn't have any portion of view for meta-review, instead it shows just plain old review. We will be checking the history of this change and try to find at what time partial view file was removed from code.
  • We will be looking into the response_controller.rb, response_helper.rb, and metareview_response_map.rb files to fix the issue.


Solution:

  • The issue was that when a user requests a meta-review, instead of getting questionnaire for meta-review we get the questionnaire for all the available rubrics for the given assignment.

We changed this such that when a meta-review is requested, the response will only have a rubric quetionnaire for meta-review.

  • The bug is found and fixed and can be checked here:

https://github.com/amundra02/expertiza/commit/f3f2343ce96a43e81eb7893a415f9077dbf7f32f


Submit button isn't available for meta-reviews

Issue Description

  • The issue at present is there is no submit button for meta-reviews.
  • Users should see save and submit buttons as part of the meta-review form similar to the behavior as the review form.

Plan:

  • We would be looking into the response.html.erb file to fix the issue.

The bug is found and fixed and can be checked here: https://github.com/amundra02/expertiza/commit/d0cd6b2c3816115360e798f8962a2b8f5e64db2a


Hide meta-review deadline under review strategy when use meta review deadline checkbox is unchecked under due date

Issue Description

  • The issue at present is that when the meta-review deadline checkbox under due-date has no impact on meta-review deadline under review strategy.
  • Users should see the meta-review deadline under review strategy only when meta-review deadline checkbox is checked.

Plan:

  • We would be looking into the review_stragtegy.html.erb file to fix the issue.

Changes:

  • We have added inline style to the section in meta-review under review strategy to hide/show the section based on a ruby variable.





#32 - Remove all debug comments in the UI

Issue Description

It was noted in the previous team's implementation that multiple debugging comments were showing up during their video testing.

Debug messages shown when viewing a completed meta review:

Changes

There was an error tag in view, which is corrected: https://github.com/amundra02/expertiza/commit/3e535992876291738334e2504bc5ef22bd8610d0

"Numbers of Meta-reviews left:” does not update after a user requests a review.

Issue Description

  • Number of meta-reviews left should decrement when a user requests a meta-review. Current behavior only decremented when a review was submitted.

Plan:

  • This isn't an issue and is the expected behavior. The same logic is for reviews also. Number of left reviews should only be reduced once any review is submitted.

"Request a new meta-review" button bugs

Issue Description

Currently after the "Request a new meta-review" button is clicked the button disappears and no new meta-review can be requested. The "Request new a meta-review" button should be hidden only when:

  • The number of meta-reviews in-progress is greater than the maximum allowed outstanding reviews.
  • The number of reviews allowed is limited AND the total number of meta-reviews completed or in-progress is greater than the number allowed.

Plan:

  • This issue deals with the view "list.html.erb" under /app/views/student_review. The code below shows the conditional used to display the "Request new meta-review" button.

The code changes are done to fix this issue are availabel here: https://github.com/amundra02/expertiza/commit/d319dbf02100501c657c1257917c325607192a1f


Request for meta review is breaking the website when there are no review

Issue Description

  • When there are no review available the request for meta-review deadline is crashing the site.

Plan:

  • Fixing it, we were able to show a small error message instead of allowing the site to crash

Code changes: https://github.com/amundra02/expertiza/commit/de9366941a7932af1cd9c0bf0c946f2ac05638c0


Testing Plan

Manual Testing

1. Log in as super_administrator2. Password is "password". Click on mange...-> Assignments.

2. Create a new Assignment by clicking the "+" button. Give the assignment these fields: Name, Course and Submission Directory.

3. Go to "Due Dates" tab and check "Use meta-review deadline" and then set the due dates 1 into the FUTURE for each review round and meta-review. Leave everything else as default as shown below:

4. Go to the "Rubrics" tab and populate as shown below:

5. OPTIONAL: Go to "Review strategy page and select "Has meta-review limit?" and set the number of allowed and required meta-reviews.

6. Once done, "Create" the assignment.

7. Go back to Assignments. manage...-> Assignments.

8. Find the newly created assignment and click on "Add participant". Add the students below:

9. Go to manage...Impersonate User and impersonate "Student485"

10. Go to the newly created assignment and submit a link for the assignment.

11. Revert back to super_administrator2 and change the submission due date to a date in the PAST so we are currently in the review stage.

12. Impersonate student486 and request for a new review under others work, submit the review.

13. Revert back to super_administrator2 as previously described and change the review due date to be passed so we are currently in the meta-review stage.

14. Impersonate as student485 and Request for a new meta review.

15. Press "Begin" and complete the meta-review

16. When the review is complete, the "Begin" should disappear and be replaced with "View" and "Edit"

Automated Test Plan

1. We will target to get all existing test passed, to make sure no existing functionality for other features break.

2. Also, we will be adding few test cases for the issues fixed in this project and hoping to increase coverage.

New test case added:

Objective: To find the behavior of questionnaire function in metareview_response_map.rb when there is no questionnaire in assignment. Expected behavior: It should return nil object, rather than some garbage object.

The test case added increased test coverage by small margin.

Code changes: https://github.com/amundra02/expertiza/commit/bca6534e78f1edb46dc22e907c23af6bcbef952b

Important Links

  1. Video Link: https://drive.google.com/file/d/1ZLcVpftAaR9XpmlgMlVeWxfVl-PlsqJn/view?usp=sharing
  2. Pull Request: https://github.com/expertiza/expertiza/pull/2397
  3. Project Repo: https://github.com/amundra02/expertiza
Retrieved from "https://wiki.expertiza.ncsu.edu/index.php?title=CSC/ECE_517_Spring_2022_-_E2235._Issues_related_to_meta-reviewing&oldid=145628"