Expertiza_Wiki - User contributions [en]

CSC/ECE 517 Fall 2019 - E1975. Generalize Review Versioning

2019-12-17T11:12:28Z

Atrived: /* Review Update */

__TOC__

== '''Introduction''' ==
Expertiza is a peer-review based learning platform in which work-products by students are evaluated and scored by peer-review of other students. Review process occurs in stages. After submission of initial feedback for a work-product by the peer-group, students are allowed time to incorporate the comments and improve their work. This second version of the work product is reviewed again and scores are given. This process might be repeated again and the average score from the last review stage is considered as the final score for that work-product. This incremental development of work-products and progressive learning is the fundamental concept underlying the Expertiza system.

=====Purpose=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The system also triggers an email to reviewer when there is a change in the submission. The scores are calculated on the fly based on the rubrics.

=====Scope=====
This project will enhance the existing scoring and peer review functionality and the underlying implementation along with the testing of the existing implementation. Testing of the score calculation, display for the weighted and non-weighted scores based on the combination of the rubrics are part of this project. This project also includes an additional feature for the author to notify the reviewers for the change in the submission.

=====Background=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The scores are calculated based on the rubrics defined in the system. There’s already an existing implementation for the computation of score but it doesn’t work for multipart rubric

=====Standards=====
The new standards that will be adhered to in this system are those given for the rails framework, and the code should follow rails and object oriented design principles. Any newly added code will adhere to the guidelines mentioned below:

● https://github.com/rubocop-hq/rails-style-guide 
● http://wiki.expertiza.ncsu.edu/index.php/Object-Oriented_Design_and_Programming.

== '''Problem Statement''' ==
=====Problem=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The functionality needs to be tested. It works in some cases, but we are suspicious that it is not entirely correct.

'''Profile Email Preference''':
User in expertiza has liberty to control what kind of emails they should receive. As the reviewers will receive an email for the change in submission. This can be controlled by selecting an option (When someone else submits work I am assigned to review) on the profile page of the user. As the checkboxes are far away on the right corner of the screen, the position of checkboxes need to be changed to make it more user friendly.
 

[[File:Generalize_review.jpg]]

Scores for students in Expertiza can be based on a combination of rubrics: review rubrics for each round of review, author feedback, and teammate review. In CSC/ECE 517, we don’t really use the student-assigned scores to grade projects, but the peer-review score shown to the student is based entirely on the second round of review 

Here is how scores are calculated: 

1. If only zero-weighted scores are available (here, in the first round of review), the average of all zero-weighted scores should be shown in grey font. An information button should explain why the scores are shown in grey (“The reviews submitted so far do not count toward the author’s final grade.”): 

2. If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is:: 
80%⨉90% + 100%⨉10% = 82%

3. If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. '''However, the previous review score that was given in the current round will count until the reviewer updates the submitted review'''. '''This functionality doesn’t work for multipart rubrics.''' 

The system also doesn’t generate emails to reviewers in case a new submission is made by the author without changing submission parameters like submission URL. Eg: A new submission is made by making a new deployment. But the deployment URL doesn’t change. In this case as well, without changing the current flow the author should have an option to notify all the reviewers to revisit the submission.

=====Interface Requirement=====
The overall user interface would remain the same as of the current system. Only the checkboxes for email preferences on user profile will be moved closer to the text to be more visible.

=====Quality Requirements=====
The interface changes are minimal and very easy to adapt. The checkboxes will be clearly visible for email preferences. The proposed changes should not affect the current system functionality in any way. The solution is only intended to enrich the project features.

=====Performance Requirements=====
There are no performance overhead created while meeting the requirements.

=====Assumptions=====
The functionality should work in case the assignment rubric is not multi-part.

== '''Proposed Solution''' ==
=====Zero-weighted scores=====
If only zero-weighted scores are available in the first round of review, the average of all zero-weighted scores will be shown in a gray font. An information button will explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade”)

Inside compute_total_score method in on_the_fly_calc.rb module, we'll check if the weighted score exists, if not then we'll calculate the non weighted score. We'll also have to add a method in the questionnaire model to get non weighted score. Few changes are required for the UI to display non-weighted score in grey with information icon and non weighted score in black.

Possible files to be modified.

models/on_the_fly_calc.rb 
models/questionnaire.rb 
models/view_team.html.erb

===== Non-zero weighted scores=====
If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is 80%⨉90% + 100%⨉10% = 82%

The score calculation was initially tested manually to verify the scenarios with multiple weights and rounds.
Also, existing spec for score calculation with a weighted average for both varying and non-varying rubric is covering this scenario.
The spec is written under /spec/models/assignment_spec.rb. The "score" function of the "model/assignment.rb" is called for online (on the fly score calculation).

Following file has the spec for weighted average score calculation.
spec/models/assignment_spec.rb 

=====Review Update=====
If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review.

 This feature can be broken down into the following sub-cases:
 1. A submission can only be made before deadline
 2. A review can edit a review only before deadline
 3. Score calculation should be only done for latest submitted review

Subcases:

 
1. Spec to ensure submission can't be made after the deadline

File : spec/features/post_deadline_review_submission_spec.rb

'''
it 'should not allow submission after deadline' do
user = User.find_by(name: "student2065")
stub_current_user(user, user.role.name, user.role)
# goto student_task page, which has link to "Your work"
visit 'student_task/view?id=1'
# the page will have text "Your work" but will be grayed
expect(page).to have_content "Your work"
# the page will not have link to content "Your work"
expect{click_link "Your work"}.to raise_error(Capybara::ElementNotFound)
end
'''

2. Spec to ensure review can be done only before deadline

File : spec/features/post_deadline_review_submission_spec.rb

'''
it "should not be able to review work after deadline" do
# The spec is written to reproduce following bug. "Others' work" link open after deadline passed
user = User.find_by(name: "student2065")
stub_current_user(user, user.role.name, user.role)
visit 'student_task/view?id=1'
# the page should have content, but after deadline passes it is displayed as gray
# but there should not be any link attached to it
expect(page).to have_content "Others' work"
# this is the bug, even after deadline has passed, the link is still present
# the ui comment in file views/student_task/view.html.erb says
# 
# But the link seems to be open even after deadline passed.
# Screenshot attached as part of wiki for E1975, Fall 2019
expect(page).to have_link("Others' work", "/student_review/list?id=1")
end

'''

The test scenario is able to produce the following bug. At times the review link is open even after the deadline has passed. The bug is also shown/reproduced in the following screenshot for Program 1. 

[[File:ScreenShotReview.jpg]]

3. Score calculation

Score calculation for assignments for review is part of models/assignment.rb. The score calculation is handled by function "score" in this file and the testing is done for both multipart varying and non-varying rubric.
The respective spec is written in file spec/models/assignment_spec.rb.

'''

describe '#scores' do
context 'when assignment is varying rubric by round assignment' do
it 'calculates scores in each round of each team in current assignment' do
allow(participant).to receive(:scores).with(review1: [question]).and_return(98)
allow(assignment).to receive(:varying_rubrics_by_round?).and_return(true)
allow(assignment).to receive(:num_review_rounds).and_return(1)
allow(ReviewResponseMap).to receive(:get_responses_for_team_round).with(team, 1).and_return([response])
allow(Answer).to receive(:compute_scores).with([response], [question]).and_return(max: 95, min: 88, avg: 90)
expect(assignment.scores(review1: [question]).inspect).to eq("{:participants=>{:\"1\"=>98}, :teams=>{:\"0\"=>{:team=>#<AssignmentTeam id: 1, "\
"name: \"no team\", parent_id: 1, type: \"AssignmentTeam\", comments_for_advertisement: nil, advertise_for_partner: nil, "\
"submitted_hyperlinks: \"---\\n- https://www.expertiza.ncsu.edu\", directory_num: 0, grade_for_submission: nil, "\
"comment_for_submission: nil>, :scores=>{:max=>95, :min=>88, :avg=>90.0}}}}")
end
end

context 'when assignment is not varying rubric by round assignment' do
it 'calculates scores of each team in current assignment' do
allow(participant).to receive(:scores).with(review: [question]).and_return(98)
allow(assignment).to receive(:varying_rubrics_by_round?).and_return(false)
allow(ReviewResponseMap).to receive(:get_assessments_for).with(team).and_return([response])
allow(Answer).to receive(:compute_scores).with([response], [question]).and_return(max: 95, min: 88, avg: 90)
expect(assignment.scores(review: [question]).inspect).to eq("{:participants=>{:\"1\"=>98}, :teams=>{:\"0\"=>{:team=>#<AssignmentTeam id: 1, "\
"name: \"no team\", parent_id: 1, type: \"AssignmentTeam\", comments_for_advertisement: nil, advertise_for_partner: nil, "\
"submitted_hyperlinks: \"---\\n- https://www.expertiza.ncsu.edu\", directory_num: 0, grade_for_submission: nil, "\
"comment_for_submission: nil>, :scores=>{:max=>95, :min=>88, :avg=>90}}}}")
end
end
end

'''

The test for "submission after deadline" and "review after deadline" are added as they were not present. A new spec file under spec/features by name post_deadline_review_submission_spec.rb has been created to accommodate the two cases.

 
[[File:Usecasenew.png]]

=='''Test Plan'''==

=====Rspec=====

New test cases will be added to test and validate the functionalities which are not working as expected. Also, the test cases for features that will be newly added to the system, will also be written.
Test cases will be added to these files:

spec/features/post_deadline_review_submission_sepc.rb 
spec/models/assignment_participant_spec.rb 

The scenarios covered by the newly introduced test cases are as follows: 
1. Zero weighted scores with gray mark: rspec spec/model/assignment_participants.rb 
2. Non-zero weighted scores 
3. Review update: rspec spec/features/post_deadline_review_submission_sepc.rb 

=====Manual Testing=====

'''Step 1'''. The instructor will create an assignment and assign the zero and non-zero weights to the rubrics. 
'''Step 2'''. The author submits the work done in expertiza. 
'''Step 3'''. Peers check the submitted work and give reviews for it. 

'''Test 1: Zero weighted scores shown in gray.''' 
Login as an author in expertiza and go to scores page to view the feedback.
Zero weighted scores should be displayed in gray to the logged in user.

'''Test 2: Information button.''' 
Login as an author in expertiza and go to scores page to view the feedback.
The information button should explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade.”)

'''Step 4'''. The mentor evaluates the final submission and give grades.

'''Test 3: Display non-zero weighted scores''' 
Login as an author in expertiza and go to scores page to view the feedback.
The scores shown for the assignment should be the weighted average of the scores.
The zero weighted scores have to be replaced with non-zero weighted scores.

'''Step 5''': The author updates the work and re-submits it again.

'''Test 4: Review Update''' 
The Review should be reopened. The reviewer should be able to go and update the submitted review.
If the review is not updated the previous review score has to be retained.

CSC/ECE 517 Fall 2019 - E1975. Generalize Review Versioning

2019-12-17T11:10:08Z

Atrived: /* Performance Requirements */

__TOC__

== '''Introduction''' ==
Expertiza is a peer-review based learning platform in which work-products by students are evaluated and scored by peer-review of other students. Review process occurs in stages. After submission of initial feedback for a work-product by the peer-group, students are allowed time to incorporate the comments and improve their work. This second version of the work product is reviewed again and scores are given. This process might be repeated again and the average score from the last review stage is considered as the final score for that work-product. This incremental development of work-products and progressive learning is the fundamental concept underlying the Expertiza system.

=====Purpose=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The system also triggers an email to reviewer when there is a change in the submission. The scores are calculated on the fly based on the rubrics.

=====Scope=====
This project will enhance the existing scoring and peer review functionality and the underlying implementation along with the testing of the existing implementation. Testing of the score calculation, display for the weighted and non-weighted scores based on the combination of the rubrics are part of this project. This project also includes an additional feature for the author to notify the reviewers for the change in the submission.

=====Background=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The scores are calculated based on the rubrics defined in the system. There’s already an existing implementation for the computation of score but it doesn’t work for multipart rubric

=====Standards=====
The new standards that will be adhered to in this system are those given for the rails framework, and the code should follow rails and object oriented design principles. Any newly added code will adhere to the guidelines mentioned below:

● https://github.com/rubocop-hq/rails-style-guide 
● http://wiki.expertiza.ncsu.edu/index.php/Object-Oriented_Design_and_Programming.

== '''Problem Statement''' ==
=====Problem=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The functionality needs to be tested. It works in some cases, but we are suspicious that it is not entirely correct.

'''Profile Email Preference''':
User in expertiza has liberty to control what kind of emails they should receive. As the reviewers will receive an email for the change in submission. This can be controlled by selecting an option (When someone else submits work I am assigned to review) on the profile page of the user. As the checkboxes are far away on the right corner of the screen, the position of checkboxes need to be changed to make it more user friendly.
 

[[File:Generalize_review.jpg]]

Scores for students in Expertiza can be based on a combination of rubrics: review rubrics for each round of review, author feedback, and teammate review. In CSC/ECE 517, we don’t really use the student-assigned scores to grade projects, but the peer-review score shown to the student is based entirely on the second round of review 

Here is how scores are calculated: 

1. If only zero-weighted scores are available (here, in the first round of review), the average of all zero-weighted scores should be shown in grey font. An information button should explain why the scores are shown in grey (“The reviews submitted so far do not count toward the author’s final grade.”): 

2. If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is:: 
80%⨉90% + 100%⨉10% = 82%

3. If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. '''However, the previous review score that was given in the current round will count until the reviewer updates the submitted review'''. '''This functionality doesn’t work for multipart rubrics.''' 

The system also doesn’t generate emails to reviewers in case a new submission is made by the author without changing submission parameters like submission URL. Eg: A new submission is made by making a new deployment. But the deployment URL doesn’t change. In this case as well, without changing the current flow the author should have an option to notify all the reviewers to revisit the submission.

=====Interface Requirement=====
The overall user interface would remain the same as of the current system. Only the checkboxes for email preferences on user profile will be moved closer to the text to be more visible.

=====Quality Requirements=====
The interface changes are minimal and very easy to adapt. The checkboxes will be clearly visible for email preferences. The proposed changes should not affect the current system functionality in any way. The solution is only intended to enrich the project features.

=====Performance Requirements=====
There are no performance overhead created while meeting the requirements.

=====Assumptions=====
The functionality should work in case the assignment rubric is not multi-part.

== '''Proposed Solution''' ==
=====Zero-weighted scores=====
If only zero-weighted scores are available in the first round of review, the average of all zero-weighted scores will be shown in a gray font. An information button will explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade”)

Inside compute_total_score method in on_the_fly_calc.rb module, we'll check if the weighted score exists, if not then we'll calculate the non weighted score. We'll also have to add a method in the questionnaire model to get non weighted score. Few changes are required for the UI to display non-weighted score in grey with information icon and non weighted score in black.

Possible files to be modified.

models/on_the_fly_calc.rb 
models/questionnaire.rb 
models/view_team.html.erb

===== Non-zero weighted scores=====
If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is 80%⨉90% + 100%⨉10% = 82%

The score calculation was initially tested manually to verify the scenarios with multiple weights and rounds.
Also, existing spec for score calculation with a weighted average for both varying and non-varying rubric is covering this scenario.
The spec is written under /spec/models/assignment_spec.rb. The "score" function of the "model/assignment.rb" is called for online (on the fly score calculation).

Following file has the spec for weighted average score calculation.
spec/models/assignment_spec.rb 

=====Review Update=====
If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review.

 This feature can be broken down into the following sub-cases:
 1. A submission can only be made before deadline
 2. A review can edit a review only before deadline
 3. Score calculation should be only done for latest submitted review

Subcases:

 
1. Spec to ensure submission can't be made after the deadline

File : spec/features/post_deadline_review_submission_spec.rb

'''
it 'should not allow submission after deadline' do
user = User.find_by(name: "student2065")
stub_current_user(user, user.role.name, user.role)
# goto student_task page, which has link to "Your work"
visit 'student_task/view?id=1'
# the page will have text "Your work" but will be grayed
expect(page).to have_content "Your work"
# the page will not have link to content "Your work"
expect{click_link "Your work"}.to raise_error(Capybara::ElementNotFound)
end
'''

2. Spec to ensure review can be done only before deadline

File : spec/features/post_deadline_review_submission_spec.rb

'''
it "should not be able to review work after deadline" do
# The spec is written to reproduce following bug. "Others' work" link open after deadline passed
user = User.find_by(name: "student2065")
stub_current_user(user, user.role.name, user.role)
visit 'student_task/view?id=1'
# the page should have content, but after deadline passes it is displayed as gray
# but there should not be any link attached to it
expect(page).to have_content "Others' work"
# this is the bug, even after deadline has passed, the link is still present
# the ui comment in file views/student_task/view.html.erb says
# 
# But the link seems to be open even after deadline passed.
# Screenshot attached as part of wiki for E1975, Fall 2019
expect(page).to have_link("Others' work", "/student_review/list?id=1")
end

'''

The test scenario is able to produce the following bug. At times the review link is open even after the deadline has passed. The bug is also shown/reproduced in the following screenshot for Program 1. 

[[File:ScreenShotReview.jpg]]

3. Score calculation

Score calculation for assignments for review is part of models/assignment.rb. The score calculation is handled by function "score" in this file and the testing is done for both multipart varying and non-varying rubric.
The respective spec is written in file spec/models/assignment_spec.rb.

'''

describe '#scores' do
context 'when assignment is varying rubric by round assignment' do
it 'calculates scores in each round of each team in current assignment' do
allow(participant).to receive(:scores).with(review1: [question]).and_return(98)
allow(assignment).to receive(:varying_rubrics_by_round?).and_return(true)
allow(assignment).to receive(:num_review_rounds).and_return(1)
allow(ReviewResponseMap).to receive(:get_responses_for_team_round).with(team, 1).and_return([response])
allow(Answer).to receive(:compute_scores).with([response], [question]).and_return(max: 95, min: 88, avg: 90)
expect(assignment.scores(review1: [question]).inspect).to eq("{:participants=>{:\"1\"=>98}, :teams=>{:\"0\"=>{:team=>#<AssignmentTeam id: 1, "\
"name: \"no team\", parent_id: 1, type: \"AssignmentTeam\", comments_for_advertisement: nil, advertise_for_partner: nil, "\
"submitted_hyperlinks: \"---\\n- https://www.expertiza.ncsu.edu\", directory_num: 0, grade_for_submission: nil, "\
"comment_for_submission: nil>, :scores=>{:max=>95, :min=>88, :avg=>90.0}}}}")
end
end

context 'when assignment is not varying rubric by round assignment' do
it 'calculates scores of each team in current assignment' do
allow(participant).to receive(:scores).with(review: [question]).and_return(98)
allow(assignment).to receive(:varying_rubrics_by_round?).and_return(false)
allow(ReviewResponseMap).to receive(:get_assessments_for).with(team).and_return([response])
allow(Answer).to receive(:compute_scores).with([response], [question]).and_return(max: 95, min: 88, avg: 90)
expect(assignment.scores(review: [question]).inspect).to eq("{:participants=>{:\"1\"=>98}, :teams=>{:\"0\"=>{:team=>#<AssignmentTeam id: 1, "\
"name: \"no team\", parent_id: 1, type: \"AssignmentTeam\", comments_for_advertisement: nil, advertise_for_partner: nil, "\
"submitted_hyperlinks: \"---\\n- https://www.expertiza.ncsu.edu\", directory_num: 0, grade_for_submission: nil, "\
"comment_for_submission: nil>, :scores=>{:max=>95, :min=>88, :avg=>90}}}}")
end
end
end

'''

 
[[File:Usecasenew.png]]

=='''Test Plan'''==

=====Rspec=====

New test cases will be added to test and validate the functionalities which are not working as expected. Also, the test cases for features that will be newly added to the system, will also be written.
Test cases will be added to these files:

spec/features/post_deadline_review_submission_sepc.rb 
spec/models/assignment_participant_spec.rb 

The scenarios covered by the newly introduced test cases are as follows: 
1. Zero weighted scores with gray mark: rspec spec/model/assignment_participants.rb 
2. Non-zero weighted scores 
3. Review update: rspec spec/features/post_deadline_review_submission_sepc.rb 

=====Manual Testing=====

'''Step 1'''. The instructor will create an assignment and assign the zero and non-zero weights to the rubrics. 
'''Step 2'''. The author submits the work done in expertiza. 
'''Step 3'''. Peers check the submitted work and give reviews for it. 

'''Test 1: Zero weighted scores shown in gray.''' 
Login as an author in expertiza and go to scores page to view the feedback.
Zero weighted scores should be displayed in gray to the logged in user.

'''Test 2: Information button.''' 
Login as an author in expertiza and go to scores page to view the feedback.
The information button should explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade.”)

'''Step 4'''. The mentor evaluates the final submission and give grades.

'''Test 3: Display non-zero weighted scores''' 
Login as an author in expertiza and go to scores page to view the feedback.
The scores shown for the assignment should be the weighted average of the scores.
The zero weighted scores have to be replaced with non-zero weighted scores.

'''Step 5''': The author updates the work and re-submits it again.

'''Test 4: Review Update''' 
The Review should be reopened. The reviewer should be able to go and update the submitted review.
If the review is not updated the previous review score has to be retained.

CSC/ECE 517 Fall 2019 - E1975. Generalize Review Versioning

2019-12-17T11:09:53Z

Atrived: /* Performance Requirements */

__TOC__

== '''Introduction''' ==
Expertiza is a peer-review based learning platform in which work-products by students are evaluated and scored by peer-review of other students. Review process occurs in stages. After submission of initial feedback for a work-product by the peer-group, students are allowed time to incorporate the comments and improve their work. This second version of the work product is reviewed again and scores are given. This process might be repeated again and the average score from the last review stage is considered as the final score for that work-product. This incremental development of work-products and progressive learning is the fundamental concept underlying the Expertiza system.

=====Purpose=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The system also triggers an email to reviewer when there is a change in the submission. The scores are calculated on the fly based on the rubrics.

=====Scope=====
This project will enhance the existing scoring and peer review functionality and the underlying implementation along with the testing of the existing implementation. Testing of the score calculation, display for the weighted and non-weighted scores based on the combination of the rubrics are part of this project. This project also includes an additional feature for the author to notify the reviewers for the change in the submission.

=====Background=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The scores are calculated based on the rubrics defined in the system. There’s already an existing implementation for the computation of score but it doesn’t work for multipart rubric

=====Standards=====
The new standards that will be adhered to in this system are those given for the rails framework, and the code should follow rails and object oriented design principles. Any newly added code will adhere to the guidelines mentioned below:

● https://github.com/rubocop-hq/rails-style-guide 
● http://wiki.expertiza.ncsu.edu/index.php/Object-Oriented_Design_and_Programming.

== '''Problem Statement''' ==
=====Problem=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The functionality needs to be tested. It works in some cases, but we are suspicious that it is not entirely correct.

'''Profile Email Preference''':
User in expertiza has liberty to control what kind of emails they should receive. As the reviewers will receive an email for the change in submission. This can be controlled by selecting an option (When someone else submits work I am assigned to review) on the profile page of the user. As the checkboxes are far away on the right corner of the screen, the position of checkboxes need to be changed to make it more user friendly.
 

[[File:Generalize_review.jpg]]

Scores for students in Expertiza can be based on a combination of rubrics: review rubrics for each round of review, author feedback, and teammate review. In CSC/ECE 517, we don’t really use the student-assigned scores to grade projects, but the peer-review score shown to the student is based entirely on the second round of review 

Here is how scores are calculated: 

1. If only zero-weighted scores are available (here, in the first round of review), the average of all zero-weighted scores should be shown in grey font. An information button should explain why the scores are shown in grey (“The reviews submitted so far do not count toward the author’s final grade.”): 

2. If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is:: 
80%⨉90% + 100%⨉10% = 82%

3. If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. '''However, the previous review score that was given in the current round will count until the reviewer updates the submitted review'''. '''This functionality doesn’t work for multipart rubrics.''' 

The system also doesn’t generate emails to reviewers in case a new submission is made by the author without changing submission parameters like submission URL. Eg: A new submission is made by making a new deployment. But the deployment URL doesn’t change. In this case as well, without changing the current flow the author should have an option to notify all the reviewers to revisit the submission.

=====Interface Requirement=====
The overall user interface would remain the same as of the current system. Only the checkboxes for email preferences on user profile will be moved closer to the text to be more visible.

=====Quality Requirements=====
The interface changes are minimal and very easy to adapt. The checkboxes will be clearly visible for email preferences. The proposed changes should not affect the current system functionality in any way. The solution is only intended to enrich the project features.

=====Performance Requirements=====
There are not performance overhead created while meeting the requirements.

=====Assumptions=====
The functionality should work in case the assignment rubric is not multi-part.

== '''Proposed Solution''' ==
=====Zero-weighted scores=====
If only zero-weighted scores are available in the first round of review, the average of all zero-weighted scores will be shown in a gray font. An information button will explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade”)

Inside compute_total_score method in on_the_fly_calc.rb module, we'll check if the weighted score exists, if not then we'll calculate the non weighted score. We'll also have to add a method in the questionnaire model to get non weighted score. Few changes are required for the UI to display non-weighted score in grey with information icon and non weighted score in black.

Possible files to be modified.

models/on_the_fly_calc.rb 
models/questionnaire.rb 
models/view_team.html.erb

===== Non-zero weighted scores=====
If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is 80%⨉90% + 100%⨉10% = 82%

The score calculation was initially tested manually to verify the scenarios with multiple weights and rounds.
Also, existing spec for score calculation with a weighted average for both varying and non-varying rubric is covering this scenario.
The spec is written under /spec/models/assignment_spec.rb. The "score" function of the "model/assignment.rb" is called for online (on the fly score calculation).

Following file has the spec for weighted average score calculation.
spec/models/assignment_spec.rb 

=====Review Update=====
If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review.

 This feature can be broken down into the following sub-cases:
 1. A submission can only be made before deadline
 2. A review can edit a review only before deadline
 3. Score calculation should be only done for latest submitted review

Subcases:

 
1. Spec to ensure submission can't be made after the deadline

File : spec/features/post_deadline_review_submission_spec.rb

'''
it 'should not allow submission after deadline' do
user = User.find_by(name: "student2065")
stub_current_user(user, user.role.name, user.role)
# goto student_task page, which has link to "Your work"
visit 'student_task/view?id=1'
# the page will have text "Your work" but will be grayed
expect(page).to have_content "Your work"
# the page will not have link to content "Your work"
expect{click_link "Your work"}.to raise_error(Capybara::ElementNotFound)
end
'''

2. Spec to ensure review can be done only before deadline

File : spec/features/post_deadline_review_submission_spec.rb

'''
it "should not be able to review work after deadline" do
# The spec is written to reproduce following bug. "Others' work" link open after deadline passed
user = User.find_by(name: "student2065")
stub_current_user(user, user.role.name, user.role)
visit 'student_task/view?id=1'
# the page should have content, but after deadline passes it is displayed as gray
# but there should not be any link attached to it
expect(page).to have_content "Others' work"
# this is the bug, even after deadline has passed, the link is still present
# the ui comment in file views/student_task/view.html.erb says
# 
# But the link seems to be open even after deadline passed.
# Screenshot attached as part of wiki for E1975, Fall 2019
expect(page).to have_link("Others' work", "/student_review/list?id=1")
end

'''

The test scenario is able to produce the following bug. At times the review link is open even after the deadline has passed. The bug is also shown/reproduced in the following screenshot for Program 1. 

[[File:ScreenShotReview.jpg]]

3. Score calculation

Score calculation for assignments for review is part of models/assignment.rb. The score calculation is handled by function "score" in this file and the testing is done for both multipart varying and non-varying rubric.
The respective spec is written in file spec/models/assignment_spec.rb.

'''

describe '#scores' do
context 'when assignment is varying rubric by round assignment' do
it 'calculates scores in each round of each team in current assignment' do
allow(participant).to receive(:scores).with(review1: [question]).and_return(98)
allow(assignment).to receive(:varying_rubrics_by_round?).and_return(true)
allow(assignment).to receive(:num_review_rounds).and_return(1)
allow(ReviewResponseMap).to receive(:get_responses_for_team_round).with(team, 1).and_return([response])
allow(Answer).to receive(:compute_scores).with([response], [question]).and_return(max: 95, min: 88, avg: 90)
expect(assignment.scores(review1: [question]).inspect).to eq("{:participants=>{:\"1\"=>98}, :teams=>{:\"0\"=>{:team=>#<AssignmentTeam id: 1, "\
"name: \"no team\", parent_id: 1, type: \"AssignmentTeam\", comments_for_advertisement: nil, advertise_for_partner: nil, "\
"submitted_hyperlinks: \"---\\n- https://www.expertiza.ncsu.edu\", directory_num: 0, grade_for_submission: nil, "\
"comment_for_submission: nil>, :scores=>{:max=>95, :min=>88, :avg=>90.0}}}}")
end
end

context 'when assignment is not varying rubric by round assignment' do
it 'calculates scores of each team in current assignment' do
allow(participant).to receive(:scores).with(review: [question]).and_return(98)
allow(assignment).to receive(:varying_rubrics_by_round?).and_return(false)
allow(ReviewResponseMap).to receive(:get_assessments_for).with(team).and_return([response])
allow(Answer).to receive(:compute_scores).with([response], [question]).and_return(max: 95, min: 88, avg: 90)
expect(assignment.scores(review: [question]).inspect).to eq("{:participants=>{:\"1\"=>98}, :teams=>{:\"0\"=>{:team=>#<AssignmentTeam id: 1, "\
"name: \"no team\", parent_id: 1, type: \"AssignmentTeam\", comments_for_advertisement: nil, advertise_for_partner: nil, "\
"submitted_hyperlinks: \"---\\n- https://www.expertiza.ncsu.edu\", directory_num: 0, grade_for_submission: nil, "\
"comment_for_submission: nil>, :scores=>{:max=>95, :min=>88, :avg=>90}}}}")
end
end
end

'''

 
[[File:Usecasenew.png]]

=='''Test Plan'''==

=====Rspec=====

New test cases will be added to test and validate the functionalities which are not working as expected. Also, the test cases for features that will be newly added to the system, will also be written.
Test cases will be added to these files:

spec/features/post_deadline_review_submission_sepc.rb 
spec/models/assignment_participant_spec.rb 

The scenarios covered by the newly introduced test cases are as follows: 
1. Zero weighted scores with gray mark: rspec spec/model/assignment_participants.rb 
2. Non-zero weighted scores 
3. Review update: rspec spec/features/post_deadline_review_submission_sepc.rb 

=====Manual Testing=====

'''Step 1'''. The instructor will create an assignment and assign the zero and non-zero weights to the rubrics. 
'''Step 2'''. The author submits the work done in expertiza. 
'''Step 3'''. Peers check the submitted work and give reviews for it. 

'''Test 1: Zero weighted scores shown in gray.''' 
Login as an author in expertiza and go to scores page to view the feedback.
Zero weighted scores should be displayed in gray to the logged in user.

'''Test 2: Information button.''' 
Login as an author in expertiza and go to scores page to view the feedback.
The information button should explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade.”)

'''Step 4'''. The mentor evaluates the final submission and give grades.

'''Test 3: Display non-zero weighted scores''' 
Login as an author in expertiza and go to scores page to view the feedback.
The scores shown for the assignment should be the weighted average of the scores.
The zero weighted scores have to be replaced with non-zero weighted scores.

'''Step 5''': The author updates the work and re-submits it again.

'''Test 4: Review Update''' 
The Review should be reopened. The reviewer should be able to go and update the submitted review.
If the review is not updated the previous review score has to be retained.

CSC/ECE 517 Fall 2019 - E1975. Generalize Review Versioning

2019-12-17T11:08:56Z

Atrived: /* Quality Requirements */

__TOC__

== '''Introduction''' ==
Expertiza is a peer-review based learning platform in which work-products by students are evaluated and scored by peer-review of other students. Review process occurs in stages. After submission of initial feedback for a work-product by the peer-group, students are allowed time to incorporate the comments and improve their work. This second version of the work product is reviewed again and scores are given. This process might be repeated again and the average score from the last review stage is considered as the final score for that work-product. This incremental development of work-products and progressive learning is the fundamental concept underlying the Expertiza system.

=====Purpose=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The system also triggers an email to reviewer when there is a change in the submission. The scores are calculated on the fly based on the rubrics.

=====Scope=====
This project will enhance the existing scoring and peer review functionality and the underlying implementation along with the testing of the existing implementation. Testing of the score calculation, display for the weighted and non-weighted scores based on the combination of the rubrics are part of this project. This project also includes an additional feature for the author to notify the reviewers for the change in the submission.

=====Background=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The scores are calculated based on the rubrics defined in the system. There’s already an existing implementation for the computation of score but it doesn’t work for multipart rubric

=====Standards=====
The new standards that will be adhered to in this system are those given for the rails framework, and the code should follow rails and object oriented design principles. Any newly added code will adhere to the guidelines mentioned below:

● https://github.com/rubocop-hq/rails-style-guide 
● http://wiki.expertiza.ncsu.edu/index.php/Object-Oriented_Design_and_Programming.

== '''Problem Statement''' ==
=====Problem=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The functionality needs to be tested. It works in some cases, but we are suspicious that it is not entirely correct.

'''Profile Email Preference''':
User in expertiza has liberty to control what kind of emails they should receive. As the reviewers will receive an email for the change in submission. This can be controlled by selecting an option (When someone else submits work I am assigned to review) on the profile page of the user. As the checkboxes are far away on the right corner of the screen, the position of checkboxes need to be changed to make it more user friendly.
 

[[File:Generalize_review.jpg]]

Scores for students in Expertiza can be based on a combination of rubrics: review rubrics for each round of review, author feedback, and teammate review. In CSC/ECE 517, we don’t really use the student-assigned scores to grade projects, but the peer-review score shown to the student is based entirely on the second round of review 

Here is how scores are calculated: 

1. If only zero-weighted scores are available (here, in the first round of review), the average of all zero-weighted scores should be shown in grey font. An information button should explain why the scores are shown in grey (“The reviews submitted so far do not count toward the author’s final grade.”): 

2. If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is:: 
80%⨉90% + 100%⨉10% = 82%

3. If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. '''However, the previous review score that was given in the current round will count until the reviewer updates the submitted review'''. '''This functionality doesn’t work for multipart rubrics.''' 

The system also doesn’t generate emails to reviewers in case a new submission is made by the author without changing submission parameters like submission URL. Eg: A new submission is made by making a new deployment. But the deployment URL doesn’t change. In this case as well, without changing the current flow the author should have an option to notify all the reviewers to revisit the submission.

=====Interface Requirement=====
The overall user interface would remain the same as of the current system. Only the checkboxes for email preferences on user profile will be moved closer to the text to be more visible.

=====Quality Requirements=====
The interface changes are minimal and very easy to adapt. The checkboxes will be clearly visible for email preferences. The proposed changes should not affect the current system functionality in any way. The solution is only intended to enrich the project features.

=====Performance Requirements=====
Adding test for multi-part rubric functionality should not cause any overhead in the performance of the system. Sending emails through “Notify All Reviewers” button should affect the system minimally in terms of performance. The only overhead is a database query to get all the reviewers. A good database design should incorporate such a query smoothly.

=====Assumptions=====
The functionality should work in case the assignment rubric is not multi-part.

== '''Proposed Solution''' ==
=====Zero-weighted scores=====
If only zero-weighted scores are available in the first round of review, the average of all zero-weighted scores will be shown in a gray font. An information button will explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade”)

Inside compute_total_score method in on_the_fly_calc.rb module, we'll check if the weighted score exists, if not then we'll calculate the non weighted score. We'll also have to add a method in the questionnaire model to get non weighted score. Few changes are required for the UI to display non-weighted score in grey with information icon and non weighted score in black.

Possible files to be modified.

models/on_the_fly_calc.rb 
models/questionnaire.rb 
models/view_team.html.erb

===== Non-zero weighted scores=====
If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is 80%⨉90% + 100%⨉10% = 82%

The score calculation was initially tested manually to verify the scenarios with multiple weights and rounds.
Also, existing spec for score calculation with a weighted average for both varying and non-varying rubric is covering this scenario.
The spec is written under /spec/models/assignment_spec.rb. The "score" function of the "model/assignment.rb" is called for online (on the fly score calculation).

Following file has the spec for weighted average score calculation.
spec/models/assignment_spec.rb 

=====Review Update=====
If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review.

 This feature can be broken down into the following sub-cases:
 1. A submission can only be made before deadline
 2. A review can edit a review only before deadline
 3. Score calculation should be only done for latest submitted review

Subcases:

 
1. Spec to ensure submission can't be made after the deadline

File : spec/features/post_deadline_review_submission_spec.rb

'''
it 'should not allow submission after deadline' do
user = User.find_by(name: "student2065")
stub_current_user(user, user.role.name, user.role)
# goto student_task page, which has link to "Your work"
visit 'student_task/view?id=1'
# the page will have text "Your work" but will be grayed
expect(page).to have_content "Your work"
# the page will not have link to content "Your work"
expect{click_link "Your work"}.to raise_error(Capybara::ElementNotFound)
end
'''

2. Spec to ensure review can be done only before deadline

File : spec/features/post_deadline_review_submission_spec.rb

'''
it "should not be able to review work after deadline" do
# The spec is written to reproduce following bug. "Others' work" link open after deadline passed
user = User.find_by(name: "student2065")
stub_current_user(user, user.role.name, user.role)
visit 'student_task/view?id=1'
# the page should have content, but after deadline passes it is displayed as gray
# but there should not be any link attached to it
expect(page).to have_content "Others' work"
# this is the bug, even after deadline has passed, the link is still present
# the ui comment in file views/student_task/view.html.erb says
# 
# But the link seems to be open even after deadline passed.
# Screenshot attached as part of wiki for E1975, Fall 2019
expect(page).to have_link("Others' work", "/student_review/list?id=1")
end

'''

The test scenario is able to produce the following bug. At times the review link is open even after the deadline has passed. The bug is also shown/reproduced in the following screenshot for Program 1. 

[[File:ScreenShotReview.jpg]]

3. Score calculation

Score calculation for assignments for review is part of models/assignment.rb. The score calculation is handled by function "score" in this file and the testing is done for both multipart varying and non-varying rubric.
The respective spec is written in file spec/models/assignment_spec.rb.

'''

describe '#scores' do
context 'when assignment is varying rubric by round assignment' do
it 'calculates scores in each round of each team in current assignment' do
allow(participant).to receive(:scores).with(review1: [question]).and_return(98)
allow(assignment).to receive(:varying_rubrics_by_round?).and_return(true)
allow(assignment).to receive(:num_review_rounds).and_return(1)
allow(ReviewResponseMap).to receive(:get_responses_for_team_round).with(team, 1).and_return([response])
allow(Answer).to receive(:compute_scores).with([response], [question]).and_return(max: 95, min: 88, avg: 90)
expect(assignment.scores(review1: [question]).inspect).to eq("{:participants=>{:\"1\"=>98}, :teams=>{:\"0\"=>{:team=>#<AssignmentTeam id: 1, "\
"name: \"no team\", parent_id: 1, type: \"AssignmentTeam\", comments_for_advertisement: nil, advertise_for_partner: nil, "\
"submitted_hyperlinks: \"---\\n- https://www.expertiza.ncsu.edu\", directory_num: 0, grade_for_submission: nil, "\
"comment_for_submission: nil>, :scores=>{:max=>95, :min=>88, :avg=>90.0}}}}")
end
end

context 'when assignment is not varying rubric by round assignment' do
it 'calculates scores of each team in current assignment' do
allow(participant).to receive(:scores).with(review: [question]).and_return(98)
allow(assignment).to receive(:varying_rubrics_by_round?).and_return(false)
allow(ReviewResponseMap).to receive(:get_assessments_for).with(team).and_return([response])
allow(Answer).to receive(:compute_scores).with([response], [question]).and_return(max: 95, min: 88, avg: 90)
expect(assignment.scores(review: [question]).inspect).to eq("{:participants=>{:\"1\"=>98}, :teams=>{:\"0\"=>{:team=>#<AssignmentTeam id: 1, "\
"name: \"no team\", parent_id: 1, type: \"AssignmentTeam\", comments_for_advertisement: nil, advertise_for_partner: nil, "\
"submitted_hyperlinks: \"---\\n- https://www.expertiza.ncsu.edu\", directory_num: 0, grade_for_submission: nil, "\
"comment_for_submission: nil>, :scores=>{:max=>95, :min=>88, :avg=>90}}}}")
end
end
end

'''

 
[[File:Usecasenew.png]]

=='''Test Plan'''==

=====Rspec=====

New test cases will be added to test and validate the functionalities which are not working as expected. Also, the test cases for features that will be newly added to the system, will also be written.
Test cases will be added to these files:

spec/features/post_deadline_review_submission_sepc.rb 
spec/models/assignment_participant_spec.rb 

The scenarios covered by the newly introduced test cases are as follows: 
1. Zero weighted scores with gray mark: rspec spec/model/assignment_participants.rb 
2. Non-zero weighted scores 
3. Review update: rspec spec/features/post_deadline_review_submission_sepc.rb 

=====Manual Testing=====

'''Step 1'''. The instructor will create an assignment and assign the zero and non-zero weights to the rubrics. 
'''Step 2'''. The author submits the work done in expertiza. 
'''Step 3'''. Peers check the submitted work and give reviews for it. 

'''Test 1: Zero weighted scores shown in gray.''' 
Login as an author in expertiza and go to scores page to view the feedback.
Zero weighted scores should be displayed in gray to the logged in user.

'''Test 2: Information button.''' 
Login as an author in expertiza and go to scores page to view the feedback.
The information button should explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade.”)

'''Step 4'''. The mentor evaluates the final submission and give grades.

'''Test 3: Display non-zero weighted scores''' 
Login as an author in expertiza and go to scores page to view the feedback.
The scores shown for the assignment should be the weighted average of the scores.
The zero weighted scores have to be replaced with non-zero weighted scores.

'''Step 5''': The author updates the work and re-submits it again.

'''Test 4: Review Update''' 
The Review should be reopened. The reviewer should be able to go and update the submitted review.
If the review is not updated the previous review score has to be retained.

CSC/ECE 517 Fall 2019 - E1975. Generalize Review Versioning

2019-12-17T11:08:06Z

Atrived: /* Interface Requirement */

__TOC__

== '''Introduction''' ==
Expertiza is a peer-review based learning platform in which work-products by students are evaluated and scored by peer-review of other students. Review process occurs in stages. After submission of initial feedback for a work-product by the peer-group, students are allowed time to incorporate the comments and improve their work. This second version of the work product is reviewed again and scores are given. This process might be repeated again and the average score from the last review stage is considered as the final score for that work-product. This incremental development of work-products and progressive learning is the fundamental concept underlying the Expertiza system.

=====Purpose=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The system also triggers an email to reviewer when there is a change in the submission. The scores are calculated on the fly based on the rubrics.

=====Scope=====
This project will enhance the existing scoring and peer review functionality and the underlying implementation along with the testing of the existing implementation. Testing of the score calculation, display for the weighted and non-weighted scores based on the combination of the rubrics are part of this project. This project also includes an additional feature for the author to notify the reviewers for the change in the submission.

=====Background=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The scores are calculated based on the rubrics defined in the system. There’s already an existing implementation for the computation of score but it doesn’t work for multipart rubric

=====Standards=====
The new standards that will be adhered to in this system are those given for the rails framework, and the code should follow rails and object oriented design principles. Any newly added code will adhere to the guidelines mentioned below:

● https://github.com/rubocop-hq/rails-style-guide 
● http://wiki.expertiza.ncsu.edu/index.php/Object-Oriented_Design_and_Programming.

== '''Problem Statement''' ==
=====Problem=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The functionality needs to be tested. It works in some cases, but we are suspicious that it is not entirely correct.

'''Profile Email Preference''':
User in expertiza has liberty to control what kind of emails they should receive. As the reviewers will receive an email for the change in submission. This can be controlled by selecting an option (When someone else submits work I am assigned to review) on the profile page of the user. As the checkboxes are far away on the right corner of the screen, the position of checkboxes need to be changed to make it more user friendly.
 

[[File:Generalize_review.jpg]]

Scores for students in Expertiza can be based on a combination of rubrics: review rubrics for each round of review, author feedback, and teammate review. In CSC/ECE 517, we don’t really use the student-assigned scores to grade projects, but the peer-review score shown to the student is based entirely on the second round of review 

Here is how scores are calculated: 

1. If only zero-weighted scores are available (here, in the first round of review), the average of all zero-weighted scores should be shown in grey font. An information button should explain why the scores are shown in grey (“The reviews submitted so far do not count toward the author’s final grade.”): 

2. If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is:: 
80%⨉90% + 100%⨉10% = 82%

3. If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. '''However, the previous review score that was given in the current round will count until the reviewer updates the submitted review'''. '''This functionality doesn’t work for multipart rubrics.''' 

The system also doesn’t generate emails to reviewers in case a new submission is made by the author without changing submission parameters like submission URL. Eg: A new submission is made by making a new deployment. But the deployment URL doesn’t change. In this case as well, without changing the current flow the author should have an option to notify all the reviewers to revisit the submission.

=====Interface Requirement=====
The overall user interface would remain the same as of the current system. Only the checkboxes for email preferences on user profile will be moved closer to the text to be more visible.

=====Quality Requirements=====
The interface changes are minimal and very easy to adapt. The new button will be clearly visible and is supposed to have a clear text to interpret its functionality. The proposed changes should not affect the current system functionality in any way. The solution is only intended to enrich the project features.

=====Performance Requirements=====
Adding test for multi-part rubric functionality should not cause any overhead in the performance of the system. Sending emails through “Notify All Reviewers” button should affect the system minimally in terms of performance. The only overhead is a database query to get all the reviewers. A good database design should incorporate such a query smoothly.

=====Assumptions=====
The functionality should work in case the assignment rubric is not multi-part.

== '''Proposed Solution''' ==
=====Zero-weighted scores=====
If only zero-weighted scores are available in the first round of review, the average of all zero-weighted scores will be shown in a gray font. An information button will explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade”)

Inside compute_total_score method in on_the_fly_calc.rb module, we'll check if the weighted score exists, if not then we'll calculate the non weighted score. We'll also have to add a method in the questionnaire model to get non weighted score. Few changes are required for the UI to display non-weighted score in grey with information icon and non weighted score in black.

Possible files to be modified.

models/on_the_fly_calc.rb 
models/questionnaire.rb 
models/view_team.html.erb

===== Non-zero weighted scores=====
If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is 80%⨉90% + 100%⨉10% = 82%

The score calculation was initially tested manually to verify the scenarios with multiple weights and rounds.
Also, existing spec for score calculation with a weighted average for both varying and non-varying rubric is covering this scenario.
The spec is written under /spec/models/assignment_spec.rb. The "score" function of the "model/assignment.rb" is called for online (on the fly score calculation).

Following file has the spec for weighted average score calculation.
spec/models/assignment_spec.rb 

=====Review Update=====
If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review.

 This feature can be broken down into the following sub-cases:
 1. A submission can only be made before deadline
 2. A review can edit a review only before deadline
 3. Score calculation should be only done for latest submitted review

Subcases:

 
1. Spec to ensure submission can't be made after the deadline

File : spec/features/post_deadline_review_submission_spec.rb

'''
it 'should not allow submission after deadline' do
user = User.find_by(name: "student2065")
stub_current_user(user, user.role.name, user.role)
# goto student_task page, which has link to "Your work"
visit 'student_task/view?id=1'
# the page will have text "Your work" but will be grayed
expect(page).to have_content "Your work"
# the page will not have link to content "Your work"
expect{click_link "Your work"}.to raise_error(Capybara::ElementNotFound)
end
'''

2. Spec to ensure review can be done only before deadline

File : spec/features/post_deadline_review_submission_spec.rb

'''
it "should not be able to review work after deadline" do
# The spec is written to reproduce following bug. "Others' work" link open after deadline passed
user = User.find_by(name: "student2065")
stub_current_user(user, user.role.name, user.role)
visit 'student_task/view?id=1'
# the page should have content, but after deadline passes it is displayed as gray
# but there should not be any link attached to it
expect(page).to have_content "Others' work"
# this is the bug, even after deadline has passed, the link is still present
# the ui comment in file views/student_task/view.html.erb says
# 
# But the link seems to be open even after deadline passed.
# Screenshot attached as part of wiki for E1975, Fall 2019
expect(page).to have_link("Others' work", "/student_review/list?id=1")
end

'''

The test scenario is able to produce the following bug. At times the review link is open even after the deadline has passed. The bug is also shown/reproduced in the following screenshot for Program 1. 

[[File:ScreenShotReview.jpg]]

3. Score calculation

Score calculation for assignments for review is part of models/assignment.rb. The score calculation is handled by function "score" in this file and the testing is done for both multipart varying and non-varying rubric.
The respective spec is written in file spec/models/assignment_spec.rb.

'''

describe '#scores' do
context 'when assignment is varying rubric by round assignment' do
it 'calculates scores in each round of each team in current assignment' do
allow(participant).to receive(:scores).with(review1: [question]).and_return(98)
allow(assignment).to receive(:varying_rubrics_by_round?).and_return(true)
allow(assignment).to receive(:num_review_rounds).and_return(1)
allow(ReviewResponseMap).to receive(:get_responses_for_team_round).with(team, 1).and_return([response])
allow(Answer).to receive(:compute_scores).with([response], [question]).and_return(max: 95, min: 88, avg: 90)
expect(assignment.scores(review1: [question]).inspect).to eq("{:participants=>{:\"1\"=>98}, :teams=>{:\"0\"=>{:team=>#<AssignmentTeam id: 1, "\
"name: \"no team\", parent_id: 1, type: \"AssignmentTeam\", comments_for_advertisement: nil, advertise_for_partner: nil, "\
"submitted_hyperlinks: \"---\\n- https://www.expertiza.ncsu.edu\", directory_num: 0, grade_for_submission: nil, "\
"comment_for_submission: nil>, :scores=>{:max=>95, :min=>88, :avg=>90.0}}}}")
end
end

context 'when assignment is not varying rubric by round assignment' do
it 'calculates scores of each team in current assignment' do
allow(participant).to receive(:scores).with(review: [question]).and_return(98)
allow(assignment).to receive(:varying_rubrics_by_round?).and_return(false)
allow(ReviewResponseMap).to receive(:get_assessments_for).with(team).and_return([response])
allow(Answer).to receive(:compute_scores).with([response], [question]).and_return(max: 95, min: 88, avg: 90)
expect(assignment.scores(review: [question]).inspect).to eq("{:participants=>{:\"1\"=>98}, :teams=>{:\"0\"=>{:team=>#<AssignmentTeam id: 1, "\
"name: \"no team\", parent_id: 1, type: \"AssignmentTeam\", comments_for_advertisement: nil, advertise_for_partner: nil, "\
"submitted_hyperlinks: \"---\\n- https://www.expertiza.ncsu.edu\", directory_num: 0, grade_for_submission: nil, "\
"comment_for_submission: nil>, :scores=>{:max=>95, :min=>88, :avg=>90}}}}")
end
end
end

'''

 
[[File:Usecasenew.png]]

=='''Test Plan'''==

=====Rspec=====

New test cases will be added to test and validate the functionalities which are not working as expected. Also, the test cases for features that will be newly added to the system, will also be written.
Test cases will be added to these files:

spec/features/post_deadline_review_submission_sepc.rb 
spec/models/assignment_participant_spec.rb 

The scenarios covered by the newly introduced test cases are as follows: 
1. Zero weighted scores with gray mark: rspec spec/model/assignment_participants.rb 
2. Non-zero weighted scores 
3. Review update: rspec spec/features/post_deadline_review_submission_sepc.rb 

=====Manual Testing=====

'''Step 1'''. The instructor will create an assignment and assign the zero and non-zero weights to the rubrics. 
'''Step 2'''. The author submits the work done in expertiza. 
'''Step 3'''. Peers check the submitted work and give reviews for it. 

'''Test 1: Zero weighted scores shown in gray.''' 
Login as an author in expertiza and go to scores page to view the feedback.
Zero weighted scores should be displayed in gray to the logged in user.

'''Test 2: Information button.''' 
Login as an author in expertiza and go to scores page to view the feedback.
The information button should explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade.”)

'''Step 4'''. The mentor evaluates the final submission and give grades.

'''Test 3: Display non-zero weighted scores''' 
Login as an author in expertiza and go to scores page to view the feedback.
The scores shown for the assignment should be the weighted average of the scores.
The zero weighted scores have to be replaced with non-zero weighted scores.

'''Step 5''': The author updates the work and re-submits it again.

'''Test 4: Review Update''' 
The Review should be reopened. The reviewer should be able to go and update the submitted review.
If the review is not updated the previous review score has to be retained.

CSC/ECE 517 Fall 2019 - E1975. Generalize Review Versioning

2019-12-17T11:05:37Z

Atrived: /* Interface Requirement */

__TOC__

== '''Introduction''' ==
Expertiza is a peer-review based learning platform in which work-products by students are evaluated and scored by peer-review of other students. Review process occurs in stages. After submission of initial feedback for a work-product by the peer-group, students are allowed time to incorporate the comments and improve their work. This second version of the work product is reviewed again and scores are given. This process might be repeated again and the average score from the last review stage is considered as the final score for that work-product. This incremental development of work-products and progressive learning is the fundamental concept underlying the Expertiza system.

=====Purpose=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The system also triggers an email to reviewer when there is a change in the submission. The scores are calculated on the fly based on the rubrics.

=====Scope=====
This project will enhance the existing scoring and peer review functionality and the underlying implementation along with the testing of the existing implementation. Testing of the score calculation, display for the weighted and non-weighted scores based on the combination of the rubrics are part of this project. This project also includes an additional feature for the author to notify the reviewers for the change in the submission.

=====Background=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The scores are calculated based on the rubrics defined in the system. There’s already an existing implementation for the computation of score but it doesn’t work for multipart rubric

=====Standards=====
The new standards that will be adhered to in this system are those given for the rails framework, and the code should follow rails and object oriented design principles. Any newly added code will adhere to the guidelines mentioned below:

● https://github.com/rubocop-hq/rails-style-guide 
● http://wiki.expertiza.ncsu.edu/index.php/Object-Oriented_Design_and_Programming.

== '''Problem Statement''' ==
=====Problem=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The functionality needs to be tested. It works in some cases, but we are suspicious that it is not entirely correct.

'''Profile Email Preference''':
User in expertiza has liberty to control what kind of emails they should receive. As the reviewers will receive an email for the change in submission. This can be controlled by selecting an option (When someone else submits work I am assigned to review) on the profile page of the user. As the checkboxes are far away on the right corner of the screen, the position of checkboxes need to be changed to make it more user friendly.
 

[[File:Generalize_review.jpg]]

Scores for students in Expertiza can be based on a combination of rubrics: review rubrics for each round of review, author feedback, and teammate review. In CSC/ECE 517, we don’t really use the student-assigned scores to grade projects, but the peer-review score shown to the student is based entirely on the second round of review 

Here is how scores are calculated: 

1. If only zero-weighted scores are available (here, in the first round of review), the average of all zero-weighted scores should be shown in grey font. An information button should explain why the scores are shown in grey (“The reviews submitted so far do not count toward the author’s final grade.”): 

2. If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is:: 
80%⨉90% + 100%⨉10% = 82%

3. If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. '''However, the previous review score that was given in the current round will count until the reviewer updates the submitted review'''. '''This functionality doesn’t work for multipart rubrics.''' 

The system also doesn’t generate emails to reviewers in case a new submission is made by the author without changing submission parameters like submission URL. Eg: A new submission is made by making a new deployment. But the deployment URL doesn’t change. In this case as well, without changing the current flow the author should have an option to notify all the reviewers to revisit the submission.

=====Interface Requirement=====
The overall user interface would remain the same as of the current system.

=====Quality Requirements=====
The interface changes are minimal and very easy to adapt. The new button will be clearly visible and is supposed to have a clear text to interpret its functionality. The proposed changes should not affect the current system functionality in any way. The solution is only intended to enrich the project features.

=====Performance Requirements=====
Adding test for multi-part rubric functionality should not cause any overhead in the performance of the system. Sending emails through “Notify All Reviewers” button should affect the system minimally in terms of performance. The only overhead is a database query to get all the reviewers. A good database design should incorporate such a query smoothly.

=====Assumptions=====
The functionality should work in case the assignment rubric is not multi-part.

== '''Proposed Solution''' ==
=====Zero-weighted scores=====
If only zero-weighted scores are available in the first round of review, the average of all zero-weighted scores will be shown in a gray font. An information button will explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade”)

Inside compute_total_score method in on_the_fly_calc.rb module, we'll check if the weighted score exists, if not then we'll calculate the non weighted score. We'll also have to add a method in the questionnaire model to get non weighted score. Few changes are required for the UI to display non-weighted score in grey with information icon and non weighted score in black.

Possible files to be modified.

models/on_the_fly_calc.rb 
models/questionnaire.rb 
models/view_team.html.erb

===== Non-zero weighted scores=====
If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is 80%⨉90% + 100%⨉10% = 82%

The score calculation was initially tested manually to verify the scenarios with multiple weights and rounds.
Also, existing spec for score calculation with a weighted average for both varying and non-varying rubric is covering this scenario.
The spec is written under /spec/models/assignment_spec.rb. The "score" function of the "model/assignment.rb" is called for online (on the fly score calculation).

Following file has the spec for weighted average score calculation.
spec/models/assignment_spec.rb 

=====Review Update=====
If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review.

 This feature can be broken down into the following sub-cases:
 1. A submission can only be made before deadline
 2. A review can edit a review only before deadline
 3. Score calculation should be only done for latest submitted review

Subcases:

 
1. Spec to ensure submission can't be made after the deadline

File : spec/features/post_deadline_review_submission_spec.rb

'''
it 'should not allow submission after deadline' do
user = User.find_by(name: "student2065")
stub_current_user(user, user.role.name, user.role)
# goto student_task page, which has link to "Your work"
visit 'student_task/view?id=1'
# the page will have text "Your work" but will be grayed
expect(page).to have_content "Your work"
# the page will not have link to content "Your work"
expect{click_link "Your work"}.to raise_error(Capybara::ElementNotFound)
end
'''

2. Spec to ensure review can be done only before deadline

File : spec/features/post_deadline_review_submission_spec.rb

'''
it "should not be able to review work after deadline" do
# The spec is written to reproduce following bug. "Others' work" link open after deadline passed
user = User.find_by(name: "student2065")
stub_current_user(user, user.role.name, user.role)
visit 'student_task/view?id=1'
# the page should have content, but after deadline passes it is displayed as gray
# but there should not be any link attached to it
expect(page).to have_content "Others' work"
# this is the bug, even after deadline has passed, the link is still present
# the ui comment in file views/student_task/view.html.erb says
# 
# But the link seems to be open even after deadline passed.
# Screenshot attached as part of wiki for E1975, Fall 2019
expect(page).to have_link("Others' work", "/student_review/list?id=1")
end

'''

The test scenario is able to produce the following bug. At times the review link is open even after the deadline has passed. The bug is also shown/reproduced in the following screenshot for Program 1. 

[[File:ScreenShotReview.jpg]]

3. Score calculation

Score calculation for assignments for review is part of models/assignment.rb. The score calculation is handled by function "score" in this file and the testing is done for both multipart varying and non-varying rubric.
The respective spec is written in file spec/models/assignment_spec.rb.

'''

describe '#scores' do
context 'when assignment is varying rubric by round assignment' do
it 'calculates scores in each round of each team in current assignment' do
allow(participant).to receive(:scores).with(review1: [question]).and_return(98)
allow(assignment).to receive(:varying_rubrics_by_round?).and_return(true)
allow(assignment).to receive(:num_review_rounds).and_return(1)
allow(ReviewResponseMap).to receive(:get_responses_for_team_round).with(team, 1).and_return([response])
allow(Answer).to receive(:compute_scores).with([response], [question]).and_return(max: 95, min: 88, avg: 90)
expect(assignment.scores(review1: [question]).inspect).to eq("{:participants=>{:\"1\"=>98}, :teams=>{:\"0\"=>{:team=>#<AssignmentTeam id: 1, "\
"name: \"no team\", parent_id: 1, type: \"AssignmentTeam\", comments_for_advertisement: nil, advertise_for_partner: nil, "\
"submitted_hyperlinks: \"---\\n- https://www.expertiza.ncsu.edu\", directory_num: 0, grade_for_submission: nil, "\
"comment_for_submission: nil>, :scores=>{:max=>95, :min=>88, :avg=>90.0}}}}")
end
end

context 'when assignment is not varying rubric by round assignment' do
it 'calculates scores of each team in current assignment' do
allow(participant).to receive(:scores).with(review: [question]).and_return(98)
allow(assignment).to receive(:varying_rubrics_by_round?).and_return(false)
allow(ReviewResponseMap).to receive(:get_assessments_for).with(team).and_return([response])
allow(Answer).to receive(:compute_scores).with([response], [question]).and_return(max: 95, min: 88, avg: 90)
expect(assignment.scores(review: [question]).inspect).to eq("{:participants=>{:\"1\"=>98}, :teams=>{:\"0\"=>{:team=>#<AssignmentTeam id: 1, "\
"name: \"no team\", parent_id: 1, type: \"AssignmentTeam\", comments_for_advertisement: nil, advertise_for_partner: nil, "\
"submitted_hyperlinks: \"---\\n- https://www.expertiza.ncsu.edu\", directory_num: 0, grade_for_submission: nil, "\
"comment_for_submission: nil>, :scores=>{:max=>95, :min=>88, :avg=>90}}}}")
end
end
end

'''

 
[[File:Usecasenew.png]]

=='''Test Plan'''==

=====Rspec=====

New test cases will be added to test and validate the functionalities which are not working as expected. Also, the test cases for features that will be newly added to the system, will also be written.
Test cases will be added to these files:

spec/features/post_deadline_review_submission_sepc.rb 
spec/models/assignment_participant_spec.rb 

The scenarios covered by the newly introduced test cases are as follows: 
1. Zero weighted scores with gray mark: rspec spec/model/assignment_participants.rb 
2. Non-zero weighted scores 
3. Review update: rspec spec/features/post_deadline_review_submission_sepc.rb 

=====Manual Testing=====

'''Step 1'''. The instructor will create an assignment and assign the zero and non-zero weights to the rubrics. 
'''Step 2'''. The author submits the work done in expertiza. 
'''Step 3'''. Peers check the submitted work and give reviews for it. 

'''Test 1: Zero weighted scores shown in gray.''' 
Login as an author in expertiza and go to scores page to view the feedback.
Zero weighted scores should be displayed in gray to the logged in user.

'''Test 2: Information button.''' 
Login as an author in expertiza and go to scores page to view the feedback.
The information button should explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade.”)

'''Step 4'''. The mentor evaluates the final submission and give grades.

'''Test 3: Display non-zero weighted scores''' 
Login as an author in expertiza and go to scores page to view the feedback.
The scores shown for the assignment should be the weighted average of the scores.
The zero weighted scores have to be replaced with non-zero weighted scores.

'''Step 5''': The author updates the work and re-submits it again.

'''Test 4: Review Update''' 
The Review should be reopened. The reviewer should be able to go and update the submitted review.
If the review is not updated the previous review score has to be retained.

CSC/ECE 517 Fall 2019 - E1975. Generalize Review Versioning

2019-12-15T22:41:07Z

Atrived: /* Rspec */

__TOC__

== '''Introduction''' ==
Expertiza is a peer-review based learning platform in which work-products by students are evaluated and scored by peer-review of other students. Review process occurs in stages. After submission of initial feedback for a work-product by the peer-group, students are allowed time to incorporate the comments and improve their work. This second version of the work product is reviewed again and scores are given. This process might be repeated again and the average score from the last review stage is considered as the final score for that work-product. This incremental development of work-products and progressive learning is the fundamental concept underlying the Expertiza system.

=====Purpose=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The system also triggers an email to reviewer when there is a change in the submission. The scores are calculated on the fly based on the rubrics.

=====Scope=====
This project will enhance the existing scoring and peer review functionality and the underlying implementation along with the testing of the existing implementation. Testing of the score calculation, display for the weighted and non-weighted scores based on the combination of the rubrics are part of this project. This project also includes an additional feature for the author to notify the reviewers for the change in the submission.

=====Background=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The scores are calculated based on the rubrics defined in the system. There’s already an existing implementation for the computation of score but it doesn’t work for multipart rubric

=====Standards=====
The new standards that will be adhered to in this system are those given for the rails framework, and the code should follow rails and object oriented design principles. Any newly added code will adhere to the guidelines mentioned below:

● https://github.com/rubocop-hq/rails-style-guide 
● http://wiki.expertiza.ncsu.edu/index.php/Object-Oriented_Design_and_Programming.

== '''Problem Statement''' ==
=====Problem=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The functionality needs to be tested. It works in some cases, but we are suspicious that it is not entirely correct.

'''Profile Email Preference''':
User in expertiza has liberty to control what kind of emails they should receive. As the reviewers will receive an email for the change in submission. This can be controlled by selecting an option (When someone else submits work I am assigned to review) on the profile page of the user. As the checkboxes are far away on the right corner of the screen, the position of checkboxes need to be changed to make it more user friendly.
 

Scores for students in Expertiza can be based on a combination of rubrics: review rubrics for each round of review, author feedback, and teammate review. In CSC/ECE 517, we don’t really use the student-assigned scores to grade projects, but the peer-review score shown to the student is based entirely on the second round of review 

Here is how scores are calculated: 

1. If only zero-weighted scores are available (here, in the first round of review), the average of all zero-weighted scores should be shown in grey font. An information button should explain why the scores are shown in grey (“The reviews submitted so far do not count toward the author’s final grade.”): 

2. If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is:: 
80%⨉90% + 100%⨉10% = 82%

3. If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. '''However, the previous review score that was given in the current round will count until the reviewer updates the submitted review'''. '''This functionality doesn’t work for multipart rubrics.''' 

The system also doesn’t generate emails to reviewers in case a new submission is made by the author without changing submission parameters like submission URL. Eg: A new submission is made by making a new deployment. But the deployment URL doesn’t change. In this case as well, without changing the current flow the author should have an option to notify all the reviewers to revisit the submission.

=====Interface Requirement=====
The overall user interface would remain the same as of the current system. Since this is a modification to the current submission page, there will be a new addition to the submission page. On the submission page, there will be a new button to notify reviewers when there is a submission which doesn’t contain a change to submission parameters like submission URL, but the deployment has been updated on the same URL. Other than that the interface remains the same for the score page, only functionalities will be tested for multi-part rubrics.

=====Quality Requirements=====
The interface changes are minimal and very easy to adapt. The new button will be clearly visible and is supposed to have a clear text to interpret its functionality. The proposed changes should not affect the current system functionality in any way. The solution is only intended to enrich the project features.

=====Performance Requirements=====
Adding test for multi-part rubric functionality should not cause any overhead in the performance of the system. Sending emails through “Notify All Reviewers” button should affect the system minimally in terms of performance. The only overhead is a database query to get all the reviewers. A good database design should incorporate such a query smoothly.

=====Assumptions=====
The functionality should work in case the assignment rubric is not multi-part.

== '''Proposed Solution''' ==
=====Zero-weighted scores=====
If only zero-weighted scores are available in the first round of review, the average of all zero-weighted scores will be shown in a gray font. An information button will explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade”)

Inside compute_total_score method in on_the_fly_calc.rb module, we'll check if the weighted score exists, if not then we'll calculate the non weighted score. We'll also have to add a method in the questionnaire model to get non weighted score. Few changes are required for the UI to display non-weighted score in grey with information icon and non weighted score in black.

Possible files to be modified.

models/on_the_fly_calc.rb 
models/questionnaire.rb 
models/view_team.html.erb

===== Non-zero weighted scores=====
If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is 80%⨉90% + 100%⨉10% = 82%

The score calculation was initially tested manually to verify the scenarios with multiple weights and rounds.
Also, existing spec for score calculation with a weighted average for both varying and non-varying rubric is covering this scenario.
The spec is written under /spec/models/assignment_spec.rb. The "score" function of the "model/assignment.rb" is called for online (on the fly score calculation).

Following file has the spec for weighted average score calculation.
spec/models/assignment_spec.rb 

=====Review Update=====
If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review.

 This feature can be broken down into the following sub-cases:
 1. A submission can only be made before deadline
 2. A review can edit a review only before deadline
 3. Score calculation should be only done for latest submitted review

Subcases:

 
1. Spec to ensure submission can't be made after the deadline

File : spec/features/post_deadline_review_submission_spec.rb

'''
it "should not allow submission after deadline" do

user = User.find_by(name: "student2065")
stub_current_user(user, user.role.name, user.role)

# goto student_task page, which has link to "Your work"
visit 'student_task/view?id=1'

# the page will have text "Your work" but will be grayed
expect(page).to have_content "Your work"

# the page will not have link to content "Your work"
expect{click_link "Your work"}.to raise_error(Capybara::ElementNotFound)

end
'''

2. Spec to ensure review can be done only before deadline

File : spec/features/post_deadline_review_submission_spec.rb

'''
it "should not be able to review work after deadline" do

# The spec is written to reproduce following bug. "Others' work" link open after deadline passed

user = User.find_by(name: "student2065")
stub_current_user(user, user.role.name, user.role)
visit 'student_task/view?id=1'

# the page should have content, but after deadline passes it is displayed as gray
# but there should not be any link attached to it
expect(page).to have_content "Others' work"

# this is the bug, even after deadline has passed, the link is still present
# the ui comment in file views/student_task/view.html.erb says
# 
# But the link seems to be open even after deadline passed.
# Screenshot attached as part of wiki for E1975, Fall 2019
expect(page).to have_link("Others' work", "/student_review/list?id=1")

end
'''

The test scenario is able to produce the following bug. At times the review link is open even after the deadline has passed. The bug is also shown/reproduced in the following screenshot for Program 1. 

[[File:ReviewBug.png]]

3. Score calculation

Score calculation for assignments for review is part of models/assignment.rb. The score calculation is handled by function "score" in this file and the testing is done for both multipart varying and non-varying rubric.
The respective spec is written in file spec/models/assignment_spec.rb.

'''

describe '#scores' do
context 'when assignment is varying rubric by round assignment' do
it 'calculates scores in each round of each team in current assignment' do
allow(participant).to receive(:scores).with(review1: [question]).and_return(98)
allow(assignment).to receive(:varying_rubrics_by_round?).and_return(true)
allow(assignment).to receive(:num_review_rounds).and_return(1)
allow(ReviewResponseMap).to receive(:get_responses_for_team_round).with(team, 1).and_return([response])
allow(Answer).to receive(:compute_scores).with([response], [question]).and_return(max: 95, min: 88, avg: 90)
expect(assignment.scores(review1: [question]).inspect).to eq("{:participants=>{:\"1\"=>98}, :teams=>{:\"0\"=>{:team=>#<AssignmentTeam id: 1, "\
"name: \"no team\", parent_id: 1, type: \"AssignmentTeam\", comments_for_advertisement: nil, advertise_for_partner: nil, "\
"submitted_hyperlinks: \"---\\n- https://www.expertiza.ncsu.edu\", directory_num: 0, grade_for_submission: nil, "\
"comment_for_submission: nil>, :scores=>{:max=>95, :min=>88, :avg=>90.0}}}}")
end
end

context 'when assignment is not varying rubric by round assignment' do
it 'calculates scores of each team in current assignment' do
allow(participant).to receive(:scores).with(review: [question]).and_return(98)
allow(assignment).to receive(:varying_rubrics_by_round?).and_return(false)
allow(ReviewResponseMap).to receive(:get_assessments_for).with(team).and_return([response])
allow(Answer).to receive(:compute_scores).with([response], [question]).and_return(max: 95, min: 88, avg: 90)
expect(assignment.scores(review: [question]).inspect).to eq("{:participants=>{:\"1\"=>98}, :teams=>{:\"0\"=>{:team=>#<AssignmentTeam id: 1, "\
"name: \"no team\", parent_id: 1, type: \"AssignmentTeam\", comments_for_advertisement: nil, advertise_for_partner: nil, "\
"submitted_hyperlinks: \"---\\n- https://www.expertiza.ncsu.edu\", directory_num: 0, grade_for_submission: nil, "\
"comment_for_submission: nil>, :scores=>{:max=>95, :min=>88, :avg=>90}}}}")
end
end
end

'''

[[File:E1975_CHECKBOX_1.jpg]]

 
[[File:Usecasenew.png]]

=='''Test Plan'''==

=====Rspec=====

New test cases will be added to test and validate the functionalities which are not working as expected. Also, the test cases for features that will be newly added to the system, will also be written.
Test cases will be added to these files:

spec/features/post_deadline_review_submission_sepc.rb 
spec/models/assignment_participant_spec.rb 

The scenarios covered by the newly introduced test cases are as follows: 
1. Zero weighted scores with gray mark: rspec spec/model/assignment_participants.rb 
2. Non-zero weighted scores 
3. Review update: rspec spec/features/post_deadline_review_submission_sepc.rb 

=====Manual Testing=====

'''Step 1'''. The instructor will create an assignment and assign the zero and non-zero weights to the rubrics. 
'''Step 2'''. The author submits the work done in expertiza. 
'''Step 3'''. Peers check the submitted work and give reviews for it. 

'''Test 1: Zero weighted scores shown in gray.''' 
Login as an author in expertiza and go to scores page to view the feedback.
Zero weighted scores should be displayed in gray to the logged in user.

'''Test 2: Information button.''' 
Login as an author in expertiza and go to scores page to view the feedback.
The information button should explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade.”)

'''Step 4'''. The mentor evaluates the final submission and give grades.

'''Test 3: Display non-zero weighted scores''' 
Login as an author in expertiza and go to scores page to view the feedback.
The scores shown for the assignment should be the weighted average of the scores.
The zero weighted scores have to be replaced with non-zero weighted scores.

'''Step 5''': The author updates the work and re-submits it again.

'''Test 4: Review Update''' 
The Review should be reopened. The reviewer should be able to go and update the submitted review.
If the review is not updated the previous review score has to be retained.

CSC/ECE 517 Fall 2019 - E1975. Generalize Review Versioning

2019-12-15T22:35:02Z

Atrived: /* Non-zero weighted scores */

__TOC__

== '''Introduction''' ==
Expertiza is a peer-review based learning platform in which work-products by students are evaluated and scored by peer-review of other students. Review process occurs in stages. After submission of initial feedback for a work-product by the peer-group, students are allowed time to incorporate the comments and improve their work. This second version of the work product is reviewed again and scores are given. This process might be repeated again and the average score from the last review stage is considered as the final score for that work-product. This incremental development of work-products and progressive learning is the fundamental concept underlying the Expertiza system.

=====Purpose=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The system also triggers an email to reviewer when there is a change in the submission. The scores are calculated on the fly based on the rubrics.

=====Scope=====
This project will enhance the existing scoring and peer review functionality and the underlying implementation along with the testing of the existing implementation. Testing of the score calculation, display for the weighted and non-weighted scores based on the combination of the rubrics are part of this project. This project also includes an additional feature for the author to notify the reviewers for the change in the submission.

=====Background=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The scores are calculated based on the rubrics defined in the system. There’s already an existing implementation for the computation of score but it doesn’t work for multipart rubric

=====Standards=====
The new standards that will be adhered to in this system are those given for the rails framework, and the code should follow rails and object oriented design principles. Any newly added code will adhere to the guidelines mentioned below:

● https://github.com/rubocop-hq/rails-style-guide 
● http://wiki.expertiza.ncsu.edu/index.php/Object-Oriented_Design_and_Programming.

== '''Problem Statement''' ==
=====Problem=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The functionality needs to be tested. It works in some cases, but we are suspicious that it is not entirely correct.

'''Profile Email Preference''':
User in expertiza has liberty to control what kind of emails they should receive. As the reviewers will receive an email for the change in submission. This can be controlled by selecting an option (When someone else submits work I am assigned to review) on the profile page of the user. As the checkboxes are far away on the right corner of the screen, the position of checkboxes need to be changed to make it more user friendly.
 

Scores for students in Expertiza can be based on a combination of rubrics: review rubrics for each round of review, author feedback, and teammate review. In CSC/ECE 517, we don’t really use the student-assigned scores to grade projects, but the peer-review score shown to the student is based entirely on the second round of review 

Here is how scores are calculated: 

1. If only zero-weighted scores are available (here, in the first round of review), the average of all zero-weighted scores should be shown in grey font. An information button should explain why the scores are shown in grey (“The reviews submitted so far do not count toward the author’s final grade.”): 

2. If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is:: 
80%⨉90% + 100%⨉10% = 82%

3. If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. '''However, the previous review score that was given in the current round will count until the reviewer updates the submitted review'''. '''This functionality doesn’t work for multipart rubrics.''' 

The system also doesn’t generate emails to reviewers in case a new submission is made by the author without changing submission parameters like submission URL. Eg: A new submission is made by making a new deployment. But the deployment URL doesn’t change. In this case as well, without changing the current flow the author should have an option to notify all the reviewers to revisit the submission.

=====Interface Requirement=====
The overall user interface would remain the same as of the current system. Since this is a modification to the current submission page, there will be a new addition to the submission page. On the submission page, there will be a new button to notify reviewers when there is a submission which doesn’t contain a change to submission parameters like submission URL, but the deployment has been updated on the same URL. Other than that the interface remains the same for the score page, only functionalities will be tested for multi-part rubrics.

=====Quality Requirements=====
The interface changes are minimal and very easy to adapt. The new button will be clearly visible and is supposed to have a clear text to interpret its functionality. The proposed changes should not affect the current system functionality in any way. The solution is only intended to enrich the project features.

=====Performance Requirements=====
Adding test for multi-part rubric functionality should not cause any overhead in the performance of the system. Sending emails through “Notify All Reviewers” button should affect the system minimally in terms of performance. The only overhead is a database query to get all the reviewers. A good database design should incorporate such a query smoothly.

=====Assumptions=====
The functionality should work in case the assignment rubric is not multi-part.

== '''Proposed Solution''' ==
=====Zero-weighted scores=====
If only zero-weighted scores are available in the first round of review, the average of all zero-weighted scores will be shown in a gray font. An information button will explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade”)

Inside compute_total_score method in on_the_fly_calc.rb module, we'll check if the weighted score exists, if not then we'll calculate the non weighted score. We'll also have to add a method in the questionnaire model to get non weighted score. Few changes are required for the UI to display non-weighted score in grey with information icon and non weighted score in black.

Possible files to be modified.

models/on_the_fly_calc.rb 
models/questionnaire.rb 
models/view_team.html.erb

===== Non-zero weighted scores=====
If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is 80%⨉90% + 100%⨉10% = 82%

The score calculation was initially tested manually to verify the scenarios with multiple weights and rounds.
Also, existing spec for score calculation with a weighted average for both varying and non-varying rubric is covering this scenario.
The spec is written under /spec/models/assignment_spec.rb. The "score" function of the "model/assignment.rb" is called for online (on the fly score calculation).

Following file has the spec for weighted average score calculation.
spec/models/assignment_spec.rb 

=====Review Update=====
If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review.

 This feature can be broken down into the following sub-cases:
 1. A submission can only be made before deadline
 2. A review can edit a review only before deadline
 3. Score calculation should be only done for latest submitted review

Subcases:

 
1. Spec to ensure submission can't be made after the deadline

File : spec/features/post_deadline_review_submission_spec.rb

'''
it "should not allow submission after deadline" do

user = User.find_by(name: "student2065")
stub_current_user(user, user.role.name, user.role)

# goto student_task page, which has link to "Your work"
visit 'student_task/view?id=1'

# the page will have text "Your work" but will be grayed
expect(page).to have_content "Your work"

# the page will not have link to content "Your work"
expect{click_link "Your work"}.to raise_error(Capybara::ElementNotFound)

end
'''

2. Spec to ensure review can be done only before deadline

File : spec/features/post_deadline_review_submission_spec.rb

'''
it "should not be able to review work after deadline" do

# The spec is written to reproduce following bug. "Others' work" link open after deadline passed

user = User.find_by(name: "student2065")
stub_current_user(user, user.role.name, user.role)
visit 'student_task/view?id=1'

# the page should have content, but after deadline passes it is displayed as gray
# but there should not be any link attached to it
expect(page).to have_content "Others' work"

# this is the bug, even after deadline has passed, the link is still present
# the ui comment in file views/student_task/view.html.erb says
# 
# But the link seems to be open even after deadline passed.
# Screenshot attached as part of wiki for E1975, Fall 2019
expect(page).to have_link("Others' work", "/student_review/list?id=1")

end
'''

The test scenario is able to produce the following bug. At times the review link is open even after the deadline has passed. The bug is also shown/reproduced in the following screenshot for Program 1. 

[[File:ReviewBug.png]]

3. Score calculation

Score calculation for assignments for review is part of models/assignment.rb. The score calculation is handled by function "score" in this file and the testing is done for both multipart varying and non-varying rubric.
The respective spec is written in file spec/models/assignment_spec.rb.

'''

describe '#scores' do
context 'when assignment is varying rubric by round assignment' do
it 'calculates scores in each round of each team in current assignment' do
allow(participant).to receive(:scores).with(review1: [question]).and_return(98)
allow(assignment).to receive(:varying_rubrics_by_round?).and_return(true)
allow(assignment).to receive(:num_review_rounds).and_return(1)
allow(ReviewResponseMap).to receive(:get_responses_for_team_round).with(team, 1).and_return([response])
allow(Answer).to receive(:compute_scores).with([response], [question]).and_return(max: 95, min: 88, avg: 90)
expect(assignment.scores(review1: [question]).inspect).to eq("{:participants=>{:\"1\"=>98}, :teams=>{:\"0\"=>{:team=>#<AssignmentTeam id: 1, "\
"name: \"no team\", parent_id: 1, type: \"AssignmentTeam\", comments_for_advertisement: nil, advertise_for_partner: nil, "\
"submitted_hyperlinks: \"---\\n- https://www.expertiza.ncsu.edu\", directory_num: 0, grade_for_submission: nil, "\
"comment_for_submission: nil>, :scores=>{:max=>95, :min=>88, :avg=>90.0}}}}")
end
end

context 'when assignment is not varying rubric by round assignment' do
it 'calculates scores of each team in current assignment' do
allow(participant).to receive(:scores).with(review: [question]).and_return(98)
allow(assignment).to receive(:varying_rubrics_by_round?).and_return(false)
allow(ReviewResponseMap).to receive(:get_assessments_for).with(team).and_return([response])
allow(Answer).to receive(:compute_scores).with([response], [question]).and_return(max: 95, min: 88, avg: 90)
expect(assignment.scores(review: [question]).inspect).to eq("{:participants=>{:\"1\"=>98}, :teams=>{:\"0\"=>{:team=>#<AssignmentTeam id: 1, "\
"name: \"no team\", parent_id: 1, type: \"AssignmentTeam\", comments_for_advertisement: nil, advertise_for_partner: nil, "\
"submitted_hyperlinks: \"---\\n- https://www.expertiza.ncsu.edu\", directory_num: 0, grade_for_submission: nil, "\
"comment_for_submission: nil>, :scores=>{:max=>95, :min=>88, :avg=>90}}}}")
end
end
end

'''

[[File:E1975_CHECKBOX_1.jpg]]

 
[[File:Usecasenew.png]]

=='''Test Plan'''==

=====Rspec=====

New test cases will be added to test and validate the functionalities which are not working as expected. Also the test cases for features that will be newly added to the system, will also be written. Test cases will be added to these files:
spec/models/on_the_fly_calc_spec.rb 
spec/models/questionnaire_spec.rb 

The scenarios covered by the newly introduced test cases are as follows: 
1. Zero weighted scores 
2. Non-zero weighted scores 
3. Review update 

=====Manual Testing=====

'''Step 1'''. The instructor will create an assignment and assign the zero and non-zero weights to the rubrics. 
'''Step 2'''. The author submits the work done in expertiza. 
'''Step 3'''. Peers check the submitted work and give reviews for it. 

'''Test 1: Zero weighted scores shown in gray.''' 
Login as an author in expertiza and go to scores page to view the feedback.
Zero weighted scores should be displayed in gray to the logged in user.

'''Test 2: Information button.''' 
Login as an author in expertiza and go to scores page to view the feedback.
The information button should explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade.”)

'''Step 4'''. The mentor evaluates the final submission and give grades.

'''Test 3: Display non-zero weighted scores''' 
Login as an author in expertiza and go to scores page to view the feedback.
The scores shown for the assignment should be the weighted average of the scores.
The zero weighted scores have to be replaced with non-zero weighted scores.

'''Step 5''': The author updates the work and re-submits it again.

'''Test 4: Review Update''' 
The Review should be reopened. The reviewer should be able to go and update the submitted review.
If the review is not updated the previous review score has to be retained.

CSC/ECE 517 Fall 2019 - E1975. Generalize Review Versioning

2019-12-15T21:45:30Z

Atrived: /* Review Update */

__TOC__

== '''Introduction''' ==
Expertiza is a peer-review based learning platform in which work-products by students are evaluated and scored by peer-review of other students. Review process occurs in stages. After submission of initial feedback for a work-product by the peer-group, students are allowed time to incorporate the comments and improve their work. This second version of the work product is reviewed again and scores are given. This process might be repeated again and the average score from the last review stage is considered as the final score for that work-product. This incremental development of work-products and progressive learning is the fundamental concept underlying the Expertiza system.

=====Purpose=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The system also triggers an email to reviewer when there is a change in the submission. The scores are calculated on the fly based on the rubrics.

=====Scope=====
This project will enhance the existing scoring and peer review functionality and the underlying implementation along with the testing of the existing implementation. Testing of the score calculation, display for the weighted and non-weighted scores based on the combination of the rubrics are part of this project. This project also includes an additional feature for the author to notify the reviewers for the change in the submission.

=====Background=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The scores are calculated based on the rubrics defined in the system. There’s already an existing implementation for the computation of score but it doesn’t work for multipart rubric

=====Standards=====
The new standards that will be adhered to in this system are those given for the rails framework, and the code should follow rails and object oriented design principles. Any newly added code will adhere to the guidelines mentioned below:

● https://github.com/rubocop-hq/rails-style-guide 
● http://wiki.expertiza.ncsu.edu/index.php/Object-Oriented_Design_and_Programming.

== '''Problem Statement''' ==
=====Problem=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The functionality needs to be tested. It works in some cases, but we are suspicious that it is not entirely correct.

'''Profile Email Preference''':
User in expertiza has liberty to control what kind of emails they should receive. As the reviewers will receive an email for the change in submission. This can be controlled by selecting an option (When someone else submits work I am assigned to review) on the profile page of the user. As the checkboxes are far away on the right corner of the screen, the position of checkboxes need to be changed to make it more user friendly.
 

Scores for students in Expertiza can be based on a combination of rubrics: review rubrics for each round of review, author feedback, and teammate review. In CSC/ECE 517, we don’t really use the student-assigned scores to grade projects, but the peer-review score shown to the student is based entirely on the second round of review 

Here is how scores are calculated: 

1. If only zero-weighted scores are available (here, in the first round of review), the average of all zero-weighted scores should be shown in grey font. An information button should explain why the scores are shown in grey (“The reviews submitted so far do not count toward the author’s final grade.”): 

2. If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is:: 
80%⨉90% + 100%⨉10% = 82%

3. If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. '''However, the previous review score that was given in the current round will count until the reviewer updates the submitted review'''. '''This functionality doesn’t work for multipart rubrics.''' 

The system also doesn’t generate emails to reviewers in case a new submission is made by the author without changing submission parameters like submission URL. Eg: A new submission is made by making a new deployment. But the deployment URL doesn’t change. In this case as well, without changing the current flow the author should have an option to notify all the reviewers to revisit the submission.

=====Interface Requirement=====
The overall user interface would remain the same as of the current system. Since this is a modification to the current submission page, there will be a new addition to the submission page. On the submission page, there will be a new button to notify reviewers when there is a submission which doesn’t contain a change to submission parameters like submission URL, but the deployment has been updated on the same URL. Other than that the interface remains the same for the score page, only functionalities will be tested for multi-part rubrics.

=====Quality Requirements=====
The interface changes are minimal and very easy to adapt. The new button will be clearly visible and is supposed to have a clear text to interpret its functionality. The proposed changes should not affect the current system functionality in any way. The solution is only intended to enrich the project features.

=====Performance Requirements=====
Adding test for multi-part rubric functionality should not cause any overhead in the performance of the system. Sending emails through “Notify All Reviewers” button should affect the system minimally in terms of performance. The only overhead is a database query to get all the reviewers. A good database design should incorporate such a query smoothly.

=====Assumptions=====
The functionality should work in case the assignment rubric is not multi-part.

== '''Proposed Solution''' ==
=====Zero-weighted scores=====
If only zero-weighted scores are available in the first round of review, the average of all zero-weighted scores will be shown in a gray font. An information button will explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade”)

Inside compute_total_score method in on_the_fly_calc.rb module, we'll check if the weighted score exists, if not then we'll calculate the non weighted score. We'll also have to add a method in the questionnaire model to get non weighted score. Few changes are required for the UI to display non-weighted score in grey with information icon and non weighted score in black.

Possible files to be modified.

models/on_the_fly_calc.rb 
models/questionnaire.rb 
models/view_team.html.erb

===== Non-zero weighted scores=====
If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is 80%⨉90% + 100%⨉10% = 82%

We'll first write the test cases by adding test cases to the '''spec/models/assignment_spec.rb'''

Also, we'll test this manually by computing the score and if there's a bug then below are the possible files to be modified.

spec/models/assignment_spec.rb 

=====Review Update=====
If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review.

 This feature can be broken down into the following sub-cases:
 1. A submission can only be made before deadline
 2. A review can edit a review only before deadline
 3. Score calculation should be only done for latest submitted review

Subcases:

 
1. Spec to ensure submission can't be made after the deadline

File : spec/features/post_deadline_review_submission_spec.rb

'''
it "should not allow submission after deadline" do

user = User.find_by(name: "student2065")
stub_current_user(user, user.role.name, user.role)

# goto student_task page, which has link to "Your work"
visit 'student_task/view?id=1'

# the page will have text "Your work" but will be grayed
expect(page).to have_content "Your work"

# the page will not have link to content "Your work"
expect{click_link "Your work"}.to raise_error(Capybara::ElementNotFound)

end
'''

2. Spec to ensure review can be done only before deadline

File : spec/features/post_deadline_review_submission_spec.rb

'''
it "should not be able to review work after deadline" do

# The spec is written to reproduce following bug. "Others' work" link open after deadline passed

user = User.find_by(name: "student2065")
stub_current_user(user, user.role.name, user.role)
visit 'student_task/view?id=1'

# the page should have content, but after deadline passes it is displayed as gray
# but there should not be any link attached to it
expect(page).to have_content "Others' work"

# this is the bug, even after deadline has passed, the link is still present
# the ui comment in file views/student_task/view.html.erb says
# 
# But the link seems to be open even after deadline passed.
# Screenshot attached as part of wiki for E1975, Fall 2019
expect(page).to have_link("Others' work", "/student_review/list?id=1")

end
'''

The test scenario is able to produce the following bug. At times the review link is open even after the deadline has passed. The bug is also shown/reproduced in the following screenshot for Program 1. 

[[File:ReviewBug.png]]

3. Score calculation

Score calculation for assignments for review is part of models/assignment.rb. The score calculation is handled by function "score" in this file and the testing is done for both multipart varying and non-varying rubric.
The respective spec is written in file spec/models/assignment_spec.rb.

'''

describe '#scores' do
context 'when assignment is varying rubric by round assignment' do
it 'calculates scores in each round of each team in current assignment' do
allow(participant).to receive(:scores).with(review1: [question]).and_return(98)
allow(assignment).to receive(:varying_rubrics_by_round?).and_return(true)
allow(assignment).to receive(:num_review_rounds).and_return(1)
allow(ReviewResponseMap).to receive(:get_responses_for_team_round).with(team, 1).and_return([response])
allow(Answer).to receive(:compute_scores).with([response], [question]).and_return(max: 95, min: 88, avg: 90)
expect(assignment.scores(review1: [question]).inspect).to eq("{:participants=>{:\"1\"=>98}, :teams=>{:\"0\"=>{:team=>#<AssignmentTeam id: 1, "\
"name: \"no team\", parent_id: 1, type: \"AssignmentTeam\", comments_for_advertisement: nil, advertise_for_partner: nil, "\
"submitted_hyperlinks: \"---\\n- https://www.expertiza.ncsu.edu\", directory_num: 0, grade_for_submission: nil, "\
"comment_for_submission: nil>, :scores=>{:max=>95, :min=>88, :avg=>90.0}}}}")
end
end

context 'when assignment is not varying rubric by round assignment' do
it 'calculates scores of each team in current assignment' do
allow(participant).to receive(:scores).with(review: [question]).and_return(98)
allow(assignment).to receive(:varying_rubrics_by_round?).and_return(false)
allow(ReviewResponseMap).to receive(:get_assessments_for).with(team).and_return([response])
allow(Answer).to receive(:compute_scores).with([response], [question]).and_return(max: 95, min: 88, avg: 90)
expect(assignment.scores(review: [question]).inspect).to eq("{:participants=>{:\"1\"=>98}, :teams=>{:\"0\"=>{:team=>#<AssignmentTeam id: 1, "\
"name: \"no team\", parent_id: 1, type: \"AssignmentTeam\", comments_for_advertisement: nil, advertise_for_partner: nil, "\
"submitted_hyperlinks: \"---\\n- https://www.expertiza.ncsu.edu\", directory_num: 0, grade_for_submission: nil, "\
"comment_for_submission: nil>, :scores=>{:max=>95, :min=>88, :avg=>90}}}}")
end
end
end

'''

[[File:E1975_CHECKBOX_1.jpg]]

 
[[File:Usecasenew.png]]

=='''Test Plan'''==

=====Rspec=====

New test cases will be added to test and validate the functionalities which are not working as expected. Also the test cases for features that will be newly added to the system, will also be written. Test cases will be added to these files:
spec/models/on_the_fly_calc_spec.rb 
spec/models/questionnaire_spec.rb 

The scenarios covered by the newly introduced test cases are as follows: 
1. Zero weighted scores 
2. Non-zero weighted scores 
3. Review update 

=====Manual Testing=====

'''Step 1'''. The instructor will create an assignment and assign the zero and non-zero weights to the rubrics. 
'''Step 2'''. The author submits the work done in expertiza. 
'''Step 3'''. Peers check the submitted work and give reviews for it. 

'''Test 1: Zero weighted scores shown in gray.''' 
Login as an author in expertiza and go to scores page to view the feedback.
Zero weighted scores should be displayed in gray to the logged in user.

'''Test 2: Information button.''' 
Login as an author in expertiza and go to scores page to view the feedback.
The information button should explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade.”)

'''Step 4'''. The mentor evaluates the final submission and give grades.

'''Test 3: Display non-zero weighted scores''' 
Login as an author in expertiza and go to scores page to view the feedback.
The scores shown for the assignment should be the weighted average of the scores.
The zero weighted scores have to be replaced with non-zero weighted scores.

'''Step 5''': The author updates the work and re-submits it again.

'''Test 4: Review Update''' 
The Review should be reopened. The reviewer should be able to go and update the submitted review.
If the review is not updated the previous review score has to be retained.

CSC/ECE 517 Fall 2019 - E1975. Generalize Review Versioning

2019-12-15T21:40:35Z

Atrived: /* Review Update */

__TOC__

== '''Introduction''' ==
Expertiza is a peer-review based learning platform in which work-products by students are evaluated and scored by peer-review of other students. Review process occurs in stages. After submission of initial feedback for a work-product by the peer-group, students are allowed time to incorporate the comments and improve their work. This second version of the work product is reviewed again and scores are given. This process might be repeated again and the average score from the last review stage is considered as the final score for that work-product. This incremental development of work-products and progressive learning is the fundamental concept underlying the Expertiza system.

=====Purpose=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The system also triggers an email to reviewer when there is a change in the submission. The scores are calculated on the fly based on the rubrics.

=====Scope=====
This project will enhance the existing scoring and peer review functionality and the underlying implementation along with the testing of the existing implementation. Testing of the score calculation, display for the weighted and non-weighted scores based on the combination of the rubrics are part of this project. This project also includes an additional feature for the author to notify the reviewers for the change in the submission.

=====Background=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The scores are calculated based on the rubrics defined in the system. There’s already an existing implementation for the computation of score but it doesn’t work for multipart rubric

=====Standards=====
The new standards that will be adhered to in this system are those given for the rails framework, and the code should follow rails and object oriented design principles. Any newly added code will adhere to the guidelines mentioned below:

● https://github.com/rubocop-hq/rails-style-guide 
● http://wiki.expertiza.ncsu.edu/index.php/Object-Oriented_Design_and_Programming.

== '''Problem Statement''' ==
=====Problem=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The functionality needs to be tested. It works in some cases, but we are suspicious that it is not entirely correct.

'''Profile Email Preference''':
User in expertiza has liberty to control what kind of emails they should receive. As the reviewers will receive an email for the change in submission. This can be controlled by selecting an option (When someone else submits work I am assigned to review) on the profile page of the user. As the checkboxes are far away on the right corner of the screen, the position of checkboxes need to be changed to make it more user friendly.
 

Scores for students in Expertiza can be based on a combination of rubrics: review rubrics for each round of review, author feedback, and teammate review. In CSC/ECE 517, we don’t really use the student-assigned scores to grade projects, but the peer-review score shown to the student is based entirely on the second round of review 

Here is how scores are calculated: 

1. If only zero-weighted scores are available (here, in the first round of review), the average of all zero-weighted scores should be shown in grey font. An information button should explain why the scores are shown in grey (“The reviews submitted so far do not count toward the author’s final grade.”): 

2. If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is:: 
80%⨉90% + 100%⨉10% = 82%

3. If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. '''However, the previous review score that was given in the current round will count until the reviewer updates the submitted review'''. '''This functionality doesn’t work for multipart rubrics.''' 

The system also doesn’t generate emails to reviewers in case a new submission is made by the author without changing submission parameters like submission URL. Eg: A new submission is made by making a new deployment. But the deployment URL doesn’t change. In this case as well, without changing the current flow the author should have an option to notify all the reviewers to revisit the submission.

=====Interface Requirement=====
The overall user interface would remain the same as of the current system. Since this is a modification to the current submission page, there will be a new addition to the submission page. On the submission page, there will be a new button to notify reviewers when there is a submission which doesn’t contain a change to submission parameters like submission URL, but the deployment has been updated on the same URL. Other than that the interface remains the same for the score page, only functionalities will be tested for multi-part rubrics.

=====Quality Requirements=====
The interface changes are minimal and very easy to adapt. The new button will be clearly visible and is supposed to have a clear text to interpret its functionality. The proposed changes should not affect the current system functionality in any way. The solution is only intended to enrich the project features.

=====Performance Requirements=====
Adding test for multi-part rubric functionality should not cause any overhead in the performance of the system. Sending emails through “Notify All Reviewers” button should affect the system minimally in terms of performance. The only overhead is a database query to get all the reviewers. A good database design should incorporate such a query smoothly.

=====Assumptions=====
The functionality should work in case the assignment rubric is not multi-part.

== '''Proposed Solution''' ==
=====Zero-weighted scores=====
If only zero-weighted scores are available in the first round of review, the average of all zero-weighted scores will be shown in a gray font. An information button will explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade”)

Inside compute_total_score method in on_the_fly_calc.rb module, we'll check if the weighted score exists, if not then we'll calculate the non weighted score. We'll also have to add a method in the questionnaire model to get non weighted score. Few changes are required for the UI to display non-weighted score in grey with information icon and non weighted score in black.

Possible files to be modified.

models/on_the_fly_calc.rb 
models/questionnaire.rb 
models/view_team.html.erb

===== Non-zero weighted scores=====
If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is 80%⨉90% + 100%⨉10% = 82%

We'll first write the test cases by adding test cases to the '''spec/models/assignment_spec.rb'''

Also, we'll test this manually by computing the score and if there's a bug then below are the possible files to be modified.

spec/models/assignment_spec.rb 

=====Review Update=====
If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review.

 This feature can be broken down into the following sub-cases:
 1. A submission can only be made before deadline
 2. A review can edit a review only before deadline
 3. Score calculation should be only done for latest submitted review

Subcases:

 
1. Spec to ensure submission can't be made after the deadline

File : spec/features/post_deadline_review_submission_spec.rb

'''
it "should not allow submission after deadline" do

user = User.find_by(name: "student2065")
stub_current_user(user, user.role.name, user.role)

# goto student_task page, which has link to "Your work"
visit 'student_task/view?id=1'

# the page will have text "Your work" but will be grayed
expect(page).to have_content "Your work"

# the page will not have link to content "Your work"
expect{click_link "Your work"}.to raise_error(Capybara::ElementNotFound)

end
'''

2. Spec to ensure review can be done only before deadline

File : spec/features/post_deadline_review_submission_spec.rb

'''
it "should not be able to review work after deadline" do

# The spec is written to reproduce following bug. "Others' work" link open after deadline passed

user = User.find_by(name: "student2065")
stub_current_user(user, user.role.name, user.role)
visit 'student_task/view?id=1'

# the page should have content, but after deadline passes it is displayed as gray
# but there should not be any link attached to it
expect(page).to have_content "Others' work"

# this is the bug, even after deadline has passed, the link is still present
# the ui comment in file views/student_task/view.html.erb says
# 
# But the link seems to be open even after deadline passed.
# Screenshot attached as part of wiki for E1975, Fall 2019
expect(page).to have_link("Others' work", "/student_review/list?id=1")

end
'''

The test scenario is able to produce the following bug. At times the review link is open even after the deadline has passed. The bug is also shown/reproduced in the following screenshot for Program 1. 

[[File:ReviewBug.png]]

[[File:E1975_CHECKBOX_1.jpg]]

 
[[File:Usecasenew.png]]

=='''Test Plan'''==

=====Rspec=====

New test cases will be added to test and validate the functionalities which are not working as expected. Also the test cases for features that will be newly added to the system, will also be written. Test cases will be added to these files:
spec/models/on_the_fly_calc_spec.rb 
spec/models/questionnaire_spec.rb 

The scenarios covered by the newly introduced test cases are as follows: 
1. Zero weighted scores 
2. Non-zero weighted scores 
3. Review update 

=====Manual Testing=====

'''Step 1'''. The instructor will create an assignment and assign the zero and non-zero weights to the rubrics. 
'''Step 2'''. The author submits the work done in expertiza. 
'''Step 3'''. Peers check the submitted work and give reviews for it. 

'''Test 1: Zero weighted scores shown in gray.''' 
Login as an author in expertiza and go to scores page to view the feedback.
Zero weighted scores should be displayed in gray to the logged in user.

'''Test 2: Information button.''' 
Login as an author in expertiza and go to scores page to view the feedback.
The information button should explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade.”)

'''Step 4'''. The mentor evaluates the final submission and give grades.

'''Test 3: Display non-zero weighted scores''' 
Login as an author in expertiza and go to scores page to view the feedback.
The scores shown for the assignment should be the weighted average of the scores.
The zero weighted scores have to be replaced with non-zero weighted scores.

'''Step 5''': The author updates the work and re-submits it again.

'''Test 4: Review Update''' 
The Review should be reopened. The reviewer should be able to go and update the submitted review.
If the review is not updated the previous review score has to be retained.

CSC/ECE 517 Fall 2019 - E1975. Generalize Review Versioning

2019-12-15T21:40:00Z

Atrived: /* Review Update */

__TOC__

== '''Introduction''' ==
Expertiza is a peer-review based learning platform in which work-products by students are evaluated and scored by peer-review of other students. Review process occurs in stages. After submission of initial feedback for a work-product by the peer-group, students are allowed time to incorporate the comments and improve their work. This second version of the work product is reviewed again and scores are given. This process might be repeated again and the average score from the last review stage is considered as the final score for that work-product. This incremental development of work-products and progressive learning is the fundamental concept underlying the Expertiza system.

=====Purpose=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The system also triggers an email to reviewer when there is a change in the submission. The scores are calculated on the fly based on the rubrics.

=====Scope=====
This project will enhance the existing scoring and peer review functionality and the underlying implementation along with the testing of the existing implementation. Testing of the score calculation, display for the weighted and non-weighted scores based on the combination of the rubrics are part of this project. This project also includes an additional feature for the author to notify the reviewers for the change in the submission.

=====Background=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The scores are calculated based on the rubrics defined in the system. There’s already an existing implementation for the computation of score but it doesn’t work for multipart rubric

=====Standards=====
The new standards that will be adhered to in this system are those given for the rails framework, and the code should follow rails and object oriented design principles. Any newly added code will adhere to the guidelines mentioned below:

● https://github.com/rubocop-hq/rails-style-guide 
● http://wiki.expertiza.ncsu.edu/index.php/Object-Oriented_Design_and_Programming.

== '''Problem Statement''' ==
=====Problem=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The functionality needs to be tested. It works in some cases, but we are suspicious that it is not entirely correct.

'''Profile Email Preference''':
User in expertiza has liberty to control what kind of emails they should receive. As the reviewers will receive an email for the change in submission. This can be controlled by selecting an option (When someone else submits work I am assigned to review) on the profile page of the user. As the checkboxes are far away on the right corner of the screen, the position of checkboxes need to be changed to make it more user friendly.
 

Scores for students in Expertiza can be based on a combination of rubrics: review rubrics for each round of review, author feedback, and teammate review. In CSC/ECE 517, we don’t really use the student-assigned scores to grade projects, but the peer-review score shown to the student is based entirely on the second round of review 

Here is how scores are calculated: 

1. If only zero-weighted scores are available (here, in the first round of review), the average of all zero-weighted scores should be shown in grey font. An information button should explain why the scores are shown in grey (“The reviews submitted so far do not count toward the author’s final grade.”): 

2. If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is:: 
80%⨉90% + 100%⨉10% = 82%

3. If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. '''However, the previous review score that was given in the current round will count until the reviewer updates the submitted review'''. '''This functionality doesn’t work for multipart rubrics.''' 

The system also doesn’t generate emails to reviewers in case a new submission is made by the author without changing submission parameters like submission URL. Eg: A new submission is made by making a new deployment. But the deployment URL doesn’t change. In this case as well, without changing the current flow the author should have an option to notify all the reviewers to revisit the submission.

=====Interface Requirement=====
The overall user interface would remain the same as of the current system. Since this is a modification to the current submission page, there will be a new addition to the submission page. On the submission page, there will be a new button to notify reviewers when there is a submission which doesn’t contain a change to submission parameters like submission URL, but the deployment has been updated on the same URL. Other than that the interface remains the same for the score page, only functionalities will be tested for multi-part rubrics.

=====Quality Requirements=====
The interface changes are minimal and very easy to adapt. The new button will be clearly visible and is supposed to have a clear text to interpret its functionality. The proposed changes should not affect the current system functionality in any way. The solution is only intended to enrich the project features.

=====Performance Requirements=====
Adding test for multi-part rubric functionality should not cause any overhead in the performance of the system. Sending emails through “Notify All Reviewers” button should affect the system minimally in terms of performance. The only overhead is a database query to get all the reviewers. A good database design should incorporate such a query smoothly.

=====Assumptions=====
The functionality should work in case the assignment rubric is not multi-part.

== '''Proposed Solution''' ==
=====Zero-weighted scores=====
If only zero-weighted scores are available in the first round of review, the average of all zero-weighted scores will be shown in a gray font. An information button will explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade”)

Inside compute_total_score method in on_the_fly_calc.rb module, we'll check if the weighted score exists, if not then we'll calculate the non weighted score. We'll also have to add a method in the questionnaire model to get non weighted score. Few changes are required for the UI to display non-weighted score in grey with information icon and non weighted score in black.

Possible files to be modified.

models/on_the_fly_calc.rb 
models/questionnaire.rb 
models/view_team.html.erb

===== Non-zero weighted scores=====
If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is 80%⨉90% + 100%⨉10% = 82%

We'll first write the test cases by adding test cases to the '''spec/models/assignment_spec.rb'''

Also, we'll test this manually by computing the score and if there's a bug then below are the possible files to be modified.

spec/models/assignment_spec.rb 

=====Review Update=====
If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review.

 This feature can be broken down into the following sub-cases:
 1. A submission can only be made before deadline
 2. A review can edit a review only before deadline
 3. Score calculation should be only done for latest submitted review

Subcases:

 
1. Spec to ensure submission can't be made after the deadline

File : spec/features/post_deadline_review_submission_spec.rb

'''
it "should not allow submission after deadline" do

user = User.find_by(name: "student2065")
stub_current_user(user, user.role.name, user.role)

# goto student_task page, which has link to "Your work"
visit 'student_task/view?id=1'

# the page will have text "Your work" but will be grayed
expect(page).to have_content "Your work"

# the page will not have link to content "Your work"
expect{click_link "Your work"}.to raise_error(Capybara::ElementNotFound)

end
'''

2. Spec to ensure review can be done only before deadline

File : spec/features/post_deadline_review_submission_spec.rb

'''
it "should not be able to review work after deadline" do

# The spec is written to reproduce following bug. "Others' work" link open after deadline passed

user = User.find_by(name: "student2065")
stub_current_user(user, user.role.name, user.role)
visit 'student_task/view?id=1'

# the page should have content, but after deadline passes it is displayed as gray
# but there should not be any link attached to it
expect(page).to have_content "Others' work"

# this is the bug, even after deadline has passed, the link is still present
# the ui comment in file views/student_task/view.html.erb says
# 
# But the link seems to be open even after deadline passed.
# Screenshot attached as part of wiki for E1975, Fall 2019
expect(page).to have_link("Others' work", "/student_review/list?id=1")

end
'''

The test scenario is able to produce the following bug. At times the review link is open even after the deadline has passed. The bug is also shown/reproduced in the following screenshot for Program 1.

[[File:ReviewBug.png]]

[[File:E1975_CHECKBOX_1.jpg]]

 
[[File:Usecasenew.png]]

=='''Test Plan'''==

=====Rspec=====

New test cases will be added to test and validate the functionalities which are not working as expected. Also the test cases for features that will be newly added to the system, will also be written. Test cases will be added to these files:
spec/models/on_the_fly_calc_spec.rb 
spec/models/questionnaire_spec.rb 

The scenarios covered by the newly introduced test cases are as follows: 
1. Zero weighted scores 
2. Non-zero weighted scores 
3. Review update 

=====Manual Testing=====

'''Step 1'''. The instructor will create an assignment and assign the zero and non-zero weights to the rubrics. 
'''Step 2'''. The author submits the work done in expertiza. 
'''Step 3'''. Peers check the submitted work and give reviews for it. 

'''Test 1: Zero weighted scores shown in gray.''' 
Login as an author in expertiza and go to scores page to view the feedback.
Zero weighted scores should be displayed in gray to the logged in user.

'''Test 2: Information button.''' 
Login as an author in expertiza and go to scores page to view the feedback.
The information button should explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade.”)

'''Step 4'''. The mentor evaluates the final submission and give grades.

'''Test 3: Display non-zero weighted scores''' 
Login as an author in expertiza and go to scores page to view the feedback.
The scores shown for the assignment should be the weighted average of the scores.
The zero weighted scores have to be replaced with non-zero weighted scores.

'''Step 5''': The author updates the work and re-submits it again.

'''Test 4: Review Update''' 
The Review should be reopened. The reviewer should be able to go and update the submitted review.
If the review is not updated the previous review score has to be retained.

File:ReviewBug.png

2019-12-15T21:38:39Z

Atrived:

CSC/ECE 517 Fall 2019 - E1975. Generalize Review Versioning

2019-12-15T21:37:41Z

Atrived: /* Review Update */

__TOC__

== '''Introduction''' ==
Expertiza is a peer-review based learning platform in which work-products by students are evaluated and scored by peer-review of other students. Review process occurs in stages. After submission of initial feedback for a work-product by the peer-group, students are allowed time to incorporate the comments and improve their work. This second version of the work product is reviewed again and scores are given. This process might be repeated again and the average score from the last review stage is considered as the final score for that work-product. This incremental development of work-products and progressive learning is the fundamental concept underlying the Expertiza system.

=====Purpose=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The system also triggers an email to reviewer when there is a change in the submission. The scores are calculated on the fly based on the rubrics.

=====Scope=====
This project will enhance the existing scoring and peer review functionality and the underlying implementation along with the testing of the existing implementation. Testing of the score calculation, display for the weighted and non-weighted scores based on the combination of the rubrics are part of this project. This project also includes an additional feature for the author to notify the reviewers for the change in the submission.

=====Background=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The scores are calculated based on the rubrics defined in the system. There’s already an existing implementation for the computation of score but it doesn’t work for multipart rubric

=====Standards=====
The new standards that will be adhered to in this system are those given for the rails framework, and the code should follow rails and object oriented design principles. Any newly added code will adhere to the guidelines mentioned below:

● https://github.com/rubocop-hq/rails-style-guide 
● http://wiki.expertiza.ncsu.edu/index.php/Object-Oriented_Design_and_Programming.

== '''Problem Statement''' ==
=====Problem=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The functionality needs to be tested. It works in some cases, but we are suspicious that it is not entirely correct.

'''Profile Email Preference''':
User in expertiza has liberty to control what kind of emails they should receive. As the reviewers will receive an email for the change in submission. This can be controlled by selecting an option (When someone else submits work I am assigned to review) on the profile page of the user. As the checkboxes are far away on the right corner of the screen, the position of checkboxes need to be changed to make it more user friendly.
 

Scores for students in Expertiza can be based on a combination of rubrics: review rubrics for each round of review, author feedback, and teammate review. In CSC/ECE 517, we don’t really use the student-assigned scores to grade projects, but the peer-review score shown to the student is based entirely on the second round of review 

Here is how scores are calculated: 

1. If only zero-weighted scores are available (here, in the first round of review), the average of all zero-weighted scores should be shown in grey font. An information button should explain why the scores are shown in grey (“The reviews submitted so far do not count toward the author’s final grade.”): 

2. If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is:: 
80%⨉90% + 100%⨉10% = 82%

3. If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. '''However, the previous review score that was given in the current round will count until the reviewer updates the submitted review'''. '''This functionality doesn’t work for multipart rubrics.''' 

The system also doesn’t generate emails to reviewers in case a new submission is made by the author without changing submission parameters like submission URL. Eg: A new submission is made by making a new deployment. But the deployment URL doesn’t change. In this case as well, without changing the current flow the author should have an option to notify all the reviewers to revisit the submission.

=====Interface Requirement=====
The overall user interface would remain the same as of the current system. Since this is a modification to the current submission page, there will be a new addition to the submission page. On the submission page, there will be a new button to notify reviewers when there is a submission which doesn’t contain a change to submission parameters like submission URL, but the deployment has been updated on the same URL. Other than that the interface remains the same for the score page, only functionalities will be tested for multi-part rubrics.

=====Quality Requirements=====
The interface changes are minimal and very easy to adapt. The new button will be clearly visible and is supposed to have a clear text to interpret its functionality. The proposed changes should not affect the current system functionality in any way. The solution is only intended to enrich the project features.

=====Performance Requirements=====
Adding test for multi-part rubric functionality should not cause any overhead in the performance of the system. Sending emails through “Notify All Reviewers” button should affect the system minimally in terms of performance. The only overhead is a database query to get all the reviewers. A good database design should incorporate such a query smoothly.

=====Assumptions=====
The functionality should work in case the assignment rubric is not multi-part.

== '''Proposed Solution''' ==
=====Zero-weighted scores=====
If only zero-weighted scores are available in the first round of review, the average of all zero-weighted scores will be shown in a gray font. An information button will explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade”)

Inside compute_total_score method in on_the_fly_calc.rb module, we'll check if the weighted score exists, if not then we'll calculate the non weighted score. We'll also have to add a method in the questionnaire model to get non weighted score. Few changes are required for the UI to display non-weighted score in grey with information icon and non weighted score in black.

Possible files to be modified.

models/on_the_fly_calc.rb 
models/questionnaire.rb 
models/view_team.html.erb

===== Non-zero weighted scores=====
If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is 80%⨉90% + 100%⨉10% = 82%

We'll first write the test cases by adding test cases to the '''spec/models/assignment_spec.rb'''

Also, we'll test this manually by computing the score and if there's a bug then below are the possible files to be modified.

spec/models/assignment_spec.rb 

=====Review Update=====
If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review.

 This feature can be broken down into the following sub-cases:
 1. A submission can only be made before deadline
 2. A review can edit a review only before deadline
 3. Score calculation should be only done for latest submitted review

Subcases:

 
1. Spec to ensure submission can't be made after the deadline

File : spec/features/post_deadline_review_submission_spec.rb

'''
it "should not allow submission after deadline" do

user = User.find_by(name: "student2065")
stub_current_user(user, user.role.name, user.role)

# goto student_task page, which has link to "Your work"
visit 'student_task/view?id=1'

# the page will have text "Your work" but will be grayed
expect(page).to have_content "Your work"

# the page will not have link to content "Your work"
expect{click_link "Your work"}.to raise_error(Capybara::ElementNotFound)

end
'''

2. Spec to ensure review can be done only before deadline

File : spec/features/post_deadline_review_submission_spec.rb

'''
it "should not be able to review work after deadline" do

# The spec is written to reproduce following bug. "Others' work" link open after deadline passed

user = User.find_by(name: "student2065")
stub_current_user(user, user.role.name, user.role)
visit 'student_task/view?id=1'

# the page should have content, but after deadline passes it is displayed as gray
# but there should not be any link attached to it
expect(page).to have_content "Others' work"

# this is the bug, even after deadline has passed, the link is still present
# the ui comment in file views/student_task/view.html.erb says
# 
# But the link seems to be open even after deadline passed.
# Screenshot attached as part of wiki for E1975, Fall 2019
expect(page).to have_link("Others' work", "/student_review/list?id=1")

end
'''

The test scenario is able to produce the following bug. At times the review link is open even after the deadline has passed. The bug is also shown/reproduced in the following screenshot for Program 1.

[[File:E1975_CHECKBOX_1.jpg]]

 
[[File:Usecasenew.png]]

=='''Test Plan'''==

=====Rspec=====

New test cases will be added to test and validate the functionalities which are not working as expected. Also the test cases for features that will be newly added to the system, will also be written. Test cases will be added to these files:
spec/models/on_the_fly_calc_spec.rb 
spec/models/questionnaire_spec.rb 

The scenarios covered by the newly introduced test cases are as follows: 
1. Zero weighted scores 
2. Non-zero weighted scores 
3. Review update 

=====Manual Testing=====

'''Step 1'''. The instructor will create an assignment and assign the zero and non-zero weights to the rubrics. 
'''Step 2'''. The author submits the work done in expertiza. 
'''Step 3'''. Peers check the submitted work and give reviews for it. 

'''Test 1: Zero weighted scores shown in gray.''' 
Login as an author in expertiza and go to scores page to view the feedback.
Zero weighted scores should be displayed in gray to the logged in user.

'''Test 2: Information button.''' 
Login as an author in expertiza and go to scores page to view the feedback.
The information button should explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade.”)

'''Step 4'''. The mentor evaluates the final submission and give grades.

'''Test 3: Display non-zero weighted scores''' 
Login as an author in expertiza and go to scores page to view the feedback.
The scores shown for the assignment should be the weighted average of the scores.
The zero weighted scores have to be replaced with non-zero weighted scores.

'''Step 5''': The author updates the work and re-submits it again.

'''Test 4: Review Update''' 
The Review should be reopened. The reviewer should be able to go and update the submitted review.
If the review is not updated the previous review score has to be retained.

CSC/ECE 517 Fall 2019 - E1975. Generalize Review Versioning

2019-12-15T21:18:39Z

Atrived: /* Non-zero weighted scores */

CSC/ECE 517 Fall 2019 - E1975. Generalize Review Versioning

2019-12-15T20:33:04Z

Atrived: /* Review Update */

CSC/ECE 517 Fall 2019 - E1975. Generalize Review Versioning

2019-11-11T20:48:47Z

Atrived: /* Test Plan */

__TOC__

== '''About Expertiza''' ==

Expertiza is an open source project based on Ruby on Rails framework and the code is available on Github. Expertiza allows the instructor to create new assignments as well as edit new or existing assignments. Instructors can also create a list of topics the students can sign up for and specify deadlines for completion of various tasks. Students can form teams in Expertiza to work on various projects and assignments as well as peer review other students' submissions. Expertiza supports submission across various document types, including the URLs Wiki pages.
 
 

== '''Introduction''' ==
Expertiza is a peer-review based learning platform in which work-products by students are evaluated and scored by peer-review of other students. Review process occurs in stages. After submission of initial feedback for a work-product by the peer-group, students are allowed time to incorporate the comments and improve their work. This second version of the work product is reviewed again and scores are given. This process might be repeated again and the average score from the last review stage is considered as the final score for that work-product. This incremental development of work-products and progressive learning is the fundamental concept underlying the Expertiza system.

=====Purpose=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The system also triggers an email to reviewer when there is a change in the submission. The scores are calculated on the fly based on the rubrics.

=====Scope=====
This project will enhance the existing scoring and peer review functionality and the underlying implementation along with the testing of the existing implementation. Testing of the score calculation, display for the weighted and non-weighted scores based on the combination of the rubrics are part of this project. This project also includes an additional feature for the author to notify the reviewers for the change in the submission.

=====Background=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The scores are calculated based on the rubrics defined in the system. There’s already an existing implementation for the computation of score but it doesn’t work for multipart rubric

=====Documentation=====
The documents that will be generated for this project are the design document, the actual files that we edit or create, and a ReadMe that explains what we did and how to use the modified product.

=====Standards=====
The new standards that will be adhered to in this system are those given for the rails framework, and the code should follow rails and object oriented design principles. Any newly added code will adhere to the guidelines mentioned below:

● https://github.com/rubocop-hq/rails-style-guide 
● http://wiki.expertiza.ncsu.edu/index.php/Object-Oriented_Design_and_Programming.

== '''System Requirements''' ==
=====Problem=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The functionality needs to be tested. It works in some cases, but we are suspicious that it is not entirely correct.

'''Profile Email Preference''':
User in expertiza has liberty to control what kind of emails they should receive. As the reviewers will receive an email for the change in submission. This can be controlled by selecting an option (When someone else submits work I am assigned to review) on the profile page of the user. As the checkboxes are far away on the right corner of the screen, the position of checkboxes need to be changed to make it more user friendly.
 

Scores for students in Expertiza can be based on a combination of rubrics: review rubrics for each round of review, author feedback, and teammate review. In CSC/ECE 517, we don’t really use the student-assigned scores to grade projects, but the peer-review score shown to the student is based entirely on the second round of review 

Here is how scores are calculated: 

1. If only zero-weighted scores are available (here, in the first round of review), the average of all zero-weighted scores should be shown in grey font. An information button should explain why the scores are shown in grey (“The reviews submitted so far do not count toward the author’s final grade.”): 

2. If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is:: 
80%⨉90% + 100%⨉10% = 82%

3. If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. '''However, the previous review score that was given in the current round will count until the reviewer updates the submitted review'''. '''This functionality doesn’t work for multipart rubrics.''' 

The system also doesn’t generate emails to reviewers in case a new submission is made by the author without changing submission parameters like submission URL. Eg: A new submission is made by making a new deployment. But the deployment URL doesn’t change. In this case as well, without changing the current flow the author should have an option to notify all the reviewers to revisit the submission.

=====Interface Requirement=====
The overall user interface would remain the same as of the current system. Since this is a modification to the current submission page, there will be a new addition to the submission page. On the submission page, there will be a new button to notify reviewers when there is a submission which doesn’t contain a change to submission parameters like submission URL, but the deployment has been updated on the same URL. Other than that the interface remains the same for the score page, only functionalities will be tested for multi-part rubrics.

=====Quality Requirements=====
The interface changes are minimal and very easy to adapt. The new button will be clearly visible and is supposed to have a clear text to interpret its functionality. The proposed changes should not affect the current system functionality in any way. The solution is only intended to enrich the project features.

=====Portability Requirements=====
All of the changes for this enhancement to Expertiza should not interfere with the platform portability in any way.

=====Performance Requirements=====
Adding test for multi-part rubric functionality should not cause any overhead in the performance of the system. Sending emails through “Notify All Reviewers” button should affect the system minimally in terms of performance. The only overhead is a database query to get all the reviewers. A good database design should incorporate such a query smoothly.

=====Assumptions=====
The functionality should work in case the assignment rubric is not multi-part.

== '''Design''' ==
=====Zero-weighted scores=====
If only zero-weighted scores are available in the first round of review, the average of all zero-weighted scores will be shown in a gray font. An information button will explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade”)

===== Non-zero weighted scores=====
If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is 80%⨉90% + 100%⨉10% = 82%

=====Review Update=====
If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review. This functionality already exists in the existing system. However, In some submissions, there’s no change in the submitted URL’s so there’s no way to let the reviewers know that the author has made some changes in the project. We’ll add a notify button for the Author in the UI which will trigger an email to the reviewers. This will reopen the review, and then the reviewer can go in and update the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review

'''Sending an email when the submission parameters don’t change:'''

A new button will be added on the submission page. The button will enable the author to notify all the reviewers to revisit the submission. We could also add a custom message in the
mail body so that the authors know the specific reason as to why they received this particular mail.

=====Profile Email Preference=====
The checkboxes are moved to the left corner. As the layout of the options have used table layout. So, we'll increase the width of the the column which includes checkboxes. We'll be using the percentage instead of fixed value to give the width, as this would adjust the column size based on the screen size. 

[[File:E1975_CHECKBOX_1.jpg]]

=='''Test Plan'''==

=====Rspec=====
We have added new testcases for the new pieces of code that were added by us.

=====Test Plan for Email Preferences=====

. The email preferences checkboxes at present appear sticking to the rightmost border of the page. By following the requirements the checkboxes will be made available closer to the text fields to make them feel more user-friendly.
The changes can be tested post code changes.

=====Manual Testing=====

The instructor will create an assignment and assign the zero and non-zero weights to the rubrics.

· Login as an author and go to scores page to view the feedback.

Test – Zero weighted scores shown in gray.

Test – Zero weighted scores shown in gray.

· Zero weighted scores should be displayed in gray to the user.

Test – Information button.

· The information button should explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade.”)

Test – Display non-zero weighted scores.

· The scores shown for the assignment should be the weighted average of the scores, if any non-zero scores are available
· The zero weighted scores have to be replaced with non-zero weighted scores.

Test – Update review score.
· If author updates the submission before the end of current round. The reviewer should be able to go and update the submitted review.
· If the review is not updated the previous review score has to be retained.

Test – Multipart part rubric.
· The functionality should work for multipart rubric.
· The scores calculated have to be accurate.

CSC/ECE 517 Fall 2019 - E1975. Generalize Review Versioning

2019-11-11T20:47:35Z

Atrived: /* Manual Testing */

__TOC__

== '''About Expertiza''' ==

Expertiza is an open source project based on Ruby on Rails framework and the code is available on Github. Expertiza allows the instructor to create new assignments as well as edit new or existing assignments. Instructors can also create a list of topics the students can sign up for and specify deadlines for completion of various tasks. Students can form teams in Expertiza to work on various projects and assignments as well as peer review other students' submissions. Expertiza supports submission across various document types, including the URLs Wiki pages.
 
 

== '''Introduction''' ==
Expertiza is a peer-review based learning platform in which work-products by students are evaluated and scored by peer-review of other students. Review process occurs in stages. After submission of initial feedback for a work-product by the peer-group, students are allowed time to incorporate the comments and improve their work. This second version of the work product is reviewed again and scores are given. This process might be repeated again and the average score from the last review stage is considered as the final score for that work-product. This incremental development of work-products and progressive learning is the fundamental concept underlying the Expertiza system.

=====Purpose=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The system also triggers an email to reviewer when there is a change in the submission. The scores are calculated on the fly based on the rubrics.

=====Scope=====
This project will enhance the existing scoring and peer review functionality and the underlying implementation along with the testing of the existing implementation. Testing of the score calculation, display for the weighted and non-weighted scores based on the combination of the rubrics are part of this project. This project also includes an additional feature for the author to notify the reviewers for the change in the submission.

=====Background=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The scores are calculated based on the rubrics defined in the system. There’s already an existing implementation for the computation of score but it doesn’t work for multipart rubric

=====Documentation=====
The documents that will be generated for this project are the design document, the actual files that we edit or create, and a ReadMe that explains what we did and how to use the modified product.

=====Standards=====
The new standards that will be adhered to in this system are those given for the rails framework, and the code should follow rails and object oriented design principles. Any newly added code will adhere to the guidelines mentioned below:

● https://github.com/rubocop-hq/rails-style-guide 
● http://wiki.expertiza.ncsu.edu/index.php/Object-Oriented_Design_and_Programming.

== '''System Requirements''' ==
=====Problem=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The functionality needs to be tested. It works in some cases, but we are suspicious that it is not entirely correct.

'''Profile Email Preference''':
User in expertiza has liberty to control what kind of emails they should receive. As the reviewers will receive an email for the change in submission. This can be controlled by selecting an option (When someone else submits work I am assigned to review) on the profile page of the user. As the checkboxes are far away on the right corner of the screen, the position of checkboxes need to be changed to make it more user friendly.
 

Scores for students in Expertiza can be based on a combination of rubrics: review rubrics for each round of review, author feedback, and teammate review. In CSC/ECE 517, we don’t really use the student-assigned scores to grade projects, but the peer-review score shown to the student is based entirely on the second round of review 

Here is how scores are calculated: 

1. If only zero-weighted scores are available (here, in the first round of review), the average of all zero-weighted scores should be shown in grey font. An information button should explain why the scores are shown in grey (“The reviews submitted so far do not count toward the author’s final grade.”): 

2. If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is:: 
80%⨉90% + 100%⨉10% = 82%

3. If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. '''However, the previous review score that was given in the current round will count until the reviewer updates the submitted review'''. '''This functionality doesn’t work for multipart rubrics.''' 

The system also doesn’t generate emails to reviewers in case a new submission is made by the author without changing submission parameters like submission URL. Eg: A new submission is made by making a new deployment. But the deployment URL doesn’t change. In this case as well, without changing the current flow the author should have an option to notify all the reviewers to revisit the submission.

=====Interface Requirement=====
The overall user interface would remain the same as of the current system. Since this is a modification to the current submission page, there will be a new addition to the submission page. On the submission page, there will be a new button to notify reviewers when there is a submission which doesn’t contain a change to submission parameters like submission URL, but the deployment has been updated on the same URL. Other than that the interface remains the same for the score page, only functionalities will be tested for multi-part rubrics.

=====Quality Requirements=====
The interface changes are minimal and very easy to adapt. The new button will be clearly visible and is supposed to have a clear text to interpret its functionality. The proposed changes should not affect the current system functionality in any way. The solution is only intended to enrich the project features.

=====Portability Requirements=====
All of the changes for this enhancement to Expertiza should not interfere with the platform portability in any way.

=====Performance Requirements=====
Adding test for multi-part rubric functionality should not cause any overhead in the performance of the system. Sending emails through “Notify All Reviewers” button should affect the system minimally in terms of performance. The only overhead is a database query to get all the reviewers. A good database design should incorporate such a query smoothly.

=====Assumptions=====
The functionality should work in case the assignment rubric is not multi-part.

== '''Design''' ==
=====Zero-weighted scores=====
If only zero-weighted scores are available in the first round of review, the average of all zero-weighted scores will be shown in a gray font. An information button will explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade”)

===== Non-zero weighted scores=====
If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is 80%⨉90% + 100%⨉10% = 82%

=====Review Update=====
If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review. This functionality already exists in the existing system. However, In some submissions, there’s no change in the submitted URL’s so there’s no way to let the reviewers know that the author has made some changes in the project. We’ll add a notify button for the Author in the UI which will trigger an email to the reviewers. This will reopen the review, and then the reviewer can go in and update the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review

'''Sending an email when the submission parameters don’t change:'''

A new button will be added on the submission page. The button will enable the author to notify all the reviewers to revisit the submission. We could also add a custom message in the
mail body so that the authors know the specific reason as to why they received this particular mail.

=====Profile Email Preference=====
The checkboxes are moved to the left corner 

[[File:E1975_CHECKBOX_1.jpg]]

=='''Test Plan'''==

=====Rspec=====
We have added new testcases for the new pieces of code that were added by us.

=====Manual Testing=====

The instructor will create an assignment and assign the zero and non-zero weights to the rubrics.

· Login as an author and go to scores page to view the feedback.

Test – Zero weighted scores shown in gray.

Test – Zero weighted scores shown in gray.

· Zero weighted scores should be displayed in gray to the user.

Test – Information button.

· The information button should explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade.”)

Test – Display non-zero weighted scores.

· The scores shown for the assignment should be the weighted average of the scores, if any non-zero scores are available
· The zero weighted scores have to be replaced with non-zero weighted scores.

Test – Update review score.
· If author updates the submission before the end of current round. The reviewer should be able to go and update the submitted review.
· If the review is not updated the previous review score has to be retained.

Test – Multipart part rubric.
· The functionality should work for multipart rubric.
· The scores calculated have to be accurate.

CSC/ECE 517 Fall 2019 - E1975. Generalize Review Versioning

2019-11-11T20:47:10Z

Atrived: /* Manual Testing */

__TOC__

== '''About Expertiza''' ==

Expertiza is an open source project based on Ruby on Rails framework and the code is available on Github. Expertiza allows the instructor to create new assignments as well as edit new or existing assignments. Instructors can also create a list of topics the students can sign up for and specify deadlines for completion of various tasks. Students can form teams in Expertiza to work on various projects and assignments as well as peer review other students' submissions. Expertiza supports submission across various document types, including the URLs Wiki pages.
 
 

== '''Introduction''' ==
Expertiza is a peer-review based learning platform in which work-products by students are evaluated and scored by peer-review of other students. Review process occurs in stages. After submission of initial feedback for a work-product by the peer-group, students are allowed time to incorporate the comments and improve their work. This second version of the work product is reviewed again and scores are given. This process might be repeated again and the average score from the last review stage is considered as the final score for that work-product. This incremental development of work-products and progressive learning is the fundamental concept underlying the Expertiza system.

=====Purpose=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The system also triggers an email to reviewer when there is a change in the submission. The scores are calculated on the fly based on the rubrics.

=====Scope=====
This project will enhance the existing scoring and peer review functionality and the underlying implementation along with the testing of the existing implementation. Testing of the score calculation, display for the weighted and non-weighted scores based on the combination of the rubrics are part of this project. This project also includes an additional feature for the author to notify the reviewers for the change in the submission.

=====Background=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The scores are calculated based on the rubrics defined in the system. There’s already an existing implementation for the computation of score but it doesn’t work for multipart rubric

=====Documentation=====
The documents that will be generated for this project are the design document, the actual files that we edit or create, and a ReadMe that explains what we did and how to use the modified product.

=====Standards=====
The new standards that will be adhered to in this system are those given for the rails framework, and the code should follow rails and object oriented design principles. Any newly added code will adhere to the guidelines mentioned below:

● https://github.com/rubocop-hq/rails-style-guide 
● http://wiki.expertiza.ncsu.edu/index.php/Object-Oriented_Design_and_Programming.

== '''System Requirements''' ==
=====Problem=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The functionality needs to be tested. It works in some cases, but we are suspicious that it is not entirely correct.

'''Profile Email Preference''':
User in expertiza has liberty to control what kind of emails they should receive. As the reviewers will receive an email for the change in submission. This can be controlled by selecting an option (When someone else submits work I am assigned to review) on the profile page of the user. As the checkboxes are far away on the right corner of the screen, the position of checkboxes need to be changed to make it more user friendly.
 

Scores for students in Expertiza can be based on a combination of rubrics: review rubrics for each round of review, author feedback, and teammate review. In CSC/ECE 517, we don’t really use the student-assigned scores to grade projects, but the peer-review score shown to the student is based entirely on the second round of review 

Here is how scores are calculated: 

1. If only zero-weighted scores are available (here, in the first round of review), the average of all zero-weighted scores should be shown in grey font. An information button should explain why the scores are shown in grey (“The reviews submitted so far do not count toward the author’s final grade.”): 

2. If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is:: 
80%⨉90% + 100%⨉10% = 82%

3. If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. '''However, the previous review score that was given in the current round will count until the reviewer updates the submitted review'''. '''This functionality doesn’t work for multipart rubrics.''' 

The system also doesn’t generate emails to reviewers in case a new submission is made by the author without changing submission parameters like submission URL. Eg: A new submission is made by making a new deployment. But the deployment URL doesn’t change. In this case as well, without changing the current flow the author should have an option to notify all the reviewers to revisit the submission.

=====Interface Requirement=====
The overall user interface would remain the same as of the current system. Since this is a modification to the current submission page, there will be a new addition to the submission page. On the submission page, there will be a new button to notify reviewers when there is a submission which doesn’t contain a change to submission parameters like submission URL, but the deployment has been updated on the same URL. Other than that the interface remains the same for the score page, only functionalities will be tested for multi-part rubrics.

=====Quality Requirements=====
The interface changes are minimal and very easy to adapt. The new button will be clearly visible and is supposed to have a clear text to interpret its functionality. The proposed changes should not affect the current system functionality in any way. The solution is only intended to enrich the project features.

=====Portability Requirements=====
All of the changes for this enhancement to Expertiza should not interfere with the platform portability in any way.

=====Performance Requirements=====
Adding test for multi-part rubric functionality should not cause any overhead in the performance of the system. Sending emails through “Notify All Reviewers” button should affect the system minimally in terms of performance. The only overhead is a database query to get all the reviewers. A good database design should incorporate such a query smoothly.

=====Assumptions=====
The functionality should work in case the assignment rubric is not multi-part.

== '''Design''' ==
=====Zero-weighted scores=====
If only zero-weighted scores are available in the first round of review, the average of all zero-weighted scores will be shown in a gray font. An information button will explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade”)

===== Non-zero weighted scores=====
If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is 80%⨉90% + 100%⨉10% = 82%

=====Review Update=====
If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review. This functionality already exists in the existing system. However, In some submissions, there’s no change in the submitted URL’s so there’s no way to let the reviewers know that the author has made some changes in the project. We’ll add a notify button for the Author in the UI which will trigger an email to the reviewers. This will reopen the review, and then the reviewer can go in and update the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review

'''Sending an email when the submission parameters don’t change:'''

A new button will be added on the submission page. The button will enable the author to notify all the reviewers to revisit the submission. We could also add a custom message in the
mail body so that the authors know the specific reason as to why they received this particular mail.

=====Profile Email Preference=====
The checkboxes are moved to the left corner 

[[File:E1975_CHECKBOX_1.jpg]]

=='''Test Plan'''==

=====Rspec=====
We have added new testcases for the new pieces of code that were added by us.

=====Manual Testing=====

Test Plan for Email Preferences

. The email preferences checkboxes at present appear sticking to the rightmost border of the page. By following the requirements the checkboxes will be made available closer to the text fields to make them feel more user-friendly.
The changes can be tested post code changes.

The instructor will create an assignment and assign the zero and non-zero weights to the rubrics.

· Login as an author and go to scores page to view the feedback.

Test – Zero weighted scores shown in gray.

Test – Zero weighted scores shown in gray.

· Zero weighted scores should be displayed in gray to the user.

Test – Information button.

· The information button should explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade.”)

Test – Display non-zero weighted scores.

· The scores shown for the assignment should be the weighted average of the scores, if any non-zero scores are available
· The zero weighted scores have to be replaced with non-zero weighted scores.

Test – Update review score.
· If author updates the submission before the end of current round. The reviewer should be able to go and update the submitted review.
· If the review is not updated the previous review score has to be retained.

Test – Multipart part rubric.
· The functionality should work for multipart rubric.
· The scores calculated have to be accurate.

CSC/ECE 517 Fall 2019 - E1940. Improving email notification

2019-11-07T03:48:25Z

Atrived: /* Issue 3 */

=E1940 Improving e-mail notification=

===Brief Introduction===

* E1940 Improving e-mail notification.

* The forked git repository for this project can be found [https://github.com/dheerajmakam/expertiza.git]

===Problem Statement===
The following tasks were accomplished in this project:

*Issue1: Send new account welcome email to user, when imported from CSV through assignment page.
*Issue2: Don't send email to reviewers for a new submission after review deadline has passed.
*Issue3: Adding relevant links to reminder emails.

====Issue 1 (717) - New user email====

'''app/models/assignment_participant.rb'''

* Call method to send mail after user imported successfully.
<pre>
password = user.reset_password
MailerHelper.send_mail_to_user(user, "Your Expertiza account has been created.", "user_welcome", password).deliver
</pre>

====Issue2 (345)- No submission email to reviewer after deadline====
*'''Before fixing this issue, we had to write the logic to send emails to reviewers on submissions.'''

 
'''app/controllers/submitted_content_controller.rb'''
*Added the logic to check for last review date to the function '''submit_hyperlink'''

<pre>
# get current date and time
cur_date = Time.now
# get the last due date for the review
max_due_date = DueDate.where(parent_id: @participant.assignment.id,deadline_type_id: 2).maximum("due_at")

# send mail only if the last due date has not passed
if cur_date <= max_due_date
email_submission_reviewers(@participant)
end
</pre>

*Function to identify the reviewers and send mails.

<pre>
def email_submission_reviewers(participant)
if participant.reviewers.length != 0
participant.reviewers.each do |reviewer|
rev_res_map = ReviewResponseMap.where(['reviewer_id = ? and reviewee_id = ?', reviewer.id, participant.team.id]).first
all_responses = Response.where(:map_id => rev_res_map.id).order("updated_at DESC").first

user = User.find(reviewer.user_id)

if user.email_on_submission?
MailerHelper.submission_mail_to_reviewr(user,
"New submission available to review.",
"update").deliver
end
end
end
end
</pre>

'''app/helpers/mailer_helper.rb'''

*Helper function to mail reviewers

<pre>
def self.submission_mail_to_reviewr(user, subject, mail_partial)
Mailer.notify_reviewer_for_new_submission ({
to: user.email,
subject: subject,
body: {
user: user,
#first_name: ApplicationHelper.get_user_first_name(user),
message: "",
partial_name: mail_partial
}
})
end
</pre>

'''app/mailers/mailer.rb'''

*Mailer function to send the mail.

<pre>
def notify_reviewer_for_new_submission(defn)
@partial_name = defn[:body][:partial_name]
@user = defn[:body][:user]
#@first_name = defn[:body][:first_name]
@message = defn[:body][:message]
mail(subject: defn[:subject],
to: defn[:to])
end
</pre>

'''app/views/mailer/notify_reviewer_for_new_submission.erb'''

*Email template for the mail

<pre>
<!DOCTYPE html>
<html>
<head>
<meta content='text/html; charset=UTF-8' http-equiv='Content-Type' />
</head>
<body>
<%= render :partial => 'mailer/partials/'+@partial_name+'_html' %>
<hr>
</body>
</html>
</pre>

====Issue3 (111)- Adding relevant links to reminder emails.====

*Modified function email_remainder to add functionality.

'''app/mailers/mail_worker.rb'''

<pre>
def email_reminder(emails, deadline_type)
assignment = Assignment.find(self.assignment_id)
subject = "Message regarding #{deadline_type} for assignment #{assignment.name}"
body = "This is a reminder to complete #{deadline_type} for assignment #{assignment.name}. \
Deadline is #{self.due_at}.If you have already done the #{deadline_type}, Please ignore this mail."
body = "This is a reminder to complete #{deadline_type} for assignment #{assignment.name}."

emails.each do |mail|
user = User.where({email: mail}).first
participant_assignment_id = Participant.where(user_id: user.id, parent_id: self.assignment_id).id

link_to_destination = "Please follow the lilnk: http://expertiza.ncsu.edu/student_task/view?id=#{participant_assignment_id}"
body = body + link_to_destination + " Deadline is #{self.due_at}.If you have already done the #{deadline_type}, Please ignore this mail.";

@mail = Mailer.delayed_message(bcc: mail, subject: subject, body: body)
@mail.deliver_now
Rails.logger.info mail
end

@mail = Mailer.delayed_message(bcc: emails, subject: subject, body: body)
@mail.deliver_now

end
</pre>

===Test Plan===

====Issue 1====

Step 1: Navigate to Manage --> Assignment page.

Step 2: Click on add participants for any of the assignments.

Step 3: Click "Import course participants"

Step 4: Choose a csv file to be imported (follow the format given on the website).

Step 5: The users mentioned in the csv file and don't exist on Expertiza should get a new user email.

Step 6: To check e-mail is received or not, log in with following credentials: username [ 'expertiza.development@gmail.com' ] password [ 'qwer@1234' ].

====Issue 2====

Step 1: Create new assignment [Manage --> Assignment --> + Button (to create new assignment)]

Step 2: Fill the details for the assignments.

Step 3: Navigate to due dates.

Step 4: Change the number of review rounds to 2.

Step 5: Select "Yes" in the dropdown for review allowed during submission and select "Yes" for submission during the review.

Step 6: Add two users to the assignment(author and reviewer).

Step 7: Log in with some user credentials (author credential).

Step 8: Make a new submission to this assignment.

Step 9: Log in with another user (reviewer).

Step 10: Submit a review of the assignment submission.

Step 11: Login as an author again.

Step 12: Edit the submission.

Step 13: After this check the mailbox of the reviewer [development mail for development].

Step 14: Reviewer should get the mail to re-review the work.

Step 15: Change the due date to some date and time which has passed.

Step 16: Now making a new submission from the author account should not send a re-review mail to the reviewer. [Repeat steps 7-15].

====Issue 3====

This test requires an approaching deadline scenario. Since the reminder mail goes through as a sidekiq background, the part of this issue was to fix the email link.

Create a new scenario with an approaching deadline. The email is sent already, we have added the link which was missing, directing users to visit the required page.

===Team Member Details===

Team members :
* Adarsh Trivedi (atrived)
* Dheeraj Makam (drmakam)
* Prachi Sheoran (psheora)

CSC/ECE 517 Fall 2019 - E1940. Improving email notification

2019-11-07T03:46:20Z

Atrived: /* Issue 2 */

=E1940 Improving e-mail notification=

===Brief Introduction===

* E1940 Improving e-mail notification.

* The forked git repository for this project can be found [https://github.com/dheerajmakam/expertiza.git]

===Problem Statement===
The following tasks were accomplished in this project:

*Issue1: Send new account welcome email to user, when imported from CSV through assignment page.
*Issue2: Don't send email to reviewers for a new submission after review deadline has passed.
*Issue3: Adding relevant links to reminder emails.

====Issue 1 (717) - New user email====

'''app/models/assignment_participant.rb'''

* Call method to send mail after user imported successfully.
<pre>
password = user.reset_password
MailerHelper.send_mail_to_user(user, "Your Expertiza account has been created.", "user_welcome", password).deliver
</pre>

====Issue2 (345)- No submission email to reviewer after deadline====
*'''Before fixing this issue, we had to write the logic to send emails to reviewers on submissions.'''

 
'''app/controllers/submitted_content_controller.rb'''
*Added the logic to check for last review date to the function '''submit_hyperlink'''

<pre>
# get current date and time
cur_date = Time.now
# get the last due date for the review
max_due_date = DueDate.where(parent_id: @participant.assignment.id,deadline_type_id: 2).maximum("due_at")

# send mail only if the last due date has not passed
if cur_date <= max_due_date
email_submission_reviewers(@participant)
end
</pre>

*Function to identify the reviewers and send mails.

<pre>
def email_submission_reviewers(participant)
if participant.reviewers.length != 0
participant.reviewers.each do |reviewer|
rev_res_map = ReviewResponseMap.where(['reviewer_id = ? and reviewee_id = ?', reviewer.id, participant.team.id]).first
all_responses = Response.where(:map_id => rev_res_map.id).order("updated_at DESC").first

user = User.find(reviewer.user_id)

if user.email_on_submission?
MailerHelper.submission_mail_to_reviewr(user,
"New submission available to review.",
"update").deliver
end
end
end
end
</pre>

'''app/helpers/mailer_helper.rb'''

*Helper function to mail reviewers

<pre>
def self.submission_mail_to_reviewr(user, subject, mail_partial)
Mailer.notify_reviewer_for_new_submission ({
to: user.email,
subject: subject,
body: {
user: user,
#first_name: ApplicationHelper.get_user_first_name(user),
message: "",
partial_name: mail_partial
}
})
end
</pre>

'''app/mailers/mailer.rb'''

*Mailer function to send the mail.

<pre>
def notify_reviewer_for_new_submission(defn)
@partial_name = defn[:body][:partial_name]
@user = defn[:body][:user]
#@first_name = defn[:body][:first_name]
@message = defn[:body][:message]
mail(subject: defn[:subject],
to: defn[:to])
end
</pre>

'''app/views/mailer/notify_reviewer_for_new_submission.erb'''

*Email template for the mail

<pre>
<!DOCTYPE html>
<html>
<head>
<meta content='text/html; charset=UTF-8' http-equiv='Content-Type' />
</head>
<body>
<%= render :partial => 'mailer/partials/'+@partial_name+'_html' %>
<hr>
</body>
</html>
</pre>

====Issue3 (111)- Adding relevant links to reminder emails.====

*Modified function email_remainder to add functionality.

'''app/mailers/mail_worker.rb'''

<pre>
def email_reminder(emails, deadline_type)
assignment = Assignment.find(self.assignment_id)
subject = "Message regarding #{deadline_type} for assignment #{assignment.name}"
body = "This is a reminder to complete #{deadline_type} for assignment #{assignment.name}. \
Deadline is #{self.due_at}.If you have already done the #{deadline_type}, Please ignore this mail."
body = "This is a reminder to complete #{deadline_type} for assignment #{assignment.name}."

emails.each do |mail|
user = User.where({email: mail}).first
participant_assignment_id = Participant.where(user_id: user.id, parent_id: self.assignment_id).id

link_to_destination = "Please follow the lilnk: http://expertiza.ncsu.edu/student_task/view?id=#{participant_assignment_id}"
body = body + link_to_destination + " Deadline is #{self.due_at}.If you have already done the #{deadline_type}, Please ignore this mail.";

@mail = Mailer.delayed_message(bcc: mail, subject: subject, body: body)
@mail.deliver_now
Rails.logger.info mail
end

@mail = Mailer.delayed_message(bcc: emails, subject: subject, body: body)
@mail.deliver_now

end
</pre>

===Test Plan===

====Issue 1====

Step 1: Navigate to Manage --> Assignment page.

Step 2: Click on add participants for any of the assignments.

Step 3: Click "Import course participants"

Step 4: Choose a csv file to be imported (follow the format given on the website).

Step 5: The users mentioned in the csv file and don't exist on Expertiza should get a new user email.

Step 6: To check e-mail is received or not, log in with following credentials: username [ 'expertiza.development@gmail.com' ] password [ 'qwer@1234' ].

====Issue 2====

Step 1: Create new assignment [Manage --> Assignment --> + Button (to create new assignment)]

Step 2: Fill the details for the assignments.

Step 3: Navigate to due dates.

Step 4: Change the number of review rounds to 2.

Step 5: Select "Yes" in the dropdown for review allowed during submission and select "Yes" for submission during the review.

Step 6: Add two users to the assignment(author and reviewer).

Step 7: Log in with some user credentials (author credential).

Step 8: Make a new submission to this assignment.

Step 9: Log in with another user (reviewer).

Step 10: Submit a review of the assignment submission.

Step 11: Login as an author again.

Step 12: Edit the submission.

Step 13: After this check the mailbox of the reviewer [development mail for development].

Step 14: Reviewer should get the mail to re-review the work.

Step 15: Change the due date to some date and time which has passed.

Step 16: Now making a new submission from the author account should not send a re-review mail to the reviewer. [Repeat steps 7-15].

====Issue 3====

This test requires an approaching deadline scenario. Since the reminder mail goes through as a sidekiq background, the part of this issue was to fix the email link.

Create a new scenario with an approaching deadline. The email is sent already, just the link is fixed.

===Team Member Details===

Team members :
* Adarsh Trivedi (atrived)
* Dheeraj Makam (drmakam)
* Prachi Sheoran (psheora)

CSC/ECE 517 Fall 2019 - E1940. Improving email notification

2019-11-07T03:40:22Z

Atrived: /* Issue 2 */

=E1940 Improving e-mail notification=

===Brief Introduction===

* E1940 Improving e-mail notification.

* The forked git repository for this project can be found [https://github.com/dheerajmakam/expertiza.git]

===Problem Statement===
The following tasks were accomplished in this project:

*Issue1: Send new account welcome email to user, when imported from CSV through assignment page.
*Issue2: Don't send email to reviewers for a new submission after review deadline has passed.
*Issue3: Adding relevant links to reminder emails.

====Issue 1 (717) - New user email====

'''app/models/assignment_participant.rb'''

* Call method to send mail after user imported successfully.
<pre>
password = user.reset_password
MailerHelper.send_mail_to_user(user, "Your Expertiza account has been created.", "user_welcome", password).deliver
</pre>

====Issue2 (345)- No submission email to reviewer after deadline====
*'''Before fixing this issue, we had to write the logic to send emails to reviewers on submissions.'''

 
'''app/controllers/submitted_content_controller.rb'''
*Added the logic to check for last review date to the function '''submit_hyperlink'''

<pre>
# get current date and time
cur_date = Time.now
# get the last due date for the review
max_due_date = DueDate.where(parent_id: @participant.assignment.id,deadline_type_id: 2).maximum("due_at")

# send mail only if the last due date has not passed
if cur_date <= max_due_date
email_submission_reviewers(@participant)
end
</pre>

*Function to identify the reviewers and send mails.

<pre>
def email_submission_reviewers(participant)
if participant.reviewers.length != 0
participant.reviewers.each do |reviewer|
rev_res_map = ReviewResponseMap.where(['reviewer_id = ? and reviewee_id = ?', reviewer.id, participant.team.id]).first
all_responses = Response.where(:map_id => rev_res_map.id).order("updated_at DESC").first

user = User.find(reviewer.user_id)

if user.email_on_submission?
MailerHelper.submission_mail_to_reviewr(user,
"New submission available to review.",
"update").deliver
end
end
end
end
</pre>

'''app/helpers/mailer_helper.rb'''

*Helper function to mail reviewers

<pre>
def self.submission_mail_to_reviewr(user, subject, mail_partial)
Mailer.notify_reviewer_for_new_submission ({
to: user.email,
subject: subject,
body: {
user: user,
#first_name: ApplicationHelper.get_user_first_name(user),
message: "",
partial_name: mail_partial
}
})
end
</pre>

'''app/mailers/mailer.rb'''

*Mailer function to send the mail.

<pre>
def notify_reviewer_for_new_submission(defn)
@partial_name = defn[:body][:partial_name]
@user = defn[:body][:user]
#@first_name = defn[:body][:first_name]
@message = defn[:body][:message]
mail(subject: defn[:subject],
to: defn[:to])
end
</pre>

'''app/views/mailer/notify_reviewer_for_new_submission.erb'''

*Email template for the mail

<pre>
<!DOCTYPE html>
<html>
<head>
<meta content='text/html; charset=UTF-8' http-equiv='Content-Type' />
</head>
<body>
<%= render :partial => 'mailer/partials/'+@partial_name+'_html' %>
<hr>
</body>
</html>
</pre>

====Issue3 (111)- Adding relevant links to reminder emails.====

*Modified function email_remainder to add functionality.

'''app/mailers/mail_worker.rb'''

<pre>
def email_reminder(emails, deadline_type)
assignment = Assignment.find(self.assignment_id)
subject = "Message regarding #{deadline_type} for assignment #{assignment.name}"
body = "This is a reminder to complete #{deadline_type} for assignment #{assignment.name}. \
Deadline is #{self.due_at}.If you have already done the #{deadline_type}, Please ignore this mail."
body = "This is a reminder to complete #{deadline_type} for assignment #{assignment.name}."

emails.each do |mail|
user = User.where({email: mail}).first
participant_assignment_id = Participant.where(user_id: user.id, parent_id: self.assignment_id).id

link_to_destination = "Please follow the lilnk: http://expertiza.ncsu.edu/student_task/view?id=#{participant_assignment_id}"
body = body + link_to_destination + " Deadline is #{self.due_at}.If you have already done the #{deadline_type}, Please ignore this mail.";

@mail = Mailer.delayed_message(bcc: mail, subject: subject, body: body)
@mail.deliver_now
Rails.logger.info mail
end

@mail = Mailer.delayed_message(bcc: emails, subject: subject, body: body)
@mail.deliver_now

end
</pre>

===Test Plan===

====Issue 1====

Step 1: Navigate to Manage --> Assignment page.

Step 2: Click on add participants for any of the assignments.

Step 3: Click "Import course participants"

Step 4: Choose a csv file to be imported (follow the format given on the website).

Step 5: The users mentioned in the csv file and don't exist on Expertiza should get a new user email.

Step 6: To check e-mail is received or not, log in with following credentials: username [ 'expertiza.development@gmail.com' ] password [ 'qwer@1234' ].

====Issue 2====

Step 1: Create new assignment [Manage --> Assignment --> + Button (to create new assignment)]

Step 2: Fill the details for the assignments.

Step 3: Navigate to due dates.

Step 4: Change the number of review rounds to 2.

Step 5: Select "Yes" in the dropdown for review allowed for submission.

Step 6: Add the user to the assignment.

Step 7: Log in with some user credentials (author credential).

Step 8: Make a new submission to this assignment.

Step 9: Log in with another user (reviewer).

Step 10: Submit a review to the assignment submission.

Step 11: Login as an author again.

Step 12: Edit the submission.

Step 13: After this check the mailbox of the reviewer [development mail for development].

Step 14: Reviewer should get the mail to re-review the work.

Step 15: Change the due date to some date and time which has passed.

Step 16: Now making a new submission from the author account should not send a re-review mail to the reviewer. [Repeat steps 7-15].

====Issue 3====

This test requires an approaching deadline scenario. Since the reminder mail goes through as a sidekiq background, the part of this issue was to fix the email link.

Create a new scenario with an approaching deadline. The email is sent already, just the link is fixed.

===Team Member Details===

Team members :
* Adarsh Trivedi (atrived)
* Dheeraj Makam (drmakam)
* Prachi Sheoran (psheora)

CSC/ECE 517 Fall 2019 - E1940. Improving email notification

2019-11-07T03:38:27Z

Atrived: /* Issue 1 */

=E1940 Improving e-mail notification=

===Brief Introduction===

* E1940 Improving e-mail notification.

* The forked git repository for this project can be found [https://github.com/dheerajmakam/expertiza.git]

===Problem Statement===
The following tasks were accomplished in this project:

*Issue1: Send new account welcome email to user, when imported from CSV through assignment page.
*Issue2: Don't send email to reviewers for a new submission after review deadline has passed.
*Issue3: Adding relevant links to reminder emails.

====Issue 1 (717) - New user email====

'''app/models/assignment_participant.rb'''

* Call method to send mail after user imported successfully.
<pre>
password = user.reset_password
MailerHelper.send_mail_to_user(user, "Your Expertiza account has been created.", "user_welcome", password).deliver
</pre>

====Issue2 (345)- No submission email to reviewer after deadline====
*'''Before fixing this issue, we had to write the logic to send emails to reviewers on submissions.'''

 
'''app/controllers/submitted_content_controller.rb'''
*Added the logic to check for last review date to the function '''submit_hyperlink'''

<pre>
# get current date and time
cur_date = Time.now
# get the last due date for the review
max_due_date = DueDate.where(parent_id: @participant.assignment.id,deadline_type_id: 2).maximum("due_at")

# send mail only if the last due date has not passed
if cur_date <= max_due_date
email_submission_reviewers(@participant)
end
</pre>

*Function to identify the reviewers and send mails.

<pre>
def email_submission_reviewers(participant)
if participant.reviewers.length != 0
participant.reviewers.each do |reviewer|
rev_res_map = ReviewResponseMap.where(['reviewer_id = ? and reviewee_id = ?', reviewer.id, participant.team.id]).first
all_responses = Response.where(:map_id => rev_res_map.id).order("updated_at DESC").first

user = User.find(reviewer.user_id)

if user.email_on_submission?
MailerHelper.submission_mail_to_reviewr(user,
"New submission available to review.",
"update").deliver
end
end
end
end
</pre>

'''app/helpers/mailer_helper.rb'''

*Helper function to mail reviewers

<pre>
def self.submission_mail_to_reviewr(user, subject, mail_partial)
Mailer.notify_reviewer_for_new_submission ({
to: user.email,
subject: subject,
body: {
user: user,
#first_name: ApplicationHelper.get_user_first_name(user),
message: "",
partial_name: mail_partial
}
})
end
</pre>

'''app/mailers/mailer.rb'''

*Mailer function to send the mail.

<pre>
def notify_reviewer_for_new_submission(defn)
@partial_name = defn[:body][:partial_name]
@user = defn[:body][:user]
#@first_name = defn[:body][:first_name]
@message = defn[:body][:message]
mail(subject: defn[:subject],
to: defn[:to])
end
</pre>

'''app/views/mailer/notify_reviewer_for_new_submission.erb'''

*Email template for the mail

<pre>
<!DOCTYPE html>
<html>
<head>
<meta content='text/html; charset=UTF-8' http-equiv='Content-Type' />
</head>
<body>
<%= render :partial => 'mailer/partials/'+@partial_name+'_html' %>
<hr>
</body>
</html>
</pre>

====Issue3 (111)- Adding relevant links to reminder emails.====

*Modified function email_remainder to add functionality.

'''app/mailers/mail_worker.rb'''

<pre>
def email_reminder(emails, deadline_type)
assignment = Assignment.find(self.assignment_id)
subject = "Message regarding #{deadline_type} for assignment #{assignment.name}"
body = "This is a reminder to complete #{deadline_type} for assignment #{assignment.name}. \
Deadline is #{self.due_at}.If you have already done the #{deadline_type}, Please ignore this mail."
body = "This is a reminder to complete #{deadline_type} for assignment #{assignment.name}."

emails.each do |mail|
user = User.where({email: mail}).first
participant_assignment_id = Participant.where(user_id: user.id, parent_id: self.assignment_id).id

link_to_destination = "Please follow the lilnk: http://expertiza.ncsu.edu/student_task/view?id=#{participant_assignment_id}"
body = body + link_to_destination + " Deadline is #{self.due_at}.If you have already done the #{deadline_type}, Please ignore this mail.";

@mail = Mailer.delayed_message(bcc: mail, subject: subject, body: body)
@mail.deliver_now
Rails.logger.info mail
end

@mail = Mailer.delayed_message(bcc: emails, subject: subject, body: body)
@mail.deliver_now

end
</pre>

===Test Plan===

====Issue 1====

Step 1: Navigate to Manage --> Assignment page.

Step 2: Click on add participants for any of the assignments.

Step 3: Click "Import course participants"

Step 4: Choose a csv file to be imported (follow the format given on the website).

Step 5: The users mentioned in the csv file and don't exist on Expertiza should get a new user email.

Step 6: To check e-mail is received or not, log in with following credentials: username [ 'expertiza.development@gmail.com' ] password [ 'qwer@1234' ].

====Issue 2====

Step 1: Create new assignment [Manage/Assignment/+ Button (to create new assignment)]

Step 2: Fill the details for the assignments.

Step 3: Navigate to due dates.

Step 4: Change the number of review rounds to 2.

Step 5: Select "Yes" in the dropdown for review allowed for submission.

Step 6: Add the user to the assignment.

Step 7: Log in with some user credentials (author credential).

Step 8: Make a new submission to this assignment.

Step 9: Log in with another user (reviewer).

Step 10: Submit a review to the assignment submission.

Step 11: Login as an author again.

Step 12: Edit the submission.

Step 13: After this check the mailbox of the reviewer [development mail for development].

Step 14: Reviewer should get the mail to re-review the work.

Step 15: Change the due date to some date and time which has passed.

Step 16: Now making a new submission from the author account should not send a re-review mail to the reviewer. [Repeat steps 7-15].

====Issue 3====

This test requires an approaching deadline scenario. Since the reminder mail goes through as a sidekiq background, the part of this issue was to fix the email link.

Create a new scenario with an approaching deadline. The email is sent already, just the link is fixed.

===Team Member Details===

Team members :
* Adarsh Trivedi (atrived)
* Dheeraj Makam (drmakam)
* Prachi Sheoran (psheora)

CSC/ECE 517 Fall 2019 - E1940. Improving email notification

2019-11-07T03:33:58Z

Atrived:

=E1940 Improving e-mail notification=

===Brief Introduction===

* E1940 Improving e-mail notification.

* The forked git repository for this project can be found [https://github.com/dheerajmakam/expertiza.git]

===Problem Statement===
The following tasks were accomplished in this project:

*Issue1: Send new account welcome email to user, when imported from CSV through assignment page.
*Issue2: Don't send email to reviewers for a new submission after review deadline has passed.
*Issue3: Adding relevant links to reminder emails.

====Issue 1 (717) - New user email====

'''app/models/assignment_participant.rb'''

* Call method to send mail after user imported successfully.
<pre>
password = user.reset_password
MailerHelper.send_mail_to_user(user, "Your Expertiza account has been created.", "user_welcome", password).deliver
</pre>

====Issue2 (345)- No submission email to reviewer after deadline====
*'''Before fixing this issue, we had to write the logic to send emails to reviewers on submissions.'''

 
'''app/controllers/submitted_content_controller.rb'''
*Added the logic to check for last review date to the function '''submit_hyperlink'''

<pre>
# get current date and time
cur_date = Time.now
# get the last due date for the review
max_due_date = DueDate.where(parent_id: @participant.assignment.id,deadline_type_id: 2).maximum("due_at")

# send mail only if the last due date has not passed
if cur_date <= max_due_date
email_submission_reviewers(@participant)
end
</pre>

*Function to identify the reviewers and send mails.

<pre>
def email_submission_reviewers(participant)
if participant.reviewers.length != 0
participant.reviewers.each do |reviewer|
rev_res_map = ReviewResponseMap.where(['reviewer_id = ? and reviewee_id = ?', reviewer.id, participant.team.id]).first
all_responses = Response.where(:map_id => rev_res_map.id).order("updated_at DESC").first

user = User.find(reviewer.user_id)

if user.email_on_submission?
MailerHelper.submission_mail_to_reviewr(user,
"New submission available to review.",
"update").deliver
end
end
end
end
</pre>

'''app/helpers/mailer_helper.rb'''

*Helper function to mail reviewers

<pre>
def self.submission_mail_to_reviewr(user, subject, mail_partial)
Mailer.notify_reviewer_for_new_submission ({
to: user.email,
subject: subject,
body: {
user: user,
#first_name: ApplicationHelper.get_user_first_name(user),
message: "",
partial_name: mail_partial
}
})
end
</pre>

'''app/mailers/mailer.rb'''

*Mailer function to send the mail.

<pre>
def notify_reviewer_for_new_submission(defn)
@partial_name = defn[:body][:partial_name]
@user = defn[:body][:user]
#@first_name = defn[:body][:first_name]
@message = defn[:body][:message]
mail(subject: defn[:subject],
to: defn[:to])
end
</pre>

'''app/views/mailer/notify_reviewer_for_new_submission.erb'''

*Email template for the mail

<pre>
<!DOCTYPE html>
<html>
<head>
<meta content='text/html; charset=UTF-8' http-equiv='Content-Type' />
</head>
<body>
<%= render :partial => 'mailer/partials/'+@partial_name+'_html' %>
<hr>
</body>
</html>
</pre>

====Issue3 (111)- Adding relevant links to reminder emails.====

*Modified function email_remainder to add functionality.

'''app/mailers/mail_worker.rb'''

<pre>
def email_reminder(emails, deadline_type)
assignment = Assignment.find(self.assignment_id)
subject = "Message regarding #{deadline_type} for assignment #{assignment.name}"
body = "This is a reminder to complete #{deadline_type} for assignment #{assignment.name}. \
Deadline is #{self.due_at}.If you have already done the #{deadline_type}, Please ignore this mail."
body = "This is a reminder to complete #{deadline_type} for assignment #{assignment.name}."

emails.each do |mail|
user = User.where({email: mail}).first
participant_assignment_id = Participant.where(user_id: user.id, parent_id: self.assignment_id).id

link_to_destination = "Please follow the lilnk: http://expertiza.ncsu.edu/student_task/view?id=#{participant_assignment_id}"
body = body + link_to_destination + " Deadline is #{self.due_at}.If you have already done the #{deadline_type}, Please ignore this mail.";

@mail = Mailer.delayed_message(bcc: mail, subject: subject, body: body)
@mail.deliver_now
Rails.logger.info mail
end

@mail = Mailer.delayed_message(bcc: emails, subject: subject, body: body)
@mail.deliver_now

end
</pre>

===Test Plan===

====Issue 1====

Step 1: Navigate to Manage --> Assignment page.

Step 2: Click on add participants for any of the assignments.

Step 3: "Import" course participants.

Step 4: Choose a csv file to be imported (follow the format given on the website).

Step 5: The users mentioned in the csv file and don't exist on Expertiza should get a new user email.

Step 6: To check e-mail is received or not, log in with following credentials: username [ 'expertiza.development@gmail.com' ] password [ 'qwer@1234' ].

====Issue 2====

Step 1: Create new assignment [Manage/Assignment/+ Button (to create new assignment)]

Step 2: Fill the details for the assignments.

Step 3: Navigate to due dates.

Step 4: Change the number of review rounds to 2.

Step 5: Select "Yes" in the dropdown for review allowed for submission.

Step 6: Add the user to the assignment.

Step 7: Log in with some user credentials (author credential).

Step 8: Make a new submission to this assignment.

Step 9: Log in with another user (reviewer).

Step 10: Submit a review to the assignment submission.

Step 11: Login as an author again.

Step 12: Edit the submission.

Step 13: After this check the mailbox of the reviewer [development mail for development].

Step 14: Reviewer should get the mail to re-review the work.

Step 15: Change the due date to some date and time which has passed.

Step 16: Now making a new submission from the author account should not send a re-review mail to the reviewer. [Repeat steps 7-15].

====Issue 3====

This test requires an approaching deadline scenario. Since the reminder mail goes through as a sidekiq background, the part of this issue was to fix the email link.

Create a new scenario with an approaching deadline. The email is sent already, just the link is fixed.

===Team Member Details===

Team members :
* Adarsh Trivedi (atrived)
* Dheeraj Makam (drmakam)
* Prachi Sheoran (psheora)

CSC/ECE 517 Fall 2019 - E1940. Improving email notification

2019-10-29T03:58:34Z

Atrived:

CSC/ECE 517 Fall 2019 - E1940. Improving email notification

2019-10-29T02:59:15Z

Atrived: /* Team Member Details */

CSC/ECE 517 Fall 2019 - E1940. Improving email notification

2019-10-29T00:35:21Z

Atrived:

CSC/ECE 517 Fall 2019 - E1940. Improving email notification

2019-10-28T23:09:39Z

Atrived:

=E1940 Improving e-mail notification=

Team members :
* Adarsh Trivedi (atrived)
* Dheeraj
* Prachi

===Brief Introduction===

* E1940 Improving e-mail notification.

* The forked git repository for this project can be found [https://github.com/dheerajmakam/expertiza.git]

===Problem Statement===
The following tasks were accomplished in this project:

*Issue1: Send new account welcome email to user, when imported from CSV through assignment page.
*Issue2: Don't send email to reviewers for a new submission after review deadline has passed.
*Issue3: Adding relevant links to reminder emails.

===Issue 1 - New user email===

'''app/models/assignment_participant.rb'''

* Call method to send mail after user imported successfully.
<pre>
password = user.reset_password
MailerHelper.send_mail_to_user(user, "Your Expertiza account has been created.", "user_welcome", password).deliver
</pre>

===Issue2 - No submission email to reviewer after deadline===
*'''Before fixing this issue, we had to write the logic to send emails to reviewers on submissions.'''

 
'''app/controllers/submitted_content_controller.rb'''
*Added the logic to check for last review date to the function '''submit_hyperlink'''

<pre>
# get current date and time
cur_date = Time.now
# get the last due date for the review
max_due_date = DueDate.where(parent_id: @participant.assignment.id,deadline_type_id: 2).maximum("due_at")

# send mail only if the last due date has not passed
if cur_date <= max_due_date
email_submission_reviewers(@participant)
end
</pre>

*Function to identify the reviewers and send mails.

<pre>
def email_submission_reviewers(participant)
if participant.reviewers.length != 0
participant.reviewers.each do |reviewer|
rev_res_map = ReviewResponseMap.where(['reviewer_id = ? and reviewee_id = ?', reviewer.id, participant.team.id]).first
all_responses = Response.where(:map_id => rev_res_map.id).order("updated_at DESC").first

user = User.find(reviewer.user_id)

if user.email_on_submission?
MailerHelper.submission_mail_to_reviewr(user,
"New submission available to review.",
"update").deliver
end
end
end
end
</pre>

'''app/helpers/mailer_helper.rb'''

*Helper function to mail reviewers

<pre>
def self.submission_mail_to_reviewr(user, subject, mail_partial)
Mailer.notify_reviewer_for_new_submission ({
to: user.email,
subject: subject,
body: {
user: user,
#first_name: ApplicationHelper.get_user_first_name(user),
message: "",
partial_name: mail_partial
}
})
end
</pre>

'''app/mailers/mailer.rb'''

*Mailer function to send the mail.

<pre>
def notify_reviewer_for_new_submission(defn)
@partial_name = defn[:body][:partial_name]
@user = defn[:body][:user]
#@first_name = defn[:body][:first_name]
@message = defn[:body][:message]
mail(subject: defn[:subject],
to: defn[:to])
end
</pre>

'''app/views/mailer/notify_reviewer_for_new_submission.erb'''

*Email template for the mail

<pre>
<!DOCTYPE html>
<html>
<head>
<meta content='text/html; charset=UTF-8' http-equiv='Content-Type' />
</head>
<body>
<%= render :partial => 'mailer/partials/'+@partial_name+'_html' %>
<hr>
</body>
</html>
</pre>

===Issue3 - Adding relevant links to reminder emails.===

CSC/ECE 517 Fall 2019 - E1940. Improving email notification

2019-10-28T23:00:54Z

Atrived:

=E1940 Improving e-mail notification=

Team members :
* Adarsh Trivedi (atrived)
* Dheeraj
* Prachi

===Brief Introduction===

* E1940 Improving e-mail notification.

* The forked git repository for this project can be found [https://github.com/dheerajmakam/expertiza.git]

===Problem Statement===
The following tasks were accomplished in this project:

*Issue1: Send new account welcome email to user, when imported from CSV through assignment page.
*Issue2: Don't send email to reviewers for a new submission after review deadline has passed.
*Issue3: Adding relevant links to reminder emails.

===Issue 1 - New user email===

'''app/models/assignment_participant.rb'''

* Call method to send mail after user imported successfully.
<pre>
password = user.reset_password
MailerHelper.send_mail_to_user(user, "Your Expertiza account has been created.", "user_welcome", password).deliver
</pre>

===Issue2 - No submission email to reviewer after deadline===
*'''Before fixing this issue, we had to write the logic to send emails to reviewers on submissions.'''

 
'''app/controllers/submitted_content_controller.rb'''
*Added the logic to check for last review date to the function '''submit_hyperlink'''

<pre>
# get current date and time
cur_date = Time.now
# get the last due date for the review
max_due_date = DueDate.where(parent_id: @participant.assignment.id,deadline_type_id: 2).maximum("due_at")

# send mail only if the last due date has not passed
if cur_date <= max_due_date
email_submission_reviewers(@participant)
end
</pre>

<pre>
def email_submission_reviewers(participant)
if participant.reviewers.length != 0
participant.reviewers.each do |reviewer|
rev_res_map = ReviewResponseMap.where(['reviewer_id = ? and reviewee_id = ?', reviewer.id, participant.team.id]).first
all_responses = Response.where(:map_id => rev_res_map.id).order("updated_at DESC").first

user = User.find(reviewer.user_id)

if user.email_on_submission?
MailerHelper.submission_mail_to_reviewr(user,
"New submission available to review.",
"update").deliver
end
end
end
end
</pre>

<pre>
def self.submission_mail_to_reviewr(user, subject, mail_partial)
Mailer.notify_reviewer_for_new_submission ({
to: user.email,
subject: subject,
body: {
user: user,
#first_name: ApplicationHelper.get_user_first_name(user),
message: "",
partial_name: mail_partial
}
})
end
</pre>

<pre>
def notify_reviewer_for_new_submission(defn)
@partial_name = defn[:body][:partial_name]
@user = defn[:body][:user]
#@first_name = defn[:body][:first_name]
@message = defn[:body][:message]
mail(subject: defn[:subject],
to: defn[:to])
end
</pre>

<pre>
<!DOCTYPE html>
<html>
<head>
<meta content='text/html; charset=UTF-8' http-equiv='Content-Type' />
</head>
<body>
<%= render :partial => 'mailer/partials/'+@partial_name+'_html' %>
<hr>
</body>
</html>
</pre>

===Issue3 - Adding relevant links to reminder emails.===

CSC/ECE 517 Fall 2019 - E1940. Improving email notification

2019-10-28T22:25:03Z

Atrived: /* E1940 Improving e-mail notification */

CSC/ECE 517 Fall 2019 - E1940. Improving email notification

2019-10-28T22:12:40Z

Atrived:

=E1940 Improving e-mail notification=

Team members :
* Adarsh Trivedi (atrived)
* Dheeraj
* Prachi

CSC/ECE 517 Fall 2019 - E1940. Improving email notification

2019-10-28T22:11:27Z

Atrived: Created page with "=E1967 Fix glitches in Author Feedback= Team members : * Adarsh Trivedi (atrived) * Dheeraj * Prachi"

=E1967 Fix glitches in Author Feedback=

Team members :
* Adarsh Trivedi (atrived)
* Dheeraj
* Prachi

CSC/ECE 517 Fall 2019

2019-10-28T20:28:24Z

Atrived:

* [[CSC/ECE 517 Fall 2019 - Project E1947. Refactor quiz_questionnaire_controller.rb]]
* [[CSC/ECE 517 Fall 2019 - Project E1965. Review report should link to the usual view for reviews]]
* [[CSC/ECE 517 Fall 2019 - E1972. OSS project J. Skellington: Accessing Assignment Rubrics]]
* [[CSC/ECE 517 Fall 2019 - E1961. Email notification to reviewers and instructors]]
* [[CSC/ECE 517 Fall 2019 - E1971. OSS project Finklestein: Instructors & Institutions]]
* [[CSC/ECE 517 Fall 2019 - E1953. Tagging report for student]]
* [[CSC/ECE 517 Fall 2019 - E1955.Write unit tests for student_task.rb]]
* [[CSC/ECE 517 Fall 2019 - E1954. Auto-generate submission directory names based on assignment names]]
* [[CSC/ECE 517 Fall 2019 - E1958. Two issues related to assignment management]]
* [[CSC/ECE 517 Fall 2019 - E1948. Refactor review_mapping_helper.rb]]
* [[CSC/ECE 517 Fall 2019 - E1959. Intelligent copying of assignments without topics]]
* [[CSC/ECE 517 Fall 2019 - E1968. Fixes for adding members to teams]]
* [[CSC/ECE 517 Fall 2019 - E1969. Fixes for reviews not being available]]
* [[CSC/ECE 517 Fall 2019 - E1951. Remove multiple topics at a time]]
* [[CSC/ECE 517 Fall 2019 - E1957. Time travel Not Allowed..!!! Restrict TAs’ ability to change their own grade + limit file-size upload]]
* [[CSC/ECE 517 Fall 2019 - E1963. Changing assignment participant role]]
* [[CSC/ECE 517 Fall 2019 - E1941. Issues related to topic deadlines]]
* [[CSC/ECE 517 Fall 2019 - E1966. Tabbed_reviews partial file refactor for displaying the alternate view of reviews]]
* [[CSC/ECE 517 Fall 2019 - E1962. Email notification upon account creation]]
* [[CSC/ECE 517 Fall 2019 - E1967. Fix glitches in author feedback]]
* [[CSC/ECE 517 Fall 2019 - E1960. Create new late policy successfully and fixing "Back" link]]
* [[CSC/ECE 517 Fall 2019 - E1939. OSS Project Juniper: Bookmark enhancements]]
* [[CSC/ECE 517 Fall 2019 - M1950. Support Asynchronous Web Assembly Compilation]]
* [[CSC/ECE 517 Fall 2019 - E1938. OSS project Duke Blue: Fix import glitches]]
* [[CSC/ECE 517 Fall 2019 - M1951. Implement missing OffscreenCanvas APIs]]
* [[CSC/ECE 517 Fall 2019 - E1940. Improving email notification]]