Expertiza_Wiki - User contributions [en]

CSC/ECE 517 Fall 2019 - E1975. Generalize Review Versioning

2019-12-18T00:16:36Z

Ajain37: /* Rspec */

__TOC__

== '''Introduction''' ==
Expertiza is a peer-review based learning platform in which work-products by students are evaluated and scored by peer-review of other students. Review process occurs in stages. After submission of initial feedback for a work-product by the peer-group, students are allowed time to incorporate the comments and improve their work. This second version of the work product is reviewed again and scores are given. This process might be repeated again and the average score from the last review stage is considered as the final score for that work-product. This incremental development of work-products and progressive learning is the fundamental concept underlying the Expertiza system.

=====Purpose=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The system also triggers an email to reviewer when there is a change in the submission. The scores are calculated on the fly based on the rubrics.

=====Scope=====
This project will enhance the existing scoring and peer review functionality and the underlying implementation along with the testing of the existing implementation. Testing of the score calculation, display for the weighted and non-weighted scores based on the combination of the rubrics are part of this project. This project also includes an additional feature for the author to notify the reviewers for the change in the submission.

=====Background=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The scores are calculated based on the rubrics defined in the system. There’s already an existing implementation for the computation of score but it doesn’t work for multipart rubric

=====Standards=====
The new standards that will be adhered to in this system are those given for the rails framework, and the code should follow rails and object oriented design principles. Any newly added code will adhere to the guidelines mentioned below:

● https://github.com/rubocop-hq/rails-style-guide 
● http://wiki.expertiza.ncsu.edu/index.php/Object-Oriented_Design_and_Programming.

== '''Problem Statement''' ==
=====Problem=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The functionality needs to be tested. It works in some cases, but we are suspicious that it is not entirely correct.

'''Profile Email Preference''':
User in expertiza has liberty to control what kind of emails they should receive. As the reviewers will receive an email for the change in submission. This can be controlled by selecting an option (When someone else submits work I am assigned to review) on the profile page of the user. As the checkboxes are far away on the right corner of the screen, the position of checkboxes need to be changed to make it more user friendly.
 

[[File:Generalize_review.jpg]]

Scores for students in Expertiza can be based on a combination of rubrics: review rubrics for each round of review, author feedback, and teammate review. In CSC/ECE 517, we don’t really use the student-assigned scores to grade projects, but the peer-review score shown to the student is based entirely on the second round of review 

Here is how scores are calculated: 

1. If only zero-weighted scores are available (here, in the first round of review), the average of all zero-weighted scores should be shown in grey font. An information button should explain why the scores are shown in grey (“The reviews submitted so far do not count toward the author’s final grade.”): 

2. If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is:: 
80%⨉90% + 100%⨉10% = 82%

3. If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. '''However, the previous review score that was given in the current round will count until the reviewer updates the submitted review'''. '''This functionality doesn’t work for multipart rubrics.''' 

The system also doesn’t generate emails to reviewers in case a new submission is made by the author without changing submission parameters like submission URL. Eg: A new submission is made by making a new deployment. But the deployment URL doesn’t change. In this case as well, without changing the current flow the author should have an option to notify all the reviewers to revisit the submission.

=====Interface Requirement=====
The overall user interface would remain the same as of the current system. Only the checkboxes for email preferences on user profile will be moved closer to the text to be more visible.

=====Quality Requirements=====
The interface changes are minimal and very easy to adapt. The checkboxes will be clearly visible for email preferences. The proposed changes should not affect the current system functionality in any way. The solution is only intended to enrich the project features.

=====Performance Requirements=====
There are no performance overhead created while meeting the requirements.

=====Assumptions=====
The functionality should work in case the assignment rubric is not multi-part.

== '''Proposed Solution''' ==
=====Profile Email Preference=====
We have modified the file app/views/users/_prefs.html.erb to move the check boxes to the left of the text.

=====Zero-weighted scores=====
If only zero-weighted scores are available in the first round of review, the average of all zero-weighted scores will be shown in a gray font. An information button will explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade”)

Inside compute_total_score method in on_the_fly_calc.rb module, we'll check if the weighted score exists, if not then we'll calculate the non weighted score. We'll also have to add a method in the questionnaire model to get non weighted score. Few changes are required for the UI to display non-weighted score in grey with information icon and weighted score in green.

Modified Files

app/models/assignment_participant.rb 
app/views/grades/view_team.html.erb

We have added a new method weighted_scores_exist which will determine if there's any round which has non-zero weight assigned to it, if yes then the flag is_weighted will be set to true or false and based on the boolean value the css class to change the color of the score will be toggled.

'''
def weighted_scores_exist(scores)
#This method checks if the round has a weight assigned to it#
#A more appropriate place for this method is the assignment_team class and needs to be moved in the future#
scores[:is_weighted] = true
scores.each do |score|
if score[0]["review"]
is_weighted = score[1][:is_weighted]
is_completed = score[1][:assessments].any?
if is_completed and !is_weighted
scores[:is_weighted] = false
end
end
end
scores
end
'''

===== Non-zero weighted scores=====
If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is 80%⨉90% + 100%⨉10% = 82%

The score calculation was initially tested manually to verify the scenarios with multiple weights and rounds.
Also, existing spec for score calculation with a weighted average for both varying and non-varying rubric is covering this scenario.
The spec is written under /spec/models/assignment_spec.rb. The "score" function of the "model/assignment.rb" is called for online (on the fly score calculation).

Following file has the spec for weighted average score calculation.
spec/models/assignment_spec.rb 

=====Review Update=====
If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review.

 This feature can be broken down into the following sub-cases:
 1. A submission can only be made before deadline
 2. A review can edit a review only before deadline
 3. Score calculation should be only done for latest submitted review

[[File:Usecasenew.png]]

=='''Test Plan'''==

=====Rspec=====

New test cases has been added to test and validate the functionalities which are not working as expected. Also, the test cases for features that will be newly added to the system, will also be written.
Test cases has been added to these files:

spec/features/post_deadline_review_submission_sepc.rb 
spec/models/assignment_participant_spec.rb 

The scenarios covered by the newly introduced test cases are as follows: 
1. Zero weighted scores with gray color: rspec spec/model/assignment_participants.rb 
2. Non-zero weighted scores with green color: rspec spec/model/assignment_participants.rb 
3. Review update: rspec spec/features/post_deadline_review_submission_sepc.rb 

The test for "submission after deadline" and "review after deadline" are added as they were not present. A new spec file under spec/features by name post_deadline_review_submission_spec.rb has been created to accommodate the two cases.

 
1. Spec to ensure submission can't be made after the deadline

File : spec/features/post_deadline_review_submission_spec.rb

'''
it 'should not allow submission after deadline' do
user = User.find_by(name: "student2065")
stub_current_user(user, user.role.name, user.role)
# goto student_task page, which has link to "Your work"
visit 'student_task/view?id=1'
# the page will have text "Your work" but will be grayed
expect(page).to have_content "Your work"
# the page will not have link to content "Your work"
expect{click_link "Your work"}.to raise_error(Capybara::ElementNotFound)
end
'''

2. Spec to ensure review can be done only before deadline

File : spec/features/post_deadline_review_submission_spec.rb

'''
it "should not be able to review work after deadline" do
# The spec is written to reproduce following bug. "Others' work" link open after deadline passed
user = User.find_by(name: "student2065")
stub_current_user(user, user.role.name, user.role)
visit 'student_task/view?id=1'
# the page should have content, but after deadline passes it is displayed as gray
# but there should not be any link attached to it
expect(page).to have_content "Others' work"
# this is the bug, even after deadline has passed, the link is still present
# the ui comment in file views/student_task/view.html.erb says
# 
# But the link seems to be open even after deadline passed.
# Screenshot attached as part of wiki for E1975, Fall 2019
expect(page).to have_link("Others' work", "/student_review/list?id=1")
end

'''

The test scenario is able to produce the following bug. At times the review link is open even after the deadline has passed. The bug is also shown/reproduced in the following screenshot for Program 1. 

[[File:ScreenShotReview.jpg]]

3. Score calculation

Score calculation for assignments for review is part of models/assignment.rb. The score calculation is handled by function "score" in this file and the testing is done for both multipart varying and non-varying rubric.
The respective spec is written in file spec/models/assignment_spec.rb.

'''

describe '#scores' do
context 'when assignment is varying rubric by round assignment' do
it 'calculates scores in each round of each team in current assignment' do
allow(participant).to receive(:scores).with(review1: [question]).and_return(98)
allow(assignment).to receive(:varying_rubrics_by_round?).and_return(true)
allow(assignment).to receive(:num_review_rounds).and_return(1)
allow(ReviewResponseMap).to receive(:get_responses_for_team_round).with(team, 1).and_return([response])
allow(Answer).to receive(:compute_scores).with([response], [question]).and_return(max: 95, min: 88, avg: 90)
expect(assignment.scores(review1: [question]).inspect).to eq("{:participants=>{:\"1\"=>98}, :teams=>{:\"0\"=>{:team=>#<AssignmentTeam id: 1, "\
"name: \"no team\", parent_id: 1, type: \"AssignmentTeam\", comments_for_advertisement: nil, advertise_for_partner: nil, "\
"submitted_hyperlinks: \"---\\n- https://www.expertiza.ncsu.edu\", directory_num: 0, grade_for_submission: nil, "\
"comment_for_submission: nil>, :scores=>{:max=>95, :min=>88, :avg=>90.0}}}}")
end
end

context 'when assignment is not varying rubric by round assignment' do
it 'calculates scores of each team in current assignment' do
allow(participant).to receive(:scores).with(review: [question]).and_return(98)
allow(assignment).to receive(:varying_rubrics_by_round?).and_return(false)
allow(ReviewResponseMap).to receive(:get_assessments_for).with(team).and_return([response])
allow(Answer).to receive(:compute_scores).with([response], [question]).and_return(max: 95, min: 88, avg: 90)
expect(assignment.scores(review: [question]).inspect).to eq("{:participants=>{:\"1\"=>98}, :teams=>{:\"0\"=>{:team=>#<AssignmentTeam id: 1, "\
"name: \"no team\", parent_id: 1, type: \"AssignmentTeam\", comments_for_advertisement: nil, advertise_for_partner: nil, "\
"submitted_hyperlinks: \"---\\n- https://www.expertiza.ncsu.edu\", directory_num: 0, grade_for_submission: nil, "\
"comment_for_submission: nil>, :scores=>{:max=>95, :min=>88, :avg=>90}}}}")
end
end
end

'''

 

=====Manual Testing=====

'''Step 1'''. The instructor will create an assignment and assign the zero and non-zero weights to the rubrics. 
'''Step 2'''. The author submits the work done in expertiza. 
'''Step 3'''. Peers check the submitted work and give reviews for it. 

'''Test 1: Zero weighted scores shown in gray.''' 
Login as an author in expertiza and go to scores page to view the feedback.
Zero weighted scores should be displayed in gray to the logged in user.

'''Test 2: Information button.''' 
Login as an author in expertiza and go to scores page to view the feedback.
The information button should explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade.”)

'''Step 4'''. The mentor evaluates the final submission and give grades.

'''Test 3: Display non-zero weighted scores''' 
Login as an author in expertiza and go to scores page to view the feedback.
The scores shown for the assignment should be the weighted average of the scores.
The zero weighted scores have to be replaced with non-zero weighted scores.

'''Step 5''': The author updates the work and re-submits it again.

'''Test 4: Review Update''' 
The Review should be reopened. The reviewer should be able to go and update the submitted review.
If the review is not updated the previous review score has to be retained.

CSC/ECE 517 Fall 2019 - E1975. Generalize Review Versioning

2019-12-18T00:15:28Z

Ajain37: /* Rspec */

__TOC__

== '''Introduction''' ==
Expertiza is a peer-review based learning platform in which work-products by students are evaluated and scored by peer-review of other students. Review process occurs in stages. After submission of initial feedback for a work-product by the peer-group, students are allowed time to incorporate the comments and improve their work. This second version of the work product is reviewed again and scores are given. This process might be repeated again and the average score from the last review stage is considered as the final score for that work-product. This incremental development of work-products and progressive learning is the fundamental concept underlying the Expertiza system.

=====Purpose=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The system also triggers an email to reviewer when there is a change in the submission. The scores are calculated on the fly based on the rubrics.

=====Scope=====
This project will enhance the existing scoring and peer review functionality and the underlying implementation along with the testing of the existing implementation. Testing of the score calculation, display for the weighted and non-weighted scores based on the combination of the rubrics are part of this project. This project also includes an additional feature for the author to notify the reviewers for the change in the submission.

=====Background=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The scores are calculated based on the rubrics defined in the system. There’s already an existing implementation for the computation of score but it doesn’t work for multipart rubric

=====Standards=====
The new standards that will be adhered to in this system are those given for the rails framework, and the code should follow rails and object oriented design principles. Any newly added code will adhere to the guidelines mentioned below:

● https://github.com/rubocop-hq/rails-style-guide 
● http://wiki.expertiza.ncsu.edu/index.php/Object-Oriented_Design_and_Programming.

== '''Problem Statement''' ==
=====Problem=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The functionality needs to be tested. It works in some cases, but we are suspicious that it is not entirely correct.

'''Profile Email Preference''':
User in expertiza has liberty to control what kind of emails they should receive. As the reviewers will receive an email for the change in submission. This can be controlled by selecting an option (When someone else submits work I am assigned to review) on the profile page of the user. As the checkboxes are far away on the right corner of the screen, the position of checkboxes need to be changed to make it more user friendly.
 

[[File:Generalize_review.jpg]]

Scores for students in Expertiza can be based on a combination of rubrics: review rubrics for each round of review, author feedback, and teammate review. In CSC/ECE 517, we don’t really use the student-assigned scores to grade projects, but the peer-review score shown to the student is based entirely on the second round of review 

Here is how scores are calculated: 

1. If only zero-weighted scores are available (here, in the first round of review), the average of all zero-weighted scores should be shown in grey font. An information button should explain why the scores are shown in grey (“The reviews submitted so far do not count toward the author’s final grade.”): 

2. If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is:: 
80%⨉90% + 100%⨉10% = 82%

3. If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. '''However, the previous review score that was given in the current round will count until the reviewer updates the submitted review'''. '''This functionality doesn’t work for multipart rubrics.''' 

The system also doesn’t generate emails to reviewers in case a new submission is made by the author without changing submission parameters like submission URL. Eg: A new submission is made by making a new deployment. But the deployment URL doesn’t change. In this case as well, without changing the current flow the author should have an option to notify all the reviewers to revisit the submission.

=====Interface Requirement=====
The overall user interface would remain the same as of the current system. Only the checkboxes for email preferences on user profile will be moved closer to the text to be more visible.

=====Quality Requirements=====
The interface changes are minimal and very easy to adapt. The checkboxes will be clearly visible for email preferences. The proposed changes should not affect the current system functionality in any way. The solution is only intended to enrich the project features.

=====Performance Requirements=====
There are no performance overhead created while meeting the requirements.

=====Assumptions=====
The functionality should work in case the assignment rubric is not multi-part.

== '''Proposed Solution''' ==
=====Profile Email Preference=====
We have modified the file app/views/users/_prefs.html.erb to move the check boxes to the left of the text.

=====Zero-weighted scores=====
If only zero-weighted scores are available in the first round of review, the average of all zero-weighted scores will be shown in a gray font. An information button will explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade”)

Inside compute_total_score method in on_the_fly_calc.rb module, we'll check if the weighted score exists, if not then we'll calculate the non weighted score. We'll also have to add a method in the questionnaire model to get non weighted score. Few changes are required for the UI to display non-weighted score in grey with information icon and weighted score in green.

Modified Files

app/models/assignment_participant.rb 
app/views/grades/view_team.html.erb

We have added a new method weighted_scores_exist which will determine if there's any round which has non-zero weight assigned to it, if yes then the flag is_weighted will be set to true or false and based on the boolean value the css class to change the color of the score will be toggled.

'''
def weighted_scores_exist(scores)
#This method checks if the round has a weight assigned to it#
#A more appropriate place for this method is the assignment_team class and needs to be moved in the future#
scores[:is_weighted] = true
scores.each do |score|
if score[0]["review"]
is_weighted = score[1][:is_weighted]
is_completed = score[1][:assessments].any?
if is_completed and !is_weighted
scores[:is_weighted] = false
end
end
end
scores
end
'''

===== Non-zero weighted scores=====
If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is 80%⨉90% + 100%⨉10% = 82%

The score calculation was initially tested manually to verify the scenarios with multiple weights and rounds.
Also, existing spec for score calculation with a weighted average for both varying and non-varying rubric is covering this scenario.
The spec is written under /spec/models/assignment_spec.rb. The "score" function of the "model/assignment.rb" is called for online (on the fly score calculation).

Following file has the spec for weighted average score calculation.
spec/models/assignment_spec.rb 

=====Review Update=====
If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review.

 This feature can be broken down into the following sub-cases:
 1. A submission can only be made before deadline
 2. A review can edit a review only before deadline
 3. Score calculation should be only done for latest submitted review

[[File:Usecasenew.png]]

=='''Test Plan'''==

=====Rspec=====

New test cases has been added to test and validate the functionalities which are not working as expected. Also, the test cases for features that will be newly added to the system, will also be written.
Test cases has been added to these files:

spec/features/post_deadline_review_submission_sepc.rb 
spec/models/assignment_participant_spec.rb 

The scenarios covered by the newly introduced test cases are as follows: 
1. Zero weighted scores with gray color: rspec spec/model/assignment_participants.rb 
2. Non-zero weighted scores with green color: rspec spec/model/assignment_participants.rb 
3. Review update: rspec spec/features/post_deadline_review_submission_sepc.rb 

Subcases:

 
1. Spec to ensure submission can't be made after the deadline

File : spec/features/post_deadline_review_submission_spec.rb

'''
it 'should not allow submission after deadline' do
user = User.find_by(name: "student2065")
stub_current_user(user, user.role.name, user.role)
# goto student_task page, which has link to "Your work"
visit 'student_task/view?id=1'
# the page will have text "Your work" but will be grayed
expect(page).to have_content "Your work"
# the page will not have link to content "Your work"
expect{click_link "Your work"}.to raise_error(Capybara::ElementNotFound)
end
'''

2. Spec to ensure review can be done only before deadline

File : spec/features/post_deadline_review_submission_spec.rb

'''
it "should not be able to review work after deadline" do
# The spec is written to reproduce following bug. "Others' work" link open after deadline passed
user = User.find_by(name: "student2065")
stub_current_user(user, user.role.name, user.role)
visit 'student_task/view?id=1'
# the page should have content, but after deadline passes it is displayed as gray
# but there should not be any link attached to it
expect(page).to have_content "Others' work"
# this is the bug, even after deadline has passed, the link is still present
# the ui comment in file views/student_task/view.html.erb says
# 
# But the link seems to be open even after deadline passed.
# Screenshot attached as part of wiki for E1975, Fall 2019
expect(page).to have_link("Others' work", "/student_review/list?id=1")
end

'''

The test scenario is able to produce the following bug. At times the review link is open even after the deadline has passed. The bug is also shown/reproduced in the following screenshot for Program 1. 

[[File:ScreenShotReview.jpg]]

3. Score calculation

Score calculation for assignments for review is part of models/assignment.rb. The score calculation is handled by function "score" in this file and the testing is done for both multipart varying and non-varying rubric.
The respective spec is written in file spec/models/assignment_spec.rb.

'''

describe '#scores' do
context 'when assignment is varying rubric by round assignment' do
it 'calculates scores in each round of each team in current assignment' do
allow(participant).to receive(:scores).with(review1: [question]).and_return(98)
allow(assignment).to receive(:varying_rubrics_by_round?).and_return(true)
allow(assignment).to receive(:num_review_rounds).and_return(1)
allow(ReviewResponseMap).to receive(:get_responses_for_team_round).with(team, 1).and_return([response])
allow(Answer).to receive(:compute_scores).with([response], [question]).and_return(max: 95, min: 88, avg: 90)
expect(assignment.scores(review1: [question]).inspect).to eq("{:participants=>{:\"1\"=>98}, :teams=>{:\"0\"=>{:team=>#<AssignmentTeam id: 1, "\
"name: \"no team\", parent_id: 1, type: \"AssignmentTeam\", comments_for_advertisement: nil, advertise_for_partner: nil, "\
"submitted_hyperlinks: \"---\\n- https://www.expertiza.ncsu.edu\", directory_num: 0, grade_for_submission: nil, "\
"comment_for_submission: nil>, :scores=>{:max=>95, :min=>88, :avg=>90.0}}}}")
end
end

context 'when assignment is not varying rubric by round assignment' do
it 'calculates scores of each team in current assignment' do
allow(participant).to receive(:scores).with(review: [question]).and_return(98)
allow(assignment).to receive(:varying_rubrics_by_round?).and_return(false)
allow(ReviewResponseMap).to receive(:get_assessments_for).with(team).and_return([response])
allow(Answer).to receive(:compute_scores).with([response], [question]).and_return(max: 95, min: 88, avg: 90)
expect(assignment.scores(review: [question]).inspect).to eq("{:participants=>{:\"1\"=>98}, :teams=>{:\"0\"=>{:team=>#<AssignmentTeam id: 1, "\
"name: \"no team\", parent_id: 1, type: \"AssignmentTeam\", comments_for_advertisement: nil, advertise_for_partner: nil, "\
"submitted_hyperlinks: \"---\\n- https://www.expertiza.ncsu.edu\", directory_num: 0, grade_for_submission: nil, "\
"comment_for_submission: nil>, :scores=>{:max=>95, :min=>88, :avg=>90}}}}")
end
end
end

'''

The test for "submission after deadline" and "review after deadline" are added as they were not present. A new spec file under spec/features by name post_deadline_review_submission_spec.rb has been created to accommodate the two cases.

 

=====Manual Testing=====

'''Step 1'''. The instructor will create an assignment and assign the zero and non-zero weights to the rubrics. 
'''Step 2'''. The author submits the work done in expertiza. 
'''Step 3'''. Peers check the submitted work and give reviews for it. 

'''Test 1: Zero weighted scores shown in gray.''' 
Login as an author in expertiza and go to scores page to view the feedback.
Zero weighted scores should be displayed in gray to the logged in user.

'''Test 2: Information button.''' 
Login as an author in expertiza and go to scores page to view the feedback.
The information button should explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade.”)

'''Step 4'''. The mentor evaluates the final submission and give grades.

'''Test 3: Display non-zero weighted scores''' 
Login as an author in expertiza and go to scores page to view the feedback.
The scores shown for the assignment should be the weighted average of the scores.
The zero weighted scores have to be replaced with non-zero weighted scores.

'''Step 5''': The author updates the work and re-submits it again.

'''Test 4: Review Update''' 
The Review should be reopened. The reviewer should be able to go and update the submitted review.
If the review is not updated the previous review score has to be retained.

CSC/ECE 517 Fall 2019 - E1975. Generalize Review Versioning

2019-12-18T00:15:20Z

Ajain37: /* Review Update */

__TOC__

== '''Introduction''' ==
Expertiza is a peer-review based learning platform in which work-products by students are evaluated and scored by peer-review of other students. Review process occurs in stages. After submission of initial feedback for a work-product by the peer-group, students are allowed time to incorporate the comments and improve their work. This second version of the work product is reviewed again and scores are given. This process might be repeated again and the average score from the last review stage is considered as the final score for that work-product. This incremental development of work-products and progressive learning is the fundamental concept underlying the Expertiza system.

=====Purpose=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The system also triggers an email to reviewer when there is a change in the submission. The scores are calculated on the fly based on the rubrics.

=====Scope=====
This project will enhance the existing scoring and peer review functionality and the underlying implementation along with the testing of the existing implementation. Testing of the score calculation, display for the weighted and non-weighted scores based on the combination of the rubrics are part of this project. This project also includes an additional feature for the author to notify the reviewers for the change in the submission.

=====Background=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The scores are calculated based on the rubrics defined in the system. There’s already an existing implementation for the computation of score but it doesn’t work for multipart rubric

=====Standards=====
The new standards that will be adhered to in this system are those given for the rails framework, and the code should follow rails and object oriented design principles. Any newly added code will adhere to the guidelines mentioned below:

● https://github.com/rubocop-hq/rails-style-guide 
● http://wiki.expertiza.ncsu.edu/index.php/Object-Oriented_Design_and_Programming.

== '''Problem Statement''' ==
=====Problem=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The functionality needs to be tested. It works in some cases, but we are suspicious that it is not entirely correct.

'''Profile Email Preference''':
User in expertiza has liberty to control what kind of emails they should receive. As the reviewers will receive an email for the change in submission. This can be controlled by selecting an option (When someone else submits work I am assigned to review) on the profile page of the user. As the checkboxes are far away on the right corner of the screen, the position of checkboxes need to be changed to make it more user friendly.
 

[[File:Generalize_review.jpg]]

Scores for students in Expertiza can be based on a combination of rubrics: review rubrics for each round of review, author feedback, and teammate review. In CSC/ECE 517, we don’t really use the student-assigned scores to grade projects, but the peer-review score shown to the student is based entirely on the second round of review 

Here is how scores are calculated: 

1. If only zero-weighted scores are available (here, in the first round of review), the average of all zero-weighted scores should be shown in grey font. An information button should explain why the scores are shown in grey (“The reviews submitted so far do not count toward the author’s final grade.”): 

2. If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is:: 
80%⨉90% + 100%⨉10% = 82%

3. If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. '''However, the previous review score that was given in the current round will count until the reviewer updates the submitted review'''. '''This functionality doesn’t work for multipart rubrics.''' 

The system also doesn’t generate emails to reviewers in case a new submission is made by the author without changing submission parameters like submission URL. Eg: A new submission is made by making a new deployment. But the deployment URL doesn’t change. In this case as well, without changing the current flow the author should have an option to notify all the reviewers to revisit the submission.

=====Interface Requirement=====
The overall user interface would remain the same as of the current system. Only the checkboxes for email preferences on user profile will be moved closer to the text to be more visible.

=====Quality Requirements=====
The interface changes are minimal and very easy to adapt. The checkboxes will be clearly visible for email preferences. The proposed changes should not affect the current system functionality in any way. The solution is only intended to enrich the project features.

=====Performance Requirements=====
There are no performance overhead created while meeting the requirements.

=====Assumptions=====
The functionality should work in case the assignment rubric is not multi-part.

== '''Proposed Solution''' ==
=====Profile Email Preference=====
We have modified the file app/views/users/_prefs.html.erb to move the check boxes to the left of the text.

=====Zero-weighted scores=====
If only zero-weighted scores are available in the first round of review, the average of all zero-weighted scores will be shown in a gray font. An information button will explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade”)

Inside compute_total_score method in on_the_fly_calc.rb module, we'll check if the weighted score exists, if not then we'll calculate the non weighted score. We'll also have to add a method in the questionnaire model to get non weighted score. Few changes are required for the UI to display non-weighted score in grey with information icon and weighted score in green.

Modified Files

app/models/assignment_participant.rb 
app/views/grades/view_team.html.erb

We have added a new method weighted_scores_exist which will determine if there's any round which has non-zero weight assigned to it, if yes then the flag is_weighted will be set to true or false and based on the boolean value the css class to change the color of the score will be toggled.

'''
def weighted_scores_exist(scores)
#This method checks if the round has a weight assigned to it#
#A more appropriate place for this method is the assignment_team class and needs to be moved in the future#
scores[:is_weighted] = true
scores.each do |score|
if score[0]["review"]
is_weighted = score[1][:is_weighted]
is_completed = score[1][:assessments].any?
if is_completed and !is_weighted
scores[:is_weighted] = false
end
end
end
scores
end
'''

===== Non-zero weighted scores=====
If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is 80%⨉90% + 100%⨉10% = 82%

The score calculation was initially tested manually to verify the scenarios with multiple weights and rounds.
Also, existing spec for score calculation with a weighted average for both varying and non-varying rubric is covering this scenario.
The spec is written under /spec/models/assignment_spec.rb. The "score" function of the "model/assignment.rb" is called for online (on the fly score calculation).

Following file has the spec for weighted average score calculation.
spec/models/assignment_spec.rb 

=====Review Update=====
If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review.

 This feature can be broken down into the following sub-cases:
 1. A submission can only be made before deadline
 2. A review can edit a review only before deadline
 3. Score calculation should be only done for latest submitted review

[[File:Usecasenew.png]]

=='''Test Plan'''==

=====Rspec=====

New test cases has been added to test and validate the functionalities which are not working as expected. Also, the test cases for features that will be newly added to the system, will also be written.
Test cases has been added to these files:

spec/features/post_deadline_review_submission_sepc.rb 
spec/models/assignment_participant_spec.rb 

The scenarios covered by the newly introduced test cases are as follows: 
1. Zero weighted scores with gray color: rspec spec/model/assignment_participants.rb 
2. Non-zero weighted scores with green color: rspec spec/model/assignment_participants.rb 
3. Review update: rspec spec/features/post_deadline_review_submission_sepc.rb 

=====Manual Testing=====

'''Step 1'''. The instructor will create an assignment and assign the zero and non-zero weights to the rubrics. 
'''Step 2'''. The author submits the work done in expertiza. 
'''Step 3'''. Peers check the submitted work and give reviews for it. 

'''Test 1: Zero weighted scores shown in gray.''' 
Login as an author in expertiza and go to scores page to view the feedback.
Zero weighted scores should be displayed in gray to the logged in user.

'''Test 2: Information button.''' 
Login as an author in expertiza and go to scores page to view the feedback.
The information button should explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade.”)

'''Step 4'''. The mentor evaluates the final submission and give grades.

'''Test 3: Display non-zero weighted scores''' 
Login as an author in expertiza and go to scores page to view the feedback.
The scores shown for the assignment should be the weighted average of the scores.
The zero weighted scores have to be replaced with non-zero weighted scores.

'''Step 5''': The author updates the work and re-submits it again.

'''Test 4: Review Update''' 
The Review should be reopened. The reviewer should be able to go and update the submitted review.
If the review is not updated the previous review score has to be retained.

CSC/ECE 517 Fall 2019 - E1975. Generalize Review Versioning

2019-12-17T15:52:58Z

Ajain37: /* Proposed Solution */

__TOC__

== '''Introduction''' ==
Expertiza is a peer-review based learning platform in which work-products by students are evaluated and scored by peer-review of other students. Review process occurs in stages. After submission of initial feedback for a work-product by the peer-group, students are allowed time to incorporate the comments and improve their work. This second version of the work product is reviewed again and scores are given. This process might be repeated again and the average score from the last review stage is considered as the final score for that work-product. This incremental development of work-products and progressive learning is the fundamental concept underlying the Expertiza system.

=====Purpose=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The system also triggers an email to reviewer when there is a change in the submission. The scores are calculated on the fly based on the rubrics.

=====Scope=====
This project will enhance the existing scoring and peer review functionality and the underlying implementation along with the testing of the existing implementation. Testing of the score calculation, display for the weighted and non-weighted scores based on the combination of the rubrics are part of this project. This project also includes an additional feature for the author to notify the reviewers for the change in the submission.

=====Background=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The scores are calculated based on the rubrics defined in the system. There’s already an existing implementation for the computation of score but it doesn’t work for multipart rubric

=====Standards=====
The new standards that will be adhered to in this system are those given for the rails framework, and the code should follow rails and object oriented design principles. Any newly added code will adhere to the guidelines mentioned below:

● https://github.com/rubocop-hq/rails-style-guide 
● http://wiki.expertiza.ncsu.edu/index.php/Object-Oriented_Design_and_Programming.

== '''Problem Statement''' ==
=====Problem=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The functionality needs to be tested. It works in some cases, but we are suspicious that it is not entirely correct.

'''Profile Email Preference''':
User in expertiza has liberty to control what kind of emails they should receive. As the reviewers will receive an email for the change in submission. This can be controlled by selecting an option (When someone else submits work I am assigned to review) on the profile page of the user. As the checkboxes are far away on the right corner of the screen, the position of checkboxes need to be changed to make it more user friendly.
 

[[File:Generalize_review.jpg]]

Scores for students in Expertiza can be based on a combination of rubrics: review rubrics for each round of review, author feedback, and teammate review. In CSC/ECE 517, we don’t really use the student-assigned scores to grade projects, but the peer-review score shown to the student is based entirely on the second round of review 

Here is how scores are calculated: 

1. If only zero-weighted scores are available (here, in the first round of review), the average of all zero-weighted scores should be shown in grey font. An information button should explain why the scores are shown in grey (“The reviews submitted so far do not count toward the author’s final grade.”): 

2. If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is:: 
80%⨉90% + 100%⨉10% = 82%

3. If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. '''However, the previous review score that was given in the current round will count until the reviewer updates the submitted review'''. '''This functionality doesn’t work for multipart rubrics.''' 

The system also doesn’t generate emails to reviewers in case a new submission is made by the author without changing submission parameters like submission URL. Eg: A new submission is made by making a new deployment. But the deployment URL doesn’t change. In this case as well, without changing the current flow the author should have an option to notify all the reviewers to revisit the submission.

=====Interface Requirement=====
The overall user interface would remain the same as of the current system. Only the checkboxes for email preferences on user profile will be moved closer to the text to be more visible.

=====Quality Requirements=====
The interface changes are minimal and very easy to adapt. The checkboxes will be clearly visible for email preferences. The proposed changes should not affect the current system functionality in any way. The solution is only intended to enrich the project features.

=====Performance Requirements=====
There are no performance overhead created while meeting the requirements.

=====Assumptions=====
The functionality should work in case the assignment rubric is not multi-part.

== '''Proposed Solution''' ==
=====Profile Email Preference=====
We have modified the file app/views/users/_prefs.html.erb to move the check boxes to the left of the text.

=====Zero-weighted scores=====
If only zero-weighted scores are available in the first round of review, the average of all zero-weighted scores will be shown in a gray font. An information button will explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade”)

Inside compute_total_score method in on_the_fly_calc.rb module, we'll check if the weighted score exists, if not then we'll calculate the non weighted score. We'll also have to add a method in the questionnaire model to get non weighted score. Few changes are required for the UI to display non-weighted score in grey with information icon and weighted score in green.

Modified Files

app/models/assignment_participant.rb 
app/views/grades/view_team.html.erb

We have added a new method weighted_scores_exist which will determine if there's any round which has non-zero weight assigned to it, if yes then the flag is_weighted will be set to true or false and based on the boolean value the css class to change the color of the score will be toggled.

'''
def weighted_scores_exist(scores)
#This method checks if the round has a weight assigned to it#
#A more appropriate place for this method is the assignment_team class and needs to be moved in the future#
scores[:is_weighted] = true
scores.each do |score|
if score[0]["review"]
is_weighted = score[1][:is_weighted]
is_completed = score[1][:assessments].any?
if is_completed and !is_weighted
scores[:is_weighted] = false
end
end
end
scores
end
'''

===== Non-zero weighted scores=====
If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is 80%⨉90% + 100%⨉10% = 82%

The score calculation was initially tested manually to verify the scenarios with multiple weights and rounds.
Also, existing spec for score calculation with a weighted average for both varying and non-varying rubric is covering this scenario.
The spec is written under /spec/models/assignment_spec.rb. The "score" function of the "model/assignment.rb" is called for online (on the fly score calculation).

Following file has the spec for weighted average score calculation.
spec/models/assignment_spec.rb 

=====Review Update=====
If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review.

 This feature can be broken down into the following sub-cases:
 1. A submission can only be made before deadline
 2. A review can edit a review only before deadline
 3. Score calculation should be only done for latest submitted review

Subcases:

 
1. Spec to ensure submission can't be made after the deadline

File : spec/features/post_deadline_review_submission_spec.rb

'''
it 'should not allow submission after deadline' do
user = User.find_by(name: "student2065")
stub_current_user(user, user.role.name, user.role)
# goto student_task page, which has link to "Your work"
visit 'student_task/view?id=1'
# the page will have text "Your work" but will be grayed
expect(page).to have_content "Your work"
# the page will not have link to content "Your work"
expect{click_link "Your work"}.to raise_error(Capybara::ElementNotFound)
end
'''

2. Spec to ensure review can be done only before deadline

File : spec/features/post_deadline_review_submission_spec.rb

'''
it "should not be able to review work after deadline" do
# The spec is written to reproduce following bug. "Others' work" link open after deadline passed
user = User.find_by(name: "student2065")
stub_current_user(user, user.role.name, user.role)
visit 'student_task/view?id=1'
# the page should have content, but after deadline passes it is displayed as gray
# but there should not be any link attached to it
expect(page).to have_content "Others' work"
# this is the bug, even after deadline has passed, the link is still present
# the ui comment in file views/student_task/view.html.erb says
# 
# But the link seems to be open even after deadline passed.
# Screenshot attached as part of wiki for E1975, Fall 2019
expect(page).to have_link("Others' work", "/student_review/list?id=1")
end

'''

The test scenario is able to produce the following bug. At times the review link is open even after the deadline has passed. The bug is also shown/reproduced in the following screenshot for Program 1. 

[[File:ScreenShotReview.jpg]]

3. Score calculation

Score calculation for assignments for review is part of models/assignment.rb. The score calculation is handled by function "score" in this file and the testing is done for both multipart varying and non-varying rubric.
The respective spec is written in file spec/models/assignment_spec.rb.

'''

describe '#scores' do
context 'when assignment is varying rubric by round assignment' do
it 'calculates scores in each round of each team in current assignment' do
allow(participant).to receive(:scores).with(review1: [question]).and_return(98)
allow(assignment).to receive(:varying_rubrics_by_round?).and_return(true)
allow(assignment).to receive(:num_review_rounds).and_return(1)
allow(ReviewResponseMap).to receive(:get_responses_for_team_round).with(team, 1).and_return([response])
allow(Answer).to receive(:compute_scores).with([response], [question]).and_return(max: 95, min: 88, avg: 90)
expect(assignment.scores(review1: [question]).inspect).to eq("{:participants=>{:\"1\"=>98}, :teams=>{:\"0\"=>{:team=>#<AssignmentTeam id: 1, "\
"name: \"no team\", parent_id: 1, type: \"AssignmentTeam\", comments_for_advertisement: nil, advertise_for_partner: nil, "\
"submitted_hyperlinks: \"---\\n- https://www.expertiza.ncsu.edu\", directory_num: 0, grade_for_submission: nil, "\
"comment_for_submission: nil>, :scores=>{:max=>95, :min=>88, :avg=>90.0}}}}")
end
end

context 'when assignment is not varying rubric by round assignment' do
it 'calculates scores of each team in current assignment' do
allow(participant).to receive(:scores).with(review: [question]).and_return(98)
allow(assignment).to receive(:varying_rubrics_by_round?).and_return(false)
allow(ReviewResponseMap).to receive(:get_assessments_for).with(team).and_return([response])
allow(Answer).to receive(:compute_scores).with([response], [question]).and_return(max: 95, min: 88, avg: 90)
expect(assignment.scores(review: [question]).inspect).to eq("{:participants=>{:\"1\"=>98}, :teams=>{:\"0\"=>{:team=>#<AssignmentTeam id: 1, "\
"name: \"no team\", parent_id: 1, type: \"AssignmentTeam\", comments_for_advertisement: nil, advertise_for_partner: nil, "\
"submitted_hyperlinks: \"---\\n- https://www.expertiza.ncsu.edu\", directory_num: 0, grade_for_submission: nil, "\
"comment_for_submission: nil>, :scores=>{:max=>95, :min=>88, :avg=>90}}}}")
end
end
end

'''

The test for "submission after deadline" and "review after deadline" are added as they were not present. A new spec file under spec/features by name post_deadline_review_submission_spec.rb has been created to accommodate the two cases.

 
[[File:Usecasenew.png]]

=='''Test Plan'''==

=====Rspec=====

New test cases has been added to test and validate the functionalities which are not working as expected. Also, the test cases for features that will be newly added to the system, will also be written.
Test cases has been added to these files:

spec/features/post_deadline_review_submission_sepc.rb 
spec/models/assignment_participant_spec.rb 

The scenarios covered by the newly introduced test cases are as follows: 
1. Zero weighted scores with gray color: rspec spec/model/assignment_participants.rb 
2. Non-zero weighted scores with green color: rspec spec/model/assignment_participants.rb 
3. Review update: rspec spec/features/post_deadline_review_submission_sepc.rb 

=====Manual Testing=====

'''Step 1'''. The instructor will create an assignment and assign the zero and non-zero weights to the rubrics. 
'''Step 2'''. The author submits the work done in expertiza. 
'''Step 3'''. Peers check the submitted work and give reviews for it. 

'''Test 1: Zero weighted scores shown in gray.''' 
Login as an author in expertiza and go to scores page to view the feedback.
Zero weighted scores should be displayed in gray to the logged in user.

'''Test 2: Information button.''' 
Login as an author in expertiza and go to scores page to view the feedback.
The information button should explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade.”)

'''Step 4'''. The mentor evaluates the final submission and give grades.

'''Test 3: Display non-zero weighted scores''' 
Login as an author in expertiza and go to scores page to view the feedback.
The scores shown for the assignment should be the weighted average of the scores.
The zero weighted scores have to be replaced with non-zero weighted scores.

'''Step 5''': The author updates the work and re-submits it again.

'''Test 4: Review Update''' 
The Review should be reopened. The reviewer should be able to go and update the submitted review.
If the review is not updated the previous review score has to be retained.

CSC/ECE 517 Fall 2019 - E1975. Generalize Review Versioning

2019-12-17T15:50:32Z

Ajain37: /* Rspec */

__TOC__

== '''Introduction''' ==
Expertiza is a peer-review based learning platform in which work-products by students are evaluated and scored by peer-review of other students. Review process occurs in stages. After submission of initial feedback for a work-product by the peer-group, students are allowed time to incorporate the comments and improve their work. This second version of the work product is reviewed again and scores are given. This process might be repeated again and the average score from the last review stage is considered as the final score for that work-product. This incremental development of work-products and progressive learning is the fundamental concept underlying the Expertiza system.

=====Purpose=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The system also triggers an email to reviewer when there is a change in the submission. The scores are calculated on the fly based on the rubrics.

=====Scope=====
This project will enhance the existing scoring and peer review functionality and the underlying implementation along with the testing of the existing implementation. Testing of the score calculation, display for the weighted and non-weighted scores based on the combination of the rubrics are part of this project. This project also includes an additional feature for the author to notify the reviewers for the change in the submission.

=====Background=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The scores are calculated based on the rubrics defined in the system. There’s already an existing implementation for the computation of score but it doesn’t work for multipart rubric

=====Standards=====
The new standards that will be adhered to in this system are those given for the rails framework, and the code should follow rails and object oriented design principles. Any newly added code will adhere to the guidelines mentioned below:

● https://github.com/rubocop-hq/rails-style-guide 
● http://wiki.expertiza.ncsu.edu/index.php/Object-Oriented_Design_and_Programming.

== '''Problem Statement''' ==
=====Problem=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The functionality needs to be tested. It works in some cases, but we are suspicious that it is not entirely correct.

'''Profile Email Preference''':
User in expertiza has liberty to control what kind of emails they should receive. As the reviewers will receive an email for the change in submission. This can be controlled by selecting an option (When someone else submits work I am assigned to review) on the profile page of the user. As the checkboxes are far away on the right corner of the screen, the position of checkboxes need to be changed to make it more user friendly.
 

[[File:Generalize_review.jpg]]

Scores for students in Expertiza can be based on a combination of rubrics: review rubrics for each round of review, author feedback, and teammate review. In CSC/ECE 517, we don’t really use the student-assigned scores to grade projects, but the peer-review score shown to the student is based entirely on the second round of review 

Here is how scores are calculated: 

1. If only zero-weighted scores are available (here, in the first round of review), the average of all zero-weighted scores should be shown in grey font. An information button should explain why the scores are shown in grey (“The reviews submitted so far do not count toward the author’s final grade.”): 

2. If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is:: 
80%⨉90% + 100%⨉10% = 82%

3. If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. '''However, the previous review score that was given in the current round will count until the reviewer updates the submitted review'''. '''This functionality doesn’t work for multipart rubrics.''' 

The system also doesn’t generate emails to reviewers in case a new submission is made by the author without changing submission parameters like submission URL. Eg: A new submission is made by making a new deployment. But the deployment URL doesn’t change. In this case as well, without changing the current flow the author should have an option to notify all the reviewers to revisit the submission.

=====Interface Requirement=====
The overall user interface would remain the same as of the current system. Only the checkboxes for email preferences on user profile will be moved closer to the text to be more visible.

=====Quality Requirements=====
The interface changes are minimal and very easy to adapt. The checkboxes will be clearly visible for email preferences. The proposed changes should not affect the current system functionality in any way. The solution is only intended to enrich the project features.

=====Performance Requirements=====
There are no performance overhead created while meeting the requirements.

=====Assumptions=====
The functionality should work in case the assignment rubric is not multi-part.

== '''Proposed Solution''' ==
=====Zero-weighted scores=====
If only zero-weighted scores are available in the first round of review, the average of all zero-weighted scores will be shown in a gray font. An information button will explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade”)

Inside compute_total_score method in on_the_fly_calc.rb module, we'll check if the weighted score exists, if not then we'll calculate the non weighted score. We'll also have to add a method in the questionnaire model to get non weighted score. Few changes are required for the UI to display non-weighted score in grey with information icon and weighted score in green.

Modified Files

app/models/assignment_participant.rb 
app/views/grades/view_team.html.erb

We have added a new method weighted_scores_exist which will determine if there's any round which has non-zero weight assigned to it, if yes then the flag is_weighted will be set to true or false and based on the boolean value the css class to change the color of the score will be toggled.

'''
def weighted_scores_exist(scores)
#This method checks if the round has a weight assigned to it#
#A more appropriate place for this method is the assignment_team class and needs to be moved in the future#
scores[:is_weighted] = true
scores.each do |score|
if score[0]["review"]
is_weighted = score[1][:is_weighted]
is_completed = score[1][:assessments].any?
if is_completed and !is_weighted
scores[:is_weighted] = false
end
end
end
scores
end
'''

===== Non-zero weighted scores=====
If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is 80%⨉90% + 100%⨉10% = 82%

The score calculation was initially tested manually to verify the scenarios with multiple weights and rounds.
Also, existing spec for score calculation with a weighted average for both varying and non-varying rubric is covering this scenario.
The spec is written under /spec/models/assignment_spec.rb. The "score" function of the "model/assignment.rb" is called for online (on the fly score calculation).

Following file has the spec for weighted average score calculation.
spec/models/assignment_spec.rb 

=====Review Update=====
If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review.

 This feature can be broken down into the following sub-cases:
 1. A submission can only be made before deadline
 2. A review can edit a review only before deadline
 3. Score calculation should be only done for latest submitted review

Subcases:

 
1. Spec to ensure submission can't be made after the deadline

File : spec/features/post_deadline_review_submission_spec.rb

'''
it 'should not allow submission after deadline' do
user = User.find_by(name: "student2065")
stub_current_user(user, user.role.name, user.role)
# goto student_task page, which has link to "Your work"
visit 'student_task/view?id=1'
# the page will have text "Your work" but will be grayed
expect(page).to have_content "Your work"
# the page will not have link to content "Your work"
expect{click_link "Your work"}.to raise_error(Capybara::ElementNotFound)
end
'''

2. Spec to ensure review can be done only before deadline

File : spec/features/post_deadline_review_submission_spec.rb

'''
it "should not be able to review work after deadline" do
# The spec is written to reproduce following bug. "Others' work" link open after deadline passed
user = User.find_by(name: "student2065")
stub_current_user(user, user.role.name, user.role)
visit 'student_task/view?id=1'
# the page should have content, but after deadline passes it is displayed as gray
# but there should not be any link attached to it
expect(page).to have_content "Others' work"
# this is the bug, even after deadline has passed, the link is still present
# the ui comment in file views/student_task/view.html.erb says
# 
# But the link seems to be open even after deadline passed.
# Screenshot attached as part of wiki for E1975, Fall 2019
expect(page).to have_link("Others' work", "/student_review/list?id=1")
end

'''

The test scenario is able to produce the following bug. At times the review link is open even after the deadline has passed. The bug is also shown/reproduced in the following screenshot for Program 1. 

[[File:ScreenShotReview.jpg]]

3. Score calculation

Score calculation for assignments for review is part of models/assignment.rb. The score calculation is handled by function "score" in this file and the testing is done for both multipart varying and non-varying rubric.
The respective spec is written in file spec/models/assignment_spec.rb.

'''

describe '#scores' do
context 'when assignment is varying rubric by round assignment' do
it 'calculates scores in each round of each team in current assignment' do
allow(participant).to receive(:scores).with(review1: [question]).and_return(98)
allow(assignment).to receive(:varying_rubrics_by_round?).and_return(true)
allow(assignment).to receive(:num_review_rounds).and_return(1)
allow(ReviewResponseMap).to receive(:get_responses_for_team_round).with(team, 1).and_return([response])
allow(Answer).to receive(:compute_scores).with([response], [question]).and_return(max: 95, min: 88, avg: 90)
expect(assignment.scores(review1: [question]).inspect).to eq("{:participants=>{:\"1\"=>98}, :teams=>{:\"0\"=>{:team=>#<AssignmentTeam id: 1, "\
"name: \"no team\", parent_id: 1, type: \"AssignmentTeam\", comments_for_advertisement: nil, advertise_for_partner: nil, "\
"submitted_hyperlinks: \"---\\n- https://www.expertiza.ncsu.edu\", directory_num: 0, grade_for_submission: nil, "\
"comment_for_submission: nil>, :scores=>{:max=>95, :min=>88, :avg=>90.0}}}}")
end
end

context 'when assignment is not varying rubric by round assignment' do
it 'calculates scores of each team in current assignment' do
allow(participant).to receive(:scores).with(review: [question]).and_return(98)
allow(assignment).to receive(:varying_rubrics_by_round?).and_return(false)
allow(ReviewResponseMap).to receive(:get_assessments_for).with(team).and_return([response])
allow(Answer).to receive(:compute_scores).with([response], [question]).and_return(max: 95, min: 88, avg: 90)
expect(assignment.scores(review: [question]).inspect).to eq("{:participants=>{:\"1\"=>98}, :teams=>{:\"0\"=>{:team=>#<AssignmentTeam id: 1, "\
"name: \"no team\", parent_id: 1, type: \"AssignmentTeam\", comments_for_advertisement: nil, advertise_for_partner: nil, "\
"submitted_hyperlinks: \"---\\n- https://www.expertiza.ncsu.edu\", directory_num: 0, grade_for_submission: nil, "\
"comment_for_submission: nil>, :scores=>{:max=>95, :min=>88, :avg=>90}}}}")
end
end
end

'''

The test for "submission after deadline" and "review after deadline" are added as they were not present. A new spec file under spec/features by name post_deadline_review_submission_spec.rb has been created to accommodate the two cases.

 
[[File:Usecasenew.png]]

=='''Test Plan'''==

=====Rspec=====

New test cases has been added to test and validate the functionalities which are not working as expected. Also, the test cases for features that will be newly added to the system, will also be written.
Test cases has been added to these files:

spec/features/post_deadline_review_submission_sepc.rb 
spec/models/assignment_participant_spec.rb 

The scenarios covered by the newly introduced test cases are as follows: 
1. Zero weighted scores with gray color: rspec spec/model/assignment_participants.rb 
2. Non-zero weighted scores with green color: rspec spec/model/assignment_participants.rb 
3. Review update: rspec spec/features/post_deadline_review_submission_sepc.rb 

=====Manual Testing=====

'''Step 1'''. The instructor will create an assignment and assign the zero and non-zero weights to the rubrics. 
'''Step 2'''. The author submits the work done in expertiza. 
'''Step 3'''. Peers check the submitted work and give reviews for it. 

'''Test 1: Zero weighted scores shown in gray.''' 
Login as an author in expertiza and go to scores page to view the feedback.
Zero weighted scores should be displayed in gray to the logged in user.

'''Test 2: Information button.''' 
Login as an author in expertiza and go to scores page to view the feedback.
The information button should explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade.”)

'''Step 4'''. The mentor evaluates the final submission and give grades.

'''Test 3: Display non-zero weighted scores''' 
Login as an author in expertiza and go to scores page to view the feedback.
The scores shown for the assignment should be the weighted average of the scores.
The zero weighted scores have to be replaced with non-zero weighted scores.

'''Step 5''': The author updates the work and re-submits it again.

'''Test 4: Review Update''' 
The Review should be reopened. The reviewer should be able to go and update the submitted review.
If the review is not updated the previous review score has to be retained.

CSC/ECE 517 Fall 2019 - E1975. Generalize Review Versioning

2019-12-17T15:50:19Z

Ajain37: /* Rspec */

__TOC__

== '''Introduction''' ==
Expertiza is a peer-review based learning platform in which work-products by students are evaluated and scored by peer-review of other students. Review process occurs in stages. After submission of initial feedback for a work-product by the peer-group, students are allowed time to incorporate the comments and improve their work. This second version of the work product is reviewed again and scores are given. This process might be repeated again and the average score from the last review stage is considered as the final score for that work-product. This incremental development of work-products and progressive learning is the fundamental concept underlying the Expertiza system.

=====Purpose=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The system also triggers an email to reviewer when there is a change in the submission. The scores are calculated on the fly based on the rubrics.

=====Scope=====
This project will enhance the existing scoring and peer review functionality and the underlying implementation along with the testing of the existing implementation. Testing of the score calculation, display for the weighted and non-weighted scores based on the combination of the rubrics are part of this project. This project also includes an additional feature for the author to notify the reviewers for the change in the submission.

=====Background=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The scores are calculated based on the rubrics defined in the system. There’s already an existing implementation for the computation of score but it doesn’t work for multipart rubric

=====Standards=====
The new standards that will be adhered to in this system are those given for the rails framework, and the code should follow rails and object oriented design principles. Any newly added code will adhere to the guidelines mentioned below:

● https://github.com/rubocop-hq/rails-style-guide 
● http://wiki.expertiza.ncsu.edu/index.php/Object-Oriented_Design_and_Programming.

== '''Problem Statement''' ==
=====Problem=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The functionality needs to be tested. It works in some cases, but we are suspicious that it is not entirely correct.

'''Profile Email Preference''':
User in expertiza has liberty to control what kind of emails they should receive. As the reviewers will receive an email for the change in submission. This can be controlled by selecting an option (When someone else submits work I am assigned to review) on the profile page of the user. As the checkboxes are far away on the right corner of the screen, the position of checkboxes need to be changed to make it more user friendly.
 

[[File:Generalize_review.jpg]]

Scores for students in Expertiza can be based on a combination of rubrics: review rubrics for each round of review, author feedback, and teammate review. In CSC/ECE 517, we don’t really use the student-assigned scores to grade projects, but the peer-review score shown to the student is based entirely on the second round of review 

Here is how scores are calculated: 

1. If only zero-weighted scores are available (here, in the first round of review), the average of all zero-weighted scores should be shown in grey font. An information button should explain why the scores are shown in grey (“The reviews submitted so far do not count toward the author’s final grade.”): 

2. If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is:: 
80%⨉90% + 100%⨉10% = 82%

3. If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. '''However, the previous review score that was given in the current round will count until the reviewer updates the submitted review'''. '''This functionality doesn’t work for multipart rubrics.''' 

The system also doesn’t generate emails to reviewers in case a new submission is made by the author without changing submission parameters like submission URL. Eg: A new submission is made by making a new deployment. But the deployment URL doesn’t change. In this case as well, without changing the current flow the author should have an option to notify all the reviewers to revisit the submission.

=====Interface Requirement=====
The overall user interface would remain the same as of the current system. Only the checkboxes for email preferences on user profile will be moved closer to the text to be more visible.

=====Quality Requirements=====
The interface changes are minimal and very easy to adapt. The checkboxes will be clearly visible for email preferences. The proposed changes should not affect the current system functionality in any way. The solution is only intended to enrich the project features.

=====Performance Requirements=====
There are no performance overhead created while meeting the requirements.

=====Assumptions=====
The functionality should work in case the assignment rubric is not multi-part.

== '''Proposed Solution''' ==
=====Zero-weighted scores=====
If only zero-weighted scores are available in the first round of review, the average of all zero-weighted scores will be shown in a gray font. An information button will explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade”)

Inside compute_total_score method in on_the_fly_calc.rb module, we'll check if the weighted score exists, if not then we'll calculate the non weighted score. We'll also have to add a method in the questionnaire model to get non weighted score. Few changes are required for the UI to display non-weighted score in grey with information icon and weighted score in green.

Modified Files

app/models/assignment_participant.rb 
app/views/grades/view_team.html.erb

We have added a new method weighted_scores_exist which will determine if there's any round which has non-zero weight assigned to it, if yes then the flag is_weighted will be set to true or false and based on the boolean value the css class to change the color of the score will be toggled.

'''
def weighted_scores_exist(scores)
#This method checks if the round has a weight assigned to it#
#A more appropriate place for this method is the assignment_team class and needs to be moved in the future#
scores[:is_weighted] = true
scores.each do |score|
if score[0]["review"]
is_weighted = score[1][:is_weighted]
is_completed = score[1][:assessments].any?
if is_completed and !is_weighted
scores[:is_weighted] = false
end
end
end
scores
end
'''

===== Non-zero weighted scores=====
If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is 80%⨉90% + 100%⨉10% = 82%

The score calculation was initially tested manually to verify the scenarios with multiple weights and rounds.
Also, existing spec for score calculation with a weighted average for both varying and non-varying rubric is covering this scenario.
The spec is written under /spec/models/assignment_spec.rb. The "score" function of the "model/assignment.rb" is called for online (on the fly score calculation).

Following file has the spec for weighted average score calculation.
spec/models/assignment_spec.rb 

=====Review Update=====
If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review.

 This feature can be broken down into the following sub-cases:
 1. A submission can only be made before deadline
 2. A review can edit a review only before deadline
 3. Score calculation should be only done for latest submitted review

Subcases:

 
1. Spec to ensure submission can't be made after the deadline

File : spec/features/post_deadline_review_submission_spec.rb

'''
it 'should not allow submission after deadline' do
user = User.find_by(name: "student2065")
stub_current_user(user, user.role.name, user.role)
# goto student_task page, which has link to "Your work"
visit 'student_task/view?id=1'
# the page will have text "Your work" but will be grayed
expect(page).to have_content "Your work"
# the page will not have link to content "Your work"
expect{click_link "Your work"}.to raise_error(Capybara::ElementNotFound)
end
'''

2. Spec to ensure review can be done only before deadline

File : spec/features/post_deadline_review_submission_spec.rb

'''
it "should not be able to review work after deadline" do
# The spec is written to reproduce following bug. "Others' work" link open after deadline passed
user = User.find_by(name: "student2065")
stub_current_user(user, user.role.name, user.role)
visit 'student_task/view?id=1'
# the page should have content, but after deadline passes it is displayed as gray
# but there should not be any link attached to it
expect(page).to have_content "Others' work"
# this is the bug, even after deadline has passed, the link is still present
# the ui comment in file views/student_task/view.html.erb says
# 
# But the link seems to be open even after deadline passed.
# Screenshot attached as part of wiki for E1975, Fall 2019
expect(page).to have_link("Others' work", "/student_review/list?id=1")
end

'''

The test scenario is able to produce the following bug. At times the review link is open even after the deadline has passed. The bug is also shown/reproduced in the following screenshot for Program 1. 

[[File:ScreenShotReview.jpg]]

3. Score calculation

Score calculation for assignments for review is part of models/assignment.rb. The score calculation is handled by function "score" in this file and the testing is done for both multipart varying and non-varying rubric.
The respective spec is written in file spec/models/assignment_spec.rb.

'''

describe '#scores' do
context 'when assignment is varying rubric by round assignment' do
it 'calculates scores in each round of each team in current assignment' do
allow(participant).to receive(:scores).with(review1: [question]).and_return(98)
allow(assignment).to receive(:varying_rubrics_by_round?).and_return(true)
allow(assignment).to receive(:num_review_rounds).and_return(1)
allow(ReviewResponseMap).to receive(:get_responses_for_team_round).with(team, 1).and_return([response])
allow(Answer).to receive(:compute_scores).with([response], [question]).and_return(max: 95, min: 88, avg: 90)
expect(assignment.scores(review1: [question]).inspect).to eq("{:participants=>{:\"1\"=>98}, :teams=>{:\"0\"=>{:team=>#<AssignmentTeam id: 1, "\
"name: \"no team\", parent_id: 1, type: \"AssignmentTeam\", comments_for_advertisement: nil, advertise_for_partner: nil, "\
"submitted_hyperlinks: \"---\\n- https://www.expertiza.ncsu.edu\", directory_num: 0, grade_for_submission: nil, "\
"comment_for_submission: nil>, :scores=>{:max=>95, :min=>88, :avg=>90.0}}}}")
end
end

context 'when assignment is not varying rubric by round assignment' do
it 'calculates scores of each team in current assignment' do
allow(participant).to receive(:scores).with(review: [question]).and_return(98)
allow(assignment).to receive(:varying_rubrics_by_round?).and_return(false)
allow(ReviewResponseMap).to receive(:get_assessments_for).with(team).and_return([response])
allow(Answer).to receive(:compute_scores).with([response], [question]).and_return(max: 95, min: 88, avg: 90)
expect(assignment.scores(review: [question]).inspect).to eq("{:participants=>{:\"1\"=>98}, :teams=>{:\"0\"=>{:team=>#<AssignmentTeam id: 1, "\
"name: \"no team\", parent_id: 1, type: \"AssignmentTeam\", comments_for_advertisement: nil, advertise_for_partner: nil, "\
"submitted_hyperlinks: \"---\\n- https://www.expertiza.ncsu.edu\", directory_num: 0, grade_for_submission: nil, "\
"comment_for_submission: nil>, :scores=>{:max=>95, :min=>88, :avg=>90}}}}")
end
end
end

'''

The test for "submission after deadline" and "review after deadline" are added as they were not present. A new spec file under spec/features by name post_deadline_review_submission_spec.rb has been created to accommodate the two cases.

 
[[File:Usecasenew.png]]

=='''Test Plan'''==

=====Rspec=====

New test cases will be added to test and validate the functionalities which are not working as expected. Also, the test cases for features that will be newly added to the system, will also be written.
Test cases has been added to these files:

spec/features/post_deadline_review_submission_sepc.rb 
spec/models/assignment_participant_spec.rb 

The scenarios covered by the newly introduced test cases are as follows: 
1. Zero weighted scores with gray color: rspec spec/model/assignment_participants.rb 
2. Non-zero weighted scores with green color: rspec spec/model/assignment_participants.rb 
3. Review update: rspec spec/features/post_deadline_review_submission_sepc.rb 

=====Manual Testing=====

'''Step 1'''. The instructor will create an assignment and assign the zero and non-zero weights to the rubrics. 
'''Step 2'''. The author submits the work done in expertiza. 
'''Step 3'''. Peers check the submitted work and give reviews for it. 

'''Test 1: Zero weighted scores shown in gray.''' 
Login as an author in expertiza and go to scores page to view the feedback.
Zero weighted scores should be displayed in gray to the logged in user.

'''Test 2: Information button.''' 
Login as an author in expertiza and go to scores page to view the feedback.
The information button should explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade.”)

'''Step 4'''. The mentor evaluates the final submission and give grades.

'''Test 3: Display non-zero weighted scores''' 
Login as an author in expertiza and go to scores page to view the feedback.
The scores shown for the assignment should be the weighted average of the scores.
The zero weighted scores have to be replaced with non-zero weighted scores.

'''Step 5''': The author updates the work and re-submits it again.

'''Test 4: Review Update''' 
The Review should be reopened. The reviewer should be able to go and update the submitted review.
If the review is not updated the previous review score has to be retained.

CSC/ECE 517 Fall 2019 - E1975. Generalize Review Versioning

2019-12-17T15:47:40Z

Ajain37: /* Zero-weighted scores */

__TOC__

== '''Introduction''' ==
Expertiza is a peer-review based learning platform in which work-products by students are evaluated and scored by peer-review of other students. Review process occurs in stages. After submission of initial feedback for a work-product by the peer-group, students are allowed time to incorporate the comments and improve their work. This second version of the work product is reviewed again and scores are given. This process might be repeated again and the average score from the last review stage is considered as the final score for that work-product. This incremental development of work-products and progressive learning is the fundamental concept underlying the Expertiza system.

=====Purpose=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The system also triggers an email to reviewer when there is a change in the submission. The scores are calculated on the fly based on the rubrics.

=====Scope=====
This project will enhance the existing scoring and peer review functionality and the underlying implementation along with the testing of the existing implementation. Testing of the score calculation, display for the weighted and non-weighted scores based on the combination of the rubrics are part of this project. This project also includes an additional feature for the author to notify the reviewers for the change in the submission.

=====Background=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The scores are calculated based on the rubrics defined in the system. There’s already an existing implementation for the computation of score but it doesn’t work for multipart rubric

=====Standards=====
The new standards that will be adhered to in this system are those given for the rails framework, and the code should follow rails and object oriented design principles. Any newly added code will adhere to the guidelines mentioned below:

● https://github.com/rubocop-hq/rails-style-guide 
● http://wiki.expertiza.ncsu.edu/index.php/Object-Oriented_Design_and_Programming.

== '''Problem Statement''' ==
=====Problem=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The functionality needs to be tested. It works in some cases, but we are suspicious that it is not entirely correct.

'''Profile Email Preference''':
User in expertiza has liberty to control what kind of emails they should receive. As the reviewers will receive an email for the change in submission. This can be controlled by selecting an option (When someone else submits work I am assigned to review) on the profile page of the user. As the checkboxes are far away on the right corner of the screen, the position of checkboxes need to be changed to make it more user friendly.
 

[[File:Generalize_review.jpg]]

Scores for students in Expertiza can be based on a combination of rubrics: review rubrics for each round of review, author feedback, and teammate review. In CSC/ECE 517, we don’t really use the student-assigned scores to grade projects, but the peer-review score shown to the student is based entirely on the second round of review 

Here is how scores are calculated: 

1. If only zero-weighted scores are available (here, in the first round of review), the average of all zero-weighted scores should be shown in grey font. An information button should explain why the scores are shown in grey (“The reviews submitted so far do not count toward the author’s final grade.”): 

2. If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is:: 
80%⨉90% + 100%⨉10% = 82%

3. If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. '''However, the previous review score that was given in the current round will count until the reviewer updates the submitted review'''. '''This functionality doesn’t work for multipart rubrics.''' 

The system also doesn’t generate emails to reviewers in case a new submission is made by the author without changing submission parameters like submission URL. Eg: A new submission is made by making a new deployment. But the deployment URL doesn’t change. In this case as well, without changing the current flow the author should have an option to notify all the reviewers to revisit the submission.

=====Interface Requirement=====
The overall user interface would remain the same as of the current system. Only the checkboxes for email preferences on user profile will be moved closer to the text to be more visible.

=====Quality Requirements=====
The interface changes are minimal and very easy to adapt. The checkboxes will be clearly visible for email preferences. The proposed changes should not affect the current system functionality in any way. The solution is only intended to enrich the project features.

=====Performance Requirements=====
There are no performance overhead created while meeting the requirements.

=====Assumptions=====
The functionality should work in case the assignment rubric is not multi-part.

== '''Proposed Solution''' ==
=====Zero-weighted scores=====
If only zero-weighted scores are available in the first round of review, the average of all zero-weighted scores will be shown in a gray font. An information button will explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade”)

Inside compute_total_score method in on_the_fly_calc.rb module, we'll check if the weighted score exists, if not then we'll calculate the non weighted score. We'll also have to add a method in the questionnaire model to get non weighted score. Few changes are required for the UI to display non-weighted score in grey with information icon and weighted score in green.

Modified Files

app/models/assignment_participant.rb 
app/views/grades/view_team.html.erb

We have added a new method weighted_scores_exist which will determine if there's any round which has non-zero weight assigned to it, if yes then the flag is_weighted will be set to true or false and based on the boolean value the css class to change the color of the score will be toggled.

'''
def weighted_scores_exist(scores)
#This method checks if the round has a weight assigned to it#
#A more appropriate place for this method is the assignment_team class and needs to be moved in the future#
scores[:is_weighted] = true
scores.each do |score|
if score[0]["review"]
is_weighted = score[1][:is_weighted]
is_completed = score[1][:assessments].any?
if is_completed and !is_weighted
scores[:is_weighted] = false
end
end
end
scores
end
'''

===== Non-zero weighted scores=====
If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is 80%⨉90% + 100%⨉10% = 82%

The score calculation was initially tested manually to verify the scenarios with multiple weights and rounds.
Also, existing spec for score calculation with a weighted average for both varying and non-varying rubric is covering this scenario.
The spec is written under /spec/models/assignment_spec.rb. The "score" function of the "model/assignment.rb" is called for online (on the fly score calculation).

Following file has the spec for weighted average score calculation.
spec/models/assignment_spec.rb 

=====Review Update=====
If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review.

 This feature can be broken down into the following sub-cases:
 1. A submission can only be made before deadline
 2. A review can edit a review only before deadline
 3. Score calculation should be only done for latest submitted review

Subcases:

 
1. Spec to ensure submission can't be made after the deadline

File : spec/features/post_deadline_review_submission_spec.rb

'''
it 'should not allow submission after deadline' do
user = User.find_by(name: "student2065")
stub_current_user(user, user.role.name, user.role)
# goto student_task page, which has link to "Your work"
visit 'student_task/view?id=1'
# the page will have text "Your work" but will be grayed
expect(page).to have_content "Your work"
# the page will not have link to content "Your work"
expect{click_link "Your work"}.to raise_error(Capybara::ElementNotFound)
end
'''

2. Spec to ensure review can be done only before deadline

File : spec/features/post_deadline_review_submission_spec.rb

'''
it "should not be able to review work after deadline" do
# The spec is written to reproduce following bug. "Others' work" link open after deadline passed
user = User.find_by(name: "student2065")
stub_current_user(user, user.role.name, user.role)
visit 'student_task/view?id=1'
# the page should have content, but after deadline passes it is displayed as gray
# but there should not be any link attached to it
expect(page).to have_content "Others' work"
# this is the bug, even after deadline has passed, the link is still present
# the ui comment in file views/student_task/view.html.erb says
# 
# But the link seems to be open even after deadline passed.
# Screenshot attached as part of wiki for E1975, Fall 2019
expect(page).to have_link("Others' work", "/student_review/list?id=1")
end

'''

The test scenario is able to produce the following bug. At times the review link is open even after the deadline has passed. The bug is also shown/reproduced in the following screenshot for Program 1. 

[[File:ScreenShotReview.jpg]]

3. Score calculation

Score calculation for assignments for review is part of models/assignment.rb. The score calculation is handled by function "score" in this file and the testing is done for both multipart varying and non-varying rubric.
The respective spec is written in file spec/models/assignment_spec.rb.

'''

describe '#scores' do
context 'when assignment is varying rubric by round assignment' do
it 'calculates scores in each round of each team in current assignment' do
allow(participant).to receive(:scores).with(review1: [question]).and_return(98)
allow(assignment).to receive(:varying_rubrics_by_round?).and_return(true)
allow(assignment).to receive(:num_review_rounds).and_return(1)
allow(ReviewResponseMap).to receive(:get_responses_for_team_round).with(team, 1).and_return([response])
allow(Answer).to receive(:compute_scores).with([response], [question]).and_return(max: 95, min: 88, avg: 90)
expect(assignment.scores(review1: [question]).inspect).to eq("{:participants=>{:\"1\"=>98}, :teams=>{:\"0\"=>{:team=>#<AssignmentTeam id: 1, "\
"name: \"no team\", parent_id: 1, type: \"AssignmentTeam\", comments_for_advertisement: nil, advertise_for_partner: nil, "\
"submitted_hyperlinks: \"---\\n- https://www.expertiza.ncsu.edu\", directory_num: 0, grade_for_submission: nil, "\
"comment_for_submission: nil>, :scores=>{:max=>95, :min=>88, :avg=>90.0}}}}")
end
end

context 'when assignment is not varying rubric by round assignment' do
it 'calculates scores of each team in current assignment' do
allow(participant).to receive(:scores).with(review: [question]).and_return(98)
allow(assignment).to receive(:varying_rubrics_by_round?).and_return(false)
allow(ReviewResponseMap).to receive(:get_assessments_for).with(team).and_return([response])
allow(Answer).to receive(:compute_scores).with([response], [question]).and_return(max: 95, min: 88, avg: 90)
expect(assignment.scores(review: [question]).inspect).to eq("{:participants=>{:\"1\"=>98}, :teams=>{:\"0\"=>{:team=>#<AssignmentTeam id: 1, "\
"name: \"no team\", parent_id: 1, type: \"AssignmentTeam\", comments_for_advertisement: nil, advertise_for_partner: nil, "\
"submitted_hyperlinks: \"---\\n- https://www.expertiza.ncsu.edu\", directory_num: 0, grade_for_submission: nil, "\
"comment_for_submission: nil>, :scores=>{:max=>95, :min=>88, :avg=>90}}}}")
end
end
end

'''

The test for "submission after deadline" and "review after deadline" are added as they were not present. A new spec file under spec/features by name post_deadline_review_submission_spec.rb has been created to accommodate the two cases.

 
[[File:Usecasenew.png]]

=='''Test Plan'''==

=====Rspec=====

New test cases will be added to test and validate the functionalities which are not working as expected. Also, the test cases for features that will be newly added to the system, will also be written.
Test cases will be added to these files:

spec/features/post_deadline_review_submission_sepc.rb 
spec/models/assignment_participant_spec.rb 

The scenarios covered by the newly introduced test cases are as follows: 
1. Zero weighted scores with gray mark: rspec spec/model/assignment_participants.rb 
2. Non-zero weighted scores 
3. Review update: rspec spec/features/post_deadline_review_submission_sepc.rb 

=====Manual Testing=====

'''Step 1'''. The instructor will create an assignment and assign the zero and non-zero weights to the rubrics. 
'''Step 2'''. The author submits the work done in expertiza. 
'''Step 3'''. Peers check the submitted work and give reviews for it. 

'''Test 1: Zero weighted scores shown in gray.''' 
Login as an author in expertiza and go to scores page to view the feedback.
Zero weighted scores should be displayed in gray to the logged in user.

'''Test 2: Information button.''' 
Login as an author in expertiza and go to scores page to view the feedback.
The information button should explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade.”)

'''Step 4'''. The mentor evaluates the final submission and give grades.

'''Test 3: Display non-zero weighted scores''' 
Login as an author in expertiza and go to scores page to view the feedback.
The scores shown for the assignment should be the weighted average of the scores.
The zero weighted scores have to be replaced with non-zero weighted scores.

'''Step 5''': The author updates the work and re-submits it again.

'''Test 4: Review Update''' 
The Review should be reopened. The reviewer should be able to go and update the submitted review.
If the review is not updated the previous review score has to be retained.

CSC/ECE 517 Fall 2019 - E1975. Generalize Review Versioning

2019-12-17T15:44:51Z

Ajain37: /* Zero-weighted scores */

__TOC__

== '''Introduction''' ==
Expertiza is a peer-review based learning platform in which work-products by students are evaluated and scored by peer-review of other students. Review process occurs in stages. After submission of initial feedback for a work-product by the peer-group, students are allowed time to incorporate the comments and improve their work. This second version of the work product is reviewed again and scores are given. This process might be repeated again and the average score from the last review stage is considered as the final score for that work-product. This incremental development of work-products and progressive learning is the fundamental concept underlying the Expertiza system.

=====Purpose=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The system also triggers an email to reviewer when there is a change in the submission. The scores are calculated on the fly based on the rubrics.

=====Scope=====
This project will enhance the existing scoring and peer review functionality and the underlying implementation along with the testing of the existing implementation. Testing of the score calculation, display for the weighted and non-weighted scores based on the combination of the rubrics are part of this project. This project also includes an additional feature for the author to notify the reviewers for the change in the submission.

=====Background=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The scores are calculated based on the rubrics defined in the system. There’s already an existing implementation for the computation of score but it doesn’t work for multipart rubric

=====Standards=====
The new standards that will be adhered to in this system are those given for the rails framework, and the code should follow rails and object oriented design principles. Any newly added code will adhere to the guidelines mentioned below:

● https://github.com/rubocop-hq/rails-style-guide 
● http://wiki.expertiza.ncsu.edu/index.php/Object-Oriented_Design_and_Programming.

== '''Problem Statement''' ==
=====Problem=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The functionality needs to be tested. It works in some cases, but we are suspicious that it is not entirely correct.

'''Profile Email Preference''':
User in expertiza has liberty to control what kind of emails they should receive. As the reviewers will receive an email for the change in submission. This can be controlled by selecting an option (When someone else submits work I am assigned to review) on the profile page of the user. As the checkboxes are far away on the right corner of the screen, the position of checkboxes need to be changed to make it more user friendly.
 

[[File:Generalize_review.jpg]]

Scores for students in Expertiza can be based on a combination of rubrics: review rubrics for each round of review, author feedback, and teammate review. In CSC/ECE 517, we don’t really use the student-assigned scores to grade projects, but the peer-review score shown to the student is based entirely on the second round of review 

Here is how scores are calculated: 

1. If only zero-weighted scores are available (here, in the first round of review), the average of all zero-weighted scores should be shown in grey font. An information button should explain why the scores are shown in grey (“The reviews submitted so far do not count toward the author’s final grade.”): 

2. If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is:: 
80%⨉90% + 100%⨉10% = 82%

3. If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. '''However, the previous review score that was given in the current round will count until the reviewer updates the submitted review'''. '''This functionality doesn’t work for multipart rubrics.''' 

The system also doesn’t generate emails to reviewers in case a new submission is made by the author without changing submission parameters like submission URL. Eg: A new submission is made by making a new deployment. But the deployment URL doesn’t change. In this case as well, without changing the current flow the author should have an option to notify all the reviewers to revisit the submission.

=====Interface Requirement=====
The overall user interface would remain the same as of the current system. Only the checkboxes for email preferences on user profile will be moved closer to the text to be more visible.

=====Quality Requirements=====
The interface changes are minimal and very easy to adapt. The checkboxes will be clearly visible for email preferences. The proposed changes should not affect the current system functionality in any way. The solution is only intended to enrich the project features.

=====Performance Requirements=====
There are no performance overhead created while meeting the requirements.

=====Assumptions=====
The functionality should work in case the assignment rubric is not multi-part.

== '''Proposed Solution''' ==
=====Zero-weighted scores=====
If only zero-weighted scores are available in the first round of review, the average of all zero-weighted scores will be shown in a gray font. An information button will explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade”)

Inside compute_total_score method in on_the_fly_calc.rb module, we'll check if the weighted score exists, if not then we'll calculate the non weighted score. We'll also have to add a method in the questionnaire model to get non weighted score. Few changes are required for the UI to display non-weighted score in grey with information icon and weighted score in green.

Modified Files

app/models/assignment_participant.rb 
app/views/grades/view_team.html.erb

'''
def weighted_scores_exist(scores)
#This method checks if the round has a weight assigned to it#
#A more appropriate place for this method is the assignment_team class and needs to be moved in the future#
scores[:is_weighted] = true
scores.each do |score|
if score[0]["review"]
is_weighted = score[1][:is_weighted]
is_completed = score[1][:assessments].any?
if is_completed and !is_weighted
scores[:is_weighted] = false
end
end
end
scores
end
'''

===== Non-zero weighted scores=====
If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is 80%⨉90% + 100%⨉10% = 82%

The score calculation was initially tested manually to verify the scenarios with multiple weights and rounds.
Also, existing spec for score calculation with a weighted average for both varying and non-varying rubric is covering this scenario.
The spec is written under /spec/models/assignment_spec.rb. The "score" function of the "model/assignment.rb" is called for online (on the fly score calculation).

Following file has the spec for weighted average score calculation.
spec/models/assignment_spec.rb 

=====Review Update=====
If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review.

 This feature can be broken down into the following sub-cases:
 1. A submission can only be made before deadline
 2. A review can edit a review only before deadline
 3. Score calculation should be only done for latest submitted review

Subcases:

 
1. Spec to ensure submission can't be made after the deadline

File : spec/features/post_deadline_review_submission_spec.rb

'''
it 'should not allow submission after deadline' do
user = User.find_by(name: "student2065")
stub_current_user(user, user.role.name, user.role)
# goto student_task page, which has link to "Your work"
visit 'student_task/view?id=1'
# the page will have text "Your work" but will be grayed
expect(page).to have_content "Your work"
# the page will not have link to content "Your work"
expect{click_link "Your work"}.to raise_error(Capybara::ElementNotFound)
end
'''

2. Spec to ensure review can be done only before deadline

File : spec/features/post_deadline_review_submission_spec.rb

'''
it "should not be able to review work after deadline" do
# The spec is written to reproduce following bug. "Others' work" link open after deadline passed
user = User.find_by(name: "student2065")
stub_current_user(user, user.role.name, user.role)
visit 'student_task/view?id=1'
# the page should have content, but after deadline passes it is displayed as gray
# but there should not be any link attached to it
expect(page).to have_content "Others' work"
# this is the bug, even after deadline has passed, the link is still present
# the ui comment in file views/student_task/view.html.erb says
# 
# But the link seems to be open even after deadline passed.
# Screenshot attached as part of wiki for E1975, Fall 2019
expect(page).to have_link("Others' work", "/student_review/list?id=1")
end

'''

The test scenario is able to produce the following bug. At times the review link is open even after the deadline has passed. The bug is also shown/reproduced in the following screenshot for Program 1. 

[[File:ScreenShotReview.jpg]]

3. Score calculation

Score calculation for assignments for review is part of models/assignment.rb. The score calculation is handled by function "score" in this file and the testing is done for both multipart varying and non-varying rubric.
The respective spec is written in file spec/models/assignment_spec.rb.

'''

describe '#scores' do
context 'when assignment is varying rubric by round assignment' do
it 'calculates scores in each round of each team in current assignment' do
allow(participant).to receive(:scores).with(review1: [question]).and_return(98)
allow(assignment).to receive(:varying_rubrics_by_round?).and_return(true)
allow(assignment).to receive(:num_review_rounds).and_return(1)
allow(ReviewResponseMap).to receive(:get_responses_for_team_round).with(team, 1).and_return([response])
allow(Answer).to receive(:compute_scores).with([response], [question]).and_return(max: 95, min: 88, avg: 90)
expect(assignment.scores(review1: [question]).inspect).to eq("{:participants=>{:\"1\"=>98}, :teams=>{:\"0\"=>{:team=>#<AssignmentTeam id: 1, "\
"name: \"no team\", parent_id: 1, type: \"AssignmentTeam\", comments_for_advertisement: nil, advertise_for_partner: nil, "\
"submitted_hyperlinks: \"---\\n- https://www.expertiza.ncsu.edu\", directory_num: 0, grade_for_submission: nil, "\
"comment_for_submission: nil>, :scores=>{:max=>95, :min=>88, :avg=>90.0}}}}")
end
end

context 'when assignment is not varying rubric by round assignment' do
it 'calculates scores of each team in current assignment' do
allow(participant).to receive(:scores).with(review: [question]).and_return(98)
allow(assignment).to receive(:varying_rubrics_by_round?).and_return(false)
allow(ReviewResponseMap).to receive(:get_assessments_for).with(team).and_return([response])
allow(Answer).to receive(:compute_scores).with([response], [question]).and_return(max: 95, min: 88, avg: 90)
expect(assignment.scores(review: [question]).inspect).to eq("{:participants=>{:\"1\"=>98}, :teams=>{:\"0\"=>{:team=>#<AssignmentTeam id: 1, "\
"name: \"no team\", parent_id: 1, type: \"AssignmentTeam\", comments_for_advertisement: nil, advertise_for_partner: nil, "\
"submitted_hyperlinks: \"---\\n- https://www.expertiza.ncsu.edu\", directory_num: 0, grade_for_submission: nil, "\
"comment_for_submission: nil>, :scores=>{:max=>95, :min=>88, :avg=>90}}}}")
end
end
end

'''

The test for "submission after deadline" and "review after deadline" are added as they were not present. A new spec file under spec/features by name post_deadline_review_submission_spec.rb has been created to accommodate the two cases.

 
[[File:Usecasenew.png]]

=='''Test Plan'''==

=====Rspec=====

New test cases will be added to test and validate the functionalities which are not working as expected. Also, the test cases for features that will be newly added to the system, will also be written.
Test cases will be added to these files:

spec/features/post_deadline_review_submission_sepc.rb 
spec/models/assignment_participant_spec.rb 

The scenarios covered by the newly introduced test cases are as follows: 
1. Zero weighted scores with gray mark: rspec spec/model/assignment_participants.rb 
2. Non-zero weighted scores 
3. Review update: rspec spec/features/post_deadline_review_submission_sepc.rb 

=====Manual Testing=====

'''Step 1'''. The instructor will create an assignment and assign the zero and non-zero weights to the rubrics. 
'''Step 2'''. The author submits the work done in expertiza. 
'''Step 3'''. Peers check the submitted work and give reviews for it. 

'''Test 1: Zero weighted scores shown in gray.''' 
Login as an author in expertiza and go to scores page to view the feedback.
Zero weighted scores should be displayed in gray to the logged in user.

'''Test 2: Information button.''' 
Login as an author in expertiza and go to scores page to view the feedback.
The information button should explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade.”)

'''Step 4'''. The mentor evaluates the final submission and give grades.

'''Test 3: Display non-zero weighted scores''' 
Login as an author in expertiza and go to scores page to view the feedback.
The scores shown for the assignment should be the weighted average of the scores.
The zero weighted scores have to be replaced with non-zero weighted scores.

'''Step 5''': The author updates the work and re-submits it again.

'''Test 4: Review Update''' 
The Review should be reopened. The reviewer should be able to go and update the submitted review.
If the review is not updated the previous review score has to be retained.

CSC/ECE 517 Fall 2019 - E1975. Generalize Review Versioning

2019-12-17T15:43:56Z

Ajain37: /* Zero-weighted scores */

__TOC__

== '''Introduction''' ==
Expertiza is a peer-review based learning platform in which work-products by students are evaluated and scored by peer-review of other students. Review process occurs in stages. After submission of initial feedback for a work-product by the peer-group, students are allowed time to incorporate the comments and improve their work. This second version of the work product is reviewed again and scores are given. This process might be repeated again and the average score from the last review stage is considered as the final score for that work-product. This incremental development of work-products and progressive learning is the fundamental concept underlying the Expertiza system.

=====Purpose=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The system also triggers an email to reviewer when there is a change in the submission. The scores are calculated on the fly based on the rubrics.

=====Scope=====
This project will enhance the existing scoring and peer review functionality and the underlying implementation along with the testing of the existing implementation. Testing of the score calculation, display for the weighted and non-weighted scores based on the combination of the rubrics are part of this project. This project also includes an additional feature for the author to notify the reviewers for the change in the submission.

=====Background=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The scores are calculated based on the rubrics defined in the system. There’s already an existing implementation for the computation of score but it doesn’t work for multipart rubric

=====Standards=====
The new standards that will be adhered to in this system are those given for the rails framework, and the code should follow rails and object oriented design principles. Any newly added code will adhere to the guidelines mentioned below:

● https://github.com/rubocop-hq/rails-style-guide 
● http://wiki.expertiza.ncsu.edu/index.php/Object-Oriented_Design_and_Programming.

== '''Problem Statement''' ==
=====Problem=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The functionality needs to be tested. It works in some cases, but we are suspicious that it is not entirely correct.

'''Profile Email Preference''':
User in expertiza has liberty to control what kind of emails they should receive. As the reviewers will receive an email for the change in submission. This can be controlled by selecting an option (When someone else submits work I am assigned to review) on the profile page of the user. As the checkboxes are far away on the right corner of the screen, the position of checkboxes need to be changed to make it more user friendly.
 

[[File:Generalize_review.jpg]]

Scores for students in Expertiza can be based on a combination of rubrics: review rubrics for each round of review, author feedback, and teammate review. In CSC/ECE 517, we don’t really use the student-assigned scores to grade projects, but the peer-review score shown to the student is based entirely on the second round of review 

Here is how scores are calculated: 

1. If only zero-weighted scores are available (here, in the first round of review), the average of all zero-weighted scores should be shown in grey font. An information button should explain why the scores are shown in grey (“The reviews submitted so far do not count toward the author’s final grade.”): 

2. If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is:: 
80%⨉90% + 100%⨉10% = 82%

3. If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. '''However, the previous review score that was given in the current round will count until the reviewer updates the submitted review'''. '''This functionality doesn’t work for multipart rubrics.''' 

The system also doesn’t generate emails to reviewers in case a new submission is made by the author without changing submission parameters like submission URL. Eg: A new submission is made by making a new deployment. But the deployment URL doesn’t change. In this case as well, without changing the current flow the author should have an option to notify all the reviewers to revisit the submission.

=====Interface Requirement=====
The overall user interface would remain the same as of the current system. Only the checkboxes for email preferences on user profile will be moved closer to the text to be more visible.

=====Quality Requirements=====
The interface changes are minimal and very easy to adapt. The checkboxes will be clearly visible for email preferences. The proposed changes should not affect the current system functionality in any way. The solution is only intended to enrich the project features.

=====Performance Requirements=====
There are no performance overhead created while meeting the requirements.

=====Assumptions=====
The functionality should work in case the assignment rubric is not multi-part.

== '''Proposed Solution''' ==
=====Zero-weighted scores=====
If only zero-weighted scores are available in the first round of review, the average of all zero-weighted scores will be shown in a gray font. An information button will explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade”)

Inside compute_total_score method in on_the_fly_calc.rb module, we'll check if the weighted score exists, if not then we'll calculate the non weighted score. We'll also have to add a method in the questionnaire model to get non weighted score. Few changes are required for the UI to display non-weighted score in grey with information icon and weighted score in green.

Modified Files

app/models/assignment_participant.rb 
app/views/grades/view_team.html.erb

===== Non-zero weighted scores=====
If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is 80%⨉90% + 100%⨉10% = 82%

The score calculation was initially tested manually to verify the scenarios with multiple weights and rounds.
Also, existing spec for score calculation with a weighted average for both varying and non-varying rubric is covering this scenario.
The spec is written under /spec/models/assignment_spec.rb. The "score" function of the "model/assignment.rb" is called for online (on the fly score calculation).

Following file has the spec for weighted average score calculation.
spec/models/assignment_spec.rb 

=====Review Update=====
If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review.

 This feature can be broken down into the following sub-cases:
 1. A submission can only be made before deadline
 2. A review can edit a review only before deadline
 3. Score calculation should be only done for latest submitted review

Subcases:

 
1. Spec to ensure submission can't be made after the deadline

File : spec/features/post_deadline_review_submission_spec.rb

'''
it 'should not allow submission after deadline' do
user = User.find_by(name: "student2065")
stub_current_user(user, user.role.name, user.role)
# goto student_task page, which has link to "Your work"
visit 'student_task/view?id=1'
# the page will have text "Your work" but will be grayed
expect(page).to have_content "Your work"
# the page will not have link to content "Your work"
expect{click_link "Your work"}.to raise_error(Capybara::ElementNotFound)
end
'''

2. Spec to ensure review can be done only before deadline

File : spec/features/post_deadline_review_submission_spec.rb

'''
it "should not be able to review work after deadline" do
# The spec is written to reproduce following bug. "Others' work" link open after deadline passed
user = User.find_by(name: "student2065")
stub_current_user(user, user.role.name, user.role)
visit 'student_task/view?id=1'
# the page should have content, but after deadline passes it is displayed as gray
# but there should not be any link attached to it
expect(page).to have_content "Others' work"
# this is the bug, even after deadline has passed, the link is still present
# the ui comment in file views/student_task/view.html.erb says
# 
# But the link seems to be open even after deadline passed.
# Screenshot attached as part of wiki for E1975, Fall 2019
expect(page).to have_link("Others' work", "/student_review/list?id=1")
end

'''

The test scenario is able to produce the following bug. At times the review link is open even after the deadline has passed. The bug is also shown/reproduced in the following screenshot for Program 1. 

[[File:ScreenShotReview.jpg]]

3. Score calculation

Score calculation for assignments for review is part of models/assignment.rb. The score calculation is handled by function "score" in this file and the testing is done for both multipart varying and non-varying rubric.
The respective spec is written in file spec/models/assignment_spec.rb.

'''

describe '#scores' do
context 'when assignment is varying rubric by round assignment' do
it 'calculates scores in each round of each team in current assignment' do
allow(participant).to receive(:scores).with(review1: [question]).and_return(98)
allow(assignment).to receive(:varying_rubrics_by_round?).and_return(true)
allow(assignment).to receive(:num_review_rounds).and_return(1)
allow(ReviewResponseMap).to receive(:get_responses_for_team_round).with(team, 1).and_return([response])
allow(Answer).to receive(:compute_scores).with([response], [question]).and_return(max: 95, min: 88, avg: 90)
expect(assignment.scores(review1: [question]).inspect).to eq("{:participants=>{:\"1\"=>98}, :teams=>{:\"0\"=>{:team=>#<AssignmentTeam id: 1, "\
"name: \"no team\", parent_id: 1, type: \"AssignmentTeam\", comments_for_advertisement: nil, advertise_for_partner: nil, "\
"submitted_hyperlinks: \"---\\n- https://www.expertiza.ncsu.edu\", directory_num: 0, grade_for_submission: nil, "\
"comment_for_submission: nil>, :scores=>{:max=>95, :min=>88, :avg=>90.0}}}}")
end
end

context 'when assignment is not varying rubric by round assignment' do
it 'calculates scores of each team in current assignment' do
allow(participant).to receive(:scores).with(review: [question]).and_return(98)
allow(assignment).to receive(:varying_rubrics_by_round?).and_return(false)
allow(ReviewResponseMap).to receive(:get_assessments_for).with(team).and_return([response])
allow(Answer).to receive(:compute_scores).with([response], [question]).and_return(max: 95, min: 88, avg: 90)
expect(assignment.scores(review: [question]).inspect).to eq("{:participants=>{:\"1\"=>98}, :teams=>{:\"0\"=>{:team=>#<AssignmentTeam id: 1, "\
"name: \"no team\", parent_id: 1, type: \"AssignmentTeam\", comments_for_advertisement: nil, advertise_for_partner: nil, "\
"submitted_hyperlinks: \"---\\n- https://www.expertiza.ncsu.edu\", directory_num: 0, grade_for_submission: nil, "\
"comment_for_submission: nil>, :scores=>{:max=>95, :min=>88, :avg=>90}}}}")
end
end
end

'''

The test for "submission after deadline" and "review after deadline" are added as they were not present. A new spec file under spec/features by name post_deadline_review_submission_spec.rb has been created to accommodate the two cases.

 
[[File:Usecasenew.png]]

=='''Test Plan'''==

=====Rspec=====

New test cases will be added to test and validate the functionalities which are not working as expected. Also, the test cases for features that will be newly added to the system, will also be written.
Test cases will be added to these files:

spec/features/post_deadline_review_submission_sepc.rb 
spec/models/assignment_participant_spec.rb 

The scenarios covered by the newly introduced test cases are as follows: 
1. Zero weighted scores with gray mark: rspec spec/model/assignment_participants.rb 
2. Non-zero weighted scores 
3. Review update: rspec spec/features/post_deadline_review_submission_sepc.rb 

=====Manual Testing=====

'''Step 1'''. The instructor will create an assignment and assign the zero and non-zero weights to the rubrics. 
'''Step 2'''. The author submits the work done in expertiza. 
'''Step 3'''. Peers check the submitted work and give reviews for it. 

'''Test 1: Zero weighted scores shown in gray.''' 
Login as an author in expertiza and go to scores page to view the feedback.
Zero weighted scores should be displayed in gray to the logged in user.

'''Test 2: Information button.''' 
Login as an author in expertiza and go to scores page to view the feedback.
The information button should explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade.”)

'''Step 4'''. The mentor evaluates the final submission and give grades.

'''Test 3: Display non-zero weighted scores''' 
Login as an author in expertiza and go to scores page to view the feedback.
The scores shown for the assignment should be the weighted average of the scores.
The zero weighted scores have to be replaced with non-zero weighted scores.

'''Step 5''': The author updates the work and re-submits it again.

'''Test 4: Review Update''' 
The Review should be reopened. The reviewer should be able to go and update the submitted review.
If the review is not updated the previous review score has to be retained.

CSC/ECE 517 Fall 2019 - E1975. Generalize Review Versioning

2019-12-17T15:43:31Z

Ajain37: /* Proposed Solution */

__TOC__

== '''Introduction''' ==
Expertiza is a peer-review based learning platform in which work-products by students are evaluated and scored by peer-review of other students. Review process occurs in stages. After submission of initial feedback for a work-product by the peer-group, students are allowed time to incorporate the comments and improve their work. This second version of the work product is reviewed again and scores are given. This process might be repeated again and the average score from the last review stage is considered as the final score for that work-product. This incremental development of work-products and progressive learning is the fundamental concept underlying the Expertiza system.

=====Purpose=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The system also triggers an email to reviewer when there is a change in the submission. The scores are calculated on the fly based on the rubrics.

=====Scope=====
This project will enhance the existing scoring and peer review functionality and the underlying implementation along with the testing of the existing implementation. Testing of the score calculation, display for the weighted and non-weighted scores based on the combination of the rubrics are part of this project. This project also includes an additional feature for the author to notify the reviewers for the change in the submission.

=====Background=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The scores are calculated based on the rubrics defined in the system. There’s already an existing implementation for the computation of score but it doesn’t work for multipart rubric

=====Standards=====
The new standards that will be adhered to in this system are those given for the rails framework, and the code should follow rails and object oriented design principles. Any newly added code will adhere to the guidelines mentioned below:

● https://github.com/rubocop-hq/rails-style-guide 
● http://wiki.expertiza.ncsu.edu/index.php/Object-Oriented_Design_and_Programming.

== '''Problem Statement''' ==
=====Problem=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The functionality needs to be tested. It works in some cases, but we are suspicious that it is not entirely correct.

'''Profile Email Preference''':
User in expertiza has liberty to control what kind of emails they should receive. As the reviewers will receive an email for the change in submission. This can be controlled by selecting an option (When someone else submits work I am assigned to review) on the profile page of the user. As the checkboxes are far away on the right corner of the screen, the position of checkboxes need to be changed to make it more user friendly.
 

[[File:Generalize_review.jpg]]

Scores for students in Expertiza can be based on a combination of rubrics: review rubrics for each round of review, author feedback, and teammate review. In CSC/ECE 517, we don’t really use the student-assigned scores to grade projects, but the peer-review score shown to the student is based entirely on the second round of review 

Here is how scores are calculated: 

1. If only zero-weighted scores are available (here, in the first round of review), the average of all zero-weighted scores should be shown in grey font. An information button should explain why the scores are shown in grey (“The reviews submitted so far do not count toward the author’s final grade.”): 

2. If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is:: 
80%⨉90% + 100%⨉10% = 82%

3. If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. '''However, the previous review score that was given in the current round will count until the reviewer updates the submitted review'''. '''This functionality doesn’t work for multipart rubrics.''' 

The system also doesn’t generate emails to reviewers in case a new submission is made by the author without changing submission parameters like submission URL. Eg: A new submission is made by making a new deployment. But the deployment URL doesn’t change. In this case as well, without changing the current flow the author should have an option to notify all the reviewers to revisit the submission.

=====Interface Requirement=====
The overall user interface would remain the same as of the current system. Only the checkboxes for email preferences on user profile will be moved closer to the text to be more visible.

=====Quality Requirements=====
The interface changes are minimal and very easy to adapt. The checkboxes will be clearly visible for email preferences. The proposed changes should not affect the current system functionality in any way. The solution is only intended to enrich the project features.

=====Performance Requirements=====
There are no performance overhead created while meeting the requirements.

=====Assumptions=====
The functionality should work in case the assignment rubric is not multi-part.

== '''Proposed Solution''' ==
=====Zero-weighted scores=====
If only zero-weighted scores are available in the first round of review, the average of all zero-weighted scores will be shown in a gray font. An information button will explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade”)

Inside compute_total_score method in on_the_fly_calc.rb module, we'll check if the weighted score exists, if not then we'll calculate the non weighted score. We'll also have to add a method in the questionnaire model to get non weighted score. Few changes are required for the UI to display non-weighted score in grey with information icon and weighted score in green.

Modified Files

app/models/assignment_participant.rb 
models/questionnaire.rb 
app/views/grades/view_team.html.erb

===== Non-zero weighted scores=====
If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is 80%⨉90% + 100%⨉10% = 82%

The score calculation was initially tested manually to verify the scenarios with multiple weights and rounds.
Also, existing spec for score calculation with a weighted average for both varying and non-varying rubric is covering this scenario.
The spec is written under /spec/models/assignment_spec.rb. The "score" function of the "model/assignment.rb" is called for online (on the fly score calculation).

Following file has the spec for weighted average score calculation.
spec/models/assignment_spec.rb 

=====Review Update=====
If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review.

 This feature can be broken down into the following sub-cases:
 1. A submission can only be made before deadline
 2. A review can edit a review only before deadline
 3. Score calculation should be only done for latest submitted review

Subcases:

 
1. Spec to ensure submission can't be made after the deadline

File : spec/features/post_deadline_review_submission_spec.rb

'''
it 'should not allow submission after deadline' do
user = User.find_by(name: "student2065")
stub_current_user(user, user.role.name, user.role)
# goto student_task page, which has link to "Your work"
visit 'student_task/view?id=1'
# the page will have text "Your work" but will be grayed
expect(page).to have_content "Your work"
# the page will not have link to content "Your work"
expect{click_link "Your work"}.to raise_error(Capybara::ElementNotFound)
end
'''

2. Spec to ensure review can be done only before deadline

File : spec/features/post_deadline_review_submission_spec.rb

'''
it "should not be able to review work after deadline" do
# The spec is written to reproduce following bug. "Others' work" link open after deadline passed
user = User.find_by(name: "student2065")
stub_current_user(user, user.role.name, user.role)
visit 'student_task/view?id=1'
# the page should have content, but after deadline passes it is displayed as gray
# but there should not be any link attached to it
expect(page).to have_content "Others' work"
# this is the bug, even after deadline has passed, the link is still present
# the ui comment in file views/student_task/view.html.erb says
# 
# But the link seems to be open even after deadline passed.
# Screenshot attached as part of wiki for E1975, Fall 2019
expect(page).to have_link("Others' work", "/student_review/list?id=1")
end

'''

The test scenario is able to produce the following bug. At times the review link is open even after the deadline has passed. The bug is also shown/reproduced in the following screenshot for Program 1. 

[[File:ScreenShotReview.jpg]]

3. Score calculation

Score calculation for assignments for review is part of models/assignment.rb. The score calculation is handled by function "score" in this file and the testing is done for both multipart varying and non-varying rubric.
The respective spec is written in file spec/models/assignment_spec.rb.

'''

describe '#scores' do
context 'when assignment is varying rubric by round assignment' do
it 'calculates scores in each round of each team in current assignment' do
allow(participant).to receive(:scores).with(review1: [question]).and_return(98)
allow(assignment).to receive(:varying_rubrics_by_round?).and_return(true)
allow(assignment).to receive(:num_review_rounds).and_return(1)
allow(ReviewResponseMap).to receive(:get_responses_for_team_round).with(team, 1).and_return([response])
allow(Answer).to receive(:compute_scores).with([response], [question]).and_return(max: 95, min: 88, avg: 90)
expect(assignment.scores(review1: [question]).inspect).to eq("{:participants=>{:\"1\"=>98}, :teams=>{:\"0\"=>{:team=>#<AssignmentTeam id: 1, "\
"name: \"no team\", parent_id: 1, type: \"AssignmentTeam\", comments_for_advertisement: nil, advertise_for_partner: nil, "\
"submitted_hyperlinks: \"---\\n- https://www.expertiza.ncsu.edu\", directory_num: 0, grade_for_submission: nil, "\
"comment_for_submission: nil>, :scores=>{:max=>95, :min=>88, :avg=>90.0}}}}")
end
end

context 'when assignment is not varying rubric by round assignment' do
it 'calculates scores of each team in current assignment' do
allow(participant).to receive(:scores).with(review: [question]).and_return(98)
allow(assignment).to receive(:varying_rubrics_by_round?).and_return(false)
allow(ReviewResponseMap).to receive(:get_assessments_for).with(team).and_return([response])
allow(Answer).to receive(:compute_scores).with([response], [question]).and_return(max: 95, min: 88, avg: 90)
expect(assignment.scores(review: [question]).inspect).to eq("{:participants=>{:\"1\"=>98}, :teams=>{:\"0\"=>{:team=>#<AssignmentTeam id: 1, "\
"name: \"no team\", parent_id: 1, type: \"AssignmentTeam\", comments_for_advertisement: nil, advertise_for_partner: nil, "\
"submitted_hyperlinks: \"---\\n- https://www.expertiza.ncsu.edu\", directory_num: 0, grade_for_submission: nil, "\
"comment_for_submission: nil>, :scores=>{:max=>95, :min=>88, :avg=>90}}}}")
end
end
end

'''

The test for "submission after deadline" and "review after deadline" are added as they were not present. A new spec file under spec/features by name post_deadline_review_submission_spec.rb has been created to accommodate the two cases.

 
[[File:Usecasenew.png]]

=='''Test Plan'''==

=====Rspec=====

New test cases will be added to test and validate the functionalities which are not working as expected. Also, the test cases for features that will be newly added to the system, will also be written.
Test cases will be added to these files:

spec/features/post_deadline_review_submission_sepc.rb 
spec/models/assignment_participant_spec.rb 

The scenarios covered by the newly introduced test cases are as follows: 
1. Zero weighted scores with gray mark: rspec spec/model/assignment_participants.rb 
2. Non-zero weighted scores 
3. Review update: rspec spec/features/post_deadline_review_submission_sepc.rb 

=====Manual Testing=====

'''Step 1'''. The instructor will create an assignment and assign the zero and non-zero weights to the rubrics. 
'''Step 2'''. The author submits the work done in expertiza. 
'''Step 3'''. Peers check the submitted work and give reviews for it. 

'''Test 1: Zero weighted scores shown in gray.''' 
Login as an author in expertiza and go to scores page to view the feedback.
Zero weighted scores should be displayed in gray to the logged in user.

'''Test 2: Information button.''' 
Login as an author in expertiza and go to scores page to view the feedback.
The information button should explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade.”)

'''Step 4'''. The mentor evaluates the final submission and give grades.

'''Test 3: Display non-zero weighted scores''' 
Login as an author in expertiza and go to scores page to view the feedback.
The scores shown for the assignment should be the weighted average of the scores.
The zero weighted scores have to be replaced with non-zero weighted scores.

'''Step 5''': The author updates the work and re-submits it again.

'''Test 4: Review Update''' 
The Review should be reopened. The reviewer should be able to go and update the submitted review.
If the review is not updated the previous review score has to be retained.

CSC/ECE 517 Fall 2019 - E1975. Generalize Review Versioning

2019-12-17T02:13:59Z

Ajain37: /* Problem */

__TOC__

== '''Introduction''' ==
Expertiza is a peer-review based learning platform in which work-products by students are evaluated and scored by peer-review of other students. Review process occurs in stages. After submission of initial feedback for a work-product by the peer-group, students are allowed time to incorporate the comments and improve their work. This second version of the work product is reviewed again and scores are given. This process might be repeated again and the average score from the last review stage is considered as the final score for that work-product. This incremental development of work-products and progressive learning is the fundamental concept underlying the Expertiza system.

=====Purpose=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The system also triggers an email to reviewer when there is a change in the submission. The scores are calculated on the fly based on the rubrics.

=====Scope=====
This project will enhance the existing scoring and peer review functionality and the underlying implementation along with the testing of the existing implementation. Testing of the score calculation, display for the weighted and non-weighted scores based on the combination of the rubrics are part of this project. This project also includes an additional feature for the author to notify the reviewers for the change in the submission.

=====Background=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The scores are calculated based on the rubrics defined in the system. There’s already an existing implementation for the computation of score but it doesn’t work for multipart rubric

=====Standards=====
The new standards that will be adhered to in this system are those given for the rails framework, and the code should follow rails and object oriented design principles. Any newly added code will adhere to the guidelines mentioned below:

● https://github.com/rubocop-hq/rails-style-guide 
● http://wiki.expertiza.ncsu.edu/index.php/Object-Oriented_Design_and_Programming.

== '''Problem Statement''' ==
=====Problem=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The functionality needs to be tested. It works in some cases, but we are suspicious that it is not entirely correct.

'''Profile Email Preference''':
User in expertiza has liberty to control what kind of emails they should receive. As the reviewers will receive an email for the change in submission. This can be controlled by selecting an option (When someone else submits work I am assigned to review) on the profile page of the user. As the checkboxes are far away on the right corner of the screen, the position of checkboxes need to be changed to make it more user friendly.
 

[[File:Generalize_review.jpg]]

Scores for students in Expertiza can be based on a combination of rubrics: review rubrics for each round of review, author feedback, and teammate review. In CSC/ECE 517, we don’t really use the student-assigned scores to grade projects, but the peer-review score shown to the student is based entirely on the second round of review 

Here is how scores are calculated: 

1. If only zero-weighted scores are available (here, in the first round of review), the average of all zero-weighted scores should be shown in grey font. An information button should explain why the scores are shown in grey (“The reviews submitted so far do not count toward the author’s final grade.”): 

2. If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is:: 
80%⨉90% + 100%⨉10% = 82%

3. If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. '''However, the previous review score that was given in the current round will count until the reviewer updates the submitted review'''. '''This functionality doesn’t work for multipart rubrics.''' 

The system also doesn’t generate emails to reviewers in case a new submission is made by the author without changing submission parameters like submission URL. Eg: A new submission is made by making a new deployment. But the deployment URL doesn’t change. In this case as well, without changing the current flow the author should have an option to notify all the reviewers to revisit the submission.

=====Interface Requirement=====
The overall user interface would remain the same as of the current system. Since this is a modification to the current submission page, there will be a new addition to the submission page. On the submission page, there will be a new button to notify reviewers when there is a submission which doesn’t contain a change to submission parameters like submission URL, but the deployment has been updated on the same URL. Other than that the interface remains the same for the score page, only functionalities will be tested for multi-part rubrics.

=====Quality Requirements=====
The interface changes are minimal and very easy to adapt. The new button will be clearly visible and is supposed to have a clear text to interpret its functionality. The proposed changes should not affect the current system functionality in any way. The solution is only intended to enrich the project features.

=====Performance Requirements=====
Adding test for multi-part rubric functionality should not cause any overhead in the performance of the system. Sending emails through “Notify All Reviewers” button should affect the system minimally in terms of performance. The only overhead is a database query to get all the reviewers. A good database design should incorporate such a query smoothly.

=====Assumptions=====
The functionality should work in case the assignment rubric is not multi-part.

== '''Proposed Solution''' ==
=====Zero-weighted scores=====
If only zero-weighted scores are available in the first round of review, the average of all zero-weighted scores will be shown in a gray font. An information button will explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade”)

Inside compute_total_score method in on_the_fly_calc.rb module, we'll check if the weighted score exists, if not then we'll calculate the non weighted score. We'll also have to add a method in the questionnaire model to get non weighted score. Few changes are required for the UI to display non-weighted score in grey with information icon and non weighted score in black.

Possible files to be modified.

models/on_the_fly_calc.rb 
models/questionnaire.rb 
models/view_team.html.erb

===== Non-zero weighted scores=====
If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is 80%⨉90% + 100%⨉10% = 82%

The score calculation was initially tested manually to verify the scenarios with multiple weights and rounds.
Also, existing spec for score calculation with a weighted average for both varying and non-varying rubric is covering this scenario.
The spec is written under /spec/models/assignment_spec.rb. The "score" function of the "model/assignment.rb" is called for online (on the fly score calculation).

Following file has the spec for weighted average score calculation.
spec/models/assignment_spec.rb 

=====Review Update=====
If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review.

 This feature can be broken down into the following sub-cases:
 1. A submission can only be made before deadline
 2. A review can edit a review only before deadline
 3. Score calculation should be only done for latest submitted review

Subcases:

 
1. Spec to ensure submission can't be made after the deadline

File : spec/features/post_deadline_review_submission_spec.rb

'''
it 'should not allow submission after deadline' do
user = User.find_by(name: "student2065")
stub_current_user(user, user.role.name, user.role)
# goto student_task page, which has link to "Your work"
visit 'student_task/view?id=1'
# the page will have text "Your work" but will be grayed
expect(page).to have_content "Your work"
# the page will not have link to content "Your work"
expect{click_link "Your work"}.to raise_error(Capybara::ElementNotFound)
end
'''

2. Spec to ensure review can be done only before deadline

File : spec/features/post_deadline_review_submission_spec.rb

'''
it "should not be able to review work after deadline" do
# The spec is written to reproduce following bug. "Others' work" link open after deadline passed
user = User.find_by(name: "student2065")
stub_current_user(user, user.role.name, user.role)
visit 'student_task/view?id=1'
# the page should have content, but after deadline passes it is displayed as gray
# but there should not be any link attached to it
expect(page).to have_content "Others' work"
# this is the bug, even after deadline has passed, the link is still present
# the ui comment in file views/student_task/view.html.erb says
# 
# But the link seems to be open even after deadline passed.
# Screenshot attached as part of wiki for E1975, Fall 2019
expect(page).to have_link("Others' work", "/student_review/list?id=1")
end

'''

The test scenario is able to produce the following bug. At times the review link is open even after the deadline has passed. The bug is also shown/reproduced in the following screenshot for Program 1. 

[[File:ScreenShotReview.jpg]]

3. Score calculation

Score calculation for assignments for review is part of models/assignment.rb. The score calculation is handled by function "score" in this file and the testing is done for both multipart varying and non-varying rubric.
The respective spec is written in file spec/models/assignment_spec.rb.

'''

describe '#scores' do
context 'when assignment is varying rubric by round assignment' do
it 'calculates scores in each round of each team in current assignment' do
allow(participant).to receive(:scores).with(review1: [question]).and_return(98)
allow(assignment).to receive(:varying_rubrics_by_round?).and_return(true)
allow(assignment).to receive(:num_review_rounds).and_return(1)
allow(ReviewResponseMap).to receive(:get_responses_for_team_round).with(team, 1).and_return([response])
allow(Answer).to receive(:compute_scores).with([response], [question]).and_return(max: 95, min: 88, avg: 90)
expect(assignment.scores(review1: [question]).inspect).to eq("{:participants=>{:\"1\"=>98}, :teams=>{:\"0\"=>{:team=>#<AssignmentTeam id: 1, "\
"name: \"no team\", parent_id: 1, type: \"AssignmentTeam\", comments_for_advertisement: nil, advertise_for_partner: nil, "\
"submitted_hyperlinks: \"---\\n- https://www.expertiza.ncsu.edu\", directory_num: 0, grade_for_submission: nil, "\
"comment_for_submission: nil>, :scores=>{:max=>95, :min=>88, :avg=>90.0}}}}")
end
end

context 'when assignment is not varying rubric by round assignment' do
it 'calculates scores of each team in current assignment' do
allow(participant).to receive(:scores).with(review: [question]).and_return(98)
allow(assignment).to receive(:varying_rubrics_by_round?).and_return(false)
allow(ReviewResponseMap).to receive(:get_assessments_for).with(team).and_return([response])
allow(Answer).to receive(:compute_scores).with([response], [question]).and_return(max: 95, min: 88, avg: 90)
expect(assignment.scores(review: [question]).inspect).to eq("{:participants=>{:\"1\"=>98}, :teams=>{:\"0\"=>{:team=>#<AssignmentTeam id: 1, "\
"name: \"no team\", parent_id: 1, type: \"AssignmentTeam\", comments_for_advertisement: nil, advertise_for_partner: nil, "\
"submitted_hyperlinks: \"---\\n- https://www.expertiza.ncsu.edu\", directory_num: 0, grade_for_submission: nil, "\
"comment_for_submission: nil>, :scores=>{:max=>95, :min=>88, :avg=>90}}}}")
end
end
end

'''

 
[[File:Usecasenew.png]]

=='''Test Plan'''==

=====Rspec=====

New test cases will be added to test and validate the functionalities which are not working as expected. Also, the test cases for features that will be newly added to the system, will also be written.
Test cases will be added to these files:

spec/features/post_deadline_review_submission_sepc.rb 
spec/models/assignment_participant_spec.rb 

The scenarios covered by the newly introduced test cases are as follows: 
1. Zero weighted scores with gray mark: rspec spec/model/assignment_participants.rb 
2. Non-zero weighted scores 
3. Review update: rspec spec/features/post_deadline_review_submission_sepc.rb 

=====Manual Testing=====

'''Step 1'''. The instructor will create an assignment and assign the zero and non-zero weights to the rubrics. 
'''Step 2'''. The author submits the work done in expertiza. 
'''Step 3'''. Peers check the submitted work and give reviews for it. 

'''Test 1: Zero weighted scores shown in gray.''' 
Login as an author in expertiza and go to scores page to view the feedback.
Zero weighted scores should be displayed in gray to the logged in user.

'''Test 2: Information button.''' 
Login as an author in expertiza and go to scores page to view the feedback.
The information button should explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade.”)

'''Step 4'''. The mentor evaluates the final submission and give grades.

'''Test 3: Display non-zero weighted scores''' 
Login as an author in expertiza and go to scores page to view the feedback.
The scores shown for the assignment should be the weighted average of the scores.
The zero weighted scores have to be replaced with non-zero weighted scores.

'''Step 5''': The author updates the work and re-submits it again.

'''Test 4: Review Update''' 
The Review should be reopened. The reviewer should be able to go and update the submitted review.
If the review is not updated the previous review score has to be retained.

CSC/ECE 517 Fall 2019 - E1975. Generalize Review Versioning

2019-12-17T02:13:46Z

Ajain37: /* Problem */

__TOC__

== '''Introduction''' ==
Expertiza is a peer-review based learning platform in which work-products by students are evaluated and scored by peer-review of other students. Review process occurs in stages. After submission of initial feedback for a work-product by the peer-group, students are allowed time to incorporate the comments and improve their work. This second version of the work product is reviewed again and scores are given. This process might be repeated again and the average score from the last review stage is considered as the final score for that work-product. This incremental development of work-products and progressive learning is the fundamental concept underlying the Expertiza system.

=====Purpose=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The system also triggers an email to reviewer when there is a change in the submission. The scores are calculated on the fly based on the rubrics.

=====Scope=====
This project will enhance the existing scoring and peer review functionality and the underlying implementation along with the testing of the existing implementation. Testing of the score calculation, display for the weighted and non-weighted scores based on the combination of the rubrics are part of this project. This project also includes an additional feature for the author to notify the reviewers for the change in the submission.

=====Background=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The scores are calculated based on the rubrics defined in the system. There’s already an existing implementation for the computation of score but it doesn’t work for multipart rubric

=====Standards=====
The new standards that will be adhered to in this system are those given for the rails framework, and the code should follow rails and object oriented design principles. Any newly added code will adhere to the guidelines mentioned below:

● https://github.com/rubocop-hq/rails-style-guide 
● http://wiki.expertiza.ncsu.edu/index.php/Object-Oriented_Design_and_Programming.

== '''Problem Statement''' ==
=====Problem=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The functionality needs to be tested. It works in some cases, but we are suspicious that it is not entirely correct.

'''Profile Email Preference''':
User in expertiza has liberty to control what kind of emails they should receive. As the reviewers will receive an email for the change in submission. This can be controlled by selecting an option (When someone else submits work I am assigned to review) on the profile page of the user. As the checkboxes are far away on the right corner of the screen, the position of checkboxes need to be changed to make it more user friendly.
 

[[File:Generalize_review.jpg]]

 

Scores for students in Expertiza can be based on a combination of rubrics: review rubrics for each round of review, author feedback, and teammate review. In CSC/ECE 517, we don’t really use the student-assigned scores to grade projects, but the peer-review score shown to the student is based entirely on the second round of review 

Here is how scores are calculated: 

1. If only zero-weighted scores are available (here, in the first round of review), the average of all zero-weighted scores should be shown in grey font. An information button should explain why the scores are shown in grey (“The reviews submitted so far do not count toward the author’s final grade.”): 

2. If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is:: 
80%⨉90% + 100%⨉10% = 82%

3. If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. '''However, the previous review score that was given in the current round will count until the reviewer updates the submitted review'''. '''This functionality doesn’t work for multipart rubrics.''' 

The system also doesn’t generate emails to reviewers in case a new submission is made by the author without changing submission parameters like submission URL. Eg: A new submission is made by making a new deployment. But the deployment URL doesn’t change. In this case as well, without changing the current flow the author should have an option to notify all the reviewers to revisit the submission.

=====Interface Requirement=====
The overall user interface would remain the same as of the current system. Since this is a modification to the current submission page, there will be a new addition to the submission page. On the submission page, there will be a new button to notify reviewers when there is a submission which doesn’t contain a change to submission parameters like submission URL, but the deployment has been updated on the same URL. Other than that the interface remains the same for the score page, only functionalities will be tested for multi-part rubrics.

=====Quality Requirements=====
The interface changes are minimal and very easy to adapt. The new button will be clearly visible and is supposed to have a clear text to interpret its functionality. The proposed changes should not affect the current system functionality in any way. The solution is only intended to enrich the project features.

=====Performance Requirements=====
Adding test for multi-part rubric functionality should not cause any overhead in the performance of the system. Sending emails through “Notify All Reviewers” button should affect the system minimally in terms of performance. The only overhead is a database query to get all the reviewers. A good database design should incorporate such a query smoothly.

=====Assumptions=====
The functionality should work in case the assignment rubric is not multi-part.

== '''Proposed Solution''' ==
=====Zero-weighted scores=====
If only zero-weighted scores are available in the first round of review, the average of all zero-weighted scores will be shown in a gray font. An information button will explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade”)

Inside compute_total_score method in on_the_fly_calc.rb module, we'll check if the weighted score exists, if not then we'll calculate the non weighted score. We'll also have to add a method in the questionnaire model to get non weighted score. Few changes are required for the UI to display non-weighted score in grey with information icon and non weighted score in black.

Possible files to be modified.

models/on_the_fly_calc.rb 
models/questionnaire.rb 
models/view_team.html.erb

===== Non-zero weighted scores=====
If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is 80%⨉90% + 100%⨉10% = 82%

The score calculation was initially tested manually to verify the scenarios with multiple weights and rounds.
Also, existing spec for score calculation with a weighted average for both varying and non-varying rubric is covering this scenario.
The spec is written under /spec/models/assignment_spec.rb. The "score" function of the "model/assignment.rb" is called for online (on the fly score calculation).

Following file has the spec for weighted average score calculation.
spec/models/assignment_spec.rb 

=====Review Update=====
If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review.

 This feature can be broken down into the following sub-cases:
 1. A submission can only be made before deadline
 2. A review can edit a review only before deadline
 3. Score calculation should be only done for latest submitted review

Subcases:

 
1. Spec to ensure submission can't be made after the deadline

File : spec/features/post_deadline_review_submission_spec.rb

'''
it 'should not allow submission after deadline' do
user = User.find_by(name: "student2065")
stub_current_user(user, user.role.name, user.role)
# goto student_task page, which has link to "Your work"
visit 'student_task/view?id=1'
# the page will have text "Your work" but will be grayed
expect(page).to have_content "Your work"
# the page will not have link to content "Your work"
expect{click_link "Your work"}.to raise_error(Capybara::ElementNotFound)
end
'''

2. Spec to ensure review can be done only before deadline

File : spec/features/post_deadline_review_submission_spec.rb

'''
it "should not be able to review work after deadline" do
# The spec is written to reproduce following bug. "Others' work" link open after deadline passed
user = User.find_by(name: "student2065")
stub_current_user(user, user.role.name, user.role)
visit 'student_task/view?id=1'
# the page should have content, but after deadline passes it is displayed as gray
# but there should not be any link attached to it
expect(page).to have_content "Others' work"
# this is the bug, even after deadline has passed, the link is still present
# the ui comment in file views/student_task/view.html.erb says
# 
# But the link seems to be open even after deadline passed.
# Screenshot attached as part of wiki for E1975, Fall 2019
expect(page).to have_link("Others' work", "/student_review/list?id=1")
end

'''

The test scenario is able to produce the following bug. At times the review link is open even after the deadline has passed. The bug is also shown/reproduced in the following screenshot for Program 1. 

[[File:ScreenShotReview.jpg]]

3. Score calculation

Score calculation for assignments for review is part of models/assignment.rb. The score calculation is handled by function "score" in this file and the testing is done for both multipart varying and non-varying rubric.
The respective spec is written in file spec/models/assignment_spec.rb.

'''

describe '#scores' do
context 'when assignment is varying rubric by round assignment' do
it 'calculates scores in each round of each team in current assignment' do
allow(participant).to receive(:scores).with(review1: [question]).and_return(98)
allow(assignment).to receive(:varying_rubrics_by_round?).and_return(true)
allow(assignment).to receive(:num_review_rounds).and_return(1)
allow(ReviewResponseMap).to receive(:get_responses_for_team_round).with(team, 1).and_return([response])
allow(Answer).to receive(:compute_scores).with([response], [question]).and_return(max: 95, min: 88, avg: 90)
expect(assignment.scores(review1: [question]).inspect).to eq("{:participants=>{:\"1\"=>98}, :teams=>{:\"0\"=>{:team=>#<AssignmentTeam id: 1, "\
"name: \"no team\", parent_id: 1, type: \"AssignmentTeam\", comments_for_advertisement: nil, advertise_for_partner: nil, "\
"submitted_hyperlinks: \"---\\n- https://www.expertiza.ncsu.edu\", directory_num: 0, grade_for_submission: nil, "\
"comment_for_submission: nil>, :scores=>{:max=>95, :min=>88, :avg=>90.0}}}}")
end
end

context 'when assignment is not varying rubric by round assignment' do
it 'calculates scores of each team in current assignment' do
allow(participant).to receive(:scores).with(review: [question]).and_return(98)
allow(assignment).to receive(:varying_rubrics_by_round?).and_return(false)
allow(ReviewResponseMap).to receive(:get_assessments_for).with(team).and_return([response])
allow(Answer).to receive(:compute_scores).with([response], [question]).and_return(max: 95, min: 88, avg: 90)
expect(assignment.scores(review: [question]).inspect).to eq("{:participants=>{:\"1\"=>98}, :teams=>{:\"0\"=>{:team=>#<AssignmentTeam id: 1, "\
"name: \"no team\", parent_id: 1, type: \"AssignmentTeam\", comments_for_advertisement: nil, advertise_for_partner: nil, "\
"submitted_hyperlinks: \"---\\n- https://www.expertiza.ncsu.edu\", directory_num: 0, grade_for_submission: nil, "\
"comment_for_submission: nil>, :scores=>{:max=>95, :min=>88, :avg=>90}}}}")
end
end
end

'''

 
[[File:Usecasenew.png]]

=='''Test Plan'''==

=====Rspec=====

New test cases will be added to test and validate the functionalities which are not working as expected. Also, the test cases for features that will be newly added to the system, will also be written.
Test cases will be added to these files:

spec/features/post_deadline_review_submission_sepc.rb 
spec/models/assignment_participant_spec.rb 

The scenarios covered by the newly introduced test cases are as follows: 
1. Zero weighted scores with gray mark: rspec spec/model/assignment_participants.rb 
2. Non-zero weighted scores 
3. Review update: rspec spec/features/post_deadline_review_submission_sepc.rb 

=====Manual Testing=====

'''Step 1'''. The instructor will create an assignment and assign the zero and non-zero weights to the rubrics. 
'''Step 2'''. The author submits the work done in expertiza. 
'''Step 3'''. Peers check the submitted work and give reviews for it. 

'''Test 1: Zero weighted scores shown in gray.''' 
Login as an author in expertiza and go to scores page to view the feedback.
Zero weighted scores should be displayed in gray to the logged in user.

'''Test 2: Information button.''' 
Login as an author in expertiza and go to scores page to view the feedback.
The information button should explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade.”)

'''Step 4'''. The mentor evaluates the final submission and give grades.

'''Test 3: Display non-zero weighted scores''' 
Login as an author in expertiza and go to scores page to view the feedback.
The scores shown for the assignment should be the weighted average of the scores.
The zero weighted scores have to be replaced with non-zero weighted scores.

'''Step 5''': The author updates the work and re-submits it again.

'''Test 4: Review Update''' 
The Review should be reopened. The reviewer should be able to go and update the submitted review.
If the review is not updated the previous review score has to be retained.

CSC/ECE 517 Fall 2019 - E1975. Generalize Review Versioning

2019-12-17T02:11:50Z

Ajain37: /* Review Update */

__TOC__

== '''Introduction''' ==
Expertiza is a peer-review based learning platform in which work-products by students are evaluated and scored by peer-review of other students. Review process occurs in stages. After submission of initial feedback for a work-product by the peer-group, students are allowed time to incorporate the comments and improve their work. This second version of the work product is reviewed again and scores are given. This process might be repeated again and the average score from the last review stage is considered as the final score for that work-product. This incremental development of work-products and progressive learning is the fundamental concept underlying the Expertiza system.

=====Purpose=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The system also triggers an email to reviewer when there is a change in the submission. The scores are calculated on the fly based on the rubrics.

=====Scope=====
This project will enhance the existing scoring and peer review functionality and the underlying implementation along with the testing of the existing implementation. Testing of the score calculation, display for the weighted and non-weighted scores based on the combination of the rubrics are part of this project. This project also includes an additional feature for the author to notify the reviewers for the change in the submission.

=====Background=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The scores are calculated based on the rubrics defined in the system. There’s already an existing implementation for the computation of score but it doesn’t work for multipart rubric

=====Standards=====
The new standards that will be adhered to in this system are those given for the rails framework, and the code should follow rails and object oriented design principles. Any newly added code will adhere to the guidelines mentioned below:

● https://github.com/rubocop-hq/rails-style-guide 
● http://wiki.expertiza.ncsu.edu/index.php/Object-Oriented_Design_and_Programming.

== '''Problem Statement''' ==
=====Problem=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The functionality needs to be tested. It works in some cases, but we are suspicious that it is not entirely correct.

'''Profile Email Preference''':
User in expertiza has liberty to control what kind of emails they should receive. As the reviewers will receive an email for the change in submission. This can be controlled by selecting an option (When someone else submits work I am assigned to review) on the profile page of the user. As the checkboxes are far away on the right corner of the screen, the position of checkboxes need to be changed to make it more user friendly.
 

Scores for students in Expertiza can be based on a combination of rubrics: review rubrics for each round of review, author feedback, and teammate review. In CSC/ECE 517, we don’t really use the student-assigned scores to grade projects, but the peer-review score shown to the student is based entirely on the second round of review 

Here is how scores are calculated: 

1. If only zero-weighted scores are available (here, in the first round of review), the average of all zero-weighted scores should be shown in grey font. An information button should explain why the scores are shown in grey (“The reviews submitted so far do not count toward the author’s final grade.”): 

2. If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is:: 
80%⨉90% + 100%⨉10% = 82%

3. If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. '''However, the previous review score that was given in the current round will count until the reviewer updates the submitted review'''. '''This functionality doesn’t work for multipart rubrics.''' 

The system also doesn’t generate emails to reviewers in case a new submission is made by the author without changing submission parameters like submission URL. Eg: A new submission is made by making a new deployment. But the deployment URL doesn’t change. In this case as well, without changing the current flow the author should have an option to notify all the reviewers to revisit the submission.

=====Interface Requirement=====
The overall user interface would remain the same as of the current system. Since this is a modification to the current submission page, there will be a new addition to the submission page. On the submission page, there will be a new button to notify reviewers when there is a submission which doesn’t contain a change to submission parameters like submission URL, but the deployment has been updated on the same URL. Other than that the interface remains the same for the score page, only functionalities will be tested for multi-part rubrics.

=====Quality Requirements=====
The interface changes are minimal and very easy to adapt. The new button will be clearly visible and is supposed to have a clear text to interpret its functionality. The proposed changes should not affect the current system functionality in any way. The solution is only intended to enrich the project features.

=====Performance Requirements=====
Adding test for multi-part rubric functionality should not cause any overhead in the performance of the system. Sending emails through “Notify All Reviewers” button should affect the system minimally in terms of performance. The only overhead is a database query to get all the reviewers. A good database design should incorporate such a query smoothly.

=====Assumptions=====
The functionality should work in case the assignment rubric is not multi-part.

== '''Proposed Solution''' ==
=====Zero-weighted scores=====
If only zero-weighted scores are available in the first round of review, the average of all zero-weighted scores will be shown in a gray font. An information button will explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade”)

Inside compute_total_score method in on_the_fly_calc.rb module, we'll check if the weighted score exists, if not then we'll calculate the non weighted score. We'll also have to add a method in the questionnaire model to get non weighted score. Few changes are required for the UI to display non-weighted score in grey with information icon and non weighted score in black.

Possible files to be modified.

models/on_the_fly_calc.rb 
models/questionnaire.rb 
models/view_team.html.erb

===== Non-zero weighted scores=====
If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is 80%⨉90% + 100%⨉10% = 82%

The score calculation was initially tested manually to verify the scenarios with multiple weights and rounds.
Also, existing spec for score calculation with a weighted average for both varying and non-varying rubric is covering this scenario.
The spec is written under /spec/models/assignment_spec.rb. The "score" function of the "model/assignment.rb" is called for online (on the fly score calculation).

Following file has the spec for weighted average score calculation.
spec/models/assignment_spec.rb 

=====Review Update=====
If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review.

 This feature can be broken down into the following sub-cases:
 1. A submission can only be made before deadline
 2. A review can edit a review only before deadline
 3. Score calculation should be only done for latest submitted review

Subcases:

 
1. Spec to ensure submission can't be made after the deadline

File : spec/features/post_deadline_review_submission_spec.rb

'''
it 'should not allow submission after deadline' do
user = User.find_by(name: "student2065")
stub_current_user(user, user.role.name, user.role)
# goto student_task page, which has link to "Your work"
visit 'student_task/view?id=1'
# the page will have text "Your work" but will be grayed
expect(page).to have_content "Your work"
# the page will not have link to content "Your work"
expect{click_link "Your work"}.to raise_error(Capybara::ElementNotFound)
end
'''

2. Spec to ensure review can be done only before deadline

File : spec/features/post_deadline_review_submission_spec.rb

'''
it "should not be able to review work after deadline" do
# The spec is written to reproduce following bug. "Others' work" link open after deadline passed
user = User.find_by(name: "student2065")
stub_current_user(user, user.role.name, user.role)
visit 'student_task/view?id=1'
# the page should have content, but after deadline passes it is displayed as gray
# but there should not be any link attached to it
expect(page).to have_content "Others' work"
# this is the bug, even after deadline has passed, the link is still present
# the ui comment in file views/student_task/view.html.erb says
# 
# But the link seems to be open even after deadline passed.
# Screenshot attached as part of wiki for E1975, Fall 2019
expect(page).to have_link("Others' work", "/student_review/list?id=1")
end

'''

The test scenario is able to produce the following bug. At times the review link is open even after the deadline has passed. The bug is also shown/reproduced in the following screenshot for Program 1. 

[[File:ScreenShotReview.jpg]]

3. Score calculation

Score calculation for assignments for review is part of models/assignment.rb. The score calculation is handled by function "score" in this file and the testing is done for both multipart varying and non-varying rubric.
The respective spec is written in file spec/models/assignment_spec.rb.

'''

describe '#scores' do
context 'when assignment is varying rubric by round assignment' do
it 'calculates scores in each round of each team in current assignment' do
allow(participant).to receive(:scores).with(review1: [question]).and_return(98)
allow(assignment).to receive(:varying_rubrics_by_round?).and_return(true)
allow(assignment).to receive(:num_review_rounds).and_return(1)
allow(ReviewResponseMap).to receive(:get_responses_for_team_round).with(team, 1).and_return([response])
allow(Answer).to receive(:compute_scores).with([response], [question]).and_return(max: 95, min: 88, avg: 90)
expect(assignment.scores(review1: [question]).inspect).to eq("{:participants=>{:\"1\"=>98}, :teams=>{:\"0\"=>{:team=>#<AssignmentTeam id: 1, "\
"name: \"no team\", parent_id: 1, type: \"AssignmentTeam\", comments_for_advertisement: nil, advertise_for_partner: nil, "\
"submitted_hyperlinks: \"---\\n- https://www.expertiza.ncsu.edu\", directory_num: 0, grade_for_submission: nil, "\
"comment_for_submission: nil>, :scores=>{:max=>95, :min=>88, :avg=>90.0}}}}")
end
end

context 'when assignment is not varying rubric by round assignment' do
it 'calculates scores of each team in current assignment' do
allow(participant).to receive(:scores).with(review: [question]).and_return(98)
allow(assignment).to receive(:varying_rubrics_by_round?).and_return(false)
allow(ReviewResponseMap).to receive(:get_assessments_for).with(team).and_return([response])
allow(Answer).to receive(:compute_scores).with([response], [question]).and_return(max: 95, min: 88, avg: 90)
expect(assignment.scores(review: [question]).inspect).to eq("{:participants=>{:\"1\"=>98}, :teams=>{:\"0\"=>{:team=>#<AssignmentTeam id: 1, "\
"name: \"no team\", parent_id: 1, type: \"AssignmentTeam\", comments_for_advertisement: nil, advertise_for_partner: nil, "\
"submitted_hyperlinks: \"---\\n- https://www.expertiza.ncsu.edu\", directory_num: 0, grade_for_submission: nil, "\
"comment_for_submission: nil>, :scores=>{:max=>95, :min=>88, :avg=>90}}}}")
end
end
end

'''

 
[[File:Usecasenew.png]]

=='''Test Plan'''==

=====Rspec=====

New test cases will be added to test and validate the functionalities which are not working as expected. Also, the test cases for features that will be newly added to the system, will also be written.
Test cases will be added to these files:

spec/features/post_deadline_review_submission_sepc.rb 
spec/models/assignment_participant_spec.rb 

The scenarios covered by the newly introduced test cases are as follows: 
1. Zero weighted scores with gray mark: rspec spec/model/assignment_participants.rb 
2. Non-zero weighted scores 
3. Review update: rspec spec/features/post_deadline_review_submission_sepc.rb 

=====Manual Testing=====

'''Step 1'''. The instructor will create an assignment and assign the zero and non-zero weights to the rubrics. 
'''Step 2'''. The author submits the work done in expertiza. 
'''Step 3'''. Peers check the submitted work and give reviews for it. 

'''Test 1: Zero weighted scores shown in gray.''' 
Login as an author in expertiza and go to scores page to view the feedback.
Zero weighted scores should be displayed in gray to the logged in user.

'''Test 2: Information button.''' 
Login as an author in expertiza and go to scores page to view the feedback.
The information button should explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade.”)

'''Step 4'''. The mentor evaluates the final submission and give grades.

'''Test 3: Display non-zero weighted scores''' 
Login as an author in expertiza and go to scores page to view the feedback.
The scores shown for the assignment should be the weighted average of the scores.
The zero weighted scores have to be replaced with non-zero weighted scores.

'''Step 5''': The author updates the work and re-submits it again.

'''Test 4: Review Update''' 
The Review should be reopened. The reviewer should be able to go and update the submitted review.
If the review is not updated the previous review score has to be retained.

CSC/ECE 517 Fall 2019 - E1975. Generalize Review Versioning

2019-12-17T02:09:17Z

Ajain37: /* Review Update */

__TOC__

== '''Introduction''' ==
Expertiza is a peer-review based learning platform in which work-products by students are evaluated and scored by peer-review of other students. Review process occurs in stages. After submission of initial feedback for a work-product by the peer-group, students are allowed time to incorporate the comments and improve their work. This second version of the work product is reviewed again and scores are given. This process might be repeated again and the average score from the last review stage is considered as the final score for that work-product. This incremental development of work-products and progressive learning is the fundamental concept underlying the Expertiza system.

=====Purpose=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The system also triggers an email to reviewer when there is a change in the submission. The scores are calculated on the fly based on the rubrics.

=====Scope=====
This project will enhance the existing scoring and peer review functionality and the underlying implementation along with the testing of the existing implementation. Testing of the score calculation, display for the weighted and non-weighted scores based on the combination of the rubrics are part of this project. This project also includes an additional feature for the author to notify the reviewers for the change in the submission.

=====Background=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The scores are calculated based on the rubrics defined in the system. There’s already an existing implementation for the computation of score but it doesn’t work for multipart rubric

=====Standards=====
The new standards that will be adhered to in this system are those given for the rails framework, and the code should follow rails and object oriented design principles. Any newly added code will adhere to the guidelines mentioned below:

● https://github.com/rubocop-hq/rails-style-guide 
● http://wiki.expertiza.ncsu.edu/index.php/Object-Oriented_Design_and_Programming.

== '''Problem Statement''' ==
=====Problem=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The functionality needs to be tested. It works in some cases, but we are suspicious that it is not entirely correct.

'''Profile Email Preference''':
User in expertiza has liberty to control what kind of emails they should receive. As the reviewers will receive an email for the change in submission. This can be controlled by selecting an option (When someone else submits work I am assigned to review) on the profile page of the user. As the checkboxes are far away on the right corner of the screen, the position of checkboxes need to be changed to make it more user friendly.
 

Scores for students in Expertiza can be based on a combination of rubrics: review rubrics for each round of review, author feedback, and teammate review. In CSC/ECE 517, we don’t really use the student-assigned scores to grade projects, but the peer-review score shown to the student is based entirely on the second round of review 

Here is how scores are calculated: 

1. If only zero-weighted scores are available (here, in the first round of review), the average of all zero-weighted scores should be shown in grey font. An information button should explain why the scores are shown in grey (“The reviews submitted so far do not count toward the author’s final grade.”): 

2. If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is:: 
80%⨉90% + 100%⨉10% = 82%

3. If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. '''However, the previous review score that was given in the current round will count until the reviewer updates the submitted review'''. '''This functionality doesn’t work for multipart rubrics.''' 

The system also doesn’t generate emails to reviewers in case a new submission is made by the author without changing submission parameters like submission URL. Eg: A new submission is made by making a new deployment. But the deployment URL doesn’t change. In this case as well, without changing the current flow the author should have an option to notify all the reviewers to revisit the submission.

=====Interface Requirement=====
The overall user interface would remain the same as of the current system. Since this is a modification to the current submission page, there will be a new addition to the submission page. On the submission page, there will be a new button to notify reviewers when there is a submission which doesn’t contain a change to submission parameters like submission URL, but the deployment has been updated on the same URL. Other than that the interface remains the same for the score page, only functionalities will be tested for multi-part rubrics.

=====Quality Requirements=====
The interface changes are minimal and very easy to adapt. The new button will be clearly visible and is supposed to have a clear text to interpret its functionality. The proposed changes should not affect the current system functionality in any way. The solution is only intended to enrich the project features.

=====Performance Requirements=====
Adding test for multi-part rubric functionality should not cause any overhead in the performance of the system. Sending emails through “Notify All Reviewers” button should affect the system minimally in terms of performance. The only overhead is a database query to get all the reviewers. A good database design should incorporate such a query smoothly.

=====Assumptions=====
The functionality should work in case the assignment rubric is not multi-part.

== '''Proposed Solution''' ==
=====Zero-weighted scores=====
If only zero-weighted scores are available in the first round of review, the average of all zero-weighted scores will be shown in a gray font. An information button will explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade”)

Inside compute_total_score method in on_the_fly_calc.rb module, we'll check if the weighted score exists, if not then we'll calculate the non weighted score. We'll also have to add a method in the questionnaire model to get non weighted score. Few changes are required for the UI to display non-weighted score in grey with information icon and non weighted score in black.

Possible files to be modified.

models/on_the_fly_calc.rb 
models/questionnaire.rb 
models/view_team.html.erb

===== Non-zero weighted scores=====
If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is 80%⨉90% + 100%⨉10% = 82%

The score calculation was initially tested manually to verify the scenarios with multiple weights and rounds.
Also, existing spec for score calculation with a weighted average for both varying and non-varying rubric is covering this scenario.
The spec is written under /spec/models/assignment_spec.rb. The "score" function of the "model/assignment.rb" is called for online (on the fly score calculation).

Following file has the spec for weighted average score calculation.
spec/models/assignment_spec.rb 

=====Review Update=====
If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review.

 This feature can be broken down into the following sub-cases:
 1. A submission can only be made before deadline
 2. A review can edit a review only before deadline
 3. Score calculation should be only done for latest submitted review

Subcases:

 
1. Spec to ensure submission can't be made after the deadline

File : spec/features/post_deadline_review_submission_spec.rb

'''
it 'should not allow submission after deadline' do
user = User.find_by(name: "student2065")
stub_current_user(user, user.role.name, user.role)
# goto student_task page, which has link to "Your work"
visit 'student_task/view?id=1'
# the page will have text "Your work" but will be grayed
expect(page).to have_content "Your work"
# the page will not have link to content "Your work"
expect{click_link "Your work"}.to raise_error(Capybara::ElementNotFound)
end
'''

2. Spec to ensure review can be done only before deadline

File : spec/features/post_deadline_review_submission_spec.rb

'''
it "should not be able to review work after deadline" do
# The spec is written to reproduce following bug. "Others' work" link open after deadline passed
user = User.find_by(name: "student2065")
stub_current_user(user, user.role.name, user.role)
visit 'student_task/view?id=1'
# the page should have content, but after deadline passes it is displayed as gray
# but there should not be any link attached to it
expect(page).to have_content "Others' work"
# this is the bug, even after deadline has passed, the link is still present
# the ui comment in file views/student_task/view.html.erb says
# 
# But the link seems to be open even after deadline passed.
# Screenshot attached as part of wiki for E1975, Fall 2019
expect(page).to have_link("Others' work", "/student_review/list?id=1")
end

'''

The test scenario is able to produce the following bug. At times the review link is open even after the deadline has passed. The bug is also shown/reproduced in the following screenshot for Program 1. 

[[File:ScreenShotReview.jpg]]

3. Score calculation

Score calculation for assignments for review is part of models/assignment.rb. The score calculation is handled by function "score" in this file and the testing is done for both multipart varying and non-varying rubric.
The respective spec is written in file spec/models/assignment_spec.rb.

'''

describe '#scores' do
context 'when assignment is varying rubric by round assignment' do
it 'calculates scores in each round of each team in current assignment' do
allow(participant).to receive(:scores).with(review1: [question]).and_return(98)
allow(assignment).to receive(:varying_rubrics_by_round?).and_return(true)
allow(assignment).to receive(:num_review_rounds).and_return(1)
allow(ReviewResponseMap).to receive(:get_responses_for_team_round).with(team, 1).and_return([response])
allow(Answer).to receive(:compute_scores).with([response], [question]).and_return(max: 95, min: 88, avg: 90)
expect(assignment.scores(review1: [question]).inspect).to eq("{:participants=>{:\"1\"=>98}, :teams=>{:\"0\"=>{:team=>#<AssignmentTeam id: 1, "\
"name: \"no team\", parent_id: 1, type: \"AssignmentTeam\", comments_for_advertisement: nil, advertise_for_partner: nil, "\
"submitted_hyperlinks: \"---\\n- https://www.expertiza.ncsu.edu\", directory_num: 0, grade_for_submission: nil, "\
"comment_for_submission: nil>, :scores=>{:max=>95, :min=>88, :avg=>90.0}}}}")
end
end

context 'when assignment is not varying rubric by round assignment' do
it 'calculates scores of each team in current assignment' do
allow(participant).to receive(:scores).with(review: [question]).and_return(98)
allow(assignment).to receive(:varying_rubrics_by_round?).and_return(false)
allow(ReviewResponseMap).to receive(:get_assessments_for).with(team).and_return([response])
allow(Answer).to receive(:compute_scores).with([response], [question]).and_return(max: 95, min: 88, avg: 90)
expect(assignment.scores(review: [question]).inspect).to eq("{:participants=>{:\"1\"=>98}, :teams=>{:\"0\"=>{:team=>#<AssignmentTeam id: 1, "\
"name: \"no team\", parent_id: 1, type: \"AssignmentTeam\", comments_for_advertisement: nil, advertise_for_partner: nil, "\
"submitted_hyperlinks: \"---\\n- https://www.expertiza.ncsu.edu\", directory_num: 0, grade_for_submission: nil, "\
"comment_for_submission: nil>, :scores=>{:max=>95, :min=>88, :avg=>90}}}}")
end
end
end

'''

[[File:Generalize_review.jpg]]

 
[[File:Usecasenew.png]]

=='''Test Plan'''==

=====Rspec=====

New test cases will be added to test and validate the functionalities which are not working as expected. Also, the test cases for features that will be newly added to the system, will also be written.
Test cases will be added to these files:

spec/features/post_deadline_review_submission_sepc.rb 
spec/models/assignment_participant_spec.rb 

The scenarios covered by the newly introduced test cases are as follows: 
1. Zero weighted scores with gray mark: rspec spec/model/assignment_participants.rb 
2. Non-zero weighted scores 
3. Review update: rspec spec/features/post_deadline_review_submission_sepc.rb 

=====Manual Testing=====

'''Step 1'''. The instructor will create an assignment and assign the zero and non-zero weights to the rubrics. 
'''Step 2'''. The author submits the work done in expertiza. 
'''Step 3'''. Peers check the submitted work and give reviews for it. 

'''Test 1: Zero weighted scores shown in gray.''' 
Login as an author in expertiza and go to scores page to view the feedback.
Zero weighted scores should be displayed in gray to the logged in user.

'''Test 2: Information button.''' 
Login as an author in expertiza and go to scores page to view the feedback.
The information button should explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade.”)

'''Step 4'''. The mentor evaluates the final submission and give grades.

'''Test 3: Display non-zero weighted scores''' 
Login as an author in expertiza and go to scores page to view the feedback.
The scores shown for the assignment should be the weighted average of the scores.
The zero weighted scores have to be replaced with non-zero weighted scores.

'''Step 5''': The author updates the work and re-submits it again.

'''Test 4: Review Update''' 
The Review should be reopened. The reviewer should be able to go and update the submitted review.
If the review is not updated the previous review score has to be retained.

CSC/ECE 517 Fall 2019 - E1975. Generalize Review Versioning

2019-12-17T02:08:58Z

Ajain37: /* Review Update */

__TOC__

== '''Introduction''' ==
Expertiza is a peer-review based learning platform in which work-products by students are evaluated and scored by peer-review of other students. Review process occurs in stages. After submission of initial feedback for a work-product by the peer-group, students are allowed time to incorporate the comments and improve their work. This second version of the work product is reviewed again and scores are given. This process might be repeated again and the average score from the last review stage is considered as the final score for that work-product. This incremental development of work-products and progressive learning is the fundamental concept underlying the Expertiza system.

=====Purpose=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The system also triggers an email to reviewer when there is a change in the submission. The scores are calculated on the fly based on the rubrics.

=====Scope=====
This project will enhance the existing scoring and peer review functionality and the underlying implementation along with the testing of the existing implementation. Testing of the score calculation, display for the weighted and non-weighted scores based on the combination of the rubrics are part of this project. This project also includes an additional feature for the author to notify the reviewers for the change in the submission.

=====Background=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The scores are calculated based on the rubrics defined in the system. There’s already an existing implementation for the computation of score but it doesn’t work for multipart rubric

=====Standards=====
The new standards that will be adhered to in this system are those given for the rails framework, and the code should follow rails and object oriented design principles. Any newly added code will adhere to the guidelines mentioned below:

● https://github.com/rubocop-hq/rails-style-guide 
● http://wiki.expertiza.ncsu.edu/index.php/Object-Oriented_Design_and_Programming.

== '''Problem Statement''' ==
=====Problem=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The functionality needs to be tested. It works in some cases, but we are suspicious that it is not entirely correct.

'''Profile Email Preference''':
User in expertiza has liberty to control what kind of emails they should receive. As the reviewers will receive an email for the change in submission. This can be controlled by selecting an option (When someone else submits work I am assigned to review) on the profile page of the user. As the checkboxes are far away on the right corner of the screen, the position of checkboxes need to be changed to make it more user friendly.
 

Scores for students in Expertiza can be based on a combination of rubrics: review rubrics for each round of review, author feedback, and teammate review. In CSC/ECE 517, we don’t really use the student-assigned scores to grade projects, but the peer-review score shown to the student is based entirely on the second round of review 

Here is how scores are calculated: 

1. If only zero-weighted scores are available (here, in the first round of review), the average of all zero-weighted scores should be shown in grey font. An information button should explain why the scores are shown in grey (“The reviews submitted so far do not count toward the author’s final grade.”): 

2. If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is:: 
80%⨉90% + 100%⨉10% = 82%

3. If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. '''However, the previous review score that was given in the current round will count until the reviewer updates the submitted review'''. '''This functionality doesn’t work for multipart rubrics.''' 

The system also doesn’t generate emails to reviewers in case a new submission is made by the author without changing submission parameters like submission URL. Eg: A new submission is made by making a new deployment. But the deployment URL doesn’t change. In this case as well, without changing the current flow the author should have an option to notify all the reviewers to revisit the submission.

=====Interface Requirement=====
The overall user interface would remain the same as of the current system. Since this is a modification to the current submission page, there will be a new addition to the submission page. On the submission page, there will be a new button to notify reviewers when there is a submission which doesn’t contain a change to submission parameters like submission URL, but the deployment has been updated on the same URL. Other than that the interface remains the same for the score page, only functionalities will be tested for multi-part rubrics.

=====Quality Requirements=====
The interface changes are minimal and very easy to adapt. The new button will be clearly visible and is supposed to have a clear text to interpret its functionality. The proposed changes should not affect the current system functionality in any way. The solution is only intended to enrich the project features.

=====Performance Requirements=====
Adding test for multi-part rubric functionality should not cause any overhead in the performance of the system. Sending emails through “Notify All Reviewers” button should affect the system minimally in terms of performance. The only overhead is a database query to get all the reviewers. A good database design should incorporate such a query smoothly.

=====Assumptions=====
The functionality should work in case the assignment rubric is not multi-part.

== '''Proposed Solution''' ==
=====Zero-weighted scores=====
If only zero-weighted scores are available in the first round of review, the average of all zero-weighted scores will be shown in a gray font. An information button will explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade”)

Inside compute_total_score method in on_the_fly_calc.rb module, we'll check if the weighted score exists, if not then we'll calculate the non weighted score. We'll also have to add a method in the questionnaire model to get non weighted score. Few changes are required for the UI to display non-weighted score in grey with information icon and non weighted score in black.

Possible files to be modified.

models/on_the_fly_calc.rb 
models/questionnaire.rb 
models/view_team.html.erb

===== Non-zero weighted scores=====
If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is 80%⨉90% + 100%⨉10% = 82%

The score calculation was initially tested manually to verify the scenarios with multiple weights and rounds.
Also, existing spec for score calculation with a weighted average for both varying and non-varying rubric is covering this scenario.
The spec is written under /spec/models/assignment_spec.rb. The "score" function of the "model/assignment.rb" is called for online (on the fly score calculation).

Following file has the spec for weighted average score calculation.
spec/models/assignment_spec.rb 

=====Review Update=====
If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review.

 This feature can be broken down into the following sub-cases:
 1. A submission can only be made before deadline
 2. A review can edit a review only before deadline
 3. Score calculation should be only done for latest submitted review

Subcases:

 
1. Spec to ensure submission can't be made after the deadline

File : spec/features/post_deadline_review_submission_spec.rb

'''
it 'should not allow submission after deadline' do
user = User.find_by(name: "student2065")
stub_current_user(user, user.role.name, user.role)
# goto student_task page, which has link to "Your work"
visit 'student_task/view?id=1'
# the page will have text "Your work" but will be grayed
expect(page).to have_content "Your work"
# the page will not have link to content "Your work"
expect{click_link "Your work"}.to raise_error(Capybara::ElementNotFound)
end
'''

2. Spec to ensure review can be done only before deadline

File : spec/features/post_deadline_review_submission_spec.rb

'''
it "should not be able to review work after deadline" do
# The spec is written to reproduce following bug. "Others' work" link open after deadline passed
user = User.find_by(name: "student2065")
stub_current_user(user, user.role.name, user.role)
visit 'student_task/view?id=1'
# the page should have content, but after deadline passes it is displayed as gray
# but there should not be any link attached to it
expect(page).to have_content "Others' work"
# this is the bug, even after deadline has passed, the link is still present
# the ui comment in file views/student_task/view.html.erb says
# 
# But the link seems to be open even after deadline passed.
# Screenshot attached as part of wiki for E1975, Fall 2019
expect(page).to have_link("Others' work", "/student_review/list?id=1")

end
'''

The test scenario is able to produce the following bug. At times the review link is open even after the deadline has passed. The bug is also shown/reproduced in the following screenshot for Program 1. 

[[File:ScreenShotReview.jpg]]

3. Score calculation

Score calculation for assignments for review is part of models/assignment.rb. The score calculation is handled by function "score" in this file and the testing is done for both multipart varying and non-varying rubric.
The respective spec is written in file spec/models/assignment_spec.rb.

'''

describe '#scores' do
context 'when assignment is varying rubric by round assignment' do
it 'calculates scores in each round of each team in current assignment' do
allow(participant).to receive(:scores).with(review1: [question]).and_return(98)
allow(assignment).to receive(:varying_rubrics_by_round?).and_return(true)
allow(assignment).to receive(:num_review_rounds).and_return(1)
allow(ReviewResponseMap).to receive(:get_responses_for_team_round).with(team, 1).and_return([response])
allow(Answer).to receive(:compute_scores).with([response], [question]).and_return(max: 95, min: 88, avg: 90)
expect(assignment.scores(review1: [question]).inspect).to eq("{:participants=>{:\"1\"=>98}, :teams=>{:\"0\"=>{:team=>#<AssignmentTeam id: 1, "\
"name: \"no team\", parent_id: 1, type: \"AssignmentTeam\", comments_for_advertisement: nil, advertise_for_partner: nil, "\
"submitted_hyperlinks: \"---\\n- https://www.expertiza.ncsu.edu\", directory_num: 0, grade_for_submission: nil, "\
"comment_for_submission: nil>, :scores=>{:max=>95, :min=>88, :avg=>90.0}}}}")
end
end

context 'when assignment is not varying rubric by round assignment' do
it 'calculates scores of each team in current assignment' do
allow(participant).to receive(:scores).with(review: [question]).and_return(98)
allow(assignment).to receive(:varying_rubrics_by_round?).and_return(false)
allow(ReviewResponseMap).to receive(:get_assessments_for).with(team).and_return([response])
allow(Answer).to receive(:compute_scores).with([response], [question]).and_return(max: 95, min: 88, avg: 90)
expect(assignment.scores(review: [question]).inspect).to eq("{:participants=>{:\"1\"=>98}, :teams=>{:\"0\"=>{:team=>#<AssignmentTeam id: 1, "\
"name: \"no team\", parent_id: 1, type: \"AssignmentTeam\", comments_for_advertisement: nil, advertise_for_partner: nil, "\
"submitted_hyperlinks: \"---\\n- https://www.expertiza.ncsu.edu\", directory_num: 0, grade_for_submission: nil, "\
"comment_for_submission: nil>, :scores=>{:max=>95, :min=>88, :avg=>90}}}}")
end
end
end

'''

[[File:Generalize_review.jpg]]

 
[[File:Usecasenew.png]]

=='''Test Plan'''==

=====Rspec=====

New test cases will be added to test and validate the functionalities which are not working as expected. Also, the test cases for features that will be newly added to the system, will also be written.
Test cases will be added to these files:

spec/features/post_deadline_review_submission_sepc.rb 
spec/models/assignment_participant_spec.rb 

The scenarios covered by the newly introduced test cases are as follows: 
1. Zero weighted scores with gray mark: rspec spec/model/assignment_participants.rb 
2. Non-zero weighted scores 
3. Review update: rspec spec/features/post_deadline_review_submission_sepc.rb 

=====Manual Testing=====

'''Step 1'''. The instructor will create an assignment and assign the zero and non-zero weights to the rubrics. 
'''Step 2'''. The author submits the work done in expertiza. 
'''Step 3'''. Peers check the submitted work and give reviews for it. 

'''Test 1: Zero weighted scores shown in gray.''' 
Login as an author in expertiza and go to scores page to view the feedback.
Zero weighted scores should be displayed in gray to the logged in user.

'''Test 2: Information button.''' 
Login as an author in expertiza and go to scores page to view the feedback.
The information button should explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade.”)

'''Step 4'''. The mentor evaluates the final submission and give grades.

'''Test 3: Display non-zero weighted scores''' 
Login as an author in expertiza and go to scores page to view the feedback.
The scores shown for the assignment should be the weighted average of the scores.
The zero weighted scores have to be replaced with non-zero weighted scores.

'''Step 5''': The author updates the work and re-submits it again.

'''Test 4: Review Update''' 
The Review should be reopened. The reviewer should be able to go and update the submitted review.
If the review is not updated the previous review score has to be retained.

CSC/ECE 517 Fall 2019 - E1975. Generalize Review Versioning

2019-12-17T02:08:35Z

Ajain37: /* Review Update */

__TOC__

== '''Introduction''' ==
Expertiza is a peer-review based learning platform in which work-products by students are evaluated and scored by peer-review of other students. Review process occurs in stages. After submission of initial feedback for a work-product by the peer-group, students are allowed time to incorporate the comments and improve their work. This second version of the work product is reviewed again and scores are given. This process might be repeated again and the average score from the last review stage is considered as the final score for that work-product. This incremental development of work-products and progressive learning is the fundamental concept underlying the Expertiza system.

=====Purpose=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The system also triggers an email to reviewer when there is a change in the submission. The scores are calculated on the fly based on the rubrics.

=====Scope=====
This project will enhance the existing scoring and peer review functionality and the underlying implementation along with the testing of the existing implementation. Testing of the score calculation, display for the weighted and non-weighted scores based on the combination of the rubrics are part of this project. This project also includes an additional feature for the author to notify the reviewers for the change in the submission.

=====Background=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The scores are calculated based on the rubrics defined in the system. There’s already an existing implementation for the computation of score but it doesn’t work for multipart rubric

=====Standards=====
The new standards that will be adhered to in this system are those given for the rails framework, and the code should follow rails and object oriented design principles. Any newly added code will adhere to the guidelines mentioned below:

● https://github.com/rubocop-hq/rails-style-guide 
● http://wiki.expertiza.ncsu.edu/index.php/Object-Oriented_Design_and_Programming.

== '''Problem Statement''' ==
=====Problem=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The functionality needs to be tested. It works in some cases, but we are suspicious that it is not entirely correct.

'''Profile Email Preference''':
User in expertiza has liberty to control what kind of emails they should receive. As the reviewers will receive an email for the change in submission. This can be controlled by selecting an option (When someone else submits work I am assigned to review) on the profile page of the user. As the checkboxes are far away on the right corner of the screen, the position of checkboxes need to be changed to make it more user friendly.
 

Scores for students in Expertiza can be based on a combination of rubrics: review rubrics for each round of review, author feedback, and teammate review. In CSC/ECE 517, we don’t really use the student-assigned scores to grade projects, but the peer-review score shown to the student is based entirely on the second round of review 

Here is how scores are calculated: 

1. If only zero-weighted scores are available (here, in the first round of review), the average of all zero-weighted scores should be shown in grey font. An information button should explain why the scores are shown in grey (“The reviews submitted so far do not count toward the author’s final grade.”): 

2. If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is:: 
80%⨉90% + 100%⨉10% = 82%

3. If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. '''However, the previous review score that was given in the current round will count until the reviewer updates the submitted review'''. '''This functionality doesn’t work for multipart rubrics.''' 

The system also doesn’t generate emails to reviewers in case a new submission is made by the author without changing submission parameters like submission URL. Eg: A new submission is made by making a new deployment. But the deployment URL doesn’t change. In this case as well, without changing the current flow the author should have an option to notify all the reviewers to revisit the submission.

=====Interface Requirement=====
The overall user interface would remain the same as of the current system. Since this is a modification to the current submission page, there will be a new addition to the submission page. On the submission page, there will be a new button to notify reviewers when there is a submission which doesn’t contain a change to submission parameters like submission URL, but the deployment has been updated on the same URL. Other than that the interface remains the same for the score page, only functionalities will be tested for multi-part rubrics.

=====Quality Requirements=====
The interface changes are minimal and very easy to adapt. The new button will be clearly visible and is supposed to have a clear text to interpret its functionality. The proposed changes should not affect the current system functionality in any way. The solution is only intended to enrich the project features.

=====Performance Requirements=====
Adding test for multi-part rubric functionality should not cause any overhead in the performance of the system. Sending emails through “Notify All Reviewers” button should affect the system minimally in terms of performance. The only overhead is a database query to get all the reviewers. A good database design should incorporate such a query smoothly.

=====Assumptions=====
The functionality should work in case the assignment rubric is not multi-part.

== '''Proposed Solution''' ==
=====Zero-weighted scores=====
If only zero-weighted scores are available in the first round of review, the average of all zero-weighted scores will be shown in a gray font. An information button will explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade”)

Inside compute_total_score method in on_the_fly_calc.rb module, we'll check if the weighted score exists, if not then we'll calculate the non weighted score. We'll also have to add a method in the questionnaire model to get non weighted score. Few changes are required for the UI to display non-weighted score in grey with information icon and non weighted score in black.

Possible files to be modified.

models/on_the_fly_calc.rb 
models/questionnaire.rb 
models/view_team.html.erb

===== Non-zero weighted scores=====
If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is 80%⨉90% + 100%⨉10% = 82%

The score calculation was initially tested manually to verify the scenarios with multiple weights and rounds.
Also, existing spec for score calculation with a weighted average for both varying and non-varying rubric is covering this scenario.
The spec is written under /spec/models/assignment_spec.rb. The "score" function of the "model/assignment.rb" is called for online (on the fly score calculation).

Following file has the spec for weighted average score calculation.
spec/models/assignment_spec.rb 

=====Review Update=====
If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review.

 This feature can be broken down into the following sub-cases:
 1. A submission can only be made before deadline
 2. A review can edit a review only before deadline
 3. Score calculation should be only done for latest submitted review

Subcases:

 
1. Spec to ensure submission can't be made after the deadline

File : spec/features/post_deadline_review_submission_spec.rb

'''
it 'should not allow submission after deadline' do
user = User.find_by(name: "student2065")
stub_current_user(user, user.role.name, user.role)
# goto student_task page, which has link to "Your work"
visit 'student_task/view?id=1'
# the page will have text "Your work" but will be grayed
expect(page).to have_content "Your work"
# the page will not have link to content "Your work"
expect{click_link "Your work"}.to raise_error(Capybara::ElementNotFound)
end
'''

2. Spec to ensure review can be done only before deadline

File : spec/features/post_deadline_review_submission_spec.rb

'''
it "should not be able to review work after deadline" do

# The spec is written to reproduce following bug. "Others' work" link open after deadline passed

user = User.find_by(name: "student2065")
stub_current_user(user, user.role.name, user.role)
visit 'student_task/view?id=1'

# the page should have content, but after deadline passes it is displayed as gray
# but there should not be any link attached to it
expect(page).to have_content "Others' work"

# this is the bug, even after deadline has passed, the link is still present
# the ui comment in file views/student_task/view.html.erb says
# 
# But the link seems to be open even after deadline passed.
# Screenshot attached as part of wiki for E1975, Fall 2019
expect(page).to have_link("Others' work", "/student_review/list?id=1")

end
'''

The test scenario is able to produce the following bug. At times the review link is open even after the deadline has passed. The bug is also shown/reproduced in the following screenshot for Program 1. 

[[File:ScreenShotReview.jpg]]

3. Score calculation

Score calculation for assignments for review is part of models/assignment.rb. The score calculation is handled by function "score" in this file and the testing is done for both multipart varying and non-varying rubric.
The respective spec is written in file spec/models/assignment_spec.rb.

'''

describe '#scores' do
context 'when assignment is varying rubric by round assignment' do
it 'calculates scores in each round of each team in current assignment' do
allow(participant).to receive(:scores).with(review1: [question]).and_return(98)
allow(assignment).to receive(:varying_rubrics_by_round?).and_return(true)
allow(assignment).to receive(:num_review_rounds).and_return(1)
allow(ReviewResponseMap).to receive(:get_responses_for_team_round).with(team, 1).and_return([response])
allow(Answer).to receive(:compute_scores).with([response], [question]).and_return(max: 95, min: 88, avg: 90)
expect(assignment.scores(review1: [question]).inspect).to eq("{:participants=>{:\"1\"=>98}, :teams=>{:\"0\"=>{:team=>#<AssignmentTeam id: 1, "\
"name: \"no team\", parent_id: 1, type: \"AssignmentTeam\", comments_for_advertisement: nil, advertise_for_partner: nil, "\
"submitted_hyperlinks: \"---\\n- https://www.expertiza.ncsu.edu\", directory_num: 0, grade_for_submission: nil, "\
"comment_for_submission: nil>, :scores=>{:max=>95, :min=>88, :avg=>90.0}}}}")
end
end

context 'when assignment is not varying rubric by round assignment' do
it 'calculates scores of each team in current assignment' do
allow(participant).to receive(:scores).with(review: [question]).and_return(98)
allow(assignment).to receive(:varying_rubrics_by_round?).and_return(false)
allow(ReviewResponseMap).to receive(:get_assessments_for).with(team).and_return([response])
allow(Answer).to receive(:compute_scores).with([response], [question]).and_return(max: 95, min: 88, avg: 90)
expect(assignment.scores(review: [question]).inspect).to eq("{:participants=>{:\"1\"=>98}, :teams=>{:\"0\"=>{:team=>#<AssignmentTeam id: 1, "\
"name: \"no team\", parent_id: 1, type: \"AssignmentTeam\", comments_for_advertisement: nil, advertise_for_partner: nil, "\
"submitted_hyperlinks: \"---\\n- https://www.expertiza.ncsu.edu\", directory_num: 0, grade_for_submission: nil, "\
"comment_for_submission: nil>, :scores=>{:max=>95, :min=>88, :avg=>90}}}}")
end
end
end

'''

[[File:Generalize_review.jpg]]

 
[[File:Usecasenew.png]]

=='''Test Plan'''==

=====Rspec=====

New test cases will be added to test and validate the functionalities which are not working as expected. Also, the test cases for features that will be newly added to the system, will also be written.
Test cases will be added to these files:

spec/features/post_deadline_review_submission_sepc.rb 
spec/models/assignment_participant_spec.rb 

The scenarios covered by the newly introduced test cases are as follows: 
1. Zero weighted scores with gray mark: rspec spec/model/assignment_participants.rb 
2. Non-zero weighted scores 
3. Review update: rspec spec/features/post_deadline_review_submission_sepc.rb 

=====Manual Testing=====

'''Step 1'''. The instructor will create an assignment and assign the zero and non-zero weights to the rubrics. 
'''Step 2'''. The author submits the work done in expertiza. 
'''Step 3'''. Peers check the submitted work and give reviews for it. 

'''Test 1: Zero weighted scores shown in gray.''' 
Login as an author in expertiza and go to scores page to view the feedback.
Zero weighted scores should be displayed in gray to the logged in user.

'''Test 2: Information button.''' 
Login as an author in expertiza and go to scores page to view the feedback.
The information button should explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade.”)

'''Step 4'''. The mentor evaluates the final submission and give grades.

'''Test 3: Display non-zero weighted scores''' 
Login as an author in expertiza and go to scores page to view the feedback.
The scores shown for the assignment should be the weighted average of the scores.
The zero weighted scores have to be replaced with non-zero weighted scores.

'''Step 5''': The author updates the work and re-submits it again.

'''Test 4: Review Update''' 
The Review should be reopened. The reviewer should be able to go and update the submitted review.
If the review is not updated the previous review score has to be retained.

CSC/ECE 517 Fall 2019 - E1975. Generalize Review Versioning

2019-12-17T02:01:07Z

Ajain37: /* Review Update */

__TOC__

== '''Introduction''' ==
Expertiza is a peer-review based learning platform in which work-products by students are evaluated and scored by peer-review of other students. Review process occurs in stages. After submission of initial feedback for a work-product by the peer-group, students are allowed time to incorporate the comments and improve their work. This second version of the work product is reviewed again and scores are given. This process might be repeated again and the average score from the last review stage is considered as the final score for that work-product. This incremental development of work-products and progressive learning is the fundamental concept underlying the Expertiza system.

=====Purpose=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The system also triggers an email to reviewer when there is a change in the submission. The scores are calculated on the fly based on the rubrics.

=====Scope=====
This project will enhance the existing scoring and peer review functionality and the underlying implementation along with the testing of the existing implementation. Testing of the score calculation, display for the weighted and non-weighted scores based on the combination of the rubrics are part of this project. This project also includes an additional feature for the author to notify the reviewers for the change in the submission.

=====Background=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The scores are calculated based on the rubrics defined in the system. There’s already an existing implementation for the computation of score but it doesn’t work for multipart rubric

=====Standards=====
The new standards that will be adhered to in this system are those given for the rails framework, and the code should follow rails and object oriented design principles. Any newly added code will adhere to the guidelines mentioned below:

● https://github.com/rubocop-hq/rails-style-guide 
● http://wiki.expertiza.ncsu.edu/index.php/Object-Oriented_Design_and_Programming.

== '''Problem Statement''' ==
=====Problem=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The functionality needs to be tested. It works in some cases, but we are suspicious that it is not entirely correct.

'''Profile Email Preference''':
User in expertiza has liberty to control what kind of emails they should receive. As the reviewers will receive an email for the change in submission. This can be controlled by selecting an option (When someone else submits work I am assigned to review) on the profile page of the user. As the checkboxes are far away on the right corner of the screen, the position of checkboxes need to be changed to make it more user friendly.
 

Scores for students in Expertiza can be based on a combination of rubrics: review rubrics for each round of review, author feedback, and teammate review. In CSC/ECE 517, we don’t really use the student-assigned scores to grade projects, but the peer-review score shown to the student is based entirely on the second round of review 

Here is how scores are calculated: 

1. If only zero-weighted scores are available (here, in the first round of review), the average of all zero-weighted scores should be shown in grey font. An information button should explain why the scores are shown in grey (“The reviews submitted so far do not count toward the author’s final grade.”): 

2. If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is:: 
80%⨉90% + 100%⨉10% = 82%

3. If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. '''However, the previous review score that was given in the current round will count until the reviewer updates the submitted review'''. '''This functionality doesn’t work for multipart rubrics.''' 

The system also doesn’t generate emails to reviewers in case a new submission is made by the author without changing submission parameters like submission URL. Eg: A new submission is made by making a new deployment. But the deployment URL doesn’t change. In this case as well, without changing the current flow the author should have an option to notify all the reviewers to revisit the submission.

=====Interface Requirement=====
The overall user interface would remain the same as of the current system. Since this is a modification to the current submission page, there will be a new addition to the submission page. On the submission page, there will be a new button to notify reviewers when there is a submission which doesn’t contain a change to submission parameters like submission URL, but the deployment has been updated on the same URL. Other than that the interface remains the same for the score page, only functionalities will be tested for multi-part rubrics.

=====Quality Requirements=====
The interface changes are minimal and very easy to adapt. The new button will be clearly visible and is supposed to have a clear text to interpret its functionality. The proposed changes should not affect the current system functionality in any way. The solution is only intended to enrich the project features.

=====Performance Requirements=====
Adding test for multi-part rubric functionality should not cause any overhead in the performance of the system. Sending emails through “Notify All Reviewers” button should affect the system minimally in terms of performance. The only overhead is a database query to get all the reviewers. A good database design should incorporate such a query smoothly.

=====Assumptions=====
The functionality should work in case the assignment rubric is not multi-part.

== '''Proposed Solution''' ==
=====Zero-weighted scores=====
If only zero-weighted scores are available in the first round of review, the average of all zero-weighted scores will be shown in a gray font. An information button will explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade”)

Inside compute_total_score method in on_the_fly_calc.rb module, we'll check if the weighted score exists, if not then we'll calculate the non weighted score. We'll also have to add a method in the questionnaire model to get non weighted score. Few changes are required for the UI to display non-weighted score in grey with information icon and non weighted score in black.

Possible files to be modified.

models/on_the_fly_calc.rb 
models/questionnaire.rb 
models/view_team.html.erb

===== Non-zero weighted scores=====
If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is 80%⨉90% + 100%⨉10% = 82%

The score calculation was initially tested manually to verify the scenarios with multiple weights and rounds.
Also, existing spec for score calculation with a weighted average for both varying and non-varying rubric is covering this scenario.
The spec is written under /spec/models/assignment_spec.rb. The "score" function of the "model/assignment.rb" is called for online (on the fly score calculation).

Following file has the spec for weighted average score calculation.
spec/models/assignment_spec.rb 

=====Review Update=====
If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review.

 This feature can be broken down into the following sub-cases:
 1. A submission can only be made before deadline
 2. A review can edit a review only before deadline
 3. Score calculation should be only done for latest submitted review

Subcases:

 
1. Spec to ensure submission can't be made after the deadline

File : spec/features/post_deadline_review_submission_spec.rb

'''
it "should not allow submission after deadline" do

user = User.find_by(name: "student2065")
stub_current_user(user, user.role.name, user.role)

# goto student_task page, which has link to "Your work"
visit 'student_task/view?id=1'

# the page will have text "Your work" but will be grayed
expect(page).to have_content "Your work"

# the page will not have link to content "Your work"
expect{click_link "Your work"}.to raise_error(Capybara::ElementNotFound)

end
'''

2. Spec to ensure review can be done only before deadline

File : spec/features/post_deadline_review_submission_spec.rb

'''
it "should not be able to review work after deadline" do

# The spec is written to reproduce following bug. "Others' work" link open after deadline passed

user = User.find_by(name: "student2065")
stub_current_user(user, user.role.name, user.role)
visit 'student_task/view?id=1'

# the page should have content, but after deadline passes it is displayed as gray
# but there should not be any link attached to it
expect(page).to have_content "Others' work"

# this is the bug, even after deadline has passed, the link is still present
# the ui comment in file views/student_task/view.html.erb says
# 
# But the link seems to be open even after deadline passed.
# Screenshot attached as part of wiki for E1975, Fall 2019
expect(page).to have_link("Others' work", "/student_review/list?id=1")

end
'''

The test scenario is able to produce the following bug. At times the review link is open even after the deadline has passed. The bug is also shown/reproduced in the following screenshot for Program 1. 

[[File:ScreenShotReview.jpg]]

3. Score calculation

Score calculation for assignments for review is part of models/assignment.rb. The score calculation is handled by function "score" in this file and the testing is done for both multipart varying and non-varying rubric.
The respective spec is written in file spec/models/assignment_spec.rb.

'''

describe '#scores' do
context 'when assignment is varying rubric by round assignment' do
it 'calculates scores in each round of each team in current assignment' do
allow(participant).to receive(:scores).with(review1: [question]).and_return(98)
allow(assignment).to receive(:varying_rubrics_by_round?).and_return(true)
allow(assignment).to receive(:num_review_rounds).and_return(1)
allow(ReviewResponseMap).to receive(:get_responses_for_team_round).with(team, 1).and_return([response])
allow(Answer).to receive(:compute_scores).with([response], [question]).and_return(max: 95, min: 88, avg: 90)
expect(assignment.scores(review1: [question]).inspect).to eq("{:participants=>{:\"1\"=>98}, :teams=>{:\"0\"=>{:team=>#<AssignmentTeam id: 1, "\
"name: \"no team\", parent_id: 1, type: \"AssignmentTeam\", comments_for_advertisement: nil, advertise_for_partner: nil, "\
"submitted_hyperlinks: \"---\\n- https://www.expertiza.ncsu.edu\", directory_num: 0, grade_for_submission: nil, "\
"comment_for_submission: nil>, :scores=>{:max=>95, :min=>88, :avg=>90.0}}}}")
end
end

context 'when assignment is not varying rubric by round assignment' do
it 'calculates scores of each team in current assignment' do
allow(participant).to receive(:scores).with(review: [question]).and_return(98)
allow(assignment).to receive(:varying_rubrics_by_round?).and_return(false)
allow(ReviewResponseMap).to receive(:get_assessments_for).with(team).and_return([response])
allow(Answer).to receive(:compute_scores).with([response], [question]).and_return(max: 95, min: 88, avg: 90)
expect(assignment.scores(review: [question]).inspect).to eq("{:participants=>{:\"1\"=>98}, :teams=>{:\"0\"=>{:team=>#<AssignmentTeam id: 1, "\
"name: \"no team\", parent_id: 1, type: \"AssignmentTeam\", comments_for_advertisement: nil, advertise_for_partner: nil, "\
"submitted_hyperlinks: \"---\\n- https://www.expertiza.ncsu.edu\", directory_num: 0, grade_for_submission: nil, "\
"comment_for_submission: nil>, :scores=>{:max=>95, :min=>88, :avg=>90}}}}")
end
end
end

'''

[[File:Generalize_review.jpg]]

 
[[File:Usecasenew.png]]

=='''Test Plan'''==

=====Rspec=====

New test cases will be added to test and validate the functionalities which are not working as expected. Also, the test cases for features that will be newly added to the system, will also be written.
Test cases will be added to these files:

spec/features/post_deadline_review_submission_sepc.rb 
spec/models/assignment_participant_spec.rb 

The scenarios covered by the newly introduced test cases are as follows: 
1. Zero weighted scores with gray mark: rspec spec/model/assignment_participants.rb 
2. Non-zero weighted scores 
3. Review update: rspec spec/features/post_deadline_review_submission_sepc.rb 

=====Manual Testing=====

'''Step 1'''. The instructor will create an assignment and assign the zero and non-zero weights to the rubrics. 
'''Step 2'''. The author submits the work done in expertiza. 
'''Step 3'''. Peers check the submitted work and give reviews for it. 

'''Test 1: Zero weighted scores shown in gray.''' 
Login as an author in expertiza and go to scores page to view the feedback.
Zero weighted scores should be displayed in gray to the logged in user.

'''Test 2: Information button.''' 
Login as an author in expertiza and go to scores page to view the feedback.
The information button should explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade.”)

'''Step 4'''. The mentor evaluates the final submission and give grades.

'''Test 3: Display non-zero weighted scores''' 
Login as an author in expertiza and go to scores page to view the feedback.
The scores shown for the assignment should be the weighted average of the scores.
The zero weighted scores have to be replaced with non-zero weighted scores.

'''Step 5''': The author updates the work and re-submits it again.

'''Test 4: Review Update''' 
The Review should be reopened. The reviewer should be able to go and update the submitted review.
If the review is not updated the previous review score has to be retained.

File:Generalize review.jpg

2019-12-17T02:00:23Z

Ajain37:

CSC/ECE 517 Fall 2019 - E1975. Generalize Review Versioning

2019-12-17T01:16:32Z

Ajain37:

__TOC__

== '''Introduction''' ==
Expertiza is a peer-review based learning platform in which work-products by students are evaluated and scored by peer-review of other students. Review process occurs in stages. After submission of initial feedback for a work-product by the peer-group, students are allowed time to incorporate the comments and improve their work. This second version of the work product is reviewed again and scores are given. This process might be repeated again and the average score from the last review stage is considered as the final score for that work-product. This incremental development of work-products and progressive learning is the fundamental concept underlying the Expertiza system.

=====Purpose=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The system also triggers an email to reviewer when there is a change in the submission. The scores are calculated on the fly based on the rubrics.

=====Scope=====
This project will enhance the existing scoring and peer review functionality and the underlying implementation along with the testing of the existing implementation. Testing of the score calculation, display for the weighted and non-weighted scores based on the combination of the rubrics are part of this project. This project also includes an additional feature for the author to notify the reviewers for the change in the submission.

=====Background=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The scores are calculated based on the rubrics defined in the system. There’s already an existing implementation for the computation of score but it doesn’t work for multipart rubric

=====Standards=====
The new standards that will be adhered to in this system are those given for the rails framework, and the code should follow rails and object oriented design principles. Any newly added code will adhere to the guidelines mentioned below:

● https://github.com/rubocop-hq/rails-style-guide 
● http://wiki.expertiza.ncsu.edu/index.php/Object-Oriented_Design_and_Programming.

== '''Problem Statement''' ==
=====Problem=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The functionality needs to be tested. It works in some cases, but we are suspicious that it is not entirely correct.

'''Profile Email Preference''':
User in expertiza has liberty to control what kind of emails they should receive. As the reviewers will receive an email for the change in submission. This can be controlled by selecting an option (When someone else submits work I am assigned to review) on the profile page of the user. As the checkboxes are far away on the right corner of the screen, the position of checkboxes need to be changed to make it more user friendly.
 

Scores for students in Expertiza can be based on a combination of rubrics: review rubrics for each round of review, author feedback, and teammate review. In CSC/ECE 517, we don’t really use the student-assigned scores to grade projects, but the peer-review score shown to the student is based entirely on the second round of review 

Here is how scores are calculated: 

1. If only zero-weighted scores are available (here, in the first round of review), the average of all zero-weighted scores should be shown in grey font. An information button should explain why the scores are shown in grey (“The reviews submitted so far do not count toward the author’s final grade.”): 

2. If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is:: 
80%⨉90% + 100%⨉10% = 82%

3. If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. '''However, the previous review score that was given in the current round will count until the reviewer updates the submitted review'''. '''This functionality doesn’t work for multipart rubrics.''' 

The system also doesn’t generate emails to reviewers in case a new submission is made by the author without changing submission parameters like submission URL. Eg: A new submission is made by making a new deployment. But the deployment URL doesn’t change. In this case as well, without changing the current flow the author should have an option to notify all the reviewers to revisit the submission.

=====Interface Requirement=====
The overall user interface would remain the same as of the current system. Since this is a modification to the current submission page, there will be a new addition to the submission page. On the submission page, there will be a new button to notify reviewers when there is a submission which doesn’t contain a change to submission parameters like submission URL, but the deployment has been updated on the same URL. Other than that the interface remains the same for the score page, only functionalities will be tested for multi-part rubrics.

=====Quality Requirements=====
The interface changes are minimal and very easy to adapt. The new button will be clearly visible and is supposed to have a clear text to interpret its functionality. The proposed changes should not affect the current system functionality in any way. The solution is only intended to enrich the project features.

=====Performance Requirements=====
Adding test for multi-part rubric functionality should not cause any overhead in the performance of the system. Sending emails through “Notify All Reviewers” button should affect the system minimally in terms of performance. The only overhead is a database query to get all the reviewers. A good database design should incorporate such a query smoothly.

=====Assumptions=====
The functionality should work in case the assignment rubric is not multi-part.

== '''Proposed Solution''' ==
=====Zero-weighted scores=====
If only zero-weighted scores are available in the first round of review, the average of all zero-weighted scores will be shown in a gray font. An information button will explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade”)

Inside compute_total_score method in on_the_fly_calc.rb module, we'll check if the weighted score exists, if not then we'll calculate the non weighted score. We'll also have to add a method in the questionnaire model to get non weighted score. Few changes are required for the UI to display non-weighted score in grey with information icon and non weighted score in black.

Possible files to be modified.

models/on_the_fly_calc.rb 
models/questionnaire.rb 
models/view_team.html.erb

===== Non-zero weighted scores=====
If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is 80%⨉90% + 100%⨉10% = 82%

The score calculation was initially tested manually to verify the scenarios with multiple weights and rounds.
Also, existing spec for score calculation with a weighted average for both varying and non-varying rubric is covering this scenario.
The spec is written under /spec/models/assignment_spec.rb. The "score" function of the "model/assignment.rb" is called for online (on the fly score calculation).

Following file has the spec for weighted average score calculation.
spec/models/assignment_spec.rb 

=====Review Update=====
If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review.

 This feature can be broken down into the following sub-cases:
 1. A submission can only be made before deadline
 2. A review can edit a review only before deadline
 3. Score calculation should be only done for latest submitted review

Subcases:

 
1. Spec to ensure submission can't be made after the deadline

File : spec/features/post_deadline_review_submission_spec.rb

'''
it "should not allow submission after deadline" do

user = User.find_by(name: "student2065")
stub_current_user(user, user.role.name, user.role)

# goto student_task page, which has link to "Your work"
visit 'student_task/view?id=1'

# the page will have text "Your work" but will be grayed
expect(page).to have_content "Your work"

# the page will not have link to content "Your work"
expect{click_link "Your work"}.to raise_error(Capybara::ElementNotFound)

end
'''

2. Spec to ensure review can be done only before deadline

File : spec/features/post_deadline_review_submission_spec.rb

'''
it "should not be able to review work after deadline" do

# The spec is written to reproduce following bug. "Others' work" link open after deadline passed

user = User.find_by(name: "student2065")
stub_current_user(user, user.role.name, user.role)
visit 'student_task/view?id=1'

# the page should have content, but after deadline passes it is displayed as gray
# but there should not be any link attached to it
expect(page).to have_content "Others' work"

# this is the bug, even after deadline has passed, the link is still present
# the ui comment in file views/student_task/view.html.erb says
# 
# But the link seems to be open even after deadline passed.
# Screenshot attached as part of wiki for E1975, Fall 2019
expect(page).to have_link("Others' work", "/student_review/list?id=1")

end
'''

The test scenario is able to produce the following bug. At times the review link is open even after the deadline has passed. The bug is also shown/reproduced in the following screenshot for Program 1. 

[[File:ScreenShotReview.jpg]]

3. Score calculation

Score calculation for assignments for review is part of models/assignment.rb. The score calculation is handled by function "score" in this file and the testing is done for both multipart varying and non-varying rubric.
The respective spec is written in file spec/models/assignment_spec.rb.

'''

describe '#scores' do
context 'when assignment is varying rubric by round assignment' do
it 'calculates scores in each round of each team in current assignment' do
allow(participant).to receive(:scores).with(review1: [question]).and_return(98)
allow(assignment).to receive(:varying_rubrics_by_round?).and_return(true)
allow(assignment).to receive(:num_review_rounds).and_return(1)
allow(ReviewResponseMap).to receive(:get_responses_for_team_round).with(team, 1).and_return([response])
allow(Answer).to receive(:compute_scores).with([response], [question]).and_return(max: 95, min: 88, avg: 90)
expect(assignment.scores(review1: [question]).inspect).to eq("{:participants=>{:\"1\"=>98}, :teams=>{:\"0\"=>{:team=>#<AssignmentTeam id: 1, "\
"name: \"no team\", parent_id: 1, type: \"AssignmentTeam\", comments_for_advertisement: nil, advertise_for_partner: nil, "\
"submitted_hyperlinks: \"---\\n- https://www.expertiza.ncsu.edu\", directory_num: 0, grade_for_submission: nil, "\
"comment_for_submission: nil>, :scores=>{:max=>95, :min=>88, :avg=>90.0}}}}")
end
end

context 'when assignment is not varying rubric by round assignment' do
it 'calculates scores of each team in current assignment' do
allow(participant).to receive(:scores).with(review: [question]).and_return(98)
allow(assignment).to receive(:varying_rubrics_by_round?).and_return(false)
allow(ReviewResponseMap).to receive(:get_assessments_for).with(team).and_return([response])
allow(Answer).to receive(:compute_scores).with([response], [question]).and_return(max: 95, min: 88, avg: 90)
expect(assignment.scores(review: [question]).inspect).to eq("{:participants=>{:\"1\"=>98}, :teams=>{:\"0\"=>{:team=>#<AssignmentTeam id: 1, "\
"name: \"no team\", parent_id: 1, type: \"AssignmentTeam\", comments_for_advertisement: nil, advertise_for_partner: nil, "\
"submitted_hyperlinks: \"---\\n- https://www.expertiza.ncsu.edu\", directory_num: 0, grade_for_submission: nil, "\
"comment_for_submission: nil>, :scores=>{:max=>95, :min=>88, :avg=>90}}}}")
end
end
end

'''

[[File:E1975_CHECKBOX_1.jpg]]

 
[[File:Usecasenew.png]]

=='''Test Plan'''==

=====Rspec=====

New test cases will be added to test and validate the functionalities which are not working as expected. Also, the test cases for features that will be newly added to the system, will also be written.
Test cases will be added to these files:

spec/features/post_deadline_review_submission_sepc.rb 
spec/models/assignment_participant_spec.rb 

The scenarios covered by the newly introduced test cases are as follows: 
1. Zero weighted scores with gray mark: rspec spec/model/assignment_participants.rb 
2. Non-zero weighted scores 
3. Review update: rspec spec/features/post_deadline_review_submission_sepc.rb 

=====Manual Testing=====

'''Step 1'''. The instructor will create an assignment and assign the zero and non-zero weights to the rubrics. 
'''Step 2'''. The author submits the work done in expertiza. 
'''Step 3'''. Peers check the submitted work and give reviews for it. 

'''Test 1: Zero weighted scores shown in gray.''' 
Login as an author in expertiza and go to scores page to view the feedback.
Zero weighted scores should be displayed in gray to the logged in user.

'''Test 2: Information button.''' 
Login as an author in expertiza and go to scores page to view the feedback.
The information button should explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade.”)

'''Step 4'''. The mentor evaluates the final submission and give grades.

'''Test 3: Display non-zero weighted scores''' 
Login as an author in expertiza and go to scores page to view the feedback.
The scores shown for the assignment should be the weighted average of the scores.
The zero weighted scores have to be replaced with non-zero weighted scores.

'''Step 5''': The author updates the work and re-submits it again.

'''Test 4: Review Update''' 
The Review should be reopened. The reviewer should be able to go and update the submitted review.
If the review is not updated the previous review score has to be retained.

File:ScreenShotReview.jpg

2019-12-17T01:15:38Z

Ajain37:

CSC/ECE 517 Fall 2019 - E1938. OSS project Duke Blue: Fix import glitches

2019-11-12T05:46:55Z

Ajain37: /* References */

__TOC__

== '''About Expertiza''' ==

Expertiza is an open source project based on Ruby on Rails framework and the code is available on Github. Expertiza allows the instructor to create new assignments as well as edit new or existing assignments. Instructors can also create a list of topics the students can sign up for and specify deadlines for completion of various tasks. Students can form teams in Expertiza to work on various projects and assignments as well as peer review other students' submissions. Expertiza supports submission across various document types, including the URLs Wiki pages.
 
 

== '''Problem Statement''' ==
The import feature is the most helpful feature for instructors to set up assignments. The instructors usually have a list of students, teams, etc from their learning management system. Being able to import these into expertiza saves a lot of time when setting up an assignment. Unfortunately this feature has some bugs that were identified as the following list.

'''Issue 918''': It seems that non-ASCII characters in the description (e.g., a curly apostrophe, and perhaps also an en-dash) caused a SQL error in inserting the description string into the db. I had to edit the import file to remove non-ASCII characters. Any Unicode character ought to be allowed in a description.

'''Issue 153''': There should be a way to import a list of who has signed up for which topic. When a signup sheet is used in Expertiza, users are expected to sign up for topics. But, the instructor might've taken signups offline, e.g., by passing around a signup sheet in class. There should be a way to import this list to Expertiza, rather than impersonate all the students one by one and sign them up for a topic. '''This issue is not a bug but an additional feature to the existing import topic functionality.'''

'''Issue 329''': When importing teams there are different options to handle conflicting team names

'''Issue 328''': When importing teams, and a conflict exists there is an option to merge the two teams by inserting the new members into an existing team. Unfortunately, this feature is broken. we tried to import a team and selected 'Insert any new members into existing team'. No new team members were in fact inserted into the team

== '''Fixes''' ==
=====Issue 918=====

'''Problem:''' The application is not able to import topics with non ascii characters in its description field. 
'''Solution:''' Before the topic object is persisted into the database, check for any non ascii characters in the description field and strip them. 
'''Code Changes''' 
New code introduced is as follows. 
File:app/helpers/import_topics_helper.rb 
Here, a new function 'trim_non_ascii' has been added which iterates a given string character by character, and replaces each non ascii character with a white space character. 

def self.define_attributes(row_hash)
attributes = {}
if !row_hash[:description].nil? and !row_hash[:description].ascii_only?
row_hash[:description] = self.trim_non_ascii(row_hash[:description])
puts row_hash[:description]
end
attributes["topic_identifier"] = row_hash[:topic_identifier].strip
attributes["topic_name"] = row_hash[:topic_name].strip
attributes["max_choosers"] = row_hash[:max_choosers].strip
attributes
end

def self.trim_non_ascii(string)
string.split('').each do |char|
!char.ascii_only? ? string.tr!(char, ' ') : nil
end
string.gsub!(/\s+/, ' ')
end

=====Issue 328=====

'''Problem''': When importing teams, and a conflict exists there is an option to merge the two teams by inserting the new members into an existing team.

'''Solution''': This was an issue to fix the functionality of Insert any new members into existing team. This was due to faulty handling of the option handle_dups and incorrect checking of parameters.
 
Code was modified to ensure minimal impact to existing testcases and functionalities. 
The handling of handle_dups in options was done assuming that the parameter was a symbol. But when tested, it was found to be a string and hence the handling was updated accordingly.
 
name = handle_duplicate(team, name, id, options["handle_dups"], teamtype)
Also, the if and else conditions for checking class type of used was wrong. This was updated based on how the variable was assigned in the '''import_file_controller.rb#import_from_hash'''
if(teamtype.is_a?(CourseTeam.class))
...
elsif teamtype.is_a?(AssignmentTeam.class)
...

 
[[File:E1938_Before_D.PNG]]
 
After fixing changes 
[[File:Insert_new_in_old.PNG]]
 
 

=====Issue 329=====
'''Problem''': When importing teams there are different options to handle conflicting team names as shown in the screenshot below. We’d like to add another option to rename EXISTING team when a conflict exist. Introduce an option to rename the existing team in import_file/start.html.erb, handle this option in import_file_controller.rb and team.rb→import.

'''Solution''': This was a request to add an option to rename existing team if there was a conflict in the team name.
New code introduced was as follows: 
In '''start.html.erb''' 
In this file, the new option as displayed to the user is added 
<option value="rename_existing">rename the existing team and import</option>
 

In '''team.rb''' 
In this file, the handling of the new tag "rename_existing" is added wherein, the old team is updated with a new name. 
# E1938: Added handling for renaming old team when conflict arises
if handle_dups == "rename_existing"
if teamtype.is_a?(CourseTeam.class)
team.update(name: self.generate_team_name(Course.find(id).name))
elsif teamtype.is_a?(AssignmentTeam.class)
team.update(name: self.generate_team_name(Assignment.find(id).name))
end
return name
end

 
'''The state of D before conflicting import''': 
 
[[File:E1938_Before_D_1.PNG]]
 
After fixing changes 
[[File:After_D.PNG]]
[[File:After_old_team.PNG]] 
As you can see the '''new team got the name D''' and the old file got renamed to '''Team_17'''.
 
 

=====Issue 153=====
'''Requirement''': There should be a way to import a list of who has signed up for which topic. When a signup sheet is used in Expertiza, users are expected to sign up for topics. But, the instructor might've taken signups offline, e.g., by passing around a signup sheet in class. There should be a way to import this list to Expertiza, rather than impersonate all the students one by one and sign them up for a topic.

'''Solution''': We have modified the existing implementation for import topics to assign the teams to the topics. We have changed the files to handle another column where the teams have been assigned to topics. The column is optional because there might be topics that have not yet been assigned to any teams

'''Expectations from the import file:
'''
 
* Assigned team name should exist in the system, otherwise the import will fail. Instructor has to make sure that the teams which are assigned in the import file should exist with same names in the system or database.
* Instructor also need to ensure that the topic for which the team is being assigned should not be full or can accommodate new team assignment to the respective topic.
 
'''Code Changes:'''
 
 
'''import_file/_sign_up_topic.html.erb'''
 
As there's already a screen with select options to map the columns with the fields for import. We are just adding another column so the best way is to extend the existing select tag with a new option for the teams assigned. All the select tags has been changed with an additional option "Assigned Team".
 
<th>
<select name="select1" id="select1" class="form-control" style="background-color:lightgrey">
<option value="topic_identifier">Topic Identifier (required)</option>
<option value="topic_name">Topic Name (required)</option>
<option value="max_choosers">Max Choosers (required)</option>
<option value="category">Category (optional)</option>
<option value="description">Description (optional)</option>
<option value="link">Link (optional)</option>
<option value="assigned_team">Assigned Team (optional)</option>
</select>
</th>

 
'''model/sign_up_topic.rb'''
 
The existing import function in the model import the topics to the topic table. As per the requirement and validations, topic should be imported first and then only the team can be assigned to the topic. So the existing import method has been modified to assign the team once the topic is imported in the database table. As per the edge case mentioned in the requirement document that few topics may not have any team assigned to it, so we first check if topic has any team assigned, then only it tries to map the topic with the team.
 
def self.import(row_hash, session, _id = nil)
if row_hash.length < 3
raise ArgumentError, "The CSV File expects the format: Topic identifier, Topic name, Max choosers, Topic Category (optional), Topic Description (Optional), Topic Link (optional), Assigned Team Name (optional)."
end
topic = SignUpTopic.where(topic_name: row_hash[:topic_name], assignment_id: session[:assignment_id]).first
if topic.nil?
attributes = ImportTopicsHelper.define_attributes(row_hash)
topic_new_id = ImportTopicsHelper.create_new_sign_up_topic(attributes, session)
unless row_hash[:assigned_team].nil?
team = Team.where(name: row_hash[:assigned_team]).first
ImportTopicsHelper.assign_team_topic(topic_new_id, team.id)
end
else
topic.max_choosers = row_hash[:max_choosers]
topic.topic_identifier = row_hash[:topic_identifier]
topic.save
unless row_hash[:assigned_team].nil?
team = Team.where(name: row_hash[:assigned_team]).first
newteam = SignedUpTeam.where(topic_id: topic.id, team_id: team.id).first
if newteam.nil?
ImportTopicsHelper.assign_team_topic(topic.id, team.id)
else
newteam.team_id = team.id
newteam.save
end
end
end
end

'''helpers/import_topics_helper.rb'''
 
Having the assign_team_topic in the helper would organize the code but also in the unit testing. We can isolate the model logic and test this method separately by writing the test cases.
 
def self.assign_team_topic(topic_id, assigned_team)
attributes = {}
attributes["topic_id"] = topic_id
attributes["team_id"] = assigned_team
assign_team = SignedUpTeam.new(attributes)
assign_team.save!
end

def self.create_new_sign_up_topic(attributes, session)
sign_up_topic = SignUpTopic.new(attributes)
sign_up_topic.assignment_id = session[:assignment_id]
sign_up_topic.save
# sign_up_topic
sign_up_topic.save!
sign_up_topic.id
end

 
'''controllers/import_file_controller.rb'''
 
Existing implementation for import have many flaws and design issues. Most of the code has been written with nested if else, which is not the practice to be followed. Also the names of the fields have been hardcoded.
 
As we cannot change the complete import implementation. So i had to use the existing one the way it is and make changes to extend the functionality.
 

if (params[:assigned_team] == 'true')
@optional_count += 1
end
 
elsif params[:optional_count] == '4'
new_header = [params[:select1], params[:select2], params[:select3], params[:select4], params[:select5], params[:select6], params[:select7]]
@header_integrated_body = hash_rows_with_headers(new_header,contents_hash[:body])

 
'''views/sign_up_sheet/_add_topics.html.erb'''
 
This code has been used to display the columns in the footer.As we already have implementation for other columns, so another column name 'Assigned Team' is added here in this part of the code.
<%= link_to 'Import topics',
{
:controller => 'import_file',
:action => 'start',
:model => 'SignUpTopic',
:id => params[:id],
:expected_fields => 'Topic Identifier' +
'&nbsp&nbsp|&nbsp&nbsp' +
'Topic Name' +
'&nbsp&nbsp|&nbsp&nbsp' +
'Max Choosers' +
'&nbsp&nbsp|&nbsp&nbsp' +
'Topic Category (optional)' +
'&nbsp&nbsp|&nbsp&nbsp' +
'Topic Description (optional)' +
'&nbsp&nbsp|&nbsp&nbsp' +
'Topic Link (optional)' +
'&nbsp&nbsp|&nbsp&nbsp' +
'Assigned Team (optional)'

''' Changes in the application with new implementation '''

An option column checkbox (Teams Assigned) has been added to the UI
 
 
[[File:Issue153SS8.jpg]]
 
 
A new column Assigned Team is added where the columns are being mapped.
 
 
[[File: Issue153SS7.jpg]]

 
 

== '''Test Plan''' ==
*'''RSpec:'''

::We have added new testcases for the new pieces of code that were added by us.

::'''[Problem 1 (918)]'''
::::New testcases have been added to test this functionality. We use it to check if description with Ascii characters are being trimmed. These test cases has been added to '''import_topics_helper_spec.rb'''

::'''[Problem 2 (153)]'''
::::As discussed with the mentor, there were no test cases for the SignUpTopic model which is being used to import the topics. As our additional functionality is to have another column to assign team to the topic. We have to have the import test case and the assigned team change could be applied to the same test case.

::'''[Problem 3 (329)]'''
::::New testcases have been added to test this functionality. We use it to check if the older team gets the new name and the new team gets the requested name. These testcases run when we run bundle exec for the '''teams_spec.rb'''

::'''[Problem 4 (328)]'''
::::As discussed with the Mentor this was a bug fix in the existing functionality. There are no new test cases needed for this.'

 
*'''UI Testing:'''

::'''[Problem 1 (918)]'''
::::1. Login to Expertiza using username: instructor6 & password: password.
::::2. Go to Assignments and click on edit for any of the listed assignments.
::::3. Click on Topics tab.
::::4. Scroll down to the bottom of the page and click on Import topics.
::::5. Click on Topic Description checkbox and import the following file. (sample.csv)
::::6. Click on next and make the header for 4th column as 'Description' from the dropdown.
::::7. Import topics.
::::8. After this, a new topic should be imported which can be viewed from the aforementioned Topics tab. This newly imported topic will not have any non-ascii characters.

::'''[Problem 2 (153)]'''
::::1. Login to Expertiza using username: instructor6 & password: password.
::::2. Select Manage >Assignments > Edit Assignment > Topics
::::3. Scroll down to the screen and click on import topics
::::4. Upload a file with the topic details and a team assigned to topic
::::5. Check the Teams Assigned optional column
::::6. Click on import and map the columns in the mapping page
::::6. After submitting, team will be assigned to appropriate topic

::'''[Problem 3 (329)]'''
::::1. Login to Expertiza using username: instructor6 & password: password.
::::2. Select Manage > Courses or Assignments > Add Teams button > Import Team
::::3. In the screen that loads, select "rename the existing team and import" in the handle conflicts section.
::::4. Upload a file with the team details and select the appropriate delimiter.
::::5. Click on Create
::::6. Verify that the new team got created with the assigned name and the older team's name got changed.

::'''[Problem 4 (328)]'''
::::1. Login to Expertiza using username: instructor6 & password: password.
::::2. Select Manage > Courses or Assignments > Add Teams button > Import Team
::::3. In the screen that loads, select "Insert any new members into existing team" in the handle conflicts section.
::::4. Upload a file with the team details and select the appropriate delimiter.
::::5. Click on Create
::::6. Verify that the existing team got updated with the new team members

== '''Possible Enhancements''' ==

====='''Import functionality'''=====

The existing import implementation does not follow DRY principle, rails code practices and even basic design principles. Most of the code is written based with nested if else conditions. This implementation can be improved.
 
Exception Handling: Before starting to work on this requirement. I tried few imports and in many cases, import was not successful and i was not able to figure out what's going wrong. My suggestion would be to develop a screen to show which rows were successfully imported and the failure reason which didn't.
 
Also, if there's no team in the system and instructor has used it in the import file so a new pop up can be displayed where the new team can be created or the import can be extended to create teams while assigning the topic.
 
====='''Non Ascii character imports'''=====
The non ascii character issue exists for '''topic identifier''' and '''topic name''' fields as well. This should also be fixed in a similar way as is done for Issue #918 above.
 

== '''References''' ==

1. Expertiza on GITHUB: https://github.com/expertiza/expertiza

2. GitHub Project Repository Fork: https://github.com/NisargC/expertiza

3. Demo link: http://152.46.19.138:8080/

4. Demo Video link: https://drive.google.com/open?id=1mPnOsO5n_44lcVrPuYlbZrX3gDXLA9j7

5. Expertiza project documentation wiki: http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Fall_2019_-_E1938._OSS_project_Duke_Blue:_Fix_import_glitches

CSC/ECE 517 Fall 2019 - E1975. Generalize Review Versioning

2019-11-12T04:31:45Z

Ajain37: /* System Requirements */

__TOC__

== '''About Expertiza''' ==

Expertiza is an open source project based on Ruby on Rails framework and the code is available on Github. Expertiza allows the instructor to create new assignments as well as edit new or existing assignments. Instructors can also create a list of topics the students can sign up for and specify deadlines for completion of various tasks. Students can form teams in Expertiza to work on various projects and assignments as well as peer review other students' submissions. Expertiza supports submission across various document types, including the URLs Wiki pages.
 
 

== '''Introduction''' ==
Expertiza is a peer-review based learning platform in which work-products by students are evaluated and scored by peer-review of other students. Review process occurs in stages. After submission of initial feedback for a work-product by the peer-group, students are allowed time to incorporate the comments and improve their work. This second version of the work product is reviewed again and scores are given. This process might be repeated again and the average score from the last review stage is considered as the final score for that work-product. This incremental development of work-products and progressive learning is the fundamental concept underlying the Expertiza system.

=====Purpose=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The system also triggers an email to reviewer when there is a change in the submission. The scores are calculated on the fly based on the rubrics.

=====Scope=====
This project will enhance the existing scoring and peer review functionality and the underlying implementation along with the testing of the existing implementation. Testing of the score calculation, display for the weighted and non-weighted scores based on the combination of the rubrics are part of this project. This project also includes an additional feature for the author to notify the reviewers for the change in the submission.

=====Background=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The scores are calculated based on the rubrics defined in the system. There’s already an existing implementation for the computation of score but it doesn’t work for multipart rubric

=====Documentation=====
The documents that will be generated for this project are the design document, the actual files that we edit or create, and a ReadMe that explains what we did and how to use the modified product.

=====Standards=====
The new standards that will be adhered to in this system are those given for the rails framework, and the code should follow rails and object oriented design principles. Any newly added code will adhere to the guidelines mentioned below:

● https://github.com/rubocop-hq/rails-style-guide 
● http://wiki.expertiza.ncsu.edu/index.php/Object-Oriented_Design_and_Programming.

== '''Problem Statement''' ==
=====Problem=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The functionality needs to be tested. It works in some cases, but we are suspicious that it is not entirely correct.

'''Profile Email Preference''':
User in expertiza has liberty to control what kind of emails they should receive. As the reviewers will receive an email for the change in submission. This can be controlled by selecting an option (When someone else submits work I am assigned to review) on the profile page of the user. As the checkboxes are far away on the right corner of the screen, the position of checkboxes need to be changed to make it more user friendly.
 

Scores for students in Expertiza can be based on a combination of rubrics: review rubrics for each round of review, author feedback, and teammate review. In CSC/ECE 517, we don’t really use the student-assigned scores to grade projects, but the peer-review score shown to the student is based entirely on the second round of review 

Here is how scores are calculated: 

1. If only zero-weighted scores are available (here, in the first round of review), the average of all zero-weighted scores should be shown in grey font. An information button should explain why the scores are shown in grey (“The reviews submitted so far do not count toward the author’s final grade.”): 

2. If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is:: 
80%⨉90% + 100%⨉10% = 82%

3. If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. '''However, the previous review score that was given in the current round will count until the reviewer updates the submitted review'''. '''This functionality doesn’t work for multipart rubrics.''' 

The system also doesn’t generate emails to reviewers in case a new submission is made by the author without changing submission parameters like submission URL. Eg: A new submission is made by making a new deployment. But the deployment URL doesn’t change. In this case as well, without changing the current flow the author should have an option to notify all the reviewers to revisit the submission.

=====Interface Requirement=====
The overall user interface would remain the same as of the current system. Since this is a modification to the current submission page, there will be a new addition to the submission page. On the submission page, there will be a new button to notify reviewers when there is a submission which doesn’t contain a change to submission parameters like submission URL, but the deployment has been updated on the same URL. Other than that the interface remains the same for the score page, only functionalities will be tested for multi-part rubrics.

=====Quality Requirements=====
The interface changes are minimal and very easy to adapt. The new button will be clearly visible and is supposed to have a clear text to interpret its functionality. The proposed changes should not affect the current system functionality in any way. The solution is only intended to enrich the project features.

=====Portability Requirements=====
All of the changes for this enhancement to Expertiza should not interfere with the platform portability in any way.

=====Performance Requirements=====
Adding test for multi-part rubric functionality should not cause any overhead in the performance of the system. Sending emails through “Notify All Reviewers” button should affect the system minimally in terms of performance. The only overhead is a database query to get all the reviewers. A good database design should incorporate such a query smoothly.

=====Assumptions=====
The functionality should work in case the assignment rubric is not multi-part.

== '''Proposed Solution''' ==
=====Zero-weighted scores=====
If only zero-weighted scores are available in the first round of review, the average of all zero-weighted scores will be shown in a gray font. An information button will explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade”)

Inside compute_total_score method in on_the_fly_calc.rb module, we'll check if the weighted score exists, if not then we'll calculate the non weighted score. We'll also have to add a method in the questionnaire model to get non weighted score. Few changes are required for the UI to display non-weighted score in grey with information icon and non weighted score in black.

Possible files to be modified.

models/on_the_fly_calc.rb 
models/questionnaire.rb 
models/view_team.html.erb

===== Non-zero weighted scores=====
If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is 80%⨉90% + 100%⨉10% = 82%

We'll first write the test cases by adding test cases to the '''spec/models/on_the_fly_calc_spec.rb'''

Also, we'll test this manually by computing the score and if there's a bug then below are the possible files to be modified.

models/on_the_fly_calc.rb 
models/questionnarie.rb

=====Review Update=====
If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review. This functionality already exists in the existing system. However, In some submissions, there’s no change in the submitted URL’s so there’s no way to let the reviewers know that the author has made some changes in the project. We’ll add a notify button for the Author in the UI which will trigger an email to the reviewers. This will reopen the review, and then the reviewer can go in and update the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review

'''Sending an email when the submission parameters don’t change:'''

A new button will be added on the submission page. The button will enable the author to notify all the reviewers to revisit the submission. The email will be sent to all the assigned reviewers only if their email preference (When someone else submits work I am assigned to review) in the profile is set to receive the email.

=====Profile Email Preference=====
The checkboxes are moved to the left corner. As the layout of the options have used table layout. So, we'll increase the width of the the column which includes checkboxes. We'll be using the percentage instead of fixed value to give the width, as this would adjust the column size based on the screen size. The CSS can be applied to the '''layout.scss''' file to using the appropriate class given to the column 

[[File:E1975_CHECKBOX_1.jpg]]

 
[[File:Use Case 1.png]]

=='''Test Plan'''==

=====Rspec=====

New test cases will be added to test the functionalities which are not working as expected. Also the test cases for the additional features and changes in the system. Test cases will be added to these files:

spec/models/on_the_fly_calc_spec.rb 
spec/models/questionnaire_spec.rb 

=====Email Preferences UI Changes=====

The email preferences checkboxes at present appear sticking to the rightmost border of the page. By following the requirements the checkboxes will be made available closer to the text fields to make them look more user-friendly.
The changes can be tested by logging into the system and on the user profile page.

=====Manual Testing=====

'''Step 1'''. The instructor will create an assignment and assign the zero and non-zero weights to the rubrics. 
'''Step 2'''. The author submits the work done in expertiza. 
'''Step 3'''. Peers check the submitted work and give reviews for it. 

'''Test 1: Zero weighted scores shown in gray.''' 
Login as an author in expertiza and go to scores page to view the feedback.
Zero weighted scores should be displayed in gray to the logged in user.

'''Test 2: Information button.''' 
Login as an author in expertiza and go to scores page to view the feedback.
The information button should explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade.”)

'''Step 4'''. The mentor evaluates the final submission and give grades.

'''Test 3: Display non-zero weighted scores''' 
Login as an author in expertiza and go to scores page to view the feedback.
The scores shown for the assignment should be the weighted average of the scores.
The zero weighted scores have to be replaced with non-zero weighted scores.

'''Step 5''': The author updates the work and re-submits it again.

'''Test 4: Review Update''' 
The Review should be reopened. The reviewer should be able to go and update the submitted review.
If the review is not updated the previous review score has to be retained.

'''Test 5: Notify reviewers''' 
The notify button on submission page should send a notification to all the corresponding reviewers to revisit the submission and update the reviews.

CSC/ECE 517 Fall 2019 - E1975. Generalize Review Versioning

2019-11-12T04:22:28Z

Ajain37: /* Profile Email Preference */

__TOC__

== '''About Expertiza''' ==

Expertiza is an open source project based on Ruby on Rails framework and the code is available on Github. Expertiza allows the instructor to create new assignments as well as edit new or existing assignments. Instructors can also create a list of topics the students can sign up for and specify deadlines for completion of various tasks. Students can form teams in Expertiza to work on various projects and assignments as well as peer review other students' submissions. Expertiza supports submission across various document types, including the URLs Wiki pages.
 
 

== '''Introduction''' ==
Expertiza is a peer-review based learning platform in which work-products by students are evaluated and scored by peer-review of other students. Review process occurs in stages. After submission of initial feedback for a work-product by the peer-group, students are allowed time to incorporate the comments and improve their work. This second version of the work product is reviewed again and scores are given. This process might be repeated again and the average score from the last review stage is considered as the final score for that work-product. This incremental development of work-products and progressive learning is the fundamental concept underlying the Expertiza system.

=====Purpose=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The system also triggers an email to reviewer when there is a change in the submission. The scores are calculated on the fly based on the rubrics.

=====Scope=====
This project will enhance the existing scoring and peer review functionality and the underlying implementation along with the testing of the existing implementation. Testing of the score calculation, display for the weighted and non-weighted scores based on the combination of the rubrics are part of this project. This project also includes an additional feature for the author to notify the reviewers for the change in the submission.

=====Background=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The scores are calculated based on the rubrics defined in the system. There’s already an existing implementation for the computation of score but it doesn’t work for multipart rubric

=====Documentation=====
The documents that will be generated for this project are the design document, the actual files that we edit or create, and a ReadMe that explains what we did and how to use the modified product.

=====Standards=====
The new standards that will be adhered to in this system are those given for the rails framework, and the code should follow rails and object oriented design principles. Any newly added code will adhere to the guidelines mentioned below:

● https://github.com/rubocop-hq/rails-style-guide 
● http://wiki.expertiza.ncsu.edu/index.php/Object-Oriented_Design_and_Programming.

== '''System Requirements''' ==
=====Problem=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The functionality needs to be tested. It works in some cases, but we are suspicious that it is not entirely correct.

'''Profile Email Preference''':
User in expertiza has liberty to control what kind of emails they should receive. As the reviewers will receive an email for the change in submission. This can be controlled by selecting an option (When someone else submits work I am assigned to review) on the profile page of the user. As the checkboxes are far away on the right corner of the screen, the position of checkboxes need to be changed to make it more user friendly.
 

Scores for students in Expertiza can be based on a combination of rubrics: review rubrics for each round of review, author feedback, and teammate review. In CSC/ECE 517, we don’t really use the student-assigned scores to grade projects, but the peer-review score shown to the student is based entirely on the second round of review 

Here is how scores are calculated: 

1. If only zero-weighted scores are available (here, in the first round of review), the average of all zero-weighted scores should be shown in grey font. An information button should explain why the scores are shown in grey (“The reviews submitted so far do not count toward the author’s final grade.”): 

2. If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is:: 
80%⨉90% + 100%⨉10% = 82%

3. If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. '''However, the previous review score that was given in the current round will count until the reviewer updates the submitted review'''. '''This functionality doesn’t work for multipart rubrics.''' 

The system also doesn’t generate emails to reviewers in case a new submission is made by the author without changing submission parameters like submission URL. Eg: A new submission is made by making a new deployment. But the deployment URL doesn’t change. In this case as well, without changing the current flow the author should have an option to notify all the reviewers to revisit the submission.

=====Interface Requirement=====
The overall user interface would remain the same as of the current system. Since this is a modification to the current submission page, there will be a new addition to the submission page. On the submission page, there will be a new button to notify reviewers when there is a submission which doesn’t contain a change to submission parameters like submission URL, but the deployment has been updated on the same URL. Other than that the interface remains the same for the score page, only functionalities will be tested for multi-part rubrics.

=====Quality Requirements=====
The interface changes are minimal and very easy to adapt. The new button will be clearly visible and is supposed to have a clear text to interpret its functionality. The proposed changes should not affect the current system functionality in any way. The solution is only intended to enrich the project features.

=====Portability Requirements=====
All of the changes for this enhancement to Expertiza should not interfere with the platform portability in any way.

=====Performance Requirements=====
Adding test for multi-part rubric functionality should not cause any overhead in the performance of the system. Sending emails through “Notify All Reviewers” button should affect the system minimally in terms of performance. The only overhead is a database query to get all the reviewers. A good database design should incorporate such a query smoothly.

=====Assumptions=====
The functionality should work in case the assignment rubric is not multi-part.

== '''Proposed Solution''' ==
=====Zero-weighted scores=====
If only zero-weighted scores are available in the first round of review, the average of all zero-weighted scores will be shown in a gray font. An information button will explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade”)

Inside compute_total_score method in on_the_fly_calc.rb module, we'll check if the weighted score exists, if not then we'll calculate the non weighted score. We'll also have to add a method in the questionnaire model to get non weighted score. Few changes are required for the UI to display non-weighted score in grey with information icon and non weighted score in black.

Possible files to be modified.

models/on_the_fly_calc.rb 
models/questionnaire.rb 
models/view_team.html.erb

===== Non-zero weighted scores=====
If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is 80%⨉90% + 100%⨉10% = 82%

We'll first write the test cases by adding test cases to the '''spec/models/on_the_fly_calc_spec.rb'''

Also, we'll test this manually by computing the score and if there's a bug then below are the possible files to be modified.

models/on_the_fly_calc.rb 
models/questionnarie.rb

=====Review Update=====
If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review. This functionality already exists in the existing system. However, In some submissions, there’s no change in the submitted URL’s so there’s no way to let the reviewers know that the author has made some changes in the project. We’ll add a notify button for the Author in the UI which will trigger an email to the reviewers. This will reopen the review, and then the reviewer can go in and update the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review

'''Sending an email when the submission parameters don’t change:'''

A new button will be added on the submission page. The button will enable the author to notify all the reviewers to revisit the submission. The email will be sent to all the assigned reviewers only if their email preference (When someone else submits work I am assigned to review) in the profile is set to receive the email.

=====Profile Email Preference=====
The checkboxes are moved to the left corner. As the layout of the options have used table layout. So, we'll increase the width of the the column which includes checkboxes. We'll be using the percentage instead of fixed value to give the width, as this would adjust the column size based on the screen size. The CSS can be applied to the '''layout.scss''' file to using the appropriate class given to the column 

[[File:E1975_CHECKBOX_1.jpg]]

 
[[File:E1975 .jpg]]

=='''Test Plan'''==

=====Rspec=====

New test cases will be added to test the functionalities which are not working as expected. Also the test cases for the additional features and changes in the system. Test cases will be added to these files:

spec/models/on_the_fly_calc_spec.rb 
spec/models/questionnaire_spec.rb 

=====Email Preferences UI Changes=====

The email preferences checkboxes at present appear sticking to the rightmost border of the page. By following the requirements the checkboxes will be made available closer to the text fields to make them look more user-friendly.
The changes can be tested by logging into the system and on the user profile page.

=====Manual Testing=====

'''Step 1'''. The instructor will create an assignment and assign the zero and non-zero weights to the rubrics. 
'''Step 2'''. The author submits the work done in expertiza. 
'''Step 3'''. Peers check the submitted work and give reviews for it. 

'''Test 1: Zero weighted scores shown in gray.''' 
Login as an author in expertiza and go to scores page to view the feedback.
Zero weighted scores should be displayed in gray to the logged in user.

'''Test 2: Information button.''' 
Login as an author in expertiza and go to scores page to view the feedback.
The information button should explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade.”)

'''Step 4'''. The mentor evaluates the final submission and give grades.

'''Test 3: Display non-zero weighted scores''' 
Login as an author in expertiza and go to scores page to view the feedback.
The scores shown for the assignment should be the weighted average of the scores.
The zero weighted scores have to be replaced with non-zero weighted scores.

'''Step 5''': The author updates the work and re-submits it again.

'''Test 4: Review Update''' 
The Review should be reopened. The reviewer should be able to go and update the submitted review.
If the review is not updated the previous review score has to be retained.

'''Test 5: Notify reviewers''' 
The notify button on submission page should send a notification to all the corresponding reviewers to revisit the submission and update the reviews.

File:E1975 .jpg

2019-11-12T04:21:45Z

Ajain37:

CSC/ECE 517 Fall 2019 - E1975. Generalize Review Versioning

2019-11-12T04:14:30Z

Ajain37: /* Review Update */

__TOC__

== '''About Expertiza''' ==

Expertiza is an open source project based on Ruby on Rails framework and the code is available on Github. Expertiza allows the instructor to create new assignments as well as edit new or existing assignments. Instructors can also create a list of topics the students can sign up for and specify deadlines for completion of various tasks. Students can form teams in Expertiza to work on various projects and assignments as well as peer review other students' submissions. Expertiza supports submission across various document types, including the URLs Wiki pages.
 
 

== '''Introduction''' ==
Expertiza is a peer-review based learning platform in which work-products by students are evaluated and scored by peer-review of other students. Review process occurs in stages. After submission of initial feedback for a work-product by the peer-group, students are allowed time to incorporate the comments and improve their work. This second version of the work product is reviewed again and scores are given. This process might be repeated again and the average score from the last review stage is considered as the final score for that work-product. This incremental development of work-products and progressive learning is the fundamental concept underlying the Expertiza system.

=====Purpose=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The system also triggers an email to reviewer when there is a change in the submission. The scores are calculated on the fly based on the rubrics.

=====Scope=====
This project will enhance the existing scoring and peer review functionality and the underlying implementation along with the testing of the existing implementation. Testing of the score calculation, display for the weighted and non-weighted scores based on the combination of the rubrics are part of this project. This project also includes an additional feature for the author to notify the reviewers for the change in the submission.

=====Background=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The scores are calculated based on the rubrics defined in the system. There’s already an existing implementation for the computation of score but it doesn’t work for multipart rubric

=====Documentation=====
The documents that will be generated for this project are the design document, the actual files that we edit or create, and a ReadMe that explains what we did and how to use the modified product.

=====Standards=====
The new standards that will be adhered to in this system are those given for the rails framework, and the code should follow rails and object oriented design principles. Any newly added code will adhere to the guidelines mentioned below:

● https://github.com/rubocop-hq/rails-style-guide 
● http://wiki.expertiza.ncsu.edu/index.php/Object-Oriented_Design_and_Programming.

== '''System Requirements''' ==
=====Problem=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The functionality needs to be tested. It works in some cases, but we are suspicious that it is not entirely correct.

'''Profile Email Preference''':
User in expertiza has liberty to control what kind of emails they should receive. As the reviewers will receive an email for the change in submission. This can be controlled by selecting an option (When someone else submits work I am assigned to review) on the profile page of the user. As the checkboxes are far away on the right corner of the screen, the position of checkboxes need to be changed to make it more user friendly.
 

Scores for students in Expertiza can be based on a combination of rubrics: review rubrics for each round of review, author feedback, and teammate review. In CSC/ECE 517, we don’t really use the student-assigned scores to grade projects, but the peer-review score shown to the student is based entirely on the second round of review 

Here is how scores are calculated: 

1. If only zero-weighted scores are available (here, in the first round of review), the average of all zero-weighted scores should be shown in grey font. An information button should explain why the scores are shown in grey (“The reviews submitted so far do not count toward the author’s final grade.”): 

2. If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is:: 
80%⨉90% + 100%⨉10% = 82%

3. If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. '''However, the previous review score that was given in the current round will count until the reviewer updates the submitted review'''. '''This functionality doesn’t work for multipart rubrics.''' 

The system also doesn’t generate emails to reviewers in case a new submission is made by the author without changing submission parameters like submission URL. Eg: A new submission is made by making a new deployment. But the deployment URL doesn’t change. In this case as well, without changing the current flow the author should have an option to notify all the reviewers to revisit the submission.

=====Interface Requirement=====
The overall user interface would remain the same as of the current system. Since this is a modification to the current submission page, there will be a new addition to the submission page. On the submission page, there will be a new button to notify reviewers when there is a submission which doesn’t contain a change to submission parameters like submission URL, but the deployment has been updated on the same URL. Other than that the interface remains the same for the score page, only functionalities will be tested for multi-part rubrics.

=====Quality Requirements=====
The interface changes are minimal and very easy to adapt. The new button will be clearly visible and is supposed to have a clear text to interpret its functionality. The proposed changes should not affect the current system functionality in any way. The solution is only intended to enrich the project features.

=====Portability Requirements=====
All of the changes for this enhancement to Expertiza should not interfere with the platform portability in any way.

=====Performance Requirements=====
Adding test for multi-part rubric functionality should not cause any overhead in the performance of the system. Sending emails through “Notify All Reviewers” button should affect the system minimally in terms of performance. The only overhead is a database query to get all the reviewers. A good database design should incorporate such a query smoothly.

=====Assumptions=====
The functionality should work in case the assignment rubric is not multi-part.

== '''Proposed Solution''' ==
=====Zero-weighted scores=====
If only zero-weighted scores are available in the first round of review, the average of all zero-weighted scores will be shown in a gray font. An information button will explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade”)

Inside compute_total_score method in on_the_fly_calc.rb module, we'll check if the weighted score exists, if not then we'll calculate the non weighted score. We'll also have to add a method in the questionnaire model to get non weighted score. Few changes are required for the UI to display non-weighted score in grey with information icon and non weighted score in black.

Possible files to be modified.

models/on_the_fly_calc.rb 
models/questionnaire.rb 
models/view_team.html.erb

===== Non-zero weighted scores=====
If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is 80%⨉90% + 100%⨉10% = 82%

We'll first write the test cases by adding test cases to the '''spec/models/on_the_fly_calc_spec.rb'''

Also, we'll test this manually by computing the score and if there's a bug then below are the possible files to be modified.

models/on_the_fly_calc.rb 
models/questionnarie.rb

=====Review Update=====
If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review. This functionality already exists in the existing system. However, In some submissions, there’s no change in the submitted URL’s so there’s no way to let the reviewers know that the author has made some changes in the project. We’ll add a notify button for the Author in the UI which will trigger an email to the reviewers. This will reopen the review, and then the reviewer can go in and update the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review

'''Sending an email when the submission parameters don’t change:'''

A new button will be added on the submission page. The button will enable the author to notify all the reviewers to revisit the submission. The email will be sent to all the assigned reviewers only if their email preference (When someone else submits work I am assigned to review) in the profile is set to receive the email.

=====Profile Email Preference=====
The checkboxes are moved to the left corner. As the layout of the options have used table layout. So, we'll increase the width of the the column which includes checkboxes. We'll be using the percentage instead of fixed value to give the width, as this would adjust the column size based on the screen size. The CSS can be applied to the '''layout.scss''' file to using the appropriate class given to the column 

[[File:E1975_CHECKBOX_1.jpg]]

=='''Test Plan'''==

=====Rspec=====

New test cases will be added to test the functionalities which are not working as expected. Also the test cases for the additional features and changes in the system. Test cases will be added to these files:

spec/models/on_the_fly_calc_spec.rb 
spec/models/questionnaire_spec.rb 

=====Email Preferences UI Changes=====

The email preferences checkboxes at present appear sticking to the rightmost border of the page. By following the requirements the checkboxes will be made available closer to the text fields to make them look more user-friendly.
The changes can be tested by logging into the system and on the user profile page.

=====Manual Testing=====

'''Step 1'''. The instructor will create an assignment and assign the zero and non-zero weights to the rubrics. 
'''Step 2'''. The author submits the work done in expertiza. 
'''Step 3'''. Peers check the submitted work and give reviews for it. 

'''Test 1: Zero weighted scores shown in gray.''' 
Login as an author in expertiza and go to scores page to view the feedback.
Zero weighted scores should be displayed in gray to the logged in user.

'''Test 2: Information button.''' 
Login as an author in expertiza and go to scores page to view the feedback.
The information button should explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade.”)

'''Step 4'''. The mentor evaluates the final submission and give grades.

'''Test 3: Display non-zero weighted scores''' 
Login as an author in expertiza and go to scores page to view the feedback.
The scores shown for the assignment should be the weighted average of the scores.
The zero weighted scores have to be replaced with non-zero weighted scores.

'''Step 5''': The author updates the work and re-submits it again.

'''Test 4: Review Update''' 
If author updates the submission before the end of current round. The reviewer should be able to go and update the submitted review.
If the review is not updated the previous review score has to be retained.

'''Test 5: Notify reviewers''' 
The notify button on submission page should send a notification to all the corresponding reviewers to revisit the submission and update the reviews.

CSC/ECE 517 Fall 2019 - E1975. Generalize Review Versioning

2019-11-12T04:09:40Z

Ajain37: /* Propose Solution */

__TOC__

== '''About Expertiza''' ==

Expertiza is an open source project based on Ruby on Rails framework and the code is available on Github. Expertiza allows the instructor to create new assignments as well as edit new or existing assignments. Instructors can also create a list of topics the students can sign up for and specify deadlines for completion of various tasks. Students can form teams in Expertiza to work on various projects and assignments as well as peer review other students' submissions. Expertiza supports submission across various document types, including the URLs Wiki pages.
 
 

== '''Introduction''' ==
Expertiza is a peer-review based learning platform in which work-products by students are evaluated and scored by peer-review of other students. Review process occurs in stages. After submission of initial feedback for a work-product by the peer-group, students are allowed time to incorporate the comments and improve their work. This second version of the work product is reviewed again and scores are given. This process might be repeated again and the average score from the last review stage is considered as the final score for that work-product. This incremental development of work-products and progressive learning is the fundamental concept underlying the Expertiza system.

=====Purpose=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The system also triggers an email to reviewer when there is a change in the submission. The scores are calculated on the fly based on the rubrics.

=====Scope=====
This project will enhance the existing scoring and peer review functionality and the underlying implementation along with the testing of the existing implementation. Testing of the score calculation, display for the weighted and non-weighted scores based on the combination of the rubrics are part of this project. This project also includes an additional feature for the author to notify the reviewers for the change in the submission.

=====Background=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The scores are calculated based on the rubrics defined in the system. There’s already an existing implementation for the computation of score but it doesn’t work for multipart rubric

=====Documentation=====
The documents that will be generated for this project are the design document, the actual files that we edit or create, and a ReadMe that explains what we did and how to use the modified product.

=====Standards=====
The new standards that will be adhered to in this system are those given for the rails framework, and the code should follow rails and object oriented design principles. Any newly added code will adhere to the guidelines mentioned below:

● https://github.com/rubocop-hq/rails-style-guide 
● http://wiki.expertiza.ncsu.edu/index.php/Object-Oriented_Design_and_Programming.

== '''System Requirements''' ==
=====Problem=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The functionality needs to be tested. It works in some cases, but we are suspicious that it is not entirely correct.

'''Profile Email Preference''':
User in expertiza has liberty to control what kind of emails they should receive. As the reviewers will receive an email for the change in submission. This can be controlled by selecting an option (When someone else submits work I am assigned to review) on the profile page of the user. As the checkboxes are far away on the right corner of the screen, the position of checkboxes need to be changed to make it more user friendly.
 

Scores for students in Expertiza can be based on a combination of rubrics: review rubrics for each round of review, author feedback, and teammate review. In CSC/ECE 517, we don’t really use the student-assigned scores to grade projects, but the peer-review score shown to the student is based entirely on the second round of review 

Here is how scores are calculated: 

1. If only zero-weighted scores are available (here, in the first round of review), the average of all zero-weighted scores should be shown in grey font. An information button should explain why the scores are shown in grey (“The reviews submitted so far do not count toward the author’s final grade.”): 

2. If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is:: 
80%⨉90% + 100%⨉10% = 82%

3. If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. '''However, the previous review score that was given in the current round will count until the reviewer updates the submitted review'''. '''This functionality doesn’t work for multipart rubrics.''' 

The system also doesn’t generate emails to reviewers in case a new submission is made by the author without changing submission parameters like submission URL. Eg: A new submission is made by making a new deployment. But the deployment URL doesn’t change. In this case as well, without changing the current flow the author should have an option to notify all the reviewers to revisit the submission.

=====Interface Requirement=====
The overall user interface would remain the same as of the current system. Since this is a modification to the current submission page, there will be a new addition to the submission page. On the submission page, there will be a new button to notify reviewers when there is a submission which doesn’t contain a change to submission parameters like submission URL, but the deployment has been updated on the same URL. Other than that the interface remains the same for the score page, only functionalities will be tested for multi-part rubrics.

=====Quality Requirements=====
The interface changes are minimal and very easy to adapt. The new button will be clearly visible and is supposed to have a clear text to interpret its functionality. The proposed changes should not affect the current system functionality in any way. The solution is only intended to enrich the project features.

=====Portability Requirements=====
All of the changes for this enhancement to Expertiza should not interfere with the platform portability in any way.

=====Performance Requirements=====
Adding test for multi-part rubric functionality should not cause any overhead in the performance of the system. Sending emails through “Notify All Reviewers” button should affect the system minimally in terms of performance. The only overhead is a database query to get all the reviewers. A good database design should incorporate such a query smoothly.

=====Assumptions=====
The functionality should work in case the assignment rubric is not multi-part.

== '''Proposed Solution''' ==
=====Zero-weighted scores=====
If only zero-weighted scores are available in the first round of review, the average of all zero-weighted scores will be shown in a gray font. An information button will explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade”)

Inside compute_total_score method in on_the_fly_calc.rb module, we'll check if the weighted score exists, if not then we'll calculate the non weighted score. We'll also have to add a method in the questionnaire model to get non weighted score. Few changes are required for the UI to display non-weighted score in grey with information icon and non weighted score in black.

Possible files to be modified.

models/on_the_fly_calc.rb 
models/questionnaire.rb 
models/view_team.html.erb

===== Non-zero weighted scores=====
If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is 80%⨉90% + 100%⨉10% = 82%

We'll first write the test cases by adding test cases to the '''spec/models/on_the_fly_calc_spec.rb'''

Also, we'll test this manually by computing the score and if there's a bug then below are the possible files to be modified.

models/on_the_fly_calc.rb 
models/questionnarie.rb

=====Review Update=====
If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review. This functionality already exists in the existing system. However, In some submissions, there’s no change in the submitted URL’s so there’s no way to let the reviewers know that the author has made some changes in the project. We’ll add a notify button for the Author in the UI which will trigger an email to the reviewers. This will reopen the review, and then the reviewer can go in and update the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review

'''Sending an email when the submission parameters don’t change:'''

A new button will be added on the submission page. The button will enable the author to notify all the reviewers to revisit the submission. We could also add a custom message in the
mail body so that the authors know the specific reason as to why they received this particular mail.

=====Profile Email Preference=====
The checkboxes are moved to the left corner. As the layout of the options have used table layout. So, we'll increase the width of the the column which includes checkboxes. We'll be using the percentage instead of fixed value to give the width, as this would adjust the column size based on the screen size. The CSS can be applied to the '''layout.scss''' file to using the appropriate class given to the column 

[[File:E1975_CHECKBOX_1.jpg]]

=='''Test Plan'''==

=====Rspec=====

New test cases will be added to test the functionalities which are not working as expected. Also the test cases for the additional features and changes in the system. Test cases will be added to these files:

spec/models/on_the_fly_calc_spec.rb 
spec/models/questionnaire_spec.rb 

=====Email Preferences UI Changes=====

The email preferences checkboxes at present appear sticking to the rightmost border of the page. By following the requirements the checkboxes will be made available closer to the text fields to make them look more user-friendly.
The changes can be tested by logging into the system and on the user profile page.

=====Manual Testing=====

'''Step 1'''. The instructor will create an assignment and assign the zero and non-zero weights to the rubrics. 
'''Step 2'''. The author submits the work done in expertiza. 
'''Step 3'''. Peers check the submitted work and give reviews for it. 

'''Test 1: Zero weighted scores shown in gray.''' 
Login as an author in expertiza and go to scores page to view the feedback.
Zero weighted scores should be displayed in gray to the logged in user.

'''Test 2: Information button.''' 
Login as an author in expertiza and go to scores page to view the feedback.
The information button should explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade.”)

'''Step 4'''. The mentor evaluates the final submission and give grades.

'''Test 3: Display non-zero weighted scores''' 
Login as an author in expertiza and go to scores page to view the feedback.
The scores shown for the assignment should be the weighted average of the scores.
The zero weighted scores have to be replaced with non-zero weighted scores.

'''Step 5''': The author updates the work and re-submits it again.

'''Test 4: Review Update''' 
If author updates the submission before the end of current round. The reviewer should be able to go and update the submitted review.
If the review is not updated the previous review score has to be retained.

'''Test 5: Notify reviewers''' 
The notify button on submission page should send a notification to all the corresponding reviewers to revisit the submission and update the reviews.

CSC/ECE 517 Fall 2019 - E1975. Generalize Review Versioning

2019-11-12T04:06:14Z

Ajain37: /* Manual Testing */

__TOC__

== '''About Expertiza''' ==

Expertiza is an open source project based on Ruby on Rails framework and the code is available on Github. Expertiza allows the instructor to create new assignments as well as edit new or existing assignments. Instructors can also create a list of topics the students can sign up for and specify deadlines for completion of various tasks. Students can form teams in Expertiza to work on various projects and assignments as well as peer review other students' submissions. Expertiza supports submission across various document types, including the URLs Wiki pages.
 
 

== '''Introduction''' ==
Expertiza is a peer-review based learning platform in which work-products by students are evaluated and scored by peer-review of other students. Review process occurs in stages. After submission of initial feedback for a work-product by the peer-group, students are allowed time to incorporate the comments and improve their work. This second version of the work product is reviewed again and scores are given. This process might be repeated again and the average score from the last review stage is considered as the final score for that work-product. This incremental development of work-products and progressive learning is the fundamental concept underlying the Expertiza system.

=====Purpose=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The system also triggers an email to reviewer when there is a change in the submission. The scores are calculated on the fly based on the rubrics.

=====Scope=====
This project will enhance the existing scoring and peer review functionality and the underlying implementation along with the testing of the existing implementation. Testing of the score calculation, display for the weighted and non-weighted scores based on the combination of the rubrics are part of this project. This project also includes an additional feature for the author to notify the reviewers for the change in the submission.

=====Background=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The scores are calculated based on the rubrics defined in the system. There’s already an existing implementation for the computation of score but it doesn’t work for multipart rubric

=====Documentation=====
The documents that will be generated for this project are the design document, the actual files that we edit or create, and a ReadMe that explains what we did and how to use the modified product.

=====Standards=====
The new standards that will be adhered to in this system are those given for the rails framework, and the code should follow rails and object oriented design principles. Any newly added code will adhere to the guidelines mentioned below:

● https://github.com/rubocop-hq/rails-style-guide 
● http://wiki.expertiza.ncsu.edu/index.php/Object-Oriented_Design_and_Programming.

== '''System Requirements''' ==
=====Problem=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The functionality needs to be tested. It works in some cases, but we are suspicious that it is not entirely correct.

'''Profile Email Preference''':
User in expertiza has liberty to control what kind of emails they should receive. As the reviewers will receive an email for the change in submission. This can be controlled by selecting an option (When someone else submits work I am assigned to review) on the profile page of the user. As the checkboxes are far away on the right corner of the screen, the position of checkboxes need to be changed to make it more user friendly.
 

Scores for students in Expertiza can be based on a combination of rubrics: review rubrics for each round of review, author feedback, and teammate review. In CSC/ECE 517, we don’t really use the student-assigned scores to grade projects, but the peer-review score shown to the student is based entirely on the second round of review 

Here is how scores are calculated: 

1. If only zero-weighted scores are available (here, in the first round of review), the average of all zero-weighted scores should be shown in grey font. An information button should explain why the scores are shown in grey (“The reviews submitted so far do not count toward the author’s final grade.”): 

2. If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is:: 
80%⨉90% + 100%⨉10% = 82%

3. If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. '''However, the previous review score that was given in the current round will count until the reviewer updates the submitted review'''. '''This functionality doesn’t work for multipart rubrics.''' 

The system also doesn’t generate emails to reviewers in case a new submission is made by the author without changing submission parameters like submission URL. Eg: A new submission is made by making a new deployment. But the deployment URL doesn’t change. In this case as well, without changing the current flow the author should have an option to notify all the reviewers to revisit the submission.

=====Interface Requirement=====
The overall user interface would remain the same as of the current system. Since this is a modification to the current submission page, there will be a new addition to the submission page. On the submission page, there will be a new button to notify reviewers when there is a submission which doesn’t contain a change to submission parameters like submission URL, but the deployment has been updated on the same URL. Other than that the interface remains the same for the score page, only functionalities will be tested for multi-part rubrics.

=====Quality Requirements=====
The interface changes are minimal and very easy to adapt. The new button will be clearly visible and is supposed to have a clear text to interpret its functionality. The proposed changes should not affect the current system functionality in any way. The solution is only intended to enrich the project features.

=====Portability Requirements=====
All of the changes for this enhancement to Expertiza should not interfere with the platform portability in any way.

=====Performance Requirements=====
Adding test for multi-part rubric functionality should not cause any overhead in the performance of the system. Sending emails through “Notify All Reviewers” button should affect the system minimally in terms of performance. The only overhead is a database query to get all the reviewers. A good database design should incorporate such a query smoothly.

=====Assumptions=====
The functionality should work in case the assignment rubric is not multi-part.

== '''Propose Solution''' ==
=====Zero-weighted scores=====
If only zero-weighted scores are available in the first round of review, the average of all zero-weighted scores will be shown in a gray font. An information button will explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade”)

Inside compute_total_score method in on_the_fly_calc.rb module, we'll check if the weighted score exists, if not then we'll calculate the non weighted score. We'll also have to add a method in the questionnaire model to get non weighted score. Few changes are required for the UI to display non-weighted score in grey with information icon and non weighted score in black.

Possible files to be modified.

models/on_the_fly_calc.rb 
models/questionnaire.rb 
models/view_team.html.erb

===== Non-zero weighted scores=====
If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is 80%⨉90% + 100%⨉10% = 82%

We'll first write the test cases by adding test cases to the '''spec/models/on_the_fly_calc_spec.rb'''

Also, we'll test this manually by computing the score and if there's a bug then below are the possible files to be modified.

models/on_the_fly_calc.rb 
models/questionnarie.rb

=====Review Update=====
If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review. This functionality already exists in the existing system. However, In some submissions, there’s no change in the submitted URL’s so there’s no way to let the reviewers know that the author has made some changes in the project. We’ll add a notify button for the Author in the UI which will trigger an email to the reviewers. This will reopen the review, and then the reviewer can go in and update the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review

'''Sending an email when the submission parameters don’t change:'''

A new button will be added on the submission page. The button will enable the author to notify all the reviewers to revisit the submission. We could also add a custom message in the
mail body so that the authors know the specific reason as to why they received this particular mail.

=====Profile Email Preference=====
The checkboxes are moved to the left corner. As the layout of the options have used table layout. So, we'll increase the width of the the column which includes checkboxes. We'll be using the percentage instead of fixed value to give the width, as this would adjust the column size based on the screen size. The CSS can be applied to the '''layout.scss''' file to using the appropriate class given to the column 

[[File:E1975_CHECKBOX_1.jpg]]

=='''Test Plan'''==

=====Rspec=====

New test cases will be added to test the functionalities which are not working as expected. Also the test cases for the additional features and changes in the system. Test cases will be added to these files:

spec/models/on_the_fly_calc_spec.rb 
spec/models/questionnaire_spec.rb 

=====Email Preferences UI Changes=====

The email preferences checkboxes at present appear sticking to the rightmost border of the page. By following the requirements the checkboxes will be made available closer to the text fields to make them look more user-friendly.
The changes can be tested by logging into the system and on the user profile page.

=====Manual Testing=====

'''Step 1'''. The instructor will create an assignment and assign the zero and non-zero weights to the rubrics. 
'''Step 2'''. The author submits the work done in expertiza. 
'''Step 3'''. Peers check the submitted work and give reviews for it. 

'''Test 1: Zero weighted scores shown in gray.''' 
Login as an author in expertiza and go to scores page to view the feedback.
Zero weighted scores should be displayed in gray to the logged in user.

'''Test 2: Information button.''' 
Login as an author in expertiza and go to scores page to view the feedback.
The information button should explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade.”)

'''Step 4'''. The mentor evaluates the final submission and give grades.

'''Test 3: Display non-zero weighted scores''' 
Login as an author in expertiza and go to scores page to view the feedback.
The scores shown for the assignment should be the weighted average of the scores.
The zero weighted scores have to be replaced with non-zero weighted scores.

'''Step 5''': The author updates the work and re-submits it again.

'''Test 4: Review Update''' 
If author updates the submission before the end of current round. The reviewer should be able to go and update the submitted review.
If the review is not updated the previous review score has to be retained.

'''Test 5: Notify reviewers''' 
The notify button on submission page should send a notification to all the corresponding reviewers to revisit the submission and update the reviews.

CSC/ECE 517 Fall 2019 - E1975. Generalize Review Versioning

2019-11-12T04:05:07Z

Ajain37: /* Manual Testing */

__TOC__

== '''About Expertiza''' ==

Expertiza is an open source project based on Ruby on Rails framework and the code is available on Github. Expertiza allows the instructor to create new assignments as well as edit new or existing assignments. Instructors can also create a list of topics the students can sign up for and specify deadlines for completion of various tasks. Students can form teams in Expertiza to work on various projects and assignments as well as peer review other students' submissions. Expertiza supports submission across various document types, including the URLs Wiki pages.
 
 

== '''Introduction''' ==
Expertiza is a peer-review based learning platform in which work-products by students are evaluated and scored by peer-review of other students. Review process occurs in stages. After submission of initial feedback for a work-product by the peer-group, students are allowed time to incorporate the comments and improve their work. This second version of the work product is reviewed again and scores are given. This process might be repeated again and the average score from the last review stage is considered as the final score for that work-product. This incremental development of work-products and progressive learning is the fundamental concept underlying the Expertiza system.

=====Purpose=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The system also triggers an email to reviewer when there is a change in the submission. The scores are calculated on the fly based on the rubrics.

=====Scope=====
This project will enhance the existing scoring and peer review functionality and the underlying implementation along with the testing of the existing implementation. Testing of the score calculation, display for the weighted and non-weighted scores based on the combination of the rubrics are part of this project. This project also includes an additional feature for the author to notify the reviewers for the change in the submission.

=====Background=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The scores are calculated based on the rubrics defined in the system. There’s already an existing implementation for the computation of score but it doesn’t work for multipart rubric

=====Documentation=====
The documents that will be generated for this project are the design document, the actual files that we edit or create, and a ReadMe that explains what we did and how to use the modified product.

=====Standards=====
The new standards that will be adhered to in this system are those given for the rails framework, and the code should follow rails and object oriented design principles. Any newly added code will adhere to the guidelines mentioned below:

● https://github.com/rubocop-hq/rails-style-guide 
● http://wiki.expertiza.ncsu.edu/index.php/Object-Oriented_Design_and_Programming.

== '''System Requirements''' ==
=====Problem=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The functionality needs to be tested. It works in some cases, but we are suspicious that it is not entirely correct.

'''Profile Email Preference''':
User in expertiza has liberty to control what kind of emails they should receive. As the reviewers will receive an email for the change in submission. This can be controlled by selecting an option (When someone else submits work I am assigned to review) on the profile page of the user. As the checkboxes are far away on the right corner of the screen, the position of checkboxes need to be changed to make it more user friendly.
 

Scores for students in Expertiza can be based on a combination of rubrics: review rubrics for each round of review, author feedback, and teammate review. In CSC/ECE 517, we don’t really use the student-assigned scores to grade projects, but the peer-review score shown to the student is based entirely on the second round of review 

Here is how scores are calculated: 

1. If only zero-weighted scores are available (here, in the first round of review), the average of all zero-weighted scores should be shown in grey font. An information button should explain why the scores are shown in grey (“The reviews submitted so far do not count toward the author’s final grade.”): 

2. If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is:: 
80%⨉90% + 100%⨉10% = 82%

3. If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. '''However, the previous review score that was given in the current round will count until the reviewer updates the submitted review'''. '''This functionality doesn’t work for multipart rubrics.''' 

The system also doesn’t generate emails to reviewers in case a new submission is made by the author without changing submission parameters like submission URL. Eg: A new submission is made by making a new deployment. But the deployment URL doesn’t change. In this case as well, without changing the current flow the author should have an option to notify all the reviewers to revisit the submission.

=====Interface Requirement=====
The overall user interface would remain the same as of the current system. Since this is a modification to the current submission page, there will be a new addition to the submission page. On the submission page, there will be a new button to notify reviewers when there is a submission which doesn’t contain a change to submission parameters like submission URL, but the deployment has been updated on the same URL. Other than that the interface remains the same for the score page, only functionalities will be tested for multi-part rubrics.

=====Quality Requirements=====
The interface changes are minimal and very easy to adapt. The new button will be clearly visible and is supposed to have a clear text to interpret its functionality. The proposed changes should not affect the current system functionality in any way. The solution is only intended to enrich the project features.

=====Portability Requirements=====
All of the changes for this enhancement to Expertiza should not interfere with the platform portability in any way.

=====Performance Requirements=====
Adding test for multi-part rubric functionality should not cause any overhead in the performance of the system. Sending emails through “Notify All Reviewers” button should affect the system minimally in terms of performance. The only overhead is a database query to get all the reviewers. A good database design should incorporate such a query smoothly.

=====Assumptions=====
The functionality should work in case the assignment rubric is not multi-part.

== '''Propose Solution''' ==
=====Zero-weighted scores=====
If only zero-weighted scores are available in the first round of review, the average of all zero-weighted scores will be shown in a gray font. An information button will explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade”)

Inside compute_total_score method in on_the_fly_calc.rb module, we'll check if the weighted score exists, if not then we'll calculate the non weighted score. We'll also have to add a method in the questionnaire model to get non weighted score. Few changes are required for the UI to display non-weighted score in grey with information icon and non weighted score in black.

Possible files to be modified.

models/on_the_fly_calc.rb 
models/questionnaire.rb 
models/view_team.html.erb

===== Non-zero weighted scores=====
If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is 80%⨉90% + 100%⨉10% = 82%

We'll first write the test cases by adding test cases to the '''spec/models/on_the_fly_calc_spec.rb'''

Also, we'll test this manually by computing the score and if there's a bug then below are the possible files to be modified.

models/on_the_fly_calc.rb 
models/questionnarie.rb

=====Review Update=====
If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review. This functionality already exists in the existing system. However, In some submissions, there’s no change in the submitted URL’s so there’s no way to let the reviewers know that the author has made some changes in the project. We’ll add a notify button for the Author in the UI which will trigger an email to the reviewers. This will reopen the review, and then the reviewer can go in and update the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review

'''Sending an email when the submission parameters don’t change:'''

A new button will be added on the submission page. The button will enable the author to notify all the reviewers to revisit the submission. We could also add a custom message in the
mail body so that the authors know the specific reason as to why they received this particular mail.

=====Profile Email Preference=====
The checkboxes are moved to the left corner. As the layout of the options have used table layout. So, we'll increase the width of the the column which includes checkboxes. We'll be using the percentage instead of fixed value to give the width, as this would adjust the column size based on the screen size. The CSS can be applied to the '''layout.scss''' file to using the appropriate class given to the column 

[[File:E1975_CHECKBOX_1.jpg]]

=='''Test Plan'''==

=====Rspec=====

New test cases will be added to test the functionalities which are not working as expected. Also the test cases for the additional features and changes in the system. Test cases will be added to these files:

spec/models/on_the_fly_calc_spec.rb 
spec/models/questionnaire_spec.rb 

=====Email Preferences UI Changes=====

The email preferences checkboxes at present appear sticking to the rightmost border of the page. By following the requirements the checkboxes will be made available closer to the text fields to make them look more user-friendly.
The changes can be tested by logging into the system and on the user profile page.

=====Manual Testing=====

'''Step 1'''. The instructor will create an assignment and assign the zero and non-zero weights to the rubrics. 
'''Step 2'''. The author submits the work done in expertiza. 
'''Step 3'''. Peers check the submitted work and give reviews for it. 

'''Test 1''': Zero weighted scores shown in gray.
Login as an author in expertiza and go to scores page to view the feedback.
Zero weighted scores should be displayed in gray to the logged in user.

'''Test 2''': Information button.
Login as an author in expertiza and go to scores page to view the feedback.
The information button should explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade.”)

'''Step 4'''. The mentor evaluates the final submission and give grades.

'''Test 3''': Display non-zero weighted scores.
Login as an author in expertiza and go to scores page to view the feedback.
The scores shown for the assignment should be the weighted average of the scores.
The zero weighted scores have to be replaced with non-zero weighted scores.

'''Step 5''': The author updates the work and re-submits it again.

'''Test 4''': Review Update.
If author updates the submission before the end of current round. The reviewer should be able to go and update the submitted review.
If the review is not updated the previous review score has to be retained.

'''Test 5''': notify reviewers
The notify button on submission page should send a notification to all the corresponding reviewers to revisit the submission and update the reviews.

CSC/ECE 517 Fall 2019 - E1975. Generalize Review Versioning

2019-11-12T04:03:31Z

Ajain37: /* Manual Testing */

__TOC__

== '''About Expertiza''' ==

Expertiza is an open source project based on Ruby on Rails framework and the code is available on Github. Expertiza allows the instructor to create new assignments as well as edit new or existing assignments. Instructors can also create a list of topics the students can sign up for and specify deadlines for completion of various tasks. Students can form teams in Expertiza to work on various projects and assignments as well as peer review other students' submissions. Expertiza supports submission across various document types, including the URLs Wiki pages.
 
 

== '''Introduction''' ==
Expertiza is a peer-review based learning platform in which work-products by students are evaluated and scored by peer-review of other students. Review process occurs in stages. After submission of initial feedback for a work-product by the peer-group, students are allowed time to incorporate the comments and improve their work. This second version of the work product is reviewed again and scores are given. This process might be repeated again and the average score from the last review stage is considered as the final score for that work-product. This incremental development of work-products and progressive learning is the fundamental concept underlying the Expertiza system.

=====Purpose=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The system also triggers an email to reviewer when there is a change in the submission. The scores are calculated on the fly based on the rubrics.

=====Scope=====
This project will enhance the existing scoring and peer review functionality and the underlying implementation along with the testing of the existing implementation. Testing of the score calculation, display for the weighted and non-weighted scores based on the combination of the rubrics are part of this project. This project also includes an additional feature for the author to notify the reviewers for the change in the submission.

=====Background=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The scores are calculated based on the rubrics defined in the system. There’s already an existing implementation for the computation of score but it doesn’t work for multipart rubric

=====Documentation=====
The documents that will be generated for this project are the design document, the actual files that we edit or create, and a ReadMe that explains what we did and how to use the modified product.

=====Standards=====
The new standards that will be adhered to in this system are those given for the rails framework, and the code should follow rails and object oriented design principles. Any newly added code will adhere to the guidelines mentioned below:

● https://github.com/rubocop-hq/rails-style-guide 
● http://wiki.expertiza.ncsu.edu/index.php/Object-Oriented_Design_and_Programming.

== '''System Requirements''' ==
=====Problem=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The functionality needs to be tested. It works in some cases, but we are suspicious that it is not entirely correct.

'''Profile Email Preference''':
User in expertiza has liberty to control what kind of emails they should receive. As the reviewers will receive an email for the change in submission. This can be controlled by selecting an option (When someone else submits work I am assigned to review) on the profile page of the user. As the checkboxes are far away on the right corner of the screen, the position of checkboxes need to be changed to make it more user friendly.
 

Scores for students in Expertiza can be based on a combination of rubrics: review rubrics for each round of review, author feedback, and teammate review. In CSC/ECE 517, we don’t really use the student-assigned scores to grade projects, but the peer-review score shown to the student is based entirely on the second round of review 

Here is how scores are calculated: 

1. If only zero-weighted scores are available (here, in the first round of review), the average of all zero-weighted scores should be shown in grey font. An information button should explain why the scores are shown in grey (“The reviews submitted so far do not count toward the author’s final grade.”): 

2. If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is:: 
80%⨉90% + 100%⨉10% = 82%

3. If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. '''However, the previous review score that was given in the current round will count until the reviewer updates the submitted review'''. '''This functionality doesn’t work for multipart rubrics.''' 

The system also doesn’t generate emails to reviewers in case a new submission is made by the author without changing submission parameters like submission URL. Eg: A new submission is made by making a new deployment. But the deployment URL doesn’t change. In this case as well, without changing the current flow the author should have an option to notify all the reviewers to revisit the submission.

=====Interface Requirement=====
The overall user interface would remain the same as of the current system. Since this is a modification to the current submission page, there will be a new addition to the submission page. On the submission page, there will be a new button to notify reviewers when there is a submission which doesn’t contain a change to submission parameters like submission URL, but the deployment has been updated on the same URL. Other than that the interface remains the same for the score page, only functionalities will be tested for multi-part rubrics.

=====Quality Requirements=====
The interface changes are minimal and very easy to adapt. The new button will be clearly visible and is supposed to have a clear text to interpret its functionality. The proposed changes should not affect the current system functionality in any way. The solution is only intended to enrich the project features.

=====Portability Requirements=====
All of the changes for this enhancement to Expertiza should not interfere with the platform portability in any way.

=====Performance Requirements=====
Adding test for multi-part rubric functionality should not cause any overhead in the performance of the system. Sending emails through “Notify All Reviewers” button should affect the system minimally in terms of performance. The only overhead is a database query to get all the reviewers. A good database design should incorporate such a query smoothly.

=====Assumptions=====
The functionality should work in case the assignment rubric is not multi-part.

== '''Propose Solution''' ==
=====Zero-weighted scores=====
If only zero-weighted scores are available in the first round of review, the average of all zero-weighted scores will be shown in a gray font. An information button will explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade”)

Inside compute_total_score method in on_the_fly_calc.rb module, we'll check if the weighted score exists, if not then we'll calculate the non weighted score. We'll also have to add a method in the questionnaire model to get non weighted score. Few changes are required for the UI to display non-weighted score in grey with information icon and non weighted score in black.

Possible files to be modified.

models/on_the_fly_calc.rb 
models/questionnaire.rb 
models/view_team.html.erb

===== Non-zero weighted scores=====
If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is 80%⨉90% + 100%⨉10% = 82%

We'll first write the test cases by adding test cases to the '''spec/models/on_the_fly_calc_spec.rb'''

Also, we'll test this manually by computing the score and if there's a bug then below are the possible files to be modified.

models/on_the_fly_calc.rb 
models/questionnarie.rb

=====Review Update=====
If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review. This functionality already exists in the existing system. However, In some submissions, there’s no change in the submitted URL’s so there’s no way to let the reviewers know that the author has made some changes in the project. We’ll add a notify button for the Author in the UI which will trigger an email to the reviewers. This will reopen the review, and then the reviewer can go in and update the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review

'''Sending an email when the submission parameters don’t change:'''

A new button will be added on the submission page. The button will enable the author to notify all the reviewers to revisit the submission. We could also add a custom message in the
mail body so that the authors know the specific reason as to why they received this particular mail.

=====Profile Email Preference=====
The checkboxes are moved to the left corner. As the layout of the options have used table layout. So, we'll increase the width of the the column which includes checkboxes. We'll be using the percentage instead of fixed value to give the width, as this would adjust the column size based on the screen size. The CSS can be applied to the '''layout.scss''' file to using the appropriate class given to the column 

[[File:E1975_CHECKBOX_1.jpg]]

=='''Test Plan'''==

=====Rspec=====

New test cases will be added to test the functionalities which are not working as expected. Also the test cases for the additional features and changes in the system. Test cases will be added to these files:

spec/models/on_the_fly_calc_spec.rb 
spec/models/questionnaire_spec.rb 

=====Email Preferences UI Changes=====

The email preferences checkboxes at present appear sticking to the rightmost border of the page. By following the requirements the checkboxes will be made available closer to the text fields to make them look more user-friendly.
The changes can be tested by logging into the system and on the user profile page.

=====Manual Testing=====

1. The instructor will create an assignment and assign the zero and non-zero weights to the rubrics.
2. The author submits the work done in expertiza.
3. Peers check the submitted work and give reviews for it.

'''Test 1''': Zero weighted scores shown in gray.
Login as an author in expertiza and go to scores page to view the feedback.
Zero weighted scores should be displayed in gray to the logged in user.

'''Test 2''': Information button.
Login as an author in expertiza and go to scores page to view the feedback.
The information button should explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade.”)

4. The mentor evaluates the final submission and give grades.

'''Test 3''': Display non-zero weighted scores.
Login as an author in expertiza and go to scores page to view the feedback.
The scores shown for the assignment should be the weighted average of the scores.
The zero weighted scores have to be replaced with non-zero weighted scores.

The author updates the work and re-submits it again.

'''Test 4''': Review Update.
If author updates the submission before the end of current round. The reviewer should be able to go and update the submitted review.
If the review is not updated the previous review score has to be retained.

'''Test 5''': notify reviewers
The notify button on submission page should send a notification to all the corresponding reviewers to revisit the submission and update the reviews.

CSC/ECE 517 Fall 2019 - E1975. Generalize Review Versioning

2019-11-12T04:03:08Z

Ajain37: /* Manual Testing */

__TOC__

== '''About Expertiza''' ==

Expertiza is an open source project based on Ruby on Rails framework and the code is available on Github. Expertiza allows the instructor to create new assignments as well as edit new or existing assignments. Instructors can also create a list of topics the students can sign up for and specify deadlines for completion of various tasks. Students can form teams in Expertiza to work on various projects and assignments as well as peer review other students' submissions. Expertiza supports submission across various document types, including the URLs Wiki pages.
 
 

== '''Introduction''' ==
Expertiza is a peer-review based learning platform in which work-products by students are evaluated and scored by peer-review of other students. Review process occurs in stages. After submission of initial feedback for a work-product by the peer-group, students are allowed time to incorporate the comments and improve their work. This second version of the work product is reviewed again and scores are given. This process might be repeated again and the average score from the last review stage is considered as the final score for that work-product. This incremental development of work-products and progressive learning is the fundamental concept underlying the Expertiza system.

=====Purpose=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The system also triggers an email to reviewer when there is a change in the submission. The scores are calculated on the fly based on the rubrics.

=====Scope=====
This project will enhance the existing scoring and peer review functionality and the underlying implementation along with the testing of the existing implementation. Testing of the score calculation, display for the weighted and non-weighted scores based on the combination of the rubrics are part of this project. This project also includes an additional feature for the author to notify the reviewers for the change in the submission.

=====Background=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The scores are calculated based on the rubrics defined in the system. There’s already an existing implementation for the computation of score but it doesn’t work for multipart rubric

=====Documentation=====
The documents that will be generated for this project are the design document, the actual files that we edit or create, and a ReadMe that explains what we did and how to use the modified product.

=====Standards=====
The new standards that will be adhered to in this system are those given for the rails framework, and the code should follow rails and object oriented design principles. Any newly added code will adhere to the guidelines mentioned below:

● https://github.com/rubocop-hq/rails-style-guide 
● http://wiki.expertiza.ncsu.edu/index.php/Object-Oriented_Design_and_Programming.

== '''System Requirements''' ==
=====Problem=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The functionality needs to be tested. It works in some cases, but we are suspicious that it is not entirely correct.

'''Profile Email Preference''':
User in expertiza has liberty to control what kind of emails they should receive. As the reviewers will receive an email for the change in submission. This can be controlled by selecting an option (When someone else submits work I am assigned to review) on the profile page of the user. As the checkboxes are far away on the right corner of the screen, the position of checkboxes need to be changed to make it more user friendly.
 

Scores for students in Expertiza can be based on a combination of rubrics: review rubrics for each round of review, author feedback, and teammate review. In CSC/ECE 517, we don’t really use the student-assigned scores to grade projects, but the peer-review score shown to the student is based entirely on the second round of review 

Here is how scores are calculated: 

1. If only zero-weighted scores are available (here, in the first round of review), the average of all zero-weighted scores should be shown in grey font. An information button should explain why the scores are shown in grey (“The reviews submitted so far do not count toward the author’s final grade.”): 

2. If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is:: 
80%⨉90% + 100%⨉10% = 82%

3. If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. '''However, the previous review score that was given in the current round will count until the reviewer updates the submitted review'''. '''This functionality doesn’t work for multipart rubrics.''' 

The system also doesn’t generate emails to reviewers in case a new submission is made by the author without changing submission parameters like submission URL. Eg: A new submission is made by making a new deployment. But the deployment URL doesn’t change. In this case as well, without changing the current flow the author should have an option to notify all the reviewers to revisit the submission.

=====Interface Requirement=====
The overall user interface would remain the same as of the current system. Since this is a modification to the current submission page, there will be a new addition to the submission page. On the submission page, there will be a new button to notify reviewers when there is a submission which doesn’t contain a change to submission parameters like submission URL, but the deployment has been updated on the same URL. Other than that the interface remains the same for the score page, only functionalities will be tested for multi-part rubrics.

=====Quality Requirements=====
The interface changes are minimal and very easy to adapt. The new button will be clearly visible and is supposed to have a clear text to interpret its functionality. The proposed changes should not affect the current system functionality in any way. The solution is only intended to enrich the project features.

=====Portability Requirements=====
All of the changes for this enhancement to Expertiza should not interfere with the platform portability in any way.

=====Performance Requirements=====
Adding test for multi-part rubric functionality should not cause any overhead in the performance of the system. Sending emails through “Notify All Reviewers” button should affect the system minimally in terms of performance. The only overhead is a database query to get all the reviewers. A good database design should incorporate such a query smoothly.

=====Assumptions=====
The functionality should work in case the assignment rubric is not multi-part.

== '''Propose Solution''' ==
=====Zero-weighted scores=====
If only zero-weighted scores are available in the first round of review, the average of all zero-weighted scores will be shown in a gray font. An information button will explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade”)

Inside compute_total_score method in on_the_fly_calc.rb module, we'll check if the weighted score exists, if not then we'll calculate the non weighted score. We'll also have to add a method in the questionnaire model to get non weighted score. Few changes are required for the UI to display non-weighted score in grey with information icon and non weighted score in black.

Possible files to be modified.

models/on_the_fly_calc.rb 
models/questionnaire.rb 
models/view_team.html.erb

===== Non-zero weighted scores=====
If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is 80%⨉90% + 100%⨉10% = 82%

We'll first write the test cases by adding test cases to the '''spec/models/on_the_fly_calc_spec.rb'''

Also, we'll test this manually by computing the score and if there's a bug then below are the possible files to be modified.

models/on_the_fly_calc.rb 
models/questionnarie.rb

=====Review Update=====
If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review. This functionality already exists in the existing system. However, In some submissions, there’s no change in the submitted URL’s so there’s no way to let the reviewers know that the author has made some changes in the project. We’ll add a notify button for the Author in the UI which will trigger an email to the reviewers. This will reopen the review, and then the reviewer can go in and update the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review

'''Sending an email when the submission parameters don’t change:'''

A new button will be added on the submission page. The button will enable the author to notify all the reviewers to revisit the submission. We could also add a custom message in the
mail body so that the authors know the specific reason as to why they received this particular mail.

=====Profile Email Preference=====
The checkboxes are moved to the left corner. As the layout of the options have used table layout. So, we'll increase the width of the the column which includes checkboxes. We'll be using the percentage instead of fixed value to give the width, as this would adjust the column size based on the screen size. The CSS can be applied to the '''layout.scss''' file to using the appropriate class given to the column 

[[File:E1975_CHECKBOX_1.jpg]]

=='''Test Plan'''==

=====Rspec=====

New test cases will be added to test the functionalities which are not working as expected. Also the test cases for the additional features and changes in the system. Test cases will be added to these files:

spec/models/on_the_fly_calc_spec.rb 
spec/models/questionnaire_spec.rb 

=====Email Preferences UI Changes=====

The email preferences checkboxes at present appear sticking to the rightmost border of the page. By following the requirements the checkboxes will be made available closer to the text fields to make them look more user-friendly.
The changes can be tested by logging into the system and on the user profile page.

=====Manual Testing=====

1. The instructor will create an assignment and assign the zero and non-zero weights to the rubrics.
2. The author submits the work done in expertiza.
3. Peers check the submitted work and give reviews for it.

Test 1: Zero weighted scores shown in gray.
Login as an author in expertiza and go to scores page to view the feedback.
Zero weighted scores should be displayed in gray to the logged in user.

Test 2: Information button.
Login as an author in expertiza and go to scores page to view the feedback.
The information button should explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade.”)

4. The mentor evaluates the final submission and give grades.

Test 3: Display non-zero weighted scores.
Login as an author in expertiza and go to scores page to view the feedback.
The scores shown for the assignment should be the weighted average of the scores.
The zero weighted scores have to be replaced with non-zero weighted scores.

The author updates the work and re-submits it again.

Test 4: Review Update.
If author updates the submission before the end of current round. The reviewer should be able to go and update the submitted review.
If the review is not updated the previous review score has to be retained.

Test 5: notify reviewers
The notify button on submission page should send a notification to all the corresponding reviewers to revisit the submission and update the reviews.

CSC/ECE 517 Fall 2019 - E1975. Generalize Review Versioning

2019-11-12T03:57:54Z

Ajain37: /* Design */

__TOC__

== '''About Expertiza''' ==

Expertiza is an open source project based on Ruby on Rails framework and the code is available on Github. Expertiza allows the instructor to create new assignments as well as edit new or existing assignments. Instructors can also create a list of topics the students can sign up for and specify deadlines for completion of various tasks. Students can form teams in Expertiza to work on various projects and assignments as well as peer review other students' submissions. Expertiza supports submission across various document types, including the URLs Wiki pages.
 
 

== '''Introduction''' ==
Expertiza is a peer-review based learning platform in which work-products by students are evaluated and scored by peer-review of other students. Review process occurs in stages. After submission of initial feedback for a work-product by the peer-group, students are allowed time to incorporate the comments and improve their work. This second version of the work product is reviewed again and scores are given. This process might be repeated again and the average score from the last review stage is considered as the final score for that work-product. This incremental development of work-products and progressive learning is the fundamental concept underlying the Expertiza system.

=====Purpose=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The system also triggers an email to reviewer when there is a change in the submission. The scores are calculated on the fly based on the rubrics.

=====Scope=====
This project will enhance the existing scoring and peer review functionality and the underlying implementation along with the testing of the existing implementation. Testing of the score calculation, display for the weighted and non-weighted scores based on the combination of the rubrics are part of this project. This project also includes an additional feature for the author to notify the reviewers for the change in the submission.

=====Background=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The scores are calculated based on the rubrics defined in the system. There’s already an existing implementation for the computation of score but it doesn’t work for multipart rubric

=====Documentation=====
The documents that will be generated for this project are the design document, the actual files that we edit or create, and a ReadMe that explains what we did and how to use the modified product.

=====Standards=====
The new standards that will be adhered to in this system are those given for the rails framework, and the code should follow rails and object oriented design principles. Any newly added code will adhere to the guidelines mentioned below:

● https://github.com/rubocop-hq/rails-style-guide 
● http://wiki.expertiza.ncsu.edu/index.php/Object-Oriented_Design_and_Programming.

== '''System Requirements''' ==
=====Problem=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The functionality needs to be tested. It works in some cases, but we are suspicious that it is not entirely correct.

'''Profile Email Preference''':
User in expertiza has liberty to control what kind of emails they should receive. As the reviewers will receive an email for the change in submission. This can be controlled by selecting an option (When someone else submits work I am assigned to review) on the profile page of the user. As the checkboxes are far away on the right corner of the screen, the position of checkboxes need to be changed to make it more user friendly.
 

Scores for students in Expertiza can be based on a combination of rubrics: review rubrics for each round of review, author feedback, and teammate review. In CSC/ECE 517, we don’t really use the student-assigned scores to grade projects, but the peer-review score shown to the student is based entirely on the second round of review 

Here is how scores are calculated: 

1. If only zero-weighted scores are available (here, in the first round of review), the average of all zero-weighted scores should be shown in grey font. An information button should explain why the scores are shown in grey (“The reviews submitted so far do not count toward the author’s final grade.”): 

2. If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is:: 
80%⨉90% + 100%⨉10% = 82%

3. If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. '''However, the previous review score that was given in the current round will count until the reviewer updates the submitted review'''. '''This functionality doesn’t work for multipart rubrics.''' 

The system also doesn’t generate emails to reviewers in case a new submission is made by the author without changing submission parameters like submission URL. Eg: A new submission is made by making a new deployment. But the deployment URL doesn’t change. In this case as well, without changing the current flow the author should have an option to notify all the reviewers to revisit the submission.

=====Interface Requirement=====
The overall user interface would remain the same as of the current system. Since this is a modification to the current submission page, there will be a new addition to the submission page. On the submission page, there will be a new button to notify reviewers when there is a submission which doesn’t contain a change to submission parameters like submission URL, but the deployment has been updated on the same URL. Other than that the interface remains the same for the score page, only functionalities will be tested for multi-part rubrics.

=====Quality Requirements=====
The interface changes are minimal and very easy to adapt. The new button will be clearly visible and is supposed to have a clear text to interpret its functionality. The proposed changes should not affect the current system functionality in any way. The solution is only intended to enrich the project features.

=====Portability Requirements=====
All of the changes for this enhancement to Expertiza should not interfere with the platform portability in any way.

=====Performance Requirements=====
Adding test for multi-part rubric functionality should not cause any overhead in the performance of the system. Sending emails through “Notify All Reviewers” button should affect the system minimally in terms of performance. The only overhead is a database query to get all the reviewers. A good database design should incorporate such a query smoothly.

=====Assumptions=====
The functionality should work in case the assignment rubric is not multi-part.

== '''Propose Solution''' ==
=====Zero-weighted scores=====
If only zero-weighted scores are available in the first round of review, the average of all zero-weighted scores will be shown in a gray font. An information button will explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade”)

Inside compute_total_score method in on_the_fly_calc.rb module, we'll check if the weighted score exists, if not then we'll calculate the non weighted score. We'll also have to add a method in the questionnaire model to get non weighted score. Few changes are required for the UI to display non-weighted score in grey with information icon and non weighted score in black.

Possible files to be modified.

models/on_the_fly_calc.rb 
models/questionnaire.rb 
models/view_team.html.erb

===== Non-zero weighted scores=====
If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is 80%⨉90% + 100%⨉10% = 82%

We'll first write the test cases by adding test cases to the '''spec/models/on_the_fly_calc_spec.rb'''

Also, we'll test this manually by computing the score and if there's a bug then below are the possible files to be modified.

models/on_the_fly_calc.rb 
models/questionnarie.rb

=====Review Update=====
If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review. This functionality already exists in the existing system. However, In some submissions, there’s no change in the submitted URL’s so there’s no way to let the reviewers know that the author has made some changes in the project. We’ll add a notify button for the Author in the UI which will trigger an email to the reviewers. This will reopen the review, and then the reviewer can go in and update the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review

'''Sending an email when the submission parameters don’t change:'''

A new button will be added on the submission page. The button will enable the author to notify all the reviewers to revisit the submission. We could also add a custom message in the
mail body so that the authors know the specific reason as to why they received this particular mail.

=====Profile Email Preference=====
The checkboxes are moved to the left corner. As the layout of the options have used table layout. So, we'll increase the width of the the column which includes checkboxes. We'll be using the percentage instead of fixed value to give the width, as this would adjust the column size based on the screen size. The CSS can be applied to the '''layout.scss''' file to using the appropriate class given to the column 

[[File:E1975_CHECKBOX_1.jpg]]

=='''Test Plan'''==

=====Rspec=====

New test cases will be added to test the functionalities which are not working as expected. Also the test cases for the additional features and changes in the system. Test cases will be added to these files:

spec/models/on_the_fly_calc_spec.rb 
spec/models/questionnaire_spec.rb 

=====Email Preferences UI Changes=====

The email preferences checkboxes at present appear sticking to the rightmost border of the page. By following the requirements the checkboxes will be made available closer to the text fields to make them look more user-friendly.
The changes can be tested by logging into the system and on the user profile page.

=====Manual Testing=====

The instructor will create an assignment and assign the zero and non-zero weights to the rubrics.

· Login as an author and go to scores page to view the feedback.

Test – Zero weighted scores shown in gray.

Test – Zero weighted scores shown in gray.

· Zero weighted scores should be displayed in gray to the user.

Test – Information button.

· The information button should explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade.”)

Test – Display non-zero weighted scores.

· The scores shown for the assignment should be the weighted average of the scores, if any non-zero scores are available
· The zero weighted scores have to be replaced with non-zero weighted scores.

Test – Update review score.
· If author updates the submission before the end of current round. The reviewer should be able to go and update the submitted review.
· If the review is not updated the previous review score has to be retained.

Test – Multipart part rubric.
· The functionality should work for multipart rubric.
· The scores calculated have to be accurate.

CSC/ECE 517 Fall 2019 - E1975. Generalize Review Versioning

2019-11-12T03:55:44Z

Ajain37: /* Email Preferences */

__TOC__

== '''About Expertiza''' ==

Expertiza is an open source project based on Ruby on Rails framework and the code is available on Github. Expertiza allows the instructor to create new assignments as well as edit new or existing assignments. Instructors can also create a list of topics the students can sign up for and specify deadlines for completion of various tasks. Students can form teams in Expertiza to work on various projects and assignments as well as peer review other students' submissions. Expertiza supports submission across various document types, including the URLs Wiki pages.
 
 

== '''Introduction''' ==
Expertiza is a peer-review based learning platform in which work-products by students are evaluated and scored by peer-review of other students. Review process occurs in stages. After submission of initial feedback for a work-product by the peer-group, students are allowed time to incorporate the comments and improve their work. This second version of the work product is reviewed again and scores are given. This process might be repeated again and the average score from the last review stage is considered as the final score for that work-product. This incremental development of work-products and progressive learning is the fundamental concept underlying the Expertiza system.

=====Purpose=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The system also triggers an email to reviewer when there is a change in the submission. The scores are calculated on the fly based on the rubrics.

=====Scope=====
This project will enhance the existing scoring and peer review functionality and the underlying implementation along with the testing of the existing implementation. Testing of the score calculation, display for the weighted and non-weighted scores based on the combination of the rubrics are part of this project. This project also includes an additional feature for the author to notify the reviewers for the change in the submission.

=====Background=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The scores are calculated based on the rubrics defined in the system. There’s already an existing implementation for the computation of score but it doesn’t work for multipart rubric

=====Documentation=====
The documents that will be generated for this project are the design document, the actual files that we edit or create, and a ReadMe that explains what we did and how to use the modified product.

=====Standards=====
The new standards that will be adhered to in this system are those given for the rails framework, and the code should follow rails and object oriented design principles. Any newly added code will adhere to the guidelines mentioned below:

● https://github.com/rubocop-hq/rails-style-guide 
● http://wiki.expertiza.ncsu.edu/index.php/Object-Oriented_Design_and_Programming.

== '''System Requirements''' ==
=====Problem=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The functionality needs to be tested. It works in some cases, but we are suspicious that it is not entirely correct.

'''Profile Email Preference''':
User in expertiza has liberty to control what kind of emails they should receive. As the reviewers will receive an email for the change in submission. This can be controlled by selecting an option (When someone else submits work I am assigned to review) on the profile page of the user. As the checkboxes are far away on the right corner of the screen, the position of checkboxes need to be changed to make it more user friendly.
 

Scores for students in Expertiza can be based on a combination of rubrics: review rubrics for each round of review, author feedback, and teammate review. In CSC/ECE 517, we don’t really use the student-assigned scores to grade projects, but the peer-review score shown to the student is based entirely on the second round of review 

Here is how scores are calculated: 

1. If only zero-weighted scores are available (here, in the first round of review), the average of all zero-weighted scores should be shown in grey font. An information button should explain why the scores are shown in grey (“The reviews submitted so far do not count toward the author’s final grade.”): 

2. If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is:: 
80%⨉90% + 100%⨉10% = 82%

3. If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. '''However, the previous review score that was given in the current round will count until the reviewer updates the submitted review'''. '''This functionality doesn’t work for multipart rubrics.''' 

The system also doesn’t generate emails to reviewers in case a new submission is made by the author without changing submission parameters like submission URL. Eg: A new submission is made by making a new deployment. But the deployment URL doesn’t change. In this case as well, without changing the current flow the author should have an option to notify all the reviewers to revisit the submission.

=====Interface Requirement=====
The overall user interface would remain the same as of the current system. Since this is a modification to the current submission page, there will be a new addition to the submission page. On the submission page, there will be a new button to notify reviewers when there is a submission which doesn’t contain a change to submission parameters like submission URL, but the deployment has been updated on the same URL. Other than that the interface remains the same for the score page, only functionalities will be tested for multi-part rubrics.

=====Quality Requirements=====
The interface changes are minimal and very easy to adapt. The new button will be clearly visible and is supposed to have a clear text to interpret its functionality. The proposed changes should not affect the current system functionality in any way. The solution is only intended to enrich the project features.

=====Portability Requirements=====
All of the changes for this enhancement to Expertiza should not interfere with the platform portability in any way.

=====Performance Requirements=====
Adding test for multi-part rubric functionality should not cause any overhead in the performance of the system. Sending emails through “Notify All Reviewers” button should affect the system minimally in terms of performance. The only overhead is a database query to get all the reviewers. A good database design should incorporate such a query smoothly.

=====Assumptions=====
The functionality should work in case the assignment rubric is not multi-part.

== '''Design''' ==
=====Zero-weighted scores=====
If only zero-weighted scores are available in the first round of review, the average of all zero-weighted scores will be shown in a gray font. An information button will explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade”)

Inside compute_total_score method in on_the_fly_calc.rb module, we'll check if the weighted score exists, if not then we'll calculate the non weighted score. We'll also have to add a method in the questionnaire model to get non weighted score. Few changes are required for the UI to display non-weighted score in grey with information icon and non weighted score in black.

Possible files to be modified.

models/on_the_fly_calc.rb 
models/questionnaire.rb 
models/view_team.html.erb

===== Non-zero weighted scores=====
If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is 80%⨉90% + 100%⨉10% = 82%

We'll first write the test cases by adding test cases to the '''spec/models/on_the_fly_calc_spec.rb'''

Also, we'll test this manually by computing the score and if there's a bug then below are the possible files to be modified.

models/on_the_fly_calc.rb 
models/questionnarie.rb

=====Review Update=====
If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review. This functionality already exists in the existing system. However, In some submissions, there’s no change in the submitted URL’s so there’s no way to let the reviewers know that the author has made some changes in the project. We’ll add a notify button for the Author in the UI which will trigger an email to the reviewers. This will reopen the review, and then the reviewer can go in and update the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review

'''Sending an email when the submission parameters don’t change:'''

A new button will be added on the submission page. The button will enable the author to notify all the reviewers to revisit the submission. We could also add a custom message in the
mail body so that the authors know the specific reason as to why they received this particular mail.

=====Profile Email Preference=====
The checkboxes are moved to the left corner. As the layout of the options have used table layout. So, we'll increase the width of the the column which includes checkboxes. We'll be using the percentage instead of fixed value to give the width, as this would adjust the column size based on the screen size. The CSS can be applied to the '''layout.scss''' file to using the appropriate class given to the column 

[[File:E1975_CHECKBOX_1.jpg]]

=='''Test Plan'''==

=====Rspec=====

New test cases will be added to test the functionalities which are not working as expected. Also the test cases for the additional features and changes in the system. Test cases will be added to these files:

spec/models/on_the_fly_calc_spec.rb 
spec/models/questionnaire_spec.rb 

=====Email Preferences UI Changes=====

The email preferences checkboxes at present appear sticking to the rightmost border of the page. By following the requirements the checkboxes will be made available closer to the text fields to make them look more user-friendly.
The changes can be tested by logging into the system and on the user profile page.

=====Manual Testing=====

The instructor will create an assignment and assign the zero and non-zero weights to the rubrics.

· Login as an author and go to scores page to view the feedback.

Test – Zero weighted scores shown in gray.

Test – Zero weighted scores shown in gray.

· Zero weighted scores should be displayed in gray to the user.

Test – Information button.

· The information button should explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade.”)

Test – Display non-zero weighted scores.

· The scores shown for the assignment should be the weighted average of the scores, if any non-zero scores are available
· The zero weighted scores have to be replaced with non-zero weighted scores.

Test – Update review score.
· If author updates the submission before the end of current round. The reviewer should be able to go and update the submitted review.
· If the review is not updated the previous review score has to be retained.

Test – Multipart part rubric.
· The functionality should work for multipart rubric.
· The scores calculated have to be accurate.

CSC/ECE 517 Fall 2019 - E1975. Generalize Review Versioning

2019-11-12T03:55:35Z

Ajain37: /* Test Plan */

__TOC__

== '''About Expertiza''' ==

Expertiza is an open source project based on Ruby on Rails framework and the code is available on Github. Expertiza allows the instructor to create new assignments as well as edit new or existing assignments. Instructors can also create a list of topics the students can sign up for and specify deadlines for completion of various tasks. Students can form teams in Expertiza to work on various projects and assignments as well as peer review other students' submissions. Expertiza supports submission across various document types, including the URLs Wiki pages.
 
 

== '''Introduction''' ==
Expertiza is a peer-review based learning platform in which work-products by students are evaluated and scored by peer-review of other students. Review process occurs in stages. After submission of initial feedback for a work-product by the peer-group, students are allowed time to incorporate the comments and improve their work. This second version of the work product is reviewed again and scores are given. This process might be repeated again and the average score from the last review stage is considered as the final score for that work-product. This incremental development of work-products and progressive learning is the fundamental concept underlying the Expertiza system.

=====Purpose=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The system also triggers an email to reviewer when there is a change in the submission. The scores are calculated on the fly based on the rubrics.

=====Scope=====
This project will enhance the existing scoring and peer review functionality and the underlying implementation along with the testing of the existing implementation. Testing of the score calculation, display for the weighted and non-weighted scores based on the combination of the rubrics are part of this project. This project also includes an additional feature for the author to notify the reviewers for the change in the submission.

=====Background=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The scores are calculated based on the rubrics defined in the system. There’s already an existing implementation for the computation of score but it doesn’t work for multipart rubric

=====Documentation=====
The documents that will be generated for this project are the design document, the actual files that we edit or create, and a ReadMe that explains what we did and how to use the modified product.

=====Standards=====
The new standards that will be adhered to in this system are those given for the rails framework, and the code should follow rails and object oriented design principles. Any newly added code will adhere to the guidelines mentioned below:

● https://github.com/rubocop-hq/rails-style-guide 
● http://wiki.expertiza.ncsu.edu/index.php/Object-Oriented_Design_and_Programming.

== '''System Requirements''' ==
=====Problem=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The functionality needs to be tested. It works in some cases, but we are suspicious that it is not entirely correct.

'''Profile Email Preference''':
User in expertiza has liberty to control what kind of emails they should receive. As the reviewers will receive an email for the change in submission. This can be controlled by selecting an option (When someone else submits work I am assigned to review) on the profile page of the user. As the checkboxes are far away on the right corner of the screen, the position of checkboxes need to be changed to make it more user friendly.
 

Scores for students in Expertiza can be based on a combination of rubrics: review rubrics for each round of review, author feedback, and teammate review. In CSC/ECE 517, we don’t really use the student-assigned scores to grade projects, but the peer-review score shown to the student is based entirely on the second round of review 

Here is how scores are calculated: 

1. If only zero-weighted scores are available (here, in the first round of review), the average of all zero-weighted scores should be shown in grey font. An information button should explain why the scores are shown in grey (“The reviews submitted so far do not count toward the author’s final grade.”): 

2. If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is:: 
80%⨉90% + 100%⨉10% = 82%

3. If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. '''However, the previous review score that was given in the current round will count until the reviewer updates the submitted review'''. '''This functionality doesn’t work for multipart rubrics.''' 

The system also doesn’t generate emails to reviewers in case a new submission is made by the author without changing submission parameters like submission URL. Eg: A new submission is made by making a new deployment. But the deployment URL doesn’t change. In this case as well, without changing the current flow the author should have an option to notify all the reviewers to revisit the submission.

=====Interface Requirement=====
The overall user interface would remain the same as of the current system. Since this is a modification to the current submission page, there will be a new addition to the submission page. On the submission page, there will be a new button to notify reviewers when there is a submission which doesn’t contain a change to submission parameters like submission URL, but the deployment has been updated on the same URL. Other than that the interface remains the same for the score page, only functionalities will be tested for multi-part rubrics.

=====Quality Requirements=====
The interface changes are minimal and very easy to adapt. The new button will be clearly visible and is supposed to have a clear text to interpret its functionality. The proposed changes should not affect the current system functionality in any way. The solution is only intended to enrich the project features.

=====Portability Requirements=====
All of the changes for this enhancement to Expertiza should not interfere with the platform portability in any way.

=====Performance Requirements=====
Adding test for multi-part rubric functionality should not cause any overhead in the performance of the system. Sending emails through “Notify All Reviewers” button should affect the system minimally in terms of performance. The only overhead is a database query to get all the reviewers. A good database design should incorporate such a query smoothly.

=====Assumptions=====
The functionality should work in case the assignment rubric is not multi-part.

== '''Design''' ==
=====Zero-weighted scores=====
If only zero-weighted scores are available in the first round of review, the average of all zero-weighted scores will be shown in a gray font. An information button will explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade”)

Inside compute_total_score method in on_the_fly_calc.rb module, we'll check if the weighted score exists, if not then we'll calculate the non weighted score. We'll also have to add a method in the questionnaire model to get non weighted score. Few changes are required for the UI to display non-weighted score in grey with information icon and non weighted score in black.

Possible files to be modified.

models/on_the_fly_calc.rb 
models/questionnaire.rb 
models/view_team.html.erb

===== Non-zero weighted scores=====
If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is 80%⨉90% + 100%⨉10% = 82%

We'll first write the test cases by adding test cases to the '''spec/models/on_the_fly_calc_spec.rb'''

Also, we'll test this manually by computing the score and if there's a bug then below are the possible files to be modified.

models/on_the_fly_calc.rb 
models/questionnarie.rb

=====Review Update=====
If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review. This functionality already exists in the existing system. However, In some submissions, there’s no change in the submitted URL’s so there’s no way to let the reviewers know that the author has made some changes in the project. We’ll add a notify button for the Author in the UI which will trigger an email to the reviewers. This will reopen the review, and then the reviewer can go in and update the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review

'''Sending an email when the submission parameters don’t change:'''

A new button will be added on the submission page. The button will enable the author to notify all the reviewers to revisit the submission. We could also add a custom message in the
mail body so that the authors know the specific reason as to why they received this particular mail.

=====Profile Email Preference=====
The checkboxes are moved to the left corner. As the layout of the options have used table layout. So, we'll increase the width of the the column which includes checkboxes. We'll be using the percentage instead of fixed value to give the width, as this would adjust the column size based on the screen size. The CSS can be applied to the '''layout.scss''' file to using the appropriate class given to the column 

[[File:E1975_CHECKBOX_1.jpg]]

=='''Test Plan'''==

=====Rspec=====

New test cases will be added to test the functionalities which are not working as expected. Also the test cases for the additional features and changes in the system. Test cases will be added to these files:

spec/models/on_the_fly_calc_spec.rb 
spec/models/questionnaire_spec.rb 

=====Email Preferences=====

The email preferences checkboxes at present appear sticking to the rightmost border of the page. By following the requirements the checkboxes will be made available closer to the text fields to make them look more user-friendly.
The changes can be tested by logging into the system and on the user profile page.

=====Manual Testing=====

The instructor will create an assignment and assign the zero and non-zero weights to the rubrics.

· Login as an author and go to scores page to view the feedback.

Test – Zero weighted scores shown in gray.

Test – Zero weighted scores shown in gray.

· Zero weighted scores should be displayed in gray to the user.

Test – Information button.

· The information button should explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade.”)

Test – Display non-zero weighted scores.

· The scores shown for the assignment should be the weighted average of the scores, if any non-zero scores are available
· The zero weighted scores have to be replaced with non-zero weighted scores.

Test – Update review score.
· If author updates the submission before the end of current round. The reviewer should be able to go and update the submitted review.
· If the review is not updated the previous review score has to be retained.

Test – Multipart part rubric.
· The functionality should work for multipart rubric.
· The scores calculated have to be accurate.

CSC/ECE 517 Fall 2019 - E1975. Generalize Review Versioning

2019-11-12T03:55:26Z

Ajain37: /* Test Plan */

__TOC__

== '''About Expertiza''' ==

Expertiza is an open source project based on Ruby on Rails framework and the code is available on Github. Expertiza allows the instructor to create new assignments as well as edit new or existing assignments. Instructors can also create a list of topics the students can sign up for and specify deadlines for completion of various tasks. Students can form teams in Expertiza to work on various projects and assignments as well as peer review other students' submissions. Expertiza supports submission across various document types, including the URLs Wiki pages.
 
 

== '''Introduction''' ==
Expertiza is a peer-review based learning platform in which work-products by students are evaluated and scored by peer-review of other students. Review process occurs in stages. After submission of initial feedback for a work-product by the peer-group, students are allowed time to incorporate the comments and improve their work. This second version of the work product is reviewed again and scores are given. This process might be repeated again and the average score from the last review stage is considered as the final score for that work-product. This incremental development of work-products and progressive learning is the fundamental concept underlying the Expertiza system.

=====Purpose=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The system also triggers an email to reviewer when there is a change in the submission. The scores are calculated on the fly based on the rubrics.

=====Scope=====
This project will enhance the existing scoring and peer review functionality and the underlying implementation along with the testing of the existing implementation. Testing of the score calculation, display for the weighted and non-weighted scores based on the combination of the rubrics are part of this project. This project also includes an additional feature for the author to notify the reviewers for the change in the submission.

=====Background=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The scores are calculated based on the rubrics defined in the system. There’s already an existing implementation for the computation of score but it doesn’t work for multipart rubric

=====Documentation=====
The documents that will be generated for this project are the design document, the actual files that we edit or create, and a ReadMe that explains what we did and how to use the modified product.

=====Standards=====
The new standards that will be adhered to in this system are those given for the rails framework, and the code should follow rails and object oriented design principles. Any newly added code will adhere to the guidelines mentioned below:

● https://github.com/rubocop-hq/rails-style-guide 
● http://wiki.expertiza.ncsu.edu/index.php/Object-Oriented_Design_and_Programming.

== '''System Requirements''' ==
=====Problem=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The functionality needs to be tested. It works in some cases, but we are suspicious that it is not entirely correct.

'''Profile Email Preference''':
User in expertiza has liberty to control what kind of emails they should receive. As the reviewers will receive an email for the change in submission. This can be controlled by selecting an option (When someone else submits work I am assigned to review) on the profile page of the user. As the checkboxes are far away on the right corner of the screen, the position of checkboxes need to be changed to make it more user friendly.
 

Scores for students in Expertiza can be based on a combination of rubrics: review rubrics for each round of review, author feedback, and teammate review. In CSC/ECE 517, we don’t really use the student-assigned scores to grade projects, but the peer-review score shown to the student is based entirely on the second round of review 

Here is how scores are calculated: 

1. If only zero-weighted scores are available (here, in the first round of review), the average of all zero-weighted scores should be shown in grey font. An information button should explain why the scores are shown in grey (“The reviews submitted so far do not count toward the author’s final grade.”): 

2. If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is:: 
80%⨉90% + 100%⨉10% = 82%

3. If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. '''However, the previous review score that was given in the current round will count until the reviewer updates the submitted review'''. '''This functionality doesn’t work for multipart rubrics.''' 

The system also doesn’t generate emails to reviewers in case a new submission is made by the author without changing submission parameters like submission URL. Eg: A new submission is made by making a new deployment. But the deployment URL doesn’t change. In this case as well, without changing the current flow the author should have an option to notify all the reviewers to revisit the submission.

=====Interface Requirement=====
The overall user interface would remain the same as of the current system. Since this is a modification to the current submission page, there will be a new addition to the submission page. On the submission page, there will be a new button to notify reviewers when there is a submission which doesn’t contain a change to submission parameters like submission URL, but the deployment has been updated on the same URL. Other than that the interface remains the same for the score page, only functionalities will be tested for multi-part rubrics.

=====Quality Requirements=====
The interface changes are minimal and very easy to adapt. The new button will be clearly visible and is supposed to have a clear text to interpret its functionality. The proposed changes should not affect the current system functionality in any way. The solution is only intended to enrich the project features.

=====Portability Requirements=====
All of the changes for this enhancement to Expertiza should not interfere with the platform portability in any way.

=====Performance Requirements=====
Adding test for multi-part rubric functionality should not cause any overhead in the performance of the system. Sending emails through “Notify All Reviewers” button should affect the system minimally in terms of performance. The only overhead is a database query to get all the reviewers. A good database design should incorporate such a query smoothly.

=====Assumptions=====
The functionality should work in case the assignment rubric is not multi-part.

== '''Design''' ==
=====Zero-weighted scores=====
If only zero-weighted scores are available in the first round of review, the average of all zero-weighted scores will be shown in a gray font. An information button will explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade”)

Inside compute_total_score method in on_the_fly_calc.rb module, we'll check if the weighted score exists, if not then we'll calculate the non weighted score. We'll also have to add a method in the questionnaire model to get non weighted score. Few changes are required for the UI to display non-weighted score in grey with information icon and non weighted score in black.

Possible files to be modified.

models/on_the_fly_calc.rb 
models/questionnaire.rb 
models/view_team.html.erb

===== Non-zero weighted scores=====
If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is 80%⨉90% + 100%⨉10% = 82%

We'll first write the test cases by adding test cases to the '''spec/models/on_the_fly_calc_spec.rb'''

Also, we'll test this manually by computing the score and if there's a bug then below are the possible files to be modified.

models/on_the_fly_calc.rb 
models/questionnarie.rb

=====Review Update=====
If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review. This functionality already exists in the existing system. However, In some submissions, there’s no change in the submitted URL’s so there’s no way to let the reviewers know that the author has made some changes in the project. We’ll add a notify button for the Author in the UI which will trigger an email to the reviewers. This will reopen the review, and then the reviewer can go in and update the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review

'''Sending an email when the submission parameters don’t change:'''

A new button will be added on the submission page. The button will enable the author to notify all the reviewers to revisit the submission. We could also add a custom message in the
mail body so that the authors know the specific reason as to why they received this particular mail.

=====Profile Email Preference=====
The checkboxes are moved to the left corner. As the layout of the options have used table layout. So, we'll increase the width of the the column which includes checkboxes. We'll be using the percentage instead of fixed value to give the width, as this would adjust the column size based on the screen size. The CSS can be applied to the '''layout.scss''' file to using the appropriate class given to the column 

[[File:E1975_CHECKBOX_1.jpg]]

=='''Test Plan'''==

=====Rspec=====

New test cases will be added to test the functionalities which are not working as expected. Also the test cases for the additional features and changes in the system. Test cases will be added to these files:

spec/models/on_the_fly_calc_spec.rb 
spec/models/questionnaire_spec.rb 

=====Test Plan for Email Preferences=====

The email preferences checkboxes at present appear sticking to the rightmost border of the page. By following the requirements the checkboxes will be made available closer to the text fields to make them look more user-friendly.
The changes can be tested by logging into the system and on the user profile page.

=====Manual Testing=====

The instructor will create an assignment and assign the zero and non-zero weights to the rubrics.

· Login as an author and go to scores page to view the feedback.

Test – Zero weighted scores shown in gray.

Test – Zero weighted scores shown in gray.

· Zero weighted scores should be displayed in gray to the user.

Test – Information button.

· The information button should explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade.”)

Test – Display non-zero weighted scores.

· The scores shown for the assignment should be the weighted average of the scores, if any non-zero scores are available
· The zero weighted scores have to be replaced with non-zero weighted scores.

Test – Update review score.
· If author updates the submission before the end of current round. The reviewer should be able to go and update the submitted review.
· If the review is not updated the previous review score has to be retained.

Test – Multipart part rubric.
· The functionality should work for multipart rubric.
· The scores calculated have to be accurate.

CSC/ECE 517 Fall 2019 - E1975. Generalize Review Versioning

2019-11-12T01:41:14Z

Ajain37: /* Rspec */

__TOC__

== '''About Expertiza''' ==

Expertiza is an open source project based on Ruby on Rails framework and the code is available on Github. Expertiza allows the instructor to create new assignments as well as edit new or existing assignments. Instructors can also create a list of topics the students can sign up for and specify deadlines for completion of various tasks. Students can form teams in Expertiza to work on various projects and assignments as well as peer review other students' submissions. Expertiza supports submission across various document types, including the URLs Wiki pages.
 
 

== '''Introduction''' ==
Expertiza is a peer-review based learning platform in which work-products by students are evaluated and scored by peer-review of other students. Review process occurs in stages. After submission of initial feedback for a work-product by the peer-group, students are allowed time to incorporate the comments and improve their work. This second version of the work product is reviewed again and scores are given. This process might be repeated again and the average score from the last review stage is considered as the final score for that work-product. This incremental development of work-products and progressive learning is the fundamental concept underlying the Expertiza system.

=====Purpose=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The system also triggers an email to reviewer when there is a change in the submission. The scores are calculated on the fly based on the rubrics.

=====Scope=====
This project will enhance the existing scoring and peer review functionality and the underlying implementation along with the testing of the existing implementation. Testing of the score calculation, display for the weighted and non-weighted scores based on the combination of the rubrics are part of this project. This project also includes an additional feature for the author to notify the reviewers for the change in the submission.

=====Background=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The scores are calculated based on the rubrics defined in the system. There’s already an existing implementation for the computation of score but it doesn’t work for multipart rubric

=====Documentation=====
The documents that will be generated for this project are the design document, the actual files that we edit or create, and a ReadMe that explains what we did and how to use the modified product.

=====Standards=====
The new standards that will be adhered to in this system are those given for the rails framework, and the code should follow rails and object oriented design principles. Any newly added code will adhere to the guidelines mentioned below:

● https://github.com/rubocop-hq/rails-style-guide 
● http://wiki.expertiza.ncsu.edu/index.php/Object-Oriented_Design_and_Programming.

== '''System Requirements''' ==
=====Problem=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The functionality needs to be tested. It works in some cases, but we are suspicious that it is not entirely correct.

'''Profile Email Preference''':
User in expertiza has liberty to control what kind of emails they should receive. As the reviewers will receive an email for the change in submission. This can be controlled by selecting an option (When someone else submits work I am assigned to review) on the profile page of the user. As the checkboxes are far away on the right corner of the screen, the position of checkboxes need to be changed to make it more user friendly.
 

Scores for students in Expertiza can be based on a combination of rubrics: review rubrics for each round of review, author feedback, and teammate review. In CSC/ECE 517, we don’t really use the student-assigned scores to grade projects, but the peer-review score shown to the student is based entirely on the second round of review 

Here is how scores are calculated: 

1. If only zero-weighted scores are available (here, in the first round of review), the average of all zero-weighted scores should be shown in grey font. An information button should explain why the scores are shown in grey (“The reviews submitted so far do not count toward the author’s final grade.”): 

2. If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is:: 
80%⨉90% + 100%⨉10% = 82%

3. If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. '''However, the previous review score that was given in the current round will count until the reviewer updates the submitted review'''. '''This functionality doesn’t work for multipart rubrics.''' 

The system also doesn’t generate emails to reviewers in case a new submission is made by the author without changing submission parameters like submission URL. Eg: A new submission is made by making a new deployment. But the deployment URL doesn’t change. In this case as well, without changing the current flow the author should have an option to notify all the reviewers to revisit the submission.

=====Interface Requirement=====
The overall user interface would remain the same as of the current system. Since this is a modification to the current submission page, there will be a new addition to the submission page. On the submission page, there will be a new button to notify reviewers when there is a submission which doesn’t contain a change to submission parameters like submission URL, but the deployment has been updated on the same URL. Other than that the interface remains the same for the score page, only functionalities will be tested for multi-part rubrics.

=====Quality Requirements=====
The interface changes are minimal and very easy to adapt. The new button will be clearly visible and is supposed to have a clear text to interpret its functionality. The proposed changes should not affect the current system functionality in any way. The solution is only intended to enrich the project features.

=====Portability Requirements=====
All of the changes for this enhancement to Expertiza should not interfere with the platform portability in any way.

=====Performance Requirements=====
Adding test for multi-part rubric functionality should not cause any overhead in the performance of the system. Sending emails through “Notify All Reviewers” button should affect the system minimally in terms of performance. The only overhead is a database query to get all the reviewers. A good database design should incorporate such a query smoothly.

=====Assumptions=====
The functionality should work in case the assignment rubric is not multi-part.

== '''Design''' ==
=====Zero-weighted scores=====
If only zero-weighted scores are available in the first round of review, the average of all zero-weighted scores will be shown in a gray font. An information button will explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade”)

Inside compute_total_score method in on_the_fly_calc.rb module, we'll check if the weighted score exists, if not then we'll calculate the non weighted score. We'll also have to add a method in the questionnaire model to get non weighted score. Few changes are required for the UI to display non-weighted score in grey with information icon and non weighted score in black.

Possible files to be modified.

models/on_the_fly_calc.rb 
models/questionnaire.rb 
models/view_team.html.erb

===== Non-zero weighted scores=====
If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is 80%⨉90% + 100%⨉10% = 82%

We'll first write the test cases by adding test cases to the '''spec/models/on_the_fly_calc_spec.rb'''

Also, we'll test this manually by computing the score and if there's a bug then below are the possible files to be modified.

models/on_the_fly_calc.rb 
models/questionnarie.rb

=====Review Update=====
If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review. This functionality already exists in the existing system. However, In some submissions, there’s no change in the submitted URL’s so there’s no way to let the reviewers know that the author has made some changes in the project. We’ll add a notify button for the Author in the UI which will trigger an email to the reviewers. This will reopen the review, and then the reviewer can go in and update the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review

'''Sending an email when the submission parameters don’t change:'''

A new button will be added on the submission page. The button will enable the author to notify all the reviewers to revisit the submission. We could also add a custom message in the
mail body so that the authors know the specific reason as to why they received this particular mail.

=====Profile Email Preference=====
The checkboxes are moved to the left corner. As the layout of the options have used table layout. So, we'll increase the width of the the column which includes checkboxes. We'll be using the percentage instead of fixed value to give the width, as this would adjust the column size based on the screen size. The CSS can be applied to the '''layout.scss''' file to using the appropriate class given to the column 

[[File:E1975_CHECKBOX_1.jpg]]

=='''Test Plan'''==

=====Rspec=====

New test cases will be added to test the functionalities which may have a problem. Also the test cases for the additional features and changes in the system. Test cases will be added to these files:
spec/models/on_the_fly_calc_spec.rb 
spec/models/questionnaire_spec.rb 

We have added new testcases for the new pieces of code that were added by us.

=====Test Plan for Email Preferences=====

The email preferences checkboxes at present appear sticking to the rightmost border of the page. By following the requirements the checkboxes will be made available closer to the text fields to make them look more user-friendly.
The changes can be tested post code changes.

=====Manual Testing=====

The instructor will create an assignment and assign the zero and non-zero weights to the rubrics.

· Login as an author and go to scores page to view the feedback.

Test – Zero weighted scores shown in gray.

Test – Zero weighted scores shown in gray.

· Zero weighted scores should be displayed in gray to the user.

Test – Information button.

· The information button should explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade.”)

Test – Display non-zero weighted scores.

· The scores shown for the assignment should be the weighted average of the scores, if any non-zero scores are available
· The zero weighted scores have to be replaced with non-zero weighted scores.

Test – Update review score.
· If author updates the submission before the end of current round. The reviewer should be able to go and update the submitted review.
· If the review is not updated the previous review score has to be retained.

Test – Multipart part rubric.
· The functionality should work for multipart rubric.
· The scores calculated have to be accurate.

CSC/ECE 517 Fall 2019 - E1975. Generalize Review Versioning

2019-11-12T01:32:47Z

Ajain37: /* Test Plan */

__TOC__

== '''About Expertiza''' ==

Expertiza is an open source project based on Ruby on Rails framework and the code is available on Github. Expertiza allows the instructor to create new assignments as well as edit new or existing assignments. Instructors can also create a list of topics the students can sign up for and specify deadlines for completion of various tasks. Students can form teams in Expertiza to work on various projects and assignments as well as peer review other students' submissions. Expertiza supports submission across various document types, including the URLs Wiki pages.
 
 

== '''Introduction''' ==
Expertiza is a peer-review based learning platform in which work-products by students are evaluated and scored by peer-review of other students. Review process occurs in stages. After submission of initial feedback for a work-product by the peer-group, students are allowed time to incorporate the comments and improve their work. This second version of the work product is reviewed again and scores are given. This process might be repeated again and the average score from the last review stage is considered as the final score for that work-product. This incremental development of work-products and progressive learning is the fundamental concept underlying the Expertiza system.

=====Purpose=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The system also triggers an email to reviewer when there is a change in the submission. The scores are calculated on the fly based on the rubrics.

=====Scope=====
This project will enhance the existing scoring and peer review functionality and the underlying implementation along with the testing of the existing implementation. Testing of the score calculation, display for the weighted and non-weighted scores based on the combination of the rubrics are part of this project. This project also includes an additional feature for the author to notify the reviewers for the change in the submission.

=====Background=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The scores are calculated based on the rubrics defined in the system. There’s already an existing implementation for the computation of score but it doesn’t work for multipart rubric

=====Documentation=====
The documents that will be generated for this project are the design document, the actual files that we edit or create, and a ReadMe that explains what we did and how to use the modified product.

=====Standards=====
The new standards that will be adhered to in this system are those given for the rails framework, and the code should follow rails and object oriented design principles. Any newly added code will adhere to the guidelines mentioned below:

● https://github.com/rubocop-hq/rails-style-guide 
● http://wiki.expertiza.ncsu.edu/index.php/Object-Oriented_Design_and_Programming.

== '''System Requirements''' ==
=====Problem=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The functionality needs to be tested. It works in some cases, but we are suspicious that it is not entirely correct.

'''Profile Email Preference''':
User in expertiza has liberty to control what kind of emails they should receive. As the reviewers will receive an email for the change in submission. This can be controlled by selecting an option (When someone else submits work I am assigned to review) on the profile page of the user. As the checkboxes are far away on the right corner of the screen, the position of checkboxes need to be changed to make it more user friendly.
 

Scores for students in Expertiza can be based on a combination of rubrics: review rubrics for each round of review, author feedback, and teammate review. In CSC/ECE 517, we don’t really use the student-assigned scores to grade projects, but the peer-review score shown to the student is based entirely on the second round of review 

Here is how scores are calculated: 

1. If only zero-weighted scores are available (here, in the first round of review), the average of all zero-weighted scores should be shown in grey font. An information button should explain why the scores are shown in grey (“The reviews submitted so far do not count toward the author’s final grade.”): 

2. If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is:: 
80%⨉90% + 100%⨉10% = 82%

3. If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. '''However, the previous review score that was given in the current round will count until the reviewer updates the submitted review'''. '''This functionality doesn’t work for multipart rubrics.''' 

The system also doesn’t generate emails to reviewers in case a new submission is made by the author without changing submission parameters like submission URL. Eg: A new submission is made by making a new deployment. But the deployment URL doesn’t change. In this case as well, without changing the current flow the author should have an option to notify all the reviewers to revisit the submission.

=====Interface Requirement=====
The overall user interface would remain the same as of the current system. Since this is a modification to the current submission page, there will be a new addition to the submission page. On the submission page, there will be a new button to notify reviewers when there is a submission which doesn’t contain a change to submission parameters like submission URL, but the deployment has been updated on the same URL. Other than that the interface remains the same for the score page, only functionalities will be tested for multi-part rubrics.

=====Quality Requirements=====
The interface changes are minimal and very easy to adapt. The new button will be clearly visible and is supposed to have a clear text to interpret its functionality. The proposed changes should not affect the current system functionality in any way. The solution is only intended to enrich the project features.

=====Portability Requirements=====
All of the changes for this enhancement to Expertiza should not interfere with the platform portability in any way.

=====Performance Requirements=====
Adding test for multi-part rubric functionality should not cause any overhead in the performance of the system. Sending emails through “Notify All Reviewers” button should affect the system minimally in terms of performance. The only overhead is a database query to get all the reviewers. A good database design should incorporate such a query smoothly.

=====Assumptions=====
The functionality should work in case the assignment rubric is not multi-part.

== '''Design''' ==
=====Zero-weighted scores=====
If only zero-weighted scores are available in the first round of review, the average of all zero-weighted scores will be shown in a gray font. An information button will explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade”)

Inside compute_total_score method in on_the_fly_calc.rb module, we'll check if the weighted score exists, if not then we'll calculate the non weighted score. We'll also have to add a method in the questionnaire model to get non weighted score. Few changes are required for the UI to display non-weighted score in grey with information icon and non weighted score in black.

Possible files to be modified.

models/on_the_fly_calc.rb 
models/questionnaire.rb 
models/view_team.html.erb

===== Non-zero weighted scores=====
If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is 80%⨉90% + 100%⨉10% = 82%

We'll first write the test cases by adding test cases to the '''spec/models/on_the_fly_calc_spec.rb'''

Also, we'll test this manually by computing the score and if there's a bug then below are the possible files to be modified.

models/on_the_fly_calc.rb 
models/questionnarie.rb

=====Review Update=====
If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review. This functionality already exists in the existing system. However, In some submissions, there’s no change in the submitted URL’s so there’s no way to let the reviewers know that the author has made some changes in the project. We’ll add a notify button for the Author in the UI which will trigger an email to the reviewers. This will reopen the review, and then the reviewer can go in and update the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review

'''Sending an email when the submission parameters don’t change:'''

A new button will be added on the submission page. The button will enable the author to notify all the reviewers to revisit the submission. We could also add a custom message in the
mail body so that the authors know the specific reason as to why they received this particular mail.

=====Profile Email Preference=====
The checkboxes are moved to the left corner. As the layout of the options have used table layout. So, we'll increase the width of the the column which includes checkboxes. We'll be using the percentage instead of fixed value to give the width, as this would adjust the column size based on the screen size. The CSS can be applied to the '''layout.scss''' file to using the appropriate class given to the column 

[[File:E1975_CHECKBOX_1.jpg]]

=='''Test Plan'''==

=====Rspec=====

New test cases will be added to test the functionalities which may have a problem. Also the test cases for the additional features and changes in the system. Test cases will be added to these files:

We have added new testcases for the new pieces of code that were added by us.

=====Test Plan for Email Preferences=====

The email preferences checkboxes at present appear sticking to the rightmost border of the page. By following the requirements the checkboxes will be made available closer to the text fields to make them look more user-friendly.
The changes can be tested post code changes.

=====Manual Testing=====

The instructor will create an assignment and assign the zero and non-zero weights to the rubrics.

· Login as an author and go to scores page to view the feedback.

Test – Zero weighted scores shown in gray.

Test – Zero weighted scores shown in gray.

· Zero weighted scores should be displayed in gray to the user.

Test – Information button.

· The information button should explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade.”)

Test – Display non-zero weighted scores.

· The scores shown for the assignment should be the weighted average of the scores, if any non-zero scores are available
· The zero weighted scores have to be replaced with non-zero weighted scores.

Test – Update review score.
· If author updates the submission before the end of current round. The reviewer should be able to go and update the submitted review.
· If the review is not updated the previous review score has to be retained.

Test – Multipart part rubric.
· The functionality should work for multipart rubric.
· The scores calculated have to be accurate.

CSC/ECE 517 Fall 2019 - E1975. Generalize Review Versioning

2019-11-11T22:18:46Z

Ajain37: /* Non-zero weighted scores */

__TOC__

== '''About Expertiza''' ==

Expertiza is an open source project based on Ruby on Rails framework and the code is available on Github. Expertiza allows the instructor to create new assignments as well as edit new or existing assignments. Instructors can also create a list of topics the students can sign up for and specify deadlines for completion of various tasks. Students can form teams in Expertiza to work on various projects and assignments as well as peer review other students' submissions. Expertiza supports submission across various document types, including the URLs Wiki pages.
 
 

== '''Introduction''' ==
Expertiza is a peer-review based learning platform in which work-products by students are evaluated and scored by peer-review of other students. Review process occurs in stages. After submission of initial feedback for a work-product by the peer-group, students are allowed time to incorporate the comments and improve their work. This second version of the work product is reviewed again and scores are given. This process might be repeated again and the average score from the last review stage is considered as the final score for that work-product. This incremental development of work-products and progressive learning is the fundamental concept underlying the Expertiza system.

=====Purpose=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The system also triggers an email to reviewer when there is a change in the submission. The scores are calculated on the fly based on the rubrics.

=====Scope=====
This project will enhance the existing scoring and peer review functionality and the underlying implementation along with the testing of the existing implementation. Testing of the score calculation, display for the weighted and non-weighted scores based on the combination of the rubrics are part of this project. This project also includes an additional feature for the author to notify the reviewers for the change in the submission.

=====Background=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The scores are calculated based on the rubrics defined in the system. There’s already an existing implementation for the computation of score but it doesn’t work for multipart rubric

=====Documentation=====
The documents that will be generated for this project are the design document, the actual files that we edit or create, and a ReadMe that explains what we did and how to use the modified product.

=====Standards=====
The new standards that will be adhered to in this system are those given for the rails framework, and the code should follow rails and object oriented design principles. Any newly added code will adhere to the guidelines mentioned below:

● https://github.com/rubocop-hq/rails-style-guide 
● http://wiki.expertiza.ncsu.edu/index.php/Object-Oriented_Design_and_Programming.

== '''System Requirements''' ==
=====Problem=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The functionality needs to be tested. It works in some cases, but we are suspicious that it is not entirely correct.

'''Profile Email Preference''':
User in expertiza has liberty to control what kind of emails they should receive. As the reviewers will receive an email for the change in submission. This can be controlled by selecting an option (When someone else submits work I am assigned to review) on the profile page of the user. As the checkboxes are far away on the right corner of the screen, the position of checkboxes need to be changed to make it more user friendly.
 

Scores for students in Expertiza can be based on a combination of rubrics: review rubrics for each round of review, author feedback, and teammate review. In CSC/ECE 517, we don’t really use the student-assigned scores to grade projects, but the peer-review score shown to the student is based entirely on the second round of review 

Here is how scores are calculated: 

1. If only zero-weighted scores are available (here, in the first round of review), the average of all zero-weighted scores should be shown in grey font. An information button should explain why the scores are shown in grey (“The reviews submitted so far do not count toward the author’s final grade.”): 

2. If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is:: 
80%⨉90% + 100%⨉10% = 82%

3. If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. '''However, the previous review score that was given in the current round will count until the reviewer updates the submitted review'''. '''This functionality doesn’t work for multipart rubrics.''' 

The system also doesn’t generate emails to reviewers in case a new submission is made by the author without changing submission parameters like submission URL. Eg: A new submission is made by making a new deployment. But the deployment URL doesn’t change. In this case as well, without changing the current flow the author should have an option to notify all the reviewers to revisit the submission.

=====Interface Requirement=====
The overall user interface would remain the same as of the current system. Since this is a modification to the current submission page, there will be a new addition to the submission page. On the submission page, there will be a new button to notify reviewers when there is a submission which doesn’t contain a change to submission parameters like submission URL, but the deployment has been updated on the same URL. Other than that the interface remains the same for the score page, only functionalities will be tested for multi-part rubrics.

=====Quality Requirements=====
The interface changes are minimal and very easy to adapt. The new button will be clearly visible and is supposed to have a clear text to interpret its functionality. The proposed changes should not affect the current system functionality in any way. The solution is only intended to enrich the project features.

=====Portability Requirements=====
All of the changes for this enhancement to Expertiza should not interfere with the platform portability in any way.

=====Performance Requirements=====
Adding test for multi-part rubric functionality should not cause any overhead in the performance of the system. Sending emails through “Notify All Reviewers” button should affect the system minimally in terms of performance. The only overhead is a database query to get all the reviewers. A good database design should incorporate such a query smoothly.

=====Assumptions=====
The functionality should work in case the assignment rubric is not multi-part.

== '''Design''' ==
=====Zero-weighted scores=====
If only zero-weighted scores are available in the first round of review, the average of all zero-weighted scores will be shown in a gray font. An information button will explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade”)

Inside compute_total_score method in on_the_fly_calc.rb module, we'll check if the weighted score exists, if not then we'll calculate the non weighted score. We'll also have to add a method in the questionnaire model to get non weighted score. Few changes are required for the UI to display non-weighted score in grey with information icon and non weighted score in black.

Possible files to be modified.

models/on_the_fly_calc.rb 
models/questionnaire.rb 
models/view_team.html.erb

===== Non-zero weighted scores=====
If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is 80%⨉90% + 100%⨉10% = 82%

We'll first write the test cases by adding test cases to the '''spec/models/on_the_fly_calc_spec.rb'''

Also, we'll test this manually by computing the score and if there's a bug then below are the possible files to be modified.

models/on_the_fly_calc.rb 
models/questionnarie.rb

=====Review Update=====
If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review. This functionality already exists in the existing system. However, In some submissions, there’s no change in the submitted URL’s so there’s no way to let the reviewers know that the author has made some changes in the project. We’ll add a notify button for the Author in the UI which will trigger an email to the reviewers. This will reopen the review, and then the reviewer can go in and update the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review

'''Sending an email when the submission parameters don’t change:'''

A new button will be added on the submission page. The button will enable the author to notify all the reviewers to revisit the submission. We could also add a custom message in the
mail body so that the authors know the specific reason as to why they received this particular mail.

=====Profile Email Preference=====
The checkboxes are moved to the left corner. As the layout of the options have used table layout. So, we'll increase the width of the the column which includes checkboxes. We'll be using the percentage instead of fixed value to give the width, as this would adjust the column size based on the screen size. The CSS can be applied to the '''layout.scss''' file to using the appropriate class given to the column 

[[File:E1975_CHECKBOX_1.jpg]]

=='''Test Plan'''==

=====Rspec=====
We have added new testcases for the new pieces of code that were added by us.

=====Test Plan for Email Preferences=====

The email preferences checkboxes at present appear sticking to the rightmost border of the page. By following the requirements the checkboxes will be made available closer to the text fields to make them look more user-friendly.
The changes can be tested post code changes.

=====Manual Testing=====

The instructor will create an assignment and assign the zero and non-zero weights to the rubrics.

· Login as an author and go to scores page to view the feedback.

Test – Zero weighted scores shown in gray.

Test – Zero weighted scores shown in gray.

· Zero weighted scores should be displayed in gray to the user.

Test – Information button.

· The information button should explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade.”)

Test – Display non-zero weighted scores.

· The scores shown for the assignment should be the weighted average of the scores, if any non-zero scores are available
· The zero weighted scores have to be replaced with non-zero weighted scores.

Test – Update review score.
· If author updates the submission before the end of current round. The reviewer should be able to go and update the submitted review.
· If the review is not updated the previous review score has to be retained.

Test – Multipart part rubric.
· The functionality should work for multipart rubric.
· The scores calculated have to be accurate.

CSC/ECE 517 Fall 2019 - E1975. Generalize Review Versioning

2019-11-11T22:17:56Z

Ajain37: /* Zero-weighted scores */

__TOC__

== '''About Expertiza''' ==

Expertiza is an open source project based on Ruby on Rails framework and the code is available on Github. Expertiza allows the instructor to create new assignments as well as edit new or existing assignments. Instructors can also create a list of topics the students can sign up for and specify deadlines for completion of various tasks. Students can form teams in Expertiza to work on various projects and assignments as well as peer review other students' submissions. Expertiza supports submission across various document types, including the URLs Wiki pages.
 
 

== '''Introduction''' ==
Expertiza is a peer-review based learning platform in which work-products by students are evaluated and scored by peer-review of other students. Review process occurs in stages. After submission of initial feedback for a work-product by the peer-group, students are allowed time to incorporate the comments and improve their work. This second version of the work product is reviewed again and scores are given. This process might be repeated again and the average score from the last review stage is considered as the final score for that work-product. This incremental development of work-products and progressive learning is the fundamental concept underlying the Expertiza system.

=====Purpose=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The system also triggers an email to reviewer when there is a change in the submission. The scores are calculated on the fly based on the rubrics.

=====Scope=====
This project will enhance the existing scoring and peer review functionality and the underlying implementation along with the testing of the existing implementation. Testing of the score calculation, display for the weighted and non-weighted scores based on the combination of the rubrics are part of this project. This project also includes an additional feature for the author to notify the reviewers for the change in the submission.

=====Background=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The scores are calculated based on the rubrics defined in the system. There’s already an existing implementation for the computation of score but it doesn’t work for multipart rubric

=====Documentation=====
The documents that will be generated for this project are the design document, the actual files that we edit or create, and a ReadMe that explains what we did and how to use the modified product.

=====Standards=====
The new standards that will be adhered to in this system are those given for the rails framework, and the code should follow rails and object oriented design principles. Any newly added code will adhere to the guidelines mentioned below:

● https://github.com/rubocop-hq/rails-style-guide 
● http://wiki.expertiza.ncsu.edu/index.php/Object-Oriented_Design_and_Programming.

== '''System Requirements''' ==
=====Problem=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The functionality needs to be tested. It works in some cases, but we are suspicious that it is not entirely correct.

'''Profile Email Preference''':
User in expertiza has liberty to control what kind of emails they should receive. As the reviewers will receive an email for the change in submission. This can be controlled by selecting an option (When someone else submits work I am assigned to review) on the profile page of the user. As the checkboxes are far away on the right corner of the screen, the position of checkboxes need to be changed to make it more user friendly.
 

Scores for students in Expertiza can be based on a combination of rubrics: review rubrics for each round of review, author feedback, and teammate review. In CSC/ECE 517, we don’t really use the student-assigned scores to grade projects, but the peer-review score shown to the student is based entirely on the second round of review 

Here is how scores are calculated: 

1. If only zero-weighted scores are available (here, in the first round of review), the average of all zero-weighted scores should be shown in grey font. An information button should explain why the scores are shown in grey (“The reviews submitted so far do not count toward the author’s final grade.”): 

2. If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is:: 
80%⨉90% + 100%⨉10% = 82%

3. If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. '''However, the previous review score that was given in the current round will count until the reviewer updates the submitted review'''. '''This functionality doesn’t work for multipart rubrics.''' 

The system also doesn’t generate emails to reviewers in case a new submission is made by the author without changing submission parameters like submission URL. Eg: A new submission is made by making a new deployment. But the deployment URL doesn’t change. In this case as well, without changing the current flow the author should have an option to notify all the reviewers to revisit the submission.

=====Interface Requirement=====
The overall user interface would remain the same as of the current system. Since this is a modification to the current submission page, there will be a new addition to the submission page. On the submission page, there will be a new button to notify reviewers when there is a submission which doesn’t contain a change to submission parameters like submission URL, but the deployment has been updated on the same URL. Other than that the interface remains the same for the score page, only functionalities will be tested for multi-part rubrics.

=====Quality Requirements=====
The interface changes are minimal and very easy to adapt. The new button will be clearly visible and is supposed to have a clear text to interpret its functionality. The proposed changes should not affect the current system functionality in any way. The solution is only intended to enrich the project features.

=====Portability Requirements=====
All of the changes for this enhancement to Expertiza should not interfere with the platform portability in any way.

=====Performance Requirements=====
Adding test for multi-part rubric functionality should not cause any overhead in the performance of the system. Sending emails through “Notify All Reviewers” button should affect the system minimally in terms of performance. The only overhead is a database query to get all the reviewers. A good database design should incorporate such a query smoothly.

=====Assumptions=====
The functionality should work in case the assignment rubric is not multi-part.

== '''Design''' ==
=====Zero-weighted scores=====
If only zero-weighted scores are available in the first round of review, the average of all zero-weighted scores will be shown in a gray font. An information button will explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade”)

Inside compute_total_score method in on_the_fly_calc.rb module, we'll check if the weighted score exists, if not then we'll calculate the non weighted score. We'll also have to add a method in the questionnaire model to get non weighted score. Few changes are required for the UI to display non-weighted score in grey with information icon and non weighted score in black.

Possible files to be modified.

models/on_the_fly_calc.rb 
models/questionnaire.rb 
models/view_team.html.erb

===== Non-zero weighted scores=====
If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is 80%⨉90% + 100%⨉10% = 82%

We'll first write the test cases by adding test cases to below files.
spec/models/on_the_fly_calc_spec.rb

Also, we'll test this manually by computing the score and if there's a bug then below are the possible files to be modified.

models/on_the_fly_calc.rb 
models/questionnarie.rb

=====Review Update=====
If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review. This functionality already exists in the existing system. However, In some submissions, there’s no change in the submitted URL’s so there’s no way to let the reviewers know that the author has made some changes in the project. We’ll add a notify button for the Author in the UI which will trigger an email to the reviewers. This will reopen the review, and then the reviewer can go in and update the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review

'''Sending an email when the submission parameters don’t change:'''

A new button will be added on the submission page. The button will enable the author to notify all the reviewers to revisit the submission. We could also add a custom message in the
mail body so that the authors know the specific reason as to why they received this particular mail.

=====Profile Email Preference=====
The checkboxes are moved to the left corner. As the layout of the options have used table layout. So, we'll increase the width of the the column which includes checkboxes. We'll be using the percentage instead of fixed value to give the width, as this would adjust the column size based on the screen size. The CSS can be applied to the '''layout.scss''' file to using the appropriate class given to the column 

[[File:E1975_CHECKBOX_1.jpg]]

=='''Test Plan'''==

=====Rspec=====
We have added new testcases for the new pieces of code that were added by us.

=====Test Plan for Email Preferences=====

The email preferences checkboxes at present appear sticking to the rightmost border of the page. By following the requirements the checkboxes will be made available closer to the text fields to make them look more user-friendly.
The changes can be tested post code changes.

=====Manual Testing=====

The instructor will create an assignment and assign the zero and non-zero weights to the rubrics.

· Login as an author and go to scores page to view the feedback.

Test – Zero weighted scores shown in gray.

Test – Zero weighted scores shown in gray.

· Zero weighted scores should be displayed in gray to the user.

Test – Information button.

· The information button should explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade.”)

Test – Display non-zero weighted scores.

· The scores shown for the assignment should be the weighted average of the scores, if any non-zero scores are available
· The zero weighted scores have to be replaced with non-zero weighted scores.

Test – Update review score.
· If author updates the submission before the end of current round. The reviewer should be able to go and update the submitted review.
· If the review is not updated the previous review score has to be retained.

Test – Multipart part rubric.
· The functionality should work for multipart rubric.
· The scores calculated have to be accurate.

CSC/ECE 517 Fall 2019 - E1975. Generalize Review Versioning

2019-11-11T22:17:43Z

Ajain37: /* Non-zero weighted scores */

__TOC__

== '''About Expertiza''' ==

Expertiza is an open source project based on Ruby on Rails framework and the code is available on Github. Expertiza allows the instructor to create new assignments as well as edit new or existing assignments. Instructors can also create a list of topics the students can sign up for and specify deadlines for completion of various tasks. Students can form teams in Expertiza to work on various projects and assignments as well as peer review other students' submissions. Expertiza supports submission across various document types, including the URLs Wiki pages.
 
 

== '''Introduction''' ==
Expertiza is a peer-review based learning platform in which work-products by students are evaluated and scored by peer-review of other students. Review process occurs in stages. After submission of initial feedback for a work-product by the peer-group, students are allowed time to incorporate the comments and improve their work. This second version of the work product is reviewed again and scores are given. This process might be repeated again and the average score from the last review stage is considered as the final score for that work-product. This incremental development of work-products and progressive learning is the fundamental concept underlying the Expertiza system.

=====Purpose=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The system also triggers an email to reviewer when there is a change in the submission. The scores are calculated on the fly based on the rubrics.

=====Scope=====
This project will enhance the existing scoring and peer review functionality and the underlying implementation along with the testing of the existing implementation. Testing of the score calculation, display for the weighted and non-weighted scores based on the combination of the rubrics are part of this project. This project also includes an additional feature for the author to notify the reviewers for the change in the submission.

=====Background=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The scores are calculated based on the rubrics defined in the system. There’s already an existing implementation for the computation of score but it doesn’t work for multipart rubric

=====Documentation=====
The documents that will be generated for this project are the design document, the actual files that we edit or create, and a ReadMe that explains what we did and how to use the modified product.

=====Standards=====
The new standards that will be adhered to in this system are those given for the rails framework, and the code should follow rails and object oriented design principles. Any newly added code will adhere to the guidelines mentioned below:

● https://github.com/rubocop-hq/rails-style-guide 
● http://wiki.expertiza.ncsu.edu/index.php/Object-Oriented_Design_and_Programming.

== '''System Requirements''' ==
=====Problem=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The functionality needs to be tested. It works in some cases, but we are suspicious that it is not entirely correct.

'''Profile Email Preference''':
User in expertiza has liberty to control what kind of emails they should receive. As the reviewers will receive an email for the change in submission. This can be controlled by selecting an option (When someone else submits work I am assigned to review) on the profile page of the user. As the checkboxes are far away on the right corner of the screen, the position of checkboxes need to be changed to make it more user friendly.
 

Scores for students in Expertiza can be based on a combination of rubrics: review rubrics for each round of review, author feedback, and teammate review. In CSC/ECE 517, we don’t really use the student-assigned scores to grade projects, but the peer-review score shown to the student is based entirely on the second round of review 

Here is how scores are calculated: 

1. If only zero-weighted scores are available (here, in the first round of review), the average of all zero-weighted scores should be shown in grey font. An information button should explain why the scores are shown in grey (“The reviews submitted so far do not count toward the author’s final grade.”): 

2. If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is:: 
80%⨉90% + 100%⨉10% = 82%

3. If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. '''However, the previous review score that was given in the current round will count until the reviewer updates the submitted review'''. '''This functionality doesn’t work for multipart rubrics.''' 

The system also doesn’t generate emails to reviewers in case a new submission is made by the author without changing submission parameters like submission URL. Eg: A new submission is made by making a new deployment. But the deployment URL doesn’t change. In this case as well, without changing the current flow the author should have an option to notify all the reviewers to revisit the submission.

=====Interface Requirement=====
The overall user interface would remain the same as of the current system. Since this is a modification to the current submission page, there will be a new addition to the submission page. On the submission page, there will be a new button to notify reviewers when there is a submission which doesn’t contain a change to submission parameters like submission URL, but the deployment has been updated on the same URL. Other than that the interface remains the same for the score page, only functionalities will be tested for multi-part rubrics.

=====Quality Requirements=====
The interface changes are minimal and very easy to adapt. The new button will be clearly visible and is supposed to have a clear text to interpret its functionality. The proposed changes should not affect the current system functionality in any way. The solution is only intended to enrich the project features.

=====Portability Requirements=====
All of the changes for this enhancement to Expertiza should not interfere with the platform portability in any way.

=====Performance Requirements=====
Adding test for multi-part rubric functionality should not cause any overhead in the performance of the system. Sending emails through “Notify All Reviewers” button should affect the system minimally in terms of performance. The only overhead is a database query to get all the reviewers. A good database design should incorporate such a query smoothly.

=====Assumptions=====
The functionality should work in case the assignment rubric is not multi-part.

== '''Design''' ==
=====Zero-weighted scores=====
If only zero-weighted scores are available in the first round of review, the average of all zero-weighted scores will be shown in a gray font. An information button will explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade”)

Inside compute_total_score method in on_the_fly_calc.rb module, we'll check if the weighted score exists, if not then we'll calculate the non weighted score. We'll also have to add a method in the questionnaire model to get non weighted score. Few changes are required for the UI to display non-weighted score in grey with information icon and non weighted score in black.

Possible files to be modified.

on_the_fly_calc.rb 
questionnaire.rb 
view_team.html.erb

===== Non-zero weighted scores=====
If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is 80%⨉90% + 100%⨉10% = 82%

We'll first write the test cases by adding test cases to below files.
spec/models/on_the_fly_calc_spec.rb

Also, we'll test this manually by computing the score and if there's a bug then below are the possible files to be modified.

models/on_the_fly_calc.rb 
models/questionnarie.rb

=====Review Update=====
If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review. This functionality already exists in the existing system. However, In some submissions, there’s no change in the submitted URL’s so there’s no way to let the reviewers know that the author has made some changes in the project. We’ll add a notify button for the Author in the UI which will trigger an email to the reviewers. This will reopen the review, and then the reviewer can go in and update the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review

'''Sending an email when the submission parameters don’t change:'''

A new button will be added on the submission page. The button will enable the author to notify all the reviewers to revisit the submission. We could also add a custom message in the
mail body so that the authors know the specific reason as to why they received this particular mail.

=====Profile Email Preference=====
The checkboxes are moved to the left corner. As the layout of the options have used table layout. So, we'll increase the width of the the column which includes checkboxes. We'll be using the percentage instead of fixed value to give the width, as this would adjust the column size based on the screen size. The CSS can be applied to the '''layout.scss''' file to using the appropriate class given to the column 

[[File:E1975_CHECKBOX_1.jpg]]

=='''Test Plan'''==

=====Rspec=====
We have added new testcases for the new pieces of code that were added by us.

=====Test Plan for Email Preferences=====

The email preferences checkboxes at present appear sticking to the rightmost border of the page. By following the requirements the checkboxes will be made available closer to the text fields to make them look more user-friendly.
The changes can be tested post code changes.

=====Manual Testing=====

The instructor will create an assignment and assign the zero and non-zero weights to the rubrics.

· Login as an author and go to scores page to view the feedback.

Test – Zero weighted scores shown in gray.

Test – Zero weighted scores shown in gray.

· Zero weighted scores should be displayed in gray to the user.

Test – Information button.

· The information button should explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade.”)

Test – Display non-zero weighted scores.

· The scores shown for the assignment should be the weighted average of the scores, if any non-zero scores are available
· The zero weighted scores have to be replaced with non-zero weighted scores.

Test – Update review score.
· If author updates the submission before the end of current round. The reviewer should be able to go and update the submitted review.
· If the review is not updated the previous review score has to be retained.

Test – Multipart part rubric.
· The functionality should work for multipart rubric.
· The scores calculated have to be accurate.

CSC/ECE 517 Fall 2019 - E1975. Generalize Review Versioning

2019-11-11T21:55:14Z

Ajain37: /* Zero-weighted scores */

__TOC__

== '''About Expertiza''' ==

Expertiza is an open source project based on Ruby on Rails framework and the code is available on Github. Expertiza allows the instructor to create new assignments as well as edit new or existing assignments. Instructors can also create a list of topics the students can sign up for and specify deadlines for completion of various tasks. Students can form teams in Expertiza to work on various projects and assignments as well as peer review other students' submissions. Expertiza supports submission across various document types, including the URLs Wiki pages.
 
 

== '''Introduction''' ==
Expertiza is a peer-review based learning platform in which work-products by students are evaluated and scored by peer-review of other students. Review process occurs in stages. After submission of initial feedback for a work-product by the peer-group, students are allowed time to incorporate the comments and improve their work. This second version of the work product is reviewed again and scores are given. This process might be repeated again and the average score from the last review stage is considered as the final score for that work-product. This incremental development of work-products and progressive learning is the fundamental concept underlying the Expertiza system.

=====Purpose=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The system also triggers an email to reviewer when there is a change in the submission. The scores are calculated on the fly based on the rubrics.

=====Scope=====
This project will enhance the existing scoring and peer review functionality and the underlying implementation along with the testing of the existing implementation. Testing of the score calculation, display for the weighted and non-weighted scores based on the combination of the rubrics are part of this project. This project also includes an additional feature for the author to notify the reviewers for the change in the submission.

=====Background=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The scores are calculated based on the rubrics defined in the system. There’s already an existing implementation for the computation of score but it doesn’t work for multipart rubric

=====Documentation=====
The documents that will be generated for this project are the design document, the actual files that we edit or create, and a ReadMe that explains what we did and how to use the modified product.

=====Standards=====
The new standards that will be adhered to in this system are those given for the rails framework, and the code should follow rails and object oriented design principles. Any newly added code will adhere to the guidelines mentioned below:

● https://github.com/rubocop-hq/rails-style-guide 
● http://wiki.expertiza.ncsu.edu/index.php/Object-Oriented_Design_and_Programming.

== '''System Requirements''' ==
=====Problem=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The functionality needs to be tested. It works in some cases, but we are suspicious that it is not entirely correct.

'''Profile Email Preference''':
User in expertiza has liberty to control what kind of emails they should receive. As the reviewers will receive an email for the change in submission. This can be controlled by selecting an option (When someone else submits work I am assigned to review) on the profile page of the user. As the checkboxes are far away on the right corner of the screen, the position of checkboxes need to be changed to make it more user friendly.
 

Scores for students in Expertiza can be based on a combination of rubrics: review rubrics for each round of review, author feedback, and teammate review. In CSC/ECE 517, we don’t really use the student-assigned scores to grade projects, but the peer-review score shown to the student is based entirely on the second round of review 

Here is how scores are calculated: 

1. If only zero-weighted scores are available (here, in the first round of review), the average of all zero-weighted scores should be shown in grey font. An information button should explain why the scores are shown in grey (“The reviews submitted so far do not count toward the author’s final grade.”): 

2. If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is:: 
80%⨉90% + 100%⨉10% = 82%

3. If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. '''However, the previous review score that was given in the current round will count until the reviewer updates the submitted review'''. '''This functionality doesn’t work for multipart rubrics.''' 

The system also doesn’t generate emails to reviewers in case a new submission is made by the author without changing submission parameters like submission URL. Eg: A new submission is made by making a new deployment. But the deployment URL doesn’t change. In this case as well, without changing the current flow the author should have an option to notify all the reviewers to revisit the submission.

=====Interface Requirement=====
The overall user interface would remain the same as of the current system. Since this is a modification to the current submission page, there will be a new addition to the submission page. On the submission page, there will be a new button to notify reviewers when there is a submission which doesn’t contain a change to submission parameters like submission URL, but the deployment has been updated on the same URL. Other than that the interface remains the same for the score page, only functionalities will be tested for multi-part rubrics.

=====Quality Requirements=====
The interface changes are minimal and very easy to adapt. The new button will be clearly visible and is supposed to have a clear text to interpret its functionality. The proposed changes should not affect the current system functionality in any way. The solution is only intended to enrich the project features.

=====Portability Requirements=====
All of the changes for this enhancement to Expertiza should not interfere with the platform portability in any way.

=====Performance Requirements=====
Adding test for multi-part rubric functionality should not cause any overhead in the performance of the system. Sending emails through “Notify All Reviewers” button should affect the system minimally in terms of performance. The only overhead is a database query to get all the reviewers. A good database design should incorporate such a query smoothly.

=====Assumptions=====
The functionality should work in case the assignment rubric is not multi-part.

== '''Design''' ==
=====Zero-weighted scores=====
If only zero-weighted scores are available in the first round of review, the average of all zero-weighted scores will be shown in a gray font. An information button will explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade”)

Inside compute_total_score method in on_the_fly_calc.rb module, we'll check if the weighted score exists, if not then we'll calculate the non weighted score. We'll also have to add a method in the questionnaire model to get non weighted score. Few changes are required for the UI to display non-weighted score in grey with information icon and non weighted score in black.

Possible files to be modified.

on_the_fly_calc.rb 
questionnaire.rb 
view_team.html.erb

===== Non-zero weighted scores=====
If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is 80%⨉90% + 100%⨉10% = 82%

=====Review Update=====
If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review. This functionality already exists in the existing system. However, In some submissions, there’s no change in the submitted URL’s so there’s no way to let the reviewers know that the author has made some changes in the project. We’ll add a notify button for the Author in the UI which will trigger an email to the reviewers. This will reopen the review, and then the reviewer can go in and update the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review

'''Sending an email when the submission parameters don’t change:'''

A new button will be added on the submission page. The button will enable the author to notify all the reviewers to revisit the submission. We could also add a custom message in the
mail body so that the authors know the specific reason as to why they received this particular mail.

=====Profile Email Preference=====
The checkboxes are moved to the left corner. As the layout of the options have used table layout. So, we'll increase the width of the the column which includes checkboxes. We'll be using the percentage instead of fixed value to give the width, as this would adjust the column size based on the screen size. The CSS can be applied to the '''layout.scss''' file to using the appropriate class given to the column 

[[File:E1975_CHECKBOX_1.jpg]]

=='''Test Plan'''==

=====Rspec=====
We have added new testcases for the new pieces of code that were added by us.

=====Test Plan for Email Preferences=====

The email preferences checkboxes at present appear sticking to the rightmost border of the page. By following the requirements the checkboxes will be made available closer to the text fields to make them look more user-friendly.
The changes can be tested post code changes.

=====Manual Testing=====

The instructor will create an assignment and assign the zero and non-zero weights to the rubrics.

· Login as an author and go to scores page to view the feedback.

Test – Zero weighted scores shown in gray.

Test – Zero weighted scores shown in gray.

· Zero weighted scores should be displayed in gray to the user.

Test – Information button.

· The information button should explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade.”)

Test – Display non-zero weighted scores.

· The scores shown for the assignment should be the weighted average of the scores, if any non-zero scores are available
· The zero weighted scores have to be replaced with non-zero weighted scores.

Test – Update review score.
· If author updates the submission before the end of current round. The reviewer should be able to go and update the submitted review.
· If the review is not updated the previous review score has to be retained.

Test – Multipart part rubric.
· The functionality should work for multipart rubric.
· The scores calculated have to be accurate.

CSC/ECE 517 Fall 2019 - E1975. Generalize Review Versioning

2019-11-11T21:54:46Z

Ajain37: /* Zero-weighted scores */

__TOC__

== '''About Expertiza''' ==

Expertiza is an open source project based on Ruby on Rails framework and the code is available on Github. Expertiza allows the instructor to create new assignments as well as edit new or existing assignments. Instructors can also create a list of topics the students can sign up for and specify deadlines for completion of various tasks. Students can form teams in Expertiza to work on various projects and assignments as well as peer review other students' submissions. Expertiza supports submission across various document types, including the URLs Wiki pages.
 
 

== '''Introduction''' ==
Expertiza is a peer-review based learning platform in which work-products by students are evaluated and scored by peer-review of other students. Review process occurs in stages. After submission of initial feedback for a work-product by the peer-group, students are allowed time to incorporate the comments and improve their work. This second version of the work product is reviewed again and scores are given. This process might be repeated again and the average score from the last review stage is considered as the final score for that work-product. This incremental development of work-products and progressive learning is the fundamental concept underlying the Expertiza system.

=====Purpose=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The system also triggers an email to reviewer when there is a change in the submission. The scores are calculated on the fly based on the rubrics.

=====Scope=====
This project will enhance the existing scoring and peer review functionality and the underlying implementation along with the testing of the existing implementation. Testing of the score calculation, display for the weighted and non-weighted scores based on the combination of the rubrics are part of this project. This project also includes an additional feature for the author to notify the reviewers for the change in the submission.

=====Background=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The scores are calculated based on the rubrics defined in the system. There’s already an existing implementation for the computation of score but it doesn’t work for multipart rubric

=====Documentation=====
The documents that will be generated for this project are the design document, the actual files that we edit or create, and a ReadMe that explains what we did and how to use the modified product.

=====Standards=====
The new standards that will be adhered to in this system are those given for the rails framework, and the code should follow rails and object oriented design principles. Any newly added code will adhere to the guidelines mentioned below:

● https://github.com/rubocop-hq/rails-style-guide 
● http://wiki.expertiza.ncsu.edu/index.php/Object-Oriented_Design_and_Programming.

== '''System Requirements''' ==
=====Problem=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The functionality needs to be tested. It works in some cases, but we are suspicious that it is not entirely correct.

'''Profile Email Preference''':
User in expertiza has liberty to control what kind of emails they should receive. As the reviewers will receive an email for the change in submission. This can be controlled by selecting an option (When someone else submits work I am assigned to review) on the profile page of the user. As the checkboxes are far away on the right corner of the screen, the position of checkboxes need to be changed to make it more user friendly.
 

Scores for students in Expertiza can be based on a combination of rubrics: review rubrics for each round of review, author feedback, and teammate review. In CSC/ECE 517, we don’t really use the student-assigned scores to grade projects, but the peer-review score shown to the student is based entirely on the second round of review 

Here is how scores are calculated: 

1. If only zero-weighted scores are available (here, in the first round of review), the average of all zero-weighted scores should be shown in grey font. An information button should explain why the scores are shown in grey (“The reviews submitted so far do not count toward the author’s final grade.”): 

2. If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is:: 
80%⨉90% + 100%⨉10% = 82%

3. If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. '''However, the previous review score that was given in the current round will count until the reviewer updates the submitted review'''. '''This functionality doesn’t work for multipart rubrics.''' 

The system also doesn’t generate emails to reviewers in case a new submission is made by the author without changing submission parameters like submission URL. Eg: A new submission is made by making a new deployment. But the deployment URL doesn’t change. In this case as well, without changing the current flow the author should have an option to notify all the reviewers to revisit the submission.

=====Interface Requirement=====
The overall user interface would remain the same as of the current system. Since this is a modification to the current submission page, there will be a new addition to the submission page. On the submission page, there will be a new button to notify reviewers when there is a submission which doesn’t contain a change to submission parameters like submission URL, but the deployment has been updated on the same URL. Other than that the interface remains the same for the score page, only functionalities will be tested for multi-part rubrics.

=====Quality Requirements=====
The interface changes are minimal and very easy to adapt. The new button will be clearly visible and is supposed to have a clear text to interpret its functionality. The proposed changes should not affect the current system functionality in any way. The solution is only intended to enrich the project features.

=====Portability Requirements=====
All of the changes for this enhancement to Expertiza should not interfere with the platform portability in any way.

=====Performance Requirements=====
Adding test for multi-part rubric functionality should not cause any overhead in the performance of the system. Sending emails through “Notify All Reviewers” button should affect the system minimally in terms of performance. The only overhead is a database query to get all the reviewers. A good database design should incorporate such a query smoothly.

=====Assumptions=====
The functionality should work in case the assignment rubric is not multi-part.

== '''Design''' ==
=====Zero-weighted scores=====
If only zero-weighted scores are available in the first round of review, the average of all zero-weighted scores will be shown in a gray font. An information button will explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade”)

Inside compute_total_score method in on_the_fly_calc.rb module, we'll check if the weighted score exists, if not then we'll calculate the non weighted score. We'll also have to add a method in the questionnaire model to get non weighted score. Few changes are required for the UI to display non-weighted score in grey with information icon and non weighted score in black.

Files which will be modified.

on_the_fly_calc.rb 
questionnaire.rb 
view_team.html.erb

===== Non-zero weighted scores=====
If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is 80%⨉90% + 100%⨉10% = 82%

=====Review Update=====
If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review. This functionality already exists in the existing system. However, In some submissions, there’s no change in the submitted URL’s so there’s no way to let the reviewers know that the author has made some changes in the project. We’ll add a notify button for the Author in the UI which will trigger an email to the reviewers. This will reopen the review, and then the reviewer can go in and update the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review

'''Sending an email when the submission parameters don’t change:'''

A new button will be added on the submission page. The button will enable the author to notify all the reviewers to revisit the submission. We could also add a custom message in the
mail body so that the authors know the specific reason as to why they received this particular mail.

=====Profile Email Preference=====
The checkboxes are moved to the left corner. As the layout of the options have used table layout. So, we'll increase the width of the the column which includes checkboxes. We'll be using the percentage instead of fixed value to give the width, as this would adjust the column size based on the screen size. The CSS can be applied to the '''layout.scss''' file to using the appropriate class given to the column 

[[File:E1975_CHECKBOX_1.jpg]]

=='''Test Plan'''==

=====Rspec=====
We have added new testcases for the new pieces of code that were added by us.

=====Test Plan for Email Preferences=====

The email preferences checkboxes at present appear sticking to the rightmost border of the page. By following the requirements the checkboxes will be made available closer to the text fields to make them look more user-friendly.
The changes can be tested post code changes.

=====Manual Testing=====

The instructor will create an assignment and assign the zero and non-zero weights to the rubrics.

· Login as an author and go to scores page to view the feedback.

Test – Zero weighted scores shown in gray.

Test – Zero weighted scores shown in gray.

· Zero weighted scores should be displayed in gray to the user.

Test – Information button.

· The information button should explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade.”)

Test – Display non-zero weighted scores.

· The scores shown for the assignment should be the weighted average of the scores, if any non-zero scores are available
· The zero weighted scores have to be replaced with non-zero weighted scores.

Test – Update review score.
· If author updates the submission before the end of current round. The reviewer should be able to go and update the submitted review.
· If the review is not updated the previous review score has to be retained.

Test – Multipart part rubric.
· The functionality should work for multipart rubric.
· The scores calculated have to be accurate.

CSC/ECE 517 Fall 2019 - E1975. Generalize Review Versioning

2019-11-11T21:52:01Z

Ajain37: /* Zero-weighted scores */

__TOC__

== '''About Expertiza''' ==

Expertiza is an open source project based on Ruby on Rails framework and the code is available on Github. Expertiza allows the instructor to create new assignments as well as edit new or existing assignments. Instructors can also create a list of topics the students can sign up for and specify deadlines for completion of various tasks. Students can form teams in Expertiza to work on various projects and assignments as well as peer review other students' submissions. Expertiza supports submission across various document types, including the URLs Wiki pages.
 
 

== '''Introduction''' ==
Expertiza is a peer-review based learning platform in which work-products by students are evaluated and scored by peer-review of other students. Review process occurs in stages. After submission of initial feedback for a work-product by the peer-group, students are allowed time to incorporate the comments and improve their work. This second version of the work product is reviewed again and scores are given. This process might be repeated again and the average score from the last review stage is considered as the final score for that work-product. This incremental development of work-products and progressive learning is the fundamental concept underlying the Expertiza system.

=====Purpose=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The system also triggers an email to reviewer when there is a change in the submission. The scores are calculated on the fly based on the rubrics.

=====Scope=====
This project will enhance the existing scoring and peer review functionality and the underlying implementation along with the testing of the existing implementation. Testing of the score calculation, display for the weighted and non-weighted scores based on the combination of the rubrics are part of this project. This project also includes an additional feature for the author to notify the reviewers for the change in the submission.

=====Background=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The scores are calculated based on the rubrics defined in the system. There’s already an existing implementation for the computation of score but it doesn’t work for multipart rubric

=====Documentation=====
The documents that will be generated for this project are the design document, the actual files that we edit or create, and a ReadMe that explains what we did and how to use the modified product.

=====Standards=====
The new standards that will be adhered to in this system are those given for the rails framework, and the code should follow rails and object oriented design principles. Any newly added code will adhere to the guidelines mentioned below:

● https://github.com/rubocop-hq/rails-style-guide 
● http://wiki.expertiza.ncsu.edu/index.php/Object-Oriented_Design_and_Programming.

== '''System Requirements''' ==
=====Problem=====
The system currently is designed to give a reviewer a new form for each round of review (rather than requiring the reviewer to edit an existing file) and automatically remove scores of reviews that are not redone. The functionality needs to be tested. It works in some cases, but we are suspicious that it is not entirely correct.

'''Profile Email Preference''':
User in expertiza has liberty to control what kind of emails they should receive. As the reviewers will receive an email for the change in submission. This can be controlled by selecting an option (When someone else submits work I am assigned to review) on the profile page of the user. As the checkboxes are far away on the right corner of the screen, the position of checkboxes need to be changed to make it more user friendly.
 

Scores for students in Expertiza can be based on a combination of rubrics: review rubrics for each round of review, author feedback, and teammate review. In CSC/ECE 517, we don’t really use the student-assigned scores to grade projects, but the peer-review score shown to the student is based entirely on the second round of review 

Here is how scores are calculated: 

1. If only zero-weighted scores are available (here, in the first round of review), the average of all zero-weighted scores should be shown in grey font. An information button should explain why the scores are shown in grey (“The reviews submitted so far do not count toward the author’s final grade.”): 

2. If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is:: 
80%⨉90% + 100%⨉10% = 82%

3. If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. '''However, the previous review score that was given in the current round will count until the reviewer updates the submitted review'''. '''This functionality doesn’t work for multipart rubrics.''' 

The system also doesn’t generate emails to reviewers in case a new submission is made by the author without changing submission parameters like submission URL. Eg: A new submission is made by making a new deployment. But the deployment URL doesn’t change. In this case as well, without changing the current flow the author should have an option to notify all the reviewers to revisit the submission.

=====Interface Requirement=====
The overall user interface would remain the same as of the current system. Since this is a modification to the current submission page, there will be a new addition to the submission page. On the submission page, there will be a new button to notify reviewers when there is a submission which doesn’t contain a change to submission parameters like submission URL, but the deployment has been updated on the same URL. Other than that the interface remains the same for the score page, only functionalities will be tested for multi-part rubrics.

=====Quality Requirements=====
The interface changes are minimal and very easy to adapt. The new button will be clearly visible and is supposed to have a clear text to interpret its functionality. The proposed changes should not affect the current system functionality in any way. The solution is only intended to enrich the project features.

=====Portability Requirements=====
All of the changes for this enhancement to Expertiza should not interfere with the platform portability in any way.

=====Performance Requirements=====
Adding test for multi-part rubric functionality should not cause any overhead in the performance of the system. Sending emails through “Notify All Reviewers” button should affect the system minimally in terms of performance. The only overhead is a database query to get all the reviewers. A good database design should incorporate such a query smoothly.

=====Assumptions=====
The functionality should work in case the assignment rubric is not multi-part.

== '''Design''' ==
=====Zero-weighted scores=====
If only zero-weighted scores are available in the first round of review, the average of all zero-weighted scores will be shown in a gray font. An information button will explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade”)

Inside compute_total_score method in on_the_fly_calc.rb module, we'll check if the weighted score exists, if not then we'll calculate the non weighted score. We'll also have to add a method in the questionnaire model to get non weighted score.

We'll also have to make changes for the UI to display non-weighted score in grey with information icon and non weighted score in black.

Files which will be modified.

on_the_fly_calc.rb 
questionnaire.rb
view_team.html.erb

===== Non-zero weighted scores=====
If any non-zero weighted scores are available, then the score shown for the assignment should be the weighted average of the scores. For example, if Round 2 reviews are weighted 90% and author feedback is weighted 10%, and two Round 2 reviews each gave the work a score of 80%, and the only author-feedback score was 100%, then the overall score is 80%⨉90% + 100%⨉10% = 82%

=====Review Update=====
If a review is submitted, and then the author(s) update the submission before the end of the current round, it will reopen the review, and then the reviewer can go in and update the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review. This functionality already exists in the existing system. However, In some submissions, there’s no change in the submitted URL’s so there’s no way to let the reviewers know that the author has made some changes in the project. We’ll add a notify button for the Author in the UI which will trigger an email to the reviewers. This will reopen the review, and then the reviewer can go in and update the submitted review. However, the previous review score that was given in the current round will count until the reviewer updates the submitted review

'''Sending an email when the submission parameters don’t change:'''

A new button will be added on the submission page. The button will enable the author to notify all the reviewers to revisit the submission. We could also add a custom message in the
mail body so that the authors know the specific reason as to why they received this particular mail.

=====Profile Email Preference=====
The checkboxes are moved to the left corner. As the layout of the options have used table layout. So, we'll increase the width of the the column which includes checkboxes. We'll be using the percentage instead of fixed value to give the width, as this would adjust the column size based on the screen size. The CSS can be applied to the '''layout.scss''' file to using the appropriate class given to the column 

[[File:E1975_CHECKBOX_1.jpg]]

=='''Test Plan'''==

=====Rspec=====
We have added new testcases for the new pieces of code that were added by us.

=====Test Plan for Email Preferences=====

The email preferences checkboxes at present appear sticking to the rightmost border of the page. By following the requirements the checkboxes will be made available closer to the text fields to make them look more user-friendly.
The changes can be tested post code changes.

=====Manual Testing=====

The instructor will create an assignment and assign the zero and non-zero weights to the rubrics.

· Login as an author and go to scores page to view the feedback.

Test – Zero weighted scores shown in gray.

Test – Zero weighted scores shown in gray.

· Zero weighted scores should be displayed in gray to the user.

Test – Information button.

· The information button should explain why the scores are shown in gray (“The reviews submitted so far do not count toward the author’s final grade.”)

Test – Display non-zero weighted scores.

· The scores shown for the assignment should be the weighted average of the scores, if any non-zero scores are available
· The zero weighted scores have to be replaced with non-zero weighted scores.

Test – Update review score.
· If author updates the submission before the end of current round. The reviewer should be able to go and update the submitted review.
· If the review is not updated the previous review score has to be retained.

Test – Multipart part rubric.
· The functionality should work for multipart rubric.
· The scores calculated have to be accurate.

CSC/ECE 517 Fall 2019 - E1975. Generalize Review Versioning

2019-11-11T21:50:24Z

Ajain37: /* Zero-weighted scores */

CSC/ECE 517 Fall 2019 - E1975. Generalize Review Versioning

2019-11-11T21:50:10Z

Ajain37: /* Zero-weighted scores */

CSC/ECE 517 Fall 2019 - E1975. Generalize Review Versioning

2019-11-11T20:50:20Z

Ajain37: /* Test Plan for Email Preferences */

CSC/ECE 517 Fall 2019 - E1975. Generalize Review Versioning

2019-11-11T20:49:56Z

Ajain37: /* Test Plan for Email Preferences */

CSC/ECE 517 Fall 2019 - E1975. Generalize Review Versioning

2019-11-11T20:49:37Z

Ajain37: /* Profile Email Preference */

CSC/ECE 517 Fall 2019 - E1975. Generalize Review Versioning

2019-11-11T20:49:25Z

Ajain37: /* Profile Email Preference */

CSC/ECE 517 Fall 2019 - E1975. Generalize Review Versioning

2019-11-11T20:48:31Z

Ajain37: /* Profile Email Preference */

CSC/ECE 517 Fall 2019 - E1975. Generalize Review Versioning

2019-11-11T20:37:34Z

Ajain37: /* Profile Email Preference */