Expertiza_Wiki - User contributions [en]

CSC/ECE 517 Fall 2017/E1777 Coherent specification of review requirements.rb

2017-11-03T02:06:12Z

Ssuresh6:

==E1777. OSS Project Green: Coherent specification of review requirements==

This page provides a description of the Expertiza based OSS project.

__TOC__

===About Expertiza===

[http://expertiza.ncsu.edu/ Expertiza] is an open source project based on [http://rubyonrails.org/ Ruby on Rails] framework. Expertiza allows the instructor to create new assignments and customize new or existing assignments. It also allows the instructor to create a list of topics the students can sign up for. Students can form teams in Expertiza to work on various projects and assignments. Students can also peer review other students' submissions. Expertiza supports submission across various document types, including the URLs and wiki pages.

===Problem Statement===
The following tasks were accomplished in this project:

* Information buttons to elaborate difference between the synonymous statements present in review strategy tab need to be included.
* The punctuation, syntax and capitalization of statements in the review strategy tab requires improvement.
* Message to a reviewer regarding how many reviews they are required to and allowed to perform is not clear enough.
* Reviewers are assigned new reviews based on number of previous reviews ''assigned'' for that submission, which is incorrect.

===Implementation===

=====Drawbacks and Solutions=====
'''Issue #402 ''': To allow reviewers a larger set of topics to choose from, the instructor can set the threshold (on the Review Strategy tab of assignment creation/editing) to some integer k > 0. Then any submission that has within k reviews of the fewest number can be chosen by a new reviewer. Let’s say that all submissions have at least 1 review. If k = 3, then the reviewer can choose any topic where there is a submission that has 4 or fewer reviews so far. Suppose that the minimum number of reviews for any submission is 2, but that I have reviewed all the submissions that have only 2 reviews. Then I’m not allowed to review at all (unless k > 0).

* '''Solution''': This issue has been fixed previously. The reviewer will get assigned a submission even if it has fulfilled the required number of reviews, to ensure that the reviewer always receives a new submission to review as long as it is not his own.

'''Issue #969''':
*a.The two statements "Set Allowed Number of Reviews per reviewer" and "Set Required Number of Reviews per reviewer." are not well differentiated. It can very confusing for an instructor to give values for them.

* '''Solution''': Information buttons have been provided beside these statements in the review strategy tab to make sure that the instructor knows the difference between the two.
This modification is made in the app/views/assignments/edit/_review_strategy.html.erb
<pre>

<img src="/assets/info.png" title='This is the minimum number of reviews a reviewer has to perform'>
<br/>
<br/>
<%= check_box_tag('review', 'true', true, {:onChange => 'hasReviewChanged()'}) %>
<%= label_tag('review', 'Has max review limit?') %>&emsp;&emsp;
<span id="reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_reviews_allowed]', ' Set allowed number of reviews per reviewer') %>
<%if @assignment_form.assignment.num_reviews_allowed && @assignment_form.assignment.num_reviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', @assignment_form.assignment.num_reviews_allowed, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', 3, size: 1) %>
<%end%>

<img src="/assets/info.png" title='This is the maximum number of reviews a reviewer can perform'<%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
</span>

</td>
</pre>
[[File:INFOBUT.PNG]]

[[File:INFOBUT2.PNG]]

*b) When the number of allowed or required reviews is not set on the Review Strategy tab, the system does not have a message to display to a reviewer about how many submissions of work they are required to and allowed to review depending on the values set by the instructor in the "allowed number of reviewers per reviewer" field and the "Set Required Number of Reviews per reviewer." field.

*'''Solution''': This view has been implemented in the app/views/student_review/list.html.erb
<pre>

<h2>Reviews for "<%= @assignment.name %>"</h2>


<% if @assignment.num_reviews_allowed.nil? || @assignment.num_reviews_allowed == -1%>
<h4>Your instructor expects you to do <%= @assignment.num_reviews_required %> reviews. You are not allowed to do any extra reviews. </h4>

<% elsif @assignment.num_reviews_allowed == @assignment.num_reviews_required %>
<h4>You should perform exactly <%= @assignment.num_reviews_allowed %> reviews </h4>
<% else %>

<h4>You may perform between <%= @assignment.num_reviews_required %> and <%= @assignment.num_reviews_allowed %> reviews</h4>
<% end %>
</pre>

*c)The capitalization and punctuation of statements such as "Set Allowed Number of Reviews per reviewer" and "Set Required Number of Reviews per reviewer." in review strategy tab are incorrect.
*'''Solution''': This view has been fixed by making changes in the respective files.
The changes were made in the file : app/views/assignments/edit/_review_strategy.html.erb
<pre>
<%= label_tag('assignment_form[assignment][num_reviews_required]', 'Set required number of reviews per reviewer') %>
<%= text_field_tag('assignment_form[assignment][num_reviews_required]', @assignment_form.assignment.num_reviews_required ||= @assignment_form.assignment.num_reviews, size: 1) %>


<img src="/assets/info.png" title='This is the minimum number of reviews a reviewer has to perform'>
<br/>
<br/>
<%= check_box_tag('review', 'true', true, {:onChange => 'hasReviewChanged()'}) %>
<%= label_tag('review', 'Has max review limit?') %>&emsp;&emsp;
<span id="reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_reviews_allowed]', ' Set allowed number of reviews per reviewer') %>
<%if @assignment_form.assignment.num_reviews_allowed && @assignment_form.assignment.num_reviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', @assignment_form.assignment.num_reviews_allowed, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', 3, size: 1) %>
<%end%>

<img src="/assets/info.png" title='This is the maximum number of reviews a reviewer can perform'<%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
</span>

</td>
<td width="50%">

<%= label_tag('assignment_form[assignment][num_metareviews_required]', 'Set required number of meta-reviews per reviewer') %>
<%= text_field_tag('assignment_form[assignment][num_metareviews_required]', @assignment_form.assignment.num_metareviews_required ||= @assignment_form.assignment.num_metareviews_allowed, size: 1) %>
<br/>
<br/>

<%= check_box_tag('meta_review', 'true', true, {:onChange => 'hasMetaReviewChanged()'}) %>
<%= label_tag('meta_review', 'Has meta-review limit?') %>&emsp;&emsp;
<span id="meta_reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_metareviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_metareviews_allowed]', ' Set allowed number of meta-reviews per reviewer') %>
<%if @assignment_form.assignment.num_metareviews_allowed && @assignment_form.assignment.num_metareviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_metareviews_allowed]', @assignment_form.assignment.num_metareviews_allowed ||= 3, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_metareviews_allowed]', 3, size: 1) %>
<%end%>
</span>

</td>
</tr>

</pre>

* '''Issue #965''': The assignment of work to review to a reviewer depends on the number of reviews it has previously received, if that work has been assigned to a reviewer previously and its review has
not been submitted then the system will still consider it such that it would not assign the same work to another reviewer if it has enough number of reviewers previously assigned to review it. This might
lead to a scenario where that work was assigned to enough number of reviewers, but it does not receive enough reviews and cannot be assigned to any new reviewers either.
*'''Solution''': This implementation has been fixed. The system now considers only the reviews that were submitted by the reviewers for that work so that every submission gets assigned to reviewers until
it receives its allowed number of reviews.

The modified file is app/models/review_assignment.rb and the modification is done in the method reject_by_max_reviews_per_submission

<pre>
def reject_by_max_reviews_per_submission(contributor_set)
contributor_set.reject! {|contributor| contributor.responses.reject {|response| !response.is_submitted }.count >= max_reviews_per_submission }
contributor_set
end
</pre>

==Test Plans==
'''Issue #402''' :This is shown graphically with dummy topics and assignment. We create a dummy assignment with two different topics oss1 and oss2. oss1 has 2 slots and oss2 has 1 slot.
So when a student who has submitted oss2 tries to review others work he will not be able to select oos2 at all because he cannot review his own submission.
When a person who has oss1 as topic goes to this page and he asks to get a new submission for review it will give another slot’s oss1.
If he does it again it says there are no more submissions available to review for this topic.
So if he has already reviewed all the other submissions of his topic oss1 and he requests for one more submission of oss1 , he will not get any more submission to review because the only one left is his own submission.

'''Issue #969''' :Set allowed number and required number of reviews per submission as 3 and 3 respectively.
Now it will display "you should perform exactly 3 reviews."

Before:

[[File:issue2.PNG]]

After:

[[File:issue2solved.PNG]]

[[File:INFOBUT3.PNG]]

'''Issue #965''' :We created a dummy assignment with a team size of 2, 3 such teams are created. Maximum allowed reviews is 2 now.
Previously for a submission after 2 people begin review its does not give any further submission to review to that user because it already reached the limit.

[[File: issue3.PNG]]

On changing the method, when we try to get a new submission we receive one. This is because the previous reviewers have only begun their reviews but have not submitted them yet.

[[File:crappy.PNG]]

Now when the previous two reviewers submit the reviews they began. Now the same new reviewer will not be able to submit his review

[[File:issue3solved.PNG]]

==Modified Files==
* app/views/assignments/edit/_review_strategy.html.erb
* app/models/review_assignment.rb
* app/views/student_review/list.html.erb

===References===

#[https://github.com/expertiza/expertiza Expertiza on GitHub]
#[https://github.com/WintersLt/expertiza GitHub Project Repository Fork]
#[http://expertiza.ncsu.edu/ The live Expertiza website]
#[http://wikis.lib.ncsu.edu/index.php/Expertiza Expertiza project documentation wiki]
#Clean Code: A handbook of agile software craftsmanship. Author: Robert C Martin

CSC/ECE 517 Fall 2017/E1777 Coherent specification of review requirements.rb

2017-11-01T18:44:43Z

Ssuresh6: /* Implementation */

==E1777. OSS Project Green: Coherent specification of review requirements==

This page provides a description of the Expertiza based OSS project.

__TOC__

===About Expertiza===

[http://expertiza.ncsu.edu/ Expertiza] is an open source project based on [http://rubyonrails.org/ Ruby on Rails] framework. Expertiza allows the instructor to create new assignments and customize new or existing assignments. It also allows the instructor to create a list of topics the students can sign up for. Students can form teams in Expertiza to work on various projects and assignments. Students can also peer review other students' submissions. Expertiza supports submission across various document types, including the URLs and wiki pages.

===Problem Statement===
The following tasks were accomplished in this project:

* Information buttons to elaborate difference between the synonymous statements present in review strategy tab need to be included.
* The punctuation, syntax and capitalization of statements in the review strategy tab requires improvement.
* Message to a reviewer regarding how many reviews they are required to and allowed to perform is not clear enough.
* Reviewers are assigned new reviews based on number of previous reviews ''assigned'' for that submission, which is incorrect.

===Implementation===

=====Drawbacks and Solutions=====
'''Issue #402 ''': To allow reviewers a larger set of topics to choose from, the instructor can set the threshold (on the Review Strategy tab of assignment creation/editing) to some integer k > 0. Then any submission that has within k reviews of the fewest number can be chosen by a new reviewer. Let’s say that all submissions have at least 1 review. If k = 3, then the reviewer can choose any topic where there is a submission that has 4 or fewer reviews so far. Suppose that the minimum number of reviews for any submission is 2, but that I have reviewed all the submissions that have only 2 reviews. Then I’m not allowed to review at all (unless k > 0).
* '''Solution''': This issue has been fixed previously. The reviewer will get assigned a submission even if it has fulfilled the required number of reviews, to ensure that the reviewer always receives a new submission to review as long as it is not his own.
*'''Test Plan''':This is shown graphically with dummy topics and assignment. We create a dummy assignment with two different topics oss1 and oss2. oss1 has 2 slots and oss2 has 1 slot.
So when a student who has submitted oss2 tries to review others work he will not be able to select oos2 at all because he cannot review his own submission.
When a person who has oss1 as topic goes to this page and he asks to get a new submission for review it will give another slot’s oss1.
If he does it again it says there are no more submissions available to review for this topic.
So if he has already reviewed all the other submissions of his topic oss1 and he requests for one more submission of oss1 , he will not get any more submission to review because the only one left is
his own submission.

'''Issue #969''':
*a.The two statements "Set Allowed Number of Reviews per reviewer" and "Set Required Number of Reviews per reviewer." are not well differentiated. It can very confusing for an instructor to give values for
them.

* '''Solution''': Information buttons have been provided beside these statements in the review strategy tab to make sure that the instructor knows the difference between the two.
This modification is made in the app/views/assignments/edit/_review_strategy.html.erb
<pre>

<img src="/assets/info.png" title='This is the minimum number of reviews a reviewer has to perform'>
<br/>
<br/>
<%= check_box_tag('review', 'true', true, {:onChange => 'hasReviewChanged()'}) %>
<%= label_tag('review', 'Has max review limit?') %>&emsp;&emsp;
<span id="reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_reviews_allowed]', ' Set allowed number of reviews per reviewer') %>
<%if @assignment_form.assignment.num_reviews_allowed && @assignment_form.assignment.num_reviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', @assignment_form.assignment.num_reviews_allowed, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', 3, size: 1) %>
<%end%>

<img src="/assets/info.png" title='This is the maximum number of reviews a reviewer can perform'<%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
</span>

</td>
</pre>
[[File:INFOBUT.PNG]]

[[File:INFOBUT2.PNG]]

*b) When the number of allowed or required reviews is not set on the Review Strategy tab, the system does not have a message to display to a reviewer about how many submissions of work they are required to and allowed to review depending on the values set by the instructor in the "allowed number of reviewers per reviewer" field and the "Set Required Number of Reviews per reviewer." field.

*'''Solution''': This view has been implemented in the app/views/student_review/list.html.erb
<pre>

<h2>Reviews for "<%= @assignment.name %>"</h2>


<% if @assignment.num_reviews_allowed.nil? || @assignment.num_reviews_allowed == -1%>
<h4>Your instructor expects you to do <%= @assignment.num_reviews_required %> reviews. You are not allowed to do any extra reviews. </h4>

<% elsif @assignment.num_reviews_allowed == @assignment.num_reviews_required %>
<h4>You should perform exactly <%= @assignment.num_reviews_allowed %> reviews </h4>
<% else %>

<h4>You may perform between <%= @assignment.num_reviews_required %> and <%= @assignment.num_reviews_allowed %> reviews</h4>
<% end %>
</pre>

*c)The capitalization and punctuation of statements such as "Set Allowed Number of Reviews per reviewer" and "Set Required Number of Reviews per reviewer." in review strategy tab are incorrect.
*'''Solution''': This view has been fixed by making changes in the respective files.
The changes were made in the file : app/views/assignments/edit/_review_strategy.html.erb
<pre>
<%= label_tag('assignment_form[assignment][num_reviews_required]', 'Set required number of reviews per reviewer') %>
<%= text_field_tag('assignment_form[assignment][num_reviews_required]', @assignment_form.assignment.num_reviews_required ||= @assignment_form.assignment.num_reviews, size: 1) %>


<img src="/assets/info.png" title='This is the minimum number of reviews a reviewer has to perform'>
<br/>
<br/>
<%= check_box_tag('review', 'true', true, {:onChange => 'hasReviewChanged()'}) %>
<%= label_tag('review', 'Has max review limit?') %>&emsp;&emsp;
<span id="reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_reviews_allowed]', ' Set allowed number of reviews per reviewer') %>
<%if @assignment_form.assignment.num_reviews_allowed && @assignment_form.assignment.num_reviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', @assignment_form.assignment.num_reviews_allowed, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', 3, size: 1) %>
<%end%>

<img src="/assets/info.png" title='This is the maximum number of reviews a reviewer can perform'<%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
</span>

</td>
<td width="50%">

<%= label_tag('assignment_form[assignment][num_metareviews_required]', 'Set required number of meta-reviews per reviewer') %>
<%= text_field_tag('assignment_form[assignment][num_metareviews_required]', @assignment_form.assignment.num_metareviews_required ||= @assignment_form.assignment.num_metareviews_allowed, size: 1) %>
<br/>
<br/>

<%= check_box_tag('meta_review', 'true', true, {:onChange => 'hasMetaReviewChanged()'}) %>
<%= label_tag('meta_review', 'Has meta-review limit?') %>&emsp;&emsp;
<span id="meta_reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_metareviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_metareviews_allowed]', ' Set allowed number of meta-reviews per reviewer') %>
<%if @assignment_form.assignment.num_metareviews_allowed && @assignment_form.assignment.num_metareviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_metareviews_allowed]', @assignment_form.assignment.num_metareviews_allowed ||= 3, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_metareviews_allowed]', 3, size: 1) %>
<%end%>
</span>

</td>
</tr>

</pre>
*'''Test Plan''': Set allowed number and required number of reviews per submission as 3 and 3 respectively.
Now it will display "you should perform exactly 3 reviews."

Before:
[[File:issue2.PNG]]

After:
[[File:issue2solved.PNG]]

[[File:INFOBUT3.PNG]]

* '''Issue #965''': The assignment of work to review to a reviewer depends on the number of reviews it has previously received, if that work has been assigned to a reviewer previously and its review has
not been submitted then the system will still consider it such that it would not assign the same work to another reviewer if it has enough number of reviewers previously assigned to review it. This might
lead to a scenario where that work was assigned to enough number of reviewers, but it does not receive enough reviews and cannot be assigned to any new reviewers either.
*'''Solution''': This implementation has been fixed. The system now considers only the reviews that were submitted by the reviewers for that work so that every submission gets assigned to reviewers until
it receives its allowed number of reviews.

The modified file is app/models/review_assignment.rb and the modification is done in the method reject_by_max_reviews_per_submission

<pre>
def reject_by_max_reviews_per_submission(contributor_set)
contributor_set.reject! {|contributor| contributor.responses.reject {|response| !response.is_submitted }.count >= max_reviews_per_submission }
contributor_set
end
</pre>

*'''Test Plan''': We created a dummy assignment with a team size of 2, 3 such teams are created. Maximum allowed reviews is 2 now.
Previously for a submission after 2 people begin review its does not give any further submission to review to that user because it already reached the limit.

[[File: issue3.PNG]]

On changing the method, when we try to get a new submission we receive one. This is because the previous reviewers have only begun their reviews but have not submitted them yet.

[[File:crappy.PNG]]

Now when the previous two reviewers submit the reviews they began. Now the same new reviewer will not be able to submit his review

[[File:issue3solved.PNG]]

==Modified Files==
* app/views/assignments/edit/_review_strategy.html.erb
* app/models/review_assignment.rb
* app/views/student_review/list.html.erb

===References===

#[https://github.com/expertiza/expertiza Expertiza on GitHub]
#[https://github.com/WintersLt/expertiza GitHub Project Repository Fork]
#[http://expertiza.ncsu.edu/ The live Expertiza website]
#[http://wikis.lib.ncsu.edu/index.php/Expertiza Expertiza project documentation wiki]
#Clean Code: A handbook of agile software craftsmanship. Author: Robert C Martin

File:Crappy.PNG

2017-11-01T18:44:10Z

Ssuresh6:

CSC/ECE 517 Fall 2017/E1777 Coherent specification of review requirements.rb

2017-11-01T18:40:43Z

Ssuresh6: /* Problem Statement */

==E1777. OSS Project Green: Coherent specification of review requirements==

This page provides a description of the Expertiza based OSS project.

__TOC__

===About Expertiza===

[http://expertiza.ncsu.edu/ Expertiza] is an open source project based on [http://rubyonrails.org/ Ruby on Rails] framework. Expertiza allows the instructor to create new assignments and customize new or existing assignments. It also allows the instructor to create a list of topics the students can sign up for. Students can form teams in Expertiza to work on various projects and assignments. Students can also peer review other students' submissions. Expertiza supports submission across various document types, including the URLs and wiki pages.

===Problem Statement===
The following tasks were accomplished in this project:

* Information buttons to elaborate difference between the synonymous statements present in review strategy tab need to be included.
* The punctuation, syntax and capitalization of statements in the review strategy tab requires improvement.
* Message to a reviewer regarding how many reviews they are required to and allowed to perform is not clear enough.
* Reviewers are assigned new reviews based on number of previous reviews ''assigned'' for that submission, which is incorrect.

===Implementation===

=====Drawbacks and Solutions=====
'''Issue #402 ''': To allow reviewers a larger set of topics to choose from, the instructor can set the threshold (on the Review Strategy tab of assignment creation/editing) to some integer k > 0. Then any submission that has within k reviews of the fewest number can be chosen by a new reviewer. Let’s say that all submissions have at least 1 review. If k = 3, then the reviewer can choose any topic where there is a submission that has 4 or fewer reviews so far. Suppose that the minimum number of reviews for any submission is 2, but that I have reviewed all the submissions that have only 2 reviews. Then I’m not allowed to review at all (unless k > 0).
* '''Solution''': This issue has been fixed previously. The reviewer will get assigned a submission even if it has fulfilled the required number of reviews, to ensure that the reviewer always receives a new submission to review as long as it is not his own.
*'''Test Plan''':This is shown graphically with dummy topics and assignment. We create a dummy assignment with two different topics oss1 and oss2. oss1 has 2 slots and oss2 has 1 slot.
So when a student who has submitted oss2 tries to review others work he will not be able to select oos2 at all because he cannot review his own submission.
When a person who has oss1 as topic goes to this page and he asks to get a new submission for review it will give another slot’s oss1.
If he does it again it says there are no more submissions available to review for this topic.
So if he has already reviewed all the other submissions of his topic oss1 and he requests for one more submission of oss1 , he will not get any more submission to review because the only one left is
his own submission.

'''Issue #969''':
*a.The two statements "Set Allowed Number of Reviews per reviewer" and "Set Required Number of Reviews per reviewer." are not well differentiated. It can very confusing for an instructor to give values for
them.

* '''Solution''': Information buttons have been provided beside these statements in the review strategy tab to make sure that the instructor knows the difference between the two.
This modification is made in the app/views/assignments/edit/_review_strategy.html.erb
<pre>

<img src="/assets/info.png" title='This is the minimum number of reviews a reviewer has to perform'>
<br/>
<br/>
<%= check_box_tag('review', 'true', true, {:onChange => 'hasReviewChanged()'}) %>
<%= label_tag('review', 'Has max review limit?') %>&emsp;&emsp;
<span id="reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_reviews_allowed]', ' Set allowed number of reviews per reviewer') %>
<%if @assignment_form.assignment.num_reviews_allowed && @assignment_form.assignment.num_reviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', @assignment_form.assignment.num_reviews_allowed, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', 3, size: 1) %>
<%end%>

<img src="/assets/info.png" title='This is the maximum number of reviews a reviewer can perform'<%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
</span>

</td>
</pre>
[[File:INFOBUT.PNG]]

[[File:INFOBUT2.PNG]]

*b) When the number of allowed or required reviews is not set on the Review Strategy tab, the system does not have a message to display to a reviewer about how many submissions of work they are required to and allowed to review depending on the values set by the instructor in the "allowed number of reviewers per reviewer" field and the "Set Required Number of Reviews per reviewer." field.

*'''Solution''': This view has been implemented in the app/views/student_review/list.html.erb
<pre>

<h2>Reviews for "<%= @assignment.name %>"</h2>


<% if @assignment.num_reviews_allowed.nil? || @assignment.num_reviews_allowed == -1%>
<h4>Your instructor expects you to do <%= @assignment.num_reviews_required %> reviews. You are not allowed to do any extra reviews. </h4>

<% elsif @assignment.num_reviews_allowed == @assignment.num_reviews_required %>
<h4>You should perform exactly <%= @assignment.num_reviews_allowed %> reviews </h4>
<% else %>

<h4>You may perform between <%= @assignment.num_reviews_required %> and <%= @assignment.num_reviews_allowed %> reviews</h4>
<% end %>
</pre>

*c)The capitalization and punctuation of statements such as "Set Allowed Number of Reviews per reviewer" and "Set Required Number of Reviews per reviewer." in review strategy tab are incorrect.
*'''Solution''': This view has been fixed by making changes in the respective files.
The changes were made in the file : app/views/assignments/edit/_review_strategy.html.erb
<pre>
<%= label_tag('assignment_form[assignment][num_reviews_required]', 'Set required number of reviews per reviewer') %>
<%= text_field_tag('assignment_form[assignment][num_reviews_required]', @assignment_form.assignment.num_reviews_required ||= @assignment_form.assignment.num_reviews, size: 1) %>


<img src="/assets/info.png" title='This is the minimum number of reviews a reviewer has to perform'>
<br/>
<br/>
<%= check_box_tag('review', 'true', true, {:onChange => 'hasReviewChanged()'}) %>
<%= label_tag('review', 'Has max review limit?') %>&emsp;&emsp;
<span id="reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_reviews_allowed]', ' Set allowed number of reviews per reviewer') %>
<%if @assignment_form.assignment.num_reviews_allowed && @assignment_form.assignment.num_reviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', @assignment_form.assignment.num_reviews_allowed, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', 3, size: 1) %>
<%end%>

<img src="/assets/info.png" title='This is the maximum number of reviews a reviewer can perform'<%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
</span>

</td>
<td width="50%">

<%= label_tag('assignment_form[assignment][num_metareviews_required]', 'Set required number of meta-reviews per reviewer') %>
<%= text_field_tag('assignment_form[assignment][num_metareviews_required]', @assignment_form.assignment.num_metareviews_required ||= @assignment_form.assignment.num_metareviews_allowed, size: 1) %>
<br/>
<br/>

<%= check_box_tag('meta_review', 'true', true, {:onChange => 'hasMetaReviewChanged()'}) %>
<%= label_tag('meta_review', 'Has meta-review limit?') %>&emsp;&emsp;
<span id="meta_reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_metareviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_metareviews_allowed]', ' Set allowed number of meta-reviews per reviewer') %>
<%if @assignment_form.assignment.num_metareviews_allowed && @assignment_form.assignment.num_metareviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_metareviews_allowed]', @assignment_form.assignment.num_metareviews_allowed ||= 3, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_metareviews_allowed]', 3, size: 1) %>
<%end%>
</span>

</td>
</tr>

</pre>
*'''Test Plan''': Set allowed number and required number of reviews per submission as 3 and 3 respectively.
Now it will display "you should perform exactly 3 reviews."

Before:
[[File:issue2.PNG]]

After:
[[File:issue2solved.PNG]]

[[File:INFOBUT3.PNG]]

* '''Issue #965''': The assignment of work to review to a reviewer depends on the number of reviews it has previously received, if that work has been assigned to a reviewer previously and its review has
not been submitted then the system will still consider it such that it would not assign the same work to another reviewer if it has enough number of reviewers previously assigned to review it. This might
lead to a scenario where that work was assigned to enough number of reviewers, but it does not receive enough reviews and cannot be assigned to any new reviewers either.
*'''Solution''': This implementation has been fixed. The system now considers only the reviews that were submitted by the reviewers for that work so that every submission gets assigned to reviewers until
it receives its allowed number of reviews.

The modified file is app/models/review_assignment.rb and the modification is done in the method reject_by_max_reviews_per_submission

<pre>
def reject_by_max_reviews_per_submission(contributor_set)
contributor_set.reject! {|contributor| contributor.responses.reject {|response| !response.is_submitted }.count >= max_reviews_per_submission }
contributor_set
end
</pre>

*'''Test Plan''': We created a dummy assignment with a team size of 2, 3 such teams are created. Maximum allowed reviews is 2 now.
Previously for a submission after 2 people begin review its does not give any further submission to review to that user because it already reached the limit.

[[File: issue3.PNG]]

On changing the method, when we try to get a new submission we receive one. This is because the previous reviewers have only begun their reviews but have not submitted them yet.

[[File:issue3^.PNG]]

Now when the previous two reviewers submit the reviews they began. Now the same new reviewer will not be able to submit his review

[[File:issue3solved.PNG]]

==Modified Files==
* app/views/assignments/edit/_review_strategy.html.erb
* app/models/review_assignment.rb
* app/views/student_review/list.html.erb

===References===

#[https://github.com/expertiza/expertiza Expertiza on GitHub]
#[https://github.com/WintersLt/expertiza GitHub Project Repository Fork]
#[http://expertiza.ncsu.edu/ The live Expertiza website]
#[http://wikis.lib.ncsu.edu/index.php/Expertiza Expertiza project documentation wiki]
#Clean Code: A handbook of agile software craftsmanship. Author: Robert C Martin

CSC/ECE 517 Fall 2017/E1777 Coherent specification of review requirements.rb

2017-11-01T18:20:32Z

Ssuresh6: /* Implementation */

==E1777. OSS Project Green: Coherent specification of review requirements==

This page provides a description of the Expertiza based OSS project.

__TOC__

===About Expertiza===

[http://expertiza.ncsu.edu/ Expertiza] is an open source project based on [http://rubyonrails.org/ Ruby on Rails] framework. Expertiza allows the instructor to create new assignments and customize new or existing assignments. It also allows the instructor to create a list of topics the students can sign up for. Students can form teams in Expertiza to work on various projects and assignments. Students can also peer review other students' submissions. Expertiza supports submission across various document types, including the URLs and wiki pages.

===Problem Statement===
The following tasks were accomplished in this project:

* Information buttons to elaborate difference between the synonymous statements present in review strategy tab need to be included.
* The punctuation, syntax and the capitalization of statements in the review strategy tab requires improvement.
* Message to a reviewer regarding how many reviews they are required to and allowed to perform is not clear enough.
* Assignment of work to review for a new reviewer is based on the number of previous reviews assigned for that submission which is incorrect.

===Implementation===

=====Drawbacks and Solutions=====
'''Issue #402 ''': To allow reviewers a larger set of topics to choose from, the instructor can set the threshold (on the Review Strategy tab of assignment creation/editing) to some integer k > 0. Then any submission that has within k reviews of the fewest number can be chosen by a new reviewer. Let’s say that all submissions have at least 1 review. If k = 3, then the reviewer can choose any topic where there is a submission that has 4 or fewer reviews so far. Suppose that the minimum number of reviews for any submission is 2, but that I have reviewed all the submissions that have only 2 reviews. Then I’m not allowed to review at all (unless k > 0).
* '''Solution''': This issue has been fixed previously. The reviewer will get assigned a submission even if it has fulfilled the required number of reviews, to ensure that the reviewer always receives a new submission to review as long as it is not his own.
*'''Test Plan''':This is shown graphically with dummy topics and assignment. We create a dummy assignment with two different topics oss1 and oss2. oss1 has 2 slots and oss2 has 1 slot.
So when a student who has submitted oss2 tries to review others work he will not be able to select oos2 at all because he cannot review his own submission.
When a person who has oss1 as topic goes to this page and he asks to get a new submission for review it will give another slot’s oss1.
If he does it again it says there are no more submissions available to review for this topic.
So if he has already reviewed all the other submissions of his topic oss1 and he requests for one more submission of oss1 , he will not get any more submission to review because the only one left is
his own submission.

'''Issue #969''':
*a.The two statements "Set Allowed Number of Reviews per reviewer" and "Set Required Number of Reviews per reviewer." are not well differentiated. It can very confusing for an instructor to give values for
them.

* '''Solution''': Information buttons have been provided beside these statements in the review strategy tab to make sure that the instructor knows the difference between the two.
This modification is made in the app/views/assignments/edit/_review_strategy.html.erb
<pre>

<img src="/assets/info.png" title='This is the minimum number of reviews a reviewer has to perform'>
<br/>
<br/>
<%= check_box_tag('review', 'true', true, {:onChange => 'hasReviewChanged()'}) %>
<%= label_tag('review', 'Has max review limit?') %>&emsp;&emsp;
<span id="reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_reviews_allowed]', ' Set allowed number of reviews per reviewer') %>
<%if @assignment_form.assignment.num_reviews_allowed && @assignment_form.assignment.num_reviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', @assignment_form.assignment.num_reviews_allowed, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', 3, size: 1) %>
<%end%>

<img src="/assets/info.png" title='This is the maximum number of reviews a reviewer can perform'<%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
</span>

</td>
</pre>
[[File:INFOBUT.PNG]]

[[File:INFOBUT2.PNG]]

*b) When the number of allowed or required reviews is not set on the Review Strategy tab, the system does not have a message to display to a reviewer about how many submissions of work they are required to and allowed to review depending on the values set by the instructor in the "allowed number of reviewers per reviewer" field and the "Set Required Number of Reviews per reviewer." field.

*'''Solution''': This view has been implemented in the app/views/student_review/list.html.erb
<pre>

<h2>Reviews for "<%= @assignment.name %>"</h2>


<% if @assignment.num_reviews_allowed.nil? || @assignment.num_reviews_allowed == -1%>
<h4>Your instructor expects you to do <%= @assignment.num_reviews_required %> reviews. You are not allowed to do any extra reviews. </h4>

<% elsif @assignment.num_reviews_allowed == @assignment.num_reviews_required %>
<h4>You should perform exactly <%= @assignment.num_reviews_allowed %> reviews </h4>
<% else %>

<h4>You may perform between <%= @assignment.num_reviews_required %> and <%= @assignment.num_reviews_allowed %> reviews</h4>
<% end %>
</pre>

*c)The capitalization and punctuation of statements such as "Set Allowed Number of Reviews per reviewer" and "Set Required Number of Reviews per reviewer." in review strategy tab are incorrect.
*'''Solution''': This view has been fixed by making changes in the respective files.
The changes were made in the file : app/views/assignments/edit/_review_strategy.html.erb
<pre>
<%= label_tag('assignment_form[assignment][num_reviews_required]', 'Set required number of reviews per reviewer') %>
<%= text_field_tag('assignment_form[assignment][num_reviews_required]', @assignment_form.assignment.num_reviews_required ||= @assignment_form.assignment.num_reviews, size: 1) %>


<img src="/assets/info.png" title='This is the minimum number of reviews a reviewer has to perform'>
<br/>
<br/>
<%= check_box_tag('review', 'true', true, {:onChange => 'hasReviewChanged()'}) %>
<%= label_tag('review', 'Has max review limit?') %>&emsp;&emsp;
<span id="reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_reviews_allowed]', ' Set allowed number of reviews per reviewer') %>
<%if @assignment_form.assignment.num_reviews_allowed && @assignment_form.assignment.num_reviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', @assignment_form.assignment.num_reviews_allowed, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', 3, size: 1) %>
<%end%>

<img src="/assets/info.png" title='This is the maximum number of reviews a reviewer can perform'<%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
</span>

</td>
<td width="50%">

<%= label_tag('assignment_form[assignment][num_metareviews_required]', 'Set required number of meta-reviews per reviewer') %>
<%= text_field_tag('assignment_form[assignment][num_metareviews_required]', @assignment_form.assignment.num_metareviews_required ||= @assignment_form.assignment.num_metareviews_allowed, size: 1) %>
<br/>
<br/>

<%= check_box_tag('meta_review', 'true', true, {:onChange => 'hasMetaReviewChanged()'}) %>
<%= label_tag('meta_review', 'Has meta-review limit?') %>&emsp;&emsp;
<span id="meta_reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_metareviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_metareviews_allowed]', ' Set allowed number of meta-reviews per reviewer') %>
<%if @assignment_form.assignment.num_metareviews_allowed && @assignment_form.assignment.num_metareviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_metareviews_allowed]', @assignment_form.assignment.num_metareviews_allowed ||= 3, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_metareviews_allowed]', 3, size: 1) %>
<%end%>
</span>

</td>
</tr>

</pre>
*'''Test Plan''': Set allowed number and required number of reviews per submission as 3 and 3 respectively.
Now it will display "you should perform exactly 3 reviews."

Before:
[[File:issue2.PNG]]

After:
[[File:issue2solved.PNG]]

[[File:INFOBUT3.PNG]]

* '''Issue #965''': The assignment of work to review to a reviewer depends on the number of reviews it has previously received, if that work has been assigned to a reviewer previously and its review has
not been submitted then the system will still consider it such that it would not assign the same work to another reviewer if it has enough number of reviewers previously assigned to review it. This might
lead to a scenario where that work was assigned to enough number of reviewers, but it does not receive enough reviews and cannot be assigned to any new reviewers either.
*'''Solution''': This implementation has been fixed. The system now considers only the reviews that were submitted by the reviewers for that work so that every submission gets assigned to reviewers until
it receives its allowed number of reviews.

The modified file is app/models/review_assignment.rb and the modification is done in the method reject_by_max_reviews_per_submission

<pre>
def reject_by_max_reviews_per_submission(contributor_set)
contributor_set.reject! {|contributor| contributor.responses.reject {|response| !response.is_submitted }.count >= max_reviews_per_submission }
contributor_set
end
</pre>

*'''Test Plan''': We created a dummy assignment with a team size of 2, 3 such teams are created. Maximum allowed reviews is 2 now.
Previously for a submission after 2 people begin review its does not give any further submission to review to that user because it already reached the limit.

[[File: issue3.PNG]]

On changing the method, when we try to get a new submission we receive one. This is because the previous reviewers have only begun their reviews but have not submitted them yet.

[[File:issue3^.PNG]]

Now when the previous two reviewers submit the reviews they began. Now the same new reviewer will not be able to submit his review

[[File:issue3solved.PNG]]

==Modified Files==
* app/views/assignments/edit/_review_strategy.html.erb
* app/models/review_assignment.rb
* app/views/student_review/list.html.erb

===References===

#[https://github.com/expertiza/expertiza Expertiza on GitHub]
#[https://github.com/WintersLt/expertiza GitHub Project Repository Fork]
#[http://expertiza.ncsu.edu/ The live Expertiza website]
#[http://wikis.lib.ncsu.edu/index.php/Expertiza Expertiza project documentation wiki]
#Clean Code: A handbook of agile software craftsmanship. Author: Robert C Martin

File:INFOBUT3.PNG

2017-11-01T18:14:10Z

Ssuresh6:

File:INFOBUT2.PNG

2017-11-01T18:14:00Z

Ssuresh6:

File:INFOBUT.PNG

2017-11-01T18:13:49Z

Ssuresh6:

CSC/ECE 517 Fall 2017/E1777 Coherent specification of review requirements.rb

2017-11-01T18:10:12Z

Ssuresh6: /* Implementation */

==E1777. OSS Project Green: Coherent specification of review requirements==

This page provides a description of the Expertiza based OSS project.

__TOC__

===About Expertiza===

[http://expertiza.ncsu.edu/ Expertiza] is an open source project based on [http://rubyonrails.org/ Ruby on Rails] framework. Expertiza allows the instructor to create new assignments and customize new or existing assignments. It also allows the instructor to create a list of topics the students can sign up for. Students can form teams in Expertiza to work on various projects and assignments. Students can also peer review other students' submissions. Expertiza supports submission across various document types, including the URLs and wiki pages.

===Problem Statement===
The following tasks were accomplished in this project:

* Information buttons to elaborate difference between the synonymous statements present in review strategy tab need to be included.
* The punctuation, syntax and the capitalization of statements in the review strategy tab requires improvement.
* Message to a reviewer regarding how many reviews they are required to and allowed to perform is not clear enough.
* Assignment of work to review for a new reviewer is based on the number of previous reviews assigned for that submission which is incorrect.

===Implementation===

=====Drawbacks and Solutions=====
'''Issue #402 ''': To allow reviewers a larger set of topics to choose from, the instructor can set the threshold (on the Review Strategy tab of assignment creation/editing) to some integer k > 0. Then any submission that has within k reviews of the fewest number can be chosen by a new reviewer. Let’s say that all submissions have at least 1 review. If k = 3, then the reviewer can choose any topic where there is a submission that has 4 or fewer reviews so far. Suppose that the minimum number of reviews for any submission is 2, but that I have reviewed all the submissions that have only 2 reviews. Then I’m not allowed to review at all (unless k > 0).
* '''Solution''': This issue has been fixed previously. The reviewer will get assigned a submission even if it has fulfilled the required number of reviews, to ensure that the reviewer always receives a new submission to review as long as it is not his own.
*'''Test Plan''':This is shown graphically with dummy topics and assignment. We create a dummy assignment with two different topics oss1 and oss2. oss1 has 2 slots and oss2 has 1 slot.
So when a student who has submitted oss2 tries to review others work he will not be able to select oos2 at all because he cannot review his own submission.
When a person who has oss1 as topic goes to this page and he asks to get a new submission for review it will give another slot’s oss1.
If he does it again it says there are no more submissions available to review for this topic.
So if he has already reviewed all the other submissions of his topic oss1 and he requests for one more submission of oss1 , he will not get any more submission to review because the only one left is
his own submission.

'''Issue #969''':
*a.The two statements "Set Allowed Number of Reviews per reviewer" and "Set Required Number of Reviews per reviewer." are not well differentiated. It can very confusing for an instructor to give values for
them.

* '''Solution''': Information buttons have been provided beside these statements in the review strategy tab to make sure that the instructor knows the difference between the two.
This modification is made in the app/views/assignments/edit/_review_strategy.html.erb
<pre>

<img src="/assets/info.png" title='This is the minimum number of reviews a reviewer has to perform'>
<br/>
<br/>
<%= check_box_tag('review', 'true', true, {:onChange => 'hasReviewChanged()'}) %>
<%= label_tag('review', 'Has max review limit?') %>&emsp;&emsp;
<span id="reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_reviews_allowed]', ' Set allowed number of reviews per reviewer') %>
<%if @assignment_form.assignment.num_reviews_allowed && @assignment_form.assignment.num_reviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', @assignment_form.assignment.num_reviews_allowed, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', 3, size: 1) %>
<%end%>

<img src="/assets/info.png" title='This is the maximum number of reviews a reviewer can perform'<%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
</span>

</td>
</pre>
[[File:morecrap.png]]

[[File:lesscrap.png]]

*b) When the number of allowed or required reviews is not set on the Review Strategy tab, the system does not have a message to display to a reviewer about how many submissions of work they are required to and allowed to review depending on the values set by the instructor in the "allowed number of reviewers per reviewer" field and the "Set Required Number of Reviews per reviewer." field.

*'''Solution''': This view has been implemented in the app/views/student_review/list.html.erb
<pre>

<h2>Reviews for "<%= @assignment.name %>"</h2>


<% if @assignment.num_reviews_allowed.nil? || @assignment.num_reviews_allowed == -1%>
<h4>Your instructor expects you to do <%= @assignment.num_reviews_required %> reviews. You are not allowed to do any extra reviews. </h4>

<% elsif @assignment.num_reviews_allowed == @assignment.num_reviews_required %>
<h4>You should perform exactly <%= @assignment.num_reviews_allowed %> reviews </h4>
<% else %>

<h4>You may perform between <%= @assignment.num_reviews_required %> and <%= @assignment.num_reviews_allowed %> reviews</h4>
<% end %>
</pre>

*c)The capitalization and punctuation of statements such as "Set Allowed Number of Reviews per reviewer" and "Set Required Number of Reviews per reviewer." in review strategy tab are incorrect.
*'''Solution''': This view has been fixed by making changes in the respective files.
The changes were made in the file : app/views/assignments/edit/_review_strategy.html.erb
<pre>
<%= label_tag('assignment_form[assignment][num_reviews_required]', 'Set required number of reviews per reviewer') %>
<%= text_field_tag('assignment_form[assignment][num_reviews_required]', @assignment_form.assignment.num_reviews_required ||= @assignment_form.assignment.num_reviews, size: 1) %>


<img src="/assets/info.png" title='This is the minimum number of reviews a reviewer has to perform'>
<br/>
<br/>
<%= check_box_tag('review', 'true', true, {:onChange => 'hasReviewChanged()'}) %>
<%= label_tag('review', 'Has max review limit?') %>&emsp;&emsp;
<span id="reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_reviews_allowed]', ' Set allowed number of reviews per reviewer') %>
<%if @assignment_form.assignment.num_reviews_allowed && @assignment_form.assignment.num_reviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', @assignment_form.assignment.num_reviews_allowed, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', 3, size: 1) %>
<%end%>

<img src="/assets/info.png" title='This is the maximum number of reviews a reviewer can perform'<%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
</span>

</td>
<td width="50%">

<%= label_tag('assignment_form[assignment][num_metareviews_required]', 'Set required number of meta-reviews per reviewer') %>
<%= text_field_tag('assignment_form[assignment][num_metareviews_required]', @assignment_form.assignment.num_metareviews_required ||= @assignment_form.assignment.num_metareviews_allowed, size: 1) %>
<br/>
<br/>

<%= check_box_tag('meta_review', 'true', true, {:onChange => 'hasMetaReviewChanged()'}) %>
<%= label_tag('meta_review', 'Has meta-review limit?') %>&emsp;&emsp;
<span id="meta_reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_metareviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_metareviews_allowed]', ' Set allowed number of meta-reviews per reviewer') %>
<%if @assignment_form.assignment.num_metareviews_allowed && @assignment_form.assignment.num_metareviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_metareviews_allowed]', @assignment_form.assignment.num_metareviews_allowed ||= 3, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_metareviews_allowed]', 3, size: 1) %>
<%end%>
</span>

</td>
</tr>

</pre>
*'''Test Plan''': Set allowed number and required number of reviews per submission as 3 and 3 respectively.
Now it will display "you should perform exactly 3 reviews."

Before:
[[File:issue2.PNG]]

After:
[[File:issue2solved.PNG]]
[[File:uifixmodi.PNG]]

* '''Issue #965''': The assignment of work to review to a reviewer depends on the number of reviews it has previously received, if that work has been assigned to a reviewer previously and its review has
not been submitted then the system will still consider it such that it would not assign the same work to another reviewer if it has enough number of reviewers previously assigned to review it. This might
lead to a scenario where that work was assigned to enough number of reviewers, but it does not receive enough reviews and cannot be assigned to any new reviewers either.
*'''Solution''': This implementation has been fixed. The system now considers only the reviews that were submitted by the reviewers for that work so that every submission gets assigned to reviewers until
it receives its allowed number of reviews.

The modified file is app/models/review_assignment.rb and the modification is done in the method reject_by_max_reviews_per_submission

<pre>
def reject_by_max_reviews_per_submission(contributor_set)
contributor_set.reject! {|contributor| contributor.responses.reject {|response| !response.is_submitted }.count >= max_reviews_per_submission }
contributor_set
end
</pre>

*'''Test Plan''': We created a dummy assignment with a team size of 2, 3 such teams are created. Maximum allowed reviews is 2 now.
Previously for a submission after 2 people begin review its does not give any further submission to review to that user because it already reached the limit.

[[File: issue3.PNG]]

On changing the method, when we try to get a new submission we receive one. This is because the previous reviewers have only begun their reviews but have not submitted them yet.

[[File:issue3^.PNG]]

Now when the previous two reviewers submit the reviews they began. Now the same new reviewer will not be able to submit his review

[[File:issue3solved.PNG]]

==Modified Files==
* app/views/assignments/edit/_review_strategy.html.erb
* app/models/review_assignment.rb
* app/views/student_review/list.html.erb

===References===

#[https://github.com/expertiza/expertiza Expertiza on GitHub]
#[https://github.com/WintersLt/expertiza GitHub Project Repository Fork]
#[http://expertiza.ncsu.edu/ The live Expertiza website]
#[http://wikis.lib.ncsu.edu/index.php/Expertiza Expertiza project documentation wiki]
#Clean Code: A handbook of agile software craftsmanship. Author: Robert C Martin

CSC/ECE 517 Fall 2017/E1777 Coherent specification of review requirements.rb

2017-11-01T18:09:41Z

Ssuresh6: /* Implementation */

==E1777. OSS Project Green: Coherent specification of review requirements==

This page provides a description of the Expertiza based OSS project.

__TOC__

===About Expertiza===

[http://expertiza.ncsu.edu/ Expertiza] is an open source project based on [http://rubyonrails.org/ Ruby on Rails] framework. Expertiza allows the instructor to create new assignments and customize new or existing assignments. It also allows the instructor to create a list of topics the students can sign up for. Students can form teams in Expertiza to work on various projects and assignments. Students can also peer review other students' submissions. Expertiza supports submission across various document types, including the URLs and wiki pages.

===Problem Statement===
The following tasks were accomplished in this project:

* Information buttons to elaborate difference between the synonymous statements present in review strategy tab need to be included.
* The punctuation, syntax and the capitalization of statements in the review strategy tab requires improvement.
* Message to a reviewer regarding how many reviews they are required to and allowed to perform is not clear enough.
* Assignment of work to review for a new reviewer is based on the number of previous reviews assigned for that submission which is incorrect.

===Implementation===

=====Drawbacks and Solutions=====
'''Issue #402 ''': To allow reviewers a larger set of topics to choose from, the instructor can set the threshold (on the Review Strategy tab of assignment creation/editing) to some integer k > 0. Then any submission that has within k reviews of the fewest number can be chosen by a new reviewer. Let’s say that all submissions have at least 1 review. If k = 3, then the reviewer can choose any topic where there is a submission that has 4 or fewer reviews so far. Suppose that the minimum number of reviews for any submission is 2, but that I have reviewed all the submissions that have only 2 reviews. Then I’m not allowed to review at all (unless k > 0).
* '''Solution''': This issue has been fixed previously. The reviewer will get assigned a submission even if it has fulfilled the required number of reviews, to ensure that the reviewer always receives a new submission to review as long as it is not his own.
*'''Test Plan''':This is shown graphically with dummy topics and assignment. We create a dummy assignment with two different topics oss1 and oss2. oss1 has 2 slots and oss2 has 1 slot.
So when a student who has submitted oss2 tries to review others work he will not be able to select oos2 at all because he cannot review his own submission.
When a person who has oss1 as topic goes to this page and he asks to get a new submission for review it will give another slot’s oss1.
If he does it again it says there are no more submissions available to review for this topic.
So if he has already reviewed all the other submissions of his topic oss1 and he requests for one more submission of oss1 , he will not get any more submission to review because the only one left is
his own submission.

'''Issue #969''':
*a.The two statements "Set Allowed Number of Reviews per reviewer" and "Set Required Number of Reviews per reviewer." are not well differentiated. It can very confusing for an instructor to give values for
them.

* '''Solution''': Information buttons have been provided beside these statements in the review strategy tab to make sure that the instructor knows the difference between the two.
This modification is made in the app/views/assignments/edit/_review_strategy.html.erb
<pre>

<img src="/assets/info.png" title='This is the minimum number of reviews a reviewer has to perform'>
<br/>
<br/>
<%= check_box_tag('review', 'true', true, {:onChange => 'hasReviewChanged()'}) %>
<%= label_tag('review', 'Has max review limit?') %>&emsp;&emsp;
<span id="reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_reviews_allowed]', ' Set allowed number of reviews per reviewer') %>
<%if @assignment_form.assignment.num_reviews_allowed && @assignment_form.assignment.num_reviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', @assignment_form.assignment.num_reviews_allowed, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', 3, size: 1) %>
<%end%>

<img src="/assets/info.png" title='This is the maximum number of reviews a reviewer can perform'<%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
</span>

</td>
</pre>
[[File:morecrap.png]]

[[File:lesscrap.png]]

*b) When the number of allowed or required reviews is not set on the Review Strategy tab, the system does not have a message to display to a reviewer about how many submissions of work they are required to and allowed to review depending on the values set by the instructor in the "allowed number of reviewers per reviewer" field and the "Set Required Number of Reviews per reviewer." field.

*'''Solution''': This view has been implemented in the app/views/student_review/list.html.erb
<pre>

<h2>Reviews for "<%= @assignment.name %>"</h2>


<% if @assignment.num_reviews_allowed.nil? || @assignment.num_reviews_allowed == -1%>
<h4>Your instructor expects you to do <%= @assignment.num_reviews_required %> reviews. You are not allowed to do any extra reviews. </h4>

<% elsif @assignment.num_reviews_allowed == @assignment.num_reviews_required %>
<h4>You should perform exactly <%= @assignment.num_reviews_allowed %> reviews </h4>
<% else %>

<h4>You may perform between <%= @assignment.num_reviews_required %> and <%= @assignment.num_reviews_allowed %> reviews</h4>
<% end %>
</pre>

*c)The capitalization and punctuation of statements such as "Set Allowed Number of Reviews per reviewer" and "Set Required Number of Reviews per reviewer." in review strategy tab are incorrect.
*'''Solution''': This view has been fixed by making changes in the respective files.
The changes were made in the file : app/views/assignments/edit/_review_strategy.html.erb
<pre>
<%= label_tag('assignment_form[assignment][num_reviews_required]', 'Set required number of reviews per reviewer') %>
<%= text_field_tag('assignment_form[assignment][num_reviews_required]', @assignment_form.assignment.num_reviews_required ||= @assignment_form.assignment.num_reviews, size: 1) %>


<img src="/assets/info.png" title='This is the minimum number of reviews a reviewer has to perform'>
<br/>
<br/>
<%= check_box_tag('review', 'true', true, {:onChange => 'hasReviewChanged()'}) %>
<%= label_tag('review', 'Has max review limit?') %>&emsp;&emsp;
<span id="reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_reviews_allowed]', ' Set allowed number of reviews per reviewer') %>
<%if @assignment_form.assignment.num_reviews_allowed && @assignment_form.assignment.num_reviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', @assignment_form.assignment.num_reviews_allowed, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', 3, size: 1) %>
<%end%>

<img src="/assets/info.png" title='This is the maximum number of reviews a reviewer can perform'<%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
</span>

</td>
<td width="50%">

<%= label_tag('assignment_form[assignment][num_metareviews_required]', 'Set required number of meta-reviews per reviewer') %>
<%= text_field_tag('assignment_form[assignment][num_metareviews_required]', @assignment_form.assignment.num_metareviews_required ||= @assignment_form.assignment.num_metareviews_allowed, size: 1) %>
<br/>
<br/>

<%= check_box_tag('meta_review', 'true', true, {:onChange => 'hasMetaReviewChanged()'}) %>
<%= label_tag('meta_review', 'Has meta-review limit?') %>&emsp;&emsp;
<span id="meta_reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_metareviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_metareviews_allowed]', ' Set allowed number of meta-reviews per reviewer') %>
<%if @assignment_form.assignment.num_metareviews_allowed && @assignment_form.assignment.num_metareviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_metareviews_allowed]', @assignment_form.assignment.num_metareviews_allowed ||= 3, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_metareviews_allowed]', 3, size: 1) %>
<%end%>
</span>

</td>
</tr>

</pre>
*'''Test Plan''': Set allowed number and required number of reviews per submission as 3 and 3 respectively.
Now it will display "you should perform exactly 3 reviews."

Before:
[[File:issue2.PNG]]

After:
[[File:issue2solved.PNG]]
[[File:uifixmodi.PNG]]

* '''Issue #965''': The assignment of work to review to a reviewer depends on the number of reviews it has previously received, if that work has been assigned to a reviewer previously and its review has
not been submitted then the system will still consider it such that it would not assign the same work to another reviewer if it has enough number of reviewers previously assigned to review it. This might
lead to a scenario where that work was assigned to enough number of reviewers, but it does not receive enough reviews and cannot be assigned to any new reviewers either.
*'''Solution''': This implementation has been fixed. The system now considers only the reviews that were submitted by the reviewers for that work so that every submission gets assigned to reviewers until
it receives its allowed number of reviews.

The modified file is app/models/review_assignment.rb and the modification is done in the method reject_by_max_reviews_per_submission

<pre>
def reject_by_max_reviews_per_submission(contributor_set)
contributor_set.reject! {|contributor| contributor.responses.reject {|response| !response.is_submitted }.count >= max_reviews_per_submission }
contributor_set
end
</pre>

*'''Test Plan''': We created a dummy assignment with a team size of 2, 3 such teams are created. Maximum allowed reviews is 2 now.
Previously for a submission after 2 people begin review its does not give any further submission to review to that user because it already reached the limit.

[[File: issue3.PNG]]
On changing the method, when we try to get a new submission we receive one. This is because the previous reviewers have only begun their reviews but have not submitted them yet.

[[File:issue3^.PNG]]
Now when the previous two reviewers submit the reviews they began. Now the same new reviewer will not be able to submit his review

[[File:issue3solved.PNG]]

==Modified Files==
* app/views/assignments/edit/_review_strategy.html.erb
* app/models/review_assignment.rb
* app/views/student_review/list.html.erb

===References===

#[https://github.com/expertiza/expertiza Expertiza on GitHub]
#[https://github.com/WintersLt/expertiza GitHub Project Repository Fork]
#[http://expertiza.ncsu.edu/ The live Expertiza website]
#[http://wikis.lib.ncsu.edu/index.php/Expertiza Expertiza project documentation wiki]
#Clean Code: A handbook of agile software craftsmanship. Author: Robert C Martin

File:Issue3solved.PNG

2017-11-01T17:44:25Z

Ssuresh6:

File:Issue3^.PNG

2017-11-01T17:44:11Z

Ssuresh6:

File:Issue3.PNG

2017-11-01T17:43:53Z

Ssuresh6:

File:Issue2solved.PNG

2017-11-01T17:43:31Z

Ssuresh6:

File:Issue2.PNG

2017-11-01T17:43:14Z

Ssuresh6:

CSC/ECE 517 Fall 2017/E1777 Coherent specification of review requirements.rb

2017-11-01T17:34:57Z

Ssuresh6: /* Drawbacks and Solutions */

==E1777. OSS Project Green: Coherent specification of review requirements==

This page provides a description of the Expertiza based OSS project.

__TOC__

===About Expertiza===

[http://expertiza.ncsu.edu/ Expertiza] is an open source project based on [http://rubyonrails.org/ Ruby on Rails] framework. Expertiza allows the instructor to create new assignments and customize new or existing assignments. It also allows the instructor to create a list of topics the students can sign up for. Students can form teams in Expertiza to work on various projects and assignments. Students can also peer review other students' submissions. Expertiza supports submission across various document types, including the URLs and wiki pages.

===Problem Statement===
The following tasks were accomplished in this project:

* Information buttons to elaborate difference between the synonymous statements present in review strategy tab need to be included.
* The punctuation, syntax and the capitalization of statements in the review strategy tab requires improvement.
* Message to a reviewer regarding how many reviews they are required to and allowed to perform is not clear enough.
* Assignment of work to review for a new reviewer is based on the number of previous reviews assigned for that submission which is incorrect.

===Implementation===

=====Drawbacks and Solutions=====
'''Issue #402 ''': To allow reviewers a larger set of topics to choose from, the instructor can set the threshold (on the Review Strategy tab of assignment creation/editing) to some integer k > 0. Then any submission that has within k reviews of the fewest number can be chosen by a new reviewer. Let’s say that all submissions have at least 1 review. If k = 3, then the reviewer can choose any topic where there is a submission that has 4 or fewer reviews so far. Suppose that the minimum number of reviews for any submission is 2, but that I have reviewed all the submissions that have only 2 reviews. Then I’m not allowed to review at all (unless k > 0).
* '''Solution''': This issue has been fixed previously. The reviewer will get assigned a submission even if it has fulfilled the required number of reviews, to ensure that the reviewer always receives a new submission to review as long as it is not his own.
*'''Test Plan''':This is shown graphically with dummy topics and assignment. We create a dummy assignment with two different topics oss1 and oss2. oss1 has 2 slots and oss2 has 1 slot.
So when a student who has submitted oss2 tries to review others work he will not be able to select oos2 at all because he cannot review his own submission.
When a person who has oss1 as topic goes to this page and he asks to get a new submission for review it will give another slot’s oss1.
If he does it again it says there are no more submissions available to review for this topic.
So if he has already reviewed all the other submissions of his topic oss1 and he requests for one more submission of oss1 , he will not get any more submission to review because the only one left is
his own submission.

'''Issue #969''':
*a.The two statements "Set Allowed Number of Reviews per reviewer" and "Set Required Number of Reviews per reviewer." are not well differentiated. It can very confusing for an instructor to give values for
them.

* '''Solution''': Information buttons have been provided beside these statements in the review strategy tab to make sure that the instructor knows the difference between the two.
This modification is made in the app/views/assignments/edit/_review_strategy.html.erb
<pre>

<img src="/assets/info.png" title='This is the minimum number of reviews a reviewer has to perform'>
<br/>
<br/>
<%= check_box_tag('review', 'true', true, {:onChange => 'hasReviewChanged()'}) %>
<%= label_tag('review', 'Has max review limit?') %>&emsp;&emsp;
<span id="reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_reviews_allowed]', ' Set allowed number of reviews per reviewer') %>
<%if @assignment_form.assignment.num_reviews_allowed && @assignment_form.assignment.num_reviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', @assignment_form.assignment.num_reviews_allowed, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', 3, size: 1) %>
<%end%>

<img src="/assets/info.png" title='This is the maximum number of reviews a reviewer can perform'<%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
</span>

</td>
</pre>
[[File:morecrap.png]]

[[File:lesscrap.png]]

*b) When the number of allowed or required reviews is not set on the Review Strategy tab, the system does not have a message to display to a reviewer about how many submissions of work they are required to and allowed to review depending on the values set by the instructor in the "allowed number of reviewers per reviewer" field and the "Set Required Number of Reviews per reviewer." field.

*'''Solution''': This view has been implemented in the app/views/student_review/list.html.erb
<pre>

<h2>Reviews for "<%= @assignment.name %>"</h2>


<% if @assignment.num_reviews_allowed.nil? || @assignment.num_reviews_allowed == -1%>
<h4>Your instructor expects you to do <%= @assignment.num_reviews_required %> reviews. You are not allowed to do any extra reviews. </h4>

<% elsif @assignment.num_reviews_allowed == @assignment.num_reviews_required %>
<h4>You should perform exactly <%= @assignment.num_reviews_allowed %> reviews </h4>
<% else %>

<h4>You may perform between <%= @assignment.num_reviews_required %> and <%= @assignment.num_reviews_allowed %> reviews</h4>
<% end %>
</pre>

*c)The capitalization and punctuation of statements such as "Set Allowed Number of Reviews per reviewer" and "Set Required Number of Reviews per reviewer." in review strategy tab are incorrect.
*'''Solution''': This view has been fixed by making changes in the respective files.
The changes were made in the file : app/views/assignments/edit/_review_strategy.html.erb
<pre>
<%= label_tag('assignment_form[assignment][num_reviews_required]', 'Set required number of reviews per reviewer') %>
<%= text_field_tag('assignment_form[assignment][num_reviews_required]', @assignment_form.assignment.num_reviews_required ||= @assignment_form.assignment.num_reviews, size: 1) %>


<img src="/assets/info.png" title='This is the minimum number of reviews a reviewer has to perform'>
<br/>
<br/>
<%= check_box_tag('review', 'true', true, {:onChange => 'hasReviewChanged()'}) %>
<%= label_tag('review', 'Has max review limit?') %>&emsp;&emsp;
<span id="reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_reviews_allowed]', ' Set allowed number of reviews per reviewer') %>
<%if @assignment_form.assignment.num_reviews_allowed && @assignment_form.assignment.num_reviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', @assignment_form.assignment.num_reviews_allowed, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', 3, size: 1) %>
<%end%>

<img src="/assets/info.png" title='This is the maximum number of reviews a reviewer can perform'<%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
</span>

</td>
<td width="50%">

<%= label_tag('assignment_form[assignment][num_metareviews_required]', 'Set required number of meta-reviews per reviewer') %>
<%= text_field_tag('assignment_form[assignment][num_metareviews_required]', @assignment_form.assignment.num_metareviews_required ||= @assignment_form.assignment.num_metareviews_allowed, size: 1) %>
<br/>
<br/>

<%= check_box_tag('meta_review', 'true', true, {:onChange => 'hasMetaReviewChanged()'}) %>
<%= label_tag('meta_review', 'Has meta-review limit?') %>&emsp;&emsp;
<span id="meta_reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_metareviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_metareviews_allowed]', ' Set allowed number of meta-reviews per reviewer') %>
<%if @assignment_form.assignment.num_metareviews_allowed && @assignment_form.assignment.num_metareviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_metareviews_allowed]', @assignment_form.assignment.num_metareviews_allowed ||= 3, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_metareviews_allowed]', 3, size: 1) %>
<%end%>
</span>

</td>
</tr>

</pre>
*'''Test Plan''': Set allowed number and required number of reviews per submission as 3 and 3 respectively.
Now it will display "you should perform exactly 3 reviews."
[[File:uifixmodi.PNG]]

* '''Issue #965''': The assignment of work to review to a reviewer depends on the number of reviews it has previously received, if that work has been assigned to a reviewer previously and its review has
not been submitted then the system will still consider it such that it would not assign the same work to another reviewer if it has enough number of reviewers previously assigned to review it. This might
lead to a scenario where that work was assigned to enough number of reviewers, but it does not receive enough reviews and cannot be assigned to any new reviewers either.
*'''Solution''': This implementation has been fixed. The system now considers only the reviews that were submitted by the reviewers for that work so that every submission gets assigned to reviewers until
it receives its allowed number of reviews.

The modified file is app/models/review_assignment.rb and the modification is done in the method reject_by_max_reviews_per_submission

<pre>
def reject_by_max_reviews_per_submission(contributor_set)
contributor_set.reject! {|contributor| contributor.responses.reject {|response| !response.is_submitted }.count >= max_reviews_per_submission }
contributor_set
end
</pre>

==Modified Files==
* app/views/assignments/edit/_review_strategy.html.erb
* app/models/review_assignment.rb
* app/views/student_review/list.html.erb

===References===

#[https://github.com/expertiza/expertiza Expertiza on GitHub]
#[https://github.com/WintersLt/expertiza GitHub Project Repository Fork]
#[http://expertiza.ncsu.edu/ The live Expertiza website]
#[http://wikis.lib.ncsu.edu/index.php/Expertiza Expertiza project documentation wiki]
#Clean Code: A handbook of agile software craftsmanship. Author: Robert C Martin

CSC/ECE 517 Fall 2017/E1777 Coherent specification of review requirements.rb

2017-11-01T17:32:58Z

Ssuresh6: /* Drawbacks and Solutions */

==E1777. OSS Project Green: Coherent specification of review requirements==

This page provides a description of the Expertiza based OSS project.

__TOC__

===About Expertiza===

[http://expertiza.ncsu.edu/ Expertiza] is an open source project based on [http://rubyonrails.org/ Ruby on Rails] framework. Expertiza allows the instructor to create new assignments and customize new or existing assignments. It also allows the instructor to create a list of topics the students can sign up for. Students can form teams in Expertiza to work on various projects and assignments. Students can also peer review other students' submissions. Expertiza supports submission across various document types, including the URLs and wiki pages.

===Problem Statement===
The following tasks were accomplished in this project:

* Information buttons to elaborate difference between the synonymous statements present in review strategy tab need to be included.
* The punctuation, syntax and the capitalization of statements in the review strategy tab requires improvement.
* Message to a reviewer regarding how many reviews they are required to and allowed to perform is not clear enough.
* Assignment of work to review for a new reviewer is based on the number of previous reviews assigned for that submission which is incorrect.

===Implementation===

=====Drawbacks and Solutions=====
'''Issue #402 ''': To allow reviewers a larger set of topics to choose from, the instructor can set the threshold (on the Review Strategy tab of assignment creation/editing) to some integer k > 0. Then any submission that has within k reviews of the fewest number can be chosen by a new reviewer. Let’s say that all submissions have at least 1 review. If k = 3, then the reviewer can choose any topic where there is a submission that has 4 or fewer reviews so far. Suppose that the minimum number of reviews for any submission is 2, but that I have reviewed all the submissions that have only 2 reviews. Then I’m not allowed to review at all (unless k > 0).
* '''Solution''': This issue has been fixed previously. The reviewer will get assigned a submission even if it has fulfilled the required number of reviews, to ensure that the reviewer always receives a new submission to review as long as it is not his own.
*'''Test Plan''':This is shown graphically with dummy topics and assignment. We create a dummy assignment with two different topics oss1 and oss2.
oss1 has 2 slots and oss2 has 1 slot.
So when a student who has submitted oss2 tries to review others work he will not be able to select oos2 at all because he cannot review his own submission.
When a person who has oss1 as topic goes to this page and he asks to get a new submission for review it will give another slot’s oss1.
If he does it again it says there are no more submissions available to review for this topic.
So if he has already reviewed all the other submissions of his topic oss1 and he requests for one more submission of oss1 , he will not get any more submission to review because the only one left is
his own submission.

'''Issue #969''':
*a.The two statements "Set Allowed Number of Reviews per reviewer" and "Set Required Number of Reviews per reviewer." are not well differentiated. It can very confusing for an instructor to give values for
them.

* '''Solution''': Information buttons have been provided beside these statements in the review strategy tab to make sure that the instructor knows the difference between the two.
This modification is made in the app/views/assignments/edit/_review_strategy.html.erb
<pre>

<img src="/assets/info.png" title='This is the minimum number of reviews a reviewer has to perform'>
<br/>
<br/>
<%= check_box_tag('review', 'true', true, {:onChange => 'hasReviewChanged()'}) %>
<%= label_tag('review', 'Has max review limit?') %>&emsp;&emsp;
<span id="reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_reviews_allowed]', ' Set allowed number of reviews per reviewer') %>
<%if @assignment_form.assignment.num_reviews_allowed && @assignment_form.assignment.num_reviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', @assignment_form.assignment.num_reviews_allowed, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', 3, size: 1) %>
<%end%>

<img src="/assets/info.png" title='This is the maximum number of reviews a reviewer can perform'<%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
</span>

</td>
</pre>
[[File:morecrap.png]]

[[File:lesscrap.png]]

*b) When the number of allowed or required reviews is not set on the Review Strategy tab, the system does not have a message to display to a reviewer about how many submissions of work they are required to and allowed to review depending on the values set by the instructor in the "allowed number of reviewers per reviewer" field and the "Set Required Number of Reviews per reviewer." field.

*'''Solution''': This view has been implemented in the app/views/student_review/list.html.erb
<pre>

<h2>Reviews for "<%= @assignment.name %>"</h2>


<% if @assignment.num_reviews_allowed.nil? || @assignment.num_reviews_allowed == -1%>
<h4>Your instructor expects you to do <%= @assignment.num_reviews_required %> reviews. You are not allowed to do any extra reviews. </h4>

<% elsif @assignment.num_reviews_allowed == @assignment.num_reviews_required %>
<h4>You should perform exactly <%= @assignment.num_reviews_allowed %> reviews </h4>
<% else %>

<h4>You may perform between <%= @assignment.num_reviews_required %> and <%= @assignment.num_reviews_allowed %> reviews</h4>
<% end %>
</pre>

*c)The capitalization and punctuation of statements such as "Set Allowed Number of Reviews per reviewer" and "Set Required Number of Reviews per reviewer." in review strategy tab are incorrect.
*'''Solution''': This view has been fixed by making changes in the respective files.
The changes were made in the file : app/views/assignments/edit/_review_strategy.html.erb
<pre>
<%= label_tag('assignment_form[assignment][num_reviews_required]', 'Set required number of reviews per reviewer') %>
<%= text_field_tag('assignment_form[assignment][num_reviews_required]', @assignment_form.assignment.num_reviews_required ||= @assignment_form.assignment.num_reviews, size: 1) %>


<img src="/assets/info.png" title='This is the minimum number of reviews a reviewer has to perform'>
<br/>
<br/>
<%= check_box_tag('review', 'true', true, {:onChange => 'hasReviewChanged()'}) %>
<%= label_tag('review', 'Has max review limit?') %>&emsp;&emsp;
<span id="reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_reviews_allowed]', ' Set allowed number of reviews per reviewer') %>
<%if @assignment_form.assignment.num_reviews_allowed && @assignment_form.assignment.num_reviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', @assignment_form.assignment.num_reviews_allowed, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', 3, size: 1) %>
<%end%>

<img src="/assets/info.png" title='This is the maximum number of reviews a reviewer can perform'<%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
</span>

</td>
<td width="50%">

<%= label_tag('assignment_form[assignment][num_metareviews_required]', 'Set required number of meta-reviews per reviewer') %>
<%= text_field_tag('assignment_form[assignment][num_metareviews_required]', @assignment_form.assignment.num_metareviews_required ||= @assignment_form.assignment.num_metareviews_allowed, size: 1) %>
<br/>
<br/>

<%= check_box_tag('meta_review', 'true', true, {:onChange => 'hasMetaReviewChanged()'}) %>
<%= label_tag('meta_review', 'Has meta-review limit?') %>&emsp;&emsp;
<span id="meta_reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_metareviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_metareviews_allowed]', ' Set allowed number of meta-reviews per reviewer') %>
<%if @assignment_form.assignment.num_metareviews_allowed && @assignment_form.assignment.num_metareviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_metareviews_allowed]', @assignment_form.assignment.num_metareviews_allowed ||= 3, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_metareviews_allowed]', 3, size: 1) %>
<%end%>
</span>

</td>
</tr>

</pre>
*'''Test Plan''': Set allowed number and required number of reviews per submission as 3 and 3 respectively.
Now it will display "you should perform exactly 3 reviews."
[[File:uifixmodi.PNG]]

* '''Issue #965''': The assignment of work to review to a reviewer depends on the number of reviews it has previously received, if that work has been assigned to a reviewer previously and its review has
not been submitted then the system will still consider it such that it would not assign the same work to another reviewer if it has enough number of reviewers previously assigned to review it. This might
lead to a scenario where that work was assigned to enough number of reviewers, but it does not receive enough reviews and cannot be assigned to any new reviewers either.
*'''Solution''': This implementation has been fixed. The system now considers only the reviews that were submitted by the reviewers for that work so that every submission gets assigned to reviewers until
it receives its allowed number of reviews.

The modified file is app/models/review_assignment.rb and the modification is done in the method reject_by_max_reviews_per_submission

<pre>
def reject_by_max_reviews_per_submission(contributor_set)
contributor_set.reject! {|contributor| contributor.responses.reject {|response| !response.is_submitted }.count >= max_reviews_per_submission }
contributor_set
end
</pre>

==Modified Files==
* app/views/assignments/edit/_review_strategy.html.erb
* app/models/review_assignment.rb
* app/views/student_review/list.html.erb

===References===

#[https://github.com/expertiza/expertiza Expertiza on GitHub]
#[https://github.com/WintersLt/expertiza GitHub Project Repository Fork]
#[http://expertiza.ncsu.edu/ The live Expertiza website]
#[http://wikis.lib.ncsu.edu/index.php/Expertiza Expertiza project documentation wiki]
#Clean Code: A handbook of agile software craftsmanship. Author: Robert C Martin

CSC/ECE 517 Fall 2017/E1777 Coherent specification of review requirements.rb

2017-11-01T17:32:20Z

Ssuresh6: /* Drawbacks and Solutions */

==E1777. OSS Project Green: Coherent specification of review requirements==

This page provides a description of the Expertiza based OSS project.

__TOC__

===About Expertiza===

[http://expertiza.ncsu.edu/ Expertiza] is an open source project based on [http://rubyonrails.org/ Ruby on Rails] framework. Expertiza allows the instructor to create new assignments and customize new or existing assignments. It also allows the instructor to create a list of topics the students can sign up for. Students can form teams in Expertiza to work on various projects and assignments. Students can also peer review other students' submissions. Expertiza supports submission across various document types, including the URLs and wiki pages.

===Problem Statement===
The following tasks were accomplished in this project:

* Information buttons to elaborate difference between the synonymous statements present in review strategy tab need to be included.
* The punctuation, syntax and the capitalization of statements in the review strategy tab requires improvement.
* Message to a reviewer regarding how many reviews they are required to and allowed to perform is not clear enough.
* Assignment of work to review for a new reviewer is based on the number of previous reviews assigned for that submission which is incorrect.

===Implementation===

=====Drawbacks and Solutions=====
'''Issue #402 ''': To allow reviewers a larger set of topics to choose from, the instructor can set the threshold (on the Review Strategy tab of assignment creation/editing) to some integer k > 0. Then any submission that has within k reviews of the fewest number can be chosen by a new reviewer. Let’s say that all submissions have at least 1 review. If k = 3, then the reviewer can choose any topic where there is a submission that has 4 or fewer reviews so far. Suppose that the minimum number of reviews for any submission is 2, but that I have reviewed all the submissions that have only 2 reviews. Then I’m not allowed to review at all (unless k > 0).
* '''Solution''': This issue has been fixed previously. The reviewer will get assigned a submission even if it has fulfilled the required number of reviews, to ensure that the reviewer always receives a new submission to review as long as it is not his own.
*'''Test Plan''':This is shown graphically with dummy topics and assignment. We create a dummy assignment with two different topics oss1 and oss2.
oss1 has 2 slots and oss2 has 1 slot.
So when a student who has submitted oss2 tries to review others work he will not be able to select oos2 at all because he cannot review his own submission.
When a person who has oss1 as topic goes to this page and he asks to get a new submission for review it will give another slot’s oss1.
If he does it again it says there are no more submissions available to review for this topic.
So if he has already reviewed all the other submissions of his topic oss1 and he requests for one more submission of oss1 , he will not get any more submission to review because the only one left is
his own submission.

'''Issue #969''':
*a.The two statements "Set Allowed Number of Reviews per reviewer" and "Set Required Number of Reviews per reviewer." are not well differentiated. It can very confusing for an instructor to give values for
them.

* '''Solution''': Information buttons have been provided beside these statements in the review strategy tab to make sure that the instructor knows the difference between the two.
This modification is made in the app/views/assignments/edit/_review_strategy.html.erb
<pre>

<img src="/assets/info.png" title='This is the minimum number of reviews a reviewer has to perform'>
<br/>
<br/>
<%= check_box_tag('review', 'true', true, {:onChange => 'hasReviewChanged()'}) %>
<%= label_tag('review', 'Has max review limit?') %>&emsp;&emsp;
<span id="reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_reviews_allowed]', ' Set allowed number of reviews per reviewer') %>
<%if @assignment_form.assignment.num_reviews_allowed && @assignment_form.assignment.num_reviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', @assignment_form.assignment.num_reviews_allowed, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', 3, size: 1) %>
<%end%>

<img src="/assets/info.png" title='This is the maximum number of reviews a reviewer can perform'<%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
</span>

</td>
</pre>
[[File:morecrap.png]]

[[File:lesscrap.png]]

*b) When the number of allowed or required reviews is not set on the Review Strategy tab, the system does not have a message to display to a reviewer about how many submissions of work they are required to and allowed to review depending on the values set by the instructor in the "allowed number of reviewers per reviewer" field and the "Set Required Number of Reviews per reviewer." field.

'''Solution''': This view has been implemented in the app/views/student_review/list.html.erb
<pre>

<h2>Reviews for "<%= @assignment.name %>"</h2>


<% if @assignment.num_reviews_allowed.nil? || @assignment.num_reviews_allowed == -1%>
<h4>Your instructor expects you to do <%= @assignment.num_reviews_required %> reviews. You are not allowed to do any extra reviews. </h4>

<% elsif @assignment.num_reviews_allowed == @assignment.num_reviews_required %>
<h4>You should perform exactly <%= @assignment.num_reviews_allowed %> reviews </h4>
<% else %>

<h4>You may perform between <%= @assignment.num_reviews_required %> and <%= @assignment.num_reviews_allowed %> reviews</h4>
<% end %>
</pre>

*c)The capitalization and punctuation of statements such as "Set Allowed Number of Reviews per reviewer" and "Set Required Number of Reviews per reviewer." in review strategy tab are incorrect.
*'''Solution''': This view has been fixed by making changes in the respective files.
The changes were made in the file : app/views/assignments/edit/_review_strategy.html.erb
<pre>
<%= label_tag('assignment_form[assignment][num_reviews_required]', 'Set required number of reviews per reviewer') %>
<%= text_field_tag('assignment_form[assignment][num_reviews_required]', @assignment_form.assignment.num_reviews_required ||= @assignment_form.assignment.num_reviews, size: 1) %>


<img src="/assets/info.png" title='This is the minimum number of reviews a reviewer has to perform'>
<br/>
<br/>
<%= check_box_tag('review', 'true', true, {:onChange => 'hasReviewChanged()'}) %>
<%= label_tag('review', 'Has max review limit?') %>&emsp;&emsp;
<span id="reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_reviews_allowed]', ' Set allowed number of reviews per reviewer') %>
<%if @assignment_form.assignment.num_reviews_allowed && @assignment_form.assignment.num_reviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', @assignment_form.assignment.num_reviews_allowed, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', 3, size: 1) %>
<%end%>

<img src="/assets/info.png" title='This is the maximum number of reviews a reviewer can perform'<%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
</span>

</td>
<td width="50%">

<%= label_tag('assignment_form[assignment][num_metareviews_required]', 'Set required number of meta-reviews per reviewer') %>
<%= text_field_tag('assignment_form[assignment][num_metareviews_required]', @assignment_form.assignment.num_metareviews_required ||= @assignment_form.assignment.num_metareviews_allowed, size: 1) %>
<br/>
<br/>

<%= check_box_tag('meta_review', 'true', true, {:onChange => 'hasMetaReviewChanged()'}) %>
<%= label_tag('meta_review', 'Has meta-review limit?') %>&emsp;&emsp;
<span id="meta_reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_metareviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_metareviews_allowed]', ' Set allowed number of meta-reviews per reviewer') %>
<%if @assignment_form.assignment.num_metareviews_allowed && @assignment_form.assignment.num_metareviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_metareviews_allowed]', @assignment_form.assignment.num_metareviews_allowed ||= 3, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_metareviews_allowed]', 3, size: 1) %>
<%end%>
</span>

</td>
</tr>

</pre>
*'''Test Plan''': Set allowed number and required number of reviews per submission as 3 and 3 respectively.
Now it will display "you should perform exactly 3 reviews."
[[File:uifixmodi.PNG]]

* '''Issue #965''': The assignment of work to review to a reviewer depends on the number of reviews it has previously received, if that work has been assigned to a reviewer previously and its review has
not been submitted then the system will still consider it such that it would not assign the same work to another reviewer if it has enough number of reviewers previously assigned to review it. This might
lead to a scenario where that work was assigned to enough number of reviewers, but it does not receive enough reviews and cannot be assigned to any new reviewers either.
*'''Solution''': This implementation has been fixed. The system now considers only the reviews that were submitted by the reviewers for that work so that every submission gets assigned to reviewers until
it receives its allowed number of reviews.

The modified file is app/models/review_assignment.rb and the modification is done in the method reject_by_max_reviews_per_submission

<pre>
def reject_by_max_reviews_per_submission(contributor_set)
contributor_set.reject! {|contributor| contributor.responses.reject {|response| !response.is_submitted }.count >= max_reviews_per_submission }
contributor_set
end
</pre>

==Modified Files==
* app/views/assignments/edit/_review_strategy.html.erb
* app/models/review_assignment.rb
* app/views/student_review/list.html.erb

===References===

#[https://github.com/expertiza/expertiza Expertiza on GitHub]
#[https://github.com/WintersLt/expertiza GitHub Project Repository Fork]
#[http://expertiza.ncsu.edu/ The live Expertiza website]
#[http://wikis.lib.ncsu.edu/index.php/Expertiza Expertiza project documentation wiki]
#Clean Code: A handbook of agile software craftsmanship. Author: Robert C Martin

CSC/ECE 517 Fall 2017/E1777 Coherent specification of review requirements.rb

2017-11-01T17:30:46Z

Ssuresh6: /* Implementation */

==E1777. OSS Project Green: Coherent specification of review requirements==

This page provides a description of the Expertiza based OSS project.

__TOC__

===About Expertiza===

[http://expertiza.ncsu.edu/ Expertiza] is an open source project based on [http://rubyonrails.org/ Ruby on Rails] framework. Expertiza allows the instructor to create new assignments and customize new or existing assignments. It also allows the instructor to create a list of topics the students can sign up for. Students can form teams in Expertiza to work on various projects and assignments. Students can also peer review other students' submissions. Expertiza supports submission across various document types, including the URLs and wiki pages.

===Problem Statement===
The following tasks were accomplished in this project:

* Information buttons to elaborate difference between the synonymous statements present in review strategy tab need to be included.
* The punctuation, syntax and the capitalization of statements in the review strategy tab requires improvement.
* Message to a reviewer regarding how many reviews they are required to and allowed to perform is not clear enough.
* Assignment of work to review for a new reviewer is based on the number of previous reviews assigned for that submission which is incorrect.

===Implementation===

=====Drawbacks and Solutions=====
'''Issue #402 ''': To allow reviewers a larger set of topics to choose from, the instructor can set the threshold (on the Review Strategy tab of assignment creation/editing) to some integer k > 0. Then any submission that has within k reviews of the fewest number can be chosen by a new reviewer. Let’s say that all submissions have at least 1 review. If k = 3, then the reviewer can choose any topic where there is a submission that has 4 or fewer reviews so far. Suppose that the minimum number of reviews for any submission is 2, but that I have reviewed all the submissions that have only 2 reviews. Then I’m not allowed to review at all (unless k > 0).
* '''Solution''': This issue has been fixed previously. The reviewer will get assigned a submission even if it has fulfilled the required number of reviews, to ensure that the reviewer always receives a new submission to review as long as it is not his own.
*'''Test Plan''':This is shown graphically with dummy topics and assignment. We create a dummy assignment with two different topics oss1 and oss2.
oss1 has 2 slots and oss2 has 1 slot.
So when a student who has submitted oss2 tries to review others work he will not be able to select oos2 at all because he cannot review his own submission.
When a person who has oss1 as topic goes to this page and he asks to get a new submission for review it will give another slot’s oss1.
If he does it again it says there are no more submissions available to review for this topic.
So if he has already reviewed all the other submissions of his topic oss1 and he requests for one more submission of oss1 , he will not get any more submission to review because the only one left is his own submission.

'''Issue #969''':
*a.The two statements "Set Allowed Number of Reviews per reviewer" and "Set Required Number of Reviews per reviewer." are not well differentiated. It can very confusing for an instructor to give values for them.

* '''Solution''': Information buttons have been provided beside these statements in the review strategy tab to make sure that the instructor knows the difference between the two.
This modification is made in the app/views/assignments/edit/_review_strategy.html.erb
<pre>

<img src="/assets/info.png" title='This is the minimum number of reviews a reviewer has to perform'>
<br/>
<br/>
<%= check_box_tag('review', 'true', true, {:onChange => 'hasReviewChanged()'}) %>
<%= label_tag('review', 'Has max review limit?') %>&emsp;&emsp;
<span id="reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_reviews_allowed]', ' Set allowed number of reviews per reviewer') %>
<%if @assignment_form.assignment.num_reviews_allowed && @assignment_form.assignment.num_reviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', @assignment_form.assignment.num_reviews_allowed, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', 3, size: 1) %>
<%end%>

<img src="/assets/info.png" title='This is the maximum number of reviews a reviewer can perform'<%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
</span>

</td>
</pre>
[[File:morecrap.png]]

[[File:lesscrap.png]]

*b) When the number of allowed or required reviews is not set on the Review Strategy tab, the system does not have a message to display to a reviewer about how many submissions of work they are required to and allowed to review depending on the values set by the instructor in the "allowed number of reviewers per reviewer" field and the "Set Required Number of Reviews per reviewer." field.

'''Solution''': This view has been implemented in the app/views/student_review/list.html.erb
<pre>

<h2>Reviews for "<%= @assignment.name %>"</h2>


<% if @assignment.num_reviews_allowed.nil? || @assignment.num_reviews_allowed == -1%>
<h4>Your instructor expects you to do <%= @assignment.num_reviews_required %> reviews. You are not allowed to do any extra reviews. </h4>

<% elsif @assignment.num_reviews_allowed == @assignment.num_reviews_required %>
<h4>You should perform exactly <%= @assignment.num_reviews_allowed %> reviews </h4>
<% else %>

<h4>You may perform between <%= @assignment.num_reviews_required %> and <%= @assignment.num_reviews_allowed %> reviews</h4>
<% end %>
</pre>

*c) The capitalization and punctuation of statements such as "Set Allowed Number of Reviews per reviewer" and "Set Required Number of Reviews per reviewer." in review strategy tab are incorrect.
*'''Solution''': This view has been fixed by making changes in the respective files.
The changes were made in the file : app/views/assignments/edit/_review_strategy.html.erb
<pre>
<%= label_tag('assignment_form[assignment][num_reviews_required]', 'Set required number of reviews per reviewer') %>
<%= text_field_tag('assignment_form[assignment][num_reviews_required]', @assignment_form.assignment.num_reviews_required ||= @assignment_form.assignment.num_reviews, size: 1) %>


<img src="/assets/info.png" title='This is the minimum number of reviews a reviewer has to perform'>
<br/>
<br/>
<%= check_box_tag('review', 'true', true, {:onChange => 'hasReviewChanged()'}) %>
<%= label_tag('review', 'Has max review limit?') %>&emsp;&emsp;
<span id="reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_reviews_allowed]', ' Set allowed number of reviews per reviewer') %>
<%if @assignment_form.assignment.num_reviews_allowed && @assignment_form.assignment.num_reviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', @assignment_form.assignment.num_reviews_allowed, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', 3, size: 1) %>
<%end%>

<img src="/assets/info.png" title='This is the maximum number of reviews a reviewer can perform'<%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
</span>

</td>
<td width="50%">

<%= label_tag('assignment_form[assignment][num_metareviews_required]', 'Set required number of meta-reviews per reviewer') %>
<%= text_field_tag('assignment_form[assignment][num_metareviews_required]', @assignment_form.assignment.num_metareviews_required ||= @assignment_form.assignment.num_metareviews_allowed, size: 1) %>
<br/>
<br/>

<%= check_box_tag('meta_review', 'true', true, {:onChange => 'hasMetaReviewChanged()'}) %>
<%= label_tag('meta_review', 'Has meta-review limit?') %>&emsp;&emsp;
<span id="meta_reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_metareviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_metareviews_allowed]', ' Set allowed number of meta-reviews per reviewer') %>
<%if @assignment_form.assignment.num_metareviews_allowed && @assignment_form.assignment.num_metareviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_metareviews_allowed]', @assignment_form.assignment.num_metareviews_allowed ||= 3, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_metareviews_allowed]', 3, size: 1) %>
<%end%>
</span>

</td>
</tr>

</pre>
*'''Test Plan''': Set allowed number and required number of reviews per submission as 3 and 3 respectively.
Now it will display "you should perform exactly 3 reviews."
[[File:uifixmodi.PNG]]

* '''Issue #965''': The assignment of work to review to a reviewer depends on the number of reviews it has previously received, if that work has been assigned to a reviewer previously and its review has not been submitted then the system will still consider it such that it would not assign the same work to another reviewer if it has enough number of reviewers previously assigned to review it. This might lead to a scenario where that work was assigned to enough number of reviewers, but it does not receive enough reviews and cannot be assigned to any new reviewers either.
*'''Solution''': This implementation has been fixed. The system now considers only the reviews that were submitted by the reviewers for that work so that every submission gets assigned to reviewers until it receives its allowed number of reviews.

The modified file is app/models/review_assignment.rb and the modification is done in the method reject_by_max_reviews_per_submission

<pre>
def reject_by_max_reviews_per_submission(contributor_set)
contributor_set.reject! {|contributor| contributor.responses.reject {|response| !response.is_submitted }.count >= max_reviews_per_submission }
contributor_set
end
</pre>

==Modified Files==
* app/views/assignments/edit/_review_strategy.html.erb
* app/models/review_assignment.rb
* app/views/student_review/list.html.erb

===References===

#[https://github.com/expertiza/expertiza Expertiza on GitHub]
#[https://github.com/WintersLt/expertiza GitHub Project Repository Fork]
#[http://expertiza.ncsu.edu/ The live Expertiza website]
#[http://wikis.lib.ncsu.edu/index.php/Expertiza Expertiza project documentation wiki]
#Clean Code: A handbook of agile software craftsmanship. Author: Robert C Martin

CSC/ECE 517 Fall 2017/E1777 Coherent specification of review requirements.rb

2017-10-30T20:53:24Z

Ssuresh6: /* Problem Statement */

==E1777. OSS Project Green: Coherent specification of review requirements==

This page provides a description of the Expertiza based OSS project.

__TOC__

===About Expertiza===

[http://expertiza.ncsu.edu/ Expertiza] is an open source project based on [http://rubyonrails.org/ Ruby on Rails] framework. Expertiza allows the instructor to create new assignments and customize new or existing assignments. It also allows the instructor to create a list of topics the students can sign up for. Students can form teams in Expertiza to work on various projects and assignments. Students can also peer review other students' submissions. Expertiza supports submission across various document types, including the URLs and wiki pages.

===Problem Statement===
The following tasks were accomplished in this project:

* Information buttons to elaborate difference between the synonymous statements present in review strategy tab need to be included.
* The punctuation, syntax and the capitalization of statements in the review strategy tab requires improvement.
* Message to a reviewer regarding how many reviews they are required to and allowed to perform is not clear enough.
* Assignment of work to review for a new reviewer is based on the number of previous reviews assigned for that submission which is incorrect.

===Implementation===

=====Drawbacks and Solutions=====
'''Issue #402 ''': To allow reviewers a larger set of topics to choose from, the instructor can set the threshold (on the Review Strategy tab of assignment creation/editing) to some integer k > 0. Then any submission that has within k reviews of the fewest number can be chosen by a new reviewer. Let’s say that all submissions have at least 1 review. If k = 3, then the reviewer can choose any topic where there is a submission that has 4 or fewer reviews so far. Suppose that the minimum number of reviews for any submission is 2, but that I have reviewed all the submissions that have only 2 reviews. Then I’m not allowed to review at all (unless k > 0).
* '''Solution''': This issue has been fixed previously. The reviewer will get assigned a submission even if it has fulfilled the required number of reviews, to ensure that the reviewer always receives a new submission to review as long as it is not his own.

'''Issue #969''':
*a.The two statements "Set Allowed Number of Reviews per reviewer" and "Set Required Number of Reviews per reviewer." are not well differentiated. It can very confusing for an instructor to give values for them.

* '''Solution''': Information buttons have been provided beside these statements in the review strategy tab to make sure that the instructor knows the difference between the two.
This modification is made in the app/views/assignments/edit/_review_strategy.html.erb
<pre>

<img src="/assets/info.png" title='This is the minimum number of reviews a reviewer has to perform'>
<br/>
<br/>
<%= check_box_tag('review', 'true', true, {:onChange => 'hasReviewChanged()'}) %>
<%= label_tag('review', 'Has max review limit?') %>&emsp;&emsp;
<span id="reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_reviews_allowed]', ' Set allowed number of reviews per reviewer') %>
<%if @assignment_form.assignment.num_reviews_allowed && @assignment_form.assignment.num_reviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', @assignment_form.assignment.num_reviews_allowed, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', 3, size: 1) %>
<%end%>

<img src="/assets/info.png" title='This is the maximum number of reviews a reviewer can perform'<%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
</span>

</td>
</pre>
[[File:morecrap.png]]

[[File:lesscrap.png]]

*b) When the number of allowed or required reviews is not set on the Review Strategy tab, the system does not have a message to display to a reviewer about how many submissions of work they are required to and allowed to review depending on the values set by the instructor in the "allowed number of reviewers per reviewer" field and the "Set Required Number of Reviews per reviewer." field.

'''Solution''': This view has been implemented in the app/views/student_review/list.html.erb
<pre>

<h2>Reviews for "<%= @assignment.name %>"</h2>


<% if @assignment.num_reviews_allowed.nil? || @assignment.num_reviews_allowed == -1%>
<h4>Your instructor expects you to do <%= @assignment.num_reviews_required %> reviews. You are not allowed to do any extra reviews. </h4>

<% elsif @assignment.num_reviews_allowed == @assignment.num_reviews_required %>
<h4>You should perform exactly <%= @assignment.num_reviews_allowed %> reviews </h4>
<% else %>

<h4>You may perform between <%= @assignment.num_reviews_required %> and <%= @assignment.num_reviews_allowed %> reviews</h4>
<% end %>
</pre>

*c) The capitalization and punctuation of statements such as "Set Allowed Number of Reviews per reviewer" and "Set Required Number of Reviews per reviewer." in review strategy tab are incorrect.
*'''Solution''': This view has been fixed by making changes in the respective files.
The changes were made in the file : app/views/assignments/edit/_review_strategy.html.erb
<pre>
<%= label_tag('assignment_form[assignment][num_reviews_required]', 'Set required number of reviews per reviewer') %>
<%= text_field_tag('assignment_form[assignment][num_reviews_required]', @assignment_form.assignment.num_reviews_required ||= @assignment_form.assignment.num_reviews, size: 1) %>


<img src="/assets/info.png" title='This is the minimum number of reviews a reviewer has to perform'>
<br/>
<br/>
<%= check_box_tag('review', 'true', true, {:onChange => 'hasReviewChanged()'}) %>
<%= label_tag('review', 'Has max review limit?') %>&emsp;&emsp;
<span id="reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_reviews_allowed]', ' Set allowed number of reviews per reviewer') %>
<%if @assignment_form.assignment.num_reviews_allowed && @assignment_form.assignment.num_reviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', @assignment_form.assignment.num_reviews_allowed, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', 3, size: 1) %>
<%end%>

<img src="/assets/info.png" title='This is the maximum number of reviews a reviewer can perform'<%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
</span>

</td>
<td width="50%">

<%= label_tag('assignment_form[assignment][num_metareviews_required]', 'Set required number of meta-reviews per reviewer') %>
<%= text_field_tag('assignment_form[assignment][num_metareviews_required]', @assignment_form.assignment.num_metareviews_required ||= @assignment_form.assignment.num_metareviews_allowed, size: 1) %>
<br/>
<br/>

<%= check_box_tag('meta_review', 'true', true, {:onChange => 'hasMetaReviewChanged()'}) %>
<%= label_tag('meta_review', 'Has meta-review limit?') %>&emsp;&emsp;
<span id="meta_reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_metareviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_metareviews_allowed]', ' Set allowed number of meta-reviews per reviewer') %>
<%if @assignment_form.assignment.num_metareviews_allowed && @assignment_form.assignment.num_metareviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_metareviews_allowed]', @assignment_form.assignment.num_metareviews_allowed ||= 3, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_metareviews_allowed]', 3, size: 1) %>
<%end%>
</span>

</td>
</tr>

</pre>
[[File:uifixmodi.PNG]]

* '''Issue #965''': The assignment of work to review to a reviewer depends on the number of reviews it has previously received, if that work has been assigned to a reviewer previously and its review has not been submitted then the system will still consider it such that it would not assign the same work to another reviewer if it has enough number of reviewers previously assigned to review it. This might lead to a scenario where that work was assigned to enough number of reviewers, but it does not receive enough reviews and cannot be assigned to any new reviewers either.
*'''Solution''': This implementation has been fixed. The system now considers only the reviews that were submitted by the reviewers for that work so that every submission gets assigned to reviewers until it receives its allowed number of reviews.

The modified file is app/models/review_assignment.rb and the modification is done in the method reject_by_max_reviews_per_submission

<pre>
def reject_by_max_reviews_per_submission(contributor_set)
contributor_set.reject! {|contributor| contributor.responses.reject {|response| !response.is_submitted }.count >= max_reviews_per_submission }
contributor_set
end
</pre>

==Modified Files==
* app/views/assignments/edit/_review_strategy.html.erb
* app/models/review_assignment.rb
* app/views/student_review/list.html.erb

===References===

#[https://github.com/expertiza/expertiza Expertiza on GitHub]
#[https://github.com/WintersLt/expertiza GitHub Project Repository Fork]
#[http://expertiza.ncsu.edu/ The live Expertiza website]
#[http://wikis.lib.ncsu.edu/index.php/Expertiza Expertiza project documentation wiki]
#Clean Code: A handbook of agile software craftsmanship. Author: Robert C Martin

CSC/ECE 517 Fall 2017/E1777 Coherent specification of review requirements.rb

2017-10-30T20:52:52Z

Ssuresh6: /* Implementation */

==E1777. OSS Project Green: Coherent specification of review requirements==

This page provides a description of the Expertiza based OSS project.

__TOC__

===About Expertiza===

[http://expertiza.ncsu.edu/ Expertiza] is an open source project based on [http://rubyonrails.org/ Ruby on Rails] framework. Expertiza allows the instructor to create new assignments and customize new or existing assignments. It also allows the instructor to create a list of topics the students can sign up for. Students can form teams in Expertiza to work on various projects and assignments. Students can also peer review other students' submissions. Expertiza supports submission across various document types, including the URLs and wiki pages.

===Problem Statement===
The following tasks were accomplished in this project:

* The punctuation, syntax and the capitalization of statements in the review strategy tab requires improvement.
* Information buttons to elaborate difference between the synonymous statements present in review strategy tab need to be included.
* Message to a reviewer regarding how many reviews they are required to and allowed to perform is not clear enough.
* Assignment of work to review for a new reviewer is based on the number of previous reviews assigned for that submission which is incorrect.

===Implementation===

=====Drawbacks and Solutions=====
'''Issue #402 ''': To allow reviewers a larger set of topics to choose from, the instructor can set the threshold (on the Review Strategy tab of assignment creation/editing) to some integer k > 0. Then any submission that has within k reviews of the fewest number can be chosen by a new reviewer. Let’s say that all submissions have at least 1 review. If k = 3, then the reviewer can choose any topic where there is a submission that has 4 or fewer reviews so far. Suppose that the minimum number of reviews for any submission is 2, but that I have reviewed all the submissions that have only 2 reviews. Then I’m not allowed to review at all (unless k > 0).
* '''Solution''': This issue has been fixed previously. The reviewer will get assigned a submission even if it has fulfilled the required number of reviews, to ensure that the reviewer always receives a new submission to review as long as it is not his own.

'''Issue #969''':
*a.The two statements "Set Allowed Number of Reviews per reviewer" and "Set Required Number of Reviews per reviewer." are not well differentiated. It can very confusing for an instructor to give values for them.

* '''Solution''': Information buttons have been provided beside these statements in the review strategy tab to make sure that the instructor knows the difference between the two.
This modification is made in the app/views/assignments/edit/_review_strategy.html.erb
<pre>

<img src="/assets/info.png" title='This is the minimum number of reviews a reviewer has to perform'>
<br/>
<br/>
<%= check_box_tag('review', 'true', true, {:onChange => 'hasReviewChanged()'}) %>
<%= label_tag('review', 'Has max review limit?') %>&emsp;&emsp;
<span id="reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_reviews_allowed]', ' Set allowed number of reviews per reviewer') %>
<%if @assignment_form.assignment.num_reviews_allowed && @assignment_form.assignment.num_reviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', @assignment_form.assignment.num_reviews_allowed, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', 3, size: 1) %>
<%end%>

<img src="/assets/info.png" title='This is the maximum number of reviews a reviewer can perform'<%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
</span>

</td>
</pre>
[[File:morecrap.png]]

[[File:lesscrap.png]]

*b) When the number of allowed or required reviews is not set on the Review Strategy tab, the system does not have a message to display to a reviewer about how many submissions of work they are required to and allowed to review depending on the values set by the instructor in the "allowed number of reviewers per reviewer" field and the "Set Required Number of Reviews per reviewer." field.

'''Solution''': This view has been implemented in the app/views/student_review/list.html.erb
<pre>

<h2>Reviews for "<%= @assignment.name %>"</h2>


<% if @assignment.num_reviews_allowed.nil? || @assignment.num_reviews_allowed == -1%>
<h4>Your instructor expects you to do <%= @assignment.num_reviews_required %> reviews. You are not allowed to do any extra reviews. </h4>

<% elsif @assignment.num_reviews_allowed == @assignment.num_reviews_required %>
<h4>You should perform exactly <%= @assignment.num_reviews_allowed %> reviews </h4>
<% else %>

<h4>You may perform between <%= @assignment.num_reviews_required %> and <%= @assignment.num_reviews_allowed %> reviews</h4>
<% end %>
</pre>

*c) The capitalization and punctuation of statements such as "Set Allowed Number of Reviews per reviewer" and "Set Required Number of Reviews per reviewer." in review strategy tab are incorrect.
*'''Solution''': This view has been fixed by making changes in the respective files.
The changes were made in the file : app/views/assignments/edit/_review_strategy.html.erb
<pre>
<%= label_tag('assignment_form[assignment][num_reviews_required]', 'Set required number of reviews per reviewer') %>
<%= text_field_tag('assignment_form[assignment][num_reviews_required]', @assignment_form.assignment.num_reviews_required ||= @assignment_form.assignment.num_reviews, size: 1) %>


<img src="/assets/info.png" title='This is the minimum number of reviews a reviewer has to perform'>
<br/>
<br/>
<%= check_box_tag('review', 'true', true, {:onChange => 'hasReviewChanged()'}) %>
<%= label_tag('review', 'Has max review limit?') %>&emsp;&emsp;
<span id="reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_reviews_allowed]', ' Set allowed number of reviews per reviewer') %>
<%if @assignment_form.assignment.num_reviews_allowed && @assignment_form.assignment.num_reviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', @assignment_form.assignment.num_reviews_allowed, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', 3, size: 1) %>
<%end%>

<img src="/assets/info.png" title='This is the maximum number of reviews a reviewer can perform'<%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
</span>

</td>
<td width="50%">

<%= label_tag('assignment_form[assignment][num_metareviews_required]', 'Set required number of meta-reviews per reviewer') %>
<%= text_field_tag('assignment_form[assignment][num_metareviews_required]', @assignment_form.assignment.num_metareviews_required ||= @assignment_form.assignment.num_metareviews_allowed, size: 1) %>
<br/>
<br/>

<%= check_box_tag('meta_review', 'true', true, {:onChange => 'hasMetaReviewChanged()'}) %>
<%= label_tag('meta_review', 'Has meta-review limit?') %>&emsp;&emsp;
<span id="meta_reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_metareviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_metareviews_allowed]', ' Set allowed number of meta-reviews per reviewer') %>
<%if @assignment_form.assignment.num_metareviews_allowed && @assignment_form.assignment.num_metareviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_metareviews_allowed]', @assignment_form.assignment.num_metareviews_allowed ||= 3, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_metareviews_allowed]', 3, size: 1) %>
<%end%>
</span>

</td>
</tr>

</pre>
[[File:uifixmodi.PNG]]

* '''Issue #965''': The assignment of work to review to a reviewer depends on the number of reviews it has previously received, if that work has been assigned to a reviewer previously and its review has not been submitted then the system will still consider it such that it would not assign the same work to another reviewer if it has enough number of reviewers previously assigned to review it. This might lead to a scenario where that work was assigned to enough number of reviewers, but it does not receive enough reviews and cannot be assigned to any new reviewers either.
*'''Solution''': This implementation has been fixed. The system now considers only the reviews that were submitted by the reviewers for that work so that every submission gets assigned to reviewers until it receives its allowed number of reviews.

The modified file is app/models/review_assignment.rb and the modification is done in the method reject_by_max_reviews_per_submission

<pre>
def reject_by_max_reviews_per_submission(contributor_set)
contributor_set.reject! {|contributor| contributor.responses.reject {|response| !response.is_submitted }.count >= max_reviews_per_submission }
contributor_set
end
</pre>

==Modified Files==
* app/views/assignments/edit/_review_strategy.html.erb
* app/models/review_assignment.rb
* app/views/student_review/list.html.erb

===References===

#[https://github.com/expertiza/expertiza Expertiza on GitHub]
#[https://github.com/WintersLt/expertiza GitHub Project Repository Fork]
#[http://expertiza.ncsu.edu/ The live Expertiza website]
#[http://wikis.lib.ncsu.edu/index.php/Expertiza Expertiza project documentation wiki]
#Clean Code: A handbook of agile software craftsmanship. Author: Robert C Martin

CSC/ECE 517 Fall 2017/E1777 Coherent specification of review requirements.rb

2017-10-30T20:52:11Z

Ssuresh6: /* Implementation */

==E1777. OSS Project Green: Coherent specification of review requirements==

This page provides a description of the Expertiza based OSS project.

__TOC__

===About Expertiza===

[http://expertiza.ncsu.edu/ Expertiza] is an open source project based on [http://rubyonrails.org/ Ruby on Rails] framework. Expertiza allows the instructor to create new assignments and customize new or existing assignments. It also allows the instructor to create a list of topics the students can sign up for. Students can form teams in Expertiza to work on various projects and assignments. Students can also peer review other students' submissions. Expertiza supports submission across various document types, including the URLs and wiki pages.

===Problem Statement===
The following tasks were accomplished in this project:

* The punctuation, syntax and the capitalization of statements in the review strategy tab requires improvement.
* Information buttons to elaborate difference between the synonymous statements present in review strategy tab need to be included.
* Message to a reviewer regarding how many reviews they are required to and allowed to perform is not clear enough.
* Assignment of work to review for a new reviewer is based on the number of previous reviews assigned for that submission which is incorrect.

===Implementation===

=====Drawbacks and Solutions=====
'''Issue #402 ''': To allow reviewers a larger set of topics to choose from, the instructor can set the threshold (on the Review Strategy tab of assignment creation/editing) to some integer k > 0. Then any submission that has within k reviews of the fewest number can be chosen by a new reviewer. Let’s say that all submissions have at least 1 review. If k = 3, then the reviewer can choose any topic where there is a submission that has 4 or fewer reviews so far. Suppose that the minimum number of reviews for any submission is 2, but that I have reviewed all the submissions that have only 2 reviews. Then I’m not allowed to review at all (unless k > 0).
* '''Solution''': This issue has been fixed previously. The reviewer will get assigned a submission even if it has fulfilled the required number of reviews, to ensure that the reviewer always receives a new submission to review as long as it is not his own.
[[File:morecrap.png]]

[[File:lesscrap.png]]
'''Issue #969''':
*a.The two statements "Set Allowed Number of Reviews per reviewer" and "Set Required Number of Reviews per reviewer." are not well differentiated. It can very confusing for an instructor to give values for them.

* '''Solution''': Information buttons have been provided beside these statements in the review strategy tab to make sure that the instructor knows the difference between the two.
This modification is made in the app/views/assignments/edit/_review_strategy.html.erb
<pre>

<img src="/assets/info.png" title='This is the minimum number of reviews a reviewer has to perform'>
<br/>
<br/>
<%= check_box_tag('review', 'true', true, {:onChange => 'hasReviewChanged()'}) %>
<%= label_tag('review', 'Has max review limit?') %>&emsp;&emsp;
<span id="reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_reviews_allowed]', ' Set allowed number of reviews per reviewer') %>
<%if @assignment_form.assignment.num_reviews_allowed && @assignment_form.assignment.num_reviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', @assignment_form.assignment.num_reviews_allowed, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', 3, size: 1) %>
<%end%>

<img src="/assets/info.png" title='This is the maximum number of reviews a reviewer can perform'<%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
</span>

</td>
</pre>

*b) When the number of allowed or required reviews is not set on the Review Strategy tab, the system does not have a message to display to a reviewer about how many submissions of work they are required to and allowed to review depending on the values set by the instructor in the "allowed number of reviewers per reviewer" field and the "Set Required Number of Reviews per reviewer." field.

'''Solution''': This view has been implemented in the app/views/student_review/list.html.erb
<pre>

<h2>Reviews for "<%= @assignment.name %>"</h2>


<% if @assignment.num_reviews_allowed.nil? || @assignment.num_reviews_allowed == -1%>
<h4>Your instructor expects you to do <%= @assignment.num_reviews_required %> reviews. You are not allowed to do any extra reviews. </h4>

<% elsif @assignment.num_reviews_allowed == @assignment.num_reviews_required %>
<h4>You should perform exactly <%= @assignment.num_reviews_allowed %> reviews </h4>
<% else %>

<h4>You may perform between <%= @assignment.num_reviews_required %> and <%= @assignment.num_reviews_allowed %> reviews</h4>
<% end %>
</pre>

*c) The capitalization and punctuation of statements such as "Set Allowed Number of Reviews per reviewer" and "Set Required Number of Reviews per reviewer." in review strategy tab are incorrect.
*'''Solution''': This view has been fixed by making changes in the respective files.
The changes were made in the file : app/views/assignments/edit/_review_strategy.html.erb
<pre>
<%= label_tag('assignment_form[assignment][num_reviews_required]', 'Set required number of reviews per reviewer') %>
<%= text_field_tag('assignment_form[assignment][num_reviews_required]', @assignment_form.assignment.num_reviews_required ||= @assignment_form.assignment.num_reviews, size: 1) %>


<img src="/assets/info.png" title='This is the minimum number of reviews a reviewer has to perform'>
<br/>
<br/>
<%= check_box_tag('review', 'true', true, {:onChange => 'hasReviewChanged()'}) %>
<%= label_tag('review', 'Has max review limit?') %>&emsp;&emsp;
<span id="reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_reviews_allowed]', ' Set allowed number of reviews per reviewer') %>
<%if @assignment_form.assignment.num_reviews_allowed && @assignment_form.assignment.num_reviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', @assignment_form.assignment.num_reviews_allowed, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', 3, size: 1) %>
<%end%>

<img src="/assets/info.png" title='This is the maximum number of reviews a reviewer can perform'<%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
</span>

</td>
<td width="50%">

<%= label_tag('assignment_form[assignment][num_metareviews_required]', 'Set required number of meta-reviews per reviewer') %>
<%= text_field_tag('assignment_form[assignment][num_metareviews_required]', @assignment_form.assignment.num_metareviews_required ||= @assignment_form.assignment.num_metareviews_allowed, size: 1) %>
<br/>
<br/>

<%= check_box_tag('meta_review', 'true', true, {:onChange => 'hasMetaReviewChanged()'}) %>
<%= label_tag('meta_review', 'Has meta-review limit?') %>&emsp;&emsp;
<span id="meta_reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_metareviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_metareviews_allowed]', ' Set allowed number of meta-reviews per reviewer') %>
<%if @assignment_form.assignment.num_metareviews_allowed && @assignment_form.assignment.num_metareviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_metareviews_allowed]', @assignment_form.assignment.num_metareviews_allowed ||= 3, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_metareviews_allowed]', 3, size: 1) %>
<%end%>
</span>

</td>
</tr>

</pre>
[[File:uifixmodi.PNG]]

* '''Issue #965''': The assignment of work to review to a reviewer depends on the number of reviews it has previously received, if that work has been assigned to a reviewer previously and its review has not been submitted then the system will still consider it such that it would not assign the same work to another reviewer if it has enough number of reviewers previously assigned to review it. This might lead to a scenario where that work was assigned to enough number of reviewers, but it does not receive enough reviews and cannot be assigned to any new reviewers either.
*'''Solution''': This implementation has been fixed. The system now considers only the reviews that were submitted by the reviewers for that work so that every submission gets assigned to reviewers until it receives its allowed number of reviews.

The modified file is app/models/review_assignment.rb and the modification is done in the method reject_by_max_reviews_per_submission

<pre>
def reject_by_max_reviews_per_submission(contributor_set)
contributor_set.reject! {|contributor| contributor.responses.reject {|response| !response.is_submitted }.count >= max_reviews_per_submission }
contributor_set
end
</pre>

==Modified Files==
* app/views/assignments/edit/_review_strategy.html.erb
* app/models/review_assignment.rb
* app/views/student_review/list.html.erb

===References===

#[https://github.com/expertiza/expertiza Expertiza on GitHub]
#[https://github.com/WintersLt/expertiza GitHub Project Repository Fork]
#[http://expertiza.ncsu.edu/ The live Expertiza website]
#[http://wikis.lib.ncsu.edu/index.php/Expertiza Expertiza project documentation wiki]
#Clean Code: A handbook of agile software craftsmanship. Author: Robert C Martin

File:Morecrap.png

2017-10-30T20:51:39Z

Ssuresh6: dvfbgnhm

dvfbgnhm

CSC/ECE 517 Fall 2017/E1777 Coherent specification of review requirements.rb

2017-10-30T20:50:56Z

Ssuresh6: /* Implementation */

==E1777. OSS Project Green: Coherent specification of review requirements==

This page provides a description of the Expertiza based OSS project.

__TOC__

===About Expertiza===

[http://expertiza.ncsu.edu/ Expertiza] is an open source project based on [http://rubyonrails.org/ Ruby on Rails] framework. Expertiza allows the instructor to create new assignments and customize new or existing assignments. It also allows the instructor to create a list of topics the students can sign up for. Students can form teams in Expertiza to work on various projects and assignments. Students can also peer review other students' submissions. Expertiza supports submission across various document types, including the URLs and wiki pages.

===Problem Statement===
The following tasks were accomplished in this project:

* The punctuation, syntax and the capitalization of statements in the review strategy tab requires improvement.
* Information buttons to elaborate difference between the synonymous statements present in review strategy tab need to be included.
* Message to a reviewer regarding how many reviews they are required to and allowed to perform is not clear enough.
* Assignment of work to review for a new reviewer is based on the number of previous reviews assigned for that submission which is incorrect.

===Implementation===

=====Drawbacks and Solutions=====
'''Issue #402 ''': To allow reviewers a larger set of topics to choose from, the instructor can set the threshold (on the Review Strategy tab of assignment creation/editing) to some integer k > 0. Then any submission that has within k reviews of the fewest number can be chosen by a new reviewer. Let’s say that all submissions have at least 1 review. If k = 3, then the reviewer can choose any topic where there is a submission that has 4 or fewer reviews so far. Suppose that the minimum number of reviews for any submission is 2, but that I have reviewed all the submissions that have only 2 reviews. Then I’m not allowed to review at all (unless k > 0).
* '''Solution''': This issue has been fixed previously. The reviewer will get assigned a submission even if it has fulfilled the required number of reviews, to ensure that the reviewer always receives a new submission to review as long as it is not his own.
[[File:lesscrap.png]]
'''Issue #969''':
*a.The two statements "Set Allowed Number of Reviews per reviewer" and "Set Required Number of Reviews per reviewer." are not well differentiated. It can very confusing for an instructor to give values for them.

* '''Solution''': Information buttons have been provided beside these statements in the review strategy tab to make sure that the instructor knows the difference between the two.
This modification is made in the app/views/assignments/edit/_review_strategy.html.erb
<pre>

<img src="/assets/info.png" title='This is the minimum number of reviews a reviewer has to perform'>
<br/>
<br/>
<%= check_box_tag('review', 'true', true, {:onChange => 'hasReviewChanged()'}) %>
<%= label_tag('review', 'Has max review limit?') %>&emsp;&emsp;
<span id="reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_reviews_allowed]', ' Set allowed number of reviews per reviewer') %>
<%if @assignment_form.assignment.num_reviews_allowed && @assignment_form.assignment.num_reviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', @assignment_form.assignment.num_reviews_allowed, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', 3, size: 1) %>
<%end%>

<img src="/assets/info.png" title='This is the maximum number of reviews a reviewer can perform'<%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
</span>

</td>
</pre>

*b) When the number of allowed or required reviews is not set on the Review Strategy tab, the system does not have a message to display to a reviewer about how many submissions of work they are required to and allowed to review depending on the values set by the instructor in the "allowed number of reviewers per reviewer" field and the "Set Required Number of Reviews per reviewer." field.

'''Solution''': This view has been implemented in the app/views/student_review/list.html.erb
<pre>

<h2>Reviews for "<%= @assignment.name %>"</h2>


<% if @assignment.num_reviews_allowed.nil? || @assignment.num_reviews_allowed == -1%>
<h4>Your instructor expects you to do <%= @assignment.num_reviews_required %> reviews. You are not allowed to do any extra reviews. </h4>

<% elsif @assignment.num_reviews_allowed == @assignment.num_reviews_required %>
<h4>You should perform exactly <%= @assignment.num_reviews_allowed %> reviews </h4>
<% else %>

<h4>You may perform between <%= @assignment.num_reviews_required %> and <%= @assignment.num_reviews_allowed %> reviews</h4>
<% end %>
</pre>

*c) The capitalization and punctuation of statements such as "Set Allowed Number of Reviews per reviewer" and "Set Required Number of Reviews per reviewer." in review strategy tab are incorrect.
*'''Solution''': This view has been fixed by making changes in the respective files.
The changes were made in the file : app/views/assignments/edit/_review_strategy.html.erb
<pre>
<%= label_tag('assignment_form[assignment][num_reviews_required]', 'Set required number of reviews per reviewer') %>
<%= text_field_tag('assignment_form[assignment][num_reviews_required]', @assignment_form.assignment.num_reviews_required ||= @assignment_form.assignment.num_reviews, size: 1) %>


<img src="/assets/info.png" title='This is the minimum number of reviews a reviewer has to perform'>
<br/>
<br/>
<%= check_box_tag('review', 'true', true, {:onChange => 'hasReviewChanged()'}) %>
<%= label_tag('review', 'Has max review limit?') %>&emsp;&emsp;
<span id="reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_reviews_allowed]', ' Set allowed number of reviews per reviewer') %>
<%if @assignment_form.assignment.num_reviews_allowed && @assignment_form.assignment.num_reviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', @assignment_form.assignment.num_reviews_allowed, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', 3, size: 1) %>
<%end%>

<img src="/assets/info.png" title='This is the maximum number of reviews a reviewer can perform'<%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
</span>

</td>
<td width="50%">

<%= label_tag('assignment_form[assignment][num_metareviews_required]', 'Set required number of meta-reviews per reviewer') %>
<%= text_field_tag('assignment_form[assignment][num_metareviews_required]', @assignment_form.assignment.num_metareviews_required ||= @assignment_form.assignment.num_metareviews_allowed, size: 1) %>
<br/>
<br/>

<%= check_box_tag('meta_review', 'true', true, {:onChange => 'hasMetaReviewChanged()'}) %>
<%= label_tag('meta_review', 'Has meta-review limit?') %>&emsp;&emsp;
<span id="meta_reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_metareviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_metareviews_allowed]', ' Set allowed number of meta-reviews per reviewer') %>
<%if @assignment_form.assignment.num_metareviews_allowed && @assignment_form.assignment.num_metareviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_metareviews_allowed]', @assignment_form.assignment.num_metareviews_allowed ||= 3, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_metareviews_allowed]', 3, size: 1) %>
<%end%>
</span>

</td>
</tr>

</pre>
[[File:uifixmodi.PNG]]

* '''Issue #965''': The assignment of work to review to a reviewer depends on the number of reviews it has previously received, if that work has been assigned to a reviewer previously and its review has not been submitted then the system will still consider it such that it would not assign the same work to another reviewer if it has enough number of reviewers previously assigned to review it. This might lead to a scenario where that work was assigned to enough number of reviewers, but it does not receive enough reviews and cannot be assigned to any new reviewers either.
*'''Solution''': This implementation has been fixed. The system now considers only the reviews that were submitted by the reviewers for that work so that every submission gets assigned to reviewers until it receives its allowed number of reviews.

The modified file is app/models/review_assignment.rb and the modification is done in the method reject_by_max_reviews_per_submission

<pre>
def reject_by_max_reviews_per_submission(contributor_set)
contributor_set.reject! {|contributor| contributor.responses.reject {|response| !response.is_submitted }.count >= max_reviews_per_submission }
contributor_set
end
</pre>

==Modified Files==
* app/views/assignments/edit/_review_strategy.html.erb
* app/models/review_assignment.rb
* app/views/student_review/list.html.erb

===References===

#[https://github.com/expertiza/expertiza Expertiza on GitHub]
#[https://github.com/WintersLt/expertiza GitHub Project Repository Fork]
#[http://expertiza.ncsu.edu/ The live Expertiza website]
#[http://wikis.lib.ncsu.edu/index.php/Expertiza Expertiza project documentation wiki]
#Clean Code: A handbook of agile software craftsmanship. Author: Robert C Martin

CSC/ECE 517 Fall 2017/E1777 Coherent specification of review requirements.rb

2017-10-30T20:50:33Z

Ssuresh6: /* Drawbacks and Solutions */

==E1777. OSS Project Green: Coherent specification of review requirements==

This page provides a description of the Expertiza based OSS project.

__TOC__

===About Expertiza===

[http://expertiza.ncsu.edu/ Expertiza] is an open source project based on [http://rubyonrails.org/ Ruby on Rails] framework. Expertiza allows the instructor to create new assignments and customize new or existing assignments. It also allows the instructor to create a list of topics the students can sign up for. Students can form teams in Expertiza to work on various projects and assignments. Students can also peer review other students' submissions. Expertiza supports submission across various document types, including the URLs and wiki pages.

===Problem Statement===
The following tasks were accomplished in this project:

* The punctuation, syntax and the capitalization of statements in the review strategy tab requires improvement.
* Information buttons to elaborate difference between the synonymous statements present in review strategy tab need to be included.
* Message to a reviewer regarding how many reviews they are required to and allowed to perform is not clear enough.
* Assignment of work to review for a new reviewer is based on the number of previous reviews assigned for that submission which is incorrect.

===Implementation===

=====Drawbacks and Solutions=====
'''Issue #402 ''': To allow reviewers a larger set of topics to choose from, the instructor can set the threshold (on the Review Strategy tab of assignment creation/editing) to some integer k > 0. Then any submission that has within k reviews of the fewest number can be chosen by a new reviewer. Let’s say that all submissions have at least 1 review. If k = 3, then the reviewer can choose any topic where there is a submission that has 4 or fewer reviews so far. Suppose that the minimum number of reviews for any submission is 2, but that I have reviewed all the submissions that have only 2 reviews. Then I’m not allowed to review at all (unless k > 0).
* '''Solution''': This issue has been fixed previously. The reviewer will get assigned a submission even if it has fulfilled the required number of reviews, to ensure that the reviewer always receives a new submission to review as long as it is not his own.
'''Issue #969''':
*a.The two statements "Set Allowed Number of Reviews per reviewer" and "Set Required Number of Reviews per reviewer." are not well differentiated. It can very confusing for an instructor to give values for them.

* '''Solution''': Information buttons have been provided beside these statements in the review strategy tab to make sure that the instructor knows the difference between the two.
This modification is made in the app/views/assignments/edit/_review_strategy.html.erb
<pre>

<img src="/assets/info.png" title='This is the minimum number of reviews a reviewer has to perform'>
<br/>
<br/>
<%= check_box_tag('review', 'true', true, {:onChange => 'hasReviewChanged()'}) %>
<%= label_tag('review', 'Has max review limit?') %>&emsp;&emsp;
<span id="reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_reviews_allowed]', ' Set allowed number of reviews per reviewer') %>
<%if @assignment_form.assignment.num_reviews_allowed && @assignment_form.assignment.num_reviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', @assignment_form.assignment.num_reviews_allowed, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', 3, size: 1) %>
<%end%>

<img src="/assets/info.png" title='This is the maximum number of reviews a reviewer can perform'<%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
</span>

</td>
</pre>

*b) When the number of allowed or required reviews is not set on the Review Strategy tab, the system does not have a message to display to a reviewer about how many submissions of work they are required to and allowed to review depending on the values set by the instructor in the "allowed number of reviewers per reviewer" field and the "Set Required Number of Reviews per reviewer." field.

'''Solution''': This view has been implemented in the app/views/student_review/list.html.erb
<pre>

<h2>Reviews for "<%= @assignment.name %>"</h2>


<% if @assignment.num_reviews_allowed.nil? || @assignment.num_reviews_allowed == -1%>
<h4>Your instructor expects you to do <%= @assignment.num_reviews_required %> reviews. You are not allowed to do any extra reviews. </h4>

<% elsif @assignment.num_reviews_allowed == @assignment.num_reviews_required %>
<h4>You should perform exactly <%= @assignment.num_reviews_allowed %> reviews </h4>
<% else %>

<h4>You may perform between <%= @assignment.num_reviews_required %> and <%= @assignment.num_reviews_allowed %> reviews</h4>
<% end %>
</pre>

*c) The capitalization and punctuation of statements such as "Set Allowed Number of Reviews per reviewer" and "Set Required Number of Reviews per reviewer." in review strategy tab are incorrect.
*'''Solution''': This view has been fixed by making changes in the respective files.
The changes were made in the file : app/views/assignments/edit/_review_strategy.html.erb
<pre>
<%= label_tag('assignment_form[assignment][num_reviews_required]', 'Set required number of reviews per reviewer') %>
<%= text_field_tag('assignment_form[assignment][num_reviews_required]', @assignment_form.assignment.num_reviews_required ||= @assignment_form.assignment.num_reviews, size: 1) %>


<img src="/assets/info.png" title='This is the minimum number of reviews a reviewer has to perform'>
<br/>
<br/>
<%= check_box_tag('review', 'true', true, {:onChange => 'hasReviewChanged()'}) %>
<%= label_tag('review', 'Has max review limit?') %>&emsp;&emsp;
<span id="reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_reviews_allowed]', ' Set allowed number of reviews per reviewer') %>
<%if @assignment_form.assignment.num_reviews_allowed && @assignment_form.assignment.num_reviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', @assignment_form.assignment.num_reviews_allowed, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', 3, size: 1) %>
<%end%>

<img src="/assets/info.png" title='This is the maximum number of reviews a reviewer can perform'<%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
</span>

</td>
<td width="50%">

<%= label_tag('assignment_form[assignment][num_metareviews_required]', 'Set required number of meta-reviews per reviewer') %>
<%= text_field_tag('assignment_form[assignment][num_metareviews_required]', @assignment_form.assignment.num_metareviews_required ||= @assignment_form.assignment.num_metareviews_allowed, size: 1) %>
<br/>
<br/>

<%= check_box_tag('meta_review', 'true', true, {:onChange => 'hasMetaReviewChanged()'}) %>
<%= label_tag('meta_review', 'Has meta-review limit?') %>&emsp;&emsp;
<span id="meta_reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_metareviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_metareviews_allowed]', ' Set allowed number of meta-reviews per reviewer') %>
<%if @assignment_form.assignment.num_metareviews_allowed && @assignment_form.assignment.num_metareviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_metareviews_allowed]', @assignment_form.assignment.num_metareviews_allowed ||= 3, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_metareviews_allowed]', 3, size: 1) %>
<%end%>
</span>

</td>
</tr>

</pre>
[[File:uifixmodi.PNG]]

* '''Issue #965''': The assignment of work to review to a reviewer depends on the number of reviews it has previously received, if that work has been assigned to a reviewer previously and its review has not been submitted then the system will still consider it such that it would not assign the same work to another reviewer if it has enough number of reviewers previously assigned to review it. This might lead to a scenario where that work was assigned to enough number of reviewers, but it does not receive enough reviews and cannot be assigned to any new reviewers either.
*'''Solution''': This implementation has been fixed. The system now considers only the reviews that were submitted by the reviewers for that work so that every submission gets assigned to reviewers until it receives its allowed number of reviews.

The modified file is app/models/review_assignment.rb and the modification is done in the method reject_by_max_reviews_per_submission

<pre>
def reject_by_max_reviews_per_submission(contributor_set)
contributor_set.reject! {|contributor| contributor.responses.reject {|response| !response.is_submitted }.count >= max_reviews_per_submission }
contributor_set
end
</pre>

==Modified Files==
* app/views/assignments/edit/_review_strategy.html.erb
* app/models/review_assignment.rb
* app/views/student_review/list.html.erb

===References===

#[https://github.com/expertiza/expertiza Expertiza on GitHub]
#[https://github.com/WintersLt/expertiza GitHub Project Repository Fork]
#[http://expertiza.ncsu.edu/ The live Expertiza website]
#[http://wikis.lib.ncsu.edu/index.php/Expertiza Expertiza project documentation wiki]
#Clean Code: A handbook of agile software craftsmanship. Author: Robert C Martin

CSC/ECE 517 Fall 2017/E1777 Coherent specification of review requirements.rb

2017-10-30T20:50:16Z

Ssuresh6: /* Implementation */

==E1777. OSS Project Green: Coherent specification of review requirements==

This page provides a description of the Expertiza based OSS project.

__TOC__

===About Expertiza===

[http://expertiza.ncsu.edu/ Expertiza] is an open source project based on [http://rubyonrails.org/ Ruby on Rails] framework. Expertiza allows the instructor to create new assignments and customize new or existing assignments. It also allows the instructor to create a list of topics the students can sign up for. Students can form teams in Expertiza to work on various projects and assignments. Students can also peer review other students' submissions. Expertiza supports submission across various document types, including the URLs and wiki pages.

===Problem Statement===
The following tasks were accomplished in this project:

* The punctuation, syntax and the capitalization of statements in the review strategy tab requires improvement.
* Information buttons to elaborate difference between the synonymous statements present in review strategy tab need to be included.
* Message to a reviewer regarding how many reviews they are required to and allowed to perform is not clear enough.
* Assignment of work to review for a new reviewer is based on the number of previous reviews assigned for that submission which is incorrect.

===Implementation===

=====Drawbacks and Solutions=====
'''Issue #402 ''': To allow reviewers a larger set of topics to choose from, the instructor can set the threshold (on the Review Strategy tab of assignment creation/editing) to some integer k > 0. Then any submission that has within k reviews of the fewest number can be chosen by a new reviewer. Let’s say that all submissions have at least 1 review. If k = 3, then the reviewer can choose any topic where there is a submission that has 4 or fewer reviews so far. Suppose that the minimum number of reviews for any submission is 2, but that I have reviewed all the submissions that have only 2 reviews. Then I’m not allowed to review at all (unless k > 0).
* '''Solution''': This issue has been fixed previously. The reviewer will get assigned a submission even if it has fulfilled the required number of reviews, to ensure that the reviewer always receives a new submission to review as long as it is not his own.
'''Issue #969''':
*a.The two statements "Set Allowed Number of Reviews per reviewer" and "Set Required Number of Reviews per reviewer." are not well differentiated. It can very confusing for an instructor to give values for them.

* '''Solution''': Information buttons have been provided beside these statements in the review strategy tab to make sure that the instructor knows the difference between the two.
This modification is made in the app/views/assignments/edit/_review_strategy.html.erb
<pre>

<img src="/assets/info.png" title='This is the minimum number of reviews a reviewer has to perform'>
<br/>
<br/>
<%= check_box_tag('review', 'true', true, {:onChange => 'hasReviewChanged()'}) %>
<%= label_tag('review', 'Has max review limit?') %>&emsp;&emsp;
<span id="reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_reviews_allowed]', ' Set allowed number of reviews per reviewer') %>
<%if @assignment_form.assignment.num_reviews_allowed && @assignment_form.assignment.num_reviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', @assignment_form.assignment.num_reviews_allowed, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', 3, size: 1) %>
<%end%>

<img src="/assets/info.png" title='This is the maximum number of reviews a reviewer can perform'<%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
</span>

</td>
</pre>

*b) When the number of allowed or required reviews is not set on the Review Strategy tab, the system does not have a message to display to a reviewer about how many submissions of work they are required to and allowed to review depending on the values set by the instructor in the "allowed number of reviewers per reviewer" field and the "Set Required Number of Reviews per reviewer." field.

'''Solution''': This view has been implemented in the app/views/student_review/list.html.erb
<pre>

<h2>Reviews for "<%= @assignment.name %>"</h2>


<% if @assignment.num_reviews_allowed.nil? || @assignment.num_reviews_allowed == -1%>
<h4>Your instructor expects you to do <%= @assignment.num_reviews_required %> reviews. You are not allowed to do any extra reviews. </h4>

<% elsif @assignment.num_reviews_allowed == @assignment.num_reviews_required %>
<h4>You should perform exactly <%= @assignment.num_reviews_allowed %> reviews </h4>
<% else %>

<h4>You may perform between <%= @assignment.num_reviews_required %> and <%= @assignment.num_reviews_allowed %> reviews</h4>
<% end %>
</pre>

*c) The capitalization and punctuation of statements such as "Set Allowed Number of Reviews per reviewer" and "Set Required Number of Reviews per reviewer." in review strategy tab are incorrect.
*'''Solution''': This view has been fixed by making changes in the respective files.
The changes were made in the file : app/views/assignments/edit/_review_strategy.html.erb
<pre>
<%= label_tag('assignment_form[assignment][num_reviews_required]', 'Set required number of reviews per reviewer') %>
<%= text_field_tag('assignment_form[assignment][num_reviews_required]', @assignment_form.assignment.num_reviews_required ||= @assignment_form.assignment.num_reviews, size: 1) %>


<img src="/assets/info.png" title='This is the minimum number of reviews a reviewer has to perform'>
<br/>
<br/>
<%= check_box_tag('review', 'true', true, {:onChange => 'hasReviewChanged()'}) %>
<%= label_tag('review', 'Has max review limit?') %>&emsp;&emsp;
<span id="reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_reviews_allowed]', ' Set allowed number of reviews per reviewer') %>
<%if @assignment_form.assignment.num_reviews_allowed && @assignment_form.assignment.num_reviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', @assignment_form.assignment.num_reviews_allowed, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', 3, size: 1) %>
<%end%>

<img src="/assets/info.png" title='This is the maximum number of reviews a reviewer can perform'<%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
</span>

</td>
<td width="50%">

<%= label_tag('assignment_form[assignment][num_metareviews_required]', 'Set required number of meta-reviews per reviewer') %>
<%= text_field_tag('assignment_form[assignment][num_metareviews_required]', @assignment_form.assignment.num_metareviews_required ||= @assignment_form.assignment.num_metareviews_allowed, size: 1) %>
<br/>
<br/>

<%= check_box_tag('meta_review', 'true', true, {:onChange => 'hasMetaReviewChanged()'}) %>
<%= label_tag('meta_review', 'Has meta-review limit?') %>&emsp;&emsp;
<span id="meta_reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_metareviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_metareviews_allowed]', ' Set allowed number of meta-reviews per reviewer') %>
<%if @assignment_form.assignment.num_metareviews_allowed && @assignment_form.assignment.num_metareviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_metareviews_allowed]', @assignment_form.assignment.num_metareviews_allowed ||= 3, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_metareviews_allowed]', 3, size: 1) %>
<%end%>
</span>

</td>
</tr>

</pre>
[[File:uifixmodi.PNG]]

* '''Issue #965''': The assignment of work to review to a reviewer depends on the number of reviews it has previously received, if that work has been assigned to a reviewer previously and its review has not been submitted then the system will still consider it such that it would not assign the same work to another reviewer if it has enough number of reviewers previously assigned to review it. This might lead to a scenario where that work was assigned to enough number of reviewers, but it does not receive enough reviews and cannot be assigned to any new reviewers either.
*'''Solution''': This implementation has been fixed. The system now considers only the reviews that were submitted by the reviewers for that work so that every submission gets assigned to reviewers until it receives its allowed number of reviews.

The modified file is app/models/review_assignment.rb and the modification is done in the method reject_by_max_reviews_per_submission

<pre>
def reject_by_max_reviews_per_submission(contributor_set)
contributor_set.reject! {|contributor| contributor.responses.reject {|response| !response.is_submitted }.count >= max_reviews_per_submission }
contributor_set
end
</pre>

==Modified Files==
* app/views/assignments/edit/_review_strategy.html.erb
* app/models/review_assignment.rb
* app/views/student_review/list.html.erb

===References===

#[https://github.com/expertiza/expertiza Expertiza on GitHub]
#[https://github.com/WintersLt/expertiza GitHub Project Repository Fork]
#[http://expertiza.ncsu.edu/ The live Expertiza website]
#[http://wikis.lib.ncsu.edu/index.php/Expertiza Expertiza project documentation wiki]
#Clean Code: A handbook of agile software craftsmanship. Author: Robert C Martin

CSC/ECE 517 Fall 2017/E1777 Coherent specification of review requirements.rb

2017-10-30T20:49:30Z

Ssuresh6: /* Implementation */

==E1777. OSS Project Green: Coherent specification of review requirements==

This page provides a description of the Expertiza based OSS project.

__TOC__

===About Expertiza===

[http://expertiza.ncsu.edu/ Expertiza] is an open source project based on [http://rubyonrails.org/ Ruby on Rails] framework. Expertiza allows the instructor to create new assignments and customize new or existing assignments. It also allows the instructor to create a list of topics the students can sign up for. Students can form teams in Expertiza to work on various projects and assignments. Students can also peer review other students' submissions. Expertiza supports submission across various document types, including the URLs and wiki pages.

===Problem Statement===
The following tasks were accomplished in this project:

* The punctuation, syntax and the capitalization of statements in the review strategy tab requires improvement.
* Information buttons to elaborate difference between the synonymous statements present in review strategy tab need to be included.
* Message to a reviewer regarding how many reviews they are required to and allowed to perform is not clear enough.
* Assignment of work to review for a new reviewer is based on the number of previous reviews assigned for that submission which is incorrect.

===Implementation===

=====Drawbacks and Solutions=====
'''Issue #402 ''': To allow reviewers a larger set of topics to choose from, the instructor can set the threshold (on the Review Strategy tab of assignment creation/editing) to some integer k > 0. Then any submission that has within k reviews of the fewest number can be chosen by a new reviewer. Let’s say that all submissions have at least 1 review. If k = 3, then the reviewer can choose any topic where there is a submission that has 4 or fewer reviews so far. Suppose that the minimum number of reviews for any submission is 2, but that I have reviewed all the submissions that have only 2 reviews. Then I’m not allowed to review at all (unless k > 0).
* '''Solution''': This issue has been fixed previously. The reviewer will get assigned a submission even if it has fulfilled the required number of reviews, to ensure that the reviewer always receives a new submission to review as long as it is not his own.
'''Issue #969''':
*a.The two statements "Set Allowed Number of Reviews per reviewer" and "Set Required Number of Reviews per reviewer." are not well differentiated. It can very confusing for an instructor to give values for them.

* '''Solution''': Information buttons have been provided beside these statements in the review strategy tab to make sure that the instructor knows the difference between the two.
This modification is made in the app/views/assignments/edit/_review_strategy.html.erb
<pre>

<img src="/assets/info.png" title='This is the minimum number of reviews a reviewer has to perform'>
<br/>
<br/>
<%= check_box_tag('review', 'true', true, {:onChange => 'hasReviewChanged()'}) %>
<%= label_tag('review', 'Has max review limit?') %>&emsp;&emsp;
<span id="reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_reviews_allowed]', ' Set allowed number of reviews per reviewer') %>
<%if @assignment_form.assignment.num_reviews_allowed && @assignment_form.assignment.num_reviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', @assignment_form.assignment.num_reviews_allowed, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', 3, size: 1) %>
<%end%>

<img src="/assets/info.png" title='This is the maximum number of reviews a reviewer can perform'<%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
</span>

</td>
</pre>

[[File:morecrap.png]]
*b) When the number of allowed or required reviews is not set on the Review Strategy tab, the system does not have a message to display to a reviewer about how many submissions of work they are required to and allowed to review depending on the values set by the instructor in the "allowed number of reviewers per reviewer" field and the "Set Required Number of Reviews per reviewer." field.

'''Solution''': This view has been implemented in the app/views/student_review/list.html.erb
<pre>

<h2>Reviews for "<%= @assignment.name %>"</h2>


<% if @assignment.num_reviews_allowed.nil? || @assignment.num_reviews_allowed == -1%>
<h4>Your instructor expects you to do <%= @assignment.num_reviews_required %> reviews. You are not allowed to do any extra reviews. </h4>

<% elsif @assignment.num_reviews_allowed == @assignment.num_reviews_required %>
<h4>You should perform exactly <%= @assignment.num_reviews_allowed %> reviews </h4>
<% else %>

<h4>You may perform between <%= @assignment.num_reviews_required %> and <%= @assignment.num_reviews_allowed %> reviews</h4>
<% end %>
</pre>

*c) The capitalization and punctuation of statements such as "Set Allowed Number of Reviews per reviewer" and "Set Required Number of Reviews per reviewer." in review strategy tab are incorrect.
*'''Solution''': This view has been fixed by making changes in the respective files.
The changes were made in the file : app/views/assignments/edit/_review_strategy.html.erb
<pre>
<%= label_tag('assignment_form[assignment][num_reviews_required]', 'Set required number of reviews per reviewer') %>
<%= text_field_tag('assignment_form[assignment][num_reviews_required]', @assignment_form.assignment.num_reviews_required ||= @assignment_form.assignment.num_reviews, size: 1) %>


<img src="/assets/info.png" title='This is the minimum number of reviews a reviewer has to perform'>
<br/>
<br/>
<%= check_box_tag('review', 'true', true, {:onChange => 'hasReviewChanged()'}) %>
<%= label_tag('review', 'Has max review limit?') %>&emsp;&emsp;
<span id="reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_reviews_allowed]', ' Set allowed number of reviews per reviewer') %>
<%if @assignment_form.assignment.num_reviews_allowed && @assignment_form.assignment.num_reviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', @assignment_form.assignment.num_reviews_allowed, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', 3, size: 1) %>
<%end%>

<img src="/assets/info.png" title='This is the maximum number of reviews a reviewer can perform'<%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
</span>

</td>
<td width="50%">

<%= label_tag('assignment_form[assignment][num_metareviews_required]', 'Set required number of meta-reviews per reviewer') %>
<%= text_field_tag('assignment_form[assignment][num_metareviews_required]', @assignment_form.assignment.num_metareviews_required ||= @assignment_form.assignment.num_metareviews_allowed, size: 1) %>
<br/>
<br/>

<%= check_box_tag('meta_review', 'true', true, {:onChange => 'hasMetaReviewChanged()'}) %>
<%= label_tag('meta_review', 'Has meta-review limit?') %>&emsp;&emsp;
<span id="meta_reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_metareviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_metareviews_allowed]', ' Set allowed number of meta-reviews per reviewer') %>
<%if @assignment_form.assignment.num_metareviews_allowed && @assignment_form.assignment.num_metareviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_metareviews_allowed]', @assignment_form.assignment.num_metareviews_allowed ||= 3, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_metareviews_allowed]', 3, size: 1) %>
<%end%>
</span>

</td>
</tr>

</pre>
[[File:uifixmodi.PNG]]

* '''Issue #965''': The assignment of work to review to a reviewer depends on the number of reviews it has previously received, if that work has been assigned to a reviewer previously and its review has not been submitted then the system will still consider it such that it would not assign the same work to another reviewer if it has enough number of reviewers previously assigned to review it. This might lead to a scenario where that work was assigned to enough number of reviewers, but it does not receive enough reviews and cannot be assigned to any new reviewers either.
*'''Solution''': This implementation has been fixed. The system now considers only the reviews that were submitted by the reviewers for that work so that every submission gets assigned to reviewers until it receives its allowed number of reviews.

The modified file is app/models/review_assignment.rb and the modification is done in the method reject_by_max_reviews_per_submission

<pre>
def reject_by_max_reviews_per_submission(contributor_set)
contributor_set.reject! {|contributor| contributor.responses.reject {|response| !response.is_submitted }.count >= max_reviews_per_submission }
contributor_set
end
</pre>

==Modified Files==
* app/views/assignments/edit/_review_strategy.html.erb
* app/models/review_assignment.rb
* app/views/student_review/list.html.erb

===References===

#[https://github.com/expertiza/expertiza Expertiza on GitHub]
#[https://github.com/WintersLt/expertiza GitHub Project Repository Fork]
#[http://expertiza.ncsu.edu/ The live Expertiza website]
#[http://wikis.lib.ncsu.edu/index.php/Expertiza Expertiza project documentation wiki]
#Clean Code: A handbook of agile software craftsmanship. Author: Robert C Martin

CSC/ECE 517 Fall 2017/E1777 Coherent specification of review requirements.rb

2017-10-30T20:44:23Z

Ssuresh6: /* Drawbacks and Solutions */

==E1777. OSS Project Green: Coherent specification of review requirements==

This page provides a description of the Expertiza based OSS project.

__TOC__

===About Expertiza===

[http://expertiza.ncsu.edu/ Expertiza] is an open source project based on [http://rubyonrails.org/ Ruby on Rails] framework. Expertiza allows the instructor to create new assignments and customize new or existing assignments. It also allows the instructor to create a list of topics the students can sign up for. Students can form teams in Expertiza to work on various projects and assignments. Students can also peer review other students' submissions. Expertiza supports submission across various document types, including the URLs and wiki pages.

===Problem Statement===
The following tasks were accomplished in this project:

* The punctuation, syntax and the capitalization of statements in the review strategy tab requires improvement.
* Information buttons to elaborate difference between the synonymous statements present in review strategy tab need to be included.
* Message to a reviewer regarding how many reviews they are required to and allowed to perform is not clear enough.
* Assignment of work to review for a new reviewer is based on the number of previous reviews assigned for that submission which is incorrect.

===Implementation===

=====Drawbacks and Solutions=====
'''Issue #402 ''': To allow reviewers a larger set of topics to choose from, the instructor can set the threshold (on the Review Strategy tab of assignment creation/editing) to some integer k > 0. Then any submission that has within k reviews of the fewest number can be chosen by a new reviewer. Let’s say that all submissions have at least 1 review. If k = 3, then the reviewer can choose any topic where there is a submission that has 4 or fewer reviews so far. Suppose that the minimum number of reviews for any submission is 2, but that I have reviewed all the submissions that have only 2 reviews. Then I’m not allowed to review at all (unless k > 0).
* '''Solution''': This issue has been fixed previously. The reviewer will get assigned a submission even if it has fulfilled the required number of reviews, to ensure that the reviewer always receives a new submission to review as long as it is not his own.
'''Issue #969''':
*a.The two statements "Set Allowed Number of Reviews per reviewer" and "Set Required Number of Reviews per reviewer." are not well differentiated. It can very confusing for an instructor to give values for them.

* '''Solution''': Information buttons have been provided beside these statements in the review strategy tab to make sure that the instructor knows the difference between the two.
This modification is made in the app/views/assignments/edit/_review_strategy.html.erb
<pre>

<img src="/assets/info.png" title='This is the minimum number of reviews a reviewer has to perform'>
<br/>
<br/>
<%= check_box_tag('review', 'true', true, {:onChange => 'hasReviewChanged()'}) %>
<%= label_tag('review', 'Has max review limit?') %>&emsp;&emsp;
<span id="reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_reviews_allowed]', ' Set allowed number of reviews per reviewer') %>
<%if @assignment_form.assignment.num_reviews_allowed && @assignment_form.assignment.num_reviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', @assignment_form.assignment.num_reviews_allowed, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', 3, size: 1) %>
<%end%>

<img src="/assets/info.png" title='This is the maximum number of reviews a reviewer can perform'<%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
</span>

</td>
</pre>

[[File:lesscrap.png]]

[[File:morecrap.png]]
*b) When the number of allowed or required reviews is not set on the Review Strategy tab, the system does not have a message to display to a reviewer about how many submissions of work they are required to and allowed to review depending on the values set by the instructor in the "allowed number of reviewers per reviewer" field and the "Set Required Number of Reviews per reviewer." field.

'''Solution''': This view has been implemented in the app/views/student_review/list.html.erb
<pre>

<h2>Reviews for "<%= @assignment.name %>"</h2>


<% if @assignment.num_reviews_allowed.nil? || @assignment.num_reviews_allowed == -1%>
<h4>Your instructor expects you to do <%= @assignment.num_reviews_required %> reviews. You are not allowed to do any extra reviews. </h4>

<% elsif @assignment.num_reviews_allowed == @assignment.num_reviews_required %>
<h4>You should perform exactly <%= @assignment.num_reviews_allowed %> reviews </h4>
<% else %>

<h4>You may perform between <%= @assignment.num_reviews_required %> and <%= @assignment.num_reviews_allowed %> reviews</h4>
<% end %>
</pre>

*c) The capitalization and punctuation of statements such as "Set Allowed Number of Reviews per reviewer" and "Set Required Number of Reviews per reviewer." in review strategy tab are incorrect.
*'''Solution''': This view has been fixed by making changes in the respective files.
The changes were made in the file : app/views/assignments/edit/_review_strategy.html.erb
<pre>
<%= label_tag('assignment_form[assignment][num_reviews_required]', 'Set required number of reviews per reviewer') %>
<%= text_field_tag('assignment_form[assignment][num_reviews_required]', @assignment_form.assignment.num_reviews_required ||= @assignment_form.assignment.num_reviews, size: 1) %>


<img src="/assets/info.png" title='This is the minimum number of reviews a reviewer has to perform'>
<br/>
<br/>
<%= check_box_tag('review', 'true', true, {:onChange => 'hasReviewChanged()'}) %>
<%= label_tag('review', 'Has max review limit?') %>&emsp;&emsp;
<span id="reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_reviews_allowed]', ' Set allowed number of reviews per reviewer') %>
<%if @assignment_form.assignment.num_reviews_allowed && @assignment_form.assignment.num_reviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', @assignment_form.assignment.num_reviews_allowed, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', 3, size: 1) %>
<%end%>

<img src="/assets/info.png" title='This is the maximum number of reviews a reviewer can perform'<%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
</span>

</td>
<td width="50%">

<%= label_tag('assignment_form[assignment][num_metareviews_required]', 'Set required number of meta-reviews per reviewer') %>
<%= text_field_tag('assignment_form[assignment][num_metareviews_required]', @assignment_form.assignment.num_metareviews_required ||= @assignment_form.assignment.num_metareviews_allowed, size: 1) %>
<br/>
<br/>

<%= check_box_tag('meta_review', 'true', true, {:onChange => 'hasMetaReviewChanged()'}) %>
<%= label_tag('meta_review', 'Has meta-review limit?') %>&emsp;&emsp;
<span id="meta_reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_metareviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_metareviews_allowed]', ' Set allowed number of meta-reviews per reviewer') %>
<%if @assignment_form.assignment.num_metareviews_allowed && @assignment_form.assignment.num_metareviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_metareviews_allowed]', @assignment_form.assignment.num_metareviews_allowed ||= 3, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_metareviews_allowed]', 3, size: 1) %>
<%end%>
</span>

</td>
</tr>

</pre>
[[File:uifixmodi.PNG]]

* '''Issue #965''': The assignment of work to review to a reviewer depends on the number of reviews it has previously received, if that work has been assigned to a reviewer previously and its review has not been submitted then the system will still consider it such that it would not assign the same work to another reviewer if it has enough number of reviewers previously assigned to review it. This might lead to a scenario where that work was assigned to enough number of reviewers, but it does not receive enough reviews and cannot be assigned to any new reviewers either.
*'''Solution''': This implementation has been fixed. The system now considers only the reviews that were submitted by the reviewers for that work so that every submission gets assigned to reviewers until it receives its allowed number of reviews.

The modified file is app/models/review_assignment.rb and the modification is done in the method reject_by_max_reviews_per_submission

<pre>
def reject_by_max_reviews_per_submission(contributor_set)
contributor_set.reject! {|contributor| contributor.responses.reject {|response| !response.is_submitted }.count >= max_reviews_per_submission }
contributor_set
end
</pre>

==Modified Files==
* app/views/assignments/edit/_review_strategy.html.erb
* app/models/review_assignment.rb
* app/views/student_review/list.html.erb

===References===

#[https://github.com/expertiza/expertiza Expertiza on GitHub]
#[https://github.com/WintersLt/expertiza GitHub Project Repository Fork]
#[http://expertiza.ncsu.edu/ The live Expertiza website]
#[http://wikis.lib.ncsu.edu/index.php/Expertiza Expertiza project documentation wiki]
#Clean Code: A handbook of agile software craftsmanship. Author: Robert C Martin

CSC/ECE 517 Fall 2017/E1777 Coherent specification of review requirements.rb

2017-10-30T20:43:13Z

Ssuresh6: /* Implementation */

==E1777. OSS Project Green: Coherent specification of review requirements==

This page provides a description of the Expertiza based OSS project.

__TOC__

===About Expertiza===

[http://expertiza.ncsu.edu/ Expertiza] is an open source project based on [http://rubyonrails.org/ Ruby on Rails] framework. Expertiza allows the instructor to create new assignments and customize new or existing assignments. It also allows the instructor to create a list of topics the students can sign up for. Students can form teams in Expertiza to work on various projects and assignments. Students can also peer review other students' submissions. Expertiza supports submission across various document types, including the URLs and wiki pages.

===Problem Statement===
The following tasks were accomplished in this project:

* The punctuation, syntax and the capitalization of statements in the review strategy tab requires improvement.
* Information buttons to elaborate difference between the synonymous statements present in review strategy tab need to be included.
* Message to a reviewer regarding how many reviews they are required to and allowed to perform is not clear enough.
* Assignment of work to review for a new reviewer is based on the number of previous reviews assigned for that submission which is incorrect.

===Implementation===

=====Drawbacks and Solutions=====
'''Issue #402 ''': To allow reviewers a larger set of topics to choose from, the instructor can set the threshold (on the Review Strategy tab of assignment creation/editing) to some integer k > 0. Then any submission that has within k reviews of the fewest number can be chosen by a new reviewer. Let’s say that all submissions have at least 1 review. If k = 3, then the reviewer can choose any topic where there is a submission that has 4 or fewer reviews so far. Suppose that the minimum number of reviews for any submission is 2, but that I have reviewed all the submissions that have only 2 reviews. Then I’m not allowed to review at all (unless k > 0).
* '''Solution''': This issue has been fixed previously. The reviewer will get assigned a submission even if it has fulfilled the required number of reviews, to ensure that the reviewer always receives a new submission to review as long as it is not his own.
'''Issue #969''':
*a.The two statements "Set Allowed Number of Reviews per reviewer" and "Set Required Number of Reviews per reviewer." are not well differentiated. It can very confusing for an instructor to give values for them.

* '''Solution''': Information buttons have been provided beside these statements in the review strategy tab to make sure that the instructor knows the difference between the two.
This modification is made in the app/views/assignments/edit/_review_strategy.html.erb
<pre>

<img src="/assets/info.png" title='This is the minimum number of reviews a reviewer has to perform'>
<br/>
<br/>
<%= check_box_tag('review', 'true', true, {:onChange => 'hasReviewChanged()'}) %>
<%= label_tag('review', 'Has max review limit?') %>&emsp;&emsp;
<span id="reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_reviews_allowed]', ' Set allowed number of reviews per reviewer') %>
<%if @assignment_form.assignment.num_reviews_allowed && @assignment_form.assignment.num_reviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', @assignment_form.assignment.num_reviews_allowed, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', 3, size: 1) %>
<%end%>

<img src="/assets/info.png" title='This is the maximum number of reviews a reviewer can perform'<%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
</span>

</td>
</pre>

[[File:morecrap.png]]
[[File:lesscrap.png]]
*b) When the number of allowed or required reviews is not set on the Review Strategy tab, the system does not have a message to display to a reviewer about how many submissions of work they are required to and allowed to review depending on the values set by the instructor in the "allowed number of reviewers per reviewer" field and the "Set Required Number of Reviews per reviewer." field.

'''Solution''': This view has been implemented in the app/views/student_review/list.html.erb
<pre>

<h2>Reviews for "<%= @assignment.name %>"</h2>


<% if @assignment.num_reviews_allowed.nil? || @assignment.num_reviews_allowed == -1%>
<h4>Your instructor expects you to do <%= @assignment.num_reviews_required %> reviews. You are not allowed to do any extra reviews. </h4>

<% elsif @assignment.num_reviews_allowed == @assignment.num_reviews_required %>
<h4>You should perform exactly <%= @assignment.num_reviews_allowed %> reviews </h4>
<% else %>

<h4>You may perform between <%= @assignment.num_reviews_required %> and <%= @assignment.num_reviews_allowed %> reviews</h4>
<% end %>
</pre>

*c) The capitalization and punctuation of statements such as "Set Allowed Number of Reviews per reviewer" and "Set Required Number of Reviews per reviewer." in review strategy tab are incorrect.
*'''Solution''': This view has been fixed by making changes in the respective files.
The changes were made in the file : app/views/assignments/edit/_review_strategy.html.erb
<pre>
<%= label_tag('assignment_form[assignment][num_reviews_required]', 'Set required number of reviews per reviewer') %>
<%= text_field_tag('assignment_form[assignment][num_reviews_required]', @assignment_form.assignment.num_reviews_required ||= @assignment_form.assignment.num_reviews, size: 1) %>


<img src="/assets/info.png" title='This is the minimum number of reviews a reviewer has to perform'>
<br/>
<br/>
<%= check_box_tag('review', 'true', true, {:onChange => 'hasReviewChanged()'}) %>
<%= label_tag('review', 'Has max review limit?') %>&emsp;&emsp;
<span id="reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_reviews_allowed]', ' Set allowed number of reviews per reviewer') %>
<%if @assignment_form.assignment.num_reviews_allowed && @assignment_form.assignment.num_reviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', @assignment_form.assignment.num_reviews_allowed, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', 3, size: 1) %>
<%end%>

<img src="/assets/info.png" title='This is the maximum number of reviews a reviewer can perform'<%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
</span>

</td>
<td width="50%">

<%= label_tag('assignment_form[assignment][num_metareviews_required]', 'Set required number of meta-reviews per reviewer') %>
<%= text_field_tag('assignment_form[assignment][num_metareviews_required]', @assignment_form.assignment.num_metareviews_required ||= @assignment_form.assignment.num_metareviews_allowed, size: 1) %>
<br/>
<br/>

<%= check_box_tag('meta_review', 'true', true, {:onChange => 'hasMetaReviewChanged()'}) %>
<%= label_tag('meta_review', 'Has meta-review limit?') %>&emsp;&emsp;
<span id="meta_reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_metareviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_metareviews_allowed]', ' Set allowed number of meta-reviews per reviewer') %>
<%if @assignment_form.assignment.num_metareviews_allowed && @assignment_form.assignment.num_metareviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_metareviews_allowed]', @assignment_form.assignment.num_metareviews_allowed ||= 3, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_metareviews_allowed]', 3, size: 1) %>
<%end%>
</span>

</td>
</tr>

</pre>
[[File:uifixmodi.PNG]]

* '''Issue #965''': The assignment of work to review to a reviewer depends on the number of reviews it has previously received, if that work has been assigned to a reviewer previously and its review has not been submitted then the system will still consider it such that it would not assign the same work to another reviewer if it has enough number of reviewers previously assigned to review it. This might lead to a scenario where that work was assigned to enough number of reviewers, but it does not receive enough reviews and cannot be assigned to any new reviewers either.
*'''Solution''': This implementation has been fixed. The system now considers only the reviews that were submitted by the reviewers for that work so that every submission gets assigned to reviewers until it receives its allowed number of reviews.

The modified file is app/models/review_assignment.rb and the modification is done in the method reject_by_max_reviews_per_submission

<pre>
def reject_by_max_reviews_per_submission(contributor_set)
contributor_set.reject! {|contributor| contributor.responses.reject {|response| !response.is_submitted }.count >= max_reviews_per_submission }
contributor_set
end
</pre>

==Modified Files==
* app/views/assignments/edit/_review_strategy.html.erb
* app/models/review_assignment.rb
* app/views/student_review/list.html.erb

===References===

#[https://github.com/expertiza/expertiza Expertiza on GitHub]
#[https://github.com/WintersLt/expertiza GitHub Project Repository Fork]
#[http://expertiza.ncsu.edu/ The live Expertiza website]
#[http://wikis.lib.ncsu.edu/index.php/Expertiza Expertiza project documentation wiki]
#Clean Code: A handbook of agile software craftsmanship. Author: Robert C Martin

CSC/ECE 517 Fall 2017/E1777 Coherent specification of review requirements.rb

2017-10-30T20:42:47Z

Ssuresh6: /* Implementation */

==E1777. OSS Project Green: Coherent specification of review requirements==

This page provides a description of the Expertiza based OSS project.

__TOC__

===About Expertiza===

[http://expertiza.ncsu.edu/ Expertiza] is an open source project based on [http://rubyonrails.org/ Ruby on Rails] framework. Expertiza allows the instructor to create new assignments and customize new or existing assignments. It also allows the instructor to create a list of topics the students can sign up for. Students can form teams in Expertiza to work on various projects and assignments. Students can also peer review other students' submissions. Expertiza supports submission across various document types, including the URLs and wiki pages.

===Problem Statement===
The following tasks were accomplished in this project:

* The punctuation, syntax and the capitalization of statements in the review strategy tab requires improvement.
* Information buttons to elaborate difference between the synonymous statements present in review strategy tab need to be included.
* Message to a reviewer regarding how many reviews they are required to and allowed to perform is not clear enough.
* Assignment of work to review for a new reviewer is based on the number of previous reviews assigned for that submission which is incorrect.

===Implementation===

=====Drawbacks and Solutions=====
'''Issue #402 ''': To allow reviewers a larger set of topics to choose from, the instructor can set the threshold (on the Review Strategy tab of assignment creation/editing) to some integer k > 0. Then any submission that has within k reviews of the fewest number can be chosen by a new reviewer. Let’s say that all submissions have at least 1 review. If k = 3, then the reviewer can choose any topic where there is a submission that has 4 or fewer reviews so far. Suppose that the minimum number of reviews for any submission is 2, but that I have reviewed all the submissions that have only 2 reviews. Then I’m not allowed to review at all (unless k > 0).
* '''Solution''': This issue has been fixed previously. The reviewer will get assigned a submission even if it has fulfilled the required number of reviews, to ensure that the reviewer always receives a new submission to review as long as it is not his own.
'''Issue #969''':
*a.The two statements "Set Allowed Number of Reviews per reviewer" and "Set Required Number of Reviews per reviewer." are not well differentiated. It can very confusing for an instructor to give values for them.

* '''Solution''': Information buttons have been provided beside these statements in the review strategy tab to make sure that the instructor knows the difference between the two.
This modification is made in the app/views/assignments/edit/_review_strategy.html.erb
<pre>

<img src="/assets/info.png" title='This is the minimum number of reviews a reviewer has to perform'>
<br/>
<br/>
<%= check_box_tag('review', 'true', true, {:onChange => 'hasReviewChanged()'}) %>
<%= label_tag('review', 'Has max review limit?') %>&emsp;&emsp;
<span id="reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_reviews_allowed]', ' Set allowed number of reviews per reviewer') %>
<%if @assignment_form.assignment.num_reviews_allowed && @assignment_form.assignment.num_reviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', @assignment_form.assignment.num_reviews_allowed, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', 3, size: 1) %>
<%end%>

<img src="/assets/info.png" title='This is the maximum number of reviews a reviewer can perform'<%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
</span>

</td>
</pre>

[[File:lesscrap.png]]
*b) When the number of allowed or required reviews is not set on the Review Strategy tab, the system does not have a message to display to a reviewer about how many submissions of work they are required to and allowed to review depending on the values set by the instructor in the "allowed number of reviewers per reviewer" field and the "Set Required Number of Reviews per reviewer." field.

'''Solution''': This view has been implemented in the app/views/student_review/list.html.erb
<pre>

<h2>Reviews for "<%= @assignment.name %>"</h2>


<% if @assignment.num_reviews_allowed.nil? || @assignment.num_reviews_allowed == -1%>
<h4>Your instructor expects you to do <%= @assignment.num_reviews_required %> reviews. You are not allowed to do any extra reviews. </h4>

<% elsif @assignment.num_reviews_allowed == @assignment.num_reviews_required %>
<h4>You should perform exactly <%= @assignment.num_reviews_allowed %> reviews </h4>
<% else %>

<h4>You may perform between <%= @assignment.num_reviews_required %> and <%= @assignment.num_reviews_allowed %> reviews</h4>
<% end %>
</pre>

*c) The capitalization and punctuation of statements such as "Set Allowed Number of Reviews per reviewer" and "Set Required Number of Reviews per reviewer." in review strategy tab are incorrect.
*'''Solution''': This view has been fixed by making changes in the respective files.
The changes were made in the file : app/views/assignments/edit/_review_strategy.html.erb
<pre>
<%= label_tag('assignment_form[assignment][num_reviews_required]', 'Set required number of reviews per reviewer') %>
<%= text_field_tag('assignment_form[assignment][num_reviews_required]', @assignment_form.assignment.num_reviews_required ||= @assignment_form.assignment.num_reviews, size: 1) %>


<img src="/assets/info.png" title='This is the minimum number of reviews a reviewer has to perform'>
<br/>
<br/>
<%= check_box_tag('review', 'true', true, {:onChange => 'hasReviewChanged()'}) %>
<%= label_tag('review', 'Has max review limit?') %>&emsp;&emsp;
<span id="reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_reviews_allowed]', ' Set allowed number of reviews per reviewer') %>
<%if @assignment_form.assignment.num_reviews_allowed && @assignment_form.assignment.num_reviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', @assignment_form.assignment.num_reviews_allowed, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', 3, size: 1) %>
<%end%>

<img src="/assets/info.png" title='This is the maximum number of reviews a reviewer can perform'<%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
</span>

</td>
<td width="50%">

<%= label_tag('assignment_form[assignment][num_metareviews_required]', 'Set required number of meta-reviews per reviewer') %>
<%= text_field_tag('assignment_form[assignment][num_metareviews_required]', @assignment_form.assignment.num_metareviews_required ||= @assignment_form.assignment.num_metareviews_allowed, size: 1) %>
<br/>
<br/>

<%= check_box_tag('meta_review', 'true', true, {:onChange => 'hasMetaReviewChanged()'}) %>
<%= label_tag('meta_review', 'Has meta-review limit?') %>&emsp;&emsp;
<span id="meta_reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_metareviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_metareviews_allowed]', ' Set allowed number of meta-reviews per reviewer') %>
<%if @assignment_form.assignment.num_metareviews_allowed && @assignment_form.assignment.num_metareviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_metareviews_allowed]', @assignment_form.assignment.num_metareviews_allowed ||= 3, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_metareviews_allowed]', 3, size: 1) %>
<%end%>
</span>

</td>
</tr>

</pre>
[[File:uifixmodi.PNG]]

* '''Issue #965''': The assignment of work to review to a reviewer depends on the number of reviews it has previously received, if that work has been assigned to a reviewer previously and its review has not been submitted then the system will still consider it such that it would not assign the same work to another reviewer if it has enough number of reviewers previously assigned to review it. This might lead to a scenario where that work was assigned to enough number of reviewers, but it does not receive enough reviews and cannot be assigned to any new reviewers either.
*'''Solution''': This implementation has been fixed. The system now considers only the reviews that were submitted by the reviewers for that work so that every submission gets assigned to reviewers until it receives its allowed number of reviews.

The modified file is app/models/review_assignment.rb and the modification is done in the method reject_by_max_reviews_per_submission

<pre>
def reject_by_max_reviews_per_submission(contributor_set)
contributor_set.reject! {|contributor| contributor.responses.reject {|response| !response.is_submitted }.count >= max_reviews_per_submission }
contributor_set
end
</pre>

==Modified Files==
* app/views/assignments/edit/_review_strategy.html.erb
* app/models/review_assignment.rb
* app/views/student_review/list.html.erb

===References===

#[https://github.com/expertiza/expertiza Expertiza on GitHub]
#[https://github.com/WintersLt/expertiza GitHub Project Repository Fork]
#[http://expertiza.ncsu.edu/ The live Expertiza website]
#[http://wikis.lib.ncsu.edu/index.php/Expertiza Expertiza project documentation wiki]
#Clean Code: A handbook of agile software craftsmanship. Author: Robert C Martin

CSC/ECE 517 Fall 2017/E1777 Coherent specification of review requirements.rb

2017-10-30T20:41:53Z

Ssuresh6: /* Implementation */

==E1777. OSS Project Green: Coherent specification of review requirements==

This page provides a description of the Expertiza based OSS project.

__TOC__

===About Expertiza===

[http://expertiza.ncsu.edu/ Expertiza] is an open source project based on [http://rubyonrails.org/ Ruby on Rails] framework. Expertiza allows the instructor to create new assignments and customize new or existing assignments. It also allows the instructor to create a list of topics the students can sign up for. Students can form teams in Expertiza to work on various projects and assignments. Students can also peer review other students' submissions. Expertiza supports submission across various document types, including the URLs and wiki pages.

===Problem Statement===
The following tasks were accomplished in this project:

* The punctuation, syntax and the capitalization of statements in the review strategy tab requires improvement.
* Information buttons to elaborate difference between the synonymous statements present in review strategy tab need to be included.
* Message to a reviewer regarding how many reviews they are required to and allowed to perform is not clear enough.
* Assignment of work to review for a new reviewer is based on the number of previous reviews assigned for that submission which is incorrect.

===Implementation===

=====Drawbacks and Solutions=====
'''Issue #402 ''': To allow reviewers a larger set of topics to choose from, the instructor can set the threshold (on the Review Strategy tab of assignment creation/editing) to some integer k > 0. Then any submission that has within k reviews of the fewest number can be chosen by a new reviewer. Let’s say that all submissions have at least 1 review. If k = 3, then the reviewer can choose any topic where there is a submission that has 4 or fewer reviews so far. Suppose that the minimum number of reviews for any submission is 2, but that I have reviewed all the submissions that have only 2 reviews. Then I’m not allowed to review at all (unless k > 0).
* '''Solution''': This issue has been fixed previously. The reviewer will get assigned a submission even if it has fulfilled the required number of reviews, to ensure that the reviewer always receives a new submission to review as long as it is not his own.
'''Issue #969''':
*a.The two statements "Set Allowed Number of Reviews per reviewer" and "Set Required Number of Reviews per reviewer." are not well differentiated. It can very confusing for an instructor to give values for them.

* '''Solution''': Information buttons have been provided beside these statements in the review strategy tab to make sure that the instructor knows the difference between the two.
This modification is made in the app/views/assignments/edit/_review_strategy.html.erb
<pre>

<img src="/assets/info.png" title='This is the minimum number of reviews a reviewer has to perform'>
<br/>
<br/>
<%= check_box_tag('review', 'true', true, {:onChange => 'hasReviewChanged()'}) %>
<%= label_tag('review', 'Has max review limit?') %>&emsp;&emsp;
<span id="reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_reviews_allowed]', ' Set allowed number of reviews per reviewer') %>
<%if @assignment_form.assignment.num_reviews_allowed && @assignment_form.assignment.num_reviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', @assignment_form.assignment.num_reviews_allowed, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', 3, size: 1) %>
<%end%>

<img src="/assets/info.png" title='This is the maximum number of reviews a reviewer can perform'<%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
</span>

</td>
</pre>
[[File:morecrap.png]]
[[File:lesscrap.png]]
*b) When the number of allowed or required reviews is not set on the Review Strategy tab, the system does not have a message to display to a reviewer about how many submissions of work they are required to and allowed to review depending on the values set by the instructor in the "allowed number of reviewers per reviewer" field and the "Set Required Number of Reviews per reviewer." field.

'''Solution''': This view has been implemented in the app/views/student_review/list.html.erb
<pre>

<h2>Reviews for "<%= @assignment.name %>"</h2>


<% if @assignment.num_reviews_allowed.nil? || @assignment.num_reviews_allowed == -1%>
<h4>Your instructor expects you to do <%= @assignment.num_reviews_required %> reviews. You are not allowed to do any extra reviews. </h4>

<% elsif @assignment.num_reviews_allowed == @assignment.num_reviews_required %>
<h4>You should perform exactly <%= @assignment.num_reviews_allowed %> reviews </h4>
<% else %>

<h4>You may perform between <%= @assignment.num_reviews_required %> and <%= @assignment.num_reviews_allowed %> reviews</h4>
<% end %>
</pre>

*c) The capitalization and punctuation of statements such as "Set Allowed Number of Reviews per reviewer" and "Set Required Number of Reviews per reviewer." in review strategy tab are incorrect.
*'''Solution''': This view has been fixed by making changes in the respective files.
The changes were made in the file : app/views/assignments/edit/_review_strategy.html.erb
<pre>
<%= label_tag('assignment_form[assignment][num_reviews_required]', 'Set required number of reviews per reviewer') %>
<%= text_field_tag('assignment_form[assignment][num_reviews_required]', @assignment_form.assignment.num_reviews_required ||= @assignment_form.assignment.num_reviews, size: 1) %>


<img src="/assets/info.png" title='This is the minimum number of reviews a reviewer has to perform'>
<br/>
<br/>
<%= check_box_tag('review', 'true', true, {:onChange => 'hasReviewChanged()'}) %>
<%= label_tag('review', 'Has max review limit?') %>&emsp;&emsp;
<span id="reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_reviews_allowed]', ' Set allowed number of reviews per reviewer') %>
<%if @assignment_form.assignment.num_reviews_allowed && @assignment_form.assignment.num_reviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', @assignment_form.assignment.num_reviews_allowed, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', 3, size: 1) %>
<%end%>

<img src="/assets/info.png" title='This is the maximum number of reviews a reviewer can perform'<%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
</span>

</td>
<td width="50%">

<%= label_tag('assignment_form[assignment][num_metareviews_required]', 'Set required number of meta-reviews per reviewer') %>
<%= text_field_tag('assignment_form[assignment][num_metareviews_required]', @assignment_form.assignment.num_metareviews_required ||= @assignment_form.assignment.num_metareviews_allowed, size: 1) %>
<br/>
<br/>

<%= check_box_tag('meta_review', 'true', true, {:onChange => 'hasMetaReviewChanged()'}) %>
<%= label_tag('meta_review', 'Has meta-review limit?') %>&emsp;&emsp;
<span id="meta_reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_metareviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_metareviews_allowed]', ' Set allowed number of meta-reviews per reviewer') %>
<%if @assignment_form.assignment.num_metareviews_allowed && @assignment_form.assignment.num_metareviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_metareviews_allowed]', @assignment_form.assignment.num_metareviews_allowed ||= 3, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_metareviews_allowed]', 3, size: 1) %>
<%end%>
</span>

</td>
</tr>

</pre>
[[File:uifixmodi.PNG]]

* '''Issue #965''': The assignment of work to review to a reviewer depends on the number of reviews it has previously received, if that work has been assigned to a reviewer previously and its review has not been submitted then the system will still consider it such that it would not assign the same work to another reviewer if it has enough number of reviewers previously assigned to review it. This might lead to a scenario where that work was assigned to enough number of reviewers, but it does not receive enough reviews and cannot be assigned to any new reviewers either.
*'''Solution''': This implementation has been fixed. The system now considers only the reviews that were submitted by the reviewers for that work so that every submission gets assigned to reviewers until it receives its allowed number of reviews.

The modified file is app/models/review_assignment.rb and the modification is done in the method reject_by_max_reviews_per_submission

<pre>
def reject_by_max_reviews_per_submission(contributor_set)
contributor_set.reject! {|contributor| contributor.responses.reject {|response| !response.is_submitted }.count >= max_reviews_per_submission }
contributor_set
end
</pre>

==Modified Files==
* app/views/assignments/edit/_review_strategy.html.erb
* app/models/review_assignment.rb
* app/views/student_review/list.html.erb

===References===

#[https://github.com/expertiza/expertiza Expertiza on GitHub]
#[https://github.com/WintersLt/expertiza GitHub Project Repository Fork]
#[http://expertiza.ncsu.edu/ The live Expertiza website]
#[http://wikis.lib.ncsu.edu/index.php/Expertiza Expertiza project documentation wiki]
#Clean Code: A handbook of agile software craftsmanship. Author: Robert C Martin

File:Lesscrap.png

2017-10-30T20:41:03Z

Ssuresh6:

File:Infobuttons.png

2017-10-30T20:39:43Z

Ssuresh6: uploaded a new version of "File:Infobuttons.png"

CSC/ECE 517 Fall 2017/E1777 Coherent specification of review requirements.rb

2017-10-30T20:38:50Z

Ssuresh6: /* Implementation */

==E1777. OSS Project Green: Coherent specification of review requirements==

This page provides a description of the Expertiza based OSS project.

__TOC__

===About Expertiza===

[http://expertiza.ncsu.edu/ Expertiza] is an open source project based on [http://rubyonrails.org/ Ruby on Rails] framework. Expertiza allows the instructor to create new assignments and customize new or existing assignments. It also allows the instructor to create a list of topics the students can sign up for. Students can form teams in Expertiza to work on various projects and assignments. Students can also peer review other students' submissions. Expertiza supports submission across various document types, including the URLs and wiki pages.

===Problem Statement===
The following tasks were accomplished in this project:

* The punctuation, syntax and the capitalization of statements in the review strategy tab requires improvement.
* Information buttons to elaborate difference between the synonymous statements present in review strategy tab need to be included.
* Message to a reviewer regarding how many reviews they are required to and allowed to perform is not clear enough.
* Assignment of work to review for a new reviewer is based on the number of previous reviews assigned for that submission which is incorrect.

===Implementation===

=====Drawbacks and Solutions=====
'''Issue #402 ''': To allow reviewers a larger set of topics to choose from, the instructor can set the threshold (on the Review Strategy tab of assignment creation/editing) to some integer k > 0. Then any submission that has within k reviews of the fewest number can be chosen by a new reviewer. Let’s say that all submissions have at least 1 review. If k = 3, then the reviewer can choose any topic where there is a submission that has 4 or fewer reviews so far. Suppose that the minimum number of reviews for any submission is 2, but that I have reviewed all the submissions that have only 2 reviews. Then I’m not allowed to review at all (unless k > 0).
* '''Solution''': This issue has been fixed previously. The reviewer will get assigned a submission even if it has fulfilled the required number of reviews, to ensure that the reviewer always receives a new submission to review as long as it is not his own.
'''Issue #969''':
*a.The two statements "Set Allowed Number of Reviews per reviewer" and "Set Required Number of Reviews per reviewer." are not well differentiated. It can very confusing for an instructor to give values for them.

* '''Solution''': Information buttons have been provided beside these statements in the review strategy tab to make sure that the instructor knows the difference between the two.
This modification is made in the app/views/assignments/edit/_review_strategy.html.erb
<pre>

<img src="/assets/info.png" title='This is the minimum number of reviews a reviewer has to perform'>
<br/>
<br/>
<%= check_box_tag('review', 'true', true, {:onChange => 'hasReviewChanged()'}) %>
<%= label_tag('review', 'Has max review limit?') %>&emsp;&emsp;
<span id="reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_reviews_allowed]', ' Set allowed number of reviews per reviewer') %>
<%if @assignment_form.assignment.num_reviews_allowed && @assignment_form.assignment.num_reviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', @assignment_form.assignment.num_reviews_allowed, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', 3, size: 1) %>
<%end%>

<img src="/assets/info.png" title='This is the maximum number of reviews a reviewer can perform'<%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
</span>

</td>
</pre>
[[File:infobuttons.png]]
[[File:infobutton2.png]]
*b) When the number of allowed or required reviews is not set on the Review Strategy tab, the system does not have a message to display to a reviewer about how many submissions of work they are required to and allowed to review depending on the values set by the instructor in the "allowed number of reviewers per reviewer" field and the "Set Required Number of Reviews per reviewer." field.

'''Solution''': This view has been implemented in the app/views/student_review/list.html.erb
<pre>

<h2>Reviews for "<%= @assignment.name %>"</h2>


<% if @assignment.num_reviews_allowed.nil? || @assignment.num_reviews_allowed == -1%>
<h4>Your instructor expects you to do <%= @assignment.num_reviews_required %> reviews. You are not allowed to do any extra reviews. </h4>

<% elsif @assignment.num_reviews_allowed == @assignment.num_reviews_required %>
<h4>You should perform exactly <%= @assignment.num_reviews_allowed %> reviews </h4>
<% else %>

<h4>You may perform between <%= @assignment.num_reviews_required %> and <%= @assignment.num_reviews_allowed %> reviews</h4>
<% end %>
</pre>

*c) The capitalization and punctuation of statements such as "Set Allowed Number of Reviews per reviewer" and "Set Required Number of Reviews per reviewer." in review strategy tab are incorrect.
*'''Solution''': This view has been fixed by making changes in the respective files.
The changes were made in the file : app/views/assignments/edit/_review_strategy.html.erb
<pre>
<%= label_tag('assignment_form[assignment][num_reviews_required]', 'Set required number of reviews per reviewer') %>
<%= text_field_tag('assignment_form[assignment][num_reviews_required]', @assignment_form.assignment.num_reviews_required ||= @assignment_form.assignment.num_reviews, size: 1) %>


<img src="/assets/info.png" title='This is the minimum number of reviews a reviewer has to perform'>
<br/>
<br/>
<%= check_box_tag('review', 'true', true, {:onChange => 'hasReviewChanged()'}) %>
<%= label_tag('review', 'Has max review limit?') %>&emsp;&emsp;
<span id="reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_reviews_allowed]', ' Set allowed number of reviews per reviewer') %>
<%if @assignment_form.assignment.num_reviews_allowed && @assignment_form.assignment.num_reviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', @assignment_form.assignment.num_reviews_allowed, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', 3, size: 1) %>
<%end%>

<img src="/assets/info.png" title='This is the maximum number of reviews a reviewer can perform'<%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
</span>

</td>
<td width="50%">

<%= label_tag('assignment_form[assignment][num_metareviews_required]', 'Set required number of meta-reviews per reviewer') %>
<%= text_field_tag('assignment_form[assignment][num_metareviews_required]', @assignment_form.assignment.num_metareviews_required ||= @assignment_form.assignment.num_metareviews_allowed, size: 1) %>
<br/>
<br/>

<%= check_box_tag('meta_review', 'true', true, {:onChange => 'hasMetaReviewChanged()'}) %>
<%= label_tag('meta_review', 'Has meta-review limit?') %>&emsp;&emsp;
<span id="meta_reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_metareviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_metareviews_allowed]', ' Set allowed number of meta-reviews per reviewer') %>
<%if @assignment_form.assignment.num_metareviews_allowed && @assignment_form.assignment.num_metareviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_metareviews_allowed]', @assignment_form.assignment.num_metareviews_allowed ||= 3, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_metareviews_allowed]', 3, size: 1) %>
<%end%>
</span>

</td>
</tr>

</pre>
[[File:uifixmodi.PNG]]

* '''Issue #965''': The assignment of work to review to a reviewer depends on the number of reviews it has previously received, if that work has been assigned to a reviewer previously and its review has not been submitted then the system will still consider it such that it would not assign the same work to another reviewer if it has enough number of reviewers previously assigned to review it. This might lead to a scenario where that work was assigned to enough number of reviewers, but it does not receive enough reviews and cannot be assigned to any new reviewers either.
*'''Solution''': This implementation has been fixed. The system now considers only the reviews that were submitted by the reviewers for that work so that every submission gets assigned to reviewers until it receives its allowed number of reviews.

The modified file is app/models/review_assignment.rb and the modification is done in the method reject_by_max_reviews_per_submission

<pre>
def reject_by_max_reviews_per_submission(contributor_set)
contributor_set.reject! {|contributor| contributor.responses.reject {|response| !response.is_submitted }.count >= max_reviews_per_submission }
contributor_set
end
</pre>

==Modified Files==
* app/views/assignments/edit/_review_strategy.html.erb
* app/models/review_assignment.rb
* app/views/student_review/list.html.erb

===References===

#[https://github.com/expertiza/expertiza Expertiza on GitHub]
#[https://github.com/WintersLt/expertiza GitHub Project Repository Fork]
#[http://expertiza.ncsu.edu/ The live Expertiza website]
#[http://wikis.lib.ncsu.edu/index.php/Expertiza Expertiza project documentation wiki]
#Clean Code: A handbook of agile software craftsmanship. Author: Robert C Martin

CSC/ECE 517 Fall 2017/E1777 Coherent specification of review requirements.rb

2017-10-30T20:37:41Z

Ssuresh6: /* Drawbacks and Solutions */

==E1777. OSS Project Green: Coherent specification of review requirements==

This page provides a description of the Expertiza based OSS project.

__TOC__

===About Expertiza===

[http://expertiza.ncsu.edu/ Expertiza] is an open source project based on [http://rubyonrails.org/ Ruby on Rails] framework. Expertiza allows the instructor to create new assignments and customize new or existing assignments. It also allows the instructor to create a list of topics the students can sign up for. Students can form teams in Expertiza to work on various projects and assignments. Students can also peer review other students' submissions. Expertiza supports submission across various document types, including the URLs and wiki pages.

===Problem Statement===
The following tasks were accomplished in this project:

* The punctuation, syntax and the capitalization of statements in the review strategy tab requires improvement.
* Information buttons to elaborate difference between the synonymous statements present in review strategy tab need to be included.
* Message to a reviewer regarding how many reviews they are required to and allowed to perform is not clear enough.
* Assignment of work to review for a new reviewer is based on the number of previous reviews assigned for that submission which is incorrect.

===Implementation===

=====Drawbacks and Solutions=====
'''Issue #402 ''': To allow reviewers a larger set of topics to choose from, the instructor can set the threshold (on the Review Strategy tab of assignment creation/editing) to some integer k > 0. Then any submission that has within k reviews of the fewest number can be chosen by a new reviewer. Let’s say that all submissions have at least 1 review. If k = 3, then the reviewer can choose any topic where there is a submission that has 4 or fewer reviews so far. Suppose that the minimum number of reviews for any submission is 2, but that I have reviewed all the submissions that have only 2 reviews. Then I’m not allowed to review at all (unless k > 0).
* '''Solution''': This issue has been fixed previously. The reviewer will get assigned a submission even if it has fulfilled the required number of reviews, to ensure that the reviewer always receives a new submission to review as long as it is not his own.
'''Issue #969''':
*a.The two statements "Set Allowed Number of Reviews per reviewer" and "Set Required Number of Reviews per reviewer." are not well differentiated. It can very confusing for an instructor to give values for them.

* '''Solution''': Information buttons have been provided beside these statements in the review strategy tab to make sure that the instructor knows the difference between the two.
This modification is made in the app/views/assignments/edit/_review_strategy.html.erb
<pre>

<img src="/assets/info.png" title='This is the minimum number of reviews a reviewer has to perform'>
<br/>
<br/>
<%= check_box_tag('review', 'true', true, {:onChange => 'hasReviewChanged()'}) %>
<%= label_tag('review', 'Has max review limit?') %>&emsp;&emsp;
<span id="reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_reviews_allowed]', ' Set allowed number of reviews per reviewer') %>
<%if @assignment_form.assignment.num_reviews_allowed && @assignment_form.assignment.num_reviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', @assignment_form.assignment.num_reviews_allowed, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', 3, size: 1) %>
<%end%>

<img src="/assets/info.png" title='This is the maximum number of reviews a reviewer can perform'<%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
</span>

</td>
</pre>
[[File:infobuttons.png]]
[[File:infobutton2.png]]
*b) When the number of allowed or required reviews is not set on the Review Strategy tab, the system does not have a message to display to a reviewer about how many submissions of work they are required to and allowed to review depending on the values set by the instructor in the "allowed number of reviewers per reviewer" field and the "Set Required Number of Reviews per reviewer." field.

'''Solution''': This view has been implemented in the app/views/student_review/list.html.erb
<pre>

<h2>Reviews for "<%= @assignment.name %>"</h2>


<% if @assignment.num_reviews_allowed.nil? || @assignment.num_reviews_allowed == -1%>
<h4>Your instructor expects you to do <%= @assignment.num_reviews_required %> reviews. You are not allowed to do any extra reviews. </h4>

<% elsif @assignment.num_reviews_allowed == @assignment.num_reviews_required %>
<h4>You should perform exactly <%= @assignment.num_reviews_allowed %> reviews </h4>
<% else %>

<h4>You may perform between <%= @assignment.num_reviews_required %> and <%= @assignment.num_reviews_allowed %> reviews</h4>
<% end %>
</pre>

*c) The capitalization and punctuation of statements such as "Set Allowed Number of Reviews per reviewer" and "Set Required Number of Reviews per reviewer." in review strategy tab are incorrect.
*'''Solution''': This view has been fixed by making changes in the respective files.
The changes were made in the file : app/views/assignments/edit/_review_strategy.html.erb
<pre>
<%= label_tag('assignment_form[assignment][num_reviews_required]', 'Set required number of reviews per reviewer') %>
<%= text_field_tag('assignment_form[assignment][num_reviews_required]', @assignment_form.assignment.num_reviews_required ||= @assignment_form.assignment.num_reviews, size: 1) %>


<img src="/assets/info.png" title='This is the minimum number of reviews a reviewer has to perform'>
<br/>
<br/>
<%= check_box_tag('review', 'true', true, {:onChange => 'hasReviewChanged()'}) %>
<%= label_tag('review', 'Has max review limit?') %>&emsp;&emsp;
<span id="reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_reviews_allowed]', ' Set allowed number of reviews per reviewer') %>
<%if @assignment_form.assignment.num_reviews_allowed && @assignment_form.assignment.num_reviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', @assignment_form.assignment.num_reviews_allowed, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', 3, size: 1) %>
<%end%>

<img src="/assets/info.png" title='This is the maximum number of reviews a reviewer can perform'<%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
</span>

</td>
<td width="50%">

<%= label_tag('assignment_form[assignment][num_metareviews_required]', 'Set required number of meta-reviews per reviewer') %>
<%= text_field_tag('assignment_form[assignment][num_metareviews_required]', @assignment_form.assignment.num_metareviews_required ||= @assignment_form.assignment.num_metareviews_allowed, size: 1) %>
<br/>
<br/>

<%= check_box_tag('meta_review', 'true', true, {:onChange => 'hasMetaReviewChanged()'}) %>
<%= label_tag('meta_review', 'Has meta-review limit?') %>&emsp;&emsp;
<span id="meta_reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_metareviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_metareviews_allowed]', ' Set allowed number of meta-reviews per reviewer') %>
<%if @assignment_form.assignment.num_metareviews_allowed && @assignment_form.assignment.num_metareviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_metareviews_allowed]', @assignment_form.assignment.num_metareviews_allowed ||= 3, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_metareviews_allowed]', 3, size: 1) %>
<%end%>
</span>

</td>
</tr>

</pre>
[[File:uifixmodi.png]]

* '''Issue #965''': The assignment of work to review to a reviewer depends on the number of reviews it has previously received, if that work has been assigned to a reviewer previously and its review has not been submitted then the system will still consider it such that it would not assign the same work to another reviewer if it has enough number of reviewers previously assigned to review it. This might lead to a scenario where that work was assigned to enough number of reviewers, but it does not receive enough reviews and cannot be assigned to any new reviewers either.
*'''Solution''': This implementation has been fixed. The system now considers only the reviews that were submitted by the reviewers for that work so that every submission gets assigned to reviewers until it receives its allowed number of reviews.

The modified file is app/models/review_assignment.rb and the modification is done in the method reject_by_max_reviews_per_submission

<pre>
def reject_by_max_reviews_per_submission(contributor_set)
contributor_set.reject! {|contributor| contributor.responses.reject {|response| !response.is_submitted }.count >= max_reviews_per_submission }
contributor_set
end
</pre>

==Modified Files==
* app/views/assignments/edit/_review_strategy.html.erb
* app/models/review_assignment.rb
* app/views/student_review/list.html.erb

===References===

#[https://github.com/expertiza/expertiza Expertiza on GitHub]
#[https://github.com/WintersLt/expertiza GitHub Project Repository Fork]
#[http://expertiza.ncsu.edu/ The live Expertiza website]
#[http://wikis.lib.ncsu.edu/index.php/Expertiza Expertiza project documentation wiki]
#Clean Code: A handbook of agile software craftsmanship. Author: Robert C Martin

File:Uifixmodi.PNG

2017-10-30T20:37:06Z

Ssuresh6:

CSC/ECE 517 Fall 2017/E1777 Coherent specification of review requirements.rb

2017-10-30T20:32:43Z

Ssuresh6: /* Drawbacks and Solutions */

==E1777. OSS Project Green: Coherent specification of review requirements==

This page provides a description of the Expertiza based OSS project.

__TOC__

===About Expertiza===

[http://expertiza.ncsu.edu/ Expertiza] is an open source project based on [http://rubyonrails.org/ Ruby on Rails] framework. Expertiza allows the instructor to create new assignments and customize new or existing assignments. It also allows the instructor to create a list of topics the students can sign up for. Students can form teams in Expertiza to work on various projects and assignments. Students can also peer review other students' submissions. Expertiza supports submission across various document types, including the URLs and wiki pages.

===Problem Statement===
The following tasks were accomplished in this project:

* The punctuation, syntax and the capitalization of statements in the review strategy tab requires improvement.
* Information buttons to elaborate difference between the synonymous statements present in review strategy tab need to be included.
* Message to a reviewer regarding how many reviews they are required to and allowed to perform is not clear enough.
* Assignment of work to review for a new reviewer is based on the number of previous reviews assigned for that submission which is incorrect.

===Implementation===

=====Drawbacks and Solutions=====
'''Issue #402 ''': To allow reviewers a larger set of topics to choose from, the instructor can set the threshold (on the Review Strategy tab of assignment creation/editing) to some integer k > 0. Then any submission that has within k reviews of the fewest number can be chosen by a new reviewer. Let’s say that all submissions have at least 1 review. If k = 3, then the reviewer can choose any topic where there is a submission that has 4 or fewer reviews so far. Suppose that the minimum number of reviews for any submission is 2, but that I have reviewed all the submissions that have only 2 reviews. Then I’m not allowed to review at all (unless k > 0).
* '''Solution''': This issue has been fixed previously. The reviewer will get assigned a submission even if it has fulfilled the required number of reviews, to ensure that the reviewer always receives a new submission to review as long as it is not his own.
'''Issue #969''':
*a.The two statements "Set Allowed Number of Reviews per reviewer" and "Set Required Number of Reviews per reviewer." are not well differentiated. It can very confusing for an instructor to give values for them.

* '''Solution''': Information buttons have been provided beside these statements in the review strategy tab to make sure that the instructor knows the difference between the two.
This modification is made in the app/views/assignments/edit/_review_strategy.html.erb
<pre>

<img src="/assets/info.png" title='This is the minimum number of reviews a reviewer has to perform'>
<br/>
<br/>
<%= check_box_tag('review', 'true', true, {:onChange => 'hasReviewChanged()'}) %>
<%= label_tag('review', 'Has max review limit?') %>&emsp;&emsp;
<span id="reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_reviews_allowed]', ' Set allowed number of reviews per reviewer') %>
<%if @assignment_form.assignment.num_reviews_allowed && @assignment_form.assignment.num_reviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', @assignment_form.assignment.num_reviews_allowed, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', 3, size: 1) %>
<%end%>

<img src="/assets/info.png" title='This is the maximum number of reviews a reviewer can perform'<%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
</span>

</td>
</pre>
[[File:infobuttons.png]]
[[File:infobutton2.png]]
*b) When the number of allowed or required reviews is not set on the Review Strategy tab, the system does not have a message to display to a reviewer about how many submissions of work they are required to and allowed to review depending on the values set by the instructor in the "allowed number of reviewers per reviewer" field and the "Set Required Number of Reviews per reviewer." field.

'''Solution''': This view has been implemented in the app/views/student_review/list.html.erb
<pre>

<h2>Reviews for "<%= @assignment.name %>"</h2>


<% if @assignment.num_reviews_allowed.nil? || @assignment.num_reviews_allowed == -1%>
<h4>Your instructor expects you to do <%= @assignment.num_reviews_required %> reviews. You are not allowed to do any extra reviews. </h4>

<% elsif @assignment.num_reviews_allowed == @assignment.num_reviews_required %>
<h4>You should perform exactly <%= @assignment.num_reviews_allowed %> reviews </h4>
<% else %>

<h4>You may perform between <%= @assignment.num_reviews_required %> and <%= @assignment.num_reviews_allowed %> reviews</h4>
<% end %>
</pre>

*c) The capitalization and punctuation of statements such as "Set Allowed Number of Reviews per reviewer" and "Set Required Number of Reviews per reviewer." in review strategy tab are incorrect.
*'''Solution''': This view has been fixed by making changes in the respective files.
The changes were made in the file : app/views/assignments/edit/_review_strategy.html.erb
<pre>
<%= label_tag('assignment_form[assignment][num_reviews_required]', 'Set required number of reviews per reviewer') %>
<%= text_field_tag('assignment_form[assignment][num_reviews_required]', @assignment_form.assignment.num_reviews_required ||= @assignment_form.assignment.num_reviews, size: 1) %>


<img src="/assets/info.png" title='This is the minimum number of reviews a reviewer has to perform'>
<br/>
<br/>
<%= check_box_tag('review', 'true', true, {:onChange => 'hasReviewChanged()'}) %>
<%= label_tag('review', 'Has max review limit?') %>&emsp;&emsp;
<span id="reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_reviews_allowed]', ' Set allowed number of reviews per reviewer') %>
<%if @assignment_form.assignment.num_reviews_allowed && @assignment_form.assignment.num_reviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', @assignment_form.assignment.num_reviews_allowed, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', 3, size: 1) %>
<%end%>

<img src="/assets/info.png" title='This is the maximum number of reviews a reviewer can perform'<%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
</span>

</td>
<td width="50%">

<%= label_tag('assignment_form[assignment][num_metareviews_required]', 'Set required number of meta-reviews per reviewer') %>
<%= text_field_tag('assignment_form[assignment][num_metareviews_required]', @assignment_form.assignment.num_metareviews_required ||= @assignment_form.assignment.num_metareviews_allowed, size: 1) %>
<br/>
<br/>

<%= check_box_tag('meta_review', 'true', true, {:onChange => 'hasMetaReviewChanged()'}) %>
<%= label_tag('meta_review', 'Has meta-review limit?') %>&emsp;&emsp;
<span id="meta_reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_metareviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_metareviews_allowed]', ' Set allowed number of meta-reviews per reviewer') %>
<%if @assignment_form.assignment.num_metareviews_allowed && @assignment_form.assignment.num_metareviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_metareviews_allowed]', @assignment_form.assignment.num_metareviews_allowed ||= 3, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_metareviews_allowed]', 3, size: 1) %>
<%end%>
</span>

</td>
</tr>

</pre>
[[File:uifix (2).png]]

* '''Issue #965''': The assignment of work to review to a reviewer depends on the number of reviews it has previously received, if that work has been assigned to a reviewer previously and its review has not been submitted then the system will still consider it such that it would not assign the same work to another reviewer if it has enough number of reviewers previously assigned to review it. This might lead to a scenario where that work was assigned to enough number of reviewers, but it does not receive enough reviews and cannot be assigned to any new reviewers either.
*'''Solution''': This implementation has been fixed. The system now considers only the reviews that were submitted by the reviewers for that work so that every submission gets assigned to reviewers until it receives its allowed number of reviews.

The modified file is app/models/review_assignment.rb and the modification is done in the method reject_by_max_reviews_per_submission

<pre>
def reject_by_max_reviews_per_submission(contributor_set)
contributor_set.reject! {|contributor| contributor.responses.reject {|response| !response.is_submitted }.count >= max_reviews_per_submission }
contributor_set
end
</pre>

==Modified Files==
* app/views/assignments/edit/_review_strategy.html.erb
* app/models/review_assignment.rb
* app/views/student_review/list.html.erb

===References===

#[https://github.com/expertiza/expertiza Expertiza on GitHub]
#[https://github.com/WintersLt/expertiza GitHub Project Repository Fork]
#[http://expertiza.ncsu.edu/ The live Expertiza website]
#[http://wikis.lib.ncsu.edu/index.php/Expertiza Expertiza project documentation wiki]
#Clean Code: A handbook of agile software craftsmanship. Author: Robert C Martin

CSC/ECE 517 Fall 2017/E1777 Coherent specification of review requirements.rb

2017-10-30T20:32:07Z

Ssuresh6: /* Implementation */

==E1777. OSS Project Green: Coherent specification of review requirements==

This page provides a description of the Expertiza based OSS project.

__TOC__

===About Expertiza===

[http://expertiza.ncsu.edu/ Expertiza] is an open source project based on [http://rubyonrails.org/ Ruby on Rails] framework. Expertiza allows the instructor to create new assignments and customize new or existing assignments. It also allows the instructor to create a list of topics the students can sign up for. Students can form teams in Expertiza to work on various projects and assignments. Students can also peer review other students' submissions. Expertiza supports submission across various document types, including the URLs and wiki pages.

===Problem Statement===
The following tasks were accomplished in this project:

* The punctuation, syntax and the capitalization of statements in the review strategy tab requires improvement.
* Information buttons to elaborate difference between the synonymous statements present in review strategy tab need to be included.
* Message to a reviewer regarding how many reviews they are required to and allowed to perform is not clear enough.
* Assignment of work to review for a new reviewer is based on the number of previous reviews assigned for that submission which is incorrect.

===Implementation===

=====Drawbacks and Solutions=====
'''Issue #402 ''': To allow reviewers a larger set of topics to choose from, the instructor can set the threshold (on the Review Strategy tab of assignment creation/editing) to some integer k > 0. Then any submission that has within k reviews of the fewest number can be chosen by a new reviewer. Let’s say that all submissions have at least 1 review. If k = 3, then the reviewer can choose any topic where there is a submission that has 4 or fewer reviews so far. Suppose that the minimum number of reviews for any submission is 2, but that I have reviewed all the submissions that have only 2 reviews. Then I’m not allowed to review at all (unless k > 0).
* '''Solution''': This issue has been fixed previously. The reviewer will get assigned a submission even if it has fulfilled the required number of reviews, to ensure that the reviewer always receives a new submission to review as long as it is not his own.
'''Issue #969''':
*a.The two statements "Set Allowed Number of Reviews per reviewer" and "Set Required Number of Reviews per reviewer." are not well differentiated. It can very confusing for an instructor to give values for them.

* '''Solution''': Information buttons have been provided beside these statements in the review strategy tab to make sure that the instructor knows the difference between the two.
This modification is made in the app/views/assignments/edit/_review_strategy.html.erb
<pre>

<img src="/assets/info.png" title='This is the minimum number of reviews a reviewer has to perform'>
<br/>
<br/>
<%= check_box_tag('review', 'true', true, {:onChange => 'hasReviewChanged()'}) %>
<%= label_tag('review', 'Has max review limit?') %>&emsp;&emsp;
<span id="reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_reviews_allowed]', ' Set allowed number of reviews per reviewer') %>
<%if @assignment_form.assignment.num_reviews_allowed && @assignment_form.assignment.num_reviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', @assignment_form.assignment.num_reviews_allowed, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', 3, size: 1) %>
<%end%>

<img src="/assets/info.png" title='This is the maximum number of reviews a reviewer can perform'<%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
</span>

</td>
</pre>
[[File:infobuttons.png]]
[[File:infobutton2.png]]
*b) When the number of allowed or required reviews is not set on the Review Strategy tab, the system does not have a message to display to a reviewer about how many submissions of work they are required to and allowed to review depending on the values set by the instructor in the "allowed number of reviewers per reviewer" field and the "Set Required Number of Reviews per reviewer." field.

'''Solution''': This view has been implemented in the app/views/student_review/list.html.erb
<pre>

<h2>Reviews for "<%= @assignment.name %>"</h2>


<% if @assignment.num_reviews_allowed.nil? || @assignment.num_reviews_allowed == -1%>
<h4>Your instructor expects you to do <%= @assignment.num_reviews_required %> reviews. You are not allowed to do any extra reviews. </h4>

<% elsif @assignment.num_reviews_allowed == @assignment.num_reviews_required %>
<h4>You should perform exactly <%= @assignment.num_reviews_allowed %> reviews </h4>
<% else %>

<h4>You may perform between <%= @assignment.num_reviews_required %> and <%= @assignment.num_reviews_allowed %> reviews</h4>
<% end %>
</pre>

*c) The capitalization and punctuation of statements such as "Set Allowed Number of Reviews per reviewer" and "Set Required Number of Reviews per reviewer." in review strategy tab are incorrect.
'''Solution''': This view has been fixed by making changes in the respective files.
The changes were made in the file : app/views/assignments/edit/_review_strategy.html.erb
<pre>
<%= label_tag('assignment_form[assignment][num_reviews_required]', 'Set required number of reviews per reviewer') %>
<%= text_field_tag('assignment_form[assignment][num_reviews_required]', @assignment_form.assignment.num_reviews_required ||= @assignment_form.assignment.num_reviews, size: 1) %>


<img src="/assets/info.png" title='This is the minimum number of reviews a reviewer has to perform'>
<br/>
<br/>
<%= check_box_tag('review', 'true', true, {:onChange => 'hasReviewChanged()'}) %>
<%= label_tag('review', 'Has max review limit?') %>&emsp;&emsp;
<span id="reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_reviews_allowed]', ' Set allowed number of reviews per reviewer') %>
<%if @assignment_form.assignment.num_reviews_allowed && @assignment_form.assignment.num_reviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', @assignment_form.assignment.num_reviews_allowed, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', 3, size: 1) %>
<%end%>

<img src="/assets/info.png" title='This is the maximum number of reviews a reviewer can perform'<%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
</span>

</td>
<td width="50%">

<%= label_tag('assignment_form[assignment][num_metareviews_required]', 'Set required number of meta-reviews per reviewer') %>
<%= text_field_tag('assignment_form[assignment][num_metareviews_required]', @assignment_form.assignment.num_metareviews_required ||= @assignment_form.assignment.num_metareviews_allowed, size: 1) %>
<br/>
<br/>

<%= check_box_tag('meta_review', 'true', true, {:onChange => 'hasMetaReviewChanged()'}) %>
<%= label_tag('meta_review', 'Has meta-review limit?') %>&emsp;&emsp;
<span id="meta_reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_metareviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_metareviews_allowed]', ' Set allowed number of meta-reviews per reviewer') %>
<%if @assignment_form.assignment.num_metareviews_allowed && @assignment_form.assignment.num_metareviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_metareviews_allowed]', @assignment_form.assignment.num_metareviews_allowed ||= 3, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_metareviews_allowed]', 3, size: 1) %>
<%end%>
</span>

</td>
</tr>

</pre>
[[File:uifix (2).png]]

* '''Issue #965''': The assignment of work to review to a reviewer depends on the number of reviews it has previously received, if that work has been assigned to a reviewer previously and its review has not been submitted then the system will still consider it such that it would not assign the same work to another reviewer if it has enough number of reviewers previously assigned to review it. This might lead to a scenario where that work was assigned to enough number of reviewers, but it does not receive enough reviews and cannot be assigned to any new reviewers either.
'''Solution''': This implementation has been fixed. The system now considers only the reviews that were submitted by the reviewers for that work so that every submission gets assigned to reviewers until it receives its allowed number of reviews.

The modified file is app/models/review_assignment.rb and the modification is done in the method reject_by_max_reviews_per_submission

<pre>
def reject_by_max_reviews_per_submission(contributor_set)
contributor_set.reject! {|contributor| contributor.responses.reject {|response| !response.is_submitted }.count >= max_reviews_per_submission }
contributor_set
end
</pre>

==Modified Files==
* app/views/assignments/edit/_review_strategy.html.erb
* app/models/review_assignment.rb
* app/views/student_review/list.html.erb

===References===

#[https://github.com/expertiza/expertiza Expertiza on GitHub]
#[https://github.com/WintersLt/expertiza GitHub Project Repository Fork]
#[http://expertiza.ncsu.edu/ The live Expertiza website]
#[http://wikis.lib.ncsu.edu/index.php/Expertiza Expertiza project documentation wiki]
#Clean Code: A handbook of agile software craftsmanship. Author: Robert C Martin

CSC/ECE 517 Fall 2017/E1777 Coherent specification of review requirements.rb

2017-10-30T20:30:33Z

Ssuresh6: /* Implementation */

==E1777. OSS Project Green: Coherent specification of review requirements==

This page provides a description of the Expertiza based OSS project.

__TOC__

===About Expertiza===

[http://expertiza.ncsu.edu/ Expertiza] is an open source project based on [http://rubyonrails.org/ Ruby on Rails] framework. Expertiza allows the instructor to create new assignments and customize new or existing assignments. It also allows the instructor to create a list of topics the students can sign up for. Students can form teams in Expertiza to work on various projects and assignments. Students can also peer review other students' submissions. Expertiza supports submission across various document types, including the URLs and wiki pages.

===Problem Statement===
The following tasks were accomplished in this project:

* The punctuation, syntax and the capitalization of statements in the review strategy tab requires improvement.
* Information buttons to elaborate difference between the synonymous statements present in review strategy tab need to be included.
* Message to a reviewer regarding how many reviews they are required to and allowed to perform is not clear enough.
* Assignment of work to review for a new reviewer is based on the number of previous reviews assigned for that submission which is incorrect.

===Implementation===

=====Drawbacks and Solutions=====
* '''Issue #402 ''': To allow reviewers a larger set of topics to choose from, the instructor can set the threshold (on the Review Strategy tab of assignment creation/editing) to some integer k > 0. Then any submission that has within k reviews of the fewest number can be chosen by a new reviewer. Let’s say that all submissions have at least 1 review. If k = 3, then the reviewer can choose any topic where there is a submission that has 4 or fewer reviews so far. Suppose that the minimum number of reviews for any submission is 2, but that I have reviewed all the submissions that have only 2 reviews. Then I’m not allowed to review at all (unless k > 0).
* '''Solution''': This issue has been fixed previously. The reviewer will get assigned a submission even if it has fulfilled the required number of reviews, to ensure that the reviewer always receives a new submission to review as long as it is not his own.
* '''Issue #969''':
a.The two statements "Set Allowed Number of Reviews per reviewer" and "Set Required Number of Reviews per reviewer." are not well differentiated. It can very confusing for an instructor to give values for them.

* '''Solution''': Information buttons have been provided beside these statements in the review strategy tab to make sure that the instructor knows the difference between the two.
* this modification is made in the app/views/assignments/edit/_review_strategy.html.erb
<pre>

<img src="/assets/info.png" title='This is the minimum number of reviews a reviewer has to perform'>
<br/>
<br/>
<%= check_box_tag('review', 'true', true, {:onChange => 'hasReviewChanged()'}) %>
<%= label_tag('review', 'Has max review limit?') %>&emsp;&emsp;
<span id="reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_reviews_allowed]', ' Set allowed number of reviews per reviewer') %>
<%if @assignment_form.assignment.num_reviews_allowed && @assignment_form.assignment.num_reviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', @assignment_form.assignment.num_reviews_allowed, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', 3, size: 1) %>
<%end%>

<img src="/assets/info.png" title='This is the maximum number of reviews a reviewer can perform'<%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
</span>

</td>
</pre>
[[File:infobuttons.png]]
[[File:infobutton2.png]]
b) When the number of allowed or required reviews is not set on the Review Strategy tab, the system does not have a message to display to a reviewer about how many submissions of work they are required to and allowed to review depending on the values set by the instructor in the "allowed number of reviewers per reviewer" field and the "Set Required Number of Reviews per reviewer." field.

* '''Solution''': This view has been implemented in the app/views/student_review/list.html.erb
<pre>

<h2>Reviews for "<%= @assignment.name %>"</h2>


<% if @assignment.num_reviews_allowed.nil? || @assignment.num_reviews_allowed == -1%>
<h4>Your instructor expects you to do <%= @assignment.num_reviews_required %> reviews. You are not allowed to do any extra reviews. </h4>

<% elsif @assignment.num_reviews_allowed == @assignment.num_reviews_required %>
<h4>You should perform exactly <%= @assignment.num_reviews_allowed %> reviews </h4>
<% else %>

<h4>You may perform between <%= @assignment.num_reviews_required %> and <%= @assignment.num_reviews_allowed %> reviews</h4>
<% end %>
</pre>

c) The capitalization and punctuation of statements such as "Set Allowed Number of Reviews per reviewer" and "Set Required Number of Reviews per reviewer." in review strategy tab are incorrect.
* '''Solution''': This view has been fixed by making changes in the respective files.
* The changes were made in the file : app/views/assignments/edit/_review_strategy.html.erb
<pre>
<%= label_tag('assignment_form[assignment][num_reviews_required]', 'Set required number of reviews per reviewer') %>
<%= text_field_tag('assignment_form[assignment][num_reviews_required]', @assignment_form.assignment.num_reviews_required ||= @assignment_form.assignment.num_reviews, size: 1) %>


<img src="/assets/info.png" title='This is the minimum number of reviews a reviewer has to perform'>
<br/>
<br/>
<%= check_box_tag('review', 'true', true, {:onChange => 'hasReviewChanged()'}) %>
<%= label_tag('review', 'Has max review limit?') %>&emsp;&emsp;
<span id="reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_reviews_allowed]', ' Set allowed number of reviews per reviewer') %>
<%if @assignment_form.assignment.num_reviews_allowed && @assignment_form.assignment.num_reviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', @assignment_form.assignment.num_reviews_allowed, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', 3, size: 1) %>
<%end%>

<img src="/assets/info.png" title='This is the maximum number of reviews a reviewer can perform'<%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
</span>

</td>
<td width="50%">

<%= label_tag('assignment_form[assignment][num_metareviews_required]', 'Set required number of meta-reviews per reviewer') %>
<%= text_field_tag('assignment_form[assignment][num_metareviews_required]', @assignment_form.assignment.num_metareviews_required ||= @assignment_form.assignment.num_metareviews_allowed, size: 1) %>
<br/>
<br/>

<%= check_box_tag('meta_review', 'true', true, {:onChange => 'hasMetaReviewChanged()'}) %>
<%= label_tag('meta_review', 'Has meta-review limit?') %>&emsp;&emsp;
<span id="meta_reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_metareviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_metareviews_allowed]', ' Set allowed number of meta-reviews per reviewer') %>
<%if @assignment_form.assignment.num_metareviews_allowed && @assignment_form.assignment.num_metareviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_metareviews_allowed]', @assignment_form.assignment.num_metareviews_allowed ||= 3, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_metareviews_allowed]', 3, size: 1) %>
<%end%>
</span>

</td>
</tr>

</pre>
[[File:uifix (2).png]]

* '''Problem 5''': The assignment of work to review to a reviewer depends on the number of reviews it has previously received, if that work has been assigned to a reviewer previously and its review has not been submitted then the system will still consider it such that it would not assign the same work to another reviewer if it has enough number of reviewers previously assigned to review it. This might lead to a scenario where that work was assigned to enough number of reviewers, but it does not receive enough reviews and cannot be assigned to any new reviewers either.
* '''Solution''': This implementation has been fixed. The system now considers only the reviews that were submitted by the reviewers for that work so that every submission gets assigned to reviewers until it receives its allowed number of reviews.

*the modified file is app/models/review_assignment.rb and the modification is done in the method reject_by_max_reviews_per_submission

<pre>
def reject_by_max_reviews_per_submission(contributor_set)
contributor_set.reject! {|contributor| contributor.responses.reject {|response| !response.is_submitted }.count >= max_reviews_per_submission }
contributor_set
end
</pre>

==Modified Files==
* app/views/assignments/edit/_review_strategy.html.erb
* app/models/review_assignment.rb
* app/views/student_review/list.html.erb

===References===

#[https://github.com/expertiza/expertiza Expertiza on GitHub]
#[https://github.com/WintersLt/expertiza GitHub Project Repository Fork]
#[http://expertiza.ncsu.edu/ The live Expertiza website]
#[http://wikis.lib.ncsu.edu/index.php/Expertiza Expertiza project documentation wiki]
#Clean Code: A handbook of agile software craftsmanship. Author: Robert C Martin

CSC/ECE 517 Fall 2017/E1777 Coherent specification of review requirements.rb

2017-10-30T20:25:18Z

Ssuresh6: /* Functionality */

==E1777. OSS Project Green: Coherent specification of review requirements==

This page provides a description of the Expertiza based OSS project.

__TOC__

===About Expertiza===

[http://expertiza.ncsu.edu/ Expertiza] is an open source project based on [http://rubyonrails.org/ Ruby on Rails] framework. Expertiza allows the instructor to create new assignments and customize new or existing assignments. It also allows the instructor to create a list of topics the students can sign up for. Students can form teams in Expertiza to work on various projects and assignments. Students can also peer review other students' submissions. Expertiza supports submission across various document types, including the URLs and wiki pages.

===Problem Statement===
The following tasks were accomplished in this project:

* The punctuation, syntax and the capitalization of statements in the review strategy tab requires improvement.
* Information buttons to elaborate difference between the synonymous statements present in review strategy tab need to be included.
* Message to a reviewer regarding how many reviews they are required to and allowed to perform is not clear enough.
* Assignment of work to review for a new reviewer is based on the number of previous reviews assigned for that submission which is incorrect.

===Implementation===

=====Drawbacks and Solutions=====
* '''Problem 1''': To allow reviewers a larger set of topics to choose from, the instructor can set the threshold (on the Review Strategy tab of assignment creation/editing) to some integer k > 0. Then any submission that has within k reviews of the fewest number can be chosen by a new reviewer. Let’s say that all submissions have at least 1 review. If k = 3, then the reviewer can choose any topic where there is a submission that has 4 or fewer reviews so far. Suppose that the minimum number of reviews for any submission is 2, but that I have reviewed all the submissions that have only 2 reviews. Then I’m not allowed to review at all (unless k > 0).
* '''Solution''': This issue has been fixed previously. The reviewer will get assigned a submission even if it has fulfilled the required number of reviews, to ensure that the reviewer always receives a new submission to review as long as it is not his own.
* '''Problem 2''': The two statements "Set Allowed Number of Reviews per reviewer" and "Set Required Number of Reviews per reviewer." are not well differentiated. It can very confusing for an instructor to give values for them.

* '''Solution''': Information buttons have been provided beside these statements in the review strategy tab to make sure that the instructor knows the difference between the two.
* this modification is made in the app/views/assignments/edit/_review_strategy.html.erb
<pre>

<img src="/assets/info.png" title='This is the minimum number of reviews a reviewer has to perform'>
<br/>
<br/>
<%= check_box_tag('review', 'true', true, {:onChange => 'hasReviewChanged()'}) %>
<%= label_tag('review', 'Has max review limit?') %>&emsp;&emsp;
<span id="reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_reviews_allowed]', ' Set allowed number of reviews per reviewer') %>
<%if @assignment_form.assignment.num_reviews_allowed && @assignment_form.assignment.num_reviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', @assignment_form.assignment.num_reviews_allowed, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', 3, size: 1) %>
<%end%>

<img src="/assets/info.png" title='This is the maximum number of reviews a reviewer can perform'<%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
</span>

</td>
</pre>
[[File:infobuttons.png]]
[[File:infobutton2.png]]
*'''Problem 3''': When the number of allowed or required reviews is not set on the Review Strategy tab, the system does not have a message to display to a reviewer about how many submissions of work they are required to and allowed to review depending on the values set by the instructor in the "allowed number of reviewers per reviewer" field and the "Set Required Number of Reviews per reviewer." field.

* '''Solution''': This view has been implemented in the app/views/student_review/list.html.erb
<pre>

<h2>Reviews for "<%= @assignment.name %>"</h2>


<% if @assignment.num_reviews_allowed.nil? || @assignment.num_reviews_allowed == -1%>
<h4>Your instructor expects you to do <%= @assignment.num_reviews_required %> reviews. You are not allowed to do any extra reviews. </h4>

<% elsif @assignment.num_reviews_allowed == @assignment.num_reviews_required %>
<h4>You should perform exactly <%= @assignment.num_reviews_allowed %> reviews </h4>
<% else %>

<h4>You may perform between <%= @assignment.num_reviews_required %> and <%= @assignment.num_reviews_allowed %> reviews</h4>
<% end %>
</pre>

* '''Problem 4''': The capitalization and punctuations of statements such as "Set Allowed Number of Reviews per reviewer" and "Set Required Number of Reviews per reviewer." in review strategy tab are incorrect.
* '''Solution''': This view has been fixed by making changes in the respective files.
* The changes were made in the file : app/views/assignments/edit/_review_strategy.html.erb
<pre>
<%= label_tag('assignment_form[assignment][num_reviews_required]', 'Set required number of reviews per reviewer') %>
<%= text_field_tag('assignment_form[assignment][num_reviews_required]', @assignment_form.assignment.num_reviews_required ||= @assignment_form.assignment.num_reviews, size: 1) %>


<img src="/assets/info.png" title='This is the minimum number of reviews a reviewer has to perform'>
<br/>
<br/>
<%= check_box_tag('review', 'true', true, {:onChange => 'hasReviewChanged()'}) %>
<%= label_tag('review', 'Has max review limit?') %>&emsp;&emsp;
<span id="reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_reviews_allowed]', ' Set allowed number of reviews per reviewer') %>
<%if @assignment_form.assignment.num_reviews_allowed && @assignment_form.assignment.num_reviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', @assignment_form.assignment.num_reviews_allowed, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', 3, size: 1) %>
<%end%>

<img src="/assets/info.png" title='This is the maximum number of reviews a reviewer can perform'<%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
</span>

</td>
<td width="50%">

<%= label_tag('assignment_form[assignment][num_metareviews_required]', 'Set required number of meta-reviews per reviewer') %>
<%= text_field_tag('assignment_form[assignment][num_metareviews_required]', @assignment_form.assignment.num_metareviews_required ||= @assignment_form.assignment.num_metareviews_allowed, size: 1) %>
<br/>
<br/>

<%= check_box_tag('meta_review', 'true', true, {:onChange => 'hasMetaReviewChanged()'}) %>
<%= label_tag('meta_review', 'Has meta-review limit?') %>&emsp;&emsp;
<span id="meta_reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_metareviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_metareviews_allowed]', ' Set allowed number of meta-reviews per reviewer') %>
<%if @assignment_form.assignment.num_metareviews_allowed && @assignment_form.assignment.num_metareviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_metareviews_allowed]', @assignment_form.assignment.num_metareviews_allowed ||= 3, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_metareviews_allowed]', 3, size: 1) %>
<%end%>
</span>

</td>
</tr>

</pre>
[[File:uifix (2).png]]

* '''Problem 5''': The assignment of work to review to a reviewer depends on the number of reviews it has previously received, if that work has been assigned to a reviewer previously and its review has not been submitted then the system will still consider it such that it would not assign the same work to another reviewer if it has enough number of reviewers previously assigned to review it. This might lead to a scenario where that work was assigned to enough number of reviewers, but it does not receive enough reviews and cannot be assigned to any new reviewers.
* '''Solution''': This implementation has been fixed. The system now considers only the reviews that were submitted by the reviewers for that work so that every submission gets assigned to reviewers until it receives its allowed number of reviews.

*the modified file is app/models/review_assignment.rb and the modification is done in the method reject_by_max_reviews_per_submission

<pre>
def reject_by_max_reviews_per_submission(contributor_set)
contributor_set.reject! {|contributor| contributor.responses.reject {|response| !response.is_submitted }.count >= max_reviews_per_submission }
contributor_set
end
</pre>

==Modified Files==
* app/views/assignments/edit/_review_strategy.html.erb
* app/models/review_assignment.rb
* app/views/student_review/list.html.erb

===References===

#[https://github.com/expertiza/expertiza Expertiza on GitHub]
#[https://github.com/WintersLt/expertiza GitHub Project Repository Fork]
#[http://expertiza.ncsu.edu/ The live Expertiza website]
#[http://wikis.lib.ncsu.edu/index.php/Expertiza Expertiza project documentation wiki]
#Clean Code: A handbook of agile software craftsmanship. Author: Robert C Martin

CSC/ECE 517 Fall 2017/E1777 Coherent specification of review requirements.rb

2017-10-30T20:23:17Z

Ssuresh6: /* Problem Statement */

==E1777. OSS Project Green: Coherent specification of review requirements==

This page provides a description of the Expertiza based OSS project.

__TOC__

===About Expertiza===

[http://expertiza.ncsu.edu/ Expertiza] is an open source project based on [http://rubyonrails.org/ Ruby on Rails] framework. Expertiza allows the instructor to create new assignments and customize new or existing assignments. It also allows the instructor to create a list of topics the students can sign up for. Students can form teams in Expertiza to work on various projects and assignments. Students can also peer review other students' submissions. Expertiza supports submission across various document types, including the URLs and wiki pages.

===Problem Statement===
The following tasks were accomplished in this project:

* The punctuation, syntax and the capitalization of statements in the review strategy tab requires improvement.
* Information buttons to elaborate difference between the synonymous statements present in review strategy tab need to be included.
* Message to a reviewer regarding how many reviews they are required to and allowed to perform is not clear enough.
* Assignment of work to review for a new reviewer is based on the number of previous reviews assigned for that submission which is incorrect.

===Implementation===

=====Functionality=====
* In Expertiza, there are two ways of assigning reviews to reviewers: either the instructor decides who reviews whom (“instructor-selected”), or “auto-selected,” in which case reviews are not assigned until a student seeks to choose something to review. To allow reviewers a larger set of topics to choose from, the instructor can set the threshold (on the Review Strategy tab of assignment creation/editing) to some integer k > 0. Then any submission that has within k reviews of the fewest number can be chosen by a new reviewer. Let’s say that all submissions have at least 1 review. If k = 3, then the reviewer can choose any topic where there is a submission that has 4 or fewer reviews so far. Suppose that the minimum number of reviews for any submission is 2, but that the reviewer has reviewed all the submissions that have only 2 reviews. Then he or she will not be allowed to review at all (unless k > 0).
* The capitalization and punctuation in many statements under the review strategy tab are random.
* On the Review Strategy tab of assignment creation, the "Maximum number of reviews per submission" checks the number of reviews that have been assigned to reviewers instead of checking the number of reviews that are submitted by the reviewers. This means that if required number of reviewers choose submissions and don't submit their reviews, those submissions cannot be reviewed any more.

=====Drawbacks and Solutions=====
* '''Problem 1''': To allow reviewers a larger set of topics to choose from, the instructor can set the threshold (on the Review Strategy tab of assignment creation/editing) to some integer k > 0. Then any submission that has within k reviews of the fewest number can be chosen by a new reviewer. Let’s say that all submissions have at least 1 review. If k = 3, then the reviewer can choose any topic where there is a submission that has 4 or fewer reviews so far. Suppose that the minimum number of reviews for any submission is 2, but that I have reviewed all the submissions that have only 2 reviews. Then I’m not allowed to review at all (unless k > 0).
* '''Solution''': This issue has been fixed previously. The reviewer will get assigned a submission even if it has fulfilled the required number of reviews, to ensure that the reviewer always receives a new submission to review as long as it is not his own.
* '''Problem 2''': The two statements "Set Allowed Number of Reviews per reviewer" and "Set Required Number of Reviews per reviewer." are not well differentiated. It can very confusing for an instructor to give values for them.

* '''Solution''': Information buttons have been provided beside these statements in the review strategy tab to make sure that the instructor knows the difference between the two.
* this modification is made in the app/views/assignments/edit/_review_strategy.html.erb
<pre>

<img src="/assets/info.png" title='This is the minimum number of reviews a reviewer has to perform'>
<br/>
<br/>
<%= check_box_tag('review', 'true', true, {:onChange => 'hasReviewChanged()'}) %>
<%= label_tag('review', 'Has max review limit?') %>&emsp;&emsp;
<span id="reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_reviews_allowed]', ' Set allowed number of reviews per reviewer') %>
<%if @assignment_form.assignment.num_reviews_allowed && @assignment_form.assignment.num_reviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', @assignment_form.assignment.num_reviews_allowed, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', 3, size: 1) %>
<%end%>

<img src="/assets/info.png" title='This is the maximum number of reviews a reviewer can perform'<%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
</span>

</td>
</pre>
[[File:infobuttons.png]]
[[File:infobutton2.png]]
*'''Problem 3''': When the number of allowed or required reviews is not set on the Review Strategy tab, the system does not have a message to display to a reviewer about how many submissions of work they are required to and allowed to review depending on the values set by the instructor in the "allowed number of reviewers per reviewer" field and the "Set Required Number of Reviews per reviewer." field.

* '''Solution''': This view has been implemented in the app/views/student_review/list.html.erb
<pre>

<h2>Reviews for "<%= @assignment.name %>"</h2>


<% if @assignment.num_reviews_allowed.nil? || @assignment.num_reviews_allowed == -1%>
<h4>Your instructor expects you to do <%= @assignment.num_reviews_required %> reviews. You are not allowed to do any extra reviews. </h4>

<% elsif @assignment.num_reviews_allowed == @assignment.num_reviews_required %>
<h4>You should perform exactly <%= @assignment.num_reviews_allowed %> reviews </h4>
<% else %>

<h4>You may perform between <%= @assignment.num_reviews_required %> and <%= @assignment.num_reviews_allowed %> reviews</h4>
<% end %>
</pre>

* '''Problem 4''': The capitalization and punctuations of statements such as "Set Allowed Number of Reviews per reviewer" and "Set Required Number of Reviews per reviewer." in review strategy tab are incorrect.
* '''Solution''': This view has been fixed by making changes in the respective files.
* The changes were made in the file : app/views/assignments/edit/_review_strategy.html.erb
<pre>
<%= label_tag('assignment_form[assignment][num_reviews_required]', 'Set required number of reviews per reviewer') %>
<%= text_field_tag('assignment_form[assignment][num_reviews_required]', @assignment_form.assignment.num_reviews_required ||= @assignment_form.assignment.num_reviews, size: 1) %>


<img src="/assets/info.png" title='This is the minimum number of reviews a reviewer has to perform'>
<br/>
<br/>
<%= check_box_tag('review', 'true', true, {:onChange => 'hasReviewChanged()'}) %>
<%= label_tag('review', 'Has max review limit?') %>&emsp;&emsp;
<span id="reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_reviews_allowed]', ' Set allowed number of reviews per reviewer') %>
<%if @assignment_form.assignment.num_reviews_allowed && @assignment_form.assignment.num_reviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', @assignment_form.assignment.num_reviews_allowed, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', 3, size: 1) %>
<%end%>

<img src="/assets/info.png" title='This is the maximum number of reviews a reviewer can perform'<%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
</span>

</td>
<td width="50%">

<%= label_tag('assignment_form[assignment][num_metareviews_required]', 'Set required number of meta-reviews per reviewer') %>
<%= text_field_tag('assignment_form[assignment][num_metareviews_required]', @assignment_form.assignment.num_metareviews_required ||= @assignment_form.assignment.num_metareviews_allowed, size: 1) %>
<br/>
<br/>

<%= check_box_tag('meta_review', 'true', true, {:onChange => 'hasMetaReviewChanged()'}) %>
<%= label_tag('meta_review', 'Has meta-review limit?') %>&emsp;&emsp;
<span id="meta_reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_metareviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_metareviews_allowed]', ' Set allowed number of meta-reviews per reviewer') %>
<%if @assignment_form.assignment.num_metareviews_allowed && @assignment_form.assignment.num_metareviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_metareviews_allowed]', @assignment_form.assignment.num_metareviews_allowed ||= 3, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_metareviews_allowed]', 3, size: 1) %>
<%end%>
</span>

</td>
</tr>

</pre>
[[File:uifix (2).png]]

* '''Problem 5''': The assignment of work to review to a reviewer depends on the number of reviews it has previously received, if that work has been assigned to a reviewer previously and its review has not been submitted then the system will still consider it such that it would not assign the same work to another reviewer if it has enough number of reviewers previously assigned to review it. This might lead to a scenario where that work was assigned to enough number of reviewers, but it does not receive enough reviews and cannot be assigned to any new reviewers.
* '''Solution''': This implementation has been fixed. The system now considers only the reviews that were submitted by the reviewers for that work so that every submission gets assigned to reviewers until it receives its allowed number of reviews.

*the modified file is app/models/review_assignment.rb and the modification is done in the method reject_by_max_reviews_per_submission

<pre>
def reject_by_max_reviews_per_submission(contributor_set)
contributor_set.reject! {|contributor| contributor.responses.reject {|response| !response.is_submitted }.count >= max_reviews_per_submission }
contributor_set
end
</pre>

==Modified Files==
* app/views/assignments/edit/_review_strategy.html.erb
* app/models/review_assignment.rb
* app/views/student_review/list.html.erb

===References===

#[https://github.com/expertiza/expertiza Expertiza on GitHub]
#[https://github.com/WintersLt/expertiza GitHub Project Repository Fork]
#[http://expertiza.ncsu.edu/ The live Expertiza website]
#[http://wikis.lib.ncsu.edu/index.php/Expertiza Expertiza project documentation wiki]
#Clean Code: A handbook of agile software craftsmanship. Author: Robert C Martin

CSC/ECE 517 Fall 2017/E1777 Coherent specification of review requirements.rb

2017-10-30T00:40:29Z

Ssuresh6: /* Modified File */

==E1777. OSS Project Green: Coherent specification of review requirements==

This page provides a description of the Expertiza based OSS project.

__TOC__

===About Expertiza===

[http://expertiza.ncsu.edu/ Expertiza] is an open source project based on [http://rubyonrails.org/ Ruby on Rails] framework. Expertiza allows the instructor to create new assignments and customize new or existing assignments. It also allows the instructor to create a list of topics the students can sign up for. Students can form teams in Expertiza to work on various projects and assignments. Students can also peer review other students' submissions. Expertiza supports submission across various document types, including the URLs and wiki pages.

===Problem Statement===
The following tasks were accomplished in this project:

* Improved the punctuation, syntax and the capitalization of statements in the review strategy tab.
* Provided information buttons to elaborate difference between the synonymous statements present in review strategy tab.
* Display of message to a reviewer regarding how many reviews they are required to and allowed to perform.
* Assignment of work to review for a new reviewer is now based on the number of previous reviews submitted for that work and not on the numbers of assigned reviews for that work.

===Implementation===

=====Functionality=====
* In Expertiza, there are two ways of assigning reviews to reviewers: either the instructor decides who reviews whom (“instructor-selected”), or “auto-selected,” in which case reviews are not assigned until a student seeks to choose something to review. To allow reviewers a larger set of topics to choose from, the instructor can set the threshold (on the Review Strategy tab of assignment creation/editing) to some integer k > 0. Then any submission that has within k reviews of the fewest number can be chosen by a new reviewer. Let’s say that all submissions have at least 1 review. If k = 3, then the reviewer can choose any topic where there is a submission that has 4 or fewer reviews so far. Suppose that the minimum number of reviews for any submission is 2, but that the reviewer has reviewed all the submissions that have only 2 reviews. Then he or she will not be allowed to review at all (unless k > 0).
* The capitalization and punctuation in many statements under the review strategy tab are random.
* On the Review Strategy tab of assignment creation, the "Maximum number of reviews per submission" checks the number of reviews that have been assigned to reviewers instead of checking the number of reviews that are submitted by the reviewers. This means that if required number of reviewers choose submissions and don't submit their reviews, those submissions cannot be reviewed any more.

=====Drawbacks and Solutions=====
* '''Problem 1''': To allow reviewers a larger set of topics to choose from, the instructor can set the threshold (on the Review Strategy tab of assignment creation/editing) to some integer k > 0. Then any submission that has within k reviews of the fewest number can be chosen by a new reviewer. Let’s say that all submissions have at least 1 review. If k = 3, then the reviewer can choose any topic where there is a submission that has 4 or fewer reviews so far. Suppose that the minimum number of reviews for any submission is 2, but that I have reviewed all the submissions that have only 2 reviews. Then I’m not allowed to review at all (unless k > 0).
* '''Solution''': This issue has been fixed previously. The reviewer will get assigned a submission even if it has fulfilled the required number of reviews, to ensure that the reviewer always receives a new submission to review as long as it is not his own.
* '''Problem 2''': The two statements "Set Allowed Number of Reviews per reviewer" and "Set Required Number of Reviews per reviewer." are not well differentiated. It can very confusing for an instructor to give values for them.

* '''Solution''': Information buttons have been provided beside these statements in the review strategy tab to make sure that the instructor knows the difference between the two.
* this modification is made in the app/views/assignments/edit/_review_strategy.html.erb
<pre>

<img src="/assets/info.png" title='This is the minimum number of reviews a reviewer has to perform'>
<br/>
<br/>
<%= check_box_tag('review', 'true', true, {:onChange => 'hasReviewChanged()'}) %>
<%= label_tag('review', 'Has max review limit?') %>&emsp;&emsp;
<span id="reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_reviews_allowed]', ' Set allowed number of reviews per reviewer') %>
<%if @assignment_form.assignment.num_reviews_allowed && @assignment_form.assignment.num_reviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', @assignment_form.assignment.num_reviews_allowed, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', 3, size: 1) %>
<%end%>

<img src="/assets/info.png" title='This is the maximum number of reviews a reviewer can perform'<%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
</span>

</td>
</pre>
[[File:infobuttons.png]]
[[File:infobutton2.png]]
*'''Problem 3''': When the number of allowed or required reviews is not set on the Review Strategy tab, the system does not have a message to display to a reviewer about how many submissions of work they are required to and allowed to review depending on the values set by the instructor in the "allowed number of reviewers per reviewer" field and the "Set Required Number of Reviews per reviewer." field.

* '''Solution''': This view has been implemented in the app/views/student_review/list.html.erb
<pre>

<h2>Reviews for "<%= @assignment.name %>"</h2>


<% if @assignment.num_reviews_allowed.nil? || @assignment.num_reviews_allowed == -1%>
<h4>Your instructor expects you to do <%= @assignment.num_reviews_required %> reviews. You are not allowed to do any extra reviews. </h4>

<% elsif @assignment.num_reviews_allowed == @assignment.num_reviews_required %>
<h4>You should perform exactly <%= @assignment.num_reviews_allowed %> reviews </h4>
<% else %>

<h4>You may perform between <%= @assignment.num_reviews_required %> and <%= @assignment.num_reviews_allowed %> reviews</h4>
<% end %>
</pre>

* '''Problem 4''': The capitalization and punctuations of statements such as "Set Allowed Number of Reviews per reviewer" and "Set Required Number of Reviews per reviewer." in review strategy tab are incorrect.
* '''Solution''': This view has been fixed by making changes in the respective files.
* The changes were made in the file : app/views/assignments/edit/_review_strategy.html.erb
<pre>
<%= label_tag('assignment_form[assignment][num_reviews_required]', 'Set required number of reviews per reviewer') %>
<%= text_field_tag('assignment_form[assignment][num_reviews_required]', @assignment_form.assignment.num_reviews_required ||= @assignment_form.assignment.num_reviews, size: 1) %>


<img src="/assets/info.png" title='This is the minimum number of reviews a reviewer has to perform'>
<br/>
<br/>
<%= check_box_tag('review', 'true', true, {:onChange => 'hasReviewChanged()'}) %>
<%= label_tag('review', 'Has max review limit?') %>&emsp;&emsp;
<span id="reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_reviews_allowed]', ' Set allowed number of reviews per reviewer') %>
<%if @assignment_form.assignment.num_reviews_allowed && @assignment_form.assignment.num_reviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', @assignment_form.assignment.num_reviews_allowed, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', 3, size: 1) %>
<%end%>

<img src="/assets/info.png" title='This is the maximum number of reviews a reviewer can perform'<%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
</span>

</td>
<td width="50%">

<%= label_tag('assignment_form[assignment][num_metareviews_required]', 'Set required number of meta-reviews per reviewer') %>
<%= text_field_tag('assignment_form[assignment][num_metareviews_required]', @assignment_form.assignment.num_metareviews_required ||= @assignment_form.assignment.num_metareviews_allowed, size: 1) %>
<br/>
<br/>

<%= check_box_tag('meta_review', 'true', true, {:onChange => 'hasMetaReviewChanged()'}) %>
<%= label_tag('meta_review', 'Has meta-review limit?') %>&emsp;&emsp;
<span id="meta_reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_metareviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_metareviews_allowed]', ' Set allowed number of meta-reviews per reviewer') %>
<%if @assignment_form.assignment.num_metareviews_allowed && @assignment_form.assignment.num_metareviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_metareviews_allowed]', @assignment_form.assignment.num_metareviews_allowed ||= 3, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_metareviews_allowed]', 3, size: 1) %>
<%end%>
</span>

</td>
</tr>

</pre>
[[File:uifix (2).png]]

* '''Problem 5''': The assignment of work to review to a reviewer depends on the number of reviews it has previously received, if that work has been assigned to a reviewer previously and its review has not been submitted then the system will still consider it such that it would not assign the same work to another reviewer if it has enough number of reviewers previously assigned to review it. This might lead to a scenario where that work was assigned to enough number of reviewers, but it does not receive enough reviews and cannot be assigned to any new reviewers.
* '''Solution''': This implementation has been fixed. The system now considers only the reviews that were submitted by the reviewers for that work so that every submission gets assigned to reviewers until it receives its allowed number of reviews.

*the modified file is app/models/review_assignment.rb and the modification is done in the method reject_by_max_reviews_per_submission

<pre>
def reject_by_max_reviews_per_submission(contributor_set)
contributor_set.reject! {|contributor| contributor.responses.reject {|response| !response.is_submitted }.count >= max_reviews_per_submission }
contributor_set
end
</pre>

==Modified Files==
* app/views/assignments/edit/_review_strategy.html.erb
* app/models/review_assignment.rb
* app/views/student_review/list.html.erb

===References===

#[https://github.com/expertiza/expertiza Expertiza on GitHub]
#[https://github.com/WintersLt/expertiza GitHub Project Repository Fork]
#[http://expertiza.ncsu.edu/ The live Expertiza website]
#[http://wikis.lib.ncsu.edu/index.php/Expertiza Expertiza project documentation wiki]
#Clean Code: A handbook of agile software craftsmanship. Author: Robert C Martin

CSC/ECE 517 Fall 2017/E1777 Coherent specification of review requirements.rb

2017-10-30T00:40:16Z

Ssuresh6: /* Drawbacks and Solutions */

==E1777. OSS Project Green: Coherent specification of review requirements==

This page provides a description of the Expertiza based OSS project.

__TOC__

===About Expertiza===

[http://expertiza.ncsu.edu/ Expertiza] is an open source project based on [http://rubyonrails.org/ Ruby on Rails] framework. Expertiza allows the instructor to create new assignments and customize new or existing assignments. It also allows the instructor to create a list of topics the students can sign up for. Students can form teams in Expertiza to work on various projects and assignments. Students can also peer review other students' submissions. Expertiza supports submission across various document types, including the URLs and wiki pages.

===Problem Statement===
The following tasks were accomplished in this project:

* Improved the punctuation, syntax and the capitalization of statements in the review strategy tab.
* Provided information buttons to elaborate difference between the synonymous statements present in review strategy tab.
* Display of message to a reviewer regarding how many reviews they are required to and allowed to perform.
* Assignment of work to review for a new reviewer is now based on the number of previous reviews submitted for that work and not on the numbers of assigned reviews for that work.

===Implementation===

=====Functionality=====
* In Expertiza, there are two ways of assigning reviews to reviewers: either the instructor decides who reviews whom (“instructor-selected”), or “auto-selected,” in which case reviews are not assigned until a student seeks to choose something to review. To allow reviewers a larger set of topics to choose from, the instructor can set the threshold (on the Review Strategy tab of assignment creation/editing) to some integer k > 0. Then any submission that has within k reviews of the fewest number can be chosen by a new reviewer. Let’s say that all submissions have at least 1 review. If k = 3, then the reviewer can choose any topic where there is a submission that has 4 or fewer reviews so far. Suppose that the minimum number of reviews for any submission is 2, but that the reviewer has reviewed all the submissions that have only 2 reviews. Then he or she will not be allowed to review at all (unless k > 0).
* The capitalization and punctuation in many statements under the review strategy tab are random.
* On the Review Strategy tab of assignment creation, the "Maximum number of reviews per submission" checks the number of reviews that have been assigned to reviewers instead of checking the number of reviews that are submitted by the reviewers. This means that if required number of reviewers choose submissions and don't submit their reviews, those submissions cannot be reviewed any more.

=====Drawbacks and Solutions=====
* '''Problem 1''': To allow reviewers a larger set of topics to choose from, the instructor can set the threshold (on the Review Strategy tab of assignment creation/editing) to some integer k > 0. Then any submission that has within k reviews of the fewest number can be chosen by a new reviewer. Let’s say that all submissions have at least 1 review. If k = 3, then the reviewer can choose any topic where there is a submission that has 4 or fewer reviews so far. Suppose that the minimum number of reviews for any submission is 2, but that I have reviewed all the submissions that have only 2 reviews. Then I’m not allowed to review at all (unless k > 0).
* '''Solution''': This issue has been fixed previously. The reviewer will get assigned a submission even if it has fulfilled the required number of reviews, to ensure that the reviewer always receives a new submission to review as long as it is not his own.
* '''Problem 2''': The two statements "Set Allowed Number of Reviews per reviewer" and "Set Required Number of Reviews per reviewer." are not well differentiated. It can very confusing for an instructor to give values for them.

* '''Solution''': Information buttons have been provided beside these statements in the review strategy tab to make sure that the instructor knows the difference between the two.
* this modification is made in the app/views/assignments/edit/_review_strategy.html.erb
<pre>

<img src="/assets/info.png" title='This is the minimum number of reviews a reviewer has to perform'>
<br/>
<br/>
<%= check_box_tag('review', 'true', true, {:onChange => 'hasReviewChanged()'}) %>
<%= label_tag('review', 'Has max review limit?') %>&emsp;&emsp;
<span id="reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_reviews_allowed]', ' Set allowed number of reviews per reviewer') %>
<%if @assignment_form.assignment.num_reviews_allowed && @assignment_form.assignment.num_reviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', @assignment_form.assignment.num_reviews_allowed, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', 3, size: 1) %>
<%end%>

<img src="/assets/info.png" title='This is the maximum number of reviews a reviewer can perform'<%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
</span>

</td>
</pre>
[[File:infobuttons.png]]
[[File:infobutton2.png]]
*'''Problem 3''': When the number of allowed or required reviews is not set on the Review Strategy tab, the system does not have a message to display to a reviewer about how many submissions of work they are required to and allowed to review depending on the values set by the instructor in the "allowed number of reviewers per reviewer" field and the "Set Required Number of Reviews per reviewer." field.

* '''Solution''': This view has been implemented in the app/views/student_review/list.html.erb
<pre>

<h2>Reviews for "<%= @assignment.name %>"</h2>


<% if @assignment.num_reviews_allowed.nil? || @assignment.num_reviews_allowed == -1%>
<h4>Your instructor expects you to do <%= @assignment.num_reviews_required %> reviews. You are not allowed to do any extra reviews. </h4>

<% elsif @assignment.num_reviews_allowed == @assignment.num_reviews_required %>
<h4>You should perform exactly <%= @assignment.num_reviews_allowed %> reviews </h4>
<% else %>

<h4>You may perform between <%= @assignment.num_reviews_required %> and <%= @assignment.num_reviews_allowed %> reviews</h4>
<% end %>
</pre>

* '''Problem 4''': The capitalization and punctuations of statements such as "Set Allowed Number of Reviews per reviewer" and "Set Required Number of Reviews per reviewer." in review strategy tab are incorrect.
* '''Solution''': This view has been fixed by making changes in the respective files.
* The changes were made in the file : app/views/assignments/edit/_review_strategy.html.erb
<pre>
<%= label_tag('assignment_form[assignment][num_reviews_required]', 'Set required number of reviews per reviewer') %>
<%= text_field_tag('assignment_form[assignment][num_reviews_required]', @assignment_form.assignment.num_reviews_required ||= @assignment_form.assignment.num_reviews, size: 1) %>


<img src="/assets/info.png" title='This is the minimum number of reviews a reviewer has to perform'>
<br/>
<br/>
<%= check_box_tag('review', 'true', true, {:onChange => 'hasReviewChanged()'}) %>
<%= label_tag('review', 'Has max review limit?') %>&emsp;&emsp;
<span id="reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_reviews_allowed]', ' Set allowed number of reviews per reviewer') %>
<%if @assignment_form.assignment.num_reviews_allowed && @assignment_form.assignment.num_reviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', @assignment_form.assignment.num_reviews_allowed, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', 3, size: 1) %>
<%end%>

<img src="/assets/info.png" title='This is the maximum number of reviews a reviewer can perform'<%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
</span>

</td>
<td width="50%">

<%= label_tag('assignment_form[assignment][num_metareviews_required]', 'Set required number of meta-reviews per reviewer') %>
<%= text_field_tag('assignment_form[assignment][num_metareviews_required]', @assignment_form.assignment.num_metareviews_required ||= @assignment_form.assignment.num_metareviews_allowed, size: 1) %>
<br/>
<br/>

<%= check_box_tag('meta_review', 'true', true, {:onChange => 'hasMetaReviewChanged()'}) %>
<%= label_tag('meta_review', 'Has meta-review limit?') %>&emsp;&emsp;
<span id="meta_reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_metareviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_metareviews_allowed]', ' Set allowed number of meta-reviews per reviewer') %>
<%if @assignment_form.assignment.num_metareviews_allowed && @assignment_form.assignment.num_metareviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_metareviews_allowed]', @assignment_form.assignment.num_metareviews_allowed ||= 3, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_metareviews_allowed]', 3, size: 1) %>
<%end%>
</span>

</td>
</tr>

</pre>
[[File:uifix (2).png]]

* '''Problem 5''': The assignment of work to review to a reviewer depends on the number of reviews it has previously received, if that work has been assigned to a reviewer previously and its review has not been submitted then the system will still consider it such that it would not assign the same work to another reviewer if it has enough number of reviewers previously assigned to review it. This might lead to a scenario where that work was assigned to enough number of reviewers, but it does not receive enough reviews and cannot be assigned to any new reviewers.
* '''Solution''': This implementation has been fixed. The system now considers only the reviews that were submitted by the reviewers for that work so that every submission gets assigned to reviewers until it receives its allowed number of reviews.

*the modified file is app/models/review_assignment.rb and the modification is done in the method reject_by_max_reviews_per_submission

<pre>
def reject_by_max_reviews_per_submission(contributor_set)
contributor_set.reject! {|contributor| contributor.responses.reject {|response| !response.is_submitted }.count >= max_reviews_per_submission }
contributor_set
end
</pre>

==Modified File==
* app/views/assignments/edit/_review_strategy.html.erb
* app/models/review_assignment.rb
* app/views/student_review/list.html.erb

===References===

#[https://github.com/expertiza/expertiza Expertiza on GitHub]
#[https://github.com/WintersLt/expertiza GitHub Project Repository Fork]
#[http://expertiza.ncsu.edu/ The live Expertiza website]
#[http://wikis.lib.ncsu.edu/index.php/Expertiza Expertiza project documentation wiki]
#Clean Code: A handbook of agile software craftsmanship. Author: Robert C Martin

CSC/ECE 517 Fall 2017/E1777 Coherent specification of review requirements.rb

2017-10-30T00:39:16Z

Ssuresh6: /* Modified File */

==E1777. OSS Project Green: Coherent specification of review requirements==

This page provides a description of the Expertiza based OSS project.

__TOC__

===About Expertiza===

[http://expertiza.ncsu.edu/ Expertiza] is an open source project based on [http://rubyonrails.org/ Ruby on Rails] framework. Expertiza allows the instructor to create new assignments and customize new or existing assignments. It also allows the instructor to create a list of topics the students can sign up for. Students can form teams in Expertiza to work on various projects and assignments. Students can also peer review other students' submissions. Expertiza supports submission across various document types, including the URLs and wiki pages.

===Problem Statement===
The following tasks were accomplished in this project:

* Improved the punctuation, syntax and the capitalization of statements in the review strategy tab.
* Provided information buttons to elaborate difference between the synonymous statements present in review strategy tab.
* Display of message to a reviewer regarding how many reviews they are required to and allowed to perform.
* Assignment of work to review for a new reviewer is now based on the number of previous reviews submitted for that work and not on the numbers of assigned reviews for that work.

===Implementation===

=====Functionality=====
* In Expertiza, there are two ways of assigning reviews to reviewers: either the instructor decides who reviews whom (“instructor-selected”), or “auto-selected,” in which case reviews are not assigned until a student seeks to choose something to review. To allow reviewers a larger set of topics to choose from, the instructor can set the threshold (on the Review Strategy tab of assignment creation/editing) to some integer k > 0. Then any submission that has within k reviews of the fewest number can be chosen by a new reviewer. Let’s say that all submissions have at least 1 review. If k = 3, then the reviewer can choose any topic where there is a submission that has 4 or fewer reviews so far. Suppose that the minimum number of reviews for any submission is 2, but that the reviewer has reviewed all the submissions that have only 2 reviews. Then he or she will not be allowed to review at all (unless k > 0).
* The capitalization and punctuation in many statements under the review strategy tab are random.
* On the Review Strategy tab of assignment creation, the "Maximum number of reviews per submission" checks the number of reviews that have been assigned to reviewers instead of checking the number of reviews that are submitted by the reviewers. This means that if required number of reviewers choose submissions and don't submit their reviews, those submissions cannot be reviewed any more.

=====Drawbacks and Solutions=====
* '''Problem 1''': To allow reviewers a larger set of topics to choose from, the instructor can set the threshold (on the Review Strategy tab of assignment creation/editing) to some integer k > 0. Then any submission that has within k reviews of the fewest number can be chosen by a new reviewer. Let’s say that all submissions have at least 1 review. If k = 3, then the reviewer can choose any topic where there is a submission that has 4 or fewer reviews so far. Suppose that the minimum number of reviews for any submission is 2, but that I have reviewed all the submissions that have only 2 reviews. Then I’m not allowed to review at all (unless k > 0).
* '''Solution''': This issue has been fixed previously. The reviewer will get assigned a submission even if it has fulfilled the required number of reviews, to ensure that the reviewer always receives a new submission to review as long as it is not his own.
* '''Problem 2''': The two statements "Set Allowed Number of Reviews per reviewer" and "Set Required Number of Reviews per reviewer." are not well differentiated. It can very confusing for an instructor to give values for them.

* '''Solution''': Information buttons have been provided beside these statements in the review strategy tab to make sure that the instructor knows the difference between the two.
* this modification is made in the app/views/assignments/edit/_review_strategy.html.erb
<pre>

<img src="/assets/info.png" title='This is the minimum number of reviews a reviewer has to perform'>
<br/>
<br/>
<%= check_box_tag('review', 'true', true, {:onChange => 'hasReviewChanged()'}) %>
<%= label_tag('review', 'Has max review limit?') %>&emsp;&emsp;
<span id="reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_reviews_allowed]', ' Set allowed number of reviews per reviewer') %>
<%if @assignment_form.assignment.num_reviews_allowed && @assignment_form.assignment.num_reviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', @assignment_form.assignment.num_reviews_allowed, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', 3, size: 1) %>
<%end%>

<img src="/assets/info.png" title='This is the maximum number of reviews a reviewer can perform'<%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
</span>

</td>
</pre>
[[File:infobuttons.png]]
[[File:infobutton2.png]]
*'''Problem 3''': When the number of allowed or required reviews is not set on the Review Strategy tab, the system does not have a message to display to a reviewer about how many submissions of work they are required to and allowed to review depending on the values set by the instructor in the "allowed number of reviewers per reviewer" field and the "Set Required Number of Reviews per reviewer." field.

* '''Solution''': This view has been implemented in the app/views/student_review/list.html.erb
<pre>

<h2>Reviews for "<%= @assignment.name %>"</h2>


<% if @assignment.num_reviews_allowed.nil? || @assignment.num_reviews_allowed == -1%>
<h4>Your instructor expects you to do <%= @assignment.num_reviews_required %> reviews. You are not allowed to do any extra reviews. </h4>

<% elsif @assignment.num_reviews_allowed == @assignment.num_reviews_required %>
<h4>You should perform exactly <%= @assignment.num_reviews_allowed %> reviews </h4>
<% else %>

<h4>You may perform between <%= @assignment.num_reviews_required %> and <%= @assignment.num_reviews_allowed %> reviews</h4>
<% end %>
</pre>

* '''Problem 4''': The capitalization and punctuations of statements such as "Set Allowed Number of Reviews per reviewer" and "Set Required Number of Reviews per reviewer." in review strategy tab are incorrect.
* '''Solution''': This view has been fixed by making changes in the respective files.
* The changes were made in the file : app/views/assignments/edit/_review_strategy.html.erb
<pre>
<%= label_tag('assignment_form[assignment][num_reviews_required]', 'Set required number of reviews per reviewer') %>
<%= text_field_tag('assignment_form[assignment][num_reviews_required]', @assignment_form.assignment.num_reviews_required ||= @assignment_form.assignment.num_reviews, size: 1) %>


<img src="/assets/info.png" title='This is the minimum number of reviews a reviewer has to perform'>
<br/>
<br/>
<%= check_box_tag('review', 'true', true, {:onChange => 'hasReviewChanged()'}) %>
<%= label_tag('review', 'Has max review limit?') %>&emsp;&emsp;
<span id="reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_reviews_allowed]', ' Set allowed number of reviews per reviewer') %>
<%if @assignment_form.assignment.num_reviews_allowed && @assignment_form.assignment.num_reviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', @assignment_form.assignment.num_reviews_allowed, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', 3, size: 1) %>
<%end%>

<img src="/assets/info.png" title='This is the maximum number of reviews a reviewer can perform'<%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
</span>

</td>
<td width="50%">

<%= label_tag('assignment_form[assignment][num_metareviews_required]', 'Set required number of meta-reviews per reviewer') %>
<%= text_field_tag('assignment_form[assignment][num_metareviews_required]', @assignment_form.assignment.num_metareviews_required ||= @assignment_form.assignment.num_metareviews_allowed, size: 1) %>
<br/>
<br/>

<%= check_box_tag('meta_review', 'true', true, {:onChange => 'hasMetaReviewChanged()'}) %>
<%= label_tag('meta_review', 'Has meta-review limit?') %>&emsp;&emsp;
<span id="meta_reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_metareviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_metareviews_allowed]', ' Set allowed number of meta-reviews per reviewer') %>
<%if @assignment_form.assignment.num_metareviews_allowed && @assignment_form.assignment.num_metareviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_metareviews_allowed]', @assignment_form.assignment.num_metareviews_allowed ||= 3, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_metareviews_allowed]', 3, size: 1) %>
<%end%>
</span>

</td>
</tr>

</pre>
[[File:uifix (2).png]]

* '''Problem 5''': The assignment of work to review to a reviewer depends on the number of reviews it has previously received, if that work has been assigned to a reviewer previously and its review has not been submitted then the system will still consider it such that it would not assign the same work to another reviewer if it has enough number of reviewers previously assigned to review it. This might lead to a scenario where that work was assigned to enough number of reviewers, but it does not receive enough reviews and cannot be assigned to any new reviewers.
* '''Solution''': This implementation has been fixed. The system now considers only the reviews that were submitted by the reviewers for that work so that every submission gets assigned to reviewers until it receives its allowed number of reviews.

*the modified file is app/models/review_assignment.rb and the modification is done in the method reject_by_max_reviews_per_submission

<pre>
def reject_by_max_reviews_per_submission(contributor_set)
contributor_set.reject! {|contributor| contributor.responses.reject {|response| !response.is_submitted }.count >= max_reviews_per_submission }
contributor_set
end
</pre>

'''==Modified File=='''
* app/views/assignments/edit/_review_strategy.html.erb
* app/models/review_assignment.rb
* app/views/student_review/list.html.erb

'''===References==='''

#[https://github.com/expertiza/expertiza Expertiza on GitHub]
#[https://github.com/WintersLt/expertiza GitHub Project Repository Fork]
#[http://expertiza.ncsu.edu/ The live Expertiza website]
#[http://wikis.lib.ncsu.edu/index.php/Expertiza Expertiza project documentation wiki]
#Clean Code: A handbook of agile software craftsmanship. Author: Robert C Martin

CSC/ECE 517 Fall 2017/E1777 Coherent specification of review requirements.rb

2017-10-30T00:38:46Z

Ssuresh6: /* Drawbacks and Solutions */

==E1777. OSS Project Green: Coherent specification of review requirements==

This page provides a description of the Expertiza based OSS project.

__TOC__

===About Expertiza===

[http://expertiza.ncsu.edu/ Expertiza] is an open source project based on [http://rubyonrails.org/ Ruby on Rails] framework. Expertiza allows the instructor to create new assignments and customize new or existing assignments. It also allows the instructor to create a list of topics the students can sign up for. Students can form teams in Expertiza to work on various projects and assignments. Students can also peer review other students' submissions. Expertiza supports submission across various document types, including the URLs and wiki pages.

===Problem Statement===
The following tasks were accomplished in this project:

* Improved the punctuation, syntax and the capitalization of statements in the review strategy tab.
* Provided information buttons to elaborate difference between the synonymous statements present in review strategy tab.
* Display of message to a reviewer regarding how many reviews they are required to and allowed to perform.
* Assignment of work to review for a new reviewer is now based on the number of previous reviews submitted for that work and not on the numbers of assigned reviews for that work.

===Implementation===

=====Functionality=====
* In Expertiza, there are two ways of assigning reviews to reviewers: either the instructor decides who reviews whom (“instructor-selected”), or “auto-selected,” in which case reviews are not assigned until a student seeks to choose something to review. To allow reviewers a larger set of topics to choose from, the instructor can set the threshold (on the Review Strategy tab of assignment creation/editing) to some integer k > 0. Then any submission that has within k reviews of the fewest number can be chosen by a new reviewer. Let’s say that all submissions have at least 1 review. If k = 3, then the reviewer can choose any topic where there is a submission that has 4 or fewer reviews so far. Suppose that the minimum number of reviews for any submission is 2, but that the reviewer has reviewed all the submissions that have only 2 reviews. Then he or she will not be allowed to review at all (unless k > 0).
* The capitalization and punctuation in many statements under the review strategy tab are random.
* On the Review Strategy tab of assignment creation, the "Maximum number of reviews per submission" checks the number of reviews that have been assigned to reviewers instead of checking the number of reviews that are submitted by the reviewers. This means that if required number of reviewers choose submissions and don't submit their reviews, those submissions cannot be reviewed any more.

=====Drawbacks and Solutions=====
* '''Problem 1''': To allow reviewers a larger set of topics to choose from, the instructor can set the threshold (on the Review Strategy tab of assignment creation/editing) to some integer k > 0. Then any submission that has within k reviews of the fewest number can be chosen by a new reviewer. Let’s say that all submissions have at least 1 review. If k = 3, then the reviewer can choose any topic where there is a submission that has 4 or fewer reviews so far. Suppose that the minimum number of reviews for any submission is 2, but that I have reviewed all the submissions that have only 2 reviews. Then I’m not allowed to review at all (unless k > 0).
* '''Solution''': This issue has been fixed previously. The reviewer will get assigned a submission even if it has fulfilled the required number of reviews, to ensure that the reviewer always receives a new submission to review as long as it is not his own.
* '''Problem 2''': The two statements "Set Allowed Number of Reviews per reviewer" and "Set Required Number of Reviews per reviewer." are not well differentiated. It can very confusing for an instructor to give values for them.

* '''Solution''': Information buttons have been provided beside these statements in the review strategy tab to make sure that the instructor knows the difference between the two.
* this modification is made in the app/views/assignments/edit/_review_strategy.html.erb
<pre>

<img src="/assets/info.png" title='This is the minimum number of reviews a reviewer has to perform'>
<br/>
<br/>
<%= check_box_tag('review', 'true', true, {:onChange => 'hasReviewChanged()'}) %>
<%= label_tag('review', 'Has max review limit?') %>&emsp;&emsp;
<span id="reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_reviews_allowed]', ' Set allowed number of reviews per reviewer') %>
<%if @assignment_form.assignment.num_reviews_allowed && @assignment_form.assignment.num_reviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', @assignment_form.assignment.num_reviews_allowed, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', 3, size: 1) %>
<%end%>

<img src="/assets/info.png" title='This is the maximum number of reviews a reviewer can perform'<%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
</span>

</td>
</pre>
[[File:infobuttons.png]]
[[File:infobutton2.png]]
*'''Problem 3''': When the number of allowed or required reviews is not set on the Review Strategy tab, the system does not have a message to display to a reviewer about how many submissions of work they are required to and allowed to review depending on the values set by the instructor in the "allowed number of reviewers per reviewer" field and the "Set Required Number of Reviews per reviewer." field.

* '''Solution''': This view has been implemented in the app/views/student_review/list.html.erb
<pre>

<h2>Reviews for "<%= @assignment.name %>"</h2>


<% if @assignment.num_reviews_allowed.nil? || @assignment.num_reviews_allowed == -1%>
<h4>Your instructor expects you to do <%= @assignment.num_reviews_required %> reviews. You are not allowed to do any extra reviews. </h4>

<% elsif @assignment.num_reviews_allowed == @assignment.num_reviews_required %>
<h4>You should perform exactly <%= @assignment.num_reviews_allowed %> reviews </h4>
<% else %>

<h4>You may perform between <%= @assignment.num_reviews_required %> and <%= @assignment.num_reviews_allowed %> reviews</h4>
<% end %>
</pre>

* '''Problem 4''': The capitalization and punctuations of statements such as "Set Allowed Number of Reviews per reviewer" and "Set Required Number of Reviews per reviewer." in review strategy tab are incorrect.
* '''Solution''': This view has been fixed by making changes in the respective files.
* The changes were made in the file : app/views/assignments/edit/_review_strategy.html.erb
<pre>
<%= label_tag('assignment_form[assignment][num_reviews_required]', 'Set required number of reviews per reviewer') %>
<%= text_field_tag('assignment_form[assignment][num_reviews_required]', @assignment_form.assignment.num_reviews_required ||= @assignment_form.assignment.num_reviews, size: 1) %>


<img src="/assets/info.png" title='This is the minimum number of reviews a reviewer has to perform'>
<br/>
<br/>
<%= check_box_tag('review', 'true', true, {:onChange => 'hasReviewChanged()'}) %>
<%= label_tag('review', 'Has max review limit?') %>&emsp;&emsp;
<span id="reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_reviews_allowed]', ' Set allowed number of reviews per reviewer') %>
<%if @assignment_form.assignment.num_reviews_allowed && @assignment_form.assignment.num_reviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', @assignment_form.assignment.num_reviews_allowed, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', 3, size: 1) %>
<%end%>

<img src="/assets/info.png" title='This is the maximum number of reviews a reviewer can perform'<%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
</span>

</td>
<td width="50%">

<%= label_tag('assignment_form[assignment][num_metareviews_required]', 'Set required number of meta-reviews per reviewer') %>
<%= text_field_tag('assignment_form[assignment][num_metareviews_required]', @assignment_form.assignment.num_metareviews_required ||= @assignment_form.assignment.num_metareviews_allowed, size: 1) %>
<br/>
<br/>

<%= check_box_tag('meta_review', 'true', true, {:onChange => 'hasMetaReviewChanged()'}) %>
<%= label_tag('meta_review', 'Has meta-review limit?') %>&emsp;&emsp;
<span id="meta_reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_metareviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_metareviews_allowed]', ' Set allowed number of meta-reviews per reviewer') %>
<%if @assignment_form.assignment.num_metareviews_allowed && @assignment_form.assignment.num_metareviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_metareviews_allowed]', @assignment_form.assignment.num_metareviews_allowed ||= 3, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_metareviews_allowed]', 3, size: 1) %>
<%end%>
</span>

</td>
</tr>

</pre>
[[File:uifix (2).png]]

* '''Problem 5''': The assignment of work to review to a reviewer depends on the number of reviews it has previously received, if that work has been assigned to a reviewer previously and its review has not been submitted then the system will still consider it such that it would not assign the same work to another reviewer if it has enough number of reviewers previously assigned to review it. This might lead to a scenario where that work was assigned to enough number of reviewers, but it does not receive enough reviews and cannot be assigned to any new reviewers.
* '''Solution''': This implementation has been fixed. The system now considers only the reviews that were submitted by the reviewers for that work so that every submission gets assigned to reviewers until it receives its allowed number of reviews.

*the modified file is app/models/review_assignment.rb and the modification is done in the method reject_by_max_reviews_per_submission

<pre>
def reject_by_max_reviews_per_submission(contributor_set)
contributor_set.reject! {|contributor| contributor.responses.reject {|response| !response.is_submitted }.count >= max_reviews_per_submission }
contributor_set
end
</pre>

==Modified File==
* app/views/assignments/edit/_review_strategy.html.erb
* app/models/review_assignment.rb
* app/views/student_review/list.html.erb

===References===

#[https://github.com/expertiza/expertiza Expertiza on GitHub]
#[https://github.com/WintersLt/expertiza GitHub Project Repository Fork]
#[http://expertiza.ncsu.edu/ The live Expertiza website]
#[http://wikis.lib.ncsu.edu/index.php/Expertiza Expertiza project documentation wiki]
#Clean Code: A handbook of agile software craftsmanship. Author: Robert C Martin

CSC/ECE 517 Fall 2017

2017-10-28T17:59:46Z

Ssuresh6:

'''Writing Assignment 2'''
*[[CSC/ECE 517 Fall 2017/E1779. Fix teammate advertisements and requests to join a team]]
*[[CSC/ECE_517_Fall_2017/E1773 Investigate and Fix Expertiza Production Version Runtime Exceptions.rb]]
*[[CSC/ECE_517_Fall_2017/E1774 Metareview fixes and improvements.rb]]
*[[CSC/ECE_517_Fall_2017/E1788 OSS project Maroon Heatmap fixes]]
*[[CSC/ECE 517 Fall 2017/E1781 Topic Management]]
*[[CSC/ECE_517_Fall_2017/E1745_Refactor_response_controller.rb]]
*[[CSC/ECE_517_Fall_2017/E1752 Refactor assignments controller]]
*[[CSC/ECE_517_Fall_2017/E1776_Enhance_Imports]]
*[[CSC/ECE_517_Fall_2017/E1756 TLD Refactor response.rb]]
*[[CSC/ECE_517_Fall_2017/E1772_Refactor reputation_web_service_controller.rb]]
*[[CSC/ECE_517_Fall_2017/M1754_Mutation Testing on Servo]]
*[[CSC/ECE_517_Fall_2017/E1753 OSS project bidding tests]]
*[[CSC/ECE_517_Fall_2017/E1766_Test team functionality]]
*[[CSC/ECE_517_Fall_2017/E1787_OSS project Bronze Score calculations]]
*[[CSC/ECE_517_Fall_2017/E1763 Fix Staggered-Deadline Assignments]]
*[[CSC/ECE 517 Fall 2017/E1757 Introduce a Student View for instructors]]
*[[CSC/ECE 517 Fall 2017/E1769_Refactor assignment_form.rb]]
*[[CSC/ECE 517 Fall 2017/E1754_Feature_test_of_rubric_advice]]
*[[CSC/ECE 517 Fall 2017/E1782 OSS Project Red Assignment Directories]]
*[[CSC/ECE 517 Fall 2017/E1748 Add past-due assignments to task list]]
*[[CSC/ECE 517 Fall 2017/E1761 Test First Development Refactor assignment.rb]]
*[[CSC/ECE_517_Fall_2017/E1780_OSS_Project_Teal_Email_Notification_Enhancements]]
*[[CSC/ECE_517_Fall_2017/E1762 Test various kinds of response-map hierarchies]]
*[[CSC/ECE_517_Fall_2017/E1759 ]]
*[[CSC/ECE 517 Fall 2017/E1777 Coherent specification of review requirements.rb]]

CSC/ECE 517 Fall 2017/E1777 Coherent specification of review requirements.rb

2017-10-28T04:51:57Z

Ssuresh6: /* Drawbacks and Solutions */

==E1777. OSS Project Green: Coherent specification of review requirements==

This page provides a description of the Expertiza based OSS project.

__TOC__

===About Expertiza===

[http://expertiza.ncsu.edu/ Expertiza] is an open source project based on [http://rubyonrails.org/ Ruby on Rails] framework. Expertiza allows the instructor to create new assignments and customize new or existing assignments. It also allows the instructor to create a list of topics the students can sign up for. Students can form teams in Expertiza to work on various projects and assignments. Students can also peer review other students' submissions. Expertiza supports submission across various document types, including the URLs and wiki pages.

===Problem Statement===
The following tasks were accomplished in this project:

* Improved the punctuation, syntax and the capitalization of statements in the review strategy tab.
* Provided information buttons to elaborate difference between the synonymous statements present in review strategy tab.
* Display of message to a reviewer regarding how many reviews they are required to and allowed to perform.
* Assignment of work to review for a new reviewer is now based on the number of previous reviews submitted for that work and not on the numbers of assigned reviews for that work.

===Implementation===

=====Functionality=====
* In Expertiza, there are two ways of assigning reviews to reviewers: either the instructor decides who reviews whom (“instructor-selected”), or “auto-selected,” in which case reviews are not assigned until a student seeks to choose something to review. To allow reviewers a larger set of topics to choose from, the instructor can set the threshold (on the Review Strategy tab of assignment creation/editing) to some integer k > 0. Then any submission that has within k reviews of the fewest number can be chosen by a new reviewer. Let’s say that all submissions have at least 1 review. If k = 3, then the reviewer can choose any topic where there is a submission that has 4 or fewer reviews so far. Suppose that the minimum number of reviews for any submission is 2, but that the reviewer has reviewed all the submissions that have only 2 reviews. Then he or she will not be allowed to review at all (unless k > 0).
* The capitalization and punctuation in many statements under the review strategy tab are random.
* On the Review Strategy tab of assignment creation, the "Maximum number of reviews per submission" checks the number of reviews that have been assigned to reviewers instead of checking the number of reviews that are submitted by the reviewers. This means that if required number of reviewers choose submissions and don't submit their reviews, those submissions cannot be reviewed any more.

=====Drawbacks and Solutions=====
* '''Problem 1''': To allow reviewers a larger set of topics to choose from, the instructor can set the threshold (on the Review Strategy tab of assignment creation/editing) to some integer k > 0. Then any submission that has within k reviews of the fewest number can be chosen by a new reviewer. Let’s say that all submissions have at least 1 review. If k = 3, then the reviewer can choose any topic where there is a submission that has 4 or fewer reviews so far. Suppose that the minimum number of reviews for any submission is 2, but that I have reviewed all the submissions that have only 2 reviews. Then I’m not allowed to review at all (unless k > 0).
* '''Solution''': This issue has been fixed previously. The reviewer will get assigned a submission even if it has fulfilled the required number of reviews, to ensure that the reviewer always receives a new submission to review as long as it is not his own.
* '''Problem 2''': The two statements "Set Allowed Number of Reviews per reviewer" and "Set Required Number of Reviews per reviewer." are not well differentiated. It can very confusing for an instructor to give values for them.

* '''Solution''': Information buttons have been provided beside these statements in the review strategy tab to make sure that the instructor knows the difference between the two.
* this modification is made in the app/views/assignments/edit/_review_strategy.html.erb
<pre>

<img src="/assets/info.png" title='This is the minimum number of reviews a reviewer has to perform'>
<br/>
<br/>
<%= check_box_tag('review', 'true', true, {:onChange => 'hasReviewChanged()'}) %>
<%= label_tag('review', 'Has max review limit?') %>&emsp;&emsp;
<span id="reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_reviews_allowed]', ' Set allowed number of reviews per reviewer') %>
<%if @assignment_form.assignment.num_reviews_allowed && @assignment_form.assignment.num_reviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', @assignment_form.assignment.num_reviews_allowed, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', 3, size: 1) %>
<%end%>

<img src="/assets/info.png" title='This is the maximum number of reviews a reviewer can perform'<%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
</span>

</td>
</pre>
[[File:infobuttons.png? maxwidth=600&maxheight=600]]
[[File:infobutton2.png]]
*'''Problem 3''': When the number of allowed or required reviews is not set on the Review Strategy tab, the system does not have a message to display to a reviewer about how many submissions of work they are required to and allowed to review depending on the values set by the instructor in the "allowed number of reviewers per reviewer" field and the "Set Required Number of Reviews per reviewer." field.

* '''Solution''': This view has been implemented in the app/views/student_review/list.html.erb
<pre>

<h2>Reviews for "<%= @assignment.name %>"</h2>


<% if @assignment.num_reviews_allowed.nil? || @assignment.num_reviews_allowed == -1%>
<h4>Your instructor expects you to do <%= @assignment.num_reviews_required %> reviews. You are not allowed to do any extra reviews. </h4>

<% elsif @assignment.num_reviews_allowed == @assignment.num_reviews_required %>
<h4>You should perform exactly <%= @assignment.num_reviews_allowed %> reviews </h4>
<% else %>

<h4>You may perform between <%= @assignment.num_reviews_required %> and <%= @assignment.num_reviews_allowed %> reviews</h4>
<% end %>
</pre>

* '''Problem 4''': The capitalization and punctuations of statements such as "Set Allowed Number of Reviews per reviewer" and "Set Required Number of Reviews per reviewer." in review strategy tab are incorrect.
* '''Solution''': This view has been fixed by making changes in the respective files.
* The changes were made in the file : app/views/assignments/edit/_review_strategy.html.erb
<pre>
<%= label_tag('assignment_form[assignment][num_reviews_required]', 'Set required number of reviews per reviewer') %>
<%= text_field_tag('assignment_form[assignment][num_reviews_required]', @assignment_form.assignment.num_reviews_required ||= @assignment_form.assignment.num_reviews, size: 1) %>


<img src="/assets/info.png" title='This is the minimum number of reviews a reviewer has to perform'>
<br/>
<br/>
<%= check_box_tag('review', 'true', true, {:onChange => 'hasReviewChanged()'}) %>
<%= label_tag('review', 'Has max review limit?') %>&emsp;&emsp;
<span id="reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_reviews_allowed]', ' Set allowed number of reviews per reviewer') %>
<%if @assignment_form.assignment.num_reviews_allowed && @assignment_form.assignment.num_reviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', @assignment_form.assignment.num_reviews_allowed, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', 3, size: 1) %>
<%end%>

<img src="/assets/info.png" title='This is the maximum number of reviews a reviewer can perform'<%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
</span>

</td>
<td width="50%">

<%= label_tag('assignment_form[assignment][num_metareviews_required]', 'Set required number of meta-reviews per reviewer') %>
<%= text_field_tag('assignment_form[assignment][num_metareviews_required]', @assignment_form.assignment.num_metareviews_required ||= @assignment_form.assignment.num_metareviews_allowed, size: 1) %>
<br/>
<br/>

<%= check_box_tag('meta_review', 'true', true, {:onChange => 'hasMetaReviewChanged()'}) %>
<%= label_tag('meta_review', 'Has meta-review limit?') %>&emsp;&emsp;
<span id="meta_reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_metareviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_metareviews_allowed]', ' Set allowed number of meta-reviews per reviewer') %>
<%if @assignment_form.assignment.num_metareviews_allowed && @assignment_form.assignment.num_metareviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_metareviews_allowed]', @assignment_form.assignment.num_metareviews_allowed ||= 3, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_metareviews_allowed]', 3, size: 1) %>
<%end%>
</span>

</td>
</tr>

</pre>
[[File:uifix (2).png]]

* '''Problem 5''': The assignment of work to review to a reviewer depends on the number of reviews it has previously received, if that work has been assigned to a reviewer previously and its review has not been submitted then the system will still consider it such that it would not assign the same work to another reviewer if it has enough number of reviewers previously assigned to review it. This might lead to a scenario where that work was assigned to enough number of reviewers, but it does not receive enough reviews and cannot be assigned to any new reviewers.
* '''Solution''': This implementation has been fixed. The system now considers only the reviews that were submitted by the reviewers for that work so that every submission gets assigned to reviewers until it receives its allowed number of reviews.

*the modified file is app/models/review_assignment.rb and the modification is done in the method reject_by_max_reviews_per_submission

<pre>
def reject_by_max_reviews_per_submission(contributor_set)
contributor_set.reject! {|contributor| contributor.responses.reject {|response| !response.is_submitted }.count >= max_reviews_per_submission }
contributor_set
end
</pre>

==Modified File==
* app/views/assignments/edit/_review_strategy.html.erb
* app/models/review_assignment.rb
* app/views/student_review/list.html.erb

===References===

#[https://github.com/expertiza/expertiza Expertiza on GitHub]
#[https://github.com/WintersLt/expertiza GitHub Project Repository Fork]
#[http://expertiza.ncsu.edu/ The live Expertiza website]
#[http://wikis.lib.ncsu.edu/index.php/Expertiza Expertiza project documentation wiki]
#Clean Code: A handbook of agile software craftsmanship. Author: Robert C Martin

CSC/ECE 517 Fall 2017/E1777 Coherent specification of review requirements.rb

2017-10-28T04:51:39Z

Ssuresh6: /* Drawbacks and Solutions */

==E1777. OSS Project Green: Coherent specification of review requirements==

This page provides a description of the Expertiza based OSS project.

__TOC__

===About Expertiza===

[http://expertiza.ncsu.edu/ Expertiza] is an open source project based on [http://rubyonrails.org/ Ruby on Rails] framework. Expertiza allows the instructor to create new assignments and customize new or existing assignments. It also allows the instructor to create a list of topics the students can sign up for. Students can form teams in Expertiza to work on various projects and assignments. Students can also peer review other students' submissions. Expertiza supports submission across various document types, including the URLs and wiki pages.

===Problem Statement===
The following tasks were accomplished in this project:

* Improved the punctuation, syntax and the capitalization of statements in the review strategy tab.
* Provided information buttons to elaborate difference between the synonymous statements present in review strategy tab.
* Display of message to a reviewer regarding how many reviews they are required to and allowed to perform.
* Assignment of work to review for a new reviewer is now based on the number of previous reviews submitted for that work and not on the numbers of assigned reviews for that work.

===Implementation===

=====Functionality=====
* In Expertiza, there are two ways of assigning reviews to reviewers: either the instructor decides who reviews whom (“instructor-selected”), or “auto-selected,” in which case reviews are not assigned until a student seeks to choose something to review. To allow reviewers a larger set of topics to choose from, the instructor can set the threshold (on the Review Strategy tab of assignment creation/editing) to some integer k > 0. Then any submission that has within k reviews of the fewest number can be chosen by a new reviewer. Let’s say that all submissions have at least 1 review. If k = 3, then the reviewer can choose any topic where there is a submission that has 4 or fewer reviews so far. Suppose that the minimum number of reviews for any submission is 2, but that the reviewer has reviewed all the submissions that have only 2 reviews. Then he or she will not be allowed to review at all (unless k > 0).
* The capitalization and punctuation in many statements under the review strategy tab are random.
* On the Review Strategy tab of assignment creation, the "Maximum number of reviews per submission" checks the number of reviews that have been assigned to reviewers instead of checking the number of reviews that are submitted by the reviewers. This means that if required number of reviewers choose submissions and don't submit their reviews, those submissions cannot be reviewed any more.

=====Drawbacks and Solutions=====
* '''Problem 1''': To allow reviewers a larger set of topics to choose from, the instructor can set the threshold (on the Review Strategy tab of assignment creation/editing) to some integer k > 0. Then any submission that has within k reviews of the fewest number can be chosen by a new reviewer. Let’s say that all submissions have at least 1 review. If k = 3, then the reviewer can choose any topic where there is a submission that has 4 or fewer reviews so far. Suppose that the minimum number of reviews for any submission is 2, but that I have reviewed all the submissions that have only 2 reviews. Then I’m not allowed to review at all (unless k > 0).
* '''Solution''': This issue has been fixed previously. The reviewer will get assigned a submission even if it has fulfilled the required number of reviews, to ensure that the reviewer always receives a new submission to review as long as it is not his own.
* '''Problem 2''': The two statements "Set Allowed Number of Reviews per reviewer" and "Set Required Number of Reviews per reviewer." are not well differentiated. It can very confusing for an instructor to give values for them.

* '''Solution''': Information buttons have been provided beside these statements in the review strategy tab to make sure that the instructor knows the difference between the two.
* this modification is made in the app/views/assignments/edit/_review_strategy.html.erb
<pre>

<img src="/assets/info.png" title='This is the minimum number of reviews a reviewer has to perform'>
<br/>
<br/>
<%= check_box_tag('review', 'true', true, {:onChange => 'hasReviewChanged()'}) %>
<%= label_tag('review', 'Has max review limit?') %>&emsp;&emsp;
<span id="reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_reviews_allowed]', ' Set allowed number of reviews per reviewer') %>
<%if @assignment_form.assignment.num_reviews_allowed && @assignment_form.assignment.num_reviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', @assignment_form.assignment.num_reviews_allowed, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', 3, size: 1) %>
<%end%>

<img src="/assets/info.png" title='This is the maximum number of reviews a reviewer can perform'<%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
</span>

</td>
</pre>
[[File:infobuttons.png?maxwidth=600&maxheight=600]]
[[File:infobutton2.png]]
*'''Problem 3''': When the number of allowed or required reviews is not set on the Review Strategy tab, the system does not have a message to display to a reviewer about how many submissions of work they are required to and allowed to review depending on the values set by the instructor in the "allowed number of reviewers per reviewer" field and the "Set Required Number of Reviews per reviewer." field.

* '''Solution''': This view has been implemented in the app/views/student_review/list.html.erb
<pre>

<h2>Reviews for "<%= @assignment.name %>"</h2>


<% if @assignment.num_reviews_allowed.nil? || @assignment.num_reviews_allowed == -1%>
<h4>Your instructor expects you to do <%= @assignment.num_reviews_required %> reviews. You are not allowed to do any extra reviews. </h4>

<% elsif @assignment.num_reviews_allowed == @assignment.num_reviews_required %>
<h4>You should perform exactly <%= @assignment.num_reviews_allowed %> reviews </h4>
<% else %>

<h4>You may perform between <%= @assignment.num_reviews_required %> and <%= @assignment.num_reviews_allowed %> reviews</h4>
<% end %>
</pre>

* '''Problem 4''': The capitalization and punctuations of statements such as "Set Allowed Number of Reviews per reviewer" and "Set Required Number of Reviews per reviewer." in review strategy tab are incorrect.
* '''Solution''': This view has been fixed by making changes in the respective files.
* The changes were made in the file : app/views/assignments/edit/_review_strategy.html.erb
<pre>
<%= label_tag('assignment_form[assignment][num_reviews_required]', 'Set required number of reviews per reviewer') %>
<%= text_field_tag('assignment_form[assignment][num_reviews_required]', @assignment_form.assignment.num_reviews_required ||= @assignment_form.assignment.num_reviews, size: 1) %>


<img src="/assets/info.png" title='This is the minimum number of reviews a reviewer has to perform'>
<br/>
<br/>
<%= check_box_tag('review', 'true', true, {:onChange => 'hasReviewChanged()'}) %>
<%= label_tag('review', 'Has max review limit?') %>&emsp;&emsp;
<span id="reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_reviews_allowed]', ' Set allowed number of reviews per reviewer') %>
<%if @assignment_form.assignment.num_reviews_allowed && @assignment_form.assignment.num_reviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', @assignment_form.assignment.num_reviews_allowed, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', 3, size: 1) %>
<%end%>

<img src="/assets/info.png" title='This is the maximum number of reviews a reviewer can perform'<%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
</span>

</td>
<td width="50%">

<%= label_tag('assignment_form[assignment][num_metareviews_required]', 'Set required number of meta-reviews per reviewer') %>
<%= text_field_tag('assignment_form[assignment][num_metareviews_required]', @assignment_form.assignment.num_metareviews_required ||= @assignment_form.assignment.num_metareviews_allowed, size: 1) %>
<br/>
<br/>

<%= check_box_tag('meta_review', 'true', true, {:onChange => 'hasMetaReviewChanged()'}) %>
<%= label_tag('meta_review', 'Has meta-review limit?') %>&emsp;&emsp;
<span id="meta_reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_metareviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_metareviews_allowed]', ' Set allowed number of meta-reviews per reviewer') %>
<%if @assignment_form.assignment.num_metareviews_allowed && @assignment_form.assignment.num_metareviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_metareviews_allowed]', @assignment_form.assignment.num_metareviews_allowed ||= 3, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_metareviews_allowed]', 3, size: 1) %>
<%end%>
</span>

</td>
</tr>

</pre>
[[File:uifix (2).png]]

* '''Problem 5''': The assignment of work to review to a reviewer depends on the number of reviews it has previously received, if that work has been assigned to a reviewer previously and its review has not been submitted then the system will still consider it such that it would not assign the same work to another reviewer if it has enough number of reviewers previously assigned to review it. This might lead to a scenario where that work was assigned to enough number of reviewers, but it does not receive enough reviews and cannot be assigned to any new reviewers.
* '''Solution''': This implementation has been fixed. The system now considers only the reviews that were submitted by the reviewers for that work so that every submission gets assigned to reviewers until it receives its allowed number of reviews.

*the modified file is app/models/review_assignment.rb and the modification is done in the method reject_by_max_reviews_per_submission

<pre>
def reject_by_max_reviews_per_submission(contributor_set)
contributor_set.reject! {|contributor| contributor.responses.reject {|response| !response.is_submitted }.count >= max_reviews_per_submission }
contributor_set
end
</pre>

==Modified File==
* app/views/assignments/edit/_review_strategy.html.erb
* app/models/review_assignment.rb
* app/views/student_review/list.html.erb

===References===

#[https://github.com/expertiza/expertiza Expertiza on GitHub]
#[https://github.com/WintersLt/expertiza GitHub Project Repository Fork]
#[http://expertiza.ncsu.edu/ The live Expertiza website]
#[http://wikis.lib.ncsu.edu/index.php/Expertiza Expertiza project documentation wiki]
#Clean Code: A handbook of agile software craftsmanship. Author: Robert C Martin

CSC/ECE 517 Fall 2017/E1777 Coherent specification of review requirements.rb

2017-10-28T04:48:01Z

Ssuresh6: /* Drawbacks and Solutions */

==E1777. OSS Project Green: Coherent specification of review requirements==

This page provides a description of the Expertiza based OSS project.

__TOC__

===About Expertiza===

[http://expertiza.ncsu.edu/ Expertiza] is an open source project based on [http://rubyonrails.org/ Ruby on Rails] framework. Expertiza allows the instructor to create new assignments and customize new or existing assignments. It also allows the instructor to create a list of topics the students can sign up for. Students can form teams in Expertiza to work on various projects and assignments. Students can also peer review other students' submissions. Expertiza supports submission across various document types, including the URLs and wiki pages.

===Problem Statement===
The following tasks were accomplished in this project:

* Improved the punctuation, syntax and the capitalization of statements in the review strategy tab.
* Provided information buttons to elaborate difference between the synonymous statements present in review strategy tab.
* Display of message to a reviewer regarding how many reviews they are required to and allowed to perform.
* Assignment of work to review for a new reviewer is now based on the number of previous reviews submitted for that work and not on the numbers of assigned reviews for that work.

===Implementation===

=====Functionality=====
* In Expertiza, there are two ways of assigning reviews to reviewers: either the instructor decides who reviews whom (“instructor-selected”), or “auto-selected,” in which case reviews are not assigned until a student seeks to choose something to review. To allow reviewers a larger set of topics to choose from, the instructor can set the threshold (on the Review Strategy tab of assignment creation/editing) to some integer k > 0. Then any submission that has within k reviews of the fewest number can be chosen by a new reviewer. Let’s say that all submissions have at least 1 review. If k = 3, then the reviewer can choose any topic where there is a submission that has 4 or fewer reviews so far. Suppose that the minimum number of reviews for any submission is 2, but that the reviewer has reviewed all the submissions that have only 2 reviews. Then he or she will not be allowed to review at all (unless k > 0).
* The capitalization and punctuation in many statements under the review strategy tab are random.
* On the Review Strategy tab of assignment creation, the "Maximum number of reviews per submission" checks the number of reviews that have been assigned to reviewers instead of checking the number of reviews that are submitted by the reviewers. This means that if required number of reviewers choose submissions and don't submit their reviews, those submissions cannot be reviewed any more.

=====Drawbacks and Solutions=====
* '''Problem 1''': To allow reviewers a larger set of topics to choose from, the instructor can set the threshold (on the Review Strategy tab of assignment creation/editing) to some integer k > 0. Then any submission that has within k reviews of the fewest number can be chosen by a new reviewer. Let’s say that all submissions have at least 1 review. If k = 3, then the reviewer can choose any topic where there is a submission that has 4 or fewer reviews so far. Suppose that the minimum number of reviews for any submission is 2, but that I have reviewed all the submissions that have only 2 reviews. Then I’m not allowed to review at all (unless k > 0).
* '''Solution''': This issue has been fixed previously. The reviewer will get assigned a submission even if it has fulfilled the required number of reviews, to ensure that the reviewer always receives a new submission to review as long as it is not his own.
* '''Problem 2''': The two statements "Set Allowed Number of Reviews per reviewer" and "Set Required Number of Reviews per reviewer." are not well differentiated. It can very confusing for an instructor to give values for them.

* '''Solution''': Information buttons have been provided beside these statements in the review strategy tab to make sure that the instructor knows the difference between the two.
* this modification is made in the app/views/assignments/edit/_review_strategy.html.erb
<pre>

<img src="/assets/info.png" title='This is the minimum number of reviews a reviewer has to perform'>
<br/>
<br/>
<%= check_box_tag('review', 'true', true, {:onChange => 'hasReviewChanged()'}) %>
<%= label_tag('review', 'Has max review limit?') %>&emsp;&emsp;
<span id="reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_reviews_allowed]', ' Set allowed number of reviews per reviewer') %>
<%if @assignment_form.assignment.num_reviews_allowed && @assignment_form.assignment.num_reviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', @assignment_form.assignment.num_reviews_allowed, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', 3, size: 1) %>
<%end%>

<img src="/assets/info.png" title='This is the maximum number of reviews a reviewer can perform'<%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
</span>

</td>
</pre>
[[File:infobuttons.png]]
[[File:infobutton2.png]]
*'''Problem 3''': When the number of allowed or required reviews is not set on the Review Strategy tab, the system does not have a message to display to a reviewer about how many submissions of work they are required to and allowed to review depending on the values set by the instructor in the "allowed number of reviewers per reviewer" field and the "Set Required Number of Reviews per reviewer." field.

* '''Solution''': This view has been implemented in the app/views/student_review/list.html.erb
<pre>

<h2>Reviews for "<%= @assignment.name %>"</h2>


<% if @assignment.num_reviews_allowed.nil? || @assignment.num_reviews_allowed == -1%>
<h4>Your instructor expects you to do <%= @assignment.num_reviews_required %> reviews. You are not allowed to do any extra reviews. </h4>

<% elsif @assignment.num_reviews_allowed == @assignment.num_reviews_required %>
<h4>You should perform exactly <%= @assignment.num_reviews_allowed %> reviews </h4>
<% else %>

<h4>You may perform between <%= @assignment.num_reviews_required %> and <%= @assignment.num_reviews_allowed %> reviews</h4>
<% end %>
</pre>

* '''Problem 4''': The capitalization and punctuations of statements such as "Set Allowed Number of Reviews per reviewer" and "Set Required Number of Reviews per reviewer." in review strategy tab are incorrect.
* '''Solution''': This view has been fixed by making changes in the respective files.
* The changes were made in the file : app/views/assignments/edit/_review_strategy.html.erb
<pre>
<%= label_tag('assignment_form[assignment][num_reviews_required]', 'Set required number of reviews per reviewer') %>
<%= text_field_tag('assignment_form[assignment][num_reviews_required]', @assignment_form.assignment.num_reviews_required ||= @assignment_form.assignment.num_reviews, size: 1) %>


<img src="/assets/info.png" title='This is the minimum number of reviews a reviewer has to perform'>
<br/>
<br/>
<%= check_box_tag('review', 'true', true, {:onChange => 'hasReviewChanged()'}) %>
<%= label_tag('review', 'Has max review limit?') %>&emsp;&emsp;
<span id="reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_reviews_allowed]', ' Set allowed number of reviews per reviewer') %>
<%if @assignment_form.assignment.num_reviews_allowed && @assignment_form.assignment.num_reviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', @assignment_form.assignment.num_reviews_allowed, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', 3, size: 1) %>
<%end%>

<img src="/assets/info.png" title='This is the maximum number of reviews a reviewer can perform'<%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
</span>

</td>
<td width="50%">

<%= label_tag('assignment_form[assignment][num_metareviews_required]', 'Set required number of meta-reviews per reviewer') %>
<%= text_field_tag('assignment_form[assignment][num_metareviews_required]', @assignment_form.assignment.num_metareviews_required ||= @assignment_form.assignment.num_metareviews_allowed, size: 1) %>
<br/>
<br/>

<%= check_box_tag('meta_review', 'true', true, {:onChange => 'hasMetaReviewChanged()'}) %>
<%= label_tag('meta_review', 'Has meta-review limit?') %>&emsp;&emsp;
<span id="meta_reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_metareviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_metareviews_allowed]', ' Set allowed number of meta-reviews per reviewer') %>
<%if @assignment_form.assignment.num_metareviews_allowed && @assignment_form.assignment.num_metareviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_metareviews_allowed]', @assignment_form.assignment.num_metareviews_allowed ||= 3, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_metareviews_allowed]', 3, size: 1) %>
<%end%>
</span>

</td>
</tr>

</pre>
[[File:uifix (2).png]]

* '''Problem 5''': The assignment of work to review to a reviewer depends on the number of reviews it has previously received, if that work has been assigned to a reviewer previously and its review has not been submitted then the system will still consider it such that it would not assign the same work to another reviewer if it has enough number of reviewers previously assigned to review it. This might lead to a scenario where that work was assigned to enough number of reviewers, but it does not receive enough reviews and cannot be assigned to any new reviewers.
* '''Solution''': This implementation has been fixed. The system now considers only the reviews that were submitted by the reviewers for that work so that every submission gets assigned to reviewers until it receives its allowed number of reviews.

*the modified file is app/models/review_assignment.rb and the modification is done in the method reject_by_max_reviews_per_submission

<pre>
def reject_by_max_reviews_per_submission(contributor_set)
contributor_set.reject! {|contributor| contributor.responses.reject {|response| !response.is_submitted }.count >= max_reviews_per_submission }
contributor_set
end
</pre>

==Modified File==
* app/views/assignments/edit/_review_strategy.html.erb
* app/models/review_assignment.rb
* app/views/student_review/list.html.erb

===References===

#[https://github.com/expertiza/expertiza Expertiza on GitHub]
#[https://github.com/WintersLt/expertiza GitHub Project Repository Fork]
#[http://expertiza.ncsu.edu/ The live Expertiza website]
#[http://wikis.lib.ncsu.edu/index.php/Expertiza Expertiza project documentation wiki]
#Clean Code: A handbook of agile software craftsmanship. Author: Robert C Martin

CSC/ECE 517 Fall 2017/E1777 Coherent specification of review requirements.rb

2017-10-28T04:47:23Z

Ssuresh6: /* Drawbacks and Solutions */

==E1777. OSS Project Green: Coherent specification of review requirements==

This page provides a description of the Expertiza based OSS project.

__TOC__

===About Expertiza===

[http://expertiza.ncsu.edu/ Expertiza] is an open source project based on [http://rubyonrails.org/ Ruby on Rails] framework. Expertiza allows the instructor to create new assignments and customize new or existing assignments. It also allows the instructor to create a list of topics the students can sign up for. Students can form teams in Expertiza to work on various projects and assignments. Students can also peer review other students' submissions. Expertiza supports submission across various document types, including the URLs and wiki pages.

===Problem Statement===
The following tasks were accomplished in this project:

* Improved the punctuation, syntax and the capitalization of statements in the review strategy tab.
* Provided information buttons to elaborate difference between the synonymous statements present in review strategy tab.
* Display of message to a reviewer regarding how many reviews they are required to and allowed to perform.
* Assignment of work to review for a new reviewer is now based on the number of previous reviews submitted for that work and not on the numbers of assigned reviews for that work.

===Implementation===

=====Functionality=====
* In Expertiza, there are two ways of assigning reviews to reviewers: either the instructor decides who reviews whom (“instructor-selected”), or “auto-selected,” in which case reviews are not assigned until a student seeks to choose something to review. To allow reviewers a larger set of topics to choose from, the instructor can set the threshold (on the Review Strategy tab of assignment creation/editing) to some integer k > 0. Then any submission that has within k reviews of the fewest number can be chosen by a new reviewer. Let’s say that all submissions have at least 1 review. If k = 3, then the reviewer can choose any topic where there is a submission that has 4 or fewer reviews so far. Suppose that the minimum number of reviews for any submission is 2, but that the reviewer has reviewed all the submissions that have only 2 reviews. Then he or she will not be allowed to review at all (unless k > 0).
* The capitalization and punctuation in many statements under the review strategy tab are random.
* On the Review Strategy tab of assignment creation, the "Maximum number of reviews per submission" checks the number of reviews that have been assigned to reviewers instead of checking the number of reviews that are submitted by the reviewers. This means that if required number of reviewers choose submissions and don't submit their reviews, those submissions cannot be reviewed any more.

=====Drawbacks and Solutions=====
* '''Problem 1''': To allow reviewers a larger set of topics to choose from, the instructor can set the threshold (on the Review Strategy tab of assignment creation/editing) to some integer k > 0. Then any submission that has within k reviews of the fewest number can be chosen by a new reviewer. Let’s say that all submissions have at least 1 review. If k = 3, then the reviewer can choose any topic where there is a submission that has 4 or fewer reviews so far. Suppose that the minimum number of reviews for any submission is 2, but that I have reviewed all the submissions that have only 2 reviews. Then I’m not allowed to review at all (unless k > 0).
* '''Solution''': This issue has been fixed previously. The reviewer will get assigned a submission even if it has fulfilled the required number of reviews, to ensure that the reviewer always receives a new submission to review as long as it is not his own.
* '''Problem 2''': The two statements "Set Allowed Number of Reviews per reviewer" and "Set Required Number of Reviews per reviewer." are not well differentiated. It can very confusing for an instructor to give values for them.

* '''Solution''': Information buttons have been provided beside these statements in the review strategy tab to make sure that the instructor knows the difference between the two.
* this modification is made in the app/views/assignments/edit/_review_strategy.html.erb
<pre>

<img src="/assets/info.png" title='This is the minimum number of reviews a reviewer has to perform'>
<br/>
<br/>
<%= check_box_tag('review', 'true', true, {:onChange => 'hasReviewChanged()'}) %>
<%= label_tag('review', 'Has max review limit?') %>&emsp;&emsp;
<span id="reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_reviews_allowed]', ' Set allowed number of reviews per reviewer') %>
<%if @assignment_form.assignment.num_reviews_allowed && @assignment_form.assignment.num_reviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', @assignment_form.assignment.num_reviews_allowed, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', 3, size: 1) %>
<%end%>

<img src="/assets/info.png" title='This is the maximum number of reviews a reviewer can perform'<%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
</span>

</td>
</pre>
[[File:infobuttons.png|200px|thumb|left|alt text]]
[[File:infobutton2.png]]
*'''Problem 3''': When the number of allowed or required reviews is not set on the Review Strategy tab, the system does not have a message to display to a reviewer about how many submissions of work they are required to and allowed to review depending on the values set by the instructor in the "allowed number of reviewers per reviewer" field and the "Set Required Number of Reviews per reviewer." field.

* '''Solution''': This view has been implemented in the app/views/student_review/list.html.erb
<pre>

<h2>Reviews for "<%= @assignment.name %>"</h2>


<% if @assignment.num_reviews_allowed.nil? || @assignment.num_reviews_allowed == -1%>
<h4>Your instructor expects you to do <%= @assignment.num_reviews_required %> reviews. You are not allowed to do any extra reviews. </h4>

<% elsif @assignment.num_reviews_allowed == @assignment.num_reviews_required %>
<h4>You should perform exactly <%= @assignment.num_reviews_allowed %> reviews </h4>
<% else %>

<h4>You may perform between <%= @assignment.num_reviews_required %> and <%= @assignment.num_reviews_allowed %> reviews</h4>
<% end %>
</pre>

* '''Problem 4''': The capitalization and punctuations of statements such as "Set Allowed Number of Reviews per reviewer" and "Set Required Number of Reviews per reviewer." in review strategy tab are incorrect.
* '''Solution''': This view has been fixed by making changes in the respective files.
* The changes were made in the file : app/views/assignments/edit/_review_strategy.html.erb
<pre>
<%= label_tag('assignment_form[assignment][num_reviews_required]', 'Set required number of reviews per reviewer') %>
<%= text_field_tag('assignment_form[assignment][num_reviews_required]', @assignment_form.assignment.num_reviews_required ||= @assignment_form.assignment.num_reviews, size: 1) %>


<img src="/assets/info.png" title='This is the minimum number of reviews a reviewer has to perform'>
<br/>
<br/>
<%= check_box_tag('review', 'true', true, {:onChange => 'hasReviewChanged()'}) %>
<%= label_tag('review', 'Has max review limit?') %>&emsp;&emsp;
<span id="reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_reviews_allowed]', ' Set allowed number of reviews per reviewer') %>
<%if @assignment_form.assignment.num_reviews_allowed && @assignment_form.assignment.num_reviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', @assignment_form.assignment.num_reviews_allowed, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_reviews_allowed]', 3, size: 1) %>
<%end%>

<img src="/assets/info.png" title='This is the maximum number of reviews a reviewer can perform'<%= 'hidden' if @assignment_form.assignment.num_reviews_allowed.nil?%>>
</span>

</td>
<td width="50%">

<%= label_tag('assignment_form[assignment][num_metareviews_required]', 'Set required number of meta-reviews per reviewer') %>
<%= text_field_tag('assignment_form[assignment][num_metareviews_required]', @assignment_form.assignment.num_metareviews_required ||= @assignment_form.assignment.num_metareviews_allowed, size: 1) %>
<br/>
<br/>

<%= check_box_tag('meta_review', 'true', true, {:onChange => 'hasMetaReviewChanged()'}) %>
<%= label_tag('meta_review', 'Has meta-review limit?') %>&emsp;&emsp;
<span id="meta_reviews_allowed" <%= 'hidden' if @assignment_form.assignment.num_metareviews_allowed.nil?%>>
<%= label_tag('assignment_form[assignment][num_metareviews_allowed]', ' Set allowed number of meta-reviews per reviewer') %>
<%if @assignment_form.assignment.num_metareviews_allowed && @assignment_form.assignment.num_metareviews_allowed >= 0 %>
<%= text_field_tag('assignment_form[assignment][num_metareviews_allowed]', @assignment_form.assignment.num_metareviews_allowed ||= 3, size: 1) %>
<%else%>
<%= text_field_tag('assignment_form[assignment][num_metareviews_allowed]', 3, size: 1) %>
<%end%>
</span>

</td>
</tr>

</pre>
[[File:uifix (2).png]]

* '''Problem 5''': The assignment of work to review to a reviewer depends on the number of reviews it has previously received, if that work has been assigned to a reviewer previously and its review has not been submitted then the system will still consider it such that it would not assign the same work to another reviewer if it has enough number of reviewers previously assigned to review it. This might lead to a scenario where that work was assigned to enough number of reviewers, but it does not receive enough reviews and cannot be assigned to any new reviewers.
* '''Solution''': This implementation has been fixed. The system now considers only the reviews that were submitted by the reviewers for that work so that every submission gets assigned to reviewers until it receives its allowed number of reviews.

*the modified file is app/models/review_assignment.rb and the modification is done in the method reject_by_max_reviews_per_submission

<pre>
def reject_by_max_reviews_per_submission(contributor_set)
contributor_set.reject! {|contributor| contributor.responses.reject {|response| !response.is_submitted }.count >= max_reviews_per_submission }
contributor_set
end
</pre>

==Modified File==
* app/views/assignments/edit/_review_strategy.html.erb
* app/models/review_assignment.rb
* app/views/student_review/list.html.erb

===References===

#[https://github.com/expertiza/expertiza Expertiza on GitHub]
#[https://github.com/WintersLt/expertiza GitHub Project Repository Fork]
#[http://expertiza.ncsu.edu/ The live Expertiza website]
#[http://wikis.lib.ncsu.edu/index.php/Expertiza Expertiza project documentation wiki]
#Clean Code: A handbook of agile software craftsmanship. Author: Robert C Martin