Expertiza_Wiki - User contributions [en]

CSC/ECE 517 Fall 2019 - E1985. Let course staff and students do reviews

2019-12-16T21:02:17Z

Mtrawic: /* Part 3: Add instructor/TA as a participant while creating the assignment (Extra Feature) */

== '''Problem Statement''' ==

Peer review is a great way for students to learn about how well they have developed their application. However, sometimes, the peer reviews may not be thorough and the team/person's work reviewed might not reflect the actual status of the development.
By letting course staff perform reviews as well, the reviewer and the reviewee both benefit, improving the overall learning experience. Currently, there exists no method for the instructor/TA to perform reviews of assignments submitted by the students. Hence this is what this project aims at, allowing instructor/TA to perform reviews.

Thus, the primary aim of this project is to allow instructors to submit reviews of student work, using the same review form that students use to do reviews.

=='''Implementation Details''' ==
Our project aims at enabling the instructor/TA to review the submissions using the same form that a student uses while peer-reviewing. For this, our implementation has been divided into 3 parts, and can be described as follows:
<ol>
<li>The first part involves the primary functionality of letting the staff perform a review on students submission.</li>
<li>The second part involves allowing the students to identify from their end, which review has been performed by an instructor/TA. This will help students improve their project as the opinions presented by a staff member would be valid.</li>
<li>Thirdly, we implemented an additional feature that allows the instructor/TA to add himself as a participant while creating/editing an Assignment (This way seemed to be more convenient rather than going to the “Add Participant” page and adding themselves as a participant.)</li>
</ol>

===Part 1: Allowing instructor/TA to perform review===
Note 1: “Instructor/TA” will be referred to by “user” here onwards.

Note 2: To be able to perform a review, the user has to be a participant of that assignment. We have implemented “Add as Participant” as an additional feature which is explained in Part 3. If the user is not a participant, he will only see the link to “Assign grade” that too after the assignment deadline passes.
Now, for each assignment, there is a page called “List Submissions” where the user can view all the submissions. So we have modified this page so that for each team submission a link for “Begin review,” “Edit review,” “View review,” “Update review,” or “Assign grade” may appear depending on the [http://wiki.expertiza.ncsu.edu/index.php/Deadline_types stage] in which the assignment is.

The following describes the flow of our project considering a fresh assignment.
<ol>
<li>When the assignment is created, it is by default is in the “submission” stage. Hence the user will not see any link on the list submission page.</li>
<li>When the first deadline passes, the assignment goes into the “review” stage. Hence the link “Begin review” will be visible for each team submission, where the instructor/TA can click on to submit the review. Two cases can occur here:
<ul>
<li>If the user “submits” a review, they will be redirected to the list submission page where they’ll see the option “View review” or “Update review.” Being an instructor/TA, they are given the flexibility to update their review at any time during the “review” stage.</li>
<li>If the user “saves” a review, they will be redirected to the list submission page where they’ll see the option “View review” and “Edit Review (as they have not submitted it yet.)”</li>
</li>
</ul>
<li>After the first review is over, if more stages of “submissions” and “reviews” exist, it will follow the above 2 steps until the final assignment deadline does not pass. Note that at any round of the review, the user can “Begin review” for a submission which they had not reviewed in the previous rounds. This is to provide flexibility to them as per the requirement.</li>
<li>Finally, after the assignment deadline passes, the assignment goes into the “Finished” stage. This is when the “Assign grade” link will be visible to the user. Along with this, depending on whether they have reviewed a particular submission or not, the begin, view, update links will be visible to them as they should be able to perform the review even after the assignment deadline for the students has passed.</li>
</ol>

Files Edited 
The following are the files (controllers and views) edited for the implementation of the review process. The comments are given along with the code to explain what is being done. 
* '''Controller: ''' ''app/controllers/response_controller.rb'' 

<pre>
# Ensuring recent submission is not nil
team_most_recent_submission = AssignmentTeam.find(@map.reviewee_id).most_recent_submission
if @response.nil? || (!team_most_recent_submission.nil? && team_most_recent_submission.updated_at > @response.updated_at)
...
# Page should be directed to list_submissions if TA/instructor performs the review
when "ta_review"
redirect_to controller: 'assignments', action: 'list_submissions', id: @map.response_map.assignment.id
</pre>

* '''Controller: ''' ''app/helpers/assignment_helper.rb'' 

<pre>
topic_id = topic.try :id
...
# Needed topic ID for review implpementation, so added it to return list of the function
[topic_id, topic_identifier ||= "", topic_name ||= "", users_for_curr_team, participants]
</pre>

* '''View: ''' ''app/views/assignments/list_submissions.html.erb'' 

<pre>
# Retrieved topic ID for review implpementation
<% topic_id, topic_identifier, topic_name, users_for_curr_team, participants = get_data_for_list_submissions(team) %>
...

<% if @assignment.get_current_stage() == "Finished" %>
<%= link_to 'Assign Grade', { controller: 'grades', action: 'view_team', id: participants.first.id}, target: '_blank' %> 
<% end %>

<% reviewer = Participant.find_by(:parent_id => @assignment.id, :user_id => session[:user].id)%>


<% if !reviewer.nil? %>

<% @mapping = ReviewResponseMap.find_by(:reviewed_object_id => @assignment.id, :reviewer_id => reviewer.id, :reviewee_id => team.id) %>

<% if @mapping.nil? && (@assignment.get_current_stage() == "review" || @assignment.get_current_stage() == "Finished") %>
<% ReviewResponseMap.create(reviewee_id: team.id, reviewer_id: reviewer.id, reviewed_object_id: @assignment.id) %>
<% map = ReviewResponseMap.find_by(reviewee_id: team.id, reviewer_id: reviewer.id, reviewed_object_id: @assignment.id) %>
<%= link_to 'Begin Review', { controller: 'response', action: 'new', :id => map.id, :return => "ta_review" } %> 

<% elsif @assignment.get_current_stage() == "review" || @assignment.get_current_stage() == "Finished" %>
<% map1 = ReviewResponseMap.find_by(reviewee_id: team.id, reviewer_id: reviewer.id, reviewed_object_id: @assignment.id) %>
<% @sorted_responses = Array.new %>
<% @sorted_responses = Response.where(:map_id => map1.id) %>

<% if @sorted_responses.empty?%>
<%= link_to 'Begin Review', { controller: 'response', action: 'new', :id => map1.id, :return => "ta_review" } %> 
<% else %>

<% @sorted_responses = @sorted_responses.sort_by {|obj| obj.updated_at} # the latest should be at the last %>
<% @latest_response = @sorted_responses.last %>
<%if @latest_response.round.nil?%> 
<% last_response_round = AssignmentDueDate.done_in_assignment_round(@assignment.id, @latest_response) %>
<%else%> 
<% last_response_round = @latest_response.round %>
<%end%>
<% current_round = @assignment.number_of_current_round(topic_id) %>

<%= link_to "View Review", {:controller => 'response', :action => 'view', :id => @latest_response.id} %> 
<% if last_response_round == current_round %>
<% latest_submission = team.most_recent_submission %>
<% if !@latest_response.is_submitted %> 
<%= link_to "Edit Review", {:controller => 'response', :action => 'edit', :id => @latest_response.id, :return => "ta_review" } %>
<% else %> 
<%= link_to "Update Review", {:controller => 'response', :action => 'new', :id => map1.id, :return => "ta_review" } %>
<%end%>
<% else %> 
<%= link_to "Update Review", {:controller => 'response', :action => 'new', :id => map1.id, :return => "ta_review" } %>
<% end %>
<% end %>
<% end %>
<% end %>
</pre>

* '''View: ''' ''app/views/grades/view_team.html.erb'' 

<pre>

<% topic_id_num, topic_id, topic_name, users_for_curr_team, participants = get_data_for_list_submissions(@team) %>
</pre>

* '''View: ''' ''app/views/student_review/_responses.html.erb'' 

<pre>
# Ensuring recent submission is not nil
<% elsif !latest_submission.nil? && @latest_response.updated_at < latest_submission.updated_at %>
</pre>

---

=== Part 2: Allow students to identify that their submission has been reviewed by the instructor===
 Currently Scenario: There is no way for the student to know which review was by the instructor. 
Requirement: This helps the student to identify the review made by the instructor from the other reviews. 
Implementation: A simple check is made to determine if the reviewer is a TA or Instructor. app/assets/images/staff.png is displayed to make an instructor performed review stand out from other reviews. If the mouse hovers over the icon a tool tip appears saying "Icon indicates this review was done by course staff" 
[[File:Icon-demo.png|center]]

Files Edited 
* '''Helper: ''' ''app/helpers/grades_helper.rb'' 

<pre>
# Determines if a review was completed by a staff participant (either a TA or an instructor)
# Called in view_team.html to determine if a review should be marked as reviewed by course staff.
def done_by_staff_participant?( review )
role = Role.find(User.find(Participant.find(ResponseMap.find(Response.find(review.id).map_id).reviewer_id).user_id).role_id).name
return (role == "Instructor") || (role == "Teaching Assistant")
end

# Determines if given participant is a staff participant (either a TA or an instructor)
# Called in views/response/view.html.erb to determine if a review should be marked as reviewed by course staff.
def is_course_staff?( participant )
role = Role.find(User.find(Participant.find(participant.id).user_id).role_id).name
return (role == "Instructor") || (role == "Teaching Assistant")
end
</pre>

* '''View: ''' ''app/views/view_team.html.erb'' 
<pre>
<% if done_by_staff_participant?(review) %>

<th class="sorter-false">
<a target="_blank" href="../response/view?id=<%= review.id.to_s %>" data-toggle="tooltip" data-placement="right" title="Click to see details">
<%= "Review " + i.to_s %></a> <img data-toggle="tooltip" data-placement="right" title="Icon indicates this review was done by course staff" class="img_icon" src="/assets/staff.png"> </th>
<% else %>
<th class="sorter-false"> <a target="_blank" href="../response/view?id=<%= review.id.to_s %>" data-toggle="tooltip" data-placement="right" title="Click to see details"><%= "Review " + i.to_s %></a> </th>
<% end %>
</pre>

=== Part 3: Add instructor/TA as a participant while creating the assignment (Extra Feature) ===
 
'''Current scenario''' 
The instructor/TA will be able to perform student reviews (after implementation of Part 1) for the assignment, only when he is a participant of the same assignment. Thus, every time he creates an assignment and wants to add himself as a participant in order to perform student reviews, he is supposed to do the following 
Go to list of assignments -> Find the required assignment -> Click on add participant -> Type user Id -> Click on add
The same flow is as shown below where the instructor/TA has created a new assignment say “Program 1 Github” wherein he wants himself as a participant to be able to perform student reviews :
 

To overcome the hassle, we added a check box through which the instructor/TA will get an option to add himself as a participant while creating the assignment. Also, he will be able to add/remove himself as a participant while editing the assignment.

'''Implementation''' 

The checkbox has been provided to Create New Assignment’s page under the Review strategy tab.
It is added as follows: 

[[File:Add instructor 4.png]] 

If the instructor/TA selects the box while creating the assignment, then he will be added as a participant for the same assignment. 

Files edited:

* '''View: ''' ''app/views/assignments/edit/_review_strategy.html.erb'' 
In this file, code for displaying the checkbox has been added which is as follows :

<pre>
<tr>
<td>
<input name='add_instructor' type="hidden" value="false"/>
<%= check_box_tag('add_instructor', value = "1", checked = is_instructor_a_participant?)%>
<%= label_tag('add_instructor', 'Add yourself as a participant?') %>
<img src="/assets/info.png" title='If you add yourself as a participant then you will be able to perform reviews of students for this assignment'>
</td>
</tr>
</pre>

Thus the checkbox "add_instructor" if selected adds the instructor as a participant to the currently being created assignment. Thus, instructor being a participant allows him to perform reviews for the students for this assignment.

* '''Controller: ''' ''app/controllers/assignments_controller.rb'' 

The file contains extra code which adds the instructor/TA as a participant if the checkbox has been selected while creating the assignment. Also, while editing the assignment, he will be able to add/remove himself as a participant for this assignment. 

The following methods are added to the controller :

<pre>
def is_instructor_a_participant?
instructor_id = session[:user].id
assignment_id = @assignment_form.assignment.id.to_s

if assignment_id != nil
@is_instructor_a_participant = Participant.where(user_id: instructor_id , parent_id: assignment_id)
if @is_instructor_a_participant.present?
return true
end
end
return false
end
</pre>

The method "is_instructor_a_participant?" returns true if the instructor is already a participant for the assignment, else returns false. This method is required to verify that the instructor/TA is not a participant of the assignment, and thus he can be added as a participant if the checkbox has been selected. Also, during editing the assignment, we need to show checkbox being selected if the instructor is a participant and vice versa. Thus, this method serves that purpose as well.

 
<pre>
def add_instructor_as_participant(assignment_id)
if params[:add_instructor] == 'false' and is_instructor_a_participant? == true
delete_instructor_as_participant(assignment_id,session[:user].id)
elsif params[:add_instructor] == '1' and is_instructor_a_participant? == false
current_assignment = Object.const_get("Assignment").find(assignment_id)
current_assignment.add_participant(session[:user].name, true, true, true)
end
end
</pre>

The method "add_instructor_as_participant" is been called from create and update methods in the Assignment controller. It checks if the "Add yourself as participant" checkbox is selected for the currently created/edited assignment. If it is selected during creation or editing, then the instructor is added as a participant to the currently created/edited assignment. The participant is added by using the already existing method "add_participant" in the model assignment.rb Also, if the option is unchecked during editing, then the instructor is removed as a participant.

 
<pre>
def delete_instructor_as_participant(assignment_id , instructor_id)
Participant.where(user_id: instructor_id , parent_id: assignment_id).destroy_all
end
</pre>
The method "delete_instructor_as_participant" removes instructor as a participant for the assignment when the checkbox is unchecked during the editing of the assignment.

== '''Test Plan''' ==

==== Part 1: Allowing instructor/TA to perform review ====
UI Testing

<ol>
<li>Login in as an instructor.
<ul>
<li>Username: instructor6</li>
<li>Password: password</li>
</ul> </li>
<li>Choose any assignment from the list of assignments which involves teams; since an assignment without teams does not include the review procedure. For example, “Final project (design and doc)” is a good assignment to choose.
<li>Now, while testing this functionality, it is recommended to keep the following pages opened in different tabs by clicking on the small icons in the same row as the assignment that you want to test: 

a. “Edit” -> “Due Dates” tab
[[File:edit_assignment.png]] 

b. “Add participants”
[[File:add_as_participant.png]] 

c. “View Submissions”
[[File:list_submissions.png]] 
</li>
<li>Now add the instructor as a participant (3.b). and edit the ALL due dates (3.a.) of different stages to any date later from now (hence the assignment goes into the “submission” stage by default). Save both these changes. </li>
<li>Now refresh the view submissions page (3.c.), you will see a list of assignments with no link to review or assign grade.</li>
<li>Now switch to edit due dates tab (3.a.) again. Set the due date for submission to a previous date which has passed, hence the next due date will be for the “review” stage by default (which will have a later due date from now.)</li>
<li>Refresh view submissions page (3.c.), you will see a list of assignments with a link to “Begin Review.”</li>
<li>You can click on it. If you “save” your response, you will see that the link has been replace by “View review” and “Edit review” now. Whereas, if you “submit” the response, the link will be replaced by “View review” and “Update review.”</li>
<li>Depending on the number of stages that an assignment has, you can edit the due date at every stage and check whether the links are displayed appropriately.</li>
<li>Finally when the last deadline of the assignment is set to a previous date which has passed, you will see the link to “Assign grade” in view submissions (3.c.), and the user will still be able to begin, edit or update a review in order to provide flexibility as per the requirements.</li>
</ol>

Edge Cases:
<ol>
<li>If the user is not a participant of the current assignment, they are shown a list of all submissions made by all teams of the current assignment. They are shown the link to “Assign Grade” only when the final deadline of the assignment passes.</li>
<li>During any review stage or even after the deadline, the user is offered the flexibility to Begin, Edit, View or Update their review.</li>
</ol>

=== Part 2: Allow students to identify that their submission has been reviewed by the instructor===
UI Testing
<ol>
<li>Login as an instructor and change the due dates for the assignments to be tested. Here the assignment chosen is Final project (and design doc).
<ul>
<li>Username: instructor6</li>
<li>Password: password</li>
</ul> </li>

<li>Login in as student. Here we have logged in as student8434
<ul>
<li>Username: student8434</li>
<li>Password: password</li>
</ul> </li>
<li> Go to Assignments-> Final project (and design doc). Here click on the assih=gment for which the due dates were changed.</li>
<li> Click on Your scores</li>

There we can see the icon for reviews reviewed by staff.

[[File:Iconpage.png|center]]

=== Part 3: Add instructor/TA as a participant while creating the assignment===
UI Testing
<ol>
<li>Login in as an instructor.
<ul>
<li>Username: instructor6</li>
<li>Password: password</li>
</ul> </li>
<li> Go to Manage-> Assignments </li>
<li> Click on add assignment </li>
<li> Enter the required details- Assignment name, course etc </li>
<li>To add instructor/TA as a participant, go to Review Tab and select the "Add yourself as a participant?" checkbox. </li>
<li> Click on Save </li>

The assignment has been created and the instructor/TA is added as a participant to the assignment.
</ol>

Edge Cases :
<ol>
<li> If the instructor/TA does not select the "Add yourself as a participant?" checkbox while creating the assignment, then he is not added as a participant</li>
<li> If the instructor/TA selects the "Add yourself as a participant?" checkbox while editing the assignment, then he is added as a participant
<li> If the instructor/TA unchecks the "Add yourself as a participant?" checkbox while editing the assignment, then he is removed as a participant </li> </ol>

 
Rspec Testing :
<pre>

describe '#create' do
before(:each) do
allow(AssignmentForm).to receive(:new).with(any_args).and_return(assignment_form)
@params = {
button: '',
assignment_form: {
assignment_questionnaire: [{"assignment_id" => "1", "questionnaire_id" => "666", "dropdown" => "true",
"questionnaire_weight" => "100", "notification_limit" => "15", "used_in_round" => "1"}],
due_date: [{"id" => "", "parent_id" => "", "round" => "1", "deadline_type_id" => "1", "due_at" => "2017/12/05 00:00", "submission_allowed_id" => "3", "review_allowed_id" => "1", "teammate_review_allowed_id" => "3", "review_of_review_allowed_id" => "1", "threshold" => "1"},
{"id" => "", "parent_id" => "", "round" => "1", "deadline_type_id" => "2", "due_at" => "2017/12/02 00:00", "submission_allowed_id" => "1", "review_allowed_id" => "3", "teammate_review_allowed_id" => "3", "review_of_review_allowed_id" => "1", "threshold" => "1"}],
assignment: {
instructor_id: 2,
course_id: 1,
max_team_size: 1,
id: 1,
name: 'test assignment',
directory_path: '/test',
spec_location: '',
private: false,
show_teammate_reviews: false,
require_quiz: false,
num_quiz_questions: 0,
staggered_deadline: false,
microtask: false,
reviews_visible_to_all: false,
is_calibrated: false,
availability_flag: true,
reputation_algorithm: 'Lauw',
simicheck: -1,
simicheck_threshold: 100
add_instructor:1 #added field
}
}
}

context 'when assignment_form is saved successfully' do
it 'redirects to assignment#edit page' do
allow(assignment_form).to receive(:assignment).and_return(assignment)
allow(assignment_form).to receive(:save).and_return(true)
allow(assignment_form).to receive(:update).with(any_args).and_return(true)
allow(assignment_form).to receive(:create_assignment_node).and_return(double('node'))
allow(assignment).to receive(:id).and_return(1)
allow(Assignment).to receive(:find_by).with(name: 'test assignment').and_return(assignment)
allow(assignment_form).to receive(:add_instructor_as_participant).with(any_args).and_return(true) #added check
allow_any_instance_of(AssignmentsController).to receive(:undo_link)
.with('Assignment "test assignment" has been created successfully. ').and_return(true)
post :create, @params
expect(response).to redirect_to('/assignments/1/edit')
end
end

describe '#update' do
context 'when params does not have key :assignment_form' do
context 'when assignment is saved successfully' do
it 'shows a note flash message and redirects to tree_display#index page' do
allow(assignment).to receive(:save).and_return(true)
params = {
id: 1,
course_id: 1
}
session = {user: instructor}
allow(assignment_form).to receive(:add_instructor_as_participant).with(any_args).and_return(true) #added check
post :update, params, session
expect(flash[:note]).to eq('The assignment was successfully saved.')
expect(response).to redirect_to('/tree_display/list')
end
end

context 'when "Add yourself as a participant?" checkbox is selected' do
it 'calls add_instructor_as_participant which in turn calls add Participant method of Participant controller' do
allow(assignment_form).to receive(:add_participant).with(any_args).and_return(true)
end
end

</pre>

== '''External Links''' ==
* Forked Repository: https://github.com/Matrawick/expertiza/tree/beta
* Pull Request: https://github.com/expertiza/expertiza/pull/1631

== '''Team Members''' ==
Mentor: Prof. Edward Gehringer (efg@ncsu.edu)
<ul>
<li>Anuja Kulkarni - apkulka2@ncsu.edu</li>
<li>Mark Trawick - mtrawic@ncsu.edu</li>
<li>Parvathy Menon - pharida@ncsu.edu</li>
<li>Samruddhi Khandale - sskhanda@ncsu.edu</li>
</ul>

CSC/ECE 517 Fall 2019 - E1985. Let course staff and students do reviews

2019-12-06T20:13:36Z

Mtrawic: /* Part 2: Allow students to identify that their submission has been reviewed by the instructor */

== '''Problem Statement''' ==

Peer review is a great way for students to learn about how well they have developed their application. However, sometimes, the peer reviews may not be thorough and the team/person's work reviewed might not reflect the actual status of the development.
By letting course staff perform reviews as well, the reviewer and the reviewee both benefit, improving the overall learning experience. Currently, there exists no method for the instructor/TA to perform reviews of assignments submitted by the students. Hence this is what this project aims at, allowing instructor/TA to perform reviews.

Thus, the primary aim of this project is to allow instructors to submit reviews of student work, using the same review form that students use to do reviews.

=='''Implementation Details''' ==
Our project aims at enabling the instructor/TA to review the submissions using the same form that a student uses while peer-reviewing. For this, our implementation has been divided into 3 parts, and can be described as follows:
<ol>
<li>The first part involves the primary functionality of letting the staff perform a review on students submission.</li>
<li>The second part involves allowing the students to identify from their end, which review has been performed by an instructor/TA. This will help students improve their project as the opinions presented by a staff member would be valid.</li>
<li>Thirdly, we implemented an additional feature that allows the instructor/TA to add himself as a participant while creating/editing an Assignment (This way seemed to be more convenient rather than going to the “Add Participant” page and adding themselves as a participant.)</li>
</ol>

===Part 1: Allowing instructor/TA to perform review===
Note 1: “Instructor/TA” will be referred to by “user” here onwards.

Note 2: To be able to perform a review, the user has to be a participant of that assignment. We have implemented “Add as Participant” as an additional feature which is explained in Part 3. If the user is not a participant, he will only see the link to “Assign grade” that too after the assignment deadline passes.
Now, for each assignment, there is a page called “List Submissions” where the user can view all the submissions. So we have modified this page so that for each team submission a link for “Begin review,” “Edit review,” “View review,” “Update review,” or “Assign grade” may appear depending on the [http://wiki.expertiza.ncsu.edu/index.php/Deadline_types stage] in which the assignment is.

The following describes the flow of our project considering a fresh assignment.
<ol>
<li>When the assignment is created, it is by default is in the “submission” stage. Hence the user will not see any link on the list submission page.</li>
<li>When the first deadline passes, the assignment goes into the “review” stage. Hence the link “Begin review” will be visible for each team submission, where the instructor/TA can click on to submit the review. Two cases can occur here:
<ul>
<li>If the user “submits” a review, they will be redirected to the list submission page where they’ll see the option “View review” or “Update review.” Being an instructor/TA, they are given the flexibility to update their review at any time during the “review” stage.</li>
<li>If the user “saves” a review, they will be redirected to the list submission page where they’ll see the option “View review” and “Edit Review (as they have not submitted it yet.)”</li>
</li>
</ul>
<li>After the first review is over, if more stages of “submissions” and “reviews” exist, it will follow the above 2 steps until the final assignment deadline does not pass. Note that at any round of the review, the user can “Begin review” for a submission which they had not reviewed in the previous rounds. This is to provide flexibility to them as per the requirement.</li>
<li>Finally, after the assignment deadline passes, the assignment goes into the “Finished” stage. This is when the “Assign grade” link will be visible to the user. Along with this, depending on whether they have reviewed a particular submission or not, the begin, view, update links will be visible to them as they should be able to perform the review even after the assignment deadline for the students has passed.</li>
</ol>

Files Edited 
The following are the files (controllers and views) edited for the implementation of the review process. The comments are given along with the code to explain what is being done. 
* '''Controller: ''' ''app/controllers/response_controller.rb'' 

<pre>
# Ensuring recent submission is not nil
team_most_recent_submission = AssignmentTeam.find(@map.reviewee_id).most_recent_submission
if @response.nil? || (!team_most_recent_submission.nil? && team_most_recent_submission.updated_at > @response.updated_at)
...
# Page should be directed to list_submissions if TA/instructor performs the review
when "ta_review"
redirect_to controller: 'assignments', action: 'list_submissions', id: @map.response_map.assignment.id
</pre>

* '''Controller: ''' ''app/helpers/assignment_helper.rb'' 

<pre>
topic_id = topic.try :id
...
# Needed topic ID for review implpementation, so added it to return list of the function
[topic_id, topic_identifier ||= "", topic_name ||= "", users_for_curr_team, participants]
</pre>

* '''View: ''' ''app/views/assignments/list_submissions.html.erb'' 

<pre>
# Retrieved topic ID for review implpementation
<% topic_id, topic_identifier, topic_name, users_for_curr_team, participants = get_data_for_list_submissions(team) %>
...

<% if @assignment.get_current_stage() == "Finished" %>
<%= link_to 'Assign Grade', { controller: 'grades', action: 'view_team', id: participants.first.id}, target: '_blank' %> 
<% end %>

<% reviewer = Participant.find_by(:parent_id => @assignment.id, :user_id => session[:user].id)%>


<% if !reviewer.nil? %>

<% @mapping = ReviewResponseMap.find_by(:reviewed_object_id => @assignment.id, :reviewer_id => reviewer.id, :reviewee_id => team.id) %>

<% if @mapping.nil? && (@assignment.get_current_stage() == "review" || @assignment.get_current_stage() == "Finished") %>
<% ReviewResponseMap.create(reviewee_id: team.id, reviewer_id: reviewer.id, reviewed_object_id: @assignment.id) %>
<% map = ReviewResponseMap.find_by(reviewee_id: team.id, reviewer_id: reviewer.id, reviewed_object_id: @assignment.id) %>
<%= link_to 'Begin Review', { controller: 'response', action: 'new', :id => map.id, :return => "ta_review" } %> 

<% elsif @assignment.get_current_stage() == "review" || @assignment.get_current_stage() == "Finished" %>
<% map1 = ReviewResponseMap.find_by(reviewee_id: team.id, reviewer_id: reviewer.id, reviewed_object_id: @assignment.id) %>
<% @sorted_responses = Array.new %>
<% @sorted_responses = Response.where(:map_id => map1.id) %>

<% if @sorted_responses.empty?%>
<%= link_to 'Begin Review', { controller: 'response', action: 'new', :id => map1.id, :return => "ta_review" } %> 
<% else %>

<% @sorted_responses = @sorted_responses.sort_by {|obj| obj.updated_at} # the latest should be at the last %>
<% @latest_response = @sorted_responses.last %>
<%if @latest_response.round.nil?%> 
<% last_response_round = AssignmentDueDate.done_in_assignment_round(@assignment.id, @latest_response) %>
<%else%> 
<% last_response_round = @latest_response.round %>
<%end%>
<% current_round = @assignment.number_of_current_round(topic_id) %>

<%= link_to "View Review", {:controller => 'response', :action => 'view', :id => @latest_response.id} %> 
<% if last_response_round == current_round %>
<% latest_submission = team.most_recent_submission %>
<% if !@latest_response.is_submitted %> 
<%= link_to "Edit Review", {:controller => 'response', :action => 'edit', :id => @latest_response.id, :return => "ta_review" } %>
<% else %> 
<%= link_to "Update Review", {:controller => 'response', :action => 'new', :id => map1.id, :return => "ta_review" } %>
<%end%>
<% else %> 
<%= link_to "Update Review", {:controller => 'response', :action => 'new', :id => map1.id, :return => "ta_review" } %>
<% end %>
<% end %>
<% end %>
<% end %>
</pre>

* '''View: ''' ''app/views/grades/view_team.html.erb'' 

<pre>

<% topic_id_num, topic_id, topic_name, users_for_curr_team, participants = get_data_for_list_submissions(@team) %>
</pre>

* '''View: ''' ''app/views/student_review/_responses.html.erb'' 

<pre>
# Ensuring recent submission is not nil
<% elsif !latest_submission.nil? && @latest_response.updated_at < latest_submission.updated_at %>
</pre>

---

=== Part 2: Allow students to identify that their submission has been reviewed by the instructor===
 Currently Scenario: There is no way for the student to know which review was by the instructor. 
Requirement: This helps the student to identify the review made by the instructor from the other reviews. 
Implementation: A simple check is made to determine if the reviewer is a TA or Instructor. app/assets/images/staff.png is displayed to make an instructor performed review stand out from other reviews. If the mouse hovers over the icon a tool tip appears saying "Icon indicates this review was done by course staff" 
[[File:Icon-demo.png|center]]

Files Edited 
* '''Helper: ''' ''app/helpers/grades_helper.rb'' 

<pre>
# Determines if a review was completed by a staff participant (either a TA or an instructor)
# Called in view_team.html to determine if a review should be marked as reviewed by course staff.
def done_by_staff_participant?( review )
role = Role.find(User.find(Participant.find(ResponseMap.find(Response.find(review.id).map_id).reviewer_id).user_id).role_id).name
return (role == "Instructor") || (role == "Teaching Assistant")
end

# Determines if given participant is a staff participant (either a TA or an instructor)
# Called in views/response/view.html.erb to determine if a review should be marked as reviewed by course staff.
def is_course_staff?( participant )
role = Role.find(User.find(Participant.find(participant.id).user_id).role_id).name
return (role == "Instructor") || (role == "Teaching Assistant")
end
</pre>

* '''View: ''' ''app/views/view_team.html.erb'' 
<pre>
<% if done_by_staff_participant?(review) %>

<th class="sorter-false">
<a target="_blank" href="../response/view?id=<%= review.id.to_s %>" data-toggle="tooltip" data-placement="right" title="Click to see details">
<%= "Review " + i.to_s %></a> <img data-toggle="tooltip" data-placement="right" title="Icon indicates this review was done by course staff" class="img_icon" src="/assets/staff.png"> </th>
<% else %>
<th class="sorter-false"> <a target="_blank" href="../response/view?id=<%= review.id.to_s %>" data-toggle="tooltip" data-placement="right" title="Click to see details"><%= "Review " + i.to_s %></a> </th>
<% end %>
</pre>

=== Part 3: Add instructor/TA as a participant while creating the assignment (Extra Feature) ===
 
'''Current scenario''' 
The instructor/TA will be able to perform student reviews (after implementation of Part 1) for the assignment, only when he is a participant of the same assignment. Thus, every time he creates an assignment and wants to add himself as a participant in order to perform student reviews, he is supposed to do the following 
Go to list of assignments -> Find the required assignment -> Click on add participant -> Type user Id -> Click on add
The same flow is as shown below where the instructor/TA has created a new assignment say “Program 1 Github” wherein he wants himself as a participant to be able to perform student reviews :
 
[[File:Addinstructor1.png|center]] 
[[File:AddInstructor2.png]] 
[[File:Add Instructor 3.png]] 

To overcome the hassle, we added a check box through which the instructor/TA will get an option to add himself as a participant while creating the assignment. Also, he will be able to add/remove himself as a participant while editing the assignment.

'''Implementation''' 

The checkbox has been provided to Create New Assignment’s page under the Review strategy tab.
It is added as follows: 

[[File:Add instructor 4.png]] 

If the instructor/TA selects the box while creating the assignment, then he will be added as a participant for the same assignment. 

Files edited:

* '''View: ''' ''app/views/assignments/edit/_review_strategy.html.erb'' 
In this file, code for displaying the checkbox has been added which is as follows :

<pre>
<tr>
<td>
<input name='add_instructor' type="hidden" value="false"/>
<%= check_box_tag('add_instructor', value = "1", checked = is_instructor_a_participant?)%>
<%= label_tag('add_instructor', 'Add yourself as a participant?') %>
<img src="/assets/info.png" title='If you add yourself as a participant then you will be able to perform reviews of students for this assignment'>
</td>
</tr>
</pre>

Thus the checkbox "add_instructor" if selected adds the instructor as a participant to the currently being created assignment. Thus, instructor being a participant allows him to perform reviews for the students for this assignment.

* '''Controller: ''' ''app/controllers/assignments_controller.rb'' 

The file contains extra code which adds the instructor/TA as a participant if the checkbox has been selected while creating the assignment. Also, while editing the assignment, he will be able to add/remove himself as a participant for this assignment. 

The following methods are added to the controller :

<pre>
def is_instructor_a_participant?
instructor_id = session[:user].id
assignment_id = @assignment_form.assignment.id.to_s

if assignment_id != nil
@is_instructor_a_participant = Participant.where(user_id: instructor_id , parent_id: assignment_id)
if @is_instructor_a_participant.present?
return true
end
end
return false
end
</pre>

The method "is_instructor_a_participant?" returns true if the instructor is already a participant for the assignment, else returns false. This method is required to verify that the instructor/TA is not a participant of the assignment, and thus he can be added as a participant if the checkbox has been selected. Also, during editing the assignment, we need to show checkbox being selected if the instructor is a participant and vice versa. Thus, this method serves that purpose as well.

 
<pre>
def add_instructor_as_participant(assignment_id)
if params[:add_instructor] == 'false' and is_instructor_a_participant? == true
delete_instructor_as_participant(assignment_id,session[:user].id)
elsif params[:add_instructor] == '1' and is_instructor_a_participant? == false
current_assignment = Object.const_get("Assignment").find(assignment_id)
current_assignment.add_participant(session[:user].name, true, true, true)
end
end
</pre>

The method "add_instructor_as_participant" is been called from create and update methods in the Assignment controller. It checks if the "Add yourself as participant" checkbox is selected for the currently created/edited assignment. If it is selected during creation or editing, then the instructor is added as a participant to the currently created/edited assignment. The participant is added by using the already existing method "add_participant" in the model assignment.rb Also, if the option is unchecked during editing, then the instructor is removed as a participant.

 
<pre>
def delete_instructor_as_participant(assignment_id , instructor_id)
Participant.where(user_id: instructor_id , parent_id: assignment_id).destroy_all
end
</pre>
The method "delete_instructor_as_participant" removes instructor as a participant for the assignment when the checkbox is unchecked during the editing of the assignment.

== '''Test Plan''' ==

==== Part 1: Allowing instructor/TA to perform review ====
UI Testing

<ol>
<li>Login in as an instructor.
<ul>
<li>Username: instructor6</li>
<li>Password: password</li>
</ul> </li>
<li>Choose any assignment from the list of assignments which involves teams; since an assignment without teams does not include the review procedure. For example, “Final project (design and doc)” is a good assignment to choose.
<li>Now, while testing this functionality, it is recommended to keep the following pages opened in different tabs by clicking on the small icons in the same row as the assignment that you want to test: 

a. “Edit” -> “Due Dates” tab
[[File:edit_assignment.png]] 

b. “Add participants”
[[File:add_as_participant.png]] 

c. “View Submissions”
[[File:list_submissions.png]] 
</li>
<li>Now add the instructor as a participant (3.b). and edit the ALL due dates (3.a.) of different stages to any date later from now (hence the assignment goes into the “submission” stage by default). Save both these changes. </li>
<li>Now refresh the view submissions page (3.c.), you will see a list of assignments with no link to review or assign grade.</li>
<li>Now switch to edit due dates tab (3.a.) again. Set the due date for submission to a previous date which has passed, hence the next due date will be for the “review” stage by default (which will have a later due date from now.)</li>
<li>Refresh view submissions page (3.c.), you will see a list of assignments with a link to “Begin Review.”</li>
<li>You can click on it. If you “save” your response, you will see that the link has been replace by “View review” and “Edit review” now. Whereas, if you “submit” the response, the link will be replaced by “View review” and “Update review.”</li>
<li>Depending on the number of stages that an assignment has, you can edit the due date at every stage and check whether the links are displayed appropriately.</li>
<li>Finally when the last deadline of the assignment is set to a previous date which has passed, you will see the link to “Assign grade” in view submissions (3.c.), and the user will still be able to begin, edit or update a review in order to provide flexibility as per the requirements.</li>
</ol>

Edge Cases:
<ol>
<li>If the user is not a participant of the current assignment, they are shown a list of all submissions made by all teams of the current assignment. They are shown the link to “Assign Grade” only when the final deadline of the assignment passes.</li>
<li>During any review stage or even after the deadline, the user is offered the flexibility to Begin, Edit, View or Update their review.</li>
</ol>

===Part 3: Add instructor/TA as a participant while creating the assignment===
UI Testing
<ol>
<li>Login in as an instructor.
<ul>
<li>Username: instructor6</li>
<li>Password: password</li>
</ul> </li>
<li> Go to Manage-> Assignments </li>
<li> Click on add assignment </li>
<li> Enter the required details- Assignment name, course etc </li>
<li>To add instructor/TA as a participant, go to Review Tab and select the "Add yourself as a participant?" checkbox. </li>
<li> Click on Save </li>

The assignment has been created and the instructor/TA is added as a participant to the assignment.
</ol>

Edge Cases :
<ol>
<li> If the instructor/TA does not select the "Add yourself as a participant?" checkbox while creating the assignment, then he is not added as a participant</li>
<li> If the instructor/TA selects the "Add yourself as a participant?" checkbox while editing the assignment, then he is added as a participant
<li> If the instructor/TA unchecks the "Add yourself as a participant?" checkbox while editing the assignment, then he is removed as a participant </li> </ol>

 
Rspec Testing :
<pre>

describe '#create' do
before(:each) do
allow(AssignmentForm).to receive(:new).with(any_args).and_return(assignment_form)
@params = {
button: '',
assignment_form: {
assignment_questionnaire: [{"assignment_id" => "1", "questionnaire_id" => "666", "dropdown" => "true",
"questionnaire_weight" => "100", "notification_limit" => "15", "used_in_round" => "1"}],
due_date: [{"id" => "", "parent_id" => "", "round" => "1", "deadline_type_id" => "1", "due_at" => "2017/12/05 00:00", "submission_allowed_id" => "3", "review_allowed_id" => "1", "teammate_review_allowed_id" => "3", "review_of_review_allowed_id" => "1", "threshold" => "1"},
{"id" => "", "parent_id" => "", "round" => "1", "deadline_type_id" => "2", "due_at" => "2017/12/02 00:00", "submission_allowed_id" => "1", "review_allowed_id" => "3", "teammate_review_allowed_id" => "3", "review_of_review_allowed_id" => "1", "threshold" => "1"}],
assignment: {
instructor_id: 2,
course_id: 1,
max_team_size: 1,
id: 1,
name: 'test assignment',
directory_path: '/test',
spec_location: '',
private: false,
show_teammate_reviews: false,
require_quiz: false,
num_quiz_questions: 0,
staggered_deadline: false,
microtask: false,
reviews_visible_to_all: false,
is_calibrated: false,
availability_flag: true,
reputation_algorithm: 'Lauw',
simicheck: -1,
simicheck_threshold: 100
add_instructor:1 #added field
}
}
}

context 'when assignment_form is saved successfully' do
it 'redirects to assignment#edit page' do
allow(assignment_form).to receive(:assignment).and_return(assignment)
allow(assignment_form).to receive(:save).and_return(true)
allow(assignment_form).to receive(:update).with(any_args).and_return(true)
allow(assignment_form).to receive(:create_assignment_node).and_return(double('node'))
allow(assignment).to receive(:id).and_return(1)
allow(Assignment).to receive(:find_by).with(name: 'test assignment').and_return(assignment)
allow(assignment_form).to receive(:add_instructor_as_participant).with(any_args).and_return(true) #added check
allow_any_instance_of(AssignmentsController).to receive(:undo_link)
.with('Assignment "test assignment" has been created successfully. ').and_return(true)
post :create, @params
expect(response).to redirect_to('/assignments/1/edit')
end
end

describe '#update' do
context 'when params does not have key :assignment_form' do
context 'when assignment is saved successfully' do
it 'shows a note flash message and redirects to tree_display#index page' do
allow(assignment).to receive(:save).and_return(true)
params = {
id: 1,
course_id: 1
}
session = {user: instructor}
allow(assignment_form).to receive(:add_instructor_as_participant).with(any_args).and_return(true) #added check
post :update, params, session
expect(flash[:note]).to eq('The assignment was successfully saved.')
expect(response).to redirect_to('/tree_display/list')
end
end

context 'when "Add yourself as a participant?" checkbox is selected' do
it 'calls add_instructor_as_participant which in turn calls add Participant method of Participant controller' do
allow(assignment_form).to receive(:add_participant).with(any_args).and_return(true)
end
end

</pre>

== '''External Links''' ==
* Forked Repository: https://github.com/Matrawick/expertiza/tree/beta
* Pull Request: https://github.com/expertiza/expertiza/pull/1631

== '''Team Members''' ==
Mentor: Prof. Edward Gehringer (efg@ncsu.edu)
<ul>
<li>Anuja Kulkarni - apkulka2@ncsu.edu</li>
<li>Mark Trawick - mtrawic@ncsu.edu</li>
<li>Parvathy Menon - pharida@ncsu.edu</li>
<li>Samruddhi Khandale - sskhanda@ncsu.edu</li>
</ul>

CSC/ECE 517 Fall 2019 - E1985. Let course staff and students do reviews

2019-12-06T20:04:29Z

Mtrawic: /* Part 2: Allow students to identify that their submission has been reviewed by the instructor */

== '''Problem Statement''' ==

Peer review is a great way for students to learn about how well they have developed their application. However, sometimes, the peer reviews may not be thorough and the team/person's work reviewed might not reflect the actual status of the development.
By letting course staff perform reviews as well, the reviewer and the reviewee both benefit, improving the overall learning experience. Currently, there exists no method for the instructor/TA to perform reviews of assignments submitted by the students. Hence this is what this project aims at, allowing instructor/TA to perform reviews.

Thus, the primary aim of this project is to allow instructors to submit reviews of student work, using the same review form that students use to do reviews.

=='''Implementation Details''' ==
Our project aims at enabling the instructor/TA to review the submissions using the same form that a student uses while peer-reviewing. For this, our implementation has been divided into 3 parts, and can be described as follows:
<ol>
<li>The first part involves the primary functionality of letting the staff perform a review on students submission.</li>
<li>The second part involves allowing the students to identify from their end, which review has been performed by an instructor/TA. This will help students improve their project as the opinions presented by a staff member would be valid.</li>
<li>Thirdly, we implemented an additional feature that allows the instructor/TA to add himself as a participant while creating/editing an Assignment (This way seemed to be more convenient rather than going to the “Add Participant” page and adding themselves as a participant.)</li>
</ol>

===Part 1: Allowing instructor/TA to perform review===
Note 1: “Instructor/TA” will be referred to by “user” here onwards.

Note 2: To be able to perform a review, the user has to be a participant of that assignment. We have implemented “Add as Participant” as an additional feature which is explained in Part 3. If the user is not a participant, he will only see the link to “Assign grade” that too after the assignment deadline passes.
Now, for each assignment, there is a page called “List Submissions” where the user can view all the submissions. So we have modified this page so that for each team submission a link for “Begin review,” “Edit review,” “View review,” “Update review,” or “Assign grade” may appear depending on the [http://wiki.expertiza.ncsu.edu/index.php/Deadline_types stage] in which the assignment is.

The following describes the flow of our project considering a fresh assignment.
<ol>
<li>When the assignment is created, it is by default is in the “submission” stage. Hence the user will not see any link on the list submission page.</li>
<li>When the first deadline passes, the assignment goes into the “review” stage. Hence the link “Begin review” will be visible for each team submission, where the instructor/TA can click on to submit the review. Two cases can occur here:
<ul>
<li>If the user “submits” a review, they will be redirected to the list submission page where they’ll see the option “View review” or “Update review.” Being an instructor/TA, they are given the flexibility to update their review at any time during the “review” stage.</li>
<li>If the user “saves” a review, they will be redirected to the list submission page where they’ll see the option “View review” and “Edit Review (as they have not submitted it yet.)”</li>
</li>
</ul>
<li>After the first review is over, if more stages of “submissions” and “reviews” exist, it will follow the above 2 steps until the final assignment deadline does not pass. Note that at any round of the review, the user can “Begin review” for a submission which they had not reviewed in the previous rounds. This is to provide flexibility to them as per the requirement.</li>
<li>Finally, after the assignment deadline passes, the assignment goes into the “Finished” stage. This is when the “Assign grade” link will be visible to the user. Along with this, depending on whether they have reviewed a particular submission or not, the begin, view, update links will be visible to them as they should be able to perform the review even after the assignment deadline for the students has passed.</li>
</ol>

Files Edited 
The following are the files (controllers and views) edited for the implementation of the review process. The comments are given along with the code to explain what is being done. 
* '''Controller: ''' ''app/controllers/response_controller.rb'' 

<pre>
# Ensuring recent submission is not nil
team_most_recent_submission = AssignmentTeam.find(@map.reviewee_id).most_recent_submission
if @response.nil? || (!team_most_recent_submission.nil? && team_most_recent_submission.updated_at > @response.updated_at)
...
# Page should be directed to list_submissions if TA/instructor performs the review
when "ta_review"
redirect_to controller: 'assignments', action: 'list_submissions', id: @map.response_map.assignment.id
</pre>

* '''Controller: ''' ''app/helpers/assignment_helper.rb'' 

<pre>
topic_id = topic.try :id
...
# Needed topic ID for review implpementation, so added it to return list of the function
[topic_id, topic_identifier ||= "", topic_name ||= "", users_for_curr_team, participants]
</pre>

* '''View: ''' ''app/views/assignments/list_submissions.html.erb'' 

<pre>
# Retrieved topic ID for review implpementation
<% topic_id, topic_identifier, topic_name, users_for_curr_team, participants = get_data_for_list_submissions(team) %>
...

<% if @assignment.get_current_stage() == "Finished" %>
<%= link_to 'Assign Grade', { controller: 'grades', action: 'view_team', id: participants.first.id}, target: '_blank' %> 
<% end %>

<% reviewer = Participant.find_by(:parent_id => @assignment.id, :user_id => session[:user].id)%>


<% if !reviewer.nil? %>

<% @mapping = ReviewResponseMap.find_by(:reviewed_object_id => @assignment.id, :reviewer_id => reviewer.id, :reviewee_id => team.id) %>

<% if @mapping.nil? && (@assignment.get_current_stage() == "review" || @assignment.get_current_stage() == "Finished") %>
<% ReviewResponseMap.create(reviewee_id: team.id, reviewer_id: reviewer.id, reviewed_object_id: @assignment.id) %>
<% map = ReviewResponseMap.find_by(reviewee_id: team.id, reviewer_id: reviewer.id, reviewed_object_id: @assignment.id) %>
<%= link_to 'Begin Review', { controller: 'response', action: 'new', :id => map.id, :return => "ta_review" } %> 

<% elsif @assignment.get_current_stage() == "review" || @assignment.get_current_stage() == "Finished" %>
<% map1 = ReviewResponseMap.find_by(reviewee_id: team.id, reviewer_id: reviewer.id, reviewed_object_id: @assignment.id) %>
<% @sorted_responses = Array.new %>
<% @sorted_responses = Response.where(:map_id => map1.id) %>

<% if @sorted_responses.empty?%>
<%= link_to 'Begin Review', { controller: 'response', action: 'new', :id => map1.id, :return => "ta_review" } %> 
<% else %>

<% @sorted_responses = @sorted_responses.sort_by {|obj| obj.updated_at} # the latest should be at the last %>
<% @latest_response = @sorted_responses.last %>
<%if @latest_response.round.nil?%> 
<% last_response_round = AssignmentDueDate.done_in_assignment_round(@assignment.id, @latest_response) %>
<%else%> 
<% last_response_round = @latest_response.round %>
<%end%>
<% current_round = @assignment.number_of_current_round(topic_id) %>

<%= link_to "View Review", {:controller => 'response', :action => 'view', :id => @latest_response.id} %> 
<% if last_response_round == current_round %>
<% latest_submission = team.most_recent_submission %>
<% if !@latest_response.is_submitted %> 
<%= link_to "Edit Review", {:controller => 'response', :action => 'edit', :id => @latest_response.id, :return => "ta_review" } %>
<% else %> 
<%= link_to "Update Review", {:controller => 'response', :action => 'new', :id => map1.id, :return => "ta_review" } %>
<%end%>
<% else %> 
<%= link_to "Update Review", {:controller => 'response', :action => 'new', :id => map1.id, :return => "ta_review" } %>
<% end %>
<% end %>
<% end %>
<% end %>
</pre>

* '''View: ''' ''app/views/grades/view_team.html.erb'' 

<pre>

<% topic_id_num, topic_id, topic_name, users_for_curr_team, participants = get_data_for_list_submissions(@team) %>
</pre>

* '''View: ''' ''app/views/student_review/_responses.html.erb'' 

<pre>
# Ensuring recent submission is not nil
<% elsif !latest_submission.nil? && @latest_response.updated_at < latest_submission.updated_at %>
</pre>

---

=== Part 2: Allow students to identify that their submission has been reviewed by the instructor===
 Currently Scenario: There is no way for the student to know which review was by the instructor. 
Requirement: This helps the student to identify the review made by the instructor from the other reviews. 
Implementation: A simple check is made to determine if the reviewer is a TA or Instructor. app/assets/images/staff.png is displayed to make an instructor performed review stand out from other reviews. If the mouse hovers over the icon a tool tip appears saying "Icon indicates this review was done by course staff" 
[[File:Icon-demo.png|center]]

Files Edited 
* '''Helper: ''' ''app/helpers/grades_helper.rb'' 

<pre>
# Determines if a review was completed by a staff participant (either a TA or an instructor)
# Called in view_team.html to determine if a review should be marked as reviewed by course staff.
def done_by_staff_participant?( review )
role = Role.find(User.find(Participant.find(ResponseMap.find(Response.find(review.id).map_id).reviewer_id).user_id).role_id).name
return (role == "Instructor") || (role == "Teaching Assistant")
end

# Determines if given participant is a staff participant (either a TA or an instructor)
# Called in views/response/view.html.erb to determine if a review should be marked as reviewed by course staff.
def is_course_staff?( participant )
role = Role.find(User.find(Participant.find(participant.id).user_id).role_id).name
return (role == "Instructor") || (role == "Teaching Assistant")
end
</pre>

=== Part 3: Add instructor/TA as a participant while creating the assignment (Extra Feature) ===
 
'''Current scenario''' 
The instructor/TA will be able to perform student reviews (after implementation of Part 1) for the assignment, only when he is a participant of the same assignment. Thus, every time he creates an assignment and wants to add himself as a participant in order to perform student reviews, he is supposed to do the following 
Go to list of assignments -> Find the required assignment -> Click on add participant -> Type user Id -> Click on add
The same flow is as shown below where the instructor/TA has created a new assignment say “Program 1 Github” wherein he wants himself as a participant to be able to perform student reviews :
 
[[File:Addinstructor1.png|center]] 
[[File:AddInstructor2.png]] 
[[File:Add Instructor 3.png]] 

To overcome the hassle, we added a check box through which the instructor/TA will get an option to add himself as a participant while creating the assignment. Also, he will be able to add/remove himself as a participant while editing the assignment.

'''Implementation''' 

The checkbox has been provided to Create New Assignment’s page under the Review strategy tab.
It is added as follows: 

[[File:Add instructor 4.png]] 

If the instructor/TA selects the box while creating the assignment, then he will be added as a participant for the same assignment. 

Files edited:

* '''View: ''' ''app/views/assignments/edit/_review_strategy.html.erb'' 
In this file, code for displaying the checkbox has been added which is as follows :

<pre>
<tr>
<td>
<input name='add_instructor' type="hidden" value="false"/>
<%= check_box_tag('add_instructor', value = "1", checked = is_instructor_a_participant?)%>
<%= label_tag('add_instructor', 'Add yourself as a participant?') %>
<img src="/assets/info.png" title='If you add yourself as a participant then you will be able to perform reviews of students for this assignment'>
</td>
</tr>
</pre>

Thus the checkbox "add_instructor" if selected adds the instructor as a participant to the currently being created assignment. Thus, instructor being a participant allows him to perform reviews for the students for this assignment.

* '''Controller: ''' ''app/controllers/assignments_controller.rb'' 

The file contains extra code which adds the instructor/TA as a participant if the checkbox has been selected while creating the assignment. Also, while editing the assignment, he will be able to add/remove himself as a participant for this assignment. 

The following methods are added to the controller :

<pre>
def is_instructor_a_participant?
instructor_id = session[:user].id
assignment_id = @assignment_form.assignment.id.to_s

if assignment_id != nil
@is_instructor_a_participant = Participant.where(user_id: instructor_id , parent_id: assignment_id)
if @is_instructor_a_participant.present?
return true
end
end
return false
end
</pre>

The method "is_instructor_a_participant?" returns true if the instructor is already a participant for the assignment, else returns false. This method is required to verify that the instructor/TA is not a participant of the assignment, and thus he can be added as a participant if the checkbox has been selected. Also, during editing the assignment, we need to show checkbox being selected if the instructor is a participant and vice versa. Thus, this method serves that purpose as well.

 
<pre>
def add_instructor_as_participant(assignment_id)
if params[:add_instructor] == 'false' and is_instructor_a_participant? == true
delete_instructor_as_participant(assignment_id,session[:user].id)
elsif params[:add_instructor] == '1' and is_instructor_a_participant? == false
current_assignment = Object.const_get("Assignment").find(assignment_id)
current_assignment.add_participant(session[:user].name, true, true, true)
end
end
</pre>

The method "add_instructor_as_participant" is been called from create and update methods in the Assignment controller. It checks if the "Add yourself as participant" checkbox is selected for the currently created/edited assignment. If it is selected during creation or editing, then the instructor is added as a participant to the currently created/edited assignment. The participant is added by using the already existing method "add_participant" in the model assignment.rb Also, if the option is unchecked during editing, then the instructor is removed as a participant.

 
<pre>
def delete_instructor_as_participant(assignment_id , instructor_id)
Participant.where(user_id: instructor_id , parent_id: assignment_id).destroy_all
end
</pre>
The method "delete_instructor_as_participant" removes instructor as a participant for the assignment when the checkbox is unchecked during the editing of the assignment.

== '''Test Plan''' ==

==== Part 1: Allowing instructor/TA to perform review ====
UI Testing

<ol>
<li>Login in as an instructor.
<ul>
<li>Username: instructor6</li>
<li>Password: password</li>
</ul> </li>
<li>Choose any assignment from the list of assignments which involves teams; since an assignment without teams does not include the review procedure. For example, “Final project (design and doc)” is a good assignment to choose.
<li>Now, while testing this functionality, it is recommended to keep the following pages opened in different tabs by clicking on the small icons in the same row as the assignment that you want to test: 

a. “Edit” -> “Due Dates” tab
[[File:edit_assignment.png]] 

b. “Add participants”
[[File:add_as_participant.png]] 

c. “View Submissions”
[[File:list_submissions.png]] 
</li>
<li>Now add the instructor as a participant (3.b). and edit the ALL due dates (3.a.) of different stages to any date later from now (hence the assignment goes into the “submission” stage by default). Save both these changes. </li>
<li>Now refresh the view submissions page (3.c.), you will see a list of assignments with no link to review or assign grade.</li>
<li>Now switch to edit due dates tab (3.a.) again. Set the due date for submission to a previous date which has passed, hence the next due date will be for the “review” stage by default (which will have a later due date from now.)</li>
<li>Refresh view submissions page (3.c.), you will see a list of assignments with a link to “Begin Review.”</li>
<li>You can click on it. If you “save” your response, you will see that the link has been replace by “View review” and “Edit review” now. Whereas, if you “submit” the response, the link will be replaced by “View review” and “Update review.”</li>
<li>Depending on the number of stages that an assignment has, you can edit the due date at every stage and check whether the links are displayed appropriately.</li>
<li>Finally when the last deadline of the assignment is set to a previous date which has passed, you will see the link to “Assign grade” in view submissions (3.c.), and the user will still be able to begin, edit or update a review in order to provide flexibility as per the requirements.</li>
</ol>

Edge Cases:
<ol>
<li>If the user is not a participant of the current assignment, they are shown a list of all submissions made by all teams of the current assignment. They are shown the link to “Assign Grade” only when the final deadline of the assignment passes.</li>
<li>During any review stage or even after the deadline, the user is offered the flexibility to Begin, Edit, View or Update their review.</li>
</ol>

===Part 3: Add instructor/TA as a participant while creating the assignment===
UI Testing
<ol>
<li>Login in as an instructor.
<ul>
<li>Username: instructor6</li>
<li>Password: password</li>
</ul> </li>
<li> Go to Manage-> Assignments </li>
<li> Click on add assignment </li>
<li> Enter the required details- Assignment name, course etc </li>
<li>To add instructor/TA as a participant, go to Review Tab and select the "Add yourself as a participant?" checkbox. </li>
<li> Click on Save </li>

The assignment has been created and the instructor/TA is added as a participant to the assignment.
</ol>

Edge Cases :
<ol>
<li> If the instructor/TA does not select the "Add yourself as a participant?" checkbox while creating the assignment, then he is not added as a participant</li>
<li> If the instructor/TA selects the "Add yourself as a participant?" checkbox while editing the assignment, then he is added as a participant
<li> If the instructor/TA unchecks the "Add yourself as a participant?" checkbox while editing the assignment, then he is removed as a participant </li> </ol>

 
Rspec Testing :
<pre>

describe '#create' do
before(:each) do
allow(AssignmentForm).to receive(:new).with(any_args).and_return(assignment_form)
@params = {
button: '',
assignment_form: {
assignment_questionnaire: [{"assignment_id" => "1", "questionnaire_id" => "666", "dropdown" => "true",
"questionnaire_weight" => "100", "notification_limit" => "15", "used_in_round" => "1"}],
due_date: [{"id" => "", "parent_id" => "", "round" => "1", "deadline_type_id" => "1", "due_at" => "2017/12/05 00:00", "submission_allowed_id" => "3", "review_allowed_id" => "1", "teammate_review_allowed_id" => "3", "review_of_review_allowed_id" => "1", "threshold" => "1"},
{"id" => "", "parent_id" => "", "round" => "1", "deadline_type_id" => "2", "due_at" => "2017/12/02 00:00", "submission_allowed_id" => "1", "review_allowed_id" => "3", "teammate_review_allowed_id" => "3", "review_of_review_allowed_id" => "1", "threshold" => "1"}],
assignment: {
instructor_id: 2,
course_id: 1,
max_team_size: 1,
id: 1,
name: 'test assignment',
directory_path: '/test',
spec_location: '',
private: false,
show_teammate_reviews: false,
require_quiz: false,
num_quiz_questions: 0,
staggered_deadline: false,
microtask: false,
reviews_visible_to_all: false,
is_calibrated: false,
availability_flag: true,
reputation_algorithm: 'Lauw',
simicheck: -1,
simicheck_threshold: 100
add_instructor:1 #added field
}
}
}

context 'when assignment_form is saved successfully' do
it 'redirects to assignment#edit page' do
allow(assignment_form).to receive(:assignment).and_return(assignment)
allow(assignment_form).to receive(:save).and_return(true)
allow(assignment_form).to receive(:update).with(any_args).and_return(true)
allow(assignment_form).to receive(:create_assignment_node).and_return(double('node'))
allow(assignment).to receive(:id).and_return(1)
allow(Assignment).to receive(:find_by).with(name: 'test assignment').and_return(assignment)
allow(assignment_form).to receive(:add_instructor_as_participant).with(any_args).and_return(true) #added check
allow_any_instance_of(AssignmentsController).to receive(:undo_link)
.with('Assignment "test assignment" has been created successfully. ').and_return(true)
post :create, @params
expect(response).to redirect_to('/assignments/1/edit')
end
end

describe '#update' do
context 'when params does not have key :assignment_form' do
context 'when assignment is saved successfully' do
it 'shows a note flash message and redirects to tree_display#index page' do
allow(assignment).to receive(:save).and_return(true)
params = {
id: 1,
course_id: 1
}
session = {user: instructor}
allow(assignment_form).to receive(:add_instructor_as_participant).with(any_args).and_return(true) #added check
post :update, params, session
expect(flash[:note]).to eq('The assignment was successfully saved.')
expect(response).to redirect_to('/tree_display/list')
end
end

context 'when "Add yourself as a participant?" checkbox is selected' do
it 'calls add_instructor_as_participant which in turn calls add Participant method of Participant controller' do
allow(assignment_form).to receive(:add_participant).with(any_args).and_return(true)
end
end

</pre>

== '''External Links''' ==
* Forked Repository: https://github.com/Matrawick/expertiza/tree/beta
* Pull Request: https://github.com/expertiza/expertiza/pull/1631

== '''Team Members''' ==
Mentor: Prof. Edward Gehringer (efg@ncsu.edu)
<ul>
<li>Anuja Kulkarni - apkulka2@ncsu.edu</li>
<li>Mark Trawick - mtrawic@ncsu.edu</li>
<li>Parvathy Menon - pharida@ncsu.edu</li>
<li>Samruddhi Khandale - sskhanda@ncsu.edu</li>
</ul>

CSC/ECE 517 Fall 2019 - E1985. Let course staff and students do reviews

2019-12-06T19:56:30Z

Mtrawic: /* Part 2: Allow students to identify that their submission has been reviewed by the instructor */

== '''Problem Statement''' ==

Peer review is a great way for students to learn about how well they have developed their application. However, sometimes, the peer reviews may not be thorough and the team/person's work reviewed might not reflect the actual status of the development.
By letting course staff perform reviews as well, the reviewer and the reviewee both benefit, improving the overall learning experience. Currently, there exists no method for the instructor/TA to perform reviews of assignments submitted by the students. Hence this is what this project aims at, allowing instructor/TA to perform reviews.

Thus, the primary aim of this project is to allow instructors to submit reviews of student work, using the same review form that students use to do reviews.

=='''Implementation Details''' ==
Our project aims at enabling the instructor/TA to review the submissions using the same form that a student uses while peer-reviewing. For this, our implementation has been divided into 3 parts, and can be described as follows:
<ol>
<li>The first part involves the primary functionality of letting the staff perform a review on students submission.</li>
<li>The second part involves allowing the students to identify from their end, which review has been performed by an instructor/TA. This will help students improve their project as the opinions presented by a staff member would be valid.</li>
<li>Thirdly, we implemented an additional feature that allows the instructor/TA to add himself as a participant while creating/editing an Assignment (This way seemed to be more convenient rather than going to the “Add Participant” page and adding themselves as a participant.)</li>
</ol>

===Part 1: Allowing instructor/TA to perform review===
Note 1: “Instructor/TA” will be referred to by “user” here onwards.

Note 2: To be able to perform a review, the user has to be a participant of that assignment. We have implemented “Add as Participant” as an additional feature which is explained in Part 3. If the user is not a participant, he will only see the link to “Assign grade” that too after the assignment deadline passes.
Now, for each assignment, there is a page called “List Submissions” where the user can view all the submissions. So we have modified this page so that for each team submission a link for “Begin review,” “Edit review,” “View review,” “Update review,” or “Assign grade” may appear depending on the [http://wiki.expertiza.ncsu.edu/index.php/Deadline_types stage] in which the assignment is.

The following describes the flow of our project considering a fresh assignment.
<ol>
<li>When the assignment is created, it is by default is in the “submission” stage. Hence the user will not see any link on the list submission page.</li>
<li>When the first deadline passes, the assignment goes into the “review” stage. Hence the link “Begin review” will be visible for each team submission, where the instructor/TA can click on to submit the review. Two cases can occur here:
<ul>
<li>If the user “submits” a review, they will be redirected to the list submission page where they’ll see the option “View review” or “Update review.” Being an instructor/TA, they are given the flexibility to update their review at any time during the “review” stage.</li>
<li>If the user “saves” a review, they will be redirected to the list submission page where they’ll see the option “View review” and “Edit Review (as they have not submitted it yet.)”</li>
</li>
</ul>
<li>After the first review is over, if more stages of “submissions” and “reviews” exist, it will follow the above 2 steps until the final assignment deadline does not pass. Note that at any round of the review, the user can “Begin review” for a submission which they had not reviewed in the previous rounds. This is to provide flexibility to them as per the requirement.</li>
<li>Finally, after the assignment deadline passes, the assignment goes into the “Finished” stage. This is when the “Assign grade” link will be visible to the user. Along with this, depending on whether they have reviewed a particular submission or not, the begin, view, update links will be visible to them as they should be able to perform the review even after the assignment deadline for the students has passed.</li>
</ol>

Files Edited 
The following are the files (controllers and views) edited for the implementation of the review process. The comments are given along with the code to explain what is being done. 
* '''Controller: ''' ''app/controllers/response_controller.rb'' 

<pre>
# Ensuring recent submission is not nil
team_most_recent_submission = AssignmentTeam.find(@map.reviewee_id).most_recent_submission
if @response.nil? || (!team_most_recent_submission.nil? && team_most_recent_submission.updated_at > @response.updated_at)
...
# Page should be directed to list_submissions if TA/instructor performs the review
when "ta_review"
redirect_to controller: 'assignments', action: 'list_submissions', id: @map.response_map.assignment.id
</pre>

* '''Controller: ''' ''app/helpers/assignment_helper.rb'' 

<pre>
topic_id = topic.try :id
...
# Needed topic ID for review implpementation, so added it to return list of the function
[topic_id, topic_identifier ||= "", topic_name ||= "", users_for_curr_team, participants]
</pre>

* '''View: ''' ''app/views/assignments/list_submissions.html.erb'' 

<pre>
# Retrieved topic ID for review implpementation
<% topic_id, topic_identifier, topic_name, users_for_curr_team, participants = get_data_for_list_submissions(team) %>
...

<% if @assignment.get_current_stage() == "Finished" %>
<%= link_to 'Assign Grade', { controller: 'grades', action: 'view_team', id: participants.first.id}, target: '_blank' %> 
<% end %>

<% reviewer = Participant.find_by(:parent_id => @assignment.id, :user_id => session[:user].id)%>


<% if !reviewer.nil? %>

<% @mapping = ReviewResponseMap.find_by(:reviewed_object_id => @assignment.id, :reviewer_id => reviewer.id, :reviewee_id => team.id) %>

<% if @mapping.nil? && (@assignment.get_current_stage() == "review" || @assignment.get_current_stage() == "Finished") %>
<% ReviewResponseMap.create(reviewee_id: team.id, reviewer_id: reviewer.id, reviewed_object_id: @assignment.id) %>
<% map = ReviewResponseMap.find_by(reviewee_id: team.id, reviewer_id: reviewer.id, reviewed_object_id: @assignment.id) %>
<%= link_to 'Begin Review', { controller: 'response', action: 'new', :id => map.id, :return => "ta_review" } %> 

<% elsif @assignment.get_current_stage() == "review" || @assignment.get_current_stage() == "Finished" %>
<% map1 = ReviewResponseMap.find_by(reviewee_id: team.id, reviewer_id: reviewer.id, reviewed_object_id: @assignment.id) %>
<% @sorted_responses = Array.new %>
<% @sorted_responses = Response.where(:map_id => map1.id) %>

<% if @sorted_responses.empty?%>
<%= link_to 'Begin Review', { controller: 'response', action: 'new', :id => map1.id, :return => "ta_review" } %> 
<% else %>

<% @sorted_responses = @sorted_responses.sort_by {|obj| obj.updated_at} # the latest should be at the last %>
<% @latest_response = @sorted_responses.last %>
<%if @latest_response.round.nil?%> 
<% last_response_round = AssignmentDueDate.done_in_assignment_round(@assignment.id, @latest_response) %>
<%else%> 
<% last_response_round = @latest_response.round %>
<%end%>
<% current_round = @assignment.number_of_current_round(topic_id) %>

<%= link_to "View Review", {:controller => 'response', :action => 'view', :id => @latest_response.id} %> 
<% if last_response_round == current_round %>
<% latest_submission = team.most_recent_submission %>
<% if !@latest_response.is_submitted %> 
<%= link_to "Edit Review", {:controller => 'response', :action => 'edit', :id => @latest_response.id, :return => "ta_review" } %>
<% else %> 
<%= link_to "Update Review", {:controller => 'response', :action => 'new', :id => map1.id, :return => "ta_review" } %>
<%end%>
<% else %> 
<%= link_to "Update Review", {:controller => 'response', :action => 'new', :id => map1.id, :return => "ta_review" } %>
<% end %>
<% end %>
<% end %>
<% end %>
</pre>

* '''View: ''' ''app/views/grades/view_team.html.erb'' 

<pre>

<% topic_id_num, topic_id, topic_name, users_for_curr_team, participants = get_data_for_list_submissions(@team) %>
</pre>

* '''View: ''' ''app/views/student_review/_responses.html.erb'' 

<pre>
# Ensuring recent submission is not nil
<% elsif !latest_submission.nil? && @latest_response.updated_at < latest_submission.updated_at %>
</pre>

---

=== Part 2: Allow students to identify that their submission has been reviewed by the instructor===
 Currently Scenario: There is no way for the student to know which review was by the instructor. 
Requirement: This helps the student to identify the review made by the instructor from the other reviews. 
Implementation: A simple check is made to determine if the reviewer is a TA or Instructor. app/assets/images/staff.png is displayed to make an instructor performed review stand out from other reviews. If the mouse hovers over the icon a tool tip appears saying "Icon indicates this review was done by course staff" 
[[File:Icon-demo.png|center]]

In grades_helper.rb the below method was added to determine if the reviewer is part of teaching staff.
<pre>
# Determines if a review was completed by a staff participant (either a TA or an instructor)
# Called in view_team.html to determine if a review should be marked as reviewed by course staff.
def done_by_staff_participant?( review )
role = Role.find(User.find(Participant.find(ResponseMap.find(Response.find(review.id).map_id).reviewer_id).user_id).role_id).name
return (role == "Instructor") || (role == "Teaching Assistant")
end
</pre>

=== Part 3: Add instructor/TA as a participant while creating the assignment (Extra Feature) ===
 
'''Current scenario''' 
The instructor/TA will be able to perform student reviews (after implementation of Part 1) for the assignment, only when he is a participant of the same assignment. Thus, every time he creates an assignment and wants to add himself as a participant in order to perform student reviews, he is supposed to do the following 
Go to list of assignments -> Find the required assignment -> Click on add participant -> Type user Id -> Click on add
The same flow is as shown below where the instructor/TA has created a new assignment say “Program 1 Github” wherein he wants himself as a participant to be able to perform student reviews :
 
[[File:Addinstructor1.png|center]] 
[[File:AddInstructor2.png]] 
[[File:Add Instructor 3.png]] 

To overcome the hassle, we added a check box through which the instructor/TA will get an option to add himself as a participant while creating the assignment. Also, he will be able to add/remove himself as a participant while editing the assignment.

'''Implementation''' 

The checkbox has been provided to Create New Assignment’s page under the Review strategy tab.
It is added as follows: 

[[File:Add instructor 4.png]] 

If the instructor/TA selects the box while creating the assignment, then he will be added as a participant for the same assignment. 

Files edited:

* '''View: ''' ''app/views/assignments/edit/_review_strategy.html.erb'' 
In this file, code for displaying the checkbox has been added which is as follows :

<pre>
<tr>
<td>
<input name='add_instructor' type="hidden" value="false"/>
<%= check_box_tag('add_instructor', value = "1", checked = is_instructor_a_participant?)%>
<%= label_tag('add_instructor', 'Add yourself as a participant?') %>
<img src="/assets/info.png" title='If you add yourself as a participant then you will be able to perform reviews of students for this assignment'>
</td>
</tr>
</pre>

Thus the checkbox "add_instructor" if selected adds the instructor as a participant to the currently being created assignment. Thus, instructor being a participant allows him to perform reviews for the students for this assignment.

* '''Controller: ''' ''app/controllers/assignments_controller.rb'' 

The file contains extra code which adds the instructor/TA as a participant if the checkbox has been selected while creating the assignment. Also, while editing the assignment, he will be able to add/remove himself as a participant for this assignment. 

The following methods are added to the controller :

<pre>
def is_instructor_a_participant?
instructor_id = session[:user].id
assignment_id = @assignment_form.assignment.id.to_s

if assignment_id != nil
@is_instructor_a_participant = Participant.where(user_id: instructor_id , parent_id: assignment_id)
if @is_instructor_a_participant.present?
return true
end
end
return false
end
</pre>

The method "is_instructor_a_participant?" returns true if the instructor is already a participant for the assignment, else returns false. This method is required to verify that the instructor/TA is not a participant of the assignment, and thus he can be added as a participant if the checkbox has been selected. Also, during editing the assignment, we need to show checkbox being selected if the instructor is a participant and vice versa. Thus, this method serves that purpose as well.

 
<pre>
def add_instructor_as_participant(assignment_id)
if params[:add_instructor] == 'false' and is_instructor_a_participant? == true
delete_instructor_as_participant(assignment_id,session[:user].id)
elsif params[:add_instructor] == '1' and is_instructor_a_participant? == false
current_assignment = Object.const_get("Assignment").find(assignment_id)
current_assignment.add_participant(session[:user].name, true, true, true)
end
end
</pre>

The method "add_instructor_as_participant" is been called from create and update methods in the Assignment controller. It checks if the "Add yourself as participant" checkbox is selected for the currently created/edited assignment. If it is selected during creation or editing, then the instructor is added as a participant to the currently created/edited assignment. The participant is added by using the already existing method "add_participant" in the model assignment.rb Also, if the option is unchecked during editing, then the instructor is removed as a participant.

 
<pre>
def delete_instructor_as_participant(assignment_id , instructor_id)
Participant.where(user_id: instructor_id , parent_id: assignment_id).destroy_all
end
</pre>
The method "delete_instructor_as_participant" removes instructor as a participant for the assignment when the checkbox is unchecked during the editing of the assignment.

== '''Test Plan''' ==

==== Part 1: Allowing instructor/TA to perform review ====
UI Testing

<ol>
<li>Login in as an instructor.
<ul>
<li>Username: instructor6</li>
<li>Password: password</li>
</ul> </li>
<li>Choose any assignment from the list of assignments which involves teams; since an assignment without teams does not include the review procedure. For example, “Final project (design and doc)” is a good assignment to choose.
<li>Now, while testing this functionality, it is recommended to keep the following pages opened in different tabs by clicking on the small icons in the same row as the assignment that you want to test: 

a. “Edit” -> “Due Dates” tab
[[File:edit_assignment.png]] 

b. “Add participants”
[[File:add_as_participant.png]] 

c. “View Submissions”
[[File:list_submissions.png]] 
</li>
<li>Now add the instructor as a participant (3.b). and edit the ALL due dates (3.a.) of different stages to any date later from now (hence the assignment goes into the “submission” stage by default). Save both these changes. </li>
<li>Now refresh the view submissions page (3.c.), you will see a list of assignments with no link to review or assign grade.</li>
<li>Now switch to edit due dates tab (3.a.) again. Set the due date for submission to a previous date which has passed, hence the next due date will be for the “review” stage by default (which will have a later due date from now.)</li>
<li>Refresh view submissions page (3.c.), you will see a list of assignments with a link to “Begin Review.”</li>
<li>You can click on it. If you “save” your response, you will see that the link has been replace by “View review” and “Edit review” now. Whereas, if you “submit” the response, the link will be replaced by “View review” and “Update review.”</li>
<li>Depending on the number of stages that an assignment has, you can edit the due date at every stage and check whether the links are displayed appropriately.</li>
<li>Finally when the last deadline of the assignment is set to a previous date which has passed, you will see the link to “Assign grade” in view submissions (3.c.), and the user will still be able to begin, edit or update a review in order to provide flexibility as per the requirements.</li>
</ol>

Edge Cases:
<ol>
<li>If the user is not a participant of the current assignment, they are shown a list of all submissions made by all teams of the current assignment. They are shown the link to “Assign Grade” only when the final deadline of the assignment passes.</li>
<li>During any review stage or even after the deadline, the user is offered the flexibility to Begin, Edit, View or Update their review.</li>
</ol>

===Part 3: Add instructor/TA as a participant while creating the assignment===
UI Testing
<ol>
<li>Login in as an instructor.
<ul>
<li>Username: instructor6</li>
<li>Password: password</li>
</ul> </li>
<li> Go to Manage-> Assignments </li>
<li> Click on add assignment </li>
<li> Enter the required details- Assignment name, course etc </li>
<li>To add instructor/TA as a participant, go to Review Tab and select the "Add yourself as a participant?" checkbox. </li>
<li> Click on Save </li>

The assignment has been created and the instructor/TA is added as a participant to the assignment.
</ol>

Edge Cases :
<ol>
<li> If the instructor/TA does not select the "Add yourself as a participant?" checkbox while creating the assignment, then he is not added as a participant</li>
<li> If the instructor/TA selects the "Add yourself as a participant?" checkbox while editing the assignment, then he is added as a participant
<li> If the instructor/TA unchecks the "Add yourself as a participant?" checkbox while editing the assignment, then he is removed as a participant </li> </ol>

 
Rspec Testing :
<pre>

describe '#create' do
before(:each) do
allow(AssignmentForm).to receive(:new).with(any_args).and_return(assignment_form)
@params = {
button: '',
assignment_form: {
assignment_questionnaire: [{"assignment_id" => "1", "questionnaire_id" => "666", "dropdown" => "true",
"questionnaire_weight" => "100", "notification_limit" => "15", "used_in_round" => "1"}],
due_date: [{"id" => "", "parent_id" => "", "round" => "1", "deadline_type_id" => "1", "due_at" => "2017/12/05 00:00", "submission_allowed_id" => "3", "review_allowed_id" => "1", "teammate_review_allowed_id" => "3", "review_of_review_allowed_id" => "1", "threshold" => "1"},
{"id" => "", "parent_id" => "", "round" => "1", "deadline_type_id" => "2", "due_at" => "2017/12/02 00:00", "submission_allowed_id" => "1", "review_allowed_id" => "3", "teammate_review_allowed_id" => "3", "review_of_review_allowed_id" => "1", "threshold" => "1"}],
assignment: {
instructor_id: 2,
course_id: 1,
max_team_size: 1,
id: 1,
name: 'test assignment',
directory_path: '/test',
spec_location: '',
private: false,
show_teammate_reviews: false,
require_quiz: false,
num_quiz_questions: 0,
staggered_deadline: false,
microtask: false,
reviews_visible_to_all: false,
is_calibrated: false,
availability_flag: true,
reputation_algorithm: 'Lauw',
simicheck: -1,
simicheck_threshold: 100
add_instructor:1 #added field
}
}
}

context 'when assignment_form is saved successfully' do
it 'redirects to assignment#edit page' do
allow(assignment_form).to receive(:assignment).and_return(assignment)
allow(assignment_form).to receive(:save).and_return(true)
allow(assignment_form).to receive(:update).with(any_args).and_return(true)
allow(assignment_form).to receive(:create_assignment_node).and_return(double('node'))
allow(assignment).to receive(:id).and_return(1)
allow(Assignment).to receive(:find_by).with(name: 'test assignment').and_return(assignment)
allow(assignment_form).to receive(:add_instructor_as_participant).with(any_args).and_return(true) #added check
allow_any_instance_of(AssignmentsController).to receive(:undo_link)
.with('Assignment "test assignment" has been created successfully. ').and_return(true)
post :create, @params
expect(response).to redirect_to('/assignments/1/edit')
end
end

describe '#update' do
context 'when params does not have key :assignment_form' do
context 'when assignment is saved successfully' do
it 'shows a note flash message and redirects to tree_display#index page' do
allow(assignment).to receive(:save).and_return(true)
params = {
id: 1,
course_id: 1
}
session = {user: instructor}
allow(assignment_form).to receive(:add_instructor_as_participant).with(any_args).and_return(true) #added check
post :update, params, session
expect(flash[:note]).to eq('The assignment was successfully saved.')
expect(response).to redirect_to('/tree_display/list')
end
end

context 'when "Add yourself as a participant?" checkbox is selected' do
it 'calls add_instructor_as_participant which in turn calls add Participant method of Participant controller' do
allow(assignment_form).to receive(:add_participant).with(any_args).and_return(true)
end
end

</pre>

== '''External Links''' ==
* Forked Repository: https://github.com/Matrawick/expertiza/tree/beta
* Pull Request: https://github.com/expertiza/expertiza/pull/1631

== '''Team Members''' ==
Mentor: Prof. Edward Gehringer (efg@ncsu.edu)
<ul>
<li>Anuja Kulkarni - apkulka2@ncsu.edu</li>
<li>Mark Trawick - mtrawic@ncsu.edu</li>
<li>Parvathy Menon - pharida@ncsu.edu</li>
<li>Samruddhi Khandale - sskhanda@ncsu.edu</li>
</ul>

CSC/ECE 517 Fall 2019 - E1985. Let course staff and students do reviews

2019-12-06T19:54:59Z

Mtrawic: /* Part 2: Allow students to identify that their submission has been reviewed by the instructor */

== '''Problem Statement''' ==

Peer review is a great way for students to learn about how well they have developed their application. However, sometimes, the peer reviews may not be thorough and the team/person's work reviewed might not reflect the actual status of the development.
By letting course staff perform reviews as well, the reviewer and the reviewee both benefit, improving the overall learning experience. Currently, there exists no method for the instructor/TA to perform reviews of assignments submitted by the students. Hence this is what this project aims at, allowing instructor/TA to perform reviews.

Thus, the primary aim of this project is to allow instructors to submit reviews of student work, using the same review form that students use to do reviews.

=='''Implementation Details''' ==
Our project aims at enabling the instructor/TA to review the submissions using the same form that a student uses while peer-reviewing. For this, our implementation has been divided into 3 parts, and can be described as follows:
<ol>
<li>The first part involves the primary functionality of letting the staff perform a review on students submission.</li>
<li>The second part involves allowing the students to identify from their end, which review has been performed by an instructor/TA. This will help students improve their project as the opinions presented by a staff member would be valid.</li>
<li>Thirdly, we implemented an additional feature that allows the instructor/TA to add himself as a participant while creating/editing an Assignment (This way seemed to be more convenient rather than going to the “Add Participant” page and adding themselves as a participant.)</li>
</ol>

===Part 1: Allowing instructor/TA to perform review===
Note 1: “Instructor/TA” will be referred to by “user” here onwards.

Note 2: To be able to perform a review, the user has to be a participant of that assignment. We have implemented “Add as Participant” as an additional feature which is explained in Part 3. If the user is not a participant, he will only see the link to “Assign grade” that too after the assignment deadline passes.
Now, for each assignment, there is a page called “List Submissions” where the user can view all the submissions. So we have modified this page so that for each team submission a link for “Begin review,” “Edit review,” “View review,” “Update review,” or “Assign grade” may appear depending on the [http://wiki.expertiza.ncsu.edu/index.php/Deadline_types stage] in which the assignment is.

The following describes the flow of our project considering a fresh assignment.
<ol>
<li>When the assignment is created, it is by default is in the “submission” stage. Hence the user will not see any link on the list submission page.</li>
<li>When the first deadline passes, the assignment goes into the “review” stage. Hence the link “Begin review” will be visible for each team submission, where the instructor/TA can click on to submit the review. Two cases can occur here:
<ul>
<li>If the user “submits” a review, they will be redirected to the list submission page where they’ll see the option “View review” or “Update review.” Being an instructor/TA, they are given the flexibility to update their review at any time during the “review” stage.</li>
<li>If the user “saves” a review, they will be redirected to the list submission page where they’ll see the option “View review” and “Edit Review (as they have not submitted it yet.)”</li>
</li>
</ul>
<li>After the first review is over, if more stages of “submissions” and “reviews” exist, it will follow the above 2 steps until the final assignment deadline does not pass. Note that at any round of the review, the user can “Begin review” for a submission which they had not reviewed in the previous rounds. This is to provide flexibility to them as per the requirement.</li>
<li>Finally, after the assignment deadline passes, the assignment goes into the “Finished” stage. This is when the “Assign grade” link will be visible to the user. Along with this, depending on whether they have reviewed a particular submission or not, the begin, view, update links will be visible to them as they should be able to perform the review even after the assignment deadline for the students has passed.</li>
</ol>

Files Edited 
The following are the files (controllers and views) edited for the implementation of the review process. The comments are given along with the code to explain what is being done. 
* '''Controller: ''' ''app/controllers/response_controller.rb'' 

<pre>
# Ensuring recent submission is not nil
team_most_recent_submission = AssignmentTeam.find(@map.reviewee_id).most_recent_submission
if @response.nil? || (!team_most_recent_submission.nil? && team_most_recent_submission.updated_at > @response.updated_at)
...
# Page should be directed to list_submissions if TA/instructor performs the review
when "ta_review"
redirect_to controller: 'assignments', action: 'list_submissions', id: @map.response_map.assignment.id
</pre>

* '''Controller: ''' ''app/helpers/assignment_helper.rb'' 

<pre>
topic_id = topic.try :id
...
# Needed topic ID for review implpementation, so added it to return list of the function
[topic_id, topic_identifier ||= "", topic_name ||= "", users_for_curr_team, participants]
</pre>

* '''View: ''' ''app/views/assignments/list_submissions.html.erb'' 

<pre>
# Retrieved topic ID for review implpementation
<% topic_id, topic_identifier, topic_name, users_for_curr_team, participants = get_data_for_list_submissions(team) %>
...

<% if @assignment.get_current_stage() == "Finished" %>
<%= link_to 'Assign Grade', { controller: 'grades', action: 'view_team', id: participants.first.id}, target: '_blank' %> 
<% end %>

<% reviewer = Participant.find_by(:parent_id => @assignment.id, :user_id => session[:user].id)%>


<% if !reviewer.nil? %>

<% @mapping = ReviewResponseMap.find_by(:reviewed_object_id => @assignment.id, :reviewer_id => reviewer.id, :reviewee_id => team.id) %>

<% if @mapping.nil? && (@assignment.get_current_stage() == "review" || @assignment.get_current_stage() == "Finished") %>
<% ReviewResponseMap.create(reviewee_id: team.id, reviewer_id: reviewer.id, reviewed_object_id: @assignment.id) %>
<% map = ReviewResponseMap.find_by(reviewee_id: team.id, reviewer_id: reviewer.id, reviewed_object_id: @assignment.id) %>
<%= link_to 'Begin Review', { controller: 'response', action: 'new', :id => map.id, :return => "ta_review" } %> 

<% elsif @assignment.get_current_stage() == "review" || @assignment.get_current_stage() == "Finished" %>
<% map1 = ReviewResponseMap.find_by(reviewee_id: team.id, reviewer_id: reviewer.id, reviewed_object_id: @assignment.id) %>
<% @sorted_responses = Array.new %>
<% @sorted_responses = Response.where(:map_id => map1.id) %>

<% if @sorted_responses.empty?%>
<%= link_to 'Begin Review', { controller: 'response', action: 'new', :id => map1.id, :return => "ta_review" } %> 
<% else %>

<% @sorted_responses = @sorted_responses.sort_by {|obj| obj.updated_at} # the latest should be at the last %>
<% @latest_response = @sorted_responses.last %>
<%if @latest_response.round.nil?%> 
<% last_response_round = AssignmentDueDate.done_in_assignment_round(@assignment.id, @latest_response) %>
<%else%> 
<% last_response_round = @latest_response.round %>
<%end%>
<% current_round = @assignment.number_of_current_round(topic_id) %>

<%= link_to "View Review", {:controller => 'response', :action => 'view', :id => @latest_response.id} %> 
<% if last_response_round == current_round %>
<% latest_submission = team.most_recent_submission %>
<% if !@latest_response.is_submitted %> 
<%= link_to "Edit Review", {:controller => 'response', :action => 'edit', :id => @latest_response.id, :return => "ta_review" } %>
<% else %> 
<%= link_to "Update Review", {:controller => 'response', :action => 'new', :id => map1.id, :return => "ta_review" } %>
<%end%>
<% else %> 
<%= link_to "Update Review", {:controller => 'response', :action => 'new', :id => map1.id, :return => "ta_review" } %>
<% end %>
<% end %>
<% end %>
<% end %>
</pre>

* '''View: ''' ''app/views/grades/view_team.html.erb'' 

<pre>

<% topic_id_num, topic_id, topic_name, users_for_curr_team, participants = get_data_for_list_submissions(@team) %>
</pre>

* '''View: ''' ''app/views/student_review/_responses.html.erb'' 

<pre>
# Ensuring recent submission is not nil
<% elsif !latest_submission.nil? && @latest_response.updated_at < latest_submission.updated_at %>
</pre>

---

=== Part 2: Allow students to identify that their submission has been reviewed by the instructor===
 Currently Scenario: There is no way for the student to know which review was by the instructor. 
Requirement: This helps the student to identify the review made by the instructor from the other reviews. 
Implementation: A simple check is made to determine if the reviewer is a TA or Instructor. app/assets/images/staff.png is displayed to make an instructor performed review stand out from other reviews. 
[[File:Icon-demo.png|center]]

In grades_helper.rb the below method was added to determine if the reviewer is part of teaching staff.
<pre>
# Determines if a review was completed by a staff participant (either a TA or an instructor)
# Called in view_team.html to determine if a review should be marked as reviewed by course staff.
def done_by_staff_participant?( review )
role = Role.find(User.find(Participant.find(ResponseMap.find(Response.find(review.id).map_id).reviewer_id).user_id).role_id).name
return (role == "Instructor") || (role == "Teaching Assistant")
end
</pre>

=== Part 3: Add instructor/TA as a participant while creating the assignment (Extra Feature) ===
 
'''Current scenario''' 
The instructor/TA will be able to perform student reviews (after implementation of Part 1) for the assignment, only when he is a participant of the same assignment. Thus, every time he creates an assignment and wants to add himself as a participant in order to perform student reviews, he is supposed to do the following 
Go to list of assignments -> Find the required assignment -> Click on add participant -> Type user Id -> Click on add
The same flow is as shown below where the instructor/TA has created a new assignment say “Program 1 Github” wherein he wants himself as a participant to be able to perform student reviews :
 
[[File:Addinstructor1.png|center]] 
[[File:AddInstructor2.png]] 
[[File:Add Instructor 3.png]] 

To overcome the hassle, we added a check box through which the instructor/TA will get an option to add himself as a participant while creating the assignment. Also, he will be able to add/remove himself as a participant while editing the assignment.

'''Implementation''' 

The checkbox has been provided to Create New Assignment’s page under the Review strategy tab.
It is added as follows: 

[[File:Add instructor 4.png]] 

If the instructor/TA selects the box while creating the assignment, then he will be added as a participant for the same assignment. 

Files edited:

* '''View: ''' ''app/views/assignments/edit/_review_strategy.html.erb'' 
In this file, code for displaying the checkbox has been added which is as follows :

<pre>
<tr>
<td>
<input name='add_instructor' type="hidden" value="false"/>
<%= check_box_tag('add_instructor', value = "1", checked = is_instructor_a_participant?)%>
<%= label_tag('add_instructor', 'Add yourself as a participant?') %>
<img src="/assets/info.png" title='If you add yourself as a participant then you will be able to perform reviews of students for this assignment'>
</td>
</tr>
</pre>

Thus the checkbox "add_instructor" if selected adds the instructor as a participant to the currently being created assignment. Thus, instructor being a participant allows him to perform reviews for the students for this assignment.

* '''Controller: ''' ''app/controllers/assignments_controller.rb'' 

The file contains extra code which adds the instructor/TA as a participant if the checkbox has been selected while creating the assignment. Also, while editing the assignment, he will be able to add/remove himself as a participant for this assignment. 

The following methods are added to the controller :

<pre>
def is_instructor_a_participant?
instructor_id = session[:user].id
assignment_id = @assignment_form.assignment.id.to_s

if assignment_id != nil
@is_instructor_a_participant = Participant.where(user_id: instructor_id , parent_id: assignment_id)
if @is_instructor_a_participant.present?
return true
end
end
return false
end
</pre>

The method "is_instructor_a_participant?" returns true if the instructor is already a participant for the assignment, else returns false. This method is required to verify that the instructor/TA is not a participant of the assignment, and thus he can be added as a participant if the checkbox has been selected. Also, during editing the assignment, we need to show checkbox being selected if the instructor is a participant and vice versa. Thus, this method serves that purpose as well.

 
<pre>
def add_instructor_as_participant(assignment_id)
if params[:add_instructor] == 'false' and is_instructor_a_participant? == true
delete_instructor_as_participant(assignment_id,session[:user].id)
elsif params[:add_instructor] == '1' and is_instructor_a_participant? == false
current_assignment = Object.const_get("Assignment").find(assignment_id)
current_assignment.add_participant(session[:user].name, true, true, true)
end
end
</pre>

The method "add_instructor_as_participant" is been called from create and update methods in the Assignment controller. It checks if the "Add yourself as participant" checkbox is selected for the currently created/edited assignment. If it is selected during creation or editing, then the instructor is added as a participant to the currently created/edited assignment. The participant is added by using the already existing method "add_participant" in the model assignment.rb Also, if the option is unchecked during editing, then the instructor is removed as a participant.

 
<pre>
def delete_instructor_as_participant(assignment_id , instructor_id)
Participant.where(user_id: instructor_id , parent_id: assignment_id).destroy_all
end
</pre>
The method "delete_instructor_as_participant" removes instructor as a participant for the assignment when the checkbox is unchecked during the editing of the assignment.

== '''Test Plan''' ==

==== Part 1: Allowing instructor/TA to perform review ====
UI Testing

<ol>
<li>Login in as an instructor.
<ul>
<li>Username: instructor6</li>
<li>Password: password</li>
</ul> </li>
<li>Choose any assignment from the list of assignments which involves teams; since an assignment without teams does not include the review procedure. For example, “Final project (design and doc)” is a good assignment to choose.
<li>Now, while testing this functionality, it is recommended to keep the following pages opened in different tabs by clicking on the small icons in the same row as the assignment that you want to test: 

a. “Edit” -> “Due Dates” tab
[[File:edit_assignment.png]] 

b. “Add participants”
[[File:add_as_participant.png]] 

c. “View Submissions”
[[File:list_submissions.png]] 
</li>
<li>Now add the instructor as a participant (3.b). and edit the ALL due dates (3.a.) of different stages to any date later from now (hence the assignment goes into the “submission” stage by default). Save both these changes. </li>
<li>Now refresh the view submissions page (3.c.), you will see a list of assignments with no link to review or assign grade.</li>
<li>Now switch to edit due dates tab (3.a.) again. Set the due date for submission to a previous date which has passed, hence the next due date will be for the “review” stage by default (which will have a later due date from now.)</li>
<li>Refresh view submissions page (3.c.), you will see a list of assignments with a link to “Begin Review.”</li>
<li>You can click on it. If you “save” your response, you will see that the link has been replace by “View review” and “Edit review” now. Whereas, if you “submit” the response, the link will be replaced by “View review” and “Update review.”</li>
<li>Depending on the number of stages that an assignment has, you can edit the due date at every stage and check whether the links are displayed appropriately.</li>
<li>Finally when the last deadline of the assignment is set to a previous date which has passed, you will see the link to “Assign grade” in view submissions (3.c.), and the user will still be able to begin, edit or update a review in order to provide flexibility as per the requirements.</li>
</ol>

Edge Cases:
<ol>
<li>If the user is not a participant of the current assignment, they are shown a list of all submissions made by all teams of the current assignment. They are shown the link to “Assign Grade” only when the final deadline of the assignment passes.</li>
<li>During any review stage or even after the deadline, the user is offered the flexibility to Begin, Edit, View or Update their review.</li>
</ol>

===Part 3: Add instructor/TA as a participant while creating the assignment===
UI Testing
<ol>
<li>Login in as an instructor.
<ul>
<li>Username: instructor6</li>
<li>Password: password</li>
</ul> </li>
<li> Go to Manage-> Assignments </li>
<li> Click on add assignment </li>
<li> Enter the required details- Assignment name, course etc </li>
<li>To add instructor/TA as a participant, go to Review Tab and select the "Add yourself as a participant?" checkbox. </li>
<li> Click on Save </li>

The assignment has been created and the instructor/TA is added as a participant to the assignment.
</ol>

Edge Cases :
<ol>
<li> If the instructor/TA does not select the "Add yourself as a participant?" checkbox while creating the assignment, then he is not added as a participant</li>
<li> If the instructor/TA selects the "Add yourself as a participant?" checkbox while editing the assignment, then he is added as a participant
<li> If the instructor/TA unchecks the "Add yourself as a participant?" checkbox while editing the assignment, then he is removed as a participant </li> </ol>

 
Rspec Testing :
<pre>

describe '#create' do
before(:each) do
allow(AssignmentForm).to receive(:new).with(any_args).and_return(assignment_form)
@params = {
button: '',
assignment_form: {
assignment_questionnaire: [{"assignment_id" => "1", "questionnaire_id" => "666", "dropdown" => "true",
"questionnaire_weight" => "100", "notification_limit" => "15", "used_in_round" => "1"}],
due_date: [{"id" => "", "parent_id" => "", "round" => "1", "deadline_type_id" => "1", "due_at" => "2017/12/05 00:00", "submission_allowed_id" => "3", "review_allowed_id" => "1", "teammate_review_allowed_id" => "3", "review_of_review_allowed_id" => "1", "threshold" => "1"},
{"id" => "", "parent_id" => "", "round" => "1", "deadline_type_id" => "2", "due_at" => "2017/12/02 00:00", "submission_allowed_id" => "1", "review_allowed_id" => "3", "teammate_review_allowed_id" => "3", "review_of_review_allowed_id" => "1", "threshold" => "1"}],
assignment: {
instructor_id: 2,
course_id: 1,
max_team_size: 1,
id: 1,
name: 'test assignment',
directory_path: '/test',
spec_location: '',
private: false,
show_teammate_reviews: false,
require_quiz: false,
num_quiz_questions: 0,
staggered_deadline: false,
microtask: false,
reviews_visible_to_all: false,
is_calibrated: false,
availability_flag: true,
reputation_algorithm: 'Lauw',
simicheck: -1,
simicheck_threshold: 100
add_instructor:1 #added field
}
}
}

context 'when assignment_form is saved successfully' do
it 'redirects to assignment#edit page' do
allow(assignment_form).to receive(:assignment).and_return(assignment)
allow(assignment_form).to receive(:save).and_return(true)
allow(assignment_form).to receive(:update).with(any_args).and_return(true)
allow(assignment_form).to receive(:create_assignment_node).and_return(double('node'))
allow(assignment).to receive(:id).and_return(1)
allow(Assignment).to receive(:find_by).with(name: 'test assignment').and_return(assignment)
allow(assignment_form).to receive(:add_instructor_as_participant).with(any_args).and_return(true) #added check
allow_any_instance_of(AssignmentsController).to receive(:undo_link)
.with('Assignment "test assignment" has been created successfully. ').and_return(true)
post :create, @params
expect(response).to redirect_to('/assignments/1/edit')
end
end

describe '#update' do
context 'when params does not have key :assignment_form' do
context 'when assignment is saved successfully' do
it 'shows a note flash message and redirects to tree_display#index page' do
allow(assignment).to receive(:save).and_return(true)
params = {
id: 1,
course_id: 1
}
session = {user: instructor}
allow(assignment_form).to receive(:add_instructor_as_participant).with(any_args).and_return(true) #added check
post :update, params, session
expect(flash[:note]).to eq('The assignment was successfully saved.')
expect(response).to redirect_to('/tree_display/list')
end
end

context 'when "Add yourself as a participant?" checkbox is selected' do
it 'calls add_instructor_as_participant which in turn calls add Participant method of Participant controller' do
allow(assignment_form).to receive(:add_participant).with(any_args).and_return(true)
end
end

</pre>

== '''External Links''' ==
* Forked Repository: https://github.com/Matrawick/expertiza/tree/beta
* Pull Request: https://github.com/expertiza/expertiza/pull/1631

== '''Team Members''' ==
Mentor: Prof. Edward Gehringer (efg@ncsu.edu)
<ul>
<li>Anuja Kulkarni - apkulka2@ncsu.edu</li>
<li>Mark Trawick - mtrawic@ncsu.edu</li>
<li>Parvathy Menon - pharida@ncsu.edu</li>
<li>Samruddhi Khandale - sskhanda@ncsu.edu</li>
</ul>

CSC/ECE 517 Fall 2019 - E1985. Let course staff and students do reviews

2019-12-06T19:41:53Z

Mtrawic: /* Part 2: Allow students to identify that their submission has been reviewed by the instructor */

== '''Problem Statement''' ==

Peer review is a great way for students to learn about how well they have developed their application. However, sometimes, the peer reviews may not be thorough and the team/person's work reviewed might not reflect the actual status of the development.
By letting course staff perform reviews as well, the reviewer and the reviewee both benefit, improving the overall learning experience. Currently, there exists no method for the instructor/TA to perform reviews of assignments submitted by the students. Hence this is what this project aims at, allowing instructor/TA to perform reviews.

Thus, the primary aim of this project is to allow instructors to submit reviews of student work, using the same review form that students use to do reviews.

=='''Implementation Details''' ==
Our project aims at enabling the instructor/TA to review the submissions using the same form that a student uses while peer-reviewing. For this, our implementation has been divided into 3 parts, and can be described as follows:
<ol>
<li>The first part involves the primary functionality of letting the staff perform a review on students submission.</li>
<li>The second part involves allowing the students to identify from their end, which review has been performed by an instructor/TA. This will help students improve their project as the opinions presented by a staff member would be valid.</li>
<li>Thirdly, we implemented an additional feature that allows the instructor/TA to add himself as a participant while creating/editing an Assignment (This way seemed to be more convenient rather than going to the “Add Participant” page and adding themselves as a participant.)</li>
</ol>

===Part 1: Allowing instructor/TA to perform review===
Note 1: “Instructor/TA” will be referred to by “user” here onwards.

Note 2: To be able to perform a review, the user has to be a participant of that assignment. We have implemented “Add as Participant” as an additional feature which is explained in Part 3. If the user is not a participant, he will only see the link to “Assign grade” that too after the assignment deadline passes.
Now, for each assignment, there is a page called “List Submissions” where the user can view all the submissions. So we have modified this page so that for each team submission a link for “Begin review,” “Edit review,” “View review,” “Update review,” or “Assign grade” may appear depending on the [http://wiki.expertiza.ncsu.edu/index.php/Deadline_types stage] in which the assignment is.

The following describes the flow of our project considering a fresh assignment.
<ol>
<li>When the assignment is created, it is by default is in the “submission” stage. Hence the user will not see any link on the list submission page.</li>
<li>When the first deadline passes, the assignment goes into the “review” stage. Hence the link “Begin review” will be visible for each team submission, where the instructor/TA can click on to submit the review. Two cases can occur here:
<ul>
<li>If the user “submits” a review, they will be redirected to the list submission page where they’ll see the option “View review” or “Update review.” Being an instructor/TA, they are given the flexibility to update their review at any time during the “review” stage.</li>
<li>If the user “saves” a review, they will be redirected to the list submission page where they’ll see the option “View review” and “Edit Review (as they have not submitted it yet.)”</li>
</li>
</ul>
<li>After the first review is over, if more stages of “submissions” and “reviews” exist, it will follow the above 2 steps until the final assignment deadline does not pass. Note that at any round of the review, the user can “Begin review” for a submission which they had not reviewed in the previous rounds. This is to provide flexibility to them as per the requirement.</li>
<li>Finally, after the assignment deadline passes, the assignment goes into the “Finished” stage. This is when the “Assign grade” link will be visible to the user. Along with this, depending on whether they have reviewed a particular submission or not, the begin, view, update links will be visible to them as they should be able to perform the review even after the assignment deadline for the students has passed.</li>
</ol>

Files Edited 
The following are the files (controllers and views) edited for the implementation of the review process. The comments are given along with the code to explain what is being done. 
* '''Controller: ''' ''app/controllers/response_controller.rb'' 

<pre>
# Ensuring recent submission is not nil
team_most_recent_submission = AssignmentTeam.find(@map.reviewee_id).most_recent_submission
if @response.nil? || (!team_most_recent_submission.nil? && team_most_recent_submission.updated_at > @response.updated_at)
...
# Page should be directed to list_submissions if TA/instructor performs the review
when "ta_review"
redirect_to controller: 'assignments', action: 'list_submissions', id: @map.response_map.assignment.id
</pre>

* '''Controller: ''' ''app/helpers/assignment_helper.rb'' 

<pre>
topic_id = topic.try :id
...
# Needed topic ID for review implpementation, so added it to return list of the function
[topic_id, topic_identifier ||= "", topic_name ||= "", users_for_curr_team, participants]
</pre>

* '''View: ''' ''app/views/assignments/list_submissions.html.erb'' 

<pre>
# Retrieved topic ID for review implpementation
<% topic_id, topic_identifier, topic_name, users_for_curr_team, participants = get_data_for_list_submissions(team) %>
...

<% if @assignment.get_current_stage() == "Finished" %>
<%= link_to 'Assign Grade', { controller: 'grades', action: 'view_team', id: participants.first.id}, target: '_blank' %> 
<% end %>

<% reviewer = Participant.find_by(:parent_id => @assignment.id, :user_id => session[:user].id)%>


<% if !reviewer.nil? %>

<% @mapping = ReviewResponseMap.find_by(:reviewed_object_id => @assignment.id, :reviewer_id => reviewer.id, :reviewee_id => team.id) %>

<% if @mapping.nil? && (@assignment.get_current_stage() == "review" || @assignment.get_current_stage() == "Finished") %>
<% ReviewResponseMap.create(reviewee_id: team.id, reviewer_id: reviewer.id, reviewed_object_id: @assignment.id) %>
<% map = ReviewResponseMap.find_by(reviewee_id: team.id, reviewer_id: reviewer.id, reviewed_object_id: @assignment.id) %>
<%= link_to 'Begin Review', { controller: 'response', action: 'new', :id => map.id, :return => "ta_review" } %> 

<% elsif @assignment.get_current_stage() == "review" || @assignment.get_current_stage() == "Finished" %>
<% map1 = ReviewResponseMap.find_by(reviewee_id: team.id, reviewer_id: reviewer.id, reviewed_object_id: @assignment.id) %>
<% @sorted_responses = Array.new %>
<% @sorted_responses = Response.where(:map_id => map1.id) %>

<% if @sorted_responses.empty?%>
<%= link_to 'Begin Review', { controller: 'response', action: 'new', :id => map1.id, :return => "ta_review" } %> 
<% else %>

<% @sorted_responses = @sorted_responses.sort_by {|obj| obj.updated_at} # the latest should be at the last %>
<% @latest_response = @sorted_responses.last %>
<%if @latest_response.round.nil?%> 
<% last_response_round = AssignmentDueDate.done_in_assignment_round(@assignment.id, @latest_response) %>
<%else%> 
<% last_response_round = @latest_response.round %>
<%end%>
<% current_round = @assignment.number_of_current_round(topic_id) %>

<%= link_to "View Review", {:controller => 'response', :action => 'view', :id => @latest_response.id} %> 
<% if last_response_round == current_round %>
<% latest_submission = team.most_recent_submission %>
<% if !@latest_response.is_submitted %> 
<%= link_to "Edit Review", {:controller => 'response', :action => 'edit', :id => @latest_response.id, :return => "ta_review" } %>
<% else %> 
<%= link_to "Update Review", {:controller => 'response', :action => 'new', :id => map1.id, :return => "ta_review" } %>
<%end%>
<% else %> 
<%= link_to "Update Review", {:controller => 'response', :action => 'new', :id => map1.id, :return => "ta_review" } %>
<% end %>
<% end %>
<% end %>
<% end %>
</pre>

* '''View: ''' ''app/views/grades/view_team.html.erb'' 

<pre>

<% topic_id_num, topic_id, topic_name, users_for_curr_team, participants = get_data_for_list_submissions(@team) %>
</pre>

* '''View: ''' ''app/views/student_review/_responses.html.erb'' 

<pre>
# Ensuring recent submission is not nil
<% elsif !latest_submission.nil? && @latest_response.updated_at < latest_submission.updated_at %>
</pre>

---

=== Part 2: Allow students to identify that their submission has been reviewed by the instructor===
 Currently Scenario: There is no way for the student to know which review was by the instructor. 
Requirement: This helps the student to identify the review made by the instructor from the other reviews. 
Implementation: A simple check is made to determine if the reviewer is a TA or Instructor. app/assets/images/staff.png is displayed to make an instructor performed review stand out from other reviews. 
[[File:Icon-demo.png|center]]

=== Part 3: Add instructor/TA as a participant while creating the assignment (Extra Feature) ===
 
'''Current scenario''' 
The instructor/TA will be able to perform student reviews (after implementation of Part 1) for the assignment, only when he is a participant of the same assignment. Thus, every time he creates an assignment and wants to add himself as a participant in order to perform student reviews, he is supposed to do the following 
Go to list of assignments -> Find the required assignment -> Click on add participant -> Type user Id -> Click on add
The same flow is as shown below where the instructor/TA has created a new assignment say “Program 1 Github” wherein he wants himself as a participant to be able to perform student reviews :
 
[[File:Addinstructor1.png|center]] 
[[File:AddInstructor2.png]] 
[[File:Add Instructor 3.png]] 

To overcome the hassle, we added a check box through which the instructor/TA will get an option to add himself as a participant while creating the assignment. Also, he will be able to add/remove himself as a participant while editing the assignment.

'''Implementation''' 

The checkbox has been provided to Create New Assignment’s page under the Review strategy tab.
It is added as follows: 

[[File:Add instructor 4.png]] 

If the instructor/TA selects the box while creating the assignment, then he will be added as a participant for the same assignment. 

Files edited:

* '''View: ''' ''app/views/assignments/edit/_review_strategy.html.erb'' 
In this file, code for displaying the checkbox has been added which is as follows :

<pre>
<tr>
<td>
<input name='add_instructor' type="hidden" value="false"/>
<%= check_box_tag('add_instructor', value = "1", checked = is_instructor_a_participant?)%>
<%= label_tag('add_instructor', 'Add yourself as a participant?') %>
<img src="/assets/info.png" title='If you add yourself as a participant then you will be able to perform reviews of students for this assignment'>
</td>
</tr>
</pre>

Thus the checkbox "add_instructor" if selected adds the instructor as a participant to the currently being created assignment. Thus, instructor being a participant allows him to perform reviews for the students for this assignment.

* '''Controller: ''' ''app/controllers/assignments_controller.rb'' 

The file contains extra code which adds the instructor/TA as a participant if the checkbox has been selected while creating the assignment. Also, while editing the assignment, he will be able to add/remove himself as a participant for this assignment. 

The following methods are added to the controller :

<pre>
def is_instructor_a_participant?
instructor_id = session[:user].id
assignment_id = @assignment_form.assignment.id.to_s

if assignment_id != nil
@is_instructor_a_participant = Participant.where(user_id: instructor_id , parent_id: assignment_id)
if @is_instructor_a_participant.present?
return true
end
end
return false
end
</pre>

The method "is_instructor_a_participant?" returns true if the instructor is already a participant for the assignment, else returns false. This method is required to verify that the instructor/TA is not a participant of the assignment, and thus he can be added as a participant if the checkbox has been selected. Also, during editing the assignment, we need to show checkbox being selected if the instructor is a participant and vice versa. Thus, this method serves that purpose as well.

 
<pre>
def add_instructor_as_participant(assignment_id)
if params[:add_instructor] == 'false' and is_instructor_a_participant? == true
delete_instructor_as_participant(assignment_id,session[:user].id)
elsif params[:add_instructor] == '1' and is_instructor_a_participant? == false
current_assignment = Object.const_get("Assignment").find(assignment_id)
current_assignment.add_participant(session[:user].name, true, true, true)
end
end
</pre>

The method "add_instructor_as_participant" is been called from create and update methods in the Assignment controller. It checks if the "Add yourself as participant" checkbox is selected for the currently created/edited assignment. If it is selected during creation or editing, then the instructor is added as a participant to the currently created/edited assignment. The participant is added by using the already existing method "add_participant" in the model assignment.rb Also, if the option is unchecked during editing, then the instructor is removed as a participant.

 
<pre>
def delete_instructor_as_participant(assignment_id , instructor_id)
Participant.where(user_id: instructor_id , parent_id: assignment_id).destroy_all
end
</pre>
The method "delete_instructor_as_participant" removes instructor as a participant for the assignment when the checkbox is unchecked during the editing of the assignment.

== '''Test Plan''' ==

==== Part 1: Allowing instructor/TA to perform review ====
UI Testing

<ol>
<li>Login in as an instructor.
<ul>
<li>Username: instructor6</li>
<li>Password: password</li>
</ul> </li>
<li>Choose any assignment from the list of assignments which involves teams; since an assignment without teams does not include the review procedure. For example, “Final project (design and doc)” is a good assignment to choose.
<li>Now, while testing this functionality, it is recommended to keep the following pages opened in different tabs by clicking on the small icons in the same row as the assignment that you want to test: 

a. “Edit” -> “Due Dates” tab
[[File:edit_assignment.png]] 

b. “Add participants”
[[File:add_as_participant.png]] 

c. “View Submissions”
[[File:list_submissions.png]] 
</li>
<li>Now add the instructor as a participant (3.b). and edit the ALL due dates (3.a.) of different stages to any date later from now (hence the assignment goes into the “submission” stage by default). Save both these changes. </li>
<li>Now refresh the view submissions page (3.c.), you will see a list of assignments with no link to review or assign grade.</li>
<li>Now switch to edit due dates tab (3.a.) again. Set the due date for submission to a previous date which has passed, hence the next due date will be for the “review” stage by default (which will have a later due date from now.)</li>
<li>Refresh view submissions page (3.c.), you will see a list of assignments with a link to “Begin Review.”</li>
<li>You can click on it. If you “save” your response, you will see that the link has been replace by “View review” and “Edit review” now. Whereas, if you “submit” the response, the link will be replaced by “View review” and “Update review.”</li>
<li>Depending on the number of stages that an assignment has, you can edit the due date at every stage and check whether the links are displayed appropriately.</li>
<li>Finally when the last deadline of the assignment is set to a previous date which has passed, you will see the link to “Assign grade” in view submissions (3.c.), and the user will still be able to begin, edit or update a review in order to provide flexibility as per the requirements.</li>
</ol>

Edge Cases:
<ol>
<li>If the user is not a participant of the current assignment, they are shown a list of all submissions made by all teams of the current assignment. They are shown the link to “Assign Grade” only when the final deadline of the assignment passes.</li>
<li>During any review stage or even after the deadline, the user is offered the flexibility to Begin, Edit, View or Update their review.</li>
</ol>

===Part 3: Add instructor/TA as a participant while creating the assignment===
UI Testing
<ol>
<li>Login in as an instructor.
<ul>
<li>Username: instructor6</li>
<li>Password: password</li>
</ul> </li>
<li> Go to Manage-> Assignments </li>
<li> Click on add assignment </li>
<li> Enter the required details- Assignment name, course etc </li>
<li>To add instructor/TA as a participant, go to Review Tab and select the "Add yourself as a participant?" checkbox. </li>
<li> Click on Save </li>

The assignment has been created and the instructor/TA is added as a participant to the assignment.
</ol>

Edge Cases :
<ol>
<li> If the instructor/TA does not select the "Add yourself as a participant?" checkbox while creating the assignment, then he is not added as a participant</li>
<li> If the instructor/TA selects the "Add yourself as a participant?" checkbox while editing the assignment, then he is added as a participant
<li> If the instructor/TA unchecks the "Add yourself as a participant?" checkbox while editing the assignment, then he is removed as a participant </li> </ol>

 
Rspec Testing :
<pre>

describe '#create' do
before(:each) do
allow(AssignmentForm).to receive(:new).with(any_args).and_return(assignment_form)
@params = {
button: '',
assignment_form: {
assignment_questionnaire: [{"assignment_id" => "1", "questionnaire_id" => "666", "dropdown" => "true",
"questionnaire_weight" => "100", "notification_limit" => "15", "used_in_round" => "1"}],
due_date: [{"id" => "", "parent_id" => "", "round" => "1", "deadline_type_id" => "1", "due_at" => "2017/12/05 00:00", "submission_allowed_id" => "3", "review_allowed_id" => "1", "teammate_review_allowed_id" => "3", "review_of_review_allowed_id" => "1", "threshold" => "1"},
{"id" => "", "parent_id" => "", "round" => "1", "deadline_type_id" => "2", "due_at" => "2017/12/02 00:00", "submission_allowed_id" => "1", "review_allowed_id" => "3", "teammate_review_allowed_id" => "3", "review_of_review_allowed_id" => "1", "threshold" => "1"}],
assignment: {
instructor_id: 2,
course_id: 1,
max_team_size: 1,
id: 1,
name: 'test assignment',
directory_path: '/test',
spec_location: '',
private: false,
show_teammate_reviews: false,
require_quiz: false,
num_quiz_questions: 0,
staggered_deadline: false,
microtask: false,
reviews_visible_to_all: false,
is_calibrated: false,
availability_flag: true,
reputation_algorithm: 'Lauw',
simicheck: -1,
simicheck_threshold: 100
add_instructor:1 #added field
}
}
}

context 'when assignment_form is saved successfully' do
it 'redirects to assignment#edit page' do
allow(assignment_form).to receive(:assignment).and_return(assignment)
allow(assignment_form).to receive(:save).and_return(true)
allow(assignment_form).to receive(:update).with(any_args).and_return(true)
allow(assignment_form).to receive(:create_assignment_node).and_return(double('node'))
allow(assignment).to receive(:id).and_return(1)
allow(Assignment).to receive(:find_by).with(name: 'test assignment').and_return(assignment)
allow(assignment_form).to receive(:add_instructor_as_participant).with(any_args).and_return(true) #added check
allow_any_instance_of(AssignmentsController).to receive(:undo_link)
.with('Assignment "test assignment" has been created successfully. ').and_return(true)
post :create, @params
expect(response).to redirect_to('/assignments/1/edit')
end
end

describe '#update' do
context 'when params does not have key :assignment_form' do
context 'when assignment is saved successfully' do
it 'shows a note flash message and redirects to tree_display#index page' do
allow(assignment).to receive(:save).and_return(true)
params = {
id: 1,
course_id: 1
}
session = {user: instructor}
allow(assignment_form).to receive(:add_instructor_as_participant).with(any_args).and_return(true) #added check
post :update, params, session
expect(flash[:note]).to eq('The assignment was successfully saved.')
expect(response).to redirect_to('/tree_display/list')
end
end

context 'when "Add yourself as a participant?" checkbox is selected' do
it 'calls add_instructor_as_participant which in turn calls add Participant method of Participant controller' do
allow(assignment_form).to receive(:add_participant).with(any_args).and_return(true)
end
end

</pre>

== '''External Links''' ==
* Forked Repository: https://github.com/Matrawick/expertiza/tree/beta
* Pull Request: https://github.com/expertiza/expertiza/pull/1631

== '''Team Members''' ==
Mentor: Prof. Edward Gehringer (efg@ncsu.edu)
<ul>
<li>Anuja Kulkarni - apkulka2@ncsu.edu</li>
<li>Mark Trawick - mtrawic@ncsu.edu</li>
<li>Parvathy Menon - pharida@ncsu.edu</li>
<li>Samruddhi Khandale - sskhanda@ncsu.edu</li>
</ul>

File:Icon-demo.png

2019-12-06T19:37:29Z

Mtrawic: uploaded a new version of "File:Icon-demo.png"

Review done by course staff has staff icon.

File:Icon-demo.png

2019-12-06T19:34:07Z

Mtrawic: Review done by course staff has staff icon.

Review done by course staff has staff icon.

CSC/ECE 517 Fall 2019 - E1985. Let course staff and students do reviews

2019-12-06T19:25:44Z

Mtrawic: /* Part 2: Allow students to identify that their submission has been reviewed by the instructor */

== '''Problem Statement''' ==

Peer review is a great way for students to learn about how well they have developed their application. However, sometimes, the peer reviews may not be thorough and the team/person's work reviewed might not reflect the actual status of the development.
By letting course staff perform reviews as well, the reviewer and the reviewee both benefit, improving the overall learning experience. Currently, there exists no method for the instructor/TA to perform reviews of assignments submitted by the students. Hence this is what this project aims at, allowing instructor/TA to perform reviews.

Thus, the primary aim of this project is to allow instructors to submit reviews of student work, using the same review form that students use to do reviews.

=='''Implementation Details''' ==
Our project aims at enabling the instructor/TA to review the submissions using the same form that a student uses while peer-reviewing. For this, our implementation has been divided into 3 parts, and can be described as follows:
<ol>
<li>The first part involves the primary functionality of letting the staff perform a review on students submission.</li>
<li>The second part involves allowing the students to identify from their end, which review has been performed by an instructor/TA. This will help students improve their project as the opinions presented by a staff member would be valid.</li>
<li>Thirdly, we implemented an additional feature that allows the instructor/TA to add himself as a participant while creating/editing an Assignment (This way seemed to be more convenient rather than going to the “Add Participant” page and adding themselves as a participant.)</li>
</ol>

===Part 1: Allowing instructor/TA to perform review===
Note 1: “Instructor/TA” will be referred to by “user” here onwards.

Note 2: To be able to perform a review, the user has to be a participant of that assignment. We have implemented “Add as Participant” as an additional feature which is explained in Part 3. If the user is not a participant, he will only see the link to “Assign grade” that too after the assignment deadline passes.
Now, for each assignment, there is a page called “List Submissions” where the user can view all the submissions. So we have modified this page so that for each team submission a link for “Begin review,” “Edit review,” “View review,” “Update review,” or “Assign grade” may appear depending on the [http://wiki.expertiza.ncsu.edu/index.php/Deadline_types stage] in which the assignment is.

The following describes the flow of our project considering a fresh assignment.
<ol>
<li>When the assignment is created, it is by default is in the “submission” stage. Hence the user will not see any link on the list submission page.</li>
<li>When the first deadline passes, the assignment goes into the “review” stage. Hence the link “Begin review” will be visible for each team submission, where the instructor/TA can click on to submit the review. Two cases can occur here:
<ul>
<li>If the user “submits” a review, they will be redirected to the list submission page where they’ll see the option “View review” or “Update review.” Being an instructor/TA, they are given the flexibility to update their review at any time during the “review” stage.</li>
<li>If the user “saves” a review, they will be redirected to the list submission page where they’ll see the option “View review” and “Edit Review (as they have not submitted it yet.)”</li>
</li>
</ul>
<li>After the first review is over, if more stages of “submissions” and “reviews” exist, it will follow the above 2 steps until the final assignment deadline does not pass. Note that at any round of the review, the user can “Begin review” for a submission which they had not reviewed in the previous rounds. This is to provide flexibility to them as per the requirement.</li>
<li>Finally, after the assignment deadline passes, the assignment goes into the “Finished” stage. This is when the “Assign grade” link will be visible to the user. Along with this, depending on whether they have reviewed a particular submission or not, the begin, view, update links will be visible to them as they should be able to perform the review even after the assignment deadline for the students has passed.</li>
</ol>

Files Edited 
The following are the files (controllers and views) edited for the implementation of the review process. The comments are given along with the code to explain what is being done. 
* '''Controller: ''' ''app/controllers/response_controller.rb'' 

<pre>
# Ensuring recent submission is not nil
team_most_recent_submission = AssignmentTeam.find(@map.reviewee_id).most_recent_submission
if @response.nil? || (!team_most_recent_submission.nil? && team_most_recent_submission.updated_at > @response.updated_at)
...
# Page should be directed to list_submissions if TA/instructor performs the review
when "ta_review"
redirect_to controller: 'assignments', action: 'list_submissions', id: @map.response_map.assignment.id
</pre>

* '''Controller: ''' ''app/helpers/assignment_helper.rb'' 

<pre>
topic_id = topic.try :id
...
# Needed topic ID for review implpementation, so added it to return list of the function
[topic_id, topic_identifier ||= "", topic_name ||= "", users_for_curr_team, participants]
</pre>

* '''View: ''' ''app/views/assignments/list_submissions.html.erb'' 

<pre>
# Retrieved topic ID for review implpementation
<% topic_id, topic_identifier, topic_name, users_for_curr_team, participants = get_data_for_list_submissions(team) %>
...

<% if @assignment.get_current_stage() == "Finished" %>
<%= link_to 'Assign Grade', { controller: 'grades', action: 'view_team', id: participants.first.id}, target: '_blank' %> 
<% end %>

<% reviewer = Participant.find_by(:parent_id => @assignment.id, :user_id => session[:user].id)%>


<% if !reviewer.nil? %>

<% @mapping = ReviewResponseMap.find_by(:reviewed_object_id => @assignment.id, :reviewer_id => reviewer.id, :reviewee_id => team.id) %>

<% if @mapping.nil? && (@assignment.get_current_stage() == "review" || @assignment.get_current_stage() == "Finished") %>
<% ReviewResponseMap.create(reviewee_id: team.id, reviewer_id: reviewer.id, reviewed_object_id: @assignment.id) %>
<% map = ReviewResponseMap.find_by(reviewee_id: team.id, reviewer_id: reviewer.id, reviewed_object_id: @assignment.id) %>
<%= link_to 'Begin Review', { controller: 'response', action: 'new', :id => map.id, :return => "ta_review" } %> 

<% elsif @assignment.get_current_stage() == "review" || @assignment.get_current_stage() == "Finished" %>
<% map1 = ReviewResponseMap.find_by(reviewee_id: team.id, reviewer_id: reviewer.id, reviewed_object_id: @assignment.id) %>
<% @sorted_responses = Array.new %>
<% @sorted_responses = Response.where(:map_id => map1.id) %>

<% if @sorted_responses.empty?%>
<%= link_to 'Begin Review', { controller: 'response', action: 'new', :id => map1.id, :return => "ta_review" } %> 
<% else %>

<% @sorted_responses = @sorted_responses.sort_by {|obj| obj.updated_at} # the latest should be at the last %>
<% @latest_response = @sorted_responses.last %>
<%if @latest_response.round.nil?%> 
<% last_response_round = AssignmentDueDate.done_in_assignment_round(@assignment.id, @latest_response) %>
<%else%> 
<% last_response_round = @latest_response.round %>
<%end%>
<% current_round = @assignment.number_of_current_round(topic_id) %>

<%= link_to "View Review", {:controller => 'response', :action => 'view', :id => @latest_response.id} %> 
<% if last_response_round == current_round %>
<% latest_submission = team.most_recent_submission %>
<% if !@latest_response.is_submitted %> 
<%= link_to "Edit Review", {:controller => 'response', :action => 'edit', :id => @latest_response.id, :return => "ta_review" } %>
<% else %> 
<%= link_to "Update Review", {:controller => 'response', :action => 'new', :id => map1.id, :return => "ta_review" } %>
<%end%>
<% else %> 
<%= link_to "Update Review", {:controller => 'response', :action => 'new', :id => map1.id, :return => "ta_review" } %>
<% end %>
<% end %>
<% end %>
<% end %>
</pre>

* '''View: ''' ''app/views/grades/view_team.html.erb'' 

<pre>

<% topic_id_num, topic_id, topic_name, users_for_curr_team, participants = get_data_for_list_submissions(@team) %>
</pre>

* '''View: ''' ''app/views/student_review/_responses.html.erb'' 

<pre>
# Ensuring recent submission is not nil
<% elsif !latest_submission.nil? && @latest_response.updated_at < latest_submission.updated_at %>
</pre>

---

=== Part 2: Allow students to identify that their submission has been reviewed by the instructor===
 Currently Scenario: There is no way for the student to know which review was by the instructor. 
Requirement: This helps the student to identify the review made by the instructor from the other reviews. 
Implementation: A simple check is made to determine if the reviewer is a TA or Instructor. app/assets/images/staff.png is displayed to make an instructor performed review stand out from other reviews. 

=== Part 3: Add instructor/TA as a participant while creating the assignment (Extra Feature) ===
 
'''Current scenario''' 
The instructor/TA will be able to perform student reviews (after implementation of Part 1) for the assignment, only when he is a participant of the same assignment. Thus, every time he creates an assignment and wants to add himself as a participant in order to perform student reviews, he is supposed to do the following 
Go to list of assignments -> Find the required assignment -> Click on add participant -> Type user Id -> Click on add
The same flow is as shown below where the instructor/TA has created a new assignment say “Program 1 Github” wherein he wants himself as a participant to be able to perform student reviews :
 
[[File:Addinstructor1.png|center]] 
[[File:AddInstructor2.png]] 
[[File:Add Instructor 3.png]] 

To overcome the hassle, we added a check box through which the instructor/TA will get an option to add himself as a participant while creating the assignment. Also, he will be able to add/remove himself as a participant while editing the assignment.

'''Implementation''' 

The checkbox has been provided to Create New Assignment’s page under the Review strategy tab.
It is added as follows: 

[[File:Add instructor 4.png]] 

If the instructor/TA selects the box while creating the assignment, then he will be added as a participant for the same assignment. 

Files edited:

* '''View: ''' ''app/views/assignments/edit/_review_strategy.html.erb'' 
In this file, code for displaying the checkbox has been added which is as follows :

<pre>
<tr>
<td>
<input name='add_instructor' type="hidden" value="false"/>
<%= check_box_tag('add_instructor', value = "1", checked = is_instructor_a_participant?)%>
<%= label_tag('add_instructor', 'Add yourself as a participant?') %>
<img src="/assets/info.png" title='If you add yourself as a participant then you will be able to perform reviews of students for this assignment'>
</td>
</tr>
</pre>

Thus the checkbox "add_instructor" if selected adds the instructor as a participant to the currently being created assignment. Thus, instructor being a participant allows him to perform reviews for the students for this assignment.

* '''Controller: ''' ''app/controllers/assignments_controller.rb'' 

The file contains extra code which adds the instructor/TA as a participant if the checkbox has been selected while creating the assignment. Also, while editing the assignment, he will be able to add/remove himself as a participant for this assignment. 

The following methods are added to the controller :

<pre>
def is_instructor_a_participant?
instructor_id = session[:user].id
assignment_id = @assignment_form.assignment.id.to_s

if assignment_id != nil
@is_instructor_a_participant = Participant.where(user_id: instructor_id , parent_id: assignment_id)
if @is_instructor_a_participant.present?
return true
end
end
return false
end
</pre>

The method "is_instructor_a_participant?" returns true if the instructor is already a participant for the assignment, else returns false. This method is required to verify that the instructor/TA is not a participant of the assignment, and thus he can be added as a participant if the checkbox has been selected. Also, during editing the assignment, we need to show checkbox being selected if the instructor is a participant and vice versa. Thus, this method serves that purpose as well.

 
<pre>
def add_instructor_as_participant(assignment_id)
if params[:add_instructor] == 'false' and is_instructor_a_participant? == true
delete_instructor_as_participant(assignment_id,session[:user].id)
elsif params[:add_instructor] == '1' and is_instructor_a_participant? == false
current_assignment = Object.const_get("Assignment").find(assignment_id)
current_assignment.add_participant(session[:user].name, true, true, true)
end
end
</pre>

The method "add_instructor_as_participant" is been called from create and update methods in the Assignment controller. It checks if the "Add yourself as participant" checkbox is selected for the currently created/edited assignment. If it is selected during creation or editing, then the instructor is added as a participant to the currently created/edited assignment. The participant is added by using the already existing method "add_participant" in the model assignment.rb Also, if the option is unchecked during editing, then the instructor is removed as a participant.

 
<pre>
def delete_instructor_as_participant(assignment_id , instructor_id)
Participant.where(user_id: instructor_id , parent_id: assignment_id).destroy_all
end
</pre>
The method "delete_instructor_as_participant" removes instructor as a participant for the assignment when the checkbox is unchecked during the editing of the assignment.

== '''Test Plan''' ==

==== Part 1: Allowing instructor/TA to perform review ====
UI Testing

<ol>
<li>Login in as an instructor.
<ul>
<li>Username: instructor6</li>
<li>Password: password</li>
</ul> </li>
<li>Choose any assignment from the list of assignments which involves teams; since an assignment without teams does not include the review procedure. For example, “Final project (design and doc)” is a good assignment to choose.
<li>Now, while testing this functionality, it is recommended to keep the following pages opened in different tabs by clicking on the small icons in the same row as the assignment that you want to test: 

a. “Edit” -> “Due Dates” tab
[[File:edit_assignment.png]] 

b. “Add participants”
[[File:add_as_participant.png]] 

c. “View Submissions”
[[File:list_submissions.png]] 
</li>
<li>Now add the instructor as a participant (3.b). and edit the ALL due dates (3.a.) of different stages to any date later from now (hence the assignment goes into the “submission” stage by default). Save both these changes. </li>
<li>Now refresh the view submissions page (3.c.), you will see a list of assignments with no link to review or assign grade.</li>
<li>Now switch to edit due dates tab (3.a.) again. Set the due date for submission to a previous date which has passed, hence the next due date will be for the “review” stage by default (which will have a later due date from now.)</li>
<li>Refresh view submissions page (3.c.), you will see a list of assignments with a link to “Begin Review.”</li>
<li>You can click on it. If you “save” your response, you will see that the link has been replace by “View review” and “Edit review” now. Whereas, if you “submit” the response, the link will be replaced by “View review” and “Update review.”</li>
<li>Depending on the number of stages that an assignment has, you can edit the due date at every stage and check whether the links are displayed appropriately.</li>
<li>Finally when the last deadline of the assignment is set to a previous date which has passed, you will see the link to “Assign grade” in view submissions (3.c.), and the user will still be able to begin, edit or update a review in order to provide flexibility as per the requirements.</li>
</ol>

Edge Cases:
<ol>
<li>If the user is not a participant of the current assignment, they are shown a list of all submissions made by all teams of the current assignment. They are shown the link to “Assign Grade” only when the final deadline of the assignment passes.</li>
<li>During any review stage or even after the deadline, the user is offered the flexibility to Begin, Edit, View or Update their review.</li>
</ol>

===Part 3: Add instructor/TA as a participant while creating the assignment===
UI Testing
<ol>
<li>Login in as an instructor.
<ul>
<li>Username: instructor6</li>
<li>Password: password</li>
</ul> </li>
<li> Go to Manage-> Assignments </li>
<li> Click on add assignment </li>
<li> Enter the required details- Assignment name, course etc </li>
<li>To add instructor/TA as a participant, go to Review Tab and select the "Add yourself as a participant?" checkbox. </li>
<li> Click on Save </li>

The assignment has been created and the instructor/TA is added as a participant to the assignment.
</ol>

Edge Cases :
<ol>
<li> If the instructor/TA does not select the "Add yourself as a participant?" checkbox while creating the assignment, then he is not added as a participant</li>
<li> If the instructor/TA selects the "Add yourself as a participant?" checkbox while editing the assignment, then he is added as a participant
<li> If the instructor/TA unchecks the "Add yourself as a participant?" checkbox while editing the assignment, then he is removed as a participant </li> </ol>

 
Rspec Testing :
<pre>

describe '#create' do
before(:each) do
allow(AssignmentForm).to receive(:new).with(any_args).and_return(assignment_form)
@params = {
button: '',
assignment_form: {
assignment_questionnaire: [{"assignment_id" => "1", "questionnaire_id" => "666", "dropdown" => "true",
"questionnaire_weight" => "100", "notification_limit" => "15", "used_in_round" => "1"}],
due_date: [{"id" => "", "parent_id" => "", "round" => "1", "deadline_type_id" => "1", "due_at" => "2017/12/05 00:00", "submission_allowed_id" => "3", "review_allowed_id" => "1", "teammate_review_allowed_id" => "3", "review_of_review_allowed_id" => "1", "threshold" => "1"},
{"id" => "", "parent_id" => "", "round" => "1", "deadline_type_id" => "2", "due_at" => "2017/12/02 00:00", "submission_allowed_id" => "1", "review_allowed_id" => "3", "teammate_review_allowed_id" => "3", "review_of_review_allowed_id" => "1", "threshold" => "1"}],
assignment: {
instructor_id: 2,
course_id: 1,
max_team_size: 1,
id: 1,
name: 'test assignment',
directory_path: '/test',
spec_location: '',
private: false,
show_teammate_reviews: false,
require_quiz: false,
num_quiz_questions: 0,
staggered_deadline: false,
microtask: false,
reviews_visible_to_all: false,
is_calibrated: false,
availability_flag: true,
reputation_algorithm: 'Lauw',
simicheck: -1,
simicheck_threshold: 100
add_instructor:1 #added field
}
}
}

context 'when assignment_form is saved successfully' do
it 'redirects to assignment#edit page' do
allow(assignment_form).to receive(:assignment).and_return(assignment)
allow(assignment_form).to receive(:save).and_return(true)
allow(assignment_form).to receive(:update).with(any_args).and_return(true)
allow(assignment_form).to receive(:create_assignment_node).and_return(double('node'))
allow(assignment).to receive(:id).and_return(1)
allow(Assignment).to receive(:find_by).with(name: 'test assignment').and_return(assignment)
allow(assignment_form).to receive(:add_instructor_as_participant).with(any_args).and_return(true) #added check
allow_any_instance_of(AssignmentsController).to receive(:undo_link)
.with('Assignment "test assignment" has been created successfully. ').and_return(true)
post :create, @params
expect(response).to redirect_to('/assignments/1/edit')
end
end

describe '#update' do
context 'when params does not have key :assignment_form' do
context 'when assignment is saved successfully' do
it 'shows a note flash message and redirects to tree_display#index page' do
allow(assignment).to receive(:save).and_return(true)
params = {
id: 1,
course_id: 1
}
session = {user: instructor}
allow(assignment_form).to receive(:add_instructor_as_participant).with(any_args).and_return(true) #added check
post :update, params, session
expect(flash[:note]).to eq('The assignment was successfully saved.')
expect(response).to redirect_to('/tree_display/list')
end
end

context 'when "Add yourself as a participant?" checkbox is selected' do
it 'calls add_instructor_as_participant which in turn calls add Participant method of Participant controller' do
allow(assignment_form).to receive(:add_participant).with(any_args).and_return(true)
end
end

</pre>

== '''External Links''' ==
* Forked Repository: https://github.com/Matrawick/expertiza/tree/beta
* Pull Request: https://github.com/expertiza/expertiza/pull/1631

== '''Team Members''' ==
Mentor: Prof. Edward Gehringer (efg@ncsu.edu)
<ul>
<li>Anuja Kulkarni - apkulka2@ncsu.edu</li>
<li>Mark Trawick - mtrawic@ncsu.edu</li>
<li>Parvathy Menon - pharida@ncsu.edu</li>
<li>Samruddhi Khandale - sskhanda@ncsu.edu</li>
</ul>

CSC/ECE 517 Fall 2019 - E1985. Let course staff and students do reviews

2019-11-11T01:43:08Z

Mtrawic: /* Issue: Allow TA/Instructor to add himself as a participant */

== '''Problem Statement''' ==

Peer review is a great way for students to learn about how well they have developed their application. However there are some problems with this:
* Sometimes, the peer reviews may not be thorough and the team/person's work reviewed might not reflect the actual status of the development.
* The reviewer might not know how well they are reviewing the peers work. They might not entirely know what tone to use or what suggestion to make.

By letting course staff perform reviews as well, the reviewer and the reviewee both benefit, improving the overall learning experience.

=== Current scenario ===

===Issue : Method to check if user is instructor / TA for selected course ===
Currently, there is no method which checks if the user is an instructor / TA for the selected course (namely a course staff member) Need : This method is required for redirection of pages which is dependent on the user role. Also, it is required to identify if the review is performed by the course staff. 
Proposed implementation : A new helper file named course_staff_helper.rb will be created in expertiza/helpers/ which will consist a method which checks if the user is a course staff member. 

=== Issue: Allow TA/Instructor to add himself as a participant ===
Currently instructors and TA's can add themselves as participants. The teaching staff reviewing a previous project thought this might be a bad idea. Here's what they said about it in our project description. "It adds instructor to the assgt. as a participant. This is probably not a good idea, because the instructor can review everyone, independent of what reviewer-assignment mechanism was in use."

I need to find out how doing a review works. I'm guessing by adding the instructor as a participant makes it easy to do a review. If we don't tell let the instructor add themselves as a participant then we may need to add code in the part for setting up reviews.

We will need a link in the list_submissions_.html.erb for "perform review".

=== Issue: Allow TA/Instructor to perform review of student assignments ===
Current Scenario: 
Currently, instructors and TA's cannot perform reviews. Previous attempts were made but they either complicated the code or did not pass the Travis CI. For this particular task, the following points describe how the current scenario looks and what needs to be improved/changed about it. The page below is the “View Submissions” page when logged in as an instructor. 
[[File:list.png]] 
The 2nd column consists of the title name with the link to “Assign Grade.” This is the part which we will have to modify. The link under the project title should be: 
1. “Perform Review,” until the deadline for the assignment hasn’t been reached. If he/she has already started the review, the link will be modified to “Edit Review,” until they don’t submit it. 
2. “Assign Grade,” after the deadline of the assignment passes. 
When clicked on “Perform Review”/”Edit Review,” it should direct to the review page seen by the students, and be able to perform the review. Thus it will direct the page to the response#new page or response#edit page respectively. 

Need: 
This functionality is needed so that the instructors can easily perform submission reviews which can also be viewed by the students. Also for future purposes, other student reviews can be benchmarked against an instructor review. 

Proposed Implementation: 
For the above-mentioned flow, the files to be edited will be as follows: 
1. app/views/assignments/list_submissions.html.erb: 
This view file refers to the page in the above screenshot. This file will be edited to check whether the deadline for the current assignment has been passed, and accordingly it will display either of the 2 options denoted above.

CSC/ECE 517 Fall 2019 - E1985. Let course staff and students do reviews

2019-11-11T01:27:39Z

Mtrawic: /* Allow TA/Instructor to add himself as a participant */

CSC/ECE 517 Fall 2019 - E1985. Let course staff and students do reviews

2019-11-11T01:27:19Z

Mtrawic: /* Issue : Method to check if user is instructor / TA for selected course */

CSC/ECE 517 Fall 2019 - E1985. Let course staff and students do reviews

2019-11-11T01:26:37Z

Mtrawic: /* Allow TA/Instructor to add himself as a participant */

CSC/ECE 517 Fall 2019 - E1985. Let course staff and students do reviews

2019-11-11T01:25:04Z

Mtrawic: /* Current scenario */

CSC/ECE 517 Fall 2019 - E1985. Let course staff and students do reviews

2019-11-11T01:03:32Z

Mtrawic: /* Allow TA/Instructor to add himself as a participant */

CSC/ECE 517 Fall 2019 - E1985. Let course staff and students do reviews

2019-11-11T00:56:09Z

Mtrawic: /* Allow TA/Instructor to add himself as a participant */

CSC/ECE 517 Fall 2019 - E1985. Let course staff and students do reviews

2019-11-11T00:48:21Z

Mtrawic: /* Current scenario */

CSC/ECE 517 Fall 2019 - E1985. Let course staff and students do reviews

2019-11-11T00:38:22Z

Mtrawic: /* Problem Statement */

CSC/ECE 517 Fall 2019 - E1985. Let course staff and students do reviews

2019-11-11T00:36:19Z

Mtrawic:

CSC/ECE 517 Fall 2019

2019-11-11T00:24:46Z

Mtrawic:

* [[CSC/ECE 517 Fall 2019 - Project E1947. Refactor quiz_questionnaire_controller.rb]]
* [[CSC/ECE 517 Fall 2019 - Project E1965. Review report should link to the usual view for reviews]]
* [[CSC/ECE 517 Fall 2019 - Project E1943. Refactor sign up sheet controller.rb]]
* [[CSC/ECE 517 Fall 2019 - E1972. OSS project J. Skellington: Accessing Assignment Rubrics]]
* [[CSC/ECE 517 Fall 2019 - E1961. Email notification to reviewers and instructors]]
* [[CSC/ECE 517 Fall 2019 - E1971. OSS project Finklestein: Instructors & Institutions]]
* [[CSC/ECE 517 Fall 2019 - E1953. Tagging report for student]]
* [[CSC/ECE 517 Fall 2019 - E1955.Write unit tests for student_task.rb]]
* [[CSC/ECE 517 Fall 2019 - E1954. Auto-generate submission directory names based on assignment names]]
* [[CSC/ECE 517 Fall 2019 - E1958. Two issues related to assignment management]]
* [[CSC/ECE 517 Fall 2019 - E1948. Refactor review_mapping_helper.rb]]
* [[CSC/ECE 517 Fall 2019 - E1959. Intelligent copying of assignments without topics]]
* [[CSC/ECE 517 Fall 2019 - E1968. Fixes for adding members to teams]]
* [[CSC/ECE 517 Fall 2019 - E1969. Fixes for reviews not being available]]
* [[CSC/ECE 517 Fall 2019 - E1951. Remove multiple topics at a time]]
* [[CSC/ECE 517 Fall 2019 - E1957. Time travel Not Allowed..!!! Restrict TAs’ ability to change their own grade + limit file-size upload]]
* [[CSC/ECE 517 Fall 2019 - E1963. Changing assignment participant role]]
* [[CSC/ECE 517 Fall 2019 - E1941. Issues related to topic deadlines]]
* [[CSC/ECE 517 Fall 2019 - E1966. Tabbed_reviews partial file refactor for displaying the alternate view of reviews]]
* [[CSC/ECE 517 Fall 2019 - E1962. Email notification upon account creation]]
* [[CSC/ECE 517 Fall 2019 - E1967. Fix glitches in author feedback]]
* [[CSC/ECE 517 Fall 2019 - E1960. Create new late policy successfully and fixing "Back" link]]
* [[CSC/ECE 517 Fall 2019 - E1939. OSS Project Juniper: Bookmark enhancements]]
* [[CSC/ECE 517 Fall 2019 - M1950. Support Asynchronous Web Assembly Compilation]]
* [[CSC/ECE 517 Fall 2019 - E1938. OSS project Duke Blue: Fix import glitches]]
* [[CSC/ECE 517 Fall 2019 - E1985. Let course staff and students do reviews]]
* [[CSC/ECE 517 Fall 2019 - M1951. Implement missing OffscreenCanvas APIs]]
* [[CSC/ECE 517 Fall 2019 - E1940. Improving email notification]]
* [[CSC/ECE 517 Fall 2019 - E1956. There is no shortcut to get free review points: Review Assignment Bug]]
* [[CSC/ECE 517 Fall 2019 - E1942. Refactor stage deadlines in assignment.rb]]
* [[CSC/ECE 517 Fall 2019 - M1952. Missing DOM features project]]
* [[CSC/ECE 517 Fall 2019 - E1945. Refactor users_controller.rb]]
* [[CSC/ECE 517 Fall 2019 - E1964. Export review scores for projects]]
* [[CSC/ECE 517 Fall 2019 - E1949. Write Unit Tests for Importing assignment participants and import glitches]]
* [[CSC/ECE 517 Fall 2019 - E1944. Refactor review mapping controller]]
* [[CSC/ECE 517 Fall 2019 - E1946. Refactor Questionnaire controller]]
* [[CSC/ECE 517 Fall 2019 - E1982. Regulate changing of rubrics while projects are in progress]]
* [[CSC/ECE 517 Fall 2019 - E1989. Track the time students look at other submissions]]
* [[CSC/ECE 517 Fall 2019 - E1990. Integrate suggestion detection algorithm]]
* [[CSC/ECE 517 Fall 2019 - E1995. Tests for email functionality]]
* [[CSC/ECE 517 Fall 2019 - E1993 Track Time Between Successive Tag Assignments]]
* [[CSC/ECE 517 Fall 2019 - E1979. Completion/Progress view]]
* [[CSC/ECE 517 Fall 2019 - E1995. Weights in grade calculation]]
* [[CSC/ECE 517 Fall 2019 - E1980. Sort instructor reports by name, ID, score, etc.]]
* [[CSC/ECE 517 Fall 2019 - E1984. Improve self-review Link peer review & self-review to derive grades]]
* [[CSC/ECE 517 Fall 2019 - E1994. Mentor management for assignments without topics]]
* [[CSC/ECE 517 Fall 2019 - E1997. Issues related to meta-reviewing]]
* [[CSC/ECE 517 Fall 2019 - E1981. Student-generated questions added to rubric]]
* [[CSC/ECE 517 Fall 2019 - E1996. Enhancements to review grader]]
* [[CSC/ECE 517 Fall 2019 - E1987. Improving search facility in Expertiza]]

CSC/ECE 517 Fall 2019 - E1985. Let course staff and students do reviews

2019-11-11T00:23:59Z

Mtrawic: Created page with "Hello World"

Hello World

CSC/ECE 517 Fall 2019 - E1985.-Let course staff and students do reviews

2019-11-11T00:20:24Z

Mtrawic: Created page with "Hello World"

Hello World

CSC/ECE 517 Fall 2019 - E1953. Tagging report for student

2019-11-04T23:42:55Z

Mtrawic: /* Deployed Example */

CSC/ECE 517 Fall 2019 - E1953. Tagging report for student

==Introduction==
===About Expertiza ===
[http://expertiza.ncsu.edu/ Expertiza] is an open source project based on [http://rubyonrails.org/ Ruby on Rails] framework. Expertiza allows the instructor to create new assignments and customize new or existing assignments. It also allows the instructor to create a list of topics the students can sign up for. Students can form teams in Expertiza to work on various projects and assignments. Students can also peer review other students' submissions. Expertiza supports submission across various document types, including the URLs and wiki pages.

===Problem Statement===
The following tasks were accomplished in this project:

* If tagging is enabled, number of total tags per round is shown on student's score page
* Implemented feature which shows number of tags student have entered
* Implemented feature to show number of tags entered and total number of tags in Assignment page.
* Implemented feature to show tagging counts dynamically when student change tagging type in score page
* Added RSPEC testcases for testing changes done for tagging

===About Tagging===
Expertiza has a feature of peer review where other teams can provide feedback on student's work. There is also another feature of "Tags", where a student can answer if the feedback provided were helpful or not. Tags can vary from assignment to assignment but the main motive of it is to provide information on whether the feedback provides helpful information for student's assignment. The picture below shows an example of tags.

[[File:Example-tags.png]]

===Motivation===
For a particular assignment, there may be a lot of questions and hence multiple feedback. Students might miss a few tags and they might go unanswered. This feature will provide students to keep track of answered tags and show them how many tags have they answered out of total tags on the page.

==Design==
For counting, we are primarily concerned with two numbers: the number of possible tags for an assignment and the number of completed tags for an assignment. The latter is easy to calculate as it's directly stored as AnswerTags. The former is more difficult: possible tags are stored as TagPromptDeployments which do not have a 1:1 correspondence to the number of prompts the user sees on their page.

[[File:E1953_UML.png]]

The green classes are used to count the number of tags done, the red classes are used to count the number of tags possible, and the yellow classes are used to calculate both counts. All classes help to narrow the scope to a specific user and assignment.

==Implementation==

=== Task 1 : Count answered tags in an assignment on Assignment Homepage ===

==== Description ====
On clicking of particular Assignment, student can see his/her teams, work, scores, etc. There is no feature which allows student to show how many of the review tags has he/she answered on this page. This task implements this feature where student can see answered tags with respect to total tags in Assignment page.

==== Code Implemented ====
'''app/controllers/student_task_controller.rb'''
- The following code counts the number of Tags present in an assignment and total number of answered tags in that particular assignment.
<pre>
@completed_tags = 0
@total_tags = 0
vmlist.each do |vm|
vm.list_of_rows.each do |r|
r.score_row.each do |row|
vm_prompts = row.vm_prompts.select {|prompt| prompt.tag_dep.tag_prompt.control_type.downcase != "checkbox"}
@total_tags += vm_prompts.count
vm_prompts.each do |vm_prompt|
answer_tag = AnswerTag.where(tag_prompt_deployment_id: vm_prompt.tag_dep, user_id: @participant.user_id, answer: vm_prompt.answer).first
if !answer_tag.nil? and answer_tag.value != "0"
@completed_tags += 1
end
end
end
end
end
</pre>

=== Task 2 : Count answered tags in an assignment per rounds on Summary Report Page ===

==== Description ====
On clicking of particular Assignment>Your Scores> student can see his/her score. Student can also see all the reviews per rounds in particular assignment. This new feature will allow student to see number of answered tags per round with respect to number of total tags present in particular round

==== Code Implemented ====
'''app/controllers/grades_controller.rb'''
- The following code counts the number of Tags present in a round and total number of answered tags in that particular round.
<pre>
@vmlist.each do |vm|
if vm.round == @round
vm.list_of_rows.each do |r|
r.score_row.each do |row|
vm_prompts = row.vm_prompts.select {|prompt| prompt.tag_dep.tag_prompt.control_type.downcase != "checkbox"}
if vm_prompts.count > 0
@total_tags += vm_prompts.count
vm_prompts.each do |vm_prompt|
answer_tag = AnswerTag.where(tag_prompt_deployment_id: vm_prompt.tag_dep, user_id: @participant.user_id, answer: vm_prompt.answer).first
if !answer_tag.nil? and answer_tag.value != "0"
@completed_tags += 1
end
end
end
end
end
end
end
@total_tags_array.append(@total_tags)
@completed_tags_array.append(@completed_tags)
</pre>

==Process==
[[File:E1953_FlowChart2.png]]

==Deployed Example==
Below are instructions for navigating through expertiza to see the review tags in action.

'''1.''' Go to http://152.46.18.192:8080/ and log in as {username: student7552 password: password}.

'''2.''' After you log in if you are not at the assignments landing page (152.46.18.192:8080//student_task/list) click on Assignments to navigate there.

'''3.''' Click on "Final project (and design doc)" from the list of assignments in the table at the bottom of the page.

'''4.''' Behold the tag counter next to the bullet labeled "Your scores". It reads (You have tagged the available tags completed / total available reviews).

[[File:Assignment-page-tags.png]]

'''5'''. Further, click on "Your scores". On the next page that shows up, click on a row in the table to see the tags. Notice in the last column of the header of the table is a cell containing the available tags completed / total available tags.

[[FIle:Per-round-tags.png‎ ]]

'''6.''' After clicking on a row to reveal tags drag the blue square towards 'No' or 'Yes' and refresh the page. Observe that the tags left / total tags has updated.

'''7.''' To observe the new per round tagging capabilities log out and then log in as {username: student8115 password: password}.

'''8.''' Select the first Program 2 from the list of assignments. In the next page click on "Your scores" as in step 4 above. Scroll down and behold tagging statistics for each round in the last column of the table headers.

[[FIle:Per-round-tagging2.png‎ ]]

==Teammates==
Mentor -Akanksha Mohan

* Sameer Adhikari (sadhika2)
* Spencer Yoder (smyoder)
* Mark Trawick (mtrawic)

==References==
# [https://github.com/expertiza/expertiza Github Repository for Expertiza]
# [http://wiki.expertiza.ncsu.edu/index.php/Documentation_on_Database_Tables Expertiza Documentation on Database Tables]
# [https://guides.rubyonrails.org/api_app.html Rails Guide Documentation]

CSC/ECE 517 Fall 2019 - E1953. Tagging report for student

2019-10-29T01:55:13Z

Mtrawic: /* Deployed Example */

CSC/ECE 517 Fall 2019 - E1953. Tagging report for student

==Introduction==
===About Expertiza ===
[http://expertiza.ncsu.edu/ Expertiza] is an open source project based on [http://rubyonrails.org/ Ruby on Rails] framework. Expertiza allows the instructor to create new assignments and customize new or existing assignments. It also allows the instructor to create a list of topics the students can sign up for. Students can form teams in Expertiza to work on various projects and assignments. Students can also peer review other students' submissions. Expertiza supports submission across various document types, including the URLs and wiki pages.

===Problem Statement===
The following tasks were accomplished in this project:

* If tagging is enabled, number of total tags per round is shown on student's score page
* Implemented feature which shows number of tags student have entered
* Implemented feature to show number of tags entered and total number of tags in Assignment page.
* Implemented feature to show tagging counts dynamically when student change tagging type in score page
* Added RSPEC testcases for testing changes done for tagging

===About Tagging===
Expertiza has a feature of peer review where other teams can provide feedback on student's work. There is also another feature of "Tags", where a student can answer if the feedback provided were helpful or not. Tags can vary from assignment to assignment but the main motive of it is to provide information on whether the feedback provides helpful information for student's assignment. The picture below shows an example of tags.

[[File:Example-tags.png]]

===Motivation===
For a particular assignment, there may be a lot of questions and hence multiple feedback. Students might miss a few tags and they might go unanswered. This feature will provide students to keep track of answered tags and show them how many tags have they answered out of total tags on the page.

==Design==
For counting, we are primarily concerned with two numbers: the number of possible tags for an assignment and the number of completed tags for an assignment. The latter is easy to calculate as it's directly stored as AnswerTags. The former is more difficult: possible tags are stored as TagPromptDeployments which do not have a 1:1 correspondence to the number of prompts the user sees on their page.

[[File:E1953_UML.png]]

The green classes are used to count the number of tags done, the red classes are used to count the number of tags possible, and the yellow classes are used to calculate both counts. All classes help to narrow the scope to a specific user and assignment.

==Implementation==

=== Task 1 : Count answered tags in an assignment on Assignment Homepage ===

==== Description ====
On clicking of particular Assignment, student can see his/her teams, work, scores, etc. There is no feature which allows student to show how many of the review tags has he/she answered on this page. This task implements this feature where student can see answered tags with respect to total tags in Assignment page.

==== Code Implemented ====
'''app/controllers/student_task_controller.rb'''
- The following code counts the number of Tags present in an assignment and total number of answered tags in that particular assignment.
<pre>
@completed_tags = 0
@total_tags = 0
vmlist.each do |vm|
vm.list_of_rows.each do |r|
r.score_row.each do |row|
vm_prompts = row.vm_prompts.select {|prompt| prompt.tag_dep.tag_prompt.control_type.downcase != "checkbox"}
@total_tags += vm_prompts.count
vm_prompts.each do |vm_prompt|
answer_tag = AnswerTag.where(tag_prompt_deployment_id: vm_prompt.tag_dep, user_id: @participant.user_id, answer: vm_prompt.answer).first
if !answer_tag.nil? and answer_tag.value != "0"
@completed_tags += 1
end
end
end
end
end
</pre>

=== Task 2 : Count answered tags in an assignment per rounds on Summary Report Page ===

==== Description ====
On clicking of particular Assignment>Your Scores> student can see his/her score. Student can also see all the reviews per rounds in particular assignment. This new feature will allow student to see number of answered tags per round with respect to number of total tags present in particular round

==== Code Implemented ====
'''app/controllers/grades_controller.rb'''
- The following code counts the number of Tags present in a round and total number of answered tags in that particular round.
<pre>
@vmlist.each do |vm|
if vm.round == @round
vm.list_of_rows.each do |r|
r.score_row.each do |row|
vm_prompts = row.vm_prompts.select {|prompt| prompt.tag_dep.tag_prompt.control_type.downcase != "checkbox"}
if vm_prompts.count > 0
@total_tags += vm_prompts.count
vm_prompts.each do |vm_prompt|
answer_tag = AnswerTag.where(tag_prompt_deployment_id: vm_prompt.tag_dep, user_id: @participant.user_id, answer: vm_prompt.answer).first
if !answer_tag.nil? and answer_tag.value != "0"
@completed_tags += 1
end
end
end
end
end
end
end
@total_tags_array.append(@total_tags)
@completed_tags_array.append(@completed_tags)
</pre>

==Process==
[[File:E1953_FlowChart2.png]]

==Deployed Example==
Below are instructions for navigating through expertiza to see the review tags in action.

'''1.''' Go to http://152.46.19.137:8080 and log in as {username: student7552 password: password}.

'''2.''' After you log in if you are not at the assignments landing page (152.46.19.137:8080/student_task/list) click on Assignments to navigate there.

'''3.''' Click on "Final project (and design doc)" from the list of assignments in the table at the bottom of the page.

'''4.''' Behold the tag counter next to the bullet labeled "Your scores". It reads (You have tagged the available tags completed / total available reviews).

[[File:Assignment-page-tags.png]]

'''5'''. Further, click on "Your scores". On the next page that shows up, click on a row in the table to see the tags. Notice in the last column of the header of the table is a cell containing the available tags completed / total available tags.

[[FIle:Per-round-tags.png‎ ]]

'''6.''' After clicking on a row to reveal tags drag the blue square towards 'No' or 'Yes' and refresh the page. Observe that the tags left / total tags has updated.

'''7.''' To observe the new per round tagging capabilities log out and then log in as {username: student8115 password: password}.

'''8.''' Select the first Program 2 from the list of assignments. In the next page click on "Your scores" as in step 4 above. Scroll down and behold tagging statistics for each round in the last column of the table headers.

[[FIle:Per-round-tagging2.png‎ ]]

==Teammates==
Mentor -Akanksha Mohan

* Sameer Adhikari (sadhika2)
* Spencer Yoder (smyoder)
* Mark Trawick (mtrawic)

==References==
# [https://github.com/expertiza/expertiza Github Repository for Expertiza]
# [http://wiki.expertiza.ncsu.edu/index.php/Documentation_on_Database_Tables Expertiza Documentation on Database Tables]
# [https://guides.rubyonrails.org/api_app.html Rails Guide Documentation]

CSC/ECE 517 Fall 2019 - E1953. Tagging report for student

2019-10-29T01:23:46Z

Mtrawic: /* About Tagging */

CSC/ECE 517 Fall 2019 - E1953. Tagging report for student

==Introduction==
===About Expertiza ===
[http://expertiza.ncsu.edu/ Expertiza] is an open source project based on [http://rubyonrails.org/ Ruby on Rails] framework. Expertiza allows the instructor to create new assignments and customize new or existing assignments. It also allows the instructor to create a list of topics the students can sign up for. Students can form teams in Expertiza to work on various projects and assignments. Students can also peer review other students' submissions. Expertiza supports submission across various document types, including the URLs and wiki pages.

===Problem Statement===
The following tasks were accomplished in this project:

* If tagging is enabled, number of total tags per round is shown on student's score page
* Implemented feature which shows number of tags student have entered
* Implemented feature to show number of tags entered and total number of tags in Assignment page.
* Implemented feature to show tagging counts dynamically when student change tagging type in score page
* Added RSPEC testcases for testing changes done for tagging

===About Tagging===
Expertiza has a feature of peer review where other teams can provide feedback on student's work. There is also another feature of "Tags", where a student can answer if the feedback provided were helpful or not. Tags can vary from assignment to assignment but the main motive of it is to provide information on whether the feedback provides helpful information for student's assignment. The picture below shows an example of tags.

[[File:Example-tags.png]]

===Motivation===
For a particular assignment, there may be a lot of questions and hence multiple feedback. Students might miss a few tags and they might go unanswered. This feature will provide students to keep track of answered tags and show them how many tags have they answered out of total tags on the page.

==Design==
//yet to do

==Implementation==

=== Task 1 : Count answered tags in an assignment on Assignment Homepage ===

==== Description ====
On clicking of particular Assignment, student can see his/her teams, work, scores, etc. There is no feature which allows student to show how many of the review tags has he/she answered on this page. This task implements this feature where student can see answered tags with respect to total tags in Assignment page.

==== Code Implemented ====
'''app/controllers/student_task_controller.rb'''
- The following code counts the number of Tags present in an assignment and total number of answered tags in that particular assignment.
<pre>
@completed_tags = 0
@total_tags = 0
vmlist.each do |vm|
vm.list_of_rows.each do |r|
r.score_row.each do |row|
vm_prompts = row.vm_prompts.select {|prompt| prompt.tag_dep.tag_prompt.control_type.downcase != "checkbox"}
@total_tags += vm_prompts.count
vm_prompts.each do |vm_prompt|
answer_tag = AnswerTag.where(tag_prompt_deployment_id: vm_prompt.tag_dep, user_id: @participant.user_id, answer: vm_prompt.answer).first
if !answer_tag.nil? and answer_tag.value != "0"
@completed_tags += 1
end
end
end
end
end
</pre>

=== Task 2 : Count answered tags in an assignment per rounds on Summary Report Page ===

==== Description ====
On clicking of particular Assignment>Your Scores> student can see his/her score. Student can also see all the reviews per rounds in particular assignment. This new feature will allow student to see number of answered tags per round with respect to number of total tags present in particular round

==== Code Implemented ====
'''app/controllers/grades_controller.rb'''
- The following code counts the number of Tags present in a round and total number of answered tags in that particular round.
<pre>
@vmlist.each do |vm|
if vm.round == @round
vm.list_of_rows.each do |r|
r.score_row.each do |row|
vm_prompts = row.vm_prompts.select {|prompt| prompt.tag_dep.tag_prompt.control_type.downcase != "checkbox"}
if vm_prompts.count > 0
@total_tags += vm_prompts.count
vm_prompts.each do |vm_prompt|
answer_tag = AnswerTag.where(tag_prompt_deployment_id: vm_prompt.tag_dep, user_id: @participant.user_id, answer: vm_prompt.answer).first
if !answer_tag.nil? and answer_tag.value != "0"
@completed_tags += 1
end
end
end
end
end
end
end
@total_tags_array.append(@total_tags)
@completed_tags_array.append(@completed_tags)
</pre>

==Process==
[[File:E1953_FlowChart2.png]]

==Deployed Example==
Below is instructions for navigating through expertiza to see the review tags in action.

'''1.''' Go to http://152.46.19.137:8080 and log in as {username: student7552 password: password}.

'''2.''' After you log in if you are not at the assignments landing page (152.46.19.137:8080/student_task/list) click on Assignments to navigate there.

'''3.''' Click on "Final project (and design doc)" from the list of assignments in the table at the bottom of the page.

'''4.''' Behold the tag counter next to the bullet labeled "Your scores". It reads (You have tagged the available tags completed / total available reviews).

[[File:Assignment-page-tags.png]]

'''5'''. Further, click on "Your scores". On the next page that shows up, click on a row in the table to see the tags. Notice in the last column of the header of the table is a cell containing the available tags completed / total available tags.

[[FIle:Per-round-tags.png‎ ]]

'''6.''' After clicking on a row to reveal tags drag the blue square towards 'No' or 'Yes' and refresh the page. Observe that the tags left / total tags has updated.

'''7.''' To observe the new per round tagging capabilities log out and then log in as {username: student8115 password: password}.

'''8.''' Select the first Program 2 from the list of assignments. In the next page click on "Your scores" as in step 4 above. Scroll down and behold tagging statistics for each round in the last column of the table headers.

[[FIle:Per-round-tagging2.png‎ ]]

==Teammates==
Mentor -Akanksha Mohan

* Sameer Adhikari (sadhika2)
* Spencer Yoder (smyoder)
* Mark Trawick (mtrawic)

==References==
# [https://github.com/expertiza/expertiza Github Repository for Expertiza]
# [http://wiki.expertiza.ncsu.edu/index.php/Documentation_on_Database_Tables Expertiza Documentation on Database Tables]
# [https://guides.rubyonrails.org/api_app.html Rails Guide Documentation]

File:Example-tags.png

2019-10-29T01:21:15Z

Mtrawic: uploaded a new version of "File:Example-tags.png"

File:Example-tags.png

2019-10-29T01:17:57Z

Mtrawic:

CSC/ECE 517 Fall 2019 - E1953. Tagging report for student

2019-10-29T01:09:01Z

Mtrawic: /* Deployed Example */

File:Per-round-tagging2.png

2019-10-29T01:08:21Z

Mtrawic:

CSC/ECE 517 Fall 2019 - E1953. Tagging report for student

2019-10-29T01:04:28Z

Mtrawic: /* Deployed Example */

CSC/ECE 517 Fall 2019 - E1953. Tagging report for student

2019-10-29T00:52:20Z

Mtrawic: /* Testing with UI */

File:Per-round-tags.png

2019-10-29T00:51:39Z

Mtrawic:

File:Assignment-page-tags.png

2019-10-29T00:44:51Z

Mtrawic: uploaded a new version of "File:Assignment-page-tags.png"

File:Assignment-page-tags.png

2019-10-29T00:33:55Z

Mtrawic:

CSC/ECE 517 Fall 2019 - E1953. Tagging report for student

2019-10-28T20:59:39Z

Mtrawic: /* Testing with UI */

CSC/ECE 517 Fall 2019 - E1953. Tagging report for student

2019-10-28T20:58:26Z

Mtrawic: /* Testing with UI */

CSC/ECE 517 Fall 2019 - E1953. Tagging report for student

2019-10-28T20:47:18Z

Mtrawic: /* Testing with UI */

CSC/ECE 517 Fall 2019 - E1953. Tagging report for student

2019-10-28T18:46:59Z

Mtrawic: