CSC/ECE 517 Fall 2019 - E1985. Let course staff and students do reviews
Problem Statement
Peer review is a great way for students to learn about how well they have developed their application. However, sometimes, the peer reviews may not be thorough and the team/person's work reviewed might not reflect the actual status of the development. By letting course staff perform reviews as well, the reviewer and the reviewee both benefit, improving the overall learning experience. Currently, there exists no method for the instructor/TA to perform reviews of assignments submitted by the students. Hence this is what this project aims at, allowing instructor/TA to perform reviews.
Thus, the primary aim of this project is to allow instructors to submit reviews of student work, using the same review form that students use to do reviews.
Implementation Details
Our project aims at enabling the instructor/TA to review the submissions using the same form that a student uses while peer-reviewing. For this, our implementation has been divided into 3 parts, and can be described as follows:
- The first part involves the primary functionality of letting the staff perform a review on students submission.
- The second part involves allowing the students to identify from their end, which review has been performed by an instructor/TA. This will help students improve their project as the opinions presented by a staff member would be valid.
- Thirdly, we implemented an additional feature that allows the instructor/TA to add himself as a participant while creating/editing an Assignment (This way seemed to be more convenient rather than going to the “Add Participant” page and adding themselves as a participant.)
Part 1: Allowing instructor/TA to perform review
Note 1: “Instructor/TA” will be referred to by “user” here onwards.
Note 2: To be able to perform a review, the user has to be a participant of that assignment. We have implemented “Add as Participant” as an additional feature which is explained in Part 3. If the user is not a participant, he will only see the link to “Assign grade” that too after the assignment deadline passes. Now, for each assignment, there is a page called “List Submissions” where the user can view all the submissions. So we have modified this page so that for each team submission a link for “Begin review,” “Edit review,” “View review,” “Update review,” or “Assign grade” may appear depending on the stage in which the assignment is.
The following describes the flow of our project considering a fresh assignment.
- When the assignment is created, it is by default is in the “submission” stage. Hence the user will not see any link on the list submission page.
- When the first deadline passes, the assignment goes into the “review” stage. Hence the link “Begin review” will be visible for each team submission, where the instructor/TA can click on to submit the review. Two cases can occur here:
- If the user “submits” a review, they will be redirected to the list submission page where they’ll see the option “View review” or “Update review.” Being an instructor/TA, they are given the flexibility to update their review at any time during the “review” stage.
- If the user “saves” a review, they will be redirected to the list submission page where they’ll see the option “View review” and “Edit Review (as they have not submitted it yet.)”
- After the first review is over, if more stages of “submissions” and “reviews” exist, it will follow the above 2 steps until the final assignment deadline does not pass. Note that at any round of the review, the user can “Begin review” for a submission which they had not reviewed in the previous rounds. This is to provide flexibility to them as per the requirement.
- Finally, after the assignment deadline passes, the assignment goes into the “Finished” stage. This is when the “Assign grade” link will be visible to the user. Along with this, depending on whether they have reviewed a particular submission or not, the begin, view, update links will be visible to them as they should be able to perform the review even after the assignment deadline for the students has passed.
Files Edited
The following are the files (controllers and views) edited for the implementation of the review process. The comments are given along with the code to explain what is being done.
- Controller: app/controllers/response_controller.rb
# Ensuring recent submission is not nil
team_most_recent_submission = AssignmentTeam.find(@map.reviewee_id).most_recent_submission
if @response.nil? || (!team_most_recent_submission.nil? && team_most_recent_submission.updated_at > @response.updated_at)
...
# Page should be directed to list_submissions if TA/instructor performs the review
when "ta_review"
redirect_to controller: 'assignments', action: 'list_submissions', id: @map.response_map.assignment.id
- Controller: app/helpers/assignment_helper.rb
topic_id = topic.try :id
...
# Needed topic ID for review implpementation, so added it to return list of the function
[topic_id, topic_identifier ||= "", topic_name ||= "", users_for_curr_team, participants]
- View: app/views/assignments/list_submissions.html.erb
# Retrieved topic ID for review implpementation
<% topic_id, topic_identifier, topic_name, users_for_curr_team, participants = get_data_for_list_submissions(team) %>
...
<!-- Display "Assign Grade" link after final deadline of assignment has passed -->
<% if @assignment.get_current_stage() == "Finished" %>
<%= link_to 'Assign Grade', { controller: 'grades', action: 'view_team', id: participants.first.id}, target: '_blank' %><br>
<% end %>
<!-- Checking if logged in user (instructor/TA) is participant in assignment -->
<% reviewer = Participant.find_by(:parent_id => @assignment.id, :user_id => session[:user].id)%>
<!-- Reviewing options are available only to "participants" of assignment -->
<!-- So only if reviewer is not nil, following applicable links will be displayed -->
<% if !reviewer.nil? %>
<!-- Checking if instructor/TA has been mapped to current team -->
<% @mapping = ReviewResponseMap.find_by(:reviewed_object_id => @assignment.id, :reviewer_id => reviewer.id, :reviewee_id => team.id) %>
<!-- If mapping is nil, a response cannot be created. So user should be able to "Begin Review" during any review stage, or after deadline has passed -->
<% if @mapping.nil? && (@assignment.get_current_stage() == "review" || @assignment.get_current_stage() == "Finished") %>
<% ReviewResponseMap.create(reviewee_id: team.id, reviewer_id: reviewer.id, reviewed_object_id: @assignment.id) %>
<% map = ReviewResponseMap.find_by(reviewee_id: team.id, reviewer_id: reviewer.id, reviewed_object_id: @assignment.id) %>
<%= link_to 'Begin Review', { controller: 'response', action: 'new', :id => map.id, :return => "ta_review" } %><br>
<!-- Any time during review or after deadline, user is given the flexibility to begin, edit or update their review -->
<% elsif @assignment.get_current_stage() == "review" || @assignment.get_current_stage() == "Finished" %>
<% map1 = ReviewResponseMap.find_by(reviewee_id: team.id, reviewer_id: reviewer.id, reviewed_object_id: @assignment.id) %>
<% @sorted_responses = Array.new %>
<% @sorted_responses = Response.where(:map_id => map1.id) %>
<!-- This means that mapping is created, but user never began the review. Hence linking to "Begin Review" -->
<% if @sorted_responses.empty?%>
<%= link_to 'Begin Review', { controller: 'response', action: 'new', :id => map1.id, :return => "ta_review" } %><br>
<% else %>
<!-- There is at least more than one response. Hence we obtain latest response of them all. -->
<% @sorted_responses = @sorted_responses.sort_by {|obj| obj.updated_at} # the latest should be at the last %>
<% @latest_response = @sorted_responses.last %>
<%if @latest_response.round.nil?%> <!-- Getting current round as last response round in unavailable -->
<% last_response_round = AssignmentDueDate.done_in_assignment_round(@assignment.id, @latest_response) %>
<%else%> <!-- Getting round in which the last response was given -->
<% last_response_round = @latest_response.round %>
<%end%>
<% current_round = @assignment.number_of_current_round(topic_id) %>
<!-- Always displaying "View Review" link for latest response, as long as at least one response exists -->
<%= link_to "View Review", {:controller => 'response', :action => 'view', :id => @latest_response.id} %><br>
<% if last_response_round == current_round %>
<% latest_submission = team.most_recent_submission %>
<% if !@latest_response.is_submitted %> <!-- Allow to "Edit" review which has not been submitted yet -->
<%= link_to "Edit Review", {:controller => 'response', :action => 'edit', :id => @latest_response.id, :return => "ta_review" } %>
<% else %> <!-- Allow to "Update" review if team has updated their submission -->
<%= link_to "Update Review", {:controller => 'response', :action => 'new', :id => map1.id, :return => "ta_review" } %>
<%end%>
<% else %> <!-- Allow to "Update" review when round changes -->
<%= link_to "Update Review", {:controller => 'response', :action => 'new', :id => map1.id, :return => "ta_review" } %>
<% end %>
<% end %>
<% end %>
<% end %>
- View: app/views/grades/view_team.html.erb
<!--Obtain topic name and topic_id from assignment helper-->
<% topic_id_num, topic_id, topic_name, users_for_curr_team, participants = get_data_for_list_submissions(@team) %>
- View: app/views/student_review/_responses.html.erb
# Ensuring recent submission is not nil
<% elsif !latest_submission.nil? && @latest_response.updated_at < latest_submission.updated_at %>
---
Part 2: Allow students to identify that their submission has been reviewed by the instructor
Currently Scenario: There is no way for the student to know which review was by the instructor.
Requirement: This helps the student to identify the review made by the instructor from the other reviews.
Implementation: A simple check is made to determine if the reviewer is a TA or Instructor. app/assets/images/staff.png is displayed to make an instructor performed review stand out from other reviews. If the mouse hovers over the icon a tool tip appears saying "Icon indicates this review was done by course staff"
Files Edited
- Helper: app/helpers/grades_helper.rb
# Determines if a review was completed by a staff participant (either a TA or an instructor)
# Called in view_team.html to determine if a review should be marked as reviewed by course staff.
def done_by_staff_participant?( review )
role = Role.find(User.find(Participant.find(ResponseMap.find(Response.find(review.id).map_id).reviewer_id).user_id).role_id).name
return (role == "Instructor") || (role == "Teaching Assistant")
end
# Determines if given participant is a staff participant (either a TA or an instructor)
# Called in views/response/view.html.erb to determine if a review should be marked as reviewed by course staff.
def is_course_staff?( participant )
role = Role.find(User.find(Participant.find(participant.id).user_id).role_id).name
return (role == "Instructor") || (role == "Teaching Assistant")
end
- View: app/views/view_team.html.erb
<% if done_by_staff_participant?(review) %>
<!-- Add icon to signal review was done by staff_participant -->
<th class="sorter-false">
<a target="_blank" href="../response/view?id=<%= review.id.to_s %>" data-toggle="tooltip" data-placement="right" title="Click to see details">
<%= "Review " + i.to_s %></a> <img data-toggle="tooltip" data-placement="right" title="Icon indicates this review was done by course staff" class="img_icon" src="/assets/staff.png"> </th>
<% else %>
<th class="sorter-false"> <a target="_blank" href="../response/view?id=<%= review.id.to_s %>" data-toggle="tooltip" data-placement="right" title="Click to see details"><%= "Review " + i.to_s %></a> </th>
<% end %>
Part 3: Add instructor/TA as a participant while creating the assignment (Extra Feature)
Current scenario
The instructor/TA will be able to perform student reviews (after implementation of Part 1) for the assignment, only when he is a participant of the same assignment. Thus, every time he creates an assignment and wants to add himself as a participant in order to perform student reviews, he is supposed to do the following
Go to list of assignments -> Find the required assignment -> Click on add participant -> Type user Id -> Click on add
The same flow is as shown below where the instructor/TA has created a new assignment say “Program 1 Github” wherein he wants himself as a participant to be able to perform student reviews :
To overcome the hassle, we added a check box through which the instructor/TA will get an option to add himself as a participant while creating the assignment. Also, he will be able to add/remove himself as a participant while editing the assignment.
Implementation
The checkbox has been provided to Create New Assignment’s page under the Review strategy tab.
It is added as follows:
If the instructor/TA selects the box while creating the assignment, then he will be added as a participant for the same assignment.
Files edited:
- View: app/views/assignments/edit/_review_strategy.html.erb
In this file, code for displaying the checkbox has been added which is as follows :
<tr>
<td>
<input name='add_instructor' type="hidden" value="false"/>
<%= check_box_tag('add_instructor', value = "1", checked = is_instructor_a_participant?)%>
<%= label_tag('add_instructor', 'Add yourself as a participant?') %>
<img src="/assets/info.png" title='If you add yourself as a participant then you will be able to perform reviews of students for this assignment'>
</td>
</tr>
Thus the checkbox "add_instructor" if selected adds the instructor as a participant to the currently being created assignment. Thus, instructor being a participant allows him to perform reviews for the students for this assignment.
- Controller: app/controllers/assignments_controller.rb
The file contains extra code which adds the instructor/TA as a participant if the checkbox has been selected while creating the assignment. Also, while editing the assignment, he will be able to add/remove himself as a participant for this assignment.
The following methods are added to the controller :
def is_instructor_a_participant?
instructor_id = session[:user].id
assignment_id = @assignment_form.assignment.id.to_s
if assignment_id != nil
@is_instructor_a_participant = Participant.where(user_id: instructor_id , parent_id: assignment_id)
if @is_instructor_a_participant.present?
return true
end
end
return false
end
The method "is_instructor_a_participant?" returns true if the instructor is already a participant for the assignment, else returns false. This method is required to verify that the instructor/TA is not a participant of the assignment, and thus he can be added as a participant if the checkbox has been selected. Also, during editing the assignment, we need to show checkbox being selected if the instructor is a participant and vice versa. Thus, this method serves that purpose as well.
def add_instructor_as_participant(assignment_id)
if params[:add_instructor] == 'false' and is_instructor_a_participant? == true
delete_instructor_as_participant(assignment_id,session[:user].id)
elsif params[:add_instructor] == '1' and is_instructor_a_participant? == false
current_assignment = Object.const_get("Assignment").find(assignment_id)
current_assignment.add_participant(session[:user].name, true, true, true)
end
end
The method "add_instructor_as_participant" is been called from create and update methods in the Assignment controller. It checks if the "Add yourself as participant" checkbox is selected for the currently created/edited assignment. If it is selected during creation or editing, then the instructor is added as a participant to the currently created/edited assignment. The participant is added by using the already existing method "add_participant" in the model assignment.rb Also, if the option is unchecked during editing, then the instructor is removed as a participant.
def delete_instructor_as_participant(assignment_id , instructor_id)
Participant.where(user_id: instructor_id , parent_id: assignment_id).destroy_all
end
The method "delete_instructor_as_participant" removes instructor as a participant for the assignment when the checkbox is unchecked during the editing of the assignment.
Test Plan
Part 1: Allowing instructor/TA to perform review
UI Testing
- Login in as an instructor.
- Username: instructor6
- Password: password
- Choose any assignment from the list of assignments which involves teams; since an assignment without teams does not include the review procedure. For example, “Final project (design and doc)” is a good assignment to choose.
- Now, while testing this functionality, it is recommended to keep the following pages opened in different tabs by clicking on the small icons in the same row as the assignment that you want to test:
a. “Edit” -> “Due Dates” tab
b. “Add participants”
c. “View Submissions”
- Now add the instructor as a participant (3.b). and edit the ALL due dates (3.a.) of different stages to any date later from now (hence the assignment goes into the “submission” stage by default). Save both these changes.
- Now refresh the view submissions page (3.c.), you will see a list of assignments with no link to review or assign grade.
- Now switch to edit due dates tab (3.a.) again. Set the due date for submission to a previous date which has passed, hence the next due date will be for the “review” stage by default (which will have a later due date from now.)
- Refresh view submissions page (3.c.), you will see a list of assignments with a link to “Begin Review.”
- You can click on it. If you “save” your response, you will see that the link has been replace by “View review” and “Edit review” now. Whereas, if you “submit” the response, the link will be replaced by “View review” and “Update review.”
- Depending on the number of stages that an assignment has, you can edit the due date at every stage and check whether the links are displayed appropriately.
- Finally when the last deadline of the assignment is set to a previous date which has passed, you will see the link to “Assign grade” in view submissions (3.c.), and the user will still be able to begin, edit or update a review in order to provide flexibility as per the requirements.
Edge Cases:
- If the user is not a participant of the current assignment, they are shown a list of all submissions made by all teams of the current assignment. They are shown the link to “Assign Grade” only when the final deadline of the assignment passes.
- During any review stage or even after the deadline, the user is offered the flexibility to Begin, Edit, View or Update their review.
Part 2: Allow students to identify that their submission has been reviewed by the instructor
UI Testing
- Login as an instructor and change the due dates for the assignments to be tested. Here the assignment chosen is Final project (and design doc).
- Username: instructor6
- Password: password
- Login in as student. Here we have logged in as student8434
- Username: student8434
- Password: password
- Go to Assignments-> Final project (and design doc). Here click on the assih=gment for which the due dates were changed.
- Click on Your scores There we can see the icon for reviews reviewed by staff.
- Login in as an instructor.
- Username: instructor6
- Password: password
- Go to Manage-> Assignments
- Click on add assignment
- Enter the required details- Assignment name, course etc
- To add instructor/TA as a participant, go to Review Tab and select the "Add yourself as a participant?" checkbox.
- Click on Save The assignment has been created and the instructor/TA is added as a participant to the assignment.
- If the instructor/TA does not select the "Add yourself as a participant?" checkbox while creating the assignment, then he is not added as a participant
- If the instructor/TA selects the "Add yourself as a participant?" checkbox while editing the assignment, then he is added as a participant
- If the instructor/TA unchecks the "Add yourself as a participant?" checkbox while editing the assignment, then he is removed as a participant
- Forked Repository: https://github.com/Matrawick/expertiza/tree/beta
- Pull Request: https://github.com/expertiza/expertiza/pull/1631
- Anuja Kulkarni - apkulka2@ncsu.edu
- Mark Trawick - mtrawic@ncsu.edu
- Parvathy Menon - pharida@ncsu.edu
- Samruddhi Khandale - sskhanda@ncsu.edu
Part 3: Add instructor/TA as a participant while creating the assignment
UI Testing
Edge Cases :
Rspec Testing :
describe '#create' do
before(:each) do
allow(AssignmentForm).to receive(:new).with(any_args).and_return(assignment_form)
@params = {
button: '',
assignment_form: {
assignment_questionnaire: [{"assignment_id" => "1", "questionnaire_id" => "666", "dropdown" => "true",
"questionnaire_weight" => "100", "notification_limit" => "15", "used_in_round" => "1"}],
due_date: [{"id" => "", "parent_id" => "", "round" => "1", "deadline_type_id" => "1", "due_at" => "2017/12/05 00:00", "submission_allowed_id" => "3", "review_allowed_id" => "1", "teammate_review_allowed_id" => "3", "review_of_review_allowed_id" => "1", "threshold" => "1"},
{"id" => "", "parent_id" => "", "round" => "1", "deadline_type_id" => "2", "due_at" => "2017/12/02 00:00", "submission_allowed_id" => "1", "review_allowed_id" => "3", "teammate_review_allowed_id" => "3", "review_of_review_allowed_id" => "1", "threshold" => "1"}],
assignment: {
instructor_id: 2,
course_id: 1,
max_team_size: 1,
id: 1,
name: 'test assignment',
directory_path: '/test',
spec_location: '',
private: false,
show_teammate_reviews: false,
require_quiz: false,
num_quiz_questions: 0,
staggered_deadline: false,
microtask: false,
reviews_visible_to_all: false,
is_calibrated: false,
availability_flag: true,
reputation_algorithm: 'Lauw',
simicheck: -1,
simicheck_threshold: 100
add_instructor:1 #added field
}
}
}
context 'when assignment_form is saved successfully' do
it 'redirects to assignment#edit page' do
allow(assignment_form).to receive(:assignment).and_return(assignment)
allow(assignment_form).to receive(:save).and_return(true)
allow(assignment_form).to receive(:update).with(any_args).and_return(true)
allow(assignment_form).to receive(:create_assignment_node).and_return(double('node'))
allow(assignment).to receive(:id).and_return(1)
allow(Assignment).to receive(:find_by).with(name: 'test assignment').and_return(assignment)
allow(assignment_form).to receive(:add_instructor_as_participant).with(any_args).and_return(true) #added check
allow_any_instance_of(AssignmentsController).to receive(:undo_link)
.with('Assignment "test assignment" has been created successfully. ').and_return(true)
post :create, @params
expect(response).to redirect_to('/assignments/1/edit')
end
end
describe '#update' do
context 'when params does not have key :assignment_form' do
context 'when assignment is saved successfully' do
it 'shows a note flash message and redirects to tree_display#index page' do
allow(assignment).to receive(:save).and_return(true)
params = {
id: 1,
course_id: 1
}
session = {user: instructor}
allow(assignment_form).to receive(:add_instructor_as_participant).with(any_args).and_return(true) #added check
post :update, params, session
expect(flash[:note]).to eq('The assignment was successfully saved.')
expect(response).to redirect_to('/tree_display/list')
end
end
context 'when "Add yourself as a participant?" checkbox is selected' do
it 'calls add_instructor_as_participant which in turn calls add Participant method of Participant controller' do
allow(assignment_form).to receive(:add_participant).with(any_args).and_return(true)
end
end
External Links
Team Members
Mentor: Prof. Edward Gehringer (efg@ncsu.edu)