CSC/ECE 517 Fall 2022 - E2284. Calibration submissions should be copied along with calibration assignments

Introduction

A “calibration assignment” is an assignment where the students are asked to review work that has also been reviewed by a member of the course staff. If the student’s review “resembles” the staff member’s review, then the student is presumed to be a competent reviewer. Here is a further description of calibration assignments. The instructor (or TA) adds a few extra participants to the assignment to set up calibration. The instructor (or TA) then impersonates the extra participants and submits work on behalf of each of the extra participants.

Problem Statement

As of now, when one clicks on the copy button next to the assignment and a copy is created, you can observe that the calibration tab of the copied assignment renders no results at all. An empty table is shown

Having the instructor (or TA) impersonate the extra participants, and submit work on behalf of each of the extra participants is extra trouble. It would be nice if an instructor didn’t have to resubmit the same calibration submissions every semester. Copying the extra participants along with their teams, submissions, and responses when copying an assignment makes things much easier.

Previous Submission

What was Done

They created a new class method that checks if the an assignment was calibrated and if it was, it calls methods from models. It performs the following:

1. They copy all submission records with the assignment_id matching the original assignment.
2. Copy all participants with the parent_id matching the original assignment ID.
3. Copy all the teams with a parent_id matching the original assignment ID.
4. Recreated mappings needed to associate participants with teams and review_response maps.
   1. Their copy methods for teams and participants return hash maps that are used to lookup previous mappings.
   2. Create new TeamsUsers. Associates participants with teams.
      1. Create new ReviewResponseMap. Associates reviewers and reviewees, linked to an assignment, together.
5. Copy all review_responses, using the original review response map as the reference.

Their 'copy_calibrated_reviews' is called in the existing assignment_form class method 'copy', if the copied assignment is able to be saved (to avoid creating new records).

They also added methods for copy naming schemes so that the assignment model validates uniqueness of both the assignment name and directory_path.

Issues

Issue 1: Not all uses of class methods are good.

• self.copy_participants_for_assignment: copies old participants to new --> assignment_participant.rb, it is in teams_users.rb, which seems misplaced.

Issue 2: Returns a mapping from the old response map to the new response map.

• self.copy_review_response_map, returns a mapping from the old response map to the new response map. It is in review_response_map.rb. This can be an instance method. There is a copy_review_responses in this class too.

Issue 3: Code missing without causing a bug

• Line 65 of submit_hyperlink has missing code.

Issue 4: Old code not reused.

• New code was written to copy a team in teams_user.rb. There already exists code to copy a team.

Issue 5: No Automated tests written

Files Changed

assignment_form.rb

• Added a new method "copy_name" to name copied assignments. Where they changed the name of the copied assignments to “copy of copy” and "copy of copy of copy", etc to "Copy of <name> <copy number>"
• Added new method for copying objects needed to recreate calibration reviews

participant.rb

• Wrote a "createparticipant" method in assignment_participant in place of a similar method in participant.rb

response.rb

• Implemented a method for copying previous responses

response_map.rb

• Implemented a method to create ReviewResponse mapping based on mapping of copied assignment

submission_record.rb

• Added method for copying submission records of an assignment

teams_user.rb

• Added a method to create new teams_users so that new participants and teams are associated the same way they were for the previous assignment

assignment_participant.rb

• Added a method for copying extra participants, to not copy participants with type 'CourseParticipant'

Design

Principles we plan to use

1. DRY: We intend to write code that is written only once and utilised wherever needed.
2. Single Responsibility: Our modules, classes and functions would have one responsibility.
3. Open-Closed: We would try to ensure that there is minimum necessity to modify our code and at the same time make sure that there is an opportunity for extension.
4. Law of Demeter: Our classes would be written in such a way that they do not have knowledge of the internal working of other objects. They would not manipulate other objects.
5. Maximize Cohesion and minimize coupling: Our modules would be written to have low dependency on other modules while depending highly on its member functions.

Proposed Workflow

Pseudocode

1. Get Participant instructor_participant with user_id = assignment.instructor_id
2. Create duplicate new_instructor_participant from instructor_participant
3. Set new_instructor_participant.parent_id = new_assignment_id
4. Save new_instructor_participant
5. Get all ResponseMaps with reviewed_object = assignment.id and calibrate_to = True
6. For each response_map
   1. Create duplicate new_response_map from response_map
   2. Set new_response_map.reviewed_object_id = new_assignment_id
   3. Find Team team where id = new_response_map.reviewee_id
   4. Create duplicate new_team from team
   5. Set new_team.parent_id = new_assignment_id
   6. Save new_team
   7. Set new_response_map.reviewer_id = new_instructor_participant.id
   8. Set new_response_map.reviewee_id = new_team.id
   9. Save new_response_map
   10. Copy members of team to new_team
   11. Get all TeamUsers where team_id = new_team.id
   12. For each team_user
       1. Get Participant participant where parent_id = assignment.id and user_id = team_user.user_id
       2. Create duplicate new_participant from participant
       3. Set new_participant.parent_id = new_assignment_id
       4. Save new_participant
   13. Get all SubmissionRecords with assignment_id = old_assignment_id and team_id = team.id
   14. For each submission_record
       1. Create duplicate new_submission_record from submission_record
       2. Set new_submission_record.assignment_id = new_assignment_id
       3. Set new_submission_record.team_id = new_team.id
       4. Save new_submission_record
   15. Get all Responses with map_id = response_map.id
   16. For each response
       1. Create duplicate new_response from response
       2. Set new_response.map_id = new_response_map.id
       3. Save new_response
       4. Get all Answers with response_id = response.id
       5. For each answer
          1. Create duplicate new_answer from answer
          2. Set new_answer.response_id = new_response.id
          3. Save new_answer

Planned Code Changes

app/models/assignment.rb

1. Add an instance method copy_calibrated_responses_from(old_assignment) that copies submissions, teams, participants, and responses from a calibrated assignment to the assignment instance.
2. Add an instance method copy_submissions_from(old_assignment) that copies SubmissionRecords from the old_assignment to the assignment.
3. Add an instance method copy_participants_from(old_assignment) that copies Participants from the old_assignment to the assignment.

app/models/team.rb

1. Add a method get_review_response_mappings() that retrieves all ReviewResponseMappings associated to a team.

app/models/review_response_map.rb

1. Add a method get_response() that retrieves the response for the review response mapping.

Our Work

Added Participant#copy_to_another_assignment(assignment)

This method copies a duplicate entry of itself to another assignment. This method will be used in multiple places ahead.

 def copy_to_another_assignment(assignment)
   new_participant = dup
   new_participant.parent_id = assignment.id
   new_participant.save
   new_participant
 end

Participants Table link

Added Answer#copy_to_response(response)

This method copies a duplicate entry of itself to another response. This method will be used while copying all answers for a response.

 def copy_to_response(response)
   new_answer = dup
   new_answer.response_id = response.id
   new_answer.save
   new_answer
 end

Answers Table Link

Added Response#copy_to_another_response_map(response_map)

This method copies a duplicate entry of itself to another response map along with all the associated answers.

 def copy_to_another_response_map(response_map)
   new_response = dup
   new_response.map_id = response_map.id
   new_response.save
   answers = Answer.where(response_id: id)
   answers.each do |answer|
     new_answer = answer.copy_to_response(new_response)
   end
   new_response
 end

Responses Table link

Added SubmissionRecord#copy_to_another_team(team)

This method copies a duplicate entry of itself to another team.

 def copy_to_another_team(team)
   new_submission_record = dup
   new_submission_record.team_id = team.id
   new_submission_record.assignment_id = team.parent_id
   new_submission_record.save
   new_submission_record
 end

Submission Records Table Link

Added Team#copy_to_another_assignment(assignment)

This method copies a duplicate entry of itself to another assignment along with associated team users, team user nodes and creates participants for the team users if not created before. Team#copy_members is reused to comply with DRY principle.

 def copy_to_another_assignment(new_assignment)
   new_team = dup
   new_team.parent_id = new_assignment.id
   new_team.save
   copy_members(new_team)
   # Creates a new Participant for every User in the Team and associates the Participant to the new Assignment
   team_users = TeamsUser.where(team_id: new_team.id)
   team_users.each do |team_user|
     # Checks if Participant for the new Assignment is missing for the given User
     participant_missing = Participant.where(
       parent_id: new_assignment.id, user_id: team_user.user_id
     ).first.nil?
     if participant_missing
       participant = Participant.where(parent_id: parent_id, user_id: team_user.user_id).first
       participant.copy_to_another_assignment(new_assignment)
     end
   end
   new_team
 end

Added ResponseMap#copy_to_another_assignment(assignment)

This method copies a duplicate entry of itself to another assignment along with the reviewee team and its associated submission records, and the responses and its associated answers.

 def copy_to_another_assignment(assignment, team, instructor_participant)
   new_response_map = dup
   new_response_map.reviewed_object_id = assignment.id
   new_response_map.reviewer_id = instructor_participant.id
   new_response_map.reviewee_id = team.id
   new_response_map.save
   submission_records = SubmissionRecord.where(assignment_id: reviewed_object_id, team_id: reviewee_id)
   submission_records.each do |submission_record|
     submission_record.copy_to_another_team(team)
   end
   responses = Response.where(map_id: id)
   responses.each do |response|
     response.copy_to_another_response_map(new_response_map)
   end
   new_response_map
 end

Added Assignment#copy_calibrated_submissions(new_assignment)

This method is the key method in our solution to copying calibrated submissions for an assignment. The method creates a participant for the instructor to allow them to be reviewers and processes each calibration response map object.

 def copy_calibration_submissions(new_assignment)
   # Create Participant entry for instructors to allow them to become reviewers
   instructor_participant = Participant.where(parent_id: id, user_id: instructor_id).first
   new_instructor_participant = instructor_participant.copy_to_another_assignment(new_assignment)
   response_maps = ResponseMap.where(reviewed_object_id: id, calibrate_to: 1)
   response_maps.each do |response_map|
     # Create a duplicate Team object from the Team associated with the ResponseMap object to add teams who's
     # submissions were used for calibration in the previous Assignments
     team = Team.where(id: response_map.reviewee_id).first
     new_team = team.copy_to_another_assignment(new_assignment)
     response_map.copy_to_another_assignment(new_assignment, new_team, new_instructor_participant)
   end

Finally the call to Assignment#copy_calibrated_submissions was added in AssignmentForm::copy.

   # Copies all calibration submissions along with Participants, Teams, TeamUsers, SubmissionRecords and Responses
   # of the old assignment on to the new Assignment
   old_assign.copy_calibration_submissions(new_assign)

Screenshots of Calibration being copied

To be copied:

Copied Assignment:

Copied Calibration responses:

Test Plan

Testing Goals

 The main goal of our testing is to ensure that when a new calibration assignment is added, all of the qualifying previous calibration submissions 
 and reviews are successfully copied over without overwriting any existing submissions.

Automated Testing using Rspec

Context

When the original assignment to be copied is a calibrated assignment

Behaviors to Verify

For the copied assignment, we need to verify if:

1. All the submission records are copied from the old assignment to the new assignment.

2. Extra participants are copied.

3. All review responses are copied.

Manual UI Testing

This feature is accessible only by an instructor.

1. Log in as an instructor.

2. Go to Assignments and select an existing assignment.

3. Make sure that the assignment has a calibration tab since this feature handles only calibration assignments.

4. Click on the copy option next to the assignment name and this creates a copy of the assignment and a number is appended to the name in order to name the new copied assignment.

5. After the successful implementation of the feature, you should be able to click on the calibration tab of the copied assignment and this should enlist all the participants, submissions, and reviews.

6. This shows that the feature of copying submissions from old calibration assignments to new calibration assignments was successful.

Team Information

Mentor

Dr. Ed Gehringer (efg@ncsu.edu)

Team Members

Pradyumna Khawas (ppkhawas@ncsu.edu)
Abhimanyu Bellam (abellam2@ncsu.edu)
Vishnu Vinod Erapalli (verapal@ncsu.edu)

Resources

Pull Request

Github Repo

Our Video

Previous Submission