CSC/ECE 517 Fall 2022 - E2284. Calibration submissions should be copied along with calibration assignments

Introduction

A “calibration assignment” is an assignment where the students are asked to review work that has also been reviewed by a member of the course staff. If the student’s review “resembles” the staff member’s review, then the student is presumed to be a competent reviewer. Here is a further description of calibration assignments. The instructor (or TA) adds a few extra participants to the assignment to set up calibration. The instructor (or TA) then impersonates the extra participants and submits work on behalf of each of the extra participants.

Problem Statement

Having the instructor (or TA) impersonate the extra participants, and submit work on behalf of each of the extra participants is extra trouble. It would be nice if an instructor didn’t have to resubmit the same calibration submissions every semester. Copying the extra participants along with their teams, submissions, and responses when copying an assignment makes things much easier.

Previous Submission

What was Done

They created a new class method that checks if the an assignment was calibrated and if it was, it calls methods from models. It performs the following:

 1. They copy all submission records with the assignment_id matching the original assignment. 
 2. Copy all participants with the parent_id matching the original assignment ID.    
 3. Copy all the teams with a parent_id matching the original assignment ID.  
 4. Recreated mappings needed to associate participants with teams and review_response maps.
   4.1. Their copy methods for teams and participants return hash maps that are used to lookup previous mappings.
     4.1.1. Create new TeamsUsers.  Associates participants with teams.  
     4.1.2. Create new ReviewResponseMap.  Associates reviewers and reviewees, linked to an assignment, together.  
 5. Copy all review_responses, using the original review response map as the reference.  

Their 'copy_calibrated_reviews' is called in the existing assignment_form class method 'copy', if the copied assignment is able to be saved (to avoid creating new records).

They also added methods for copy naming schemes so that the assignment model validates uniqueness of both the assignment name and directory_path.

Issues

Issue 1: To set up calibration, the instructor (or TA) adds a few extra participants to the assignment. The instructor (or TA) then impersonates the extra participants, and submits work on behalf of each of the extra participants. -

 This is extra trouble. An instructor needn't have to resubmit the same calibration submissions every semester.

Issue 2: Not all uses of class methods are good.

 self.copy_participants_for_assignment: copies old participants to new --> assignment_participant.rb, it is in teams_users.rb, which seems misplaced.

Issue 3: Returns a mapping from the old response map to the new response map.

 self.copy_review_response_map, returns a mapping from the old response map to the new response map. It is in review_response_map.rb. This can be an instance method. There is a copy_review_responses in this class too. 

Issue 4: Code missing without causing a bug

 Line 65 of submit_hyperlink has missing code.

Issue 5: Old code not reused.

 New code was written to copy a team in teams_user.rb. There already exists code to copy a team.

Issue 6: No Automated tests written

Files Changed

assignment_form.rb

 1. Added a new method "copy_name" to name copied assignments. Where they changed the name of the copied assignments to “copy of copy” and "copy of copy of copy", etc to "Copy of <name> <copy number>"

participant.rb:

response.rb:

response_map.rb:

submission_record.rb:

team.rb:

teams_user.rb:

assignment_participant.rb:

Design

Changed Files

1. app/assets/javascripts/tree_display.jsx

2. app/controllers/assessment360_controller.rb

3. app/controllers/review_mapping_controller.rb

4. app/models/assignment_participant.rb

5. app/models/metareview_questionnaire.rb

6. app/models/vm_question_response.rb

7. app/views/assessment360/all_students_all_reviews.html.erb

8. app/views/assessment360/index.html.erb

9. config/routes.rb

10. lib/average.rb

11. spec/controllers/assessment360_controller_spec.rb

12. spec/controllers/review_mapping_controller_spec.rb

13. spec/models/assignment_participant_spec.rb

14. spec/models/student_task_spec.rb

15. spec/models/vm_question_response_spec.rb

Test Plan

Testing Goals

 The main goal of our testing is to ensure that when a new calibration assignment is added, all of the qualifying previous calibration submissions 
 and reviews are successfully copied over without overwriting any existing submissions.

Team Information

Mentor

Dr. Ed Gehringer (efg@ncsu.edu)

Team Members

Pradyumna Khawas (ppkhawas@ncsu.edu)
Abhimanyu Bellam (abellam2@ncsu.edu)
Vishnu Vinod Erapalli (verapal@ncsu.edu)