CSC/ECE 517 Spring 2022 - E2234. Calibration submissions should be copied along with calibration assignments

Introduction

A “calibration assignment” is an assignment where the students are asked to review work that has also been reviewed by a member of the course staff. If the student’s review “resembles” the staff-member’s review, then the student is presumed to be a competent reviewer. Here is a further description of calibration assignments. To set up calibration, the instructor (or TA) adds a few extra participants to the assignment. The instructor (or TA) then impersonates the extra participants, and submits work on behalf of each of the extra participants.

Problem Statement

Having the instructor (or TA) impersonate the extra participants, and submit work on behalf of each of the extra participants is extra trouble. It would be nice if an instructor didn’t have to resubmit the same calibration submissions every semester. Copying the extra participants along with their teams, submissions, and responses when copying an assignment makes things much easier.

Previous Implementation

The previous project did fully implement code that copies over the calibration submissions whenever a calibrated assignment is submitted. It was done by ensuring that when a assignment is copied that 1) the same extra participants as the assignment you are copying are used 2) the URLs and/or files that were submitted to the previous assignment are also copied and 3) the URLs and files are submitted to the assignment as well

Shown below is a diagram of how the current system works (copying the extra participants/reviews):

The previous implementation needs to be revisited and made more elegant. A description of the needed refactoring and changes is below.

What needs to be done

• Ensure what gets copied (when and how) is done properly
  • Reviews done by the creator of the assignment are copied, but only reviews that have been started --> how do you recognize a calibration review?: Can check 'calibrate_to' field in response maps where 'reviewed_object_id' equals the assignment ID being copied
  • If an author did any non-calibration reviews in the assignment being copied, those reviews would also be treated as calibration reviews
  • Submitted files are copied too, but if the calibration submissions are done as part of the COPYING of the assignment, they will necessarily use directories 0, 1,
• Handling the case of students submitting assignments before calibration assignments are done
  • Project assumes that calibrations start at directory 0
  • Not the case if any students submit before the calibration submissions are done
• Submitter_count needs to be set to the largest directory number that was used for calibration
  • First student to submit will have their submission show up in the same directory with the first calibration submission
• Several class methods in assignment_form.rb should be moved to different classes as instance methods in those classes
  • Add better comments - eg. a comment for a method that copies an assignment should not reveal details about calibrations. Currently, only 1/4 methods in this class have comments.
  • Currently, for assignment_form.rb we change the name of the copied assignments to “copy of copy” and "copy of copy of copy" etc. We will improve this by making it "Copy of <name> <copy number>" and putting this function in a new method.

  • 

    
    

• Clean up participant.rb
  • Look for a more elegant way of (self.create_participant) copying/creating of participants → DRY violation
  • Write the create method in assignment_participant instead of directly in participant.rb
• Clean up team.rb
  • Create a copy method in team that copies the field used by all teams
  • Rest of the stuff is just used by assignment teams (submitted hyperlinks etc.)
  • Create a copy method and override in assignment_team
• Reevaluate instance variables. Are these needed, or could they be local variables instead?
• Look at editing out 9min video due to long waits for responses during copying
• Verify our work using not only user 'Instructor6' but also Professor Jennifer Kidd's account - 'Instructor 2026'

Design

Files Requiring Modification

• assignment_form.rb
  • Intead of setting the name of the copied assignments to “copy of copy” and "copy of copy of copy" etc, We will change it to name "Copy of <name> <copy number>" and put this function in a new method.
    This code currently resides in the self.copy method (which is too long and does too much for one method), so we will move it to a new method called copy_name.

  • 

    
    
  • Implement new method for copying objects needed to recreate calibration reviews

• participant.rb
  • Write the create method in assignment_participant instead of directly in participant.rb
  • Here is the current create method in participant.rb:

  • 

    
    

• response.rb
  • Implement method for copying previous responses

• response_map.rb
  • Implement method that creates ReviewResponse mapping, based on mapping of copied assignment

• submission_record.rb
  • Implement method for copying submission records of an assignment

• team.rb
  • Create a copy method in team.rb that copies all teams of a previous assignment

• teams_user.rb
  • Create method for creating new teams_users so that new participants and teams are associated the same way they were for the previous assignment

• assignment_participant.rb
  • Method for copying extra participants should be here, we wouldn't need to copy participants with type 'CourseParticipant'

Test Plan

Testing Goals

 The main goal of our testing is to ensure that when a new calibration assignment is added, all of the qualifying previous calibration submissions 
 and reviews are successfully copied over without overwriting any existing submissions.

Automated Unit Tests

Using RSpec, we have to test that the following unit tests pass:

 1. When the TA adds a calibration assignment, extra participants are added to the assignment.

 2. When extra participants are added to an assignment, calibration submissions are copied over.

 3. When calibration submissions are copied over, the submitter_count value increases by the number of calibration submissions, so that previous submissions are 
    not overwritten.

 4. After calibration assignments are copied over, only calibration reviews from the creator of the assignment are copied over.

 5. After calibration assignments are copied over, only calibration reviews that have been started are copied over.

 6. Calling specific methods in assignment.rb return a NoMethodError to indicate that these methods are no longer class methods and have been moved elsewhere as 
    instance methods.

Team Information

Mentor: Dr. Ed Gehringer (efg@ncsu.edu)
Nolan Dowdle (njdowdle@ncsu.edu)
Laura Fox (lmcampb5@ncsu.edu)
Soeun Jo (sjo@ncsu.edu)
Harshil Shah (hshah6@ncsu.edu)

Resources

A link to our pull request can be found here
A link to the Expertiza GitHub can be found here
A link to our repository can be found here
Our implementation video can be found here
A link to the previous Wiki can be found here
The previous implementation video can be found here
A link to the previous GitHub pull request can be found here