CSC/ECE 517 Fall 2021 - E2155. Calibration submissions should be copied along with calibration assignments

Project Overview

Background

To improve the experience of peer-reviewing in Expertiza and determine the competent reviewers amongst all the reviewers, creating a new form of review for assignments was considered. As a result, "calibration assignments" were introduced to the Expertiza system. The calibration assignments were to be assigned before peer review occurred so that students would know if they reviewed the same way that the instructor had reviewed the assignment. Calibration assignments show whether or not the student had given the same or similar score as the instructor for rubric items.

To set up a calibration assignment that both the instructor and the course students can review, the instructor needs to create a new assignment, add the extra participant manually, impersonate the participant, and finally, submit the review, which is not a great experience. What's more is that currently, if an instructor wishes to do this every semester or every year with the same course to copy over the calibration assignments, they have to repeat this process yet again.

Objectives

Expertiza can fix the problem of repeatedly setting up the calibration assignments by automatically copying the previous calibration assignments with all attached reviews.

Previous Implementation

In the previous implementation, the implementors designed the calibration review as part of the assignment and students were able to participate in calibration review training before reviewing other students' work.

The instructor can enable the calibration review in two ways:

1. Check "Calibration for training?" in the general tab of an assignment then copy the participants from the course.

2. Copy existing assignment and its participants with calibration review enabled.

Detailed System Overview can be found at the following:
CSC/ECE 517 Fall 2017/E17A4 Allow calibration to be part of an assignment

However, their implementation introduced several assumptions about the system that we will address to allow merging into production.

Issues with the Previous work

1. The project assumes that calibration submissions starts at directory 0, which is only correct if the users who submit the calibration submissions are the first to be added as participants to the assignment.
   1. Suggested Fix: set submitter_count to the largest directory number that Expertiza used for calibration.
      1. Allows us to keep track of the most recent calibration submission.
      2. Gives us something we can reference during copying to get the correct directory.
2. The due date of calibration review are missing.
3. The copied assignments do not have participants copied.

Design

1. TODO: add UML diagram, preliminary class design

Testing Plan

1. TODO: test if the assignment and its participants were copied correctly, test if the participants can be added to the calibration review, test if the due date of calibration review was saved successfully

Testing scenarios:

1. The instructor copies the existing assignment
   The copied assignment stored in the correct directory
   Expect the number of calibration submitters of the copied assignment=the number of participants of the original assignment
2. The instructor enables the calibration review of an assignment
   The calibration tab appears next to the due dates tab
3. The student attempt his/her calibration review
   The student submits the calibration review
   The webpage redirects to the instructor's calibration review

Links

Github repository

Team

1. John Marsden (ProfDoof)
2. Lee Shyu (ls-hyu)
3. Angela Ho (angelaho0504)
4. Kasimir Schulz (Kasimir123)