CSC/ECE 517 Fall 2017/E17A4 Allow calibration to be part of an assignment Team34

Introduction

1. Purpose

Expertiza is an open source website based on Ruby on Rails. Calibration is one of the main feature Expertiza provides, which allows students to compare their reviews with experts' reviews and learn from the comparisons.

2. Problem Definition

Currently, the instructor is able to calibrate the students’ peer review abilities on Expertiza. However, the instructor needs to create a separate assignment just for calibration purpose of a project . Then the instructor and students can start doing peer reviews for the project with the newly created assignment . However, calibration should become one part of the normal assignment. The instructor could turn on this feature if necessary. Adding a separate assignment every time a project needs a calibration process is not efficient.

This project proposes a solution to include the process of calibration in a regular assignment.

Overall Description

“Calibration for training?” feature of an assignment in Expertiza provides a solution for the Instructor of a course to effectively add a calibration to a newly created or existing assignment. The instructor should be able to turn on this feature by checking the check box "Calibration for training?" under the General tab of an assignment. The instructor will also be able to provide or change the corresponding deadline for calibration. Then students will be pre-assigned several sample assignments. After students finish one peer review , there will be a link named “Show calibration results” to show the calibration report.

Picture above is "Calibration for training?" box in editing/creating assignment page

Requirements

1. The instructor shall be able to add calibration process to an newly created or existing assignment.

2. In student_task/list page, current stage column shall display calibration when assignment is in calibration period.

3. The added calibration of an assignment shall have it owns type of due date.

4. The added calibration of an assignment shall have the same behavior as the current working calibration process.

Use Cases

1. Edit/Create Assignment

Use Case Id: 1

Use Case Description: Instructor can click to edit an existing assignment or create a new assignment.

Actors: Instructor

Pre Conditions: The user is logged in as an instructor.

Post Conditions: Instructor can edit the assignment.

2. Enable Calibration

Use Case Id: 2

Use Case Description: Instructor can select the option - "Calibration for training?" and save the assignment.

Actors: Instructor

Pre Conditions: Instructor is in the editing or creating assignment page.

Post Conditions: The assignment becomes a calibration assignment and the deadline for calibration now appears in the due dates tab.

3. Add calibration due date

Use Case Id: 3

Use Case Description: Instructor can set due date for calibration under due dates tab.

Actors: Instructor

Pre Conditions: Calibration for training? is enabled.

Post Conditions: The deadline for calibration is now updated.

Constraints: Calibration deadline should be before the first round submission deadline.

4. Expert Review

Use Case Id: 5

Use Case Description: Instructor can do expert reviews for works.

Actors: Instructor

Pre Conditions: The assignment is a calibration assignment and the work is submitted.

Post Conditions: The work has the expert review.

5. Calibration review

Use Case Id: 6

Use Case Description: Students can do calibration review.

Actors: Student

Pre Conditions: The assignment is a calibration assignment and the current stage is in calibration review.

Post Conditions: Students have done the calibration review.

6. View calibration result

Use Case Id: 7

Use Case Description: Student can see the comparison between their reviews and the experts' reviews.

Actors: Student

Pre Conditions: Instructors finished the expert reviews and this student submitted his reviews.

Post Conditions: NA

7. Submit work

Use Case Id: 4

Use Case Description: Students can submit their works when the current stage is "submit".

Actors: Student

Pre Conditions: Current stage is "submit".

Post Conditions: The database has the new works.

Test-Driven Development(TDD)

A software development process begins by writing an (initially failing) automated test case that defines a desired improvement or a new function, and generate the minimum amount of code to pass that test. It helps the developer to focus on a smaller portions of functionality at a time. By following the process of TDD we can ensure to maintain the minimum amount of code while writing that specific function, and the developer can gain an overview of the function.

What did we implement

1. We created a file "calibration_spec.rb" in "spec/ features" and write feature tests for the calibration function.

2. We created a migration file "222222222222222_add_new_deadline_type.rb" to add a new record to deadline_types model

class AddNewDeadlineType < ActiveRecord::Migration
  def self.up
    execute "INSERT INTO `deadline_types` VALUES (12,'calibration_review');"
  end
  def self.down
    execute "DELETE FROM `deadline_types` WHERE `name` = 'calibration_review';"
  end
end

3. We added a calibration due date in due date tab if it's a calibration assignment (Modified file: app/views/assignments/edit/_due_dates.html.erb)

 <% if @assignment_form.assignment.is_calibrated %>
           addDueDateTableElement(<%= @due_date_nameurl_notempty==nil ? false:@due_date_nameurl_notempty %>,'calibration', 
                                                 0, <%= due_date(@assignment_form.assignment, 'calibration_review').to_json.html_safe %>);
 <%end %>

4. We added calibration_review in deadline_right file (Modified file: app/models/deadline_right.rb)

 'calibration_review' => {
        'submission_allowed' => NO,
        'can_review' => OK,
        'review_of_review_allowed' => NO
  }

5. We changed the function of finding next due date from find_by to where().order('due_at').first (Modified file: app/models/due_date.rb)

next_due_date = AssignmentDueDate.where(['parent_id = ? && due_at >= ?', assignment_id, Time.zone.now]).order('due_at').first

6. We changed "app/controllers/student_review_controller.rb" file to exclude calibration reviews when counting the review number. We filtered the reviews with timestamp earlier than submission due date, which means reviews are done in calibration period.

   @regular_review_mappings = []
   @calibration_review_mappings = []
   @participant = AssignmentParticipant.find(params[:id])
   return unless current_user_id?(@participant.user_id)
   @assignment = @participant.assignment
   # Find the current phase that the assignment is in.
   @topic_id = SignedUpTeam.topic_id(@participant.parent_id, @participant.user_id)
   @review_phase = @assignment.get_current_stage(@topic_id)
   # ACS Removed the if condition(and corressponding else) which differentiate assignments as team and individual assignments
   # to treat all assignments as team assignments
   @review_mappings = ReviewResponseMap.where(reviewer_id: @participant.id)
   
   # if it is an calibrated assignment, change the response_map order in a certain way
   @review_mappings = @review_mappings.sort_by {|mapping| mapping.id % 5} if @assignment.is_calibrated == true
   @metareview_mappings = MetareviewResponseMap.where(reviewer_id: @participant.id)
   
   # Calculate the number of reviews that the user has completed so far.
   @calibration_deadline = @assignment.due_dates.find_by_deadline_type_id(12)
   @num_reviews_total = @review_mappings.size
   
   # Add the reviews which are requested and not began.
   @num_regular_reviews_completed = 0
   @num_calibration_reviews_completed = 0
   @num_regular_reviews_total = 0
   @num_calibration_reviews_total = 0
   assignment_submission_due_date = @assignment.due_dates.select {|due_date| due_date.deadline_type_id == 1}.first.due_at
   @review_mappings.each do |map|

     next if map.response.empty?
     if map.response.last.updated_at < assignment_submission_due_date
       @num_calibration_reviews_completed += 1 if map.response.last.is_submitted
     else
       # @regular_review_mappings.push(map)
       @num_regular_reviews_completed += 1 if map.response.last.is_submitted
     end
   end
   @review_mappings.each do |map|
     if map.updated_at < assignment_submission_due_date
       @calibration_review_mappings.push(map)
       @num_calibration_reviews_total += 1
     else
       @regular_review_mappings.push(map)
       @num_regular_reviews_total += 1
     end
   end

   @num_calibration_reviews_in_progress = @num_calibration_reviews_total - @num_calibration_reviews_completed
       @num_regular_reviews_in_progress = @num_regular_reviews_total - @num_regular_reviews_completed

   # Calculate the number of metareviews that the user has completed so far.
   @num_metareviews_total = @metareview_mappings.size
   @num_metareviews_completed = 0
   @metareview_mappings.each do |map|
       @num_metareviews_completed += 1 unless map.response.empty?
   end
   @num_metareviews_in_progress = @num_metareviews_total - @num_metareviews_completed
   @topic_id = SignedUpTeam.topic_id(@assignment.id, @participant.user_id)

7.

Test Plan

1. When it is in assignments#edit page:

• To check it has a checkbox with title "Calibration for training?" in assignments#edit page

2. When it is in assignments#edit page and clicking "Calibration for training?" checkbox and clicking "save" button:

• To check it displays a new tab named "Calibration" and adds a calibration due date in "Due dates" tab
• To check it allows instructors to change and save date & time and permissions of calibration due date

3.When current assignment is calibration stage:

• To check it shows current stage of the assignment to be "Calibration" on student_task#view page when current assignment is in calibration stage
• To check it shows "Calibration review 1,2,3..." instead of "Review 1,2,3..." on student_review#list page
• To check it allows students to do calibration review and the date can be saved successfully
• To check the student is able to compare the results of expert review by clicking "show calibration results" link

4. When current assignment if in review_stage:

• To check it excludes calibration reviews from outstanding review restriction and total review restriction
• To check it shows "Review 1,2,3..." instead of "Calibration review 1,2,3..." on student_review#list page