CSC/ECE 517 Fall 2021 - E2155. Calibration submissions should be copied along with calibration assignments: Difference between revisions

From Expertiza_Wiki
Jump to navigation Jump to search
 
(33 intermediate revisions by 4 users not shown)
Line 1: Line 1:
==Project Overview==
==Project Overview==
===Introduction===
===Background===
[https://expertiza.ncsu.edu/ Expertiza] is an open-source education and classroom web-tool founded by the [http://www.nsf.gov/ National Science Foundation]. Built with Ruby on Rails, it is designed to manage complete courses and students’ work within those courses. Each course can have a collection of instructors and students, though the interaction between instructors and students is minimal. The real emphasis of Expertiza is placed on peer-to-peer interactions, fostering a student-driven learning environment. Courses are comprised of assignments which users complete individually or with a team. Assignments usually encourage or require a team to enforce practicing peer-to-peer interaction.
To improve the experience of peer-reviewing in Expertiza and determine the competent reviewers amongst all the reviewers, creating a new form of review for assignments was considered. As a result, "calibration assignments" were introduced to the Expertiza system. The calibration assignments were to be assigned before peer review occurred so that students would know if they reviewed the same way that the instructor had reviewed the assignment. Calibration assignments show whether or not the student had given the same or similar score as the instructor for rubric items.


===Background===
To set up a calibration assignment that both the instructor and the course students can review, the instructor needs to create a new assignment, add the extra participant manually, impersonate the participant, and finally, submit the review, which is not a great experience. What's more is that currently, if an instructor wishes to do this every semester or every year with the same course to copy over the calibration assignments, they have to repeat this process yet again.
To improve the experience of peer reviewing and to find out the competent reviewers among all the reviewers, designing a new form of review for assignments seems to be a fix. As a result, "calibration assignment" was introduced to the Expertiza system. The calibration assignment should be assigned before the peer review so that the students can know if they reviewed using the same rubrics as the instructor.
To set up the calibration assignment that allows both the instructor and the students of the course to review, the instructor needs to create a new assignment and add the extra participant manually, impersonate the participant then submit the review, which is not a great experience.


===Objectives===
===Objectives===
To fix the issue of setting up the calibration assignment repeatedly every semester, the system should be able to make a copy of the previous calibration assignment and let the instructor submit a review as the extra participant by selecting calibration training when the instructor creates the assignment.
Expertiza can fix the problem of repeatedly setting up the calibration assignments by automatically copying the previous calibration assignments with all attached reviews.


===Previous Implementation===
===Previous Implementation===
In the previous implementation, the calibration review was designed as part of the assignment and students should be able to participate in calibration review training before reviewing other students' work.
In the previous implementation, the implementors designed the calibration review as part of the assignment and students were able to participate in calibration review training before reviewing other students' work.
 
The instructor can enable the calibration review in two ways:
 
====1. Check "Calibration for training?" in the general tab of an assignment then copy the participants from the course.====
 
[[File:2021-11-03 011751.png]]
 
====2. Copy existing assignment and its participants with calibration review enabled.====
 


Detailed System Overview can be found at the following: <br>
[https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2017/E17A4_Allow_calibration_to_be_part_of_an_assignment CSC/ECE 517 Fall 2017/E17A4 Allow calibration to be part of an assignment]
[https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Fall_2017/E17A4_Allow_calibration_to_be_part_of_an_assignment CSC/ECE 517 Fall 2017/E17A4 Allow calibration to be part of an assignment]


===Issues of Previous work===
However, their implementation introduced several assumptions about the system that we will address to allow merging into production.
#If any student submits a review before completing the calibration review training, the first review submitted will be stored at directory 0, where the calibration review was supposed to be stored.
 
#
===Issues with the Previous work===
#The project assumes that calibration submissions starts at directory 0, which is only correct if the users who submit the calibration submissions are the first to be added as participants to the assignment.
##Suggested Fix: set submitter_count to the largest directory number that Expertiza used for calibration.
###Allows us to keep track of the most recent calibration submission.
###Gives us something we can reference during copying to get the correct directory.
#The due date of calibration review are missing. [[File:2021-11-03 235457.png]]
#The copied assignments do not have participants copied. [[File:2021-11-04 013403.png]]


==Design==
==Design==
#TODO: add UML diagram, preliminary class design
[[File:2021-11-04 230435.png]]
==Implementation==
===Files we originally planned to modify===
#app/controllers/student_review_controller.rb<br/>The mappings of reviews or calibration reviews should be redesigned.
#views/student_review/list.html.erb<br/>The view of the list should be modified so that the calibration review can be displayed correctly.
#views/assignments/edit/_due_dates.html.erb<br/>
#app/models/deadline_right.rb
#app/controllers/assignments_controller.rb<br/>The assignments_controller might need a fix so that the assignments with calibration review can be deleted properly.
#app/views/student_review/_responses.html.erb




==Implementation==
===Files we modified===
#app/models/assignment_form.rb
##Removed vary_by_topic_desired from the update function.
##Removed topic references from update_assignment_questionnaires.
##Updated Questionnaire to AssignmentQuestionnaire.
##Added self.copy_calibration function.
##Updated the copy counter.
#app/models/assignment_team.rb
##Removed code for E1973.
##Updated review response map return.
##Created directory_path function.
#app/models/deadline_right.rb
##Added calibration to the model.
#app/models/participant.rb
##Added self.createparticipant function.
#app/models/review_response_map.rb
##Removed E1973 code.
##Removed topic from questionnaire function.
##Shortened self.import function.
##Added self.newreviewresp function.
#app/models/submission_record.rb
##Added self.copycalibratedsubmissions function.
#app/models/team.rb
##Added self.createnewteam function.
#app/views/assignments/edit/_due_dates.html.erb
##Added checkmark for calibration.
#spec/features/assignment_creation_general_tab_spec.rb
##See tests section.
#spec/models/assignment_form_spec.rb
##See tests section.
#db/migrate/20211129124620_add_calibration_to_deadline_type.rb
##We added a new migration for AddCalibrationToDeadlineType.
#Screenshots:
[[File:2021-11-29 235044.png]]
[[File:2021-11-29 235328.png]]
[[File:2021-11-29 235441.png]]
[[File:2021-11-29 235827.png]]
[[File:2021-11-29 235912.png]]
 
==Testing Plan==
 
#Unit test files added:
##spec/models/assignment_form_spec.rb
###Added tests for function 'copy'. Test to make sure that the assignment id is updated on copy and no longer references the original assignment. We also make sure that the name of the new file is correct.
##spec/features/assignment_creation_general_tab_spec.rb
###Added unit test to make sure that the calibration shows in the due dates tab.
 
Testing scenarios:
 
#The instructor copies the existing assignment<br />The copied assignment stored in the correct directory<br />Expect the number of calibration submitters of the copied assignment=the number of participants of the original assignment
#The instructor enables the calibration review of an assignment<br />The calibration tab appears next to the due dates tab
#The student attempts his/her calibration review<br />The student submits the calibration review<br />The webpage shows what the student has submitted as well as a long that says "Show Calibration Results".
 
 
==Errors Found==
 
We found, and fixed, a bug in app/controllers/response_controller.rb where the code was using get_reviewer instead of reviewer. We needed to fix this bug in order for our code to work.
 
==Future Improvements==
 
We believe that submitter_count can be removed from the database since it is never used in the project and in order to update the student directory number we do set_student_directory_num on any of team object within an assignment.
 
==Links==
[https://github.com/Kasimir123/expertiza/tree/beta Github repository]
 
[https://youtu.be/_53Zyyz59q0 Demonstration]
 
==Team==
#John Marsden (ProfDoof)
#Lee Shyu (ls-hyu)
#Angela Ho (angelaho0504)
#Kasimir Schulz (Kasimir123)

Latest revision as of 05:03, 30 November 2021

Project Overview

Background

To improve the experience of peer-reviewing in Expertiza and determine the competent reviewers amongst all the reviewers, creating a new form of review for assignments was considered. As a result, "calibration assignments" were introduced to the Expertiza system. The calibration assignments were to be assigned before peer review occurred so that students would know if they reviewed the same way that the instructor had reviewed the assignment. Calibration assignments show whether or not the student had given the same or similar score as the instructor for rubric items.

To set up a calibration assignment that both the instructor and the course students can review, the instructor needs to create a new assignment, add the extra participant manually, impersonate the participant, and finally, submit the review, which is not a great experience. What's more is that currently, if an instructor wishes to do this every semester or every year with the same course to copy over the calibration assignments, they have to repeat this process yet again.

Objectives

Expertiza can fix the problem of repeatedly setting up the calibration assignments by automatically copying the previous calibration assignments with all attached reviews.

Previous Implementation

In the previous implementation, the implementors designed the calibration review as part of the assignment and students were able to participate in calibration review training before reviewing other students' work.

The instructor can enable the calibration review in two ways:

1. Check "Calibration for training?" in the general tab of an assignment then copy the participants from the course.

2. Copy existing assignment and its participants with calibration review enabled.

Detailed System Overview can be found at the following:
CSC/ECE 517 Fall 2017/E17A4 Allow calibration to be part of an assignment

However, their implementation introduced several assumptions about the system that we will address to allow merging into production.

Issues with the Previous work

  1. The project assumes that calibration submissions starts at directory 0, which is only correct if the users who submit the calibration submissions are the first to be added as participants to the assignment.
    1. Suggested Fix: set submitter_count to the largest directory number that Expertiza used for calibration.
      1. Allows us to keep track of the most recent calibration submission.
      2. Gives us something we can reference during copying to get the correct directory.
  2. The due date of calibration review are missing.
  3. The copied assignments do not have participants copied.

Design

  1. TODO: add UML diagram, preliminary class design

Implementation

Files we originally planned to modify

  1. app/controllers/student_review_controller.rb
    The mappings of reviews or calibration reviews should be redesigned.
  2. views/student_review/list.html.erb
    The view of the list should be modified so that the calibration review can be displayed correctly.
  3. views/assignments/edit/_due_dates.html.erb
  4. app/models/deadline_right.rb
  5. app/controllers/assignments_controller.rb
    The assignments_controller might need a fix so that the assignments with calibration review can be deleted properly.
  6. app/views/student_review/_responses.html.erb


Files we modified

  1. app/models/assignment_form.rb
    1. Removed vary_by_topic_desired from the update function.
    2. Removed topic references from update_assignment_questionnaires.
    3. Updated Questionnaire to AssignmentQuestionnaire.
    4. Added self.copy_calibration function.
    5. Updated the copy counter.
  2. app/models/assignment_team.rb
    1. Removed code for E1973.
    2. Updated review response map return.
    3. Created directory_path function.
  3. app/models/deadline_right.rb
    1. Added calibration to the model.
  4. app/models/participant.rb
    1. Added self.createparticipant function.
  5. app/models/review_response_map.rb
    1. Removed E1973 code.
    2. Removed topic from questionnaire function.
    3. Shortened self.import function.
    4. Added self.newreviewresp function.
  6. app/models/submission_record.rb
    1. Added self.copycalibratedsubmissions function.
  7. app/models/team.rb
    1. Added self.createnewteam function.
  8. app/views/assignments/edit/_due_dates.html.erb
    1. Added checkmark for calibration.
  9. spec/features/assignment_creation_general_tab_spec.rb
    1. See tests section.
  10. spec/models/assignment_form_spec.rb
    1. See tests section.
  11. db/migrate/20211129124620_add_calibration_to_deadline_type.rb
    1. We added a new migration for AddCalibrationToDeadlineType.
  12. Screenshots:

Testing Plan

  1. Unit test files added:
    1. spec/models/assignment_form_spec.rb
      1. Added tests for function 'copy'. Test to make sure that the assignment id is updated on copy and no longer references the original assignment. We also make sure that the name of the new file is correct.
    2. spec/features/assignment_creation_general_tab_spec.rb
      1. Added unit test to make sure that the calibration shows in the due dates tab.

Testing scenarios:

  1. The instructor copies the existing assignment
    The copied assignment stored in the correct directory
    Expect the number of calibration submitters of the copied assignment=the number of participants of the original assignment
  2. The instructor enables the calibration review of an assignment
    The calibration tab appears next to the due dates tab
  3. The student attempts his/her calibration review
    The student submits the calibration review
    The webpage shows what the student has submitted as well as a long that says "Show Calibration Results".


Errors Found

We found, and fixed, a bug in app/controllers/response_controller.rb where the code was using get_reviewer instead of reviewer. We needed to fix this bug in order for our code to work.

Future Improvements

We believe that submitter_count can be removed from the database since it is never used in the project and in order to update the student directory number we do set_student_directory_num on any of team object within an assignment.

Links

Github repository

Demonstration

Team

  1. John Marsden (ProfDoof)
  2. Lee Shyu (ls-hyu)
  3. Angela Ho (angelaho0504)
  4. Kasimir Schulz (Kasimir123)
Retrieved from "https://wiki.expertiza.ncsu.edu/index.php?title=CSC/ECE_517_Fall_2021_-_E2155._Calibration_submissions_should_be_copied_along_with_calibration_assignments&oldid=142368"