CSC/ECE 517 Fall 2020 - E2081. Add a "cake" item type to rubrics
Problem Definition
Expertiza rubrics are utilized to build questionnaires and these rubrics incorporate several kinds of items, including Criterion (dropdown + comment), Checkbox, MultipleChoice, and Scale. When we use these questionnaires for reviews, for example the teammate review assessment we encounter a few problems. One “problem” with all of these types is that there is nothing to stop a reviewer (say some student) from assigning the maximum score to all the reviewees (student's teammates). This is indeed a problem for teammate assessment, when the faculty asks for what fraction of the work each teammate did.So an alternative is needed, let’s call it a “Cake” item type, that allows a reviewer to divide a “cake” in any way between the reviewees, but does not allow him/her to divvy up more than 100% of the cake.
- When the reviewer submits a score that would bring the total assigned for this item to > 100%, the system needs to warn.
- Allow a reviewer to give him/herself a score
- Proposed design needs to be compatible with existing self reviews code and teammates reviews
- Should be extensible to other kind of reviews apart from teammate reviews as well
For instance, the above figure consists of a team of 4 members, with self included as a team-member when reviewing every member's contribution.
If A has reviewed his other 3 teammates B, C, D with contributions of 15%, 10%, and 45% respectively, he should only be allowed to review self with a contribution of 30% or lesser.
Design
Proposed Solution:
The issue asks us to have a Cake type for the question taking in a participant’s contribution, whenever s/he is reviewing the other teammates. We add in a new Question type ‘Cake’, which will be extended from the Scored Question model [cake < scored question < choice question < question].
Potential ways of displaying the Cake question on the UI:
- Stars: Existing design for teammate reviews uses stars to symbolize the contribution provided by each student. We can implement the cake type using the same.
Cons: Stars are not very versatile when there are a greater number of students per team and if the student wants to equally rate their contribution.
- Drop Down: In order to give the student more flexibility, another way a student can pick the contribution of each team member is using a dropdown of the % values.
Cons: Drop down values need to be restricted to intervals of 5 or more, as the drop down becomes too long to display all values from 0-100.
- Text box with up-down arrows: Provides utmost flexibility and precision to the student while adding contribution of his team members, we can provide a text box with necessary validations which lets the student provide the contribution % for his teammates as any integral number within the limits.
We decided to implement the text box, looks as follows:
Addressing self-reviews for Cake Type:
The amount of credit assigned to self will be the remaining amount of percentage not given to the other teammates for the reviews. This will eliminate the need to do a self-review to specifically assign the leftover percentage to oneself. The self percentages for the cake questions will only be assigned to a student once they have completed the reviews for all other teammates. This will ensure that the students are motivated to complete the teammate reviews and the self-review calculation is executed once (instead of needing to re-calculate after each review).
Steps to reproduce the proposed workflow:
- Log in to expertiza to view the home page
- Login as an Instructor, and then impersonate a student or login as a Student
- Go to Assignments -> Your team
- You will see a list of your teammates with a link: ‘Review’
- You will see yourself with no present link, this will become visible once you complete all reviews as: 'View'
- You can see the questions for asking the contribution as a cake type
- There will be a text description next to it denoting what part of the cake is taken (what contribution factor of the work is used)
- For yourself the view will automatically be filled out with the leftover percentages for each cake contribution
Implementation
Test plan
Deployment
Team Members
- Jordan, Dylan Tyler dtjordan@ncsu.edu
- Ruoyun Ma rma9@ncsu.edu
- Han, Lige lhan6@ncsu.edu
- Kollipara, Siddhartha skollip@ncsu.edu
- Mentor: Ed Gehringer (efg@ncsu.edu)