CSC/ECE 517 Fall 2020 - E2081. Add a "cake" item type to rubrics

Problem Definition

Expertiza rubrics are utilized to build questionnaires and these rubrics incorporate several kinds of items, including Criterion (dropdown + comment), Checkbox, MultipleChoice, and Scale. When we use these questionnaires for reviews, for example the teammate review assessment we encounter a few problems. One “problem” with all of these types is that there is nothing to stop a reviewer (say some student) from assigning the maximum score to all the reviewees (student's teammates). This is indeed a problem for teammate assessment, when the faculty asks for what fraction of the work each teammate did.So an alternative is needed, let’s call it a “Cake” item type, that allows a reviewer to divide a “cake” in any way between the reviewees, but does not allow him/her to divvy up more than 100% of the cake.

1. When the reviewer submits a score that would bring the total assigned for this item to > 100%, the system needs to warn.
2. Allow a reviewer to give him/herself a score
3. Proposed design needs to be compatible with existing self reviews code and teammates reviews
4. Should be extensible to other kind of reviews apart from teammate reviews as well

For instance, the above figure consists of a team of 4 members, with self included as a team-member when reviewing every member's contribution.
If A has reviewed his other 3 teammates B, C, D with contributions of 15%, 10%, and 45% respectively, he should only be allowed to review self with a contribution of 30% or lesser.

Design

Proposed Solution:

The issue asks us to have a Cake type for the question taking in a participant’s contribution, whenever s/he is reviewing the other teammates. We add in a new Question type ‘Cake’, which will be extended from the Scored Question model [cake < scored question < choice question < question].

Potential ways of displaying the Cake question on the UI:

• Stars: Existing design for teammate reviews uses stars to symbolize the contribution provided by each student. We can implement the cake type using the same.

Cons: Stars are not very versatile when there are a greater number of students per team and if the student wants to equally rate their contribution.

• Drop Down: In order to give the student more flexibility, another way a student can pick the contribution of each team member is using a dropdown of the % values.

Cons: Drop down values need to be restricted to intervals of 5 or more, as the drop down becomes too long to display all values from 0-100.

• Text box with up-down arrows: Provides utmost flexibility and precision to the student while adding contribution of his team members, we can provide a text box with necessary validations which lets the student provide the contribution % for his teammates as any integral number within the limits.

We decided to implement the text box, looks as follows:

Addressing self-reviews for Cake Type:

The amount of credit assigned to self will be the remaining amount of percentage not given to the other teammates for the reviews. This will eliminate the need to do a self-review to specifically assign the leftover percentage to oneself. The self percentages for the cake questions will only be assigned to a student once they have completed the reviews for all other teammates. This will ensure that the students are motivated to complete the teammate reviews and the self-review calculation is executed once (instead of needing to re-calculate after each review).

Steps to reproduce the proposed workflow:

1. Log in to expertiza to view the home page
2. Login as an Instructor, and then impersonate a student or login as a Student
3. Go to Assignments -> Your team
4. You will see a list of your teammates with a link: ‘Review’
5. You will see yourself with no present link, this will become visible once you complete all reviews as: 'View'
6. You can see the questions for asking the contribution as a cake type
7. There will be a text description next to it denoting what part of the cake is taken (what contribution factor of the work is used)
8. For yourself the view will automatically be filled out with the leftover percentages for each cake contribution

Team Members

• Jordan, Dylan Tyler dtjordan@ncsu.edu

• Ruoyun Ma rma9@ncsu.edu

• Han, Lige lhan6@ncsu.edu

• Kollipara, Siddhartha skollip@ncsu.edu

• Mentor: Ed Gehringer (efg@ncsu.edu)

References

• CSC/ECE 517 Fall 2019 - E1992. Add cake type to rubrics design
• Expertiza on GitHub
• The live Expertiza website
• Expertiza project Details
• Expertiza YouTube Channel