CSC/ECE 517 Fall 2020 - E2081. Add a "cake" item type to rubrics
Problem Definition
Expertiza rubrics are utilized to build questionnaires and these rubrics incorporate several kinds of items, including Criterion (dropdown + comment), Checkbox, MultipleChoice, and Scale. When we use these questionnaires for reviews, for example the teammate review assessment we encounter a few problems. One “problem” with all of these types is that there is nothing to stop a reviewer (say some student) from assigning the maximum score to all the reviewees (student's teammates). This is indeed a problem for teammate assessment, when the faculty asks for what fraction of the work each teammate did.So an alternative is needed, let’s call it a “Cake” item type, that allows a reviewer to divide a “cake” in any way between the reviewees, but does not allow him/her to divvy up more than 100% of the cake.
- When the reviewer submits a score that would bring the total assigned for this item to > 100%, the system needs to warn.
- The system must a reviewer to give him/herself a score
- The proposed design needs to be compatible with existing self reviews code and teammates reviews
- Design should be extensible to other kind of reviews apart from teammate reviews as well
For instance, the above figure consists of a team of 4 members, with self included as a team-member when reviewing every member's contribution.
If A has reviewed his other 3 teammates B, C, D with contributions of 15%, 10%, and 45% respectively, he should only be allowed to review self with a contribution of 30% or lesser.
Design
Proposed Solution
The issue asks us to have a Cake type for the question taking in a participant’s contribution, whenever s/he is reviewing the other teammates. We add in a new Question type ‘Cake’, which will be extended from the Scored Question model [cake < scored question < choice question < question].
Potential ways of displaying the Cake question on the UI:
- Stars: Existing design for teammate reviews uses stars to symbolize the contribution provided by each student. We can implement the cake type using the same.
Cons: Stars are not very versatile when there are a greater number of students per team and if the student wants to equally rate their contribution.
- Drop Down: In order to give the student more flexibility, another way a student can pick the contribution of each team member is using a dropdown of the % values.
Cons: Drop down values need to be restricted to intervals of 5 or more, as the drop down becomes too long to display all values from 0-100.
- Text box with up-down arrows: Provides utmost flexibility and precision to the student while adding contribution of his team members, we can provide a text box with necessary validations which lets the student provide the contribution % for his teammates as any integral number within the limits. There will also be text to warn the user about how much contribution is remaining based on what is typed into the box and any previously completed teammate reviews.
We plan to implement the text box as follows:
Addressing self-reviews for Cake Type:
The amount of credit assigned to self will be the remaining amount of percentage not given to the other teammates for the reviews. This will eliminate the need to do a self-review to specifically assign the leftover percentage to oneself. The self percentages for the cake questions will only be assigned to a student once they have completed the reviews for all other teammates. This will ensure that the students are motivated to complete the teammate reviews and the self-review calculation is executed once (instead of needing to re-calculate after each review).
In the current system workflow, we found that the teammate reviews are taken as instances of TeammateReviewResponseMap. The questionnaire for reviewing teammates includes the question asking the cake (contribution) factor. Ideally, the cake should include the reviewer as well.
Currently, whenever a participant is reviewing their teammates they are shown a view where the names of their teammates are displayed with a 'Review' link next to them. The last line has the user name displayed with no link but reminder text to complete their reviews. We plan on updating this view once the reviews are complete for all teammates with a 'View' link in the user's line item that will show all of the self cake contributions.
Steps to reproduce the proposed workflow:
- Log in to expertiza to view the home page
- Login as an Instructor, and then impersonate a student or login as a Student
- Go to Assignments -> Your team
- You will see a list of your teammates with a link: ‘Review’
- You will see yourself with no present link, this will become visible once you complete all reviews as: 'View'
- You can see the questions for asking the contribution as a cake type
- There will be a text description next to it denoting what part of the cake is taken (what contribution factor of the work is used)
- For yourself the view will automatically be filled out with the leftover percentages for each cake contribution
Why we chose this approach
We found that it was efficient calculate a self contribution as the remaining percentage not assigned to all other teammates. This eliminates the need to create a specialized self-review to assign a contribution to oneself. We decided that it would be best to allow access to the self review once all other teammate reviews are complete because it encourages users to do their reviews and the final calculation for self contribution needs to only be done once.
Implementation
Test plan
Deployment
Team Members
- Jordan, Dylan Tyler dtjordan@ncsu.edu
- Ruoyun Ma rma9@ncsu.edu
- Han, Lige lhan6@ncsu.edu
- Kollipara, Siddhartha skollip@ncsu.edu
- Mentor: Ed Gehringer (efg@ncsu.edu)