CSC/ECE 517 Fall 2020 - E2081. Add a "cake" item type to rubrics
Problem Definition
Expertiza rubrics are utilized to build questionnaires and these rubrics incorporate several kinds of items, including Criterion (dropdown + comment), Checkbox, MultipleChoice, and Scale. When we use these questionnaires for reviews, for example the teammate review assessment we encounter a few problems. One “problem” with all of these types is that there is nothing to stop a reviewer (say some student) from assigning the maximum score to all the reviewees (student's teammates). This is indeed a problem for teammate assessment, when the faculty asks for what fraction of the work each teammate did.So an alternative is needed, let’s call it a “Cake” item type, that allows a reviewer to divide a “cake” in any way between the reviewees, but does not allow him/her to divvy up more than 100% of the cake.
- When the reviewer submits a score that would bring the total assigned for this item to > 100%, the system needs to warn.
- The system must a reviewer to give him/herself a score
- The proposed design needs to be compatible with existing self reviews code and teammates reviews
- Design should be extensible to other kind of reviews apart from teammate reviews as well
For instance, the above figure consists of a team of 4 members, with self included as a team-member when reviewing every member's contribution.
If A has reviewed his other 3 teammates B, C, D with contributions of 15%, 10%, and 45% respectively, he should only be allowed to review self with a contribution of 30% or lesser.
Design
Proposed Solution
The issue asks us to have a Cake type for the question taking in a participant’s contribution, whenever s/he is reviewing the other teammates. We add in a new Question type ‘Cake’, which will be extended from the Scored Question model [cake < scored question < choice question < question].
Potential ways of displaying the Cake question on the UI:
- Stars: Existing design for teammate reviews uses stars to symbolize the contribution provided by each student. We can implement the cake type using the same.
Cons: Stars are not very versatile when there are a greater number of students per team and if the student wants to equally rate their contribution.
- Drop Down: In order to give the student more flexibility, another way a student can pick the contribution of each team member is using a dropdown of the % values.
Cons: Drop down values need to be restricted to intervals of 5 or more, as the drop down becomes too long to display all values from 0-100.
- Text box with up-down arrows: Provides utmost flexibility and precision to the student while adding contribution of his team members, we can provide a text box with necessary validations which lets the student provide the contribution % for his teammates as any integral number within the limits. There will also be text to warn the user about how much contribution is remaining based on what is typed into the box and any previously completed teammate reviews.
We plan to implement the text box as follows:
Addressing self-reviews for Cake Type:
The amount of credit assigned to self will be the remaining amount of percentage not given to the other teammates for the reviews. This will eliminate the need to do a self-review to specifically assign the leftover percentage to oneself. The self percentages for the cake questions will only be assigned to a student once they have completed the reviews for all other teammates. This will ensure that the students are motivated to complete the teammate reviews and the self-review calculation is executed once (instead of needing to re-calculate after each review).
In the current system workflow, we found that the teammate reviews are taken as instances of TeammateReviewResponses. The questionnaire for reviewing teammates includes the question asking the cake (contribution) factor. Ideally, the cake should include the reviewer as well.
Currently, whenever a participant is reviewing their teammates they are shown a view where the names of their teammates are displayed with a 'Review' link next to them. The last line has the user name displayed with no link but reminder text to complete their reviews. We plan on updating this view once the reviews are complete for all teammates with a 'View' link in the user's line item that will show all of the self cake contributions.
Steps to reproduce the proposed workflow:
- Log in to expertiza to view the home page
- Login as an Instructor, and then impersonate a student or login as a Student
- Go to Assignments -> Your team
- You will see a list of your teammates with a link: ‘Review’
- You will see yourself with no present link, this will become visible once you complete all reviews as: 'View'
- You can see the questions for asking the contribution as a cake type
- There will be a text description next to it denoting what part of the cake is taken (what contribution factor of the work is used)
- For yourself the view will automatically be filled out with the leftover percentages for each cake contribution
Why we chose this approach
We found that it was efficient to calculate a self contribution as the remaining percentage not assigned to all other teammates. This eliminates the need to create a specialized self-review to assign a contribution to oneself. We decided that it would be best to allow access to the self-review once all other teammate reviews are complete because it encourages users to do their reviews and the final calculation for self contribution needs to only be done once.
Implementation
A new class Cake has been newly introduced as a subclass of ScoredQuestion. The Cake class is a type of question that can be offered as part of any questionnaire, which keeps an account of all the answer values recorded for one specific question. The maximum value that can be given to a question of type cake is 100, and any value entered above 100 is automatically void, setting the answer entered to zero by default. The user is informed of the same and also can keep track of how much of the “cake” has already been taken, which helps him determine the value that he can enter. Upon entering a value greater than 100, a warning is displayed, informing the user that the value has exceeded 100, and setting the value back to 0. A textbox input with up-down arrows is being used, to help the user increment/decrement values as he pleases.
Files involved:
1. app/controllers/questionnaires_controller.rb
Added cake type in editing the questionnaire.
def add_new_questions
...
question.size = '50, 3' if question.is_a? Criterion
question.size = '50, 3' if question.is_a? Cake
question.alternatives = '0|1|2|3|4|5' if question.is_a? Dropdown
...
end
2. app/controllers/response_controller.rb
A new method calculate_total_score was added. And it is called in methods new and edit to calculate and update the total contribution for a cake question when a student wants to create or edit a review.
def calculate_total_score
@total_score = Hash.new
@questions.each do |question|
if question.instance_of? Cake
reviewee_id = ResponseMap.select(:reviewee_id, :type).where(id: @response.map_id.to_s).first
total_score = question.get_total_score_for_question(reviewee_id.type, question.id, @participant.id, @assignment.id, reviewee_id.reviewee_id).to_s
if total_score.nil?
total_score = 0
end
@total_score[question.id] = total_score
end
end
end
3. *app/models/cake.rb
A new class Cake was added as a subclass of the ScoredQuestion class. A cake type question has a textbox that takes the percentage contribution assign to the reviewee for this specific question. Only positive integers under 100 are legal in this textbox, and illegal inputs will be prompted with corresponding error messages.
class Cake < ScoredQuestion
include ActionView::Helpers
validates :size, presence: true
#method is called during creation of questionnaire --> when cake type is added to the questionnaire.
def edit(_count)
...
4. app/views/questionnaires/_questionnaire.html.erb
Added cake to the dropdown list. Now the instructor can choose to add a cake type teammate review question to the questionnaire.
<%= select "question", "type",
{"Criterion" =>"Criterion",
"Scale"=>"Scale",
"Dropdown"=>"Dropdown",
"Checkbox"=>"Checkbox",
"TextArea"=>"TextArea",
"TextField"=>"TextField",
"UploadFile"=>"UploadFile",
"SectionHeader"=>"SectionHeader",
"TableHeader"=>"TableHeader",
"ColumnHeader"=>"ColumnHeader",
"Cake" => "Cake"
}, {}, {class: "form-control", onchange: "checkQuestionType()"} %> question(s)
5. app/views/response/response.html.erb
Added response showing how much of the total contribution has been assigned for a cake-type question.
<% elsif question.instance_of? Cake %>
<%= question.complete(i, answer, @total_score[question.id].to_s, @return) %>
6. app/views/student_teams/view.html.erb
Added a new table intended to display the remaining score in each question the reviewer currently has.
</tr> <!--self contribution for Cake questions-->
<% if @teammate_review_allowed %>
<tr style="border: 1px outset #000000; padding: 10px 20px" >
<th class="head">Teammate review question</th>
<th class="head">Self contribution percentage</th>
</tr>
<% end %> <!--end self contribution for Cake questions-->
Test plan
There are several things we plan on testing in this project as follows:
- Test the Cake question type to see that it properly renders and is editable
- Test the view for 'Your team' to verify the table containing Review Questions and Self Contributions is present when reviews are enabled
- Test the string reminder text for the cake questions that indicate the amount of contribution left to assign
- Test the cake contribution text boxes for valid responses (no negative numbers, over-allocation, or non-numeric characters)
Future scope
The implementation that is done in this project adds the Cake type question for questionnaires and verifies that a total of 100% is not exceeded by all contributions given to team members. It also validates that non-numeric entities are not allowed to be saved as a contribution. For the future scope of this project, the next step would be to add to the table view in student_teams so that the cake questions included in the teammate review rubric display along with the self contribution percentages. This can be done by pulling the response map for the user id and team id they are assigned to and populating all cake questions in the list. By using the calculation functions that exist in cake.rb, it should be possible to find the total contribution at any given time that has been assigned to teammates and subtract that that value from 100 to receive a self contribution. This value can then be displayed next to each cake question and be assignable to a grade for that student when being evaluated by the instructor.
Team Members
- Jordan, Dylan Tyler dtjordan@ncsu.edu
- Ruoyun Ma rma9@ncsu.edu
- Han, Lige lhan6@ncsu.edu
- Kollipara, Siddhartha skollip@ncsu.edu
- Mentor: Ed Gehringer (efg@ncsu.edu)