CSC/ECE 517 Fall 2019 - E1998. Weights in grade calculation

This wiki page has been created to document the changes that will be made under E1998 - Weights in grade calculation as a part of the final project for Fall 2019, CSC/ECE 517.

About Expertiza

Expertiza is an open source project which can be used by instructors for a multitude of use cases such as creating assignments of various types, creating rubrics, questionnaires, allowing teams to be created (for group assignments) etc. It also allows grading of the assignments created, and provides the ability to let students peer review other students' work. Additionally, it also allows an instructor to export data for various objects created into multiple formats for further analysis.

Project Overview

Background and Problem Statement

The questionnaire/rubric creation feature of Expertiza also allows weights to be added to the questions being created, thereby allowing an instructor to create a questionnaire with relative ordering among the questions from a grading standpoint i.e., correctly answering a question with a higher weight will fetch more marks as compared to correctly answering a question with a lower weight.

However, the same functionality is not present for quiz questionnaires. Quiz questionnaires are essentially a way to ensure that students review others' work genuinely. When an assignment has quizzes enabled, each student gets an option to create a quiz for each of his/her reviewers after the initial submission is done and a review round has occurred. While creating this quiz, a student is supposed to ask questions related to his/her work which ideally a reviewer must be able to answer. Once it is created, reviewers are able to answer these quizzes and the corresponding scores are evaluated for each quiz separately. Ideally, each question should have a weight associated with it which a student creating the quiz should be able to specify, which in turn should be used while calculating scores for those taking these quizzes. However, currently, there is no way to specify weights for individual questions while creating quizzes and hence scores are being calculated by giving each question equal weightage.

Goal of the Project

The goal of this project is to understand the cause of the aforementioned issue and to ensure that the weights associated with questions are taken into consideration when grading all types of questionnaires involving weights. Our main focus will be to implement weights for quiz questionnaires, To achieve this, first we will have to introduce a way for a student creating a quiz to be able to enter weights for individual questions. Once weights for each question are being successfully stored, we can proceed to change how scores for each reviewer(for a particular quiz) are being calculated such that these newly added weights are also being taken into consideration.

Current Implementation

Expertiza allows multiple questionnaires to be associated with an assignment. For instance, an assignment may have 3 questionnaires - one each for reviewing teammates, one for reviewing others' work, and one for giving feedback to the reviews received. There are various questions that can be created for each type of questionnaire but they can be broadly classified into two types - weighted and non-weighted.

The user-submitted responses to these questions can be accessed via a view called score_view which extracts results from three different tables - questions, questionnaires, and answers. This view has a column named question_weight which is populated by the user entered weights for weighted questions and is set to null for non-weighted questions. This is the view which the code uses for calculating grades for a particular assignment.

Currently, for questionnaires other than quiz questionnaires, an instructor creates a rubric for the questionnaire where he/she can specify weights for each weighted question that exists for that questionnaire. However, the way it is implemented is that the instructor first has to create a question without specifying any weight. The weight for this newly created question is set to 1 by default. After this is done, the instructor has to edit that question to trigger a way to enter a weight for that question. This method for specifying weights is non-intuitive and cumbersome.

For quiz questionnaires on the other hand, there is no way to enter weights for newly added questions at all. The weights are set to 1 by default for all the weighted questions and are not modified henceforth. As a result, scores are calculated for each quiz for a particular reviewer by giving equal weightage for each question. This calls for a complete overhaul of how scores for quiz questionnaires are being calculated to include weights which will need to be accepted from the student creating the quiz.

Changes made by the team

We followed a two-step approach for this project.

Questionnaires except quiz questionnaire

• The first step involved handling all questionnaires except the quiz questionnaire, ensuring that all of them store weights associated with any weighted questions they might have. If so, during score calculation, these weights need to be taken into consideration and the appropriate score displayed.

• We found that method get_total_score was responsible for the score calculation and that it took weights into consideration according to this equation:

---def get_total_score screenshot---

• Our next step was to address the issue of how weight entry while creating a rubric was being handled. As mentioned in the previous section, the user had to first create a question and then edit it to specify a weight. What we did was add a text box during the creation of a question itself where the user can enter the appropriate weight. One important point we needed to address was to only allow weight entry for weighted questions(Criterion and Scale). In the file itself, we changed the assignment of question.weight variable to implement the above change.

Code screenshot: ---text box for add question in questionnaire screenshot(weighted and unweighted)---

• The last stage in this step was to ensure that appropriate changes are made to tests written for questionnaires controller if they fail.

Quiz questionnaire

• For the quiz questionnaire, the first step was to implement a UI change to allow the student creating a quiz to be able to enter the weight for each question.

--UI quiz questionnaire screenshot--

Corresponding code screenshot ensuring entered weights are being stored for the appropriate questions:

• The next course of action was to actually implement a way to use these stored weights while calculating the score for a quiz. This involved making changes in the app/models/quiz_response_map.rb file and replacing the existing logic to calculate the score with the following:

SANVEG RANE'S SECTION

• The last stage was to write automated tests for the new functionality we implemented such that it tested as many kinds of quizzes as possible and ensured that the final score reflected the weights being set for individual weighted questions.

Please find below a high level overview of the approach that we'll be following to tackle this problem:

Files modified

The following files were modified to accomplish the aforementioned changes:

• questionnaires_controller.rb

add_new_questions

The weights were being hard coded earlier, which have now been replaced by the user specified weights via the UI. This method creates new questions when creating questions in questionnaires of types such as review questionnaire, author feedback questionnaire, teammate review questionnaire, etc.

save_new_questions

This method saves questions when a student creates a quiz questionnaire. There was no way to associate weights with questions in quiz questionnaire earlier, we've made changes to allow adding weights to questions in a quiz questionnaire. The corresponding changes are present in this method.

• _new_question_template.html.erb

The input field for adding question weights at the time of creation of questions in a quiz has been added to this file. This functionality was not present earlier, all questions in a quiz were associated with a default weight of 1. The weights entered in this input field are used by the save_new_questions method in questionnaires_controller.rb

• _questionnaire.html.erb

The input field for adding question weights at the time of creation of scored questions for questionnaires of all types except quiz questionnaire has been added in this file. This functionality was not present earlier, as questions had to be created first and then edited to change the default weight (1) associated with it. The weights entered in this input field are used by the add_new_questions method in questionnaires_controller.rb

• _question_weight.html.erb

This file contains the JavaScript code which handles the conditional display of the field to input weights depending on the type of question the user wants to create in the above view. This code ensures that the field is displayed only if the user is trying to created a question with which weights can be associated - questions of type Criterion and Scale.

• edit.html.erb

This files was had to be modified to include the above partial which contains the JavaScript for conditional display of the input field for entering question weight.

• quiz_response_map.rb

quiz_score

This method has been modified to include the logic for calculation of quiz scores. The added logic incorporates the weights which the user can now input corresponding to questions when creating quizzes.

The following file was modified to included automated tests for creation and grading of a quiz questionnaire:

• quiz_spec.rb

This file contains the necessary changes for testing the entire flow, right from creating a quiz to submitting it, and finally scoring it. The test has been created to check all the modifications we've made to the above files.

Test Plan

We created an example in this file which sets up the necessary actions before a student A can create a quiz questionnaire. We then proceed with quiz questionnaire creation and associate weights with questions created. We create all three types of questions that can be created in a quiz questionnaire - True/False questions, Multiple Choice Radio questions and Multiple Choice Checkbox questions. Further, we login as student B who has reviewed student A's work and take the quiz. Finally, we submit the quiz and ensure the that the quiz is scored after taking the weights associated with the questions into consideration.

Quick Links

• Forked Repository Link: https://github.com/theSaurabhMhatre/expertiza
• Pull Request Link: https://github.com/expertiza/expertiza/pull/1633
• Demo Link: To be created

Team

Our team:

• Sanket Pai (sgpai2@ncsu.edu)
• Sanveg Rane (ssrane2@ncsu.edu)
• Saurabh Mhatre (smhatre@ncsu.edu)
• Saurabh Shingte (svshingt@ncsu.edu)

Mentor: Carmen Bentley (cnaiken@ncsu.edu)