CSC/ECE 517 Fall 2019 - E1998. Weights in grade calculation

This wiki page has been created to document the changes that will be made under E1998 - Weights in grade calculation as a part of the final project for Fall 2019, CSC/ECE 517.

About Expertiza

Expertiza is an open source project which can be used by instructors for a multitude of use cases such as creating assignments of various types, creating rubrics, questionnaires, allowing teams to be created (for group assignments) etc. It also allows grading of the assignments created, and provides the ability to let students peer review other students' work. Additionally, it also allows an instructor to export data for various objects created into multiple formats for further analysis.

Project Overview

Background and Problem Statement

The questionnaire/rubric creation feature of Expertiza also allows weights to be added to the questions being created, thereby allowing an instructor to create a questionnaire with relative ordering among the questions from a grading standpoint i.e., correctly answering a question with a higher weight will fetch more marks as compared to correctly answering a question with a lower weight. However, it has been observed that when grading is done for questionnaires, the module responsible for grading seems to ignore the weights associated with questions.

Goal of the Project

The goal of this project is to understand the cause of the aforementioned issue, and to ensure that the weights associated with questions are taken into consideration when grading all types of questionnaires involving weights.

Current Implementation

Expertiza allows multiple questionnaires to be associated with an assignment. For instance, an assignment may have 3 questionnaires - one each for reviewing teammates, one for reviewing others work, and one for giving feedback to the reviews received. There are various questions that can be created for each types of questionnaire but they can be broadly classified into two types - weighted and non-weighted.

The user submitted responses to these questions can be accessed via a view - score_view which extracts results from three different tables - questions, questionnaires and answers. This view has a column named question_weight which is populated by the user entered weights for weighted questions and is set to null for non-weighted questions. This is the view which the code uses for calculating grades for a particular assignment.

Plan of Attack

Our main plan is to ensure that all the code that is responsible for calculating scores is taking weights into account if the final scores depend on doing so. Since the questions can be weighted or unweighted, while calculating the final scores, the score assigned by a user to individual questions will be handled (aggregated) differently depending on the type of question. Our job will be to verify whether weights are being multiplied with the individual scores for weighted questions and modify the code if weights are not considered.

To achieve the aforementioned objective, the following steps will need to be followed:

• We need to first identify the files where the code for score calculation exists in the current implementation
• Then we will need to peruse the code responsible for creating rubrics/questionnaires and editing or manipulating them in any way
• In particular, a module named on_the_fly_calc.rb is responsible for score calculation, which will be our main area of focus

Another important task for our team will be to ensure our changes do not break existing functionality by writing relevant tests for the different types of questionnaires(review, teammate review, author feedback, meta-review, quiz, etc).

Please find below a high level overview of the approach that we'll be following to tackle this problem:

Files which require modification (Tentative)

The following methods in the files might require modifications:

• answer.rb
  • compute_scores
  • get_total_score

• grades_controller.rb
  • view_team

• assignment_participant.rb
  • scores
  • compute_assignment_score

• questionnaires_controller.rb
  • add_new_questions
  • save_all_questions

• questionnaire.rb
  • compute_weighted_score
  • save_questions

• on_the_fly_calc.rb
  • compute_avg_and_ranges_hash
  • calc_review_score

Test Plan

• All of the above-mentioned files and the corresponding methods already have rspec files defined to test their functionalities, so our first task after making our changes will be to ensure they do not break for the new changes.
• The next course of action will be to write new tests to cover all the code changes we will be making.
• Since this project involves locating, replicating and then fixing a bug, we are yet to identify precisely which methods will be changed.
• That being said, we can say with reasonable certainty that we will require additional tests for questionnaires_controller.rb to ensure that while adding new questions they are assigned weights if they are weighted questions. In case they are, appropriate weights must be assigned.
• Another test we will have to either modify or write will be for compute_weighted_score method of questionnaire.rb model to ensure that while calculating the final score weights are being considered. Similar tests will have to be written if applicable for calc_review_score and compute_avg_and_ranges_hash for the on_the_fly_calc.rb file.
• Finally, we will attempt to create tests which would include creating questionnaires of every relevant type with weighted questions, and ensuring that the weights are included correctly in the grade calculations.

Quick Links

• Forked Repository Link: https://github.com/theSaurabhMhatre/expertiza
• Pull Request Link: To be created
• Demo Link: To be created

Team

Our team:

• Sanket Pai (sgpai2@ncsu.edu)
• Sanveg Rane (ssrane2@ncsu.edu)
• Saurabh Mhatre (smhatre@ncsu.edu)
• Saurabh Shingte (svshingt@ncsu.edu)

Mentor: Carmen Bentley (cnaiken@ncsu.edu)