CSC/ECE 517 Spring 2017/E1731 Improve Score Calculation

Introduction

Expertiza is an opensource web based platform developed to assist students in educational institutions to perform peer reviews and group projects. The reviewing is done through a mechanism where students will be sent request to review work and the system will analyze and publish a summarized report. The software is designed in MVC (Model-View-Controller) architecture using Ruby on Rails.

Problem Description

Current Scenario

Today, Expertiza stores the scores based on each response to each criterion, but no holistic scores are stored. The answers table only contains the score that A gives B on each criterion. This means that if we need to know what score user A gives to user B based on 100, we have to rely on the code to calculate it every time the scores are requested. This design slows down Expertiza as every time instructor clicks on "View Scores", all the calculations are done to display the score. The situation is even worse if we want to weight the scores by the reputation of each reviewer, which is a measure of how closely that reviewer's scores match other reviewers. The system potentially has to do thousands of arithmetic calculations to display the score for a particular assignment.

Proposed Solution

We propose that we have two mechanism to handle holistic scores, depending on the current state of the assignment:

• OnTheFlyCalc: Calculates holistic scores over a set of data: A set of records in answers table, namely the responses from user A to user B on each criterion.
  • When an assignment is still ongoing, the reputations for each reviewer will be calculated and handled by OnTheFlyCalc.
• LocalDBCalc: Calculates the holistic score over a single db query: "get the score that user A gives user B on assignment 999 on round 2"
  • When an asignment has finished, all the reputations will be calculated by LocalDBCalc and stored, since they are finalized. So, when the scores are requested, it is fetched using a single db query and displayed.

In this project, our goal is to make the OnTheFlyCalc and LocalDBCalc work only for peer-review scores. As future projects will introduce other types of scores, the solution must be extensible.

Design Plan

• A general ScoreCalc class will be created
• OnTheFlyCalc and LocalDBCalc classes will be created which will be subclasses of ScoreCalc class
• A new database table "local_db_scores" with following columns will be created to store the holistic scores:
  • id: int
  • type: string
  • round: int
  • score: int
  • reference_id: int
• When an assignment finishes (that is when the deadline has passed), holistic scores will be calculated in LocalDBCalc and then inserted in this new db table "local_db_score"
  • This will be done using scheduled tasks feature
  • If the record is a finalized peer-review score, the type stored in the database will be "ReviewLocalDBScore" and the reference_id will be the response_map id
  • The peer-review scores will be calculated for all the review rounds because it is possible that user A only review user B in one out of several rounds
• When a request is made to view scores, depending on whether the assignment is ongoing or finished, either OnTheFlyCalc or LocalDBCalc will be called to calculate the score or query the score from the "local_db_scores" table