CSC/ECE 517 Fall 2016 E1682: Improve score calculation

E1682. Improve score calculation

Introduction

Purpose

The methodology behind score calculation in expertiza can be significantly improved. The design behind calculating and storing scores can be tweaked to offer significant performance improvements. To achieve this, it is ideal to store overall holistic stores in addition to individual criterion scores.

Problem Statement

The current drawback of the expertiza system lies in how the scores are stored: Expertiza stores the scores based on each response to each criterion, but no holistic scores are stored. As a result of this mechanism, the process slows down. This means that if we need to know what score user A gives to user B based on 100, we have to rely on the code to calculate it (since in answers table we only have the score that A gives B on each criterion).

To overcome this bottleneck, two additional mechanisms can be included for handling the holistic scores.

Proposed changes

The two suggested mechanisms for handling the storage and calculation of holistic scores for peer-reviews are :

OnTheFlyCalc:

Calculate holistic scores over a set of data (in this case, a set of record in answers table, namely the responses from user A to user B on each criterion)

Handles the calculation of reputations for each reviewer in the case of an ongoing assignment.

LocalDBCalc:

Calculate the holistic score over a single db query (in this case, a query would be: “get the score that user A gives user B on assignment 999 on round 2”)

Handles the calculation of reputations for each reviewer in the case of a finished assignment. (all the reputations are calculated and stored since they are finalized)

Implementation

Parent Class:ScoreCalc

ScoreCalc stores the score calculated by its sub classes: on_the_fly_calc and LocalDBcalc to the database. In the database, a new table local_db_scoresn is created with the schema as explained in section 2.2. Depending upon the deadline_type, object of either on_the_fly_calc or LocalDBcalc is created and the appropriate functions within the classes are called.

Database design

A table local_db_scores will be created as:

create_table "local_db_scores",force::cascade do|t|

"local_db_scores" will have the following scores

	t.integer "id", limit: 24	id	Primary Key, Auto generated
	t.string "Type", limit: 255	Type	Type of score: either final or intermediate
	t.integer "Score", limit: 24	Score	Total score calculated based on each response to each criterion
	t.integer "Round", limit:24	Round	The completed round number
	t.integer "Reference_id" , limit:45	Reference_id	Acts as the foreign Key

Sub class 1: on_the_fly_calc

An object of on_the_fly_calc is called when the deadline_type is not "review of review" which means that the assignment is ongoing and has not been completed and stores the scores till the current stage that will change once the assignment passes the deadline

The function compute_total_scores(scores) is used to calculate the total score based on each response to each criterion. The function is as below:

def compute_total_scores(scores)

total=0

self.questionnaires.each{|questionnaire| total+=questionnaire.get_weighted_score(self,scores)}

total

end

The value in variable "total" needs to be stored in local_db_scores under "Score" and will be retrieved by using another function retrieve_total_score to display the total score instead of calculating it every time a user wants to view the scores.

Sub class 2: LocalDB_calc

An object of LocalDB_calc is called when the deadline_type is "review of review" which means that the assignment is completed and stores the final score in the database local_db_scores after computing it rather than computing it every time the score is required.

Function compute_total_scores(scores) is added to the class which calculates the total score based on each response to each criterion. The function(polymorphism) is as below:

def compute_total_scores(scores)

total=0

self.questionnaires.each{|questionnaire| total+=questionnaire.get_weighted_score(self,scores)}

total

end

The value in variable "total" needs to be stored in local_db_scores under "Score" and will be retrieved by using another function retrieve_total_score to display the total score instead of calculating it every time a user wants to view the scores.