Score Calculation: Difference between revisions
| (One intermediate revision by the same user not shown) | |||
| Line 5: | Line 5: | ||
== Layer 1: Response (Item Weights) == | == Layer 1: Response (Item Weights) == | ||
<code>Response#aggregate_questionnaire_score</code> sums <code>answer * item.weight</code> for all answered, scored items in a single submitted review. Section header items are excluded (they carry no weight). | <code>Response#aggregate_questionnaire_score</code> sums <code>answer * item.weight</code> for all answered, scored items in a single submitted review. Section-header items, text-area items, etc., are excluded (they carry no weight). | ||
<code>Response#maximum_score</code> computes the maximum possible score for the same set of answered items: <code>sum(item.weight) * questionnaire.max_question_score</code>. | <code>Response#maximum_score</code> computes the maximum possible score for the same set of answered items: <code>sum(item.weight) * questionnaire.max_question_score</code>. | ||
| Line 11: | Line 11: | ||
== Layer 2: ResponseMap (Round Weights) == | == Layer 2: ResponseMap (Round Weights) == | ||
<code>ResponseMap#review_grade</code> | <code>ResponseMap#review_grade</code> normalizes each round's score (<code>aggregate_questionnaire_score / maximum_score</code>) and computes a weighted average across rounds using <code>AssignmentQuestionnaire#questionnaire_weight</code>. | ||
Only the latest submitted response per round is used. Rounds with no submitted response are excluded from both the numerator and denominator. | Only the latest submitted response per round is used. Rounds with no submitted response are excluded from both the numerator and denominator. | ||
Latest revision as of 00:36, 7 July 2026
Score Calculation
Peer review grades in Expertiza are computed in three models.
Layer 1: Response (Item Weights)
Response#aggregate_questionnaire_score sums answer * item.weight for all answered, scored items in a single submitted review. Section-header items, text-area items, etc., are excluded (they carry no weight).
Response#maximum_score computes the maximum possible score for the same set of answered items: sum(item.weight) * questionnaire.max_question_score.
Layer 2: ResponseMap (Round Weights)
ResponseMap#review_grade normalizes each round's score (aggregate_questionnaire_score / maximum_score) and computes a weighted average across rounds using AssignmentQuestionnaire#questionnaire_weight.
Only the latest submitted response per round is used. Rounds with no submitted response are excluded from both the numerator and denominator.
Layer 3: AssignmentTeam (Reviewer Reputation)
AssignmentTeam#aggregate_reviewer_score averages ResponseMap#review_grade across all reviewers of the team, weighted by each reviewer's reputation score (currently defaulting to 1.0 — placeholder for future Uchswas integration).
The same computation is used by AssignmentParticipant#aggregate_teammate_review_grade for teammate reviews.
Validation
AssignmentQuestionnaire validates that questionnaire_weight is 0 when the linked rubric contains no scored items (only SectionHeader items), since a non-zero weight on an unscored rubric would produce meaningless grades.