Reviewer Assignment implementation - Revision history

Admin: /* Least‑Reviewed Submission Strategy */

2026-03-15T20:06:39Z

Least‑Reviewed Submission Strategy

← Older revision		Revision as of 20:06, 15 March 2026
Line 57:		Line 57:
	'''Eligibility rules:'''		'''Eligibility rules:'''

	A reviewer may not be assigned to their own team.		*A reviewer may not be assigned to their own team.
			*A reviewer may not be assigned to a team they have already reviewed.
	A reviewer may not be assigned to a team they have already reviewed.		*Among eligible teams, the one with the lowest review count is selected.

	Among eligible teams, the one with the lowest review count is selected.

	This ensures a balanced distribution of reviews across submissions.		This ensures a balanced distribution of reviews across submissions.

2026-03-15T20:06:04Z

Overview

← Older revision		Revision as of 20:06, 15 March 2026
Line 1:		Line 1:
	= Overview =		= Overview =
	The review‑assignment subsystem		The review‑assignment subsystem ''in the reimplemented Expertiza'' supports both '''static''' and '''dynamic''' allocation of reviews within an assignment. The goal is to give instructors flexible control over how reviewers are matched to submissions, while also supporting on‑demand, fairness‑aware assignment for students who request reviews dynamically.
	'in the reimplemented Expertiza'
	supports both '''static''' and '''dynamic''' allocation of reviews within an assignment. The goal is to give instructors flexible control over how reviewers are matched to submissions, while also supporting on‑demand, fairness‑aware assignment for students who request reviews dynamically.

	This page documents the behavior implemented in ReviewMappingsController and the strategy classes under ReviewMappingStrategies.		This page documents the behavior implemented in ReviewMappingsController and the strategy classes under ReviewMappingStrategies.

2026-03-15T20:05:26Z

Overview

← Older revision		Revision as of 20:05, 15 March 2026
Line 1:		Line 1:
	= Overview =		= Overview =
	The review‑assignment subsystem 'in the reimplemented Expertiza' supports both '''static''' and '''dynamic''' allocation of reviews within an assignment. The goal is to give instructors flexible control over how reviewers are matched to submissions, while also supporting on‑demand, fairness‑aware assignment for students who request reviews dynamically.		The review‑assignment subsystem
			'in the reimplemented Expertiza'
			supports both '''static''' and '''dynamic''' allocation of reviews within an assignment. The goal is to give instructors flexible control over how reviewers are matched to submissions, while also supporting on‑demand, fairness‑aware assignment for students who request reviews dynamically.

	This page documents the behavior implemented in ReviewMappingsController and the strategy classes under ReviewMappingStrategies.		This page documents the behavior implemented in ReviewMappingsController and the strategy classes under ReviewMappingStrategies.

2026-03-15T20:04:10Z

Overview

← Older revision		Revision as of 20:04, 15 March 2026
Line 1:		Line 1:
	= Overview =		= Overview =
	The review‑assignment subsystem supports both '''static''' and '''dynamic''' allocation of reviews within an assignment. The goal is to give instructors flexible control over how reviewers are matched to submissions, while also supporting on‑demand, fairness‑aware assignment for students who request reviews dynamically.		The review‑assignment subsystem 'in the reimplemented Expertiza' supports both '''static''' and '''dynamic''' allocation of reviews within an assignment. The goal is to give instructors flexible control over how reviewers are matched to submissions, while also supporting on‑demand, fairness‑aware assignment for students who request reviews dynamically.

	This page documents the behavior implemented in ReviewMappingsController and the strategy classes under ReviewMappingStrategies.		This page documents the behavior implemented in ReviewMappingsController and the strategy classes under ReviewMappingStrategies.

2026-03-15T20:03:33Z

← Older revision		Revision as of 20:03, 15 March 2026
Line 1:		Line 1:
	~~Review Assignment System~~ Overview		= Overview =
	The review‑assignment subsystem supports both '''static''' and '''dynamic''' allocation of reviews within an assignment. The goal is to give instructors flexible control over how reviewers are matched to submissions, while also supporting on‑demand, fairness‑aware assignment for students who request reviews dynamically.		The review‑assignment subsystem supports both '''static''' and '''dynamic''' allocation of reviews within an assignment. The goal is to give instructors flexible control over how reviewers are matched to submissions, while also supporting on‑demand, fairness‑aware assignment for students who request reviews dynamically.

2026-03-15T20:03:09Z

Common Eligibility Rules

← Older revision		Revision as of 20:03, 15 March 2026
Line 141:		Line 141:
	Across all dynamic strategies, a reviewer is never assigned to:		Across all dynamic strategies, a reviewer is never assigned to:

	Their own team		*Their own team
			*A team they have already reviewed
	A team they have already reviewed		*A team outside the selected topic (for topic‑based assignment)

	A team outside the selected topic (for topic‑based assignment)

2026-03-15T20:02:49Z

Strategy Classes

← Older revision		Revision as of 20:02, 15 March 2026
Line 119:		Line 119:
	All strategies inherit from ReviewMappingStrategies::BaseStrategy, which defines two abstract methods:		All strategies inherit from ReviewMappingStrategies::BaseStrategy, which defines two abstract methods:

	each_review_pair — for static strategies		*each_review_pair — for static strategies

	assign_one — for dynamic strategies		*assign_one — for dynamic strategies

	Each concrete strategy implements one of these depending on its purpose.		Each concrete strategy implements one of these depending on its purpose.

2026-03-15T20:02:25Z

Instructor Grading of Reviews

← Older revision		Revision as of 20:02, 15 March 2026
Line 111:		Line 111:
	Instructors may grade individual reviews. This action:		Instructors may grade individual reviews. This action:

	Locates the review mapping		*Locates the review mapping
			*Records the instructor’s grade and comment
	Records the instructor’s grade and comment		*Returns a success message

	Returns a success message

	= Strategy Classes =		= Strategy Classes =

2026-03-15T20:02:03Z

Topic‑Balanced Dynamic Assignment

← Older revision		Revision as of 20:02, 15 March 2026
Line 83:		Line 83:

	This ensures:		This ensures:
			*Reviewers can restrict themselves to topics they prefer.
	Reviewers can restrict themselves to topics they prefer.		*Topics remain approximately balanced, with fairness controlled by the threshold k.

	Topics remain approximately balanced, with fairness controlled by the threshold k.

	= Calibration Review Assignment =		= Calibration Review Assignment =

2026-03-15T20:01:43Z

Topic‑Balanced Dynamic Assignment

← Older revision		Revision as of 20:01, 15 March 2026
Line 78:		Line 78:
	The first eligible topic (deterministically) is selected.		The first eligible topic (deterministically) is selected.
	Within that topic, the reviewer is assigned to a team that:		Within that topic, the reviewer is assigned to a team that:
	** is not their own team,		*is not their own team,
	** has not already been reviewed by them,		*has not already been reviewed by them,
	** has the fewest reviews among eligible teams.		*has the fewest reviews among eligible teams.

	This ensures:		This ensures: