Expertiza_Wiki - User contributions [en]

CSC/ECE 517 Spring 2026 - E2605. Specialized Rubrics for Different Topic Types

2026-05-03T01:40:01Z

Vagarwa3: /* Links */

__TOC__

== Introduction ==

This wiki page describes the design and implementation for Expertiza project '''E2605: Specialized Rubrics for Different Topic Types'''.

The goal of this project is to let an instructor use different '''review rubrics''' for different '''project topics''' in the same assignment.

Before this project, Expertiza could use:

* one review rubric for the whole assignment, or
* different review rubrics for different review rounds.

But it could not use different review rubrics for different topics in the same assignment.

This project fixes that problem.

== Project Overview ==

Some assignments have topics that are very different from each other.

For example:

* one topic may be about security
* one topic may be about user interface design
* one topic may be about machine learning

These topics should not always be reviewed with the same rubric.

A security topic may need questions about:

* threat modeling
* risk analysis
* vulnerability handling

A user interface topic may need questions about:

* usability
* layout
* design quality

This project lets the system choose the right review rubric based on the topic of the team being reviewed.

== Problem Statement ==

Expertiza already knows:

* which team is being reviewed
* which topic that team signed up for
* which questionnaires are attached to the assignment

But before this project, Expertiza did not store which review rubric should be used for each topic.

Because of that:

* reviewers could receive a rubric that did not match the topic
* instructors could not configure topic-based review rubrics in the assignment editor
* changing rubric mappings after reviews started could leave old reviews inconsistent with the new rubric

== Design Goals ==

The design goals for this project are:

* let instructors assign different review rubrics to different topics
* keep the current review workflow the same for reviewers
* keep old assignments working the same way unless topic variation is enabled
* work correctly with round-based rubric variation
* use the existing database structure as much as possible
* keep rubric selection clear and predictable
* safely handle rubric changes after reviews have already started

== Non-Goals ==

This project does not:

* redesign topic signup
* redesign waitlist behavior
* redesign team formation
* redesign review mapping
* redesign questionnaire authoring from scratch

It only changes how the system stores and finds the correct review rubric.

== Main Idea ==

The main idea of the project is:

# Start from the review response map.
# Find the team being reviewed.
# Find that team’s confirmed topic signup.
# Find the correct review rubric for that assignment, topic, and round.

In simple form, the lookup path is:

<code>ReviewResponseMap -> reviewee team -> signed_up_teams -> topic -> assignment_questionnaires -> questionnaire</code>

== Important Models ==

The models used in this feature are:

* '''Assignment'''
** stores assignment settings
** now stores whether rubric varies by topic
* '''ProjectTopic'''
** stores topics for an assignment
* '''SignedUpTeam'''
** stores which topic a team selected
* '''AssignmentQuestionnaire'''
** stores which questionnaire is used for an assignment
** now can also store which topic that mapping belongs to
* '''ReviewResponseMap'''
** stores who reviews whom
* '''Response'''
** stores the actual review response
* '''ReviewResetArchive'''
** stores old reviews when rubric changes make them invalid

== UML Diagram ==

The diagram below shows the main models and relationships used in topic-based rubric selection.

[[File:E2605_class_diagram.png|800px|center|Class diagram for topic-based rubric selection]]

== Sequence Diagram ==

The diagram below shows how Expertiza finds the correct review rubric when a reviewer opens a review.

[[File:E2605_rubric_lookup_sequence.png|800px|center|Sequence diagram for topic-based rubric lookup]]

== Sequence Diagram for Rubric Change Handling ==

The diagram below shows what happens when an instructor changes a rubric or a topic-rubric mapping after reviews have already started.

[[File:E2605_rubric_reset_sequence.png|800px|center|Sequence diagram for archive, notify, and reset flow]]

== Original Design ==

The original design was based on a small change to the current rubric system.

=== Original Schema Design ===

The original plan was to add:

* <code>assignments.vary_by_topic</code>
* <code>assignment_questionnaires.project_topic_id</code>

The meaning was:

* <code>project_topic_id = NULL</code> means the mapping is the default rubric for the assignment
* <code>project_topic_id = some topic id</code> means the mapping is for that topic

=== Original Lookup Design ===

The original lookup order was:

# topic + current round
# topic + default round
# default topic + current round
# default topic + default round

Here:

* '''default topic''' means <code>project_topic_id IS NULL</code>
* '''default round''' means <code>used_in_round IS NULL</code>

This keeps lookup clear when both topic variation and round variation are enabled.

=== Original UI Design ===

The original UI plan was:

* instructor enables topic-based review rubric variation
* instructor chooses topic-specific review rubrics in the assignment editor
* if round variation is enabled, the instructor can also choose rubrics by topic and round

== What We Implemented ==

The final implementation follows the original design, but also adds some extra support that was needed to make the feature work in the real system.

{| class="wikitable" style="width:100%;"
|-
! Area
! Original Design
! What We Implemented
|-
| Assignment setting
| Add topic variation flag
| Added <code>vary_by_topic</code> to <code>assignments</code>
|-
| Rubric mapping
| Add topic id to assignment-questionnaire mapping
| Added <code>project_topic_id</code> to <code>assignment_questionnaires</code>
|-
| Rubric lookup
| Find rubric from topic and round
| Added shared lookup logic in <code>TopicBasedRubricLookup</code>
|-
| Instructor UI
| Configure topic rubrics in assignment editor
| Added topic rubric controls in the React assignment editor
|-
| Reload existing data
| Not described in detail
| Added serializer and frontend transformation logic so saved mappings reload correctly
|-
| Reviewer flow
| Reviewer gets correct rubric automatically
| Backend resolves rubric and frontend shows the resolved rubric information
|-
| Rubric changes after reviews start
| Mentioned as a concern
| Implemented archive, email notification, and review reset logic
|}

== Schema Changes ==

=== 1. <code>assignments.vary_by_topic</code> ===

A new boolean field was added to the assignments table.

<syntaxhighlight lang="ruby">
add_column :assignments, :vary_by_topic, :boolean, default: false, null: false
</syntaxhighlight>

This means:

* old assignments still work the same way
* topic-based rubric selection happens only when the instructor enables it

=== 2. <code>assignment_questionnaires.project_topic_id</code> ===

A new nullable topic reference was added to <code>assignment_questionnaires</code>.

<syntaxhighlight lang="ruby">
add_reference :assignment_questionnaires, :project_topic, foreign_key: true, null: true
</syntaxhighlight>

This lets the same table store:

* default assignment review rubric
* topic-specific review rubric
* round-based review rubric
* topic-and-round-specific review rubric

=== 3. Indexes ===

Indexes were added so the system can quickly find the correct rubric.

The most important lookup index is on:

* <code>assignment_id</code>
* <code>project_topic_id</code>
* <code>used_in_round</code>

=== 4. Review Archive Table ===

A new table was added:

* <code>review_reset_archives</code>

This table stores old reviews when a rubric change makes those reviews no longer valid.

It stores:

* response id
* response map id
* assignment id
* questionnaire id
* topic id
* round
* reviewer id
* reviewee id
* reset reason
* snapshot data

== Rubric Lookup Design ==

The final lookup logic is placed in a shared concern:

* <code>app/models/concerns/topic_based_rubric_lookup.rb</code>

This concern is included in the <code>Assignment</code> model.

=== Main Lookup Method ===

The main method is:

* <code>assignment_questionnaire_for_response_map(response_map, round:)</code>

This method is used to find the correct rubric for a review.

=== Reviewee Topic Lookup ===

When the response map is a <code>ReviewResponseMap</code>, the system does this:

# finds the reviewee team
# finds that team’s confirmed topic signup
# ignores waitlisted signups
# uses that topic id in rubric lookup

The method used is:

* <code>confirmed_topic_id_for(reviewee)</code>

=== Lookup Order ===

The final lookup order is still:

# topic + current round
# topic + default round
# assignment default topic + current round
# assignment default topic + default round

This keeps rubric selection simple and predictable.

== Original vs Final Lookup Code ==

{| class="wikitable" style="width:100%;"
|-
! Before
! After
|-
| Topic-based lookup did not exist.
| <code>Assignment</code> now includes <code>TopicBasedRubricLookup</code> and uses topic-aware rubric resolution.
|}

=== Final Lookup Code ===

<syntaxhighlight lang="ruby">
def assignment_questionnaire_for_response_map(response_map, round:)
return assignment_questionnaires.find_by(used_in_round: round) unless response_map.is_a?(ReviewResponseMap)

review_assignment_questionnaire(response_map, round) || assignment_questionnaires.find_by(used_in_round: round)
end
</syntaxhighlight>

== Model Validation ==

The <code>AssignmentQuestionnaire</code> model now validates two important things.

=== 1. Topic Must Belong to the Same Assignment ===

<syntaxhighlight lang="ruby">
def project_topic_belongs_to_assignment
return if project_topic_id.blank? || assignment_id.blank?
return if project_topic&.assignment_id == assignment_id

errors.add(:project_topic, 'must belong to the same assignment')
end
</syntaxhighlight>

This prevents a topic from one assignment from being used in another assignment’s rubric mapping.

=== 2. No Duplicate Review Mapping for the Same Assignment, Topic, and Round ===

<syntaxhighlight lang="ruby">
def unique_review_rubric_for_assignment_topic_round
return unless review_questionnaire?

duplicate = AssignmentQuestionnaire
.joins(:questionnaire)
.where(
assignment_id: assignment_id,
project_topic_id: project_topic_id,
used_in_round: used_in_round,
questionnaires: { questionnaire_type: 'ReviewQuestionnaire' }
)
duplicate = duplicate.where.not(id: id) if persisted?

errors.add(:base, 'review rubric already exists for this assignment, topic, and round') if duplicate.exists?
end
</syntaxhighlight>

This makes sure lookup returns one correct rubric.

== Backend Controller and API Changes ==

=== Assignment Save and Reload ===

To support this feature, the assignment edit flow had to save and reload more data.

The backend now permits:

* <code>vary_by_topic</code>
* <code>vary_by_round</code>
* <code>rounds_of_reviews</code>
* nested <code>assignment_questionnaires_attributes</code>

The <code>Assignment</code> model also now accepts nested questionnaire mappings:

<syntaxhighlight lang="ruby">
accepts_nested_attributes_for :assignment_questionnaires, allow_destroy: true
</syntaxhighlight>

=== Assignment Serializer ===

The assignment serializer now returns:

* <code>vary_by_topic</code>
* <code>assignment_questionnaires</code>
* <code>due_dates</code>
* <code>num_review_rounds</code>

This is needed so the frontend can reopen an assignment and show the saved topic and round rubric settings.

=== Topic Rubric Mapping API ===

The final implementation adds a dedicated controller for topic-rubric mappings:

* <code>AssignmentQuestionnairesController#index</code>
* <code>AssignmentQuestionnairesController#create</code>
* <code>AssignmentQuestionnairesController#update</code>
* <code>AssignmentQuestionnairesController#destroy</code>

This API is used by the React assignment editor.

=== Reviewer Rubric Lookup API ===

The backend also adds:

* <code>StudentTasksController#rubric_for</code>

This endpoint returns:

* assignment questionnaire id
* questionnaire id
* questionnaire name
* topic id
* round

This gives the frontend a way to ask: “Which rubric did the backend choose for this review?”

== Original vs Final API Design ==

{| class="wikitable" style="width:100%;"
|-
! Before
! After
|-
| No API for topic-rubric mappings.
| Added CRUD API in <code>AssignmentQuestionnairesController</code>.
|-
| No endpoint to ask which rubric was resolved for a review.
| Added <code>/student_tasks/rubric_for</code>.
|}

== Frontend Design ==

The topic-based rubric feature is configured in the React assignment editor.

Main frontend files are:

* <code>src/pages/Assignments/AssignmentUtil.ts</code>
* <code>src/pages/Assignments/AssignmentEditor.tsx</code>
* <code>src/pages/Assignments/tabs/TopicsTab.tsx</code>

=== Instructor Workflow ===

The instructor workflow is:

# open assignment editor
# enable topic-based rubric variation
# optionally enable round-based rubric variation
# go to Topics tab
# choose the review rubric for each topic
# save the assignment

If round variation is also enabled, the instructor can choose a rubric for each topic and round.

=== Request Transformation ===

The frontend now maps the form fields back to backend assignment fields.

Example:

<syntaxhighlight lang="typescript">
vary_by_round: values.review_rubric_varies_by_round,
vary_by_topic: values.review_rubric_varies_by_topic,
rounds_of_reviews: values.number_of_review_rounds,
assignment_questionnaires_attributes: assignmentQuestionnaires,
</syntaxhighlight>

=== Response Transformation ===

When loading an assignment, the frontend now restores:

* whether rubric varies by topic
* whether rubric varies by round
* number of review rounds
* saved assignment-questionnaire mappings
* due dates

This allows saved rubric settings to appear again when editing the assignment later.

== Original vs Final Frontend Design ==

{| class="wikitable" style="width:100%;"
|-
! Before
! After
|-
| No topic-rubric controls in assignment editor.
| Added topic-rubric controls in the Topics tab.
|-
| Assignment editor did not save topic rubric settings.
| Assignment editor now saves <code>vary_by_topic</code> and topic-based mappings.
|-
| Assignment editor did not reload topic rubric settings cleanly.
| Serializer and frontend transformation now reload saved mappings.
|}

== Questionnaire UI Support ==

The original design assumed instructors could already create the review rubrics they needed.

In practice, the React questionnaire UI needed some fixes.

The final implementation adds:

* correct mapping for the Review questionnaire type
* working edit navigation path for questionnaires
* item type loading from <code>/questions/types</code>
* fallback item type list when needed

This was important because the feature is only useful if instructors can create review rubrics in the frontend.

== Review Reset and Archive Design ==

== Why This Was Needed ==

If an instructor changes a rubric after reviews have already started, the old reviews may no longer match the new rubric.

For example:

* a topic is mapped to a different review rubric
* a review questionnaire is edited
* a question inside a review questionnaire is added, updated, or deleted

In these cases, the old completed reviews should not stay active as if nothing changed.

But at the same time, the old student work should not be lost.

== Final Reset Design ==

The final implementation adds a shared concern:

* <code>app/controllers/concerns/review_reset_handler.rb</code>

This concern does four things:

# find the affected reviews
# archive the old review data
# notify reviewers that they must redo the review
# delete the old live responses

== Review Archive ==

The archived review stores:

* response details
* questionnaire details
* reviewer and reviewee ids
* topic id
* round
* comments
* answers
* item text
* item type
* scores
* reset reason

This means the system keeps the student’s old completed work in the database.

== Reset Cases Implemented ==

The following changes now trigger archive + reset:

* topic-rubric mapping created
* topic-rubric mapping updated
* topic-rubric mapping deleted
* review questionnaire updated
* question created in a review questionnaire
* question updated in a review questionnaire
* question deleted from a review questionnaire
* all questions deleted from a review questionnaire

== Original vs Final Review-Reset Design ==

{| class="wikitable" style="width:100%;"
|-
! Original Design
! Final Implementation
|-
| Mentioned that old reviews should not stay valid after rubric changes.
| Fully implemented archive + email + reset behavior.
|-
| Did not say where old reviews would be stored.
| Added <code>review_reset_archives</code> table with full snapshot data.
|}

== Reviewer View ==

The reviewer does not pick a rubric manually.

Instead:

# reviewer opens a review
# frontend sends <code>response_map_id</code> and optional round
# backend finds the correct rubric
# frontend receives the resolved rubric details

The React review page now shows:

* resolved questionnaire name
* topic id, if topic-specific
* round, if round-specific

This gives clear proof that the backend selected a rubric.

== Known Limitation ==

The current review page does '''not yet''' fully render the actual questionnaire items from the resolved rubric.

So:

* the backend does resolve the correct rubric
* the frontend does show which rubric was resolved
* but the review page still uses mostly static review content

This is a known limit of the current implementation.

== Testing ==

=== Backend Tests ===

Backend tests were added for:

* topic-rubric mapping API
* topic-aware rubric lookup
* archive and reset behavior after rubric changes

Main backend test files include:

* <code>spec/controllers/assignment_questionnaires_controller_spec.rb</code>
* <code>spec/controllers/questionnaires_controller_spec.rb</code>
* <code>spec/controllers/student_tasks_controller_spec.rb</code>
* <code>spec/models/assignment_spec.rb</code>

=== Frontend Tests ===

Frontend tests were added for:

* assignment request mapping for <code>vary_by_topic</code>
* assignment response mapping for <code>vary_by_topic</code>
* topic tab rubric behavior
* questionnaire type mapping

Main frontend test files include:

* <code>src/pages/Assignments/AssignmentEditor.test.tsx</code>
* <code>src/pages/Assignments/tabs/TopicsTab.test.tsx</code>
* <code>src/pages/Questionnaires/QuestionnaireUtils.test.ts</code>

=== Manual Testing ===

The following manual cases were also checked:

* topic-based rubric selection can be saved
* saved topic-rubric mappings reload correctly
* topic and round fallback order works
* waitlisted signups do not affect rubric lookup
* old completed reviews are archived when the rubric changes
* reviewers are notified and old live reviews are reset

== Risks ==

The main risks in this project were:

* unclear rubric precedence
* incomplete topic-rubric setup
* mixing topic-based and round-based rules
* losing student work when rubrics change

These were handled by:

* a strict lookup order
* validation in the mapping model
* safe fallback behavior
* review archiving before reset

== Known Limits ==

The current implementation still has some limits:

* the review page does not yet fully render the real questionnaire items dynamically
* archived reviews are stored in the database, but there is no frontend page to view them
* the current main-branch implementation allows one review rubric per topic per round

== Design Principles Used ==

The project uses these design ideas:

* '''Single Responsibility Principle'''
** rubric lookup is placed in <code>TopicBasedRubricLookup</code>
** review reset logic is placed in <code>ReviewResetHandler</code>
** assignment request/response conversion is placed in frontend utility code

* '''Separation of Concerns'''
** backend decides which rubric to use
** frontend lets the instructor configure topic mappings
** model validation keeps data safe

* '''Backward Compatibility'''
** assignments that do not enable topic variation still work the old way

* '''Deterministic Behavior'''
** the same topic and round always follow the same lookup order

== Team ==

=== Mentor ===

* Ed Gehringer

=== Team Members ===

* Vanaja Agarwal
* Himashu Agarwal
* Bhargavi Jadhav

== Links ==

Pull Requests:
* https://github.com/expertiza/reimplementation-back-end/pull/325
* https://github.com/expertiza/reimplementation-front-end/pull/166

Demo Video:
* https://youtu.be/TRxBHXo7GQQ

== References ==

* [https://wiki.expertiza.ncsu.edu Expertiza Wiki]
* [https://github.com/expertiza/reimplementation-back-end Reimplementation Backend Repository]
* [https://github.com/expertiza/reimplementation-front-end Reimplementation Frontend Repository]

File:E2605 rubric reset sequence.png

2026-04-29T02:00:06Z

Vagarwa3:

File:E2605 rubric lookup sequence.png

2026-04-29T01:59:42Z

Vagarwa3:

File:E2605 class diagram.png

2026-04-29T01:57:41Z

Vagarwa3:

CSC/ECE 517 Spring 2026 - E2605. Specialized Rubrics for Different Topic Types

2026-04-29T01:56:08Z

Vagarwa3:

__TOC__

== Introduction ==

This wiki page describes the design and implementation for Expertiza project '''E2605: Specialized Rubrics for Different Topic Types'''.

The goal of this project is to let an instructor use different '''review rubrics''' for different '''project topics''' in the same assignment.

Before this project, Expertiza could use:

* one review rubric for the whole assignment, or
* different review rubrics for different review rounds.

But it could not use different review rubrics for different topics in the same assignment.

This project fixes that problem.

== Project Overview ==

Some assignments have topics that are very different from each other.

For example:

* one topic may be about security
* one topic may be about user interface design
* one topic may be about machine learning

These topics should not always be reviewed with the same rubric.

A security topic may need questions about:

* threat modeling
* risk analysis
* vulnerability handling

A user interface topic may need questions about:

* usability
* layout
* design quality

This project lets the system choose the right review rubric based on the topic of the team being reviewed.

== Problem Statement ==

Expertiza already knows:

* which team is being reviewed
* which topic that team signed up for
* which questionnaires are attached to the assignment

But before this project, Expertiza did not store which review rubric should be used for each topic.

Because of that:

* reviewers could receive a rubric that did not match the topic
* instructors could not configure topic-based review rubrics in the assignment editor
* changing rubric mappings after reviews started could leave old reviews inconsistent with the new rubric

== Design Goals ==

The design goals for this project are:

* let instructors assign different review rubrics to different topics
* keep the current review workflow the same for reviewers
* keep old assignments working the same way unless topic variation is enabled
* work correctly with round-based rubric variation
* use the existing database structure as much as possible
* keep rubric selection clear and predictable
* safely handle rubric changes after reviews have already started

== Non-Goals ==

This project does not:

* redesign topic signup
* redesign waitlist behavior
* redesign team formation
* redesign review mapping
* redesign questionnaire authoring from scratch

It only changes how the system stores and finds the correct review rubric.

== Main Idea ==

The main idea of the project is:

# Start from the review response map.
# Find the team being reviewed.
# Find that team’s confirmed topic signup.
# Find the correct review rubric for that assignment, topic, and round.

In simple form, the lookup path is:

<code>ReviewResponseMap -> reviewee team -> signed_up_teams -> topic -> assignment_questionnaires -> questionnaire</code>

== Important Models ==

The models used in this feature are:

* '''Assignment'''
** stores assignment settings
** now stores whether rubric varies by topic
* '''ProjectTopic'''
** stores topics for an assignment
* '''SignedUpTeam'''
** stores which topic a team selected
* '''AssignmentQuestionnaire'''
** stores which questionnaire is used for an assignment
** now can also store which topic that mapping belongs to
* '''ReviewResponseMap'''
** stores who reviews whom
* '''Response'''
** stores the actual review response
* '''ReviewResetArchive'''
** stores old reviews when rubric changes make them invalid

== UML Diagram ==

The diagram below shows the main models and relationships used in topic-based rubric selection.

[[File:E2605_class_diagram.png|800px|center|Class diagram for topic-based rubric selection]]

== Sequence Diagram ==

The diagram below shows how Expertiza finds the correct review rubric when a reviewer opens a review.

[[File:E2605_rubric_lookup_sequence.png|800px|center|Sequence diagram for topic-based rubric lookup]]

== Sequence Diagram for Rubric Change Handling ==

The diagram below shows what happens when an instructor changes a rubric or a topic-rubric mapping after reviews have already started.

[[File:E2605_rubric_reset_sequence.png|800px|center|Sequence diagram for archive, notify, and reset flow]]

== Original Design ==

The original design was based on a small change to the current rubric system.

=== Original Schema Design ===

The original plan was to add:

* <code>assignments.vary_by_topic</code>
* <code>assignment_questionnaires.project_topic_id</code>

The meaning was:

* <code>project_topic_id = NULL</code> means the mapping is the default rubric for the assignment
* <code>project_topic_id = some topic id</code> means the mapping is for that topic

=== Original Lookup Design ===

The original lookup order was:

# topic + current round
# topic + default round
# default topic + current round
# default topic + default round

Here:

* '''default topic''' means <code>project_topic_id IS NULL</code>
* '''default round''' means <code>used_in_round IS NULL</code>

This keeps lookup clear when both topic variation and round variation are enabled.

=== Original UI Design ===

The original UI plan was:

* instructor enables topic-based review rubric variation
* instructor chooses topic-specific review rubrics in the assignment editor
* if round variation is enabled, the instructor can also choose rubrics by topic and round

== What We Implemented ==

The final implementation follows the original design, but also adds some extra support that was needed to make the feature work in the real system.

{| class="wikitable" style="width:100%;"
|-
! Area
! Original Design
! What We Implemented
|-
| Assignment setting
| Add topic variation flag
| Added <code>vary_by_topic</code> to <code>assignments</code>
|-
| Rubric mapping
| Add topic id to assignment-questionnaire mapping
| Added <code>project_topic_id</code> to <code>assignment_questionnaires</code>
|-
| Rubric lookup
| Find rubric from topic and round
| Added shared lookup logic in <code>TopicBasedRubricLookup</code>
|-
| Instructor UI
| Configure topic rubrics in assignment editor
| Added topic rubric controls in the React assignment editor
|-
| Reload existing data
| Not described in detail
| Added serializer and frontend transformation logic so saved mappings reload correctly
|-
| Reviewer flow
| Reviewer gets correct rubric automatically
| Backend resolves rubric and frontend shows the resolved rubric information
|-
| Rubric changes after reviews start
| Mentioned as a concern
| Implemented archive, email notification, and review reset logic
|}

== Schema Changes ==

=== 1. <code>assignments.vary_by_topic</code> ===

A new boolean field was added to the assignments table.

<syntaxhighlight lang="ruby">
add_column :assignments, :vary_by_topic, :boolean, default: false, null: false
</syntaxhighlight>

This means:

* old assignments still work the same way
* topic-based rubric selection happens only when the instructor enables it

=== 2. <code>assignment_questionnaires.project_topic_id</code> ===

A new nullable topic reference was added to <code>assignment_questionnaires</code>.

<syntaxhighlight lang="ruby">
add_reference :assignment_questionnaires, :project_topic, foreign_key: true, null: true
</syntaxhighlight>

This lets the same table store:

* default assignment review rubric
* topic-specific review rubric
* round-based review rubric
* topic-and-round-specific review rubric

=== 3. Indexes ===

Indexes were added so the system can quickly find the correct rubric.

The most important lookup index is on:

* <code>assignment_id</code>
* <code>project_topic_id</code>
* <code>used_in_round</code>

=== 4. Review Archive Table ===

A new table was added:

* <code>review_reset_archives</code>

This table stores old reviews when a rubric change makes those reviews no longer valid.

It stores:

* response id
* response map id
* assignment id
* questionnaire id
* topic id
* round
* reviewer id
* reviewee id
* reset reason
* snapshot data

== Rubric Lookup Design ==

The final lookup logic is placed in a shared concern:

* <code>app/models/concerns/topic_based_rubric_lookup.rb</code>

This concern is included in the <code>Assignment</code> model.

=== Main Lookup Method ===

The main method is:

* <code>assignment_questionnaire_for_response_map(response_map, round:)</code>

This method is used to find the correct rubric for a review.

=== Reviewee Topic Lookup ===

When the response map is a <code>ReviewResponseMap</code>, the system does this:

# finds the reviewee team
# finds that team’s confirmed topic signup
# ignores waitlisted signups
# uses that topic id in rubric lookup

The method used is:

* <code>confirmed_topic_id_for(reviewee)</code>

=== Lookup Order ===

The final lookup order is still:

# topic + current round
# topic + default round
# assignment default topic + current round
# assignment default topic + default round

This keeps rubric selection simple and predictable.

== Original vs Final Lookup Code ==

{| class="wikitable" style="width:100%;"
|-
! Before
! After
|-
| Topic-based lookup did not exist.
| <code>Assignment</code> now includes <code>TopicBasedRubricLookup</code> and uses topic-aware rubric resolution.
|}

=== Final Lookup Code ===

<syntaxhighlight lang="ruby">
def assignment_questionnaire_for_response_map(response_map, round:)
return assignment_questionnaires.find_by(used_in_round: round) unless response_map.is_a?(ReviewResponseMap)

review_assignment_questionnaire(response_map, round) || assignment_questionnaires.find_by(used_in_round: round)
end
</syntaxhighlight>

== Model Validation ==

The <code>AssignmentQuestionnaire</code> model now validates two important things.

=== 1. Topic Must Belong to the Same Assignment ===

<syntaxhighlight lang="ruby">
def project_topic_belongs_to_assignment
return if project_topic_id.blank? || assignment_id.blank?
return if project_topic&.assignment_id == assignment_id

errors.add(:project_topic, 'must belong to the same assignment')
end
</syntaxhighlight>

This prevents a topic from one assignment from being used in another assignment’s rubric mapping.

=== 2. No Duplicate Review Mapping for the Same Assignment, Topic, and Round ===

<syntaxhighlight lang="ruby">
def unique_review_rubric_for_assignment_topic_round
return unless review_questionnaire?

duplicate = AssignmentQuestionnaire
.joins(:questionnaire)
.where(
assignment_id: assignment_id,
project_topic_id: project_topic_id,
used_in_round: used_in_round,
questionnaires: { questionnaire_type: 'ReviewQuestionnaire' }
)
duplicate = duplicate.where.not(id: id) if persisted?

errors.add(:base, 'review rubric already exists for this assignment, topic, and round') if duplicate.exists?
end
</syntaxhighlight>

This makes sure lookup returns one correct rubric.

== Backend Controller and API Changes ==

=== Assignment Save and Reload ===

To support this feature, the assignment edit flow had to save and reload more data.

The backend now permits:

* <code>vary_by_topic</code>
* <code>vary_by_round</code>
* <code>rounds_of_reviews</code>
* nested <code>assignment_questionnaires_attributes</code>

The <code>Assignment</code> model also now accepts nested questionnaire mappings:

<syntaxhighlight lang="ruby">
accepts_nested_attributes_for :assignment_questionnaires, allow_destroy: true
</syntaxhighlight>

=== Assignment Serializer ===

The assignment serializer now returns:

* <code>vary_by_topic</code>
* <code>assignment_questionnaires</code>
* <code>due_dates</code>
* <code>num_review_rounds</code>

This is needed so the frontend can reopen an assignment and show the saved topic and round rubric settings.

=== Topic Rubric Mapping API ===

The final implementation adds a dedicated controller for topic-rubric mappings:

* <code>AssignmentQuestionnairesController#index</code>
* <code>AssignmentQuestionnairesController#create</code>
* <code>AssignmentQuestionnairesController#update</code>
* <code>AssignmentQuestionnairesController#destroy</code>

This API is used by the React assignment editor.

=== Reviewer Rubric Lookup API ===

The backend also adds:

* <code>StudentTasksController#rubric_for</code>

This endpoint returns:

* assignment questionnaire id
* questionnaire id
* questionnaire name
* topic id
* round

This gives the frontend a way to ask: “Which rubric did the backend choose for this review?”

== Original vs Final API Design ==

{| class="wikitable" style="width:100%;"
|-
! Before
! After
|-
| No API for topic-rubric mappings.
| Added CRUD API in <code>AssignmentQuestionnairesController</code>.
|-
| No endpoint to ask which rubric was resolved for a review.
| Added <code>/student_tasks/rubric_for</code>.
|}

== Frontend Design ==

The topic-based rubric feature is configured in the React assignment editor.

Main frontend files are:

* <code>src/pages/Assignments/AssignmentUtil.ts</code>
* <code>src/pages/Assignments/AssignmentEditor.tsx</code>
* <code>src/pages/Assignments/tabs/TopicsTab.tsx</code>

=== Instructor Workflow ===

The instructor workflow is:

# open assignment editor
# enable topic-based rubric variation
# optionally enable round-based rubric variation
# go to Topics tab
# choose the review rubric for each topic
# save the assignment

If round variation is also enabled, the instructor can choose a rubric for each topic and round.

=== Request Transformation ===

The frontend now maps the form fields back to backend assignment fields.

Example:

<syntaxhighlight lang="typescript">
vary_by_round: values.review_rubric_varies_by_round,
vary_by_topic: values.review_rubric_varies_by_topic,
rounds_of_reviews: values.number_of_review_rounds,
assignment_questionnaires_attributes: assignmentQuestionnaires,
</syntaxhighlight>

=== Response Transformation ===

When loading an assignment, the frontend now restores:

* whether rubric varies by topic
* whether rubric varies by round
* number of review rounds
* saved assignment-questionnaire mappings
* due dates

This allows saved rubric settings to appear again when editing the assignment later.

== Original vs Final Frontend Design ==

{| class="wikitable" style="width:100%;"
|-
! Before
! After
|-
| No topic-rubric controls in assignment editor.
| Added topic-rubric controls in the Topics tab.
|-
| Assignment editor did not save topic rubric settings.
| Assignment editor now saves <code>vary_by_topic</code> and topic-based mappings.
|-
| Assignment editor did not reload topic rubric settings cleanly.
| Serializer and frontend transformation now reload saved mappings.
|}

== Questionnaire UI Support ==

The original design assumed instructors could already create the review rubrics they needed.

In practice, the React questionnaire UI needed some fixes.

The final implementation adds:

* correct mapping for the Review questionnaire type
* working edit navigation path for questionnaires
* item type loading from <code>/questions/types</code>
* fallback item type list when needed

This was important because the feature is only useful if instructors can create review rubrics in the frontend.

== Review Reset and Archive Design ==

== Why This Was Needed ==

If an instructor changes a rubric after reviews have already started, the old reviews may no longer match the new rubric.

For example:

* a topic is mapped to a different review rubric
* a review questionnaire is edited
* a question inside a review questionnaire is added, updated, or deleted

In these cases, the old completed reviews should not stay active as if nothing changed.

But at the same time, the old student work should not be lost.

== Final Reset Design ==

The final implementation adds a shared concern:

* <code>app/controllers/concerns/review_reset_handler.rb</code>

This concern does four things:

# find the affected reviews
# archive the old review data
# notify reviewers that they must redo the review
# delete the old live responses

== Review Archive ==

The archived review stores:

* response details
* questionnaire details
* reviewer and reviewee ids
* topic id
* round
* comments
* answers
* item text
* item type
* scores
* reset reason

This means the system keeps the student’s old completed work in the database.

== Reset Cases Implemented ==

The following changes now trigger archive + reset:

* topic-rubric mapping created
* topic-rubric mapping updated
* topic-rubric mapping deleted
* review questionnaire updated
* question created in a review questionnaire
* question updated in a review questionnaire
* question deleted from a review questionnaire
* all questions deleted from a review questionnaire

== Original vs Final Review-Reset Design ==

{| class="wikitable" style="width:100%;"
|-
! Original Design
! Final Implementation
|-
| Mentioned that old reviews should not stay valid after rubric changes.
| Fully implemented archive + email + reset behavior.
|-
| Did not say where old reviews would be stored.
| Added <code>review_reset_archives</code> table with full snapshot data.
|}

== Reviewer View ==

The reviewer does not pick a rubric manually.

Instead:

# reviewer opens a review
# frontend sends <code>response_map_id</code> and optional round
# backend finds the correct rubric
# frontend receives the resolved rubric details

The React review page now shows:

* resolved questionnaire name
* topic id, if topic-specific
* round, if round-specific

This gives clear proof that the backend selected a rubric.

== Known Limitation ==

The current review page does '''not yet''' fully render the actual questionnaire items from the resolved rubric.

So:

* the backend does resolve the correct rubric
* the frontend does show which rubric was resolved
* but the review page still uses mostly static review content

This is a known limit of the current implementation.

== Testing ==

=== Backend Tests ===

Backend tests were added for:

* topic-rubric mapping API
* topic-aware rubric lookup
* archive and reset behavior after rubric changes

Main backend test files include:

* <code>spec/controllers/assignment_questionnaires_controller_spec.rb</code>
* <code>spec/controllers/questionnaires_controller_spec.rb</code>
* <code>spec/controllers/student_tasks_controller_spec.rb</code>
* <code>spec/models/assignment_spec.rb</code>

=== Frontend Tests ===

Frontend tests were added for:

* assignment request mapping for <code>vary_by_topic</code>
* assignment response mapping for <code>vary_by_topic</code>
* topic tab rubric behavior
* questionnaire type mapping

Main frontend test files include:

* <code>src/pages/Assignments/AssignmentEditor.test.tsx</code>
* <code>src/pages/Assignments/tabs/TopicsTab.test.tsx</code>
* <code>src/pages/Questionnaires/QuestionnaireUtils.test.ts</code>

=== Manual Testing ===

The following manual cases were also checked:

* topic-based rubric selection can be saved
* saved topic-rubric mappings reload correctly
* topic and round fallback order works
* waitlisted signups do not affect rubric lookup
* old completed reviews are archived when the rubric changes
* reviewers are notified and old live reviews are reset

== Risks ==

The main risks in this project were:

* unclear rubric precedence
* incomplete topic-rubric setup
* mixing topic-based and round-based rules
* losing student work when rubrics change

These were handled by:

* a strict lookup order
* validation in the mapping model
* safe fallback behavior
* review archiving before reset

== Known Limits ==

The current implementation still has some limits:

* the review page does not yet fully render the real questionnaire items dynamically
* archived reviews are stored in the database, but there is no frontend page to view them
* the current main-branch implementation allows one review rubric per topic per round

== Design Principles Used ==

The project uses these design ideas:

* '''Single Responsibility Principle'''
** rubric lookup is placed in <code>TopicBasedRubricLookup</code>
** review reset logic is placed in <code>ReviewResetHandler</code>
** assignment request/response conversion is placed in frontend utility code

* '''Separation of Concerns'''
** backend decides which rubric to use
** frontend lets the instructor configure topic mappings
** model validation keeps data safe

* '''Backward Compatibility'''
** assignments that do not enable topic variation still work the old way

* '''Deterministic Behavior'''
** the same topic and round always follow the same lookup order

== Team ==

=== Mentor ===

* Ed Gehringer

=== Team Members ===

* Vanaja Agarwal
* Himashu Agarwal
* Bhargavi Jadhav

== Links ==

Pull Requests:
* https://github.com/expertiza/reimplementation-back-end/pull/325
* https://github.com/expertiza/reimplementation-front-end/pull/166

Demo Video:
* Add demo video link here

== References ==

* [https://wiki.expertiza.ncsu.edu Expertiza Wiki]
* [https://github.com/expertiza/reimplementation-back-end Reimplementation Backend Repository]
* [https://github.com/expertiza/reimplementation-front-end Reimplementation Frontend Repository]

CSC/ECE 517 Spring 2026 - E2605. Specialized Rubrics for Different Topic Types

2026-04-29T01:44:19Z

Vagarwa3:

__TOC__

== Introduction ==

This wiki page describes the design and implementation for Expertiza project '''E2605: Specialized Rubrics for Different Topic Types'''.

The goal of this project is to let an instructor use different '''review rubrics''' for different '''project topics''' in the same assignment.

Before this project, Expertiza could use:

* one review rubric for the whole assignment, or
* different review rubrics for different review rounds.

But it could not use different review rubrics for different topics in the same assignment.

This project fixes that problem.

== Project Overview ==

Some assignments have topics that are very different from each other.

For example:

* one topic may be about security
* one topic may be about user interface design
* one topic may be about machine learning

These topics should not always be reviewed with the same rubric.

A security topic may need questions about:

* threat modeling
* risk analysis
* vulnerability handling

A user interface topic may need questions about:

* usability
* layout
* design quality

This project lets the system choose the right review rubric based on the topic of the team being reviewed.

== Problem Statement ==

Expertiza already knows:

* which team is being reviewed
* which topic that team signed up for
* which questionnaires are attached to the assignment

But before this project, Expertiza did not store which review rubric should be used for each topic.

Because of that:

* reviewers could receive a rubric that did not match the topic
* instructors could not configure topic-based review rubrics in the assignment editor
* changing rubric mappings after reviews started could leave old reviews inconsistent with the new rubric

== Design Goals ==

The design goals for this project are:

* let instructors assign different review rubrics to different topics
* keep the current review workflow the same for reviewers
* keep old assignments working the same way unless topic variation is enabled
* work correctly with round-based rubric variation
* use the existing database structure as much as possible
* keep rubric selection clear and predictable
* safely handle rubric changes after reviews have already started

== Non-Goals ==

This project does not:

* redesign topic signup
* redesign waitlist behavior
* redesign team formation
* redesign review mapping
* redesign questionnaire authoring from scratch

It only changes how the system stores and finds the correct review rubric.

== Main Idea ==

The main idea of the project is:

# Start from the review response map.
# Find the team being reviewed.
# Find that team’s confirmed topic signup.
# Find the correct review rubric for that assignment, topic, and round.

In simple form, the lookup path is:

<code>ReviewResponseMap -> reviewee team -> signed_up_teams -> topic -> assignment_questionnaires -> questionnaire</code>

== Important Models ==

The models used in this feature are:

* '''Assignment'''
** stores assignment settings
** now stores whether rubric varies by topic
* '''ProjectTopic'''
** stores topics for an assignment
* '''SignedUpTeam'''
** stores which topic a team selected
* '''AssignmentQuestionnaire'''
** stores which questionnaire is used for an assignment
** now can also store which topic that mapping belongs to
* '''ReviewResponseMap'''
** stores who reviews whom
* '''Response'''
** stores the actual review response
* '''ReviewResetArchive'''
** stores old reviews when rubric changes make them invalid

== UML Diagram ==

Add the final class diagram here.

[[File:E2605_class_diagram.png|800px|center|Class diagram for topic-based rubric selection]]

== Sequence Diagram ==

Add the final sequence diagram here.

Suggested sequence:

* reviewer opens a review
* frontend sends <code>response_map_id</code> and round to backend
* backend finds the reviewee team
* backend finds the team’s confirmed topic
* backend finds the correct assignment questionnaire
* backend returns the resolved rubric

[[File:E2605_sequence_diagram.png|800px|center|Sequence diagram for rubric lookup]]

== Original Design ==

The original design was based on a small change to the current rubric system.

=== Original Schema Design ===

The original plan was to add:

* <code>assignments.vary_by_topic</code>
* <code>assignment_questionnaires.project_topic_id</code>

The meaning was:

* <code>project_topic_id = NULL</code> means the mapping is the default rubric for the assignment
* <code>project_topic_id = some topic id</code> means the mapping is for that topic

=== Original Lookup Design ===

The original lookup order was:

# topic + current round
# topic + default round
# default topic + current round
# default topic + default round

Here:

* '''default topic''' means <code>project_topic_id IS NULL</code>
* '''default round''' means <code>used_in_round IS NULL</code>

This keeps lookup clear when both topic variation and round variation are enabled.

=== Original UI Design ===

The original UI plan was:

* instructor enables topic-based review rubric variation
* instructor chooses topic-specific review rubrics in the assignment editor
* if round variation is enabled, the instructor can also choose rubrics by topic and round

== What We Implemented ==

The final implementation follows the original design, but also adds some extra support that was needed to make the feature work in the real system.

{| class="wikitable" style="width:100%;"
|-
! Area
! Original Design
! What We Implemented
|-
| Assignment setting
| Add topic variation flag
| Added <code>vary_by_topic</code> to <code>assignments</code>
|-
| Rubric mapping
| Add topic id to assignment-questionnaire mapping
| Added <code>project_topic_id</code> to <code>assignment_questionnaires</code>
|-
| Rubric lookup
| Find rubric from topic and round
| Added shared lookup logic in <code>TopicBasedRubricLookup</code>
|-
| Instructor UI
| Configure topic rubrics in assignment editor
| Added topic rubric controls in the React assignment editor
|-
| Reload existing data
| Not described in detail
| Added serializer and frontend transformation logic so saved mappings reload correctly
|-
| Reviewer flow
| Reviewer gets correct rubric automatically
| Backend resolves rubric and frontend shows the resolved rubric information
|-
| Rubric changes after reviews start
| Mentioned as a concern
| Implemented archive, email notification, and review reset logic
|}

== Schema Changes ==

=== 1. <code>assignments.vary_by_topic</code> ===

A new boolean field was added to the assignments table.

<syntaxhighlight lang="ruby">
add_column :assignments, :vary_by_topic, :boolean, default: false, null: false
</syntaxhighlight>

This means:

* old assignments still work the same way
* topic-based rubric selection happens only when the instructor enables it

=== 2. <code>assignment_questionnaires.project_topic_id</code> ===

A new nullable topic reference was added to <code>assignment_questionnaires</code>.

<syntaxhighlight lang="ruby">
add_reference :assignment_questionnaires, :project_topic, foreign_key: true, null: true
</syntaxhighlight>

This lets the same table store:

* default assignment review rubric
* topic-specific review rubric
* round-based review rubric
* topic-and-round-specific review rubric

=== 3. Indexes ===

Indexes were added so the system can quickly find the correct rubric.

The most important lookup index is on:

* <code>assignment_id</code>
* <code>project_topic_id</code>
* <code>used_in_round</code>

=== 4. Review Archive Table ===

A new table was added:

* <code>review_reset_archives</code>

This table stores old reviews when a rubric change makes those reviews no longer valid.

It stores:

* response id
* response map id
* assignment id
* questionnaire id
* topic id
* round
* reviewer id
* reviewee id
* reset reason
* snapshot data

== Rubric Lookup Design ==

The final lookup logic is placed in a shared concern:

* <code>app/models/concerns/topic_based_rubric_lookup.rb</code>

This concern is included in the <code>Assignment</code> model.

=== Main Lookup Method ===

The main method is:

* <code>assignment_questionnaire_for_response_map(response_map, round:)</code>

This method is used to find the correct rubric for a review.

=== Reviewee Topic Lookup ===

When the response map is a <code>ReviewResponseMap</code>, the system does this:

# finds the reviewee team
# finds that team’s confirmed topic signup
# ignores waitlisted signups
# uses that topic id in rubric lookup

The method used is:

* <code>confirmed_topic_id_for(reviewee)</code>

=== Lookup Order ===

The final lookup order is still:

# topic + current round
# topic + default round
# assignment default topic + current round
# assignment default topic + default round

This keeps rubric selection simple and predictable.

== Original vs Final Lookup Code ==

{| class="wikitable" style="width:100%;"
|-
! Before
! After
|-
| Topic-based lookup did not exist.
| <code>Assignment</code> now includes <code>TopicBasedRubricLookup</code> and uses topic-aware rubric resolution.
|}

=== Final Lookup Code ===

<syntaxhighlight lang="ruby">
def assignment_questionnaire_for_response_map(response_map, round:)
return assignment_questionnaires.find_by(used_in_round: round) unless response_map.is_a?(ReviewResponseMap)

review_assignment_questionnaire(response_map, round) || assignment_questionnaires.find_by(used_in_round: round)
end
</syntaxhighlight>

== Model Validation ==

The <code>AssignmentQuestionnaire</code> model now validates two important things.

=== 1. Topic Must Belong to the Same Assignment ===

<syntaxhighlight lang="ruby">
def project_topic_belongs_to_assignment
return if project_topic_id.blank? || assignment_id.blank?
return if project_topic&.assignment_id == assignment_id

errors.add(:project_topic, 'must belong to the same assignment')
end
</syntaxhighlight>

This prevents a topic from one assignment from being used in another assignment’s rubric mapping.

=== 2. No Duplicate Review Mapping for the Same Assignment, Topic, and Round ===

<syntaxhighlight lang="ruby">
def unique_review_rubric_for_assignment_topic_round
return unless review_questionnaire?

duplicate = AssignmentQuestionnaire
.joins(:questionnaire)
.where(
assignment_id: assignment_id,
project_topic_id: project_topic_id,
used_in_round: used_in_round,
questionnaires: { questionnaire_type: 'ReviewQuestionnaire' }
)
duplicate = duplicate.where.not(id: id) if persisted?

errors.add(:base, 'review rubric already exists for this assignment, topic, and round') if duplicate.exists?
end
</syntaxhighlight>

This makes sure lookup returns one correct rubric.

== Backend Controller and API Changes ==

=== Assignment Save and Reload ===

To support this feature, the assignment edit flow had to save and reload more data.

The backend now permits:

* <code>vary_by_topic</code>
* <code>vary_by_round</code>
* <code>rounds_of_reviews</code>
* nested <code>assignment_questionnaires_attributes</code>

The <code>Assignment</code> model also now accepts nested questionnaire mappings:

<syntaxhighlight lang="ruby">
accepts_nested_attributes_for :assignment_questionnaires, allow_destroy: true
</syntaxhighlight>

=== Assignment Serializer ===

The assignment serializer now returns:

* <code>vary_by_topic</code>
* <code>assignment_questionnaires</code>
* <code>due_dates</code>
* <code>num_review_rounds</code>

This is needed so the frontend can reopen an assignment and show the saved topic and round rubric settings.

=== Topic Rubric Mapping API ===

The final implementation adds a dedicated controller for topic-rubric mappings:

* <code>AssignmentQuestionnairesController#index</code>
* <code>AssignmentQuestionnairesController#create</code>
* <code>AssignmentQuestionnairesController#update</code>
* <code>AssignmentQuestionnairesController#destroy</code>

This API is used by the React assignment editor.

=== Reviewer Rubric Lookup API ===

The backend also adds:

* <code>StudentTasksController#rubric_for</code>

This endpoint returns:

* assignment questionnaire id
* questionnaire id
* questionnaire name
* topic id
* round

This gives the frontend a way to ask: “Which rubric did the backend choose for this review?”

== Original vs Final API Design ==

{| class="wikitable" style="width:100%;"
|-
! Before
! After
|-
| No API for topic-rubric mappings.
| Added CRUD API in <code>AssignmentQuestionnairesController</code>.
|-
| No endpoint to ask which rubric was resolved for a review.
| Added <code>/student_tasks/rubric_for</code>.
|}

== Frontend Design ==

The topic-based rubric feature is configured in the React assignment editor.

Main frontend files are:

* <code>src/pages/Assignments/AssignmentUtil.ts</code>
* <code>src/pages/Assignments/AssignmentEditor.tsx</code>
* <code>src/pages/Assignments/tabs/TopicsTab.tsx</code>

=== Instructor Workflow ===

The instructor workflow is:

# open assignment editor
# enable topic-based rubric variation
# optionally enable round-based rubric variation
# go to Topics tab
# choose the review rubric for each topic
# save the assignment

If round variation is also enabled, the instructor can choose a rubric for each topic and round.

=== Request Transformation ===

The frontend now maps the form fields back to backend assignment fields.

Example:

<syntaxhighlight lang="typescript">
vary_by_round: values.review_rubric_varies_by_round,
vary_by_topic: values.review_rubric_varies_by_topic,
rounds_of_reviews: values.number_of_review_rounds,
assignment_questionnaires_attributes: assignmentQuestionnaires,
</syntaxhighlight>

=== Response Transformation ===

When loading an assignment, the frontend now restores:

* whether rubric varies by topic
* whether rubric varies by round
* number of review rounds
* saved assignment-questionnaire mappings
* due dates

This allows saved rubric settings to appear again when editing the assignment later.

== Original vs Final Frontend Design ==

{| class="wikitable" style="width:100%;"
|-
! Before
! After
|-
| No topic-rubric controls in assignment editor.
| Added topic-rubric controls in the Topics tab.
|-
| Assignment editor did not save topic rubric settings.
| Assignment editor now saves <code>vary_by_topic</code> and topic-based mappings.
|-
| Assignment editor did not reload topic rubric settings cleanly.
| Serializer and frontend transformation now reload saved mappings.
|}

== Questionnaire UI Support ==

The original design assumed instructors could already create the review rubrics they needed.

In practice, the React questionnaire UI needed some fixes.

The final implementation adds:

* correct mapping for the Review questionnaire type
* working edit navigation path for questionnaires
* item type loading from <code>/questions/types</code>
* fallback item type list when needed

This was important because the feature is only useful if instructors can create review rubrics in the frontend.

== Review Reset and Archive Design ==

== Why This Was Needed ==

If an instructor changes a rubric after reviews have already started, the old reviews may no longer match the new rubric.

For example:

* a topic is mapped to a different review rubric
* a review questionnaire is edited
* a question inside a review questionnaire is added, updated, or deleted

In these cases, the old completed reviews should not stay active as if nothing changed.

But at the same time, the old student work should not be lost.

== Final Reset Design ==

The final implementation adds a shared concern:

* <code>app/controllers/concerns/review_reset_handler.rb</code>

This concern does four things:

# find the affected reviews
# archive the old review data
# notify reviewers that they must redo the review
# delete the old live responses

== Review Archive ==

The archived review stores:

* response details
* questionnaire details
* reviewer and reviewee ids
* topic id
* round
* comments
* answers
* item text
* item type
* scores
* reset reason

This means the system keeps the student’s old completed work in the database.

== Reset Cases Implemented ==

The following changes now trigger archive + reset:

* topic-rubric mapping created
* topic-rubric mapping updated
* topic-rubric mapping deleted
* review questionnaire updated
* question created in a review questionnaire
* question updated in a review questionnaire
* question deleted from a review questionnaire
* all questions deleted from a review questionnaire

== Original vs Final Review-Reset Design ==

{| class="wikitable" style="width:100%;"
|-
! Original Design
! Final Implementation
|-
| Mentioned that old reviews should not stay valid after rubric changes.
| Fully implemented archive + email + reset behavior.
|-
| Did not say where old reviews would be stored.
| Added <code>review_reset_archives</code> table with full snapshot data.
|}

== Reviewer View ==

The reviewer does not pick a rubric manually.

Instead:

# reviewer opens a review
# frontend sends <code>response_map_id</code> and optional round
# backend finds the correct rubric
# frontend receives the resolved rubric details

The React review page now shows:

* resolved questionnaire name
* topic id, if topic-specific
* round, if round-specific

This gives clear proof that the backend selected a rubric.

== Known Limitation ==

The current review page does '''not yet''' fully render the actual questionnaire items from the resolved rubric.

So:

* the backend does resolve the correct rubric
* the frontend does show which rubric was resolved
* but the review page still uses mostly static review content

This is a known limit of the current implementation.

== Testing ==

=== Backend Tests ===

Backend tests were added for:

* topic-rubric mapping API
* topic-aware rubric lookup
* archive and reset behavior after rubric changes

Main backend test files include:

* <code>spec/controllers/assignment_questionnaires_controller_spec.rb</code>
* <code>spec/controllers/questionnaires_controller_spec.rb</code>
* <code>spec/controllers/student_tasks_controller_spec.rb</code>
* <code>spec/models/assignment_spec.rb</code>

=== Frontend Tests ===

Frontend tests were added for:

* assignment request mapping for <code>vary_by_topic</code>
* assignment response mapping for <code>vary_by_topic</code>
* topic tab rubric behavior
* questionnaire type mapping

Main frontend test files include:

* <code>src/pages/Assignments/AssignmentEditor.test.tsx</code>
* <code>src/pages/Assignments/tabs/TopicsTab.test.tsx</code>
* <code>src/pages/Questionnaires/QuestionnaireUtils.test.ts</code>

=== Manual Testing ===

The following manual cases were also checked:

* topic-based rubric selection can be saved
* saved topic-rubric mappings reload correctly
* topic and round fallback order works
* waitlisted signups do not affect rubric lookup
* old completed reviews are archived when the rubric changes
* reviewers are notified and old live reviews are reset

== Risks ==

The main risks in this project were:

* unclear rubric precedence
* incomplete topic-rubric setup
* mixing topic-based and round-based rules
* losing student work when rubrics change

These were handled by:

* a strict lookup order
* validation in the mapping model
* safe fallback behavior
* review archiving before reset

== Known Limits ==

The current implementation still has some limits:

* the review page does not yet fully render the real questionnaire items dynamically
* archived reviews are stored in the database, but there is no frontend page to view them
* the current main-branch implementation allows one review rubric per topic per round

== Design Principles Used ==

The project uses these design ideas:

* '''Single Responsibility Principle'''
** rubric lookup is placed in <code>TopicBasedRubricLookup</code>
** review reset logic is placed in <code>ReviewResetHandler</code>
** assignment request/response conversion is placed in frontend utility code

* '''Separation of Concerns'''
** backend decides which rubric to use
** frontend lets the instructor configure topic mappings
** model validation keeps data safe

* '''Backward Compatibility'''
** assignments that do not enable topic variation still work the old way

* '''Deterministic Behavior'''
** the same topic and round always follow the same lookup order

== Team ==

=== Mentor ===

* Ed Gehringer

=== Team Members ===

* Vanaja Agarwal
* Himashu Agarwal
* Bhargavi Jadhav

== Links ==

Pull Requests:
* https://github.com/expertiza/reimplementation-back-end/pull/325
* https://github.com/expertiza/reimplementation-front-end/pull/166

Demo Video:
* Add demo video link here

== References ==

* [https://wiki.expertiza.ncsu.edu Expertiza Wiki]
* [https://github.com/expertiza/reimplementation-back-end Reimplementation Backend Repository]
* [https://github.com/expertiza/reimplementation-front-end Reimplementation Frontend Repository]

CSC/ECE 517 Spring 2026 - E2605. Specialized Rubrics for Different Topic Types

2026-04-15T23:59:20Z

Vagarwa3: /* API and Validation */

__TOC__

== Introduction ==

This wiki page presents the design for Expertiza project '''E2605: Specialized Rubrics for Different Topic Types''' for CSC/ECE 517 Object-Oriented Design and Development, Spring 2026.

The goal of this project is to let instructors use different review rubrics for different project topics within the same assignment. Expertiza currently uses the same review rubric for all submissions in an assignment, unless the assignment is configured to use different rubrics in different review rounds. This works for assignments where every topic should be evaluated with the same criteria, but it does not work well when topics require different kinds of evaluation.

For example, an assignment might include topics related to user-interface design, machine learning, and security. A security topic may need rubric questions about threat modeling or vulnerability analysis, while a user-interface topic may need rubric questions about usability and design quality. Topic-based rubric selection allows the rubric to match the topic of the submission being reviewed.

This document describes the motivation for the change, the proposed data model, the instructor workflow, and how Expertiza will resolve the correct review rubric based on assignment, topic, and review-round information.

== Overview ==

This design adds topic-based review rubric selection while keeping existing behavior for assignments where topic-based rubric selection is not enabled. Instructors will associate review rubrics with topics directly from the Topics tab of Assignment creation, using dropdowns. If an assignment is configured to vary by topic, the review rubric is chosen based on the topic of the submission that is being reviewed. If topic variation is not enabled, rubric selection follows the current assignment and round logic.

A [[Response maps|ReviewResponseMap]] identifies the team whose submission is being reviewed. The [[Signed up teams|signed_up_teams]] table identifies which topic that team selected. The [[Assignments questionnaires|assignments_questionnaires]] table will then identify which review rubric should be used for that assignment and topic.

== Problem Statement ==

Different project topics can require different evaluation criteria, but currently all submissions for an assignment use the same review rubric unless round-based rubric variation is configured. The current tables do not yet store which review rubric should be used for each topic, and there is no assignment flag for topic-based rubric variation. Because of this, reviewers can receive rubrics that do not match the reviewed team's topic.

== Design Goals ==

The goal is to let instructors define topic-specific review rubrics without changing the overall review workflow. The design must preserve backward compatibility for existing assignments, continue to support round-based variation, and keep rubric resolution deterministic when both round and topic variation are enabled.

== Non-Goals ==

This change does not redesign topic signup or waitlist behavior, questionnaire authoring or import, or review mapping strategies. It only changes how the system stores and resolves rubric selection for reviews.

== Current State ==

The existing data model already includes the relationships needed to determine a reviewee team's topic. The [[Signed up teams|signed_up_teams]] table identifies which team has selected which topic, and review mappings already identify the reviewee team. Rubric resolution currently happens in several places, including response and scoring code paths, and is mainly driven by assignment and round context.

The current tables do not yet store all the information needed for topic-based rubric selection. Assignments currently include <code>vary_by_round</code> but no equivalent <code>vary_by_topic</code> flag, and [[Assignments questionnaires|assignments_questionnaires]] records do not have a <code>project_topic_id</code> foreign key.

== Proposed Data Model ==

This design introduces minimal extensions to the existing schema while reusing the current rubric infrastructure. A new <code>vary_by_topic</code> boolean field is added to the assignments table with a default value of false. This ensures that existing assignments continue to function without any changes, and instructors can explicitly enable topic-based rubric variation only when needed.

The [[Assignments questionnaires|assignments_questionnaires]] table indicates which rubrics are used for an assignment. Adding <code>project_topic_id</code> allows one of those rubric records to apply to a specific topic instead of applying to the whole assignment. A project topic will normally have at most one review-rubric mapping, or one per round if round-based variation is enabled.

In addition, an optional <code>project_topic_id</code> foreign key is added to the [[Assignments questionnaires|assignments_questionnaires]] table. A null <code>project_topic_id</code> represents the default review rubric for the assignment. This approach allows topic-specific mappings to coexist naturally with round-based mappings through <code>used_in_round</code>.

Indexes should be added on the new topic field and on the fields used for rubric lookup so that Expertiza can efficiently find the correct rubric for an assignment, topic, and round.

No new questionnaire type is introduced. The new <code>project_topic_id</code> field only connects an existing assignment-rubric record to a topic, and the rubric itself remains the existing questionnaire referenced by <code>questionnaire_id</code>.

== Rubric Resolution Behavior ==

Rubric selection remains within existing Expertiza code paths. The workflow proceeds by resolving the assignment from the response map, determining whether topic variation is enabled, and, if so, resolving the reviewee team's topic. The system then looks for the best matching assignment questionnaire for that assignment, topic, and round context.

When both round and topic variation are active, matching is deterministic. The precedence is: assignment/topic/current round, then assignment/topic/default round, then assignment/default topic/current round, and finally assignment/default topic/default round. In this precedence rule, the default topic corresponds to <code>project_topic_id</code> being null, and the default round corresponds to <code>used_in_round</code> being null.

If no topic-specific mapping is found, the system falls back according to the defined precedence, using default round-specific rubrics for the assignment when applicable, before using the default review rubric for the assignment.

Only the reviewee team's confirmed topic signup for the current assignment is considered, and waitlisted topic signups do not determine rubric selection.

== UI and Instructor Workflow ==

[[File:E2605architecture.png|800px|center|Figure: Rubric resolution architecture]]

Instructors configure topic-based rubrics directly in assignment setup on the Topics tab. For each topic, the UI exposes dropdowns for selecting the review rubric, and when <code>vary_by_round</code> is enabled, round-specific rubric choices are also supported. This approach keeps all configuration within the standard assignment authoring flow and avoids introducing any external components.

The existing default review rubric for the assignment remains available as the fallback when a topic-specific rubric is not configured. Reviewers do not choose the rubric themselves; instead, they receive the questionnaire resolved automatically for the team they are reviewing.

== API and Validation ==

Assignment create and update flows persist <code>vary_by_topic</code>. Assignment questionnaire updates accept an optional <code>project_topic_id</code> and validate that the topic belongs to the same assignment.

Validation ensures that <code>project_topic_id</code> references an existing topic and does not cross assignment boundaries. The system also prevents duplicate review-rubric mappings for the same assignment, topic, and round combination, ensuring that rubric resolution always produces a single correct result.

Only instructors or administrators who can edit the assignment are allowed to enable topic-based rubric variation or modify topic-rubric mappings.

== Testing Strategy ==

Tests verify that assignments without topic variation remain unchanged, while topic-enabled assignments correctly select topic-specific rubrics when configured. They also ensure that fallback behavior functions properly when a team has no topic or no topic-specific rubric mapping.

Additional tests confirm deterministic precedence when both topic and round variation are enabled and ensure that scoring behavior remains consistent with the selected questionnaire.

== Risks ==

The main risk is ambiguity in precedence when both topic and round variation are present. This is mitigated by defining a strict lookup order and enforcing it through testing. Another risk is partial configuration, where topic variation is enabled but mappings are incomplete; this is mitigated by safe fallback behavior and validation in configuration paths.

== Rubric Changes After Reviews Begin ==

If an instructor changes a topic-rubric mapping after reviews have already begun, existing reviews that used the previous rubric should not remain valid. Since the rubric determines the questions and scoring criteria for a review, changing the rubric can make previously submitted reviews inconsistent with the new configuration.

When a topic-rubric mapping is changed after reviews have been performed, the affected reviews should be deleted, and the affected reviewers should be notified that they need to perform the reviews again using the updated rubric.

== Implementation Notes ==

Implementation focuses on schema changes, shared internal rubric lookup logic, integration at current rubric selection call sites, user interface support on the Topics tab, and regression coverage for response and scoring paths.

Known integration points include response questionnaire lookup, scoring maximum-score calculation, and any review form initialization path that currently assumes assignment-level rubric selection.

CSC/ECE 517 Spring 2026 - E2605. Specialized Rubrics for Different Topic Types

2026-04-15T23:57:44Z

Vagarwa3: /* Open Questions */

__TOC__

== Introduction ==

This wiki page presents the design for Expertiza project '''E2605: Specialized Rubrics for Different Topic Types''' for CSC/ECE 517 Object-Oriented Design and Development, Spring 2026.

The goal of this project is to let instructors use different review rubrics for different project topics within the same assignment. Expertiza currently uses the same review rubric for all submissions in an assignment, unless the assignment is configured to use different rubrics in different review rounds. This works for assignments where every topic should be evaluated with the same criteria, but it does not work well when topics require different kinds of evaluation.

For example, an assignment might include topics related to user-interface design, machine learning, and security. A security topic may need rubric questions about threat modeling or vulnerability analysis, while a user-interface topic may need rubric questions about usability and design quality. Topic-based rubric selection allows the rubric to match the topic of the submission being reviewed.

This document describes the motivation for the change, the proposed data model, the instructor workflow, and how Expertiza will resolve the correct review rubric based on assignment, topic, and review-round information.

== Overview ==

This design adds topic-based review rubric selection while keeping existing behavior for assignments where topic-based rubric selection is not enabled. Instructors will associate review rubrics with topics directly from the Topics tab of Assignment creation, using dropdowns. If an assignment is configured to vary by topic, the review rubric is chosen based on the topic of the submission that is being reviewed. If topic variation is not enabled, rubric selection follows the current assignment and round logic.

A [[Response maps|ReviewResponseMap]] identifies the team whose submission is being reviewed. The [[Signed up teams|signed_up_teams]] table identifies which topic that team selected. The [[Assignments questionnaires|assignments_questionnaires]] table will then identify which review rubric should be used for that assignment and topic.

== Problem Statement ==

Different project topics can require different evaluation criteria, but currently all submissions for an assignment use the same review rubric unless round-based rubric variation is configured. The current tables do not yet store which review rubric should be used for each topic, and there is no assignment flag for topic-based rubric variation. Because of this, reviewers can receive rubrics that do not match the reviewed team's topic.

== Design Goals ==

The goal is to let instructors define topic-specific review rubrics without changing the overall review workflow. The design must preserve backward compatibility for existing assignments, continue to support round-based variation, and keep rubric resolution deterministic when both round and topic variation are enabled.

== Non-Goals ==

This change does not redesign topic signup or waitlist behavior, questionnaire authoring or import, or review mapping strategies. It only changes how the system stores and resolves rubric selection for reviews.

== Current State ==

The existing data model already includes the relationships needed to determine a reviewee team's topic. The [[Signed up teams|signed_up_teams]] table identifies which team has selected which topic, and review mappings already identify the reviewee team. Rubric resolution currently happens in several places, including response and scoring code paths, and is mainly driven by assignment and round context.

The current tables do not yet store all the information needed for topic-based rubric selection. Assignments currently include <code>vary_by_round</code> but no equivalent <code>vary_by_topic</code> flag, and [[Assignments questionnaires|assignments_questionnaires]] records do not have a <code>project_topic_id</code> foreign key.

== Proposed Data Model ==

This design introduces minimal extensions to the existing schema while reusing the current rubric infrastructure. A new <code>vary_by_topic</code> boolean field is added to the assignments table with a default value of false. This ensures that existing assignments continue to function without any changes, and instructors can explicitly enable topic-based rubric variation only when needed.

The [[Assignments questionnaires|assignments_questionnaires]] table indicates which rubrics are used for an assignment. Adding <code>project_topic_id</code> allows one of those rubric records to apply to a specific topic instead of applying to the whole assignment. A project topic will normally have at most one review-rubric mapping, or one per round if round-based variation is enabled.

In addition, an optional <code>project_topic_id</code> foreign key is added to the [[Assignments questionnaires|assignments_questionnaires]] table. A null <code>project_topic_id</code> represents the default review rubric for the assignment. This approach allows topic-specific mappings to coexist naturally with round-based mappings through <code>used_in_round</code>.

Indexes should be added on the new topic field and on the fields used for rubric lookup so that Expertiza can efficiently find the correct rubric for an assignment, topic, and round.

No new questionnaire type is introduced. The new <code>project_topic_id</code> field only connects an existing assignment-rubric record to a topic, and the rubric itself remains the existing questionnaire referenced by <code>questionnaire_id</code>.

== Rubric Resolution Behavior ==

Rubric selection remains within existing Expertiza code paths. The workflow proceeds by resolving the assignment from the response map, determining whether topic variation is enabled, and, if so, resolving the reviewee team's topic. The system then looks for the best matching assignment questionnaire for that assignment, topic, and round context.

When both round and topic variation are active, matching is deterministic. The precedence is: assignment/topic/current round, then assignment/topic/default round, then assignment/default topic/current round, and finally assignment/default topic/default round. In this precedence rule, the default topic corresponds to <code>project_topic_id</code> being null, and the default round corresponds to <code>used_in_round</code> being null.

If no topic-specific mapping is found, the system falls back according to the defined precedence, using default round-specific rubrics for the assignment when applicable, before using the default review rubric for the assignment.

Only the reviewee team's confirmed topic signup for the current assignment is considered, and waitlisted topic signups do not determine rubric selection.

== UI and Instructor Workflow ==

[[File:E2605architecture.png|800px|center|Figure: Rubric resolution architecture]]

Instructors configure topic-based rubrics directly in assignment setup on the Topics tab. For each topic, the UI exposes dropdowns for selecting the review rubric, and when <code>vary_by_round</code> is enabled, round-specific rubric choices are also supported. This approach keeps all configuration within the standard assignment authoring flow and avoids introducing any external components.

The existing default review rubric for the assignment remains available as the fallback when a topic-specific rubric is not configured. Reviewers do not choose the rubric themselves; instead, they receive the questionnaire resolved automatically for the team they are reviewing.

== API and Validation ==

Assignment create and update flows persist <code>vary_by_topic</code> cleanly, and parameter naming is normalized so that there is a single canonical internal field. Assignment questionnaire updates accept an optional <code>project_topic_id</code> and validate that the topic belongs to the same assignment.

Validation ensures that <code>project_topic_id</code> references an existing topic and does not cross assignment boundaries. The system also prevents duplicate review-rubric mappings for the same assignment, topic, and round combination, ensuring that rubric resolution always produces a single correct result.

Only instructors or administrators who can edit the assignment are allowed to enable topic-based rubric variation or modify topic-rubric mappings.

== Testing Strategy ==

Tests verify that assignments without topic variation remain unchanged, while topic-enabled assignments correctly select topic-specific rubrics when configured. They also ensure that fallback behavior functions properly when a team has no topic or no topic-specific rubric mapping.

Additional tests confirm deterministic precedence when both topic and round variation are enabled and ensure that scoring behavior remains consistent with the selected questionnaire.

== Risks ==

The main risk is ambiguity in precedence when both topic and round variation are present. This is mitigated by defining a strict lookup order and enforcing it through testing. Another risk is partial configuration, where topic variation is enabled but mappings are incomplete; this is mitigated by safe fallback behavior and validation in configuration paths.

== Rubric Changes After Reviews Begin ==

If an instructor changes a topic-rubric mapping after reviews have already begun, existing reviews that used the previous rubric should not remain valid. Since the rubric determines the questions and scoring criteria for a review, changing the rubric can make previously submitted reviews inconsistent with the new configuration.

When a topic-rubric mapping is changed after reviews have been performed, the affected reviews should be deleted, and the affected reviewers should be notified that they need to perform the reviews again using the updated rubric.

== Implementation Notes ==

Implementation focuses on schema changes, shared internal rubric lookup logic, integration at current rubric selection call sites, user interface support on the Topics tab, and regression coverage for response and scoring paths.

Known integration points include response questionnaire lookup, scoring maximum-score calculation, and any review form initialization path that currently assumes assignment-level rubric selection.

CSC/ECE 517 Spring 2026 - E2605. Specialized Rubrics for Different Topic Types

2026-04-15T23:55:53Z

Vagarwa3: /* Migration and Rollout */

__TOC__

== Introduction ==

This wiki page presents the design for Expertiza project '''E2605: Specialized Rubrics for Different Topic Types''' for CSC/ECE 517 Object-Oriented Design and Development, Spring 2026.

The goal of this project is to let instructors use different review rubrics for different project topics within the same assignment. Expertiza currently uses the same review rubric for all submissions in an assignment, unless the assignment is configured to use different rubrics in different review rounds. This works for assignments where every topic should be evaluated with the same criteria, but it does not work well when topics require different kinds of evaluation.

For example, an assignment might include topics related to user-interface design, machine learning, and security. A security topic may need rubric questions about threat modeling or vulnerability analysis, while a user-interface topic may need rubric questions about usability and design quality. Topic-based rubric selection allows the rubric to match the topic of the submission being reviewed.

This document describes the motivation for the change, the proposed data model, the instructor workflow, and how Expertiza will resolve the correct review rubric based on assignment, topic, and review-round information.

== Overview ==

This design adds topic-based review rubric selection while keeping existing behavior for assignments where topic-based rubric selection is not enabled. Instructors will associate review rubrics with topics directly from the Topics tab of Assignment creation, using dropdowns. If an assignment is configured to vary by topic, the review rubric is chosen based on the topic of the submission that is being reviewed. If topic variation is not enabled, rubric selection follows the current assignment and round logic.

A [[Response maps|ReviewResponseMap]] identifies the team whose submission is being reviewed. The [[Signed up teams|signed_up_teams]] table identifies which topic that team selected. The [[Assignments questionnaires|assignments_questionnaires]] table will then identify which review rubric should be used for that assignment and topic.

== Problem Statement ==

Different project topics can require different evaluation criteria, but currently all submissions for an assignment use the same review rubric unless round-based rubric variation is configured. The current tables do not yet store which review rubric should be used for each topic, and there is no assignment flag for topic-based rubric variation. Because of this, reviewers can receive rubrics that do not match the reviewed team's topic.

== Design Goals ==

The goal is to let instructors define topic-specific review rubrics without changing the overall review workflow. The design must preserve backward compatibility for existing assignments, continue to support round-based variation, and keep rubric resolution deterministic when both round and topic variation are enabled.

== Non-Goals ==

This change does not redesign topic signup or waitlist behavior, questionnaire authoring or import, or review mapping strategies. It only changes how the system stores and resolves rubric selection for reviews.

== Current State ==

The existing data model already includes the relationships needed to determine a reviewee team's topic. The [[Signed up teams|signed_up_teams]] table identifies which team has selected which topic, and review mappings already identify the reviewee team. Rubric resolution currently happens in several places, including response and scoring code paths, and is mainly driven by assignment and round context.

The current tables do not yet store all the information needed for topic-based rubric selection. Assignments currently include <code>vary_by_round</code> but no equivalent <code>vary_by_topic</code> flag, and [[Assignments questionnaires|assignments_questionnaires]] records do not have a <code>project_topic_id</code> foreign key.

== Proposed Data Model ==

This design introduces minimal extensions to the existing schema while reusing the current rubric infrastructure. A new <code>vary_by_topic</code> boolean field is added to the assignments table with a default value of false. This ensures that existing assignments continue to function without any changes, and instructors can explicitly enable topic-based rubric variation only when needed.

The [[Assignments questionnaires|assignments_questionnaires]] table indicates which rubrics are used for an assignment. Adding <code>project_topic_id</code> allows one of those rubric records to apply to a specific topic instead of applying to the whole assignment. A project topic will normally have at most one review-rubric mapping, or one per round if round-based variation is enabled.

In addition, an optional <code>project_topic_id</code> foreign key is added to the [[Assignments questionnaires|assignments_questionnaires]] table. A null <code>project_topic_id</code> represents the default review rubric for the assignment. This approach allows topic-specific mappings to coexist naturally with round-based mappings through <code>used_in_round</code>.

Indexes should be added on the new topic field and on the fields used for rubric lookup so that Expertiza can efficiently find the correct rubric for an assignment, topic, and round.

No new questionnaire type is introduced. The new <code>project_topic_id</code> field only connects an existing assignment-rubric record to a topic, and the rubric itself remains the existing questionnaire referenced by <code>questionnaire_id</code>.

== Rubric Resolution Behavior ==

Rubric selection remains within existing Expertiza code paths. The workflow proceeds by resolving the assignment from the response map, determining whether topic variation is enabled, and, if so, resolving the reviewee team's topic. The system then looks for the best matching assignment questionnaire for that assignment, topic, and round context.

When both round and topic variation are active, matching is deterministic. The precedence is: assignment/topic/current round, then assignment/topic/default round, then assignment/default topic/current round, and finally assignment/default topic/default round. In this precedence rule, the default topic corresponds to <code>project_topic_id</code> being null, and the default round corresponds to <code>used_in_round</code> being null.

If no topic-specific mapping is found, the system falls back according to the defined precedence, using default round-specific rubrics for the assignment when applicable, before using the default review rubric for the assignment.

Only the reviewee team's confirmed topic signup for the current assignment is considered, and waitlisted topic signups do not determine rubric selection.

== UI and Instructor Workflow ==

[[File:E2605architecture.png|800px|center|Figure: Rubric resolution architecture]]

Instructors configure topic-based rubrics directly in assignment setup on the Topics tab. For each topic, the UI exposes dropdowns for selecting the review rubric, and when <code>vary_by_round</code> is enabled, round-specific rubric choices are also supported. This approach keeps all configuration within the standard assignment authoring flow and avoids introducing any external components.

The existing default review rubric for the assignment remains available as the fallback when a topic-specific rubric is not configured. Reviewers do not choose the rubric themselves; instead, they receive the questionnaire resolved automatically for the team they are reviewing.

== API and Validation ==

Assignment create and update flows persist <code>vary_by_topic</code> cleanly, and parameter naming is normalized so that there is a single canonical internal field. Assignment questionnaire updates accept an optional <code>project_topic_id</code> and validate that the topic belongs to the same assignment.

Validation ensures that <code>project_topic_id</code> references an existing topic and does not cross assignment boundaries. The system also prevents duplicate review-rubric mappings for the same assignment, topic, and round combination, ensuring that rubric resolution always produces a single correct result.

Only instructors or administrators who can edit the assignment are allowed to enable topic-based rubric variation or modify topic-rubric mappings.

== Testing Strategy ==

Tests verify that assignments without topic variation remain unchanged, while topic-enabled assignments correctly select topic-specific rubrics when configured. They also ensure that fallback behavior functions properly when a team has no topic or no topic-specific rubric mapping.

Additional tests confirm deterministic precedence when both topic and round variation are enabled and ensure that scoring behavior remains consistent with the selected questionnaire.

== Risks ==

The main risk is ambiguity in precedence when both topic and round variation are present. This is mitigated by defining a strict lookup order and enforcing it through testing. Another risk is partial configuration, where topic variation is enabled but mappings are incomplete; this is mitigated by safe fallback behavior and validation in configuration paths.

== Open Questions ==

If an instructor changes a topic-rubric mapping after reviews have begun, existing responses could resolve to a different rubric unless the selected mapping is preserved. The implementation should either store the selected assignment questionnaire on the response or prevent mapping changes once reviews are underway.

== Implementation Notes ==

Implementation focuses on schema changes, shared internal rubric lookup logic, integration at current rubric selection call sites, user interface support on the Topics tab, and regression coverage for response and scoring paths.

Known integration points include response questionnaire lookup, scoring maximum-score calculation, and any review form initialization path that currently assumes assignment-level rubric selection.

CSC/ECE 517 Spring 2026 - E2605. Specialized Rubrics for Different Topic Types

2026-04-15T23:55:17Z

Vagarwa3:

__TOC__

== Introduction ==

This wiki page presents the design for Expertiza project '''E2605: Specialized Rubrics for Different Topic Types''' for CSC/ECE 517 Object-Oriented Design and Development, Spring 2026.

The goal of this project is to let instructors use different review rubrics for different project topics within the same assignment. Expertiza currently uses the same review rubric for all submissions in an assignment, unless the assignment is configured to use different rubrics in different review rounds. This works for assignments where every topic should be evaluated with the same criteria, but it does not work well when topics require different kinds of evaluation.

For example, an assignment might include topics related to user-interface design, machine learning, and security. A security topic may need rubric questions about threat modeling or vulnerability analysis, while a user-interface topic may need rubric questions about usability and design quality. Topic-based rubric selection allows the rubric to match the topic of the submission being reviewed.

This document describes the motivation for the change, the proposed data model, the instructor workflow, and how Expertiza will resolve the correct review rubric based on assignment, topic, and review-round information.

== Overview ==

This design adds topic-based review rubric selection while keeping existing behavior for assignments where topic-based rubric selection is not enabled. Instructors will associate review rubrics with topics directly from the Topics tab of Assignment creation, using dropdowns. If an assignment is configured to vary by topic, the review rubric is chosen based on the topic of the submission that is being reviewed. If topic variation is not enabled, rubric selection follows the current assignment and round logic.

A [[Response maps|ReviewResponseMap]] identifies the team whose submission is being reviewed. The [[Signed up teams|signed_up_teams]] table identifies which topic that team selected. The [[Assignments questionnaires|assignments_questionnaires]] table will then identify which review rubric should be used for that assignment and topic.

== Problem Statement ==

Different project topics can require different evaluation criteria, but currently all submissions for an assignment use the same review rubric unless round-based rubric variation is configured. The current tables do not yet store which review rubric should be used for each topic, and there is no assignment flag for topic-based rubric variation. Because of this, reviewers can receive rubrics that do not match the reviewed team's topic.

== Design Goals ==

The goal is to let instructors define topic-specific review rubrics without changing the overall review workflow. The design must preserve backward compatibility for existing assignments, continue to support round-based variation, and keep rubric resolution deterministic when both round and topic variation are enabled.

== Non-Goals ==

This change does not redesign topic signup or waitlist behavior, questionnaire authoring or import, or review mapping strategies. It only changes how the system stores and resolves rubric selection for reviews.

== Current State ==

The existing data model already includes the relationships needed to determine a reviewee team's topic. The [[Signed up teams|signed_up_teams]] table identifies which team has selected which topic, and review mappings already identify the reviewee team. Rubric resolution currently happens in several places, including response and scoring code paths, and is mainly driven by assignment and round context.

The current tables do not yet store all the information needed for topic-based rubric selection. Assignments currently include <code>vary_by_round</code> but no equivalent <code>vary_by_topic</code> flag, and [[Assignments questionnaires|assignments_questionnaires]] records do not have a <code>project_topic_id</code> foreign key.

== Proposed Data Model ==

This design introduces minimal extensions to the existing schema while reusing the current rubric infrastructure. A new <code>vary_by_topic</code> boolean field is added to the assignments table with a default value of false. This ensures that existing assignments continue to function without any changes, and instructors can explicitly enable topic-based rubric variation only when needed.

The [[Assignments questionnaires|assignments_questionnaires]] table indicates which rubrics are used for an assignment. Adding <code>project_topic_id</code> allows one of those rubric records to apply to a specific topic instead of applying to the whole assignment. A project topic will normally have at most one review-rubric mapping, or one per round if round-based variation is enabled.

In addition, an optional <code>project_topic_id</code> foreign key is added to the [[Assignments questionnaires|assignments_questionnaires]] table. A null <code>project_topic_id</code> represents the default review rubric for the assignment. This approach allows topic-specific mappings to coexist naturally with round-based mappings through <code>used_in_round</code>.

Indexes should be added on the new topic field and on the fields used for rubric lookup so that Expertiza can efficiently find the correct rubric for an assignment, topic, and round.

No new questionnaire type is introduced. The new <code>project_topic_id</code> field only connects an existing assignment-rubric record to a topic, and the rubric itself remains the existing questionnaire referenced by <code>questionnaire_id</code>.

== Rubric Resolution Behavior ==

Rubric selection remains within existing Expertiza code paths. The workflow proceeds by resolving the assignment from the response map, determining whether topic variation is enabled, and, if so, resolving the reviewee team's topic. The system then looks for the best matching assignment questionnaire for that assignment, topic, and round context.

When both round and topic variation are active, matching is deterministic. The precedence is: assignment/topic/current round, then assignment/topic/default round, then assignment/default topic/current round, and finally assignment/default topic/default round. In this precedence rule, the default topic corresponds to <code>project_topic_id</code> being null, and the default round corresponds to <code>used_in_round</code> being null.

If no topic-specific mapping is found, the system falls back according to the defined precedence, using default round-specific rubrics for the assignment when applicable, before using the default review rubric for the assignment.

Only the reviewee team's confirmed topic signup for the current assignment is considered, and waitlisted topic signups do not determine rubric selection.

== UI and Instructor Workflow ==

[[File:E2605architecture.png|800px|center|Figure: Rubric resolution architecture]]

Instructors configure topic-based rubrics directly in assignment setup on the Topics tab. For each topic, the UI exposes dropdowns for selecting the review rubric, and when <code>vary_by_round</code> is enabled, round-specific rubric choices are also supported. This approach keeps all configuration within the standard assignment authoring flow and avoids introducing any external components.

The existing default review rubric for the assignment remains available as the fallback when a topic-specific rubric is not configured. Reviewers do not choose the rubric themselves; instead, they receive the questionnaire resolved automatically for the team they are reviewing.

== API and Validation ==

Assignment create and update flows persist <code>vary_by_topic</code> cleanly, and parameter naming is normalized so that there is a single canonical internal field. Assignment questionnaire updates accept an optional <code>project_topic_id</code> and validate that the topic belongs to the same assignment.

Validation ensures that <code>project_topic_id</code> references an existing topic and does not cross assignment boundaries. The system also prevents duplicate review-rubric mappings for the same assignment, topic, and round combination, ensuring that rubric resolution always produces a single correct result.

Only instructors or administrators who can edit the assignment are allowed to enable topic-based rubric variation or modify topic-rubric mappings.

== Migration and Rollout ==

Migration first adds <code>vary_by_topic</code> to assignments, followed by adding a nullable <code>project_topic_id</code> to the [[Assignments questionnaires|assignments_questionnaires]] table along with the necessary foreign key and indexes. Existing rows remain valid with a null topic id and continue to represent default review rubrics for assignments.

After deployment, instructors can enable topic variation gradually on a per-assignment basis using the existing user interface.

== Testing Strategy ==

Tests verify that assignments without topic variation remain unchanged, while topic-enabled assignments correctly select topic-specific rubrics when configured. They also ensure that fallback behavior functions properly when a team has no topic or no topic-specific rubric mapping.

Additional tests confirm deterministic precedence when both topic and round variation are enabled and ensure that scoring behavior remains consistent with the selected questionnaire.

== Risks ==

The main risk is ambiguity in precedence when both topic and round variation are present. This is mitigated by defining a strict lookup order and enforcing it through testing. Another risk is partial configuration, where topic variation is enabled but mappings are incomplete; this is mitigated by safe fallback behavior and validation in configuration paths.

== Open Questions ==

If an instructor changes a topic-rubric mapping after reviews have begun, existing responses could resolve to a different rubric unless the selected mapping is preserved. The implementation should either store the selected assignment questionnaire on the response or prevent mapping changes once reviews are underway.

== Implementation Notes ==

Implementation focuses on schema changes, shared internal rubric lookup logic, integration at current rubric selection call sites, user interface support on the Topics tab, and regression coverage for response and scoring paths.

Known integration points include response questionnaire lookup, scoring maximum-score calculation, and any review form initialization path that currently assumes assignment-level rubric selection.

CSC/ECE 517 Spring 2026 - E2605. Specialized Rubrics for Different Topic Types

2026-04-14T01:40:27Z

Vagarwa3:

__TOC__

== Introduction ==

This wiki page presents the design for Expertiza project '''E2605: Specialized Rubrics for Different Topic Types''' for CSC/ECE 517 Object-Oriented Design and Development, Spring 2026.

The goal of this project is to let instructors use different review rubrics for different project topics within the same assignment. Expertiza currently uses the same review rubric for all submissions in an assignment, unless the assignment is configured to use different rubrics in different review rounds. This works for assignments where every topic should be evaluated with the same criteria, but it does not work well when topics require different kinds of evaluation.

For example, an assignment might include topics related to user-interface design, machine learning, and security. A security topic may need rubric questions about threat modeling or vulnerability analysis, while a user-interface topic may need rubric questions about usability and design quality. Topic-based rubric selection allows the rubric to match the topic of the submission being reviewed.

This document describes the motivation for the change, the proposed data model, the instructor workflow, and how Expertiza will resolve the correct review rubric based on assignment, topic, and review-round information.

== Overview ==

This design adds topic-based review rubric selection while keeping existing behavior for assignments where topic-based rubric selection is not enabled. Instructors will associate review rubrics with topics directly from the Topics tab of Assignment creation, using dropdowns. If an assignment is configured to vary by topic, the review rubric is chosen based on the topic of the submission that is being reviewed. If topic variation is not enabled, rubric selection follows the current assignment and round logic.

A [[Response maps|ReviewResponseMap]] identifies the team whose submission is being reviewed. The [[Signed up teams|signed_up_teams]] table identifies which topic that team selected. The [[Assignments questionnaires|assignments_questionnaires]] table will then identify which review rubric should be used for that assignment and topic.

== Problem Statement ==

Different project topics can require different evaluation criteria, but currently all submissions for an assignment use the same review rubric unless round-based rubric variation is configured. The current tables do not yet store which review rubric should be used for each topic, and there is no assignment flag for topic-based rubric variation. Because of this, reviewers can receive rubrics that do not match the reviewed team's topic.

== Design Goals ==

The goal is to let instructors define topic-specific review rubrics without changing the overall review workflow. The design must preserve backward compatibility for existing assignments, continue to support round-based variation, and keep rubric resolution deterministic when both round and topic variation are enabled.

== Non-Goals ==

This change does not redesign topic signup or waitlist behavior, questionnaire authoring or import, or review mapping strategies. It only changes how the system stores and resolves rubric selection for reviews.

== Current State ==

The existing data model already includes the relationships needed to determine a reviewee team's topic. The [[Signed up teams|signed_up_teams]] table identifies which team has selected which topic, and review mappings already identify the reviewee team. Rubric resolution currently happens in several places, including response and scoring code paths, and is mainly driven by assignment and round context.

The current tables do not yet store all the information needed for topic-based rubric selection. Assignments currently include <code>vary_by_round</code> but no equivalent <code>vary_by_topic</code> flag, and [[Assignments questionnaires|assignments_questionnaires]] records do not have a <code>project_topic_id</code> foreign key.

== Proposed Data Model ==

This design introduces minimal extensions to the existing schema while reusing the current rubric infrastructure. A new <code>vary_by_topic</code> boolean field is added to the assignments table with a default value of false. This ensures that existing assignments continue to function without any changes, and instructors can explicitly enable topic-based rubric variation only when needed.

In addition, an optional <code>project_topic_id</code> foreign key is added to the [[Assignments questionnaires|assignments_questionnaires]] table. A null <code>project_topic_id</code> represents the default review rubric for the assignment. This approach allows topic-specific mappings to coexist naturally with round-based mappings through <code>used_in_round</code>.

Indexes should be added on the new topic field and on the fields used for rubric lookup so that Expertiza can efficiently find the correct rubric for an assignment, topic, and round.

The [[Assignments questionnaires|assignments_questionnaires]] table already indicates which rubrics are used for an assignment. Adding <code>project_topic_id</code> allows one of those rubric records to apply to a specific topic instead of applying to the whole assignment. A project topic will normally have at most one review-rubric mapping, or one per round if round-based variation is enabled.

No new questionnaire type is introduced. The new <code>project_topic_id</code> field only connects an existing assignment-rubric record to a topic, and the rubric itself remains the existing questionnaire referenced by <code>questionnaire_id</code>.

== Rubric Resolution Behavior ==

Rubric selection remains within existing Expertiza code paths. The workflow proceeds by resolving the assignment from the response map, determining whether topic variation is enabled, and, if so, resolving the reviewee team's topic. The system then looks for the best matching assignment questionnaire for that assignment, topic, and round context.

When both round and topic variation are active, matching is deterministic. The precedence is: assignment/topic/current round, then assignment/topic/default round, then assignment/default topic/current round, and finally assignment/default topic/default round. In this precedence rule, the default topic corresponds to <code>project_topic_id</code> being null, and the default round corresponds to <code>used_in_round</code> being null.

If no topic-specific mapping is found, the system falls back according to the defined precedence, including default round-specific rubrics for the assignment when applicable, before using the default review rubric for the assignment.

Only the reviewee team's confirmed topic signup for the current assignment is considered, and waitlisted topic signups do not determine rubric selection.

== UI and Instructor Workflow ==

Instructors configure topic-based rubrics directly in assignment setup on the Topics tab. For each topic, the UI exposes dropdowns for selecting the review rubric, and when <code>vary_by_round</code> is enabled, round-specific rubric choices are also supported. This approach keeps all configuration within the standard assignment authoring flow and avoids introducing any external components.

The existing default review rubric for the assignment remains available as the fallback when a topic-specific rubric is not configured. Reviewers do not choose the rubric themselves; instead, they receive the questionnaire resolved automatically for the team they are reviewing.

== API and Validation ==

Assignment create and update flows persist <code>vary_by_topic</code> cleanly, and parameter naming is normalized so that there is a single canonical internal field. Assignment questionnaire updates accept an optional <code>project_topic_id</code> and validate that the topic belongs to the same assignment.

Validation ensures that <code>project_topic_id</code> references an existing topic and does not cross assignment boundaries. The system also prevents duplicate review-rubric mappings for the same assignment, topic, and round combination, ensuring that rubric resolution always produces a single correct result.

Only instructors or administrators who can edit the assignment are allowed to enable topic-based rubric variation or modify topic-rubric mappings.

== Migration and Rollout ==

Migration first adds <code>vary_by_topic</code> to assignments, followed by adding a nullable <code>project_topic_id</code> to the [[Assignments questionnaires|assignments_questionnaires]] table along with the necessary foreign key and indexes. Existing rows remain valid with a null topic id and continue to represent default review rubrics for assignments.

After deployment, instructors can enable topic variation gradually on a per-assignment basis using the existing user interface.

== Testing Strategy ==

Tests verify that assignments without topic variation remain unchanged, while topic-enabled assignments correctly select topic-specific rubrics when configured. They also ensure that fallback behavior functions properly when a team has no topic or no topic-specific rubric mapping.

Additional tests confirm deterministic precedence when both topic and round variation are enabled and ensure that scoring behavior remains consistent with the selected questionnaire.

== Risks ==

The main risk is ambiguity in precedence when both topic and round variation are present. This is mitigated by defining a strict lookup order and enforcing it through testing. Another risk is partial configuration, where topic variation is enabled but mappings are incomplete; this is mitigated by safe fallback behavior and validation in configuration paths.

== Open Questions ==

If an instructor changes a topic-rubric mapping after reviews have begun, existing responses could resolve to a different rubric unless the selected mapping is preserved. The implementation should either store the selected assignment questionnaire on the response or prevent mapping changes once reviews are underway.

== Implementation Notes ==

Implementation focuses on schema changes, shared internal rubric lookup logic, integration at current rubric selection call sites, user interface support on the Topics tab, and regression coverage for response and scoring paths.

Known integration points include response questionnaire lookup, scoring maximum-score calculation, and any review form initialization path that currently assumes assignment-level rubric selection.

CSC/ECE 517 Spring 2026 - E2605. Specialized Rubrics for Different Topic Types

2026-04-14T01:37:34Z

Vagarwa3:

__TOC__

== Introduction ==

This wiki page presents the design for Expertiza project '''E2605: Specialized Rubrics for Different Topic Types''' for CSC/ECE 517 Object-Oriented Design and Development, Spring 2026.

The goal of this project is to let instructors use different review rubrics for different project topics within the same assignment. Expertiza currently uses the same review rubric for all submissions in an assignment, unless the assignment is configured to use different rubrics in different review rounds. This works for assignments where every topic should be evaluated with the same criteria, but it does not work well when topics require different kinds of evaluation.

For example, an assignment might include topics related to user-interface design, machine learning, and security. A security topic may need rubric questions about threat modeling or vulnerability analysis, while a user-interface topic may need rubric questions about usability and design quality. Topic-based rubric selection allows the rubric to match the topic of the submission being reviewed.

This document describes the motivation for the change, the proposed data model, the instructor workflow, and how Expertiza will resolve the correct review rubric based on assignment, topic, and review-round information.

== Overview ==

This design adds topic-based review rubric selection while keeping existing behavior for assignments where topic-based rubric selection is not enabled. Instructors will associate review rubrics with topics directly from the Topics tab of Assignment creation, using dropdowns. If an assignment is configured to vary by topic, the review rubric is chosen based on the topic of the submission that is being reviewed. If topic variation is not enabled, rubric selection follows the current assignment and round logic.

A [[Response maps|ReviewResponseMap]] identifies the team whose submission is being reviewed. The [[Signed up teams|signed_up_teams]] table identifies which topic that team selected. The [[Assignments questionnaires|assignments_questionnaires]] table will then identify which review rubric should be used for that assignment and topic.

== Problem Statement ==

Different project topics can require different evaluation criteria, but currently all submissions for an assignment use the same review rubric unless round-based rubric variation is configured. The current tables do not yet store which review rubric should be used for each topic, and there is no assignment flag for topic-based rubric variation. Because of this, reviewers can receive rubrics that do not match the reviewed team's topic.

== Design Goals ==

The goal is to let instructors define topic-specific review rubrics without changing the overall review workflow. The design must preserve backward compatibility for existing assignments, continue to support round-based variation, and keep rubric resolution deterministic when both round and topic variation are enabled.

== Non-Goals ==

This change does not redesign topic signup or waitlist behavior, questionnaire authoring or import, or review mapping strategies. It only changes how the system stores and resolves rubric selection for reviews.

== Current State ==

The existing data model already includes the relationships needed to determine a reviewee team's topic. The [[Signed up teams|signed_up_teams]] table identifies which team has selected which topic, and review mappings already identify the reviewee team. Rubric resolution currently happens in several places, including response and scoring code paths, and is mainly driven by assignment and round context.

The current tables do not yet store all the information needed for topic-based rubric selection. Assignments currently include <code>vary_by_round</code> but no equivalent <code>vary_by_topic</code> flag, and [[Assignments questionnaires|assignments_questionnaires]] records do not have a <code>project_topic_id</code> foreign key.

== Proposed Data Model ==

This design introduces minimal extensions to the existing schema while reusing the current rubric infrastructure. A new <code>vary_by_topic</code> boolean field is added to the assignments table with a default value of false. This ensures that existing assignments continue to function without any changes, and instructors can explicitly enable topic-based rubric variation only when needed.

In addition, an optional <code>project_topic_id</code> foreign key is added to the [[Assignments questionnaires|assignments_questionnaires]] table. A null <code>project_topic_id</code> represents the default review rubric for the assignment. This approach allows topic-specific mappings to coexist naturally with round-based mappings through <code>used_in_round</code>.

Indexes should be added on the new topic field and on the fields used for rubric lookup so that Expertiza can efficiently find the correct rubric for an assignment, topic, and round.

The [[Assignments questionnaires|assignments_questionnaires]] table already indicates which rubrics are used for an assignment. Adding <code>project_topic_id</code> allows one of those rubric records to apply to a specific topic instead of applying to the whole assignment. A project topic will normally have at most one review-rubric mapping, or one per round if round-based variation is enabled.

No new questionnaire type is introduced. The new <code>project_topic_id</code> field only connects an existing assignment-rubric record to a topic, and the rubric itself remains the existing questionnaire referenced by <code>questionnaire_id</code>.

== Rubric Resolution Behavior ==

Rubric selection remains within existing Expertiza code paths. The workflow proceeds by resolving the assignment from the map, determining whether topic variation is enabled, and, if so, resolving the reviewee team's topic. The system then looks for the best matching assignment questionnaire for that assignment, topic, and round context.

When both round and topic variation are active, matching is deterministic. The precedence is: assignment/topic/current round, then assignment/topic/default round, then assignment/default topic/current round, and finally assignment/default topic/default round. In this precedence rule, the default topic corresponds to <code>project_topic_id</code> being null, and the default round corresponds to <code>used_in_round</code> being null.

If no topic-specific mapping is found, the system falls back according to the defined precedence, including default round-specific rubrics for the assignment when applicable, before using the default review rubric for the assignment.

Only the reviewee team's confirmed topic signup for the current assignment is considered, and waitlisted topic signups do not determine rubric selection.

== UI and Instructor Workflow ==

Instructors configure topic-based rubrics directly in assignment setup on the Topics tab. For each topic, the UI exposes dropdowns for selecting the review rubric, and when <code>vary_by_round</code> is enabled, round-specific rubric choices are also supported. This approach keeps all configuration within the standard assignment authoring flow and avoids introducing any external components.

The existing default review rubric for the assignment remains available as the fallback when a topic-specific rubric is not configured. Reviewers do not choose the rubric themselves; instead, they receive the questionnaire resolved automatically for the team they are reviewing.

== API and Validation ==

Assignment create and update flows persist <code>vary_by_topic</code> cleanly, and parameter naming is normalized so that there is a single canonical internal field. Assignment questionnaire updates accept an optional <code>project_topic_id</code> and validate that the topic belongs to the same assignment.

Validation ensures that <code>project_topic_id</code> references an existing topic and does not cross assignment boundaries. The system also prevents duplicate review-rubric mappings for the same assignment, topic, and round combination, ensuring that rubric resolution always produces a single correct result.

Only instructors or administrators who can edit the assignment are allowed to enable topic-based rubric variation or modify topic-rubric mappings.

== Migration and Rollout ==

Migration first adds <code>vary_by_topic</code> to assignments, followed by adding a nullable <code>project_topic_id</code> to assignment questionnaires along with the necessary foreign key and indexes. Existing rows remain valid with a null topic id and continue to represent default review rubrics for assignments.

After deployment, instructors can enable topic variation gradually on a per-assignment basis using the existing user interface.

== Testing Strategy ==

Tests verify that assignments without topic variation remain unchanged, while topic-enabled assignments correctly select topic-specific rubrics when configured. They also ensure that fallback behavior functions properly when a team has no topic or no topic-specific rubric mapping.

Additional tests confirm deterministic precedence when both topic and round variation are enabled and ensure that scoring behavior remains consistent with the selected questionnaire.

== Risks ==

The main risk is ambiguity in precedence when both topic and round variation are present. This is mitigated by defining a strict lookup order and enforcing it through testing. Another risk is partial configuration, where topic variation is enabled but mappings are incomplete; this is mitigated by safe fallback behavior and validation in configuration paths.

== Open Questions ==

If an instructor changes a topic-rubric mapping after reviews have begun, existing responses could resolve to a different rubric unless the selected mapping is preserved. The implementation should either store the selected assignment questionnaire on the response or prevent mapping changes once reviews are underway.

== Implementation Notes ==

Implementation focuses on schema changes, shared internal rubric lookup logic, integration at current rubric selection call sites, user interface support on the Topics tab, and regression coverage for response and scoring paths.

Known integration points include response questionnaire lookup, scoring maximum-score calculation, and any review form initialization path that currently assumes assignment-level rubric selection.

CSC/ECE 517 Spring 2026 - E2605. Specialized Rubrics for Different Topic Types

2026-04-14T01:24:00Z

Vagarwa3:

__TOC__

== Introduction ==

This wiki page presents the design for Expertiza project '''E2605: Specialized Rubrics for Different Topic Types''' for CSC/ECE 517 Object-Oriented Design and Development, Spring 2026.

The goal of this project is to let instructors use different review rubrics for different project topics within the same assignment. Expertiza currently uses the same review rubric for all submissions in an assignment, unless the assignment is configured to use different rubrics in different review rounds. This works for assignments where every topic should be evaluated with the same criteria, but it does not work well when topics require different kinds of evaluation.

For example, an assignment might include topics related to user-interface design, machine learning, and security. A security topic may need rubric questions about threat modeling or vulnerability analysis, while a user-interface topic may need rubric questions about usability and design quality. Topic-based rubric selection allows the rubric to match the topic of the submission being reviewed.

This document describes the motivation for the change, the proposed data model, the instructor workflow, and how Expertiza will resolve the correct review rubric based on assignment, topic, and review-round information.

== Overview ==

This design adds topic-based review rubric selection while keeping existing behavior for assignments where topic-based rubric selection is not enabled. Instructors will associate review rubrics with topics directly from the Topics tab of Assignment creation, using dropdowns. If an assignment is configured to vary by topic, the review rubric is chosen based on the topic of the submission that is being reviewed. If topic variation is not enabled, rubric selection follows the current assignment and round logic.

A ReviewResponseMap identifies the team whose submission is being reviewed. The <code>signed_up_teams</code> table identifies which topic that team selected. The <code>assignments_questionnaires</code> table will then identify which review rubric should be used for that assignment and topic.

== Problem Statement ==

Different project topics can require different evaluation criteria, but currently all submissions for an assignment use the same review rubric unless round-based rubric variation is configured. The schema does not yet provide a first-class topic-to-rubric mapping, and there is no assignment flag for topic-based rubric variation. Because of this, reviewers can receive rubrics that do not match the reviewed team's topic.

== Design Goals ==

The goal is to let instructors define topic-specific review rubrics without changing the overall review workflow. The design must preserve backward compatibility for existing assignments, continue to support round-based variation, and keep rubric resolution deterministic when both round and topic variation are enabled.

== Non-Goals ==

This change does not redesign topic signup or waitlist behavior, questionnaire authoring or import, or review mapping strategies. It only changes how the system stores and resolves rubric selection for reviews.

== Current State ==

The existing data model already includes the relationships needed to determine a reviewee team's topic. The <code>signed_up_teams</code> table identifies which team has selected which topic, and review mappings already identify the reviewee team. Rubric resolution currently happens in several places, including response and scoring code paths, and is mainly driven by assignment and round context.

The current tables do not yet store all the information needed for topic-based rubric selection. Assignments currently include <code>vary_by_round</code> but no equivalent <code>vary_by_topic</code> flag, and assignment questionnaire records do not have a topic foreign key.

== Proposed Data Model ==

This design introduces minimal extensions to the existing schema while reusing the current rubric infrastructure. A new <code>vary_by_topic</code> boolean field is added to the assignments table with a default value of false. This ensures that existing assignments continue to function without any changes, and instructors can explicitly enable topic-based rubric variation only when needed.

In addition, an optional <code>project_topic_id</code> foreign key is added to the <code>assignments_questionnaires</code> table. A null <code>project_topic_id</code> represents the default review rubric for the assignment. This approach allows topic-specific mappings to coexist naturally with round-based mappings through <code>used_in_round</code>.

Indexes should be added on the new topic field and on the fields used for rubric lookup so that Expertiza can efficiently find the correct rubric for an assignment, topic, and round.

The <code>assignments_questionnaires</code> table already indicates which rubrics are used for an assignment. Adding <code>project_topic_id</code> allows one of those rubric records to apply to a specific topic instead of applying to the whole assignment. A project topic will normally have at most one review-rubric mapping, or one per round if round-based variation is enabled.

No new questionnaire type is introduced. The new <code>project_topic_id</code> field only connects an existing assignment-rubric record to a topic, and the rubric itself remains the existing questionnaire referenced by <code>questionnaire_id</code>.

== Rubric Resolution Behavior ==

No separate selection service is required, and rubric selection remains within existing Expertiza code paths. The workflow proceeds by resolving the assignment from the map, determining whether topic variation is enabled, and, if so, resolving the reviewee team's topic. The system then looks for the best matching assignment questionnaire for that assignment, topic, and round context.

When both round and topic variation are active, matching is deterministic. The precedence is: assignment/topic/current round, then assignment/topic/default round, then assignment/default topic/current round, and finally assignment/default topic/default round. In this precedence rule, the default topic corresponds to <code>project_topic_id</code> being null, and the default round corresponds to <code>used_in_round</code> being null.

If no topic-specific mapping is found, the system falls back according to the defined precedence, including assignment-level round-specific rubrics when applicable, before using the default review rubric for the assignment.

Only the reviewee team's confirmed topic signup for the current assignment is considered, and waitlisted topic signups do not determine rubric selection.

== UI and Instructor Workflow ==

Instructors configure topic-based rubrics directly in assignment setup on the Topics tab. For each topic, the UI exposes dropdowns for selecting the review rubric, and when <code>vary_by_round</code> is enabled, round-specific rubric choices are also supported. This approach keeps all configuration within the standard assignment authoring flow and avoids introducing any external components.

The existing assignment-level review rubric remains available as the default fallback when a topic-specific rubric is not configured. Reviewers do not choose the rubric themselves; instead, they receive the questionnaire resolved automatically for the team they are reviewing.

== API and Validation ==

Assignment create and update flows persist <code>vary_by_topic</code> cleanly, and parameter naming is normalized so that there is a single canonical internal field. Assignment questionnaire updates accept an optional <code>project_topic_id</code> and validate that the topic belongs to the same assignment.

Validation ensures that <code>project_topic_id</code> references an existing topic and does not cross assignment boundaries. The system also prevents duplicate review-rubric mappings for the same assignment, topic, and round combination, ensuring that rubric resolution always produces a single correct result.

Only instructors or administrators who can edit the assignment are allowed to enable topic-based rubric variation or modify topic-rubric mappings.

== Migration and Rollout ==

Migration first adds <code>vary_by_topic</code> to assignments, followed by adding a nullable <code>project_topic_id</code> to assignment questionnaires along with the necessary foreign key and indexes. Existing rows remain valid with a null topic id and continue to represent assignment-level defaults.

After deployment, instructors can enable topic variation gradually on a per-assignment basis using the existing user interface.

== Testing Strategy ==

Tests verify that assignments without topic variation remain unchanged, while topic-enabled assignments correctly select topic-specific rubrics when configured. They also ensure that fallback behavior functions properly when a team has no topic or no topic-specific rubric mapping.

Additional tests confirm deterministic precedence when both topic and round variation are enabled and ensure that scoring behavior remains consistent with the selected questionnaire.

== Risks ==

The main risk is ambiguity in precedence when both topic and round variation are present. This is mitigated by defining a strict lookup order and enforcing it through testing. Another risk is partial configuration, where topic variation is enabled but mappings are incomplete; this is mitigated by safe fallback behavior and validation in configuration paths.

== Open Questions ==

If an instructor changes a topic-rubric mapping after reviews have begun, existing responses could resolve to a different rubric unless the selected mapping is preserved. The implementation should either store the selected assignment questionnaire on the response or prevent mapping changes once reviews are underway.

== Implementation Notes ==

Implementation focuses on schema changes, shared internal rubric lookup logic, integration at current rubric selection call sites, user interface support on the Topics tab, and regression coverage for response and scoring paths.

Known integration points include response questionnaire lookup, scoring maximum-score calculation, and any review form initialization path that currently assumes assignment-level rubric selection.

CSC/ECE 517 Spring 2026 - E2605. Specialized Rubrics for Different Topic Types

2026-04-14T01:21:44Z

Vagarwa3:

= CSC/ECE 517 Spring 2026 - E2605: Specialized Rubrics for Different Topic Types =

__TOC__

== Introduction ==

This wiki page presents the design for Expertiza project '''E2605: Specialized Rubrics for Different Topic Types''' for CSC/ECE 517 Object-Oriented Design and Development, Spring 2026.

The goal of this project is to let instructors use different review rubrics for different project topics within the same assignment. Expertiza currently uses the same review rubric for all submissions in an assignment, unless the assignment is configured to use different rubrics in different review rounds. This works for assignments where every topic should be evaluated with the same criteria, but it does not work well when topics require different kinds of evaluation.

For example, an assignment might include topics related to user-interface design, machine learning, and security. A security topic may need rubric questions about threat modeling or vulnerability analysis, while a user-interface topic may need rubric questions about usability and design quality. Topic-based rubric selection allows the rubric to match the topic of the submission being reviewed.

This document describes the motivation for the change, the proposed data model, the instructor workflow, and how Expertiza will resolve the correct review rubric based on assignment, topic, and review-round information.

== Overview ==

This design adds topic-based review rubric selection while keeping existing behavior for assignments where topic-based rubric selection is not enabled. Instructors will associate review rubrics with topics directly from the Topics tab of Assignment creation, using dropdowns. If an assignment is configured to vary by topic, the review rubric is chosen based on the topic of the submission that is being reviewed. If topic variation is not enabled, rubric selection follows the current assignment and round logic.

A ReviewResponseMap identifies the team whose submission is being reviewed. The <code>signed_up_teams</code> table identifies which topic that team selected. The <code>assignments_questionnaires</code> table will then identify which review rubric should be used for that assignment and topic.

== Problem Statement ==

Different project topics can require different evaluation criteria, but currently all submissions for an assignment use the same review rubric unless round-based rubric variation is configured. The schema does not yet provide a first-class topic-to-rubric mapping, and there is no assignment flag for topic-based rubric variation. Because of this, reviewers can receive rubrics that do not match the reviewed team's topic.

== Design Goals ==

The goal is to let instructors define topic-specific review rubrics without changing the overall review workflow. The design must preserve backward compatibility for existing assignments, continue to support round-based variation, and keep rubric resolution deterministic when both round and topic variation are enabled.

== Non-Goals ==

This change does not redesign topic signup or waitlist behavior, questionnaire authoring or import, or review mapping strategies. It only changes how the system stores and resolves rubric selection for reviews.

== Current State ==

The existing data model already includes the relationships needed to determine a reviewee team's topic. The <code>signed_up_teams</code> table identifies which team has selected which topic, and review mappings already identify the reviewee team. Rubric resolution currently happens in several places, including response and scoring code paths, and is mainly driven by assignment and round context.

The current tables do not yet store all the information needed for topic-based rubric selection. Assignments currently include <code>vary_by_round</code> but no equivalent <code>vary_by_topic</code> flag, and assignment questionnaire records do not have a topic foreign key.

== Proposed Data Model ==

This design introduces minimal extensions to the existing schema while reusing the current rubric infrastructure. A new <code>vary_by_topic</code> boolean field is added to the assignments table with a default value of false. This ensures that existing assignments continue to function without any changes, and instructors can explicitly enable topic-based rubric variation only when needed.

In addition, an optional <code>project_topic_id</code> foreign key is added to the <code>assignments_questionnaires</code> table. A null <code>project_topic_id</code> represents the default review rubric for the assignment. This approach allows topic-specific mappings to coexist naturally with round-based mappings through <code>used_in_round</code>.

Indexes should be added on the new topic field and on the fields used for rubric lookup so that Expertiza can efficiently find the correct rubric for an assignment, topic, and round.

The <code>assignments_questionnaires</code> table already indicates which rubrics are used for an assignment. Adding <code>project_topic_id</code> allows one of those rubric records to apply to a specific topic instead of applying to the whole assignment. A project topic will normally have at most one review-rubric mapping, or one per round if round-based variation is enabled.

No new questionnaire type is introduced. The new <code>project_topic_id</code> field only connects an existing assignment-rubric record to a topic, and the rubric itself remains the existing questionnaire referenced by <code>questionnaire_id</code>.

== Rubric Resolution Behavior ==

No separate selection service is required, and rubric selection remains within existing Expertiza code paths. The workflow proceeds by resolving the assignment from the map, determining whether topic variation is enabled, and, if so, resolving the reviewee team's topic. The system then looks for the best matching assignment questionnaire for that assignment, topic, and round context.

When both round and topic variation are active, matching is deterministic. The precedence is: assignment/topic/current round, then assignment/topic/default round, then assignment/default topic/current round, and finally assignment/default topic/default round. In this precedence rule, the default topic corresponds to <code>project_topic_id</code> being null, and the default round corresponds to <code>used_in_round</code> being null.

If no topic-specific mapping is found, the system falls back according to the defined precedence, including assignment-level round-specific rubrics when applicable, before using the default review rubric for the assignment.

Only the reviewee team's confirmed topic signup for the current assignment is considered, and waitlisted topic signups do not determine rubric selection.

== UI and Instructor Workflow ==

Instructors configure topic-based rubrics directly in assignment setup on the Topics tab. For each topic, the UI exposes dropdowns for selecting the review rubric, and when <code>vary_by_round</code> is enabled, round-specific rubric choices are also supported. This approach keeps all configuration within the standard assignment authoring flow and avoids introducing any external components.

The existing assignment-level review rubric remains available as the default fallback when a topic-specific rubric is not configured. Reviewers do not choose the rubric themselves; instead, they receive the questionnaire resolved automatically for the team they are reviewing.

== API and Validation ==

Assignment create and update flows persist <code>vary_by_topic</code> cleanly, and parameter naming is normalized so that there is a single canonical internal field. Assignment questionnaire updates accept an optional <code>project_topic_id</code> and validate that the topic belongs to the same assignment.

Validation ensures that <code>project_topic_id</code> references an existing topic and does not cross assignment boundaries. The system also prevents duplicate review-rubric mappings for the same assignment, topic, and round combination, ensuring that rubric resolution always produces a single correct result.

Only instructors or administrators who can edit the assignment are allowed to enable topic-based rubric variation or modify topic-rubric mappings.

== Migration and Rollout ==

Migration first adds <code>vary_by_topic</code> to assignments, followed by adding a nullable <code>project_topic_id</code> to assignment questionnaires along with the necessary foreign key and indexes. Existing rows remain valid with a null topic id and continue to represent assignment-level defaults.

After deployment, instructors can enable topic variation gradually on a per-assignment basis using the existing user interface.

== Testing Strategy ==

Tests verify that assignments without topic variation remain unchanged, while topic-enabled assignments correctly select topic-specific rubrics when configured. They also ensure that fallback behavior functions properly when a team has no topic or no topic-specific rubric mapping.

Additional tests confirm deterministic precedence when both topic and round variation are enabled and ensure that scoring behavior remains consistent with the selected questionnaire.

== Risks ==

The main risk is ambiguity in precedence when both topic and round variation are present. This is mitigated by defining a strict lookup order and enforcing it through testing. Another risk is partial configuration, where topic variation is enabled but mappings are incomplete; this is mitigated by safe fallback behavior and validation in configuration paths.

== Open Questions ==

If an instructor changes a topic-rubric mapping after reviews have begun, existing responses could resolve to a different rubric unless the selected mapping is preserved. The implementation should either store the selected assignment questionnaire on the response or prevent mapping changes once reviews are underway.

== Implementation Notes ==

Implementation focuses on schema changes, shared internal rubric lookup logic, integration at current rubric selection call sites, user interface support on the Topics tab, and regression coverage for response and scoring paths.

Known integration points include response questionnaire lookup, scoring maximum-score calculation, and any review form initialization path that currently assumes assignment-level rubric selection.

CSC/ECE 517 Spring 2026 - E2605. Specialized Rubrics for Different Topic Types

2026-04-14T01:16:52Z

Vagarwa3:

= CSC/ECE 517 Spring 2026 - E2605: Specialized Rubrics for Different Topic Types =

__TOC__

== Introduction ==

This wiki page presents the design for Expertiza project '''E2605: Specialized Rubrics for Different Topic Types''' for CSC/ECE 517 Object-Oriented Design and Development (Spring 2026).

The goal of this project is to extend the existing rubric selection mechanism in Expertiza to support topic-based variation. Currently, rubric selection is driven primarily by assignment and round context, which limits the ability to tailor evaluation criteria for different types of topics within the same assignment.

This document describes the motivation for the change, the proposed data model and behavior, and how the system resolves the correct rubric based on assignment, topic, and round context. It also outlines implementation considerations, validation rules, and potential risks associated with the design.

== Overview ==

This design adds topic-based review rubric selection while keeping existing behavior for assignments that do not opt in. Instructors will associate review rubrics with topics directly from the Topics tab of Assignment creation, using dropdowns. If an assignment is configured to vary by topic, the review rubric is chosen based on the topic of the submission that is being reviewed. If topic variation is not enabled, rubric selection follows the current assignment and round logic.

A ReviewResponseMap identifies the team whose submission is being reviewed. The <code>signed_up_teams</code> table identifies which topic that team selected. The <code>assignments_questionnaires</code> table will then identify which review rubric should be used for that assignment and topic.

== Problem Statement ==

Different project topics can require different evaluation criteria, but currently all submissions for an assignment use the same review rubric unless round-based rubric variation is configured. The schema does not yet provide a first-class topic-to-rubric mapping, and there is no assignment flag for topic-based rubric variation. Because of this, reviewers can receive rubrics that do not match the reviewed team's topic.

== Design Goals ==

The goal is to let instructors define topic-specific review rubrics without changing the overall review workflow. The design must preserve backward compatibility for existing assignments, continue to support round-based variation, and keep rubric resolution deterministic when both round and topic variation are enabled.

== Non-Goals ==

This change does not redesign topic signup or waitlist behavior, questionnaire authoring or import, or review mapping strategies. It only changes how the system stores and resolves rubric selection for reviews.

== Current State ==

The existing data model already includes the relationships needed to determine a reviewee team's topic. The <code>signed_up_teams</code> table identifies which team has selected which topic, and review mappings already identify the reviewee team. Rubric resolution currently happens in several places, including response and scoring code paths, and is mainly driven by assignment and round context.

The current tables do not yet store all the information needed for topic-based rubric selection. Assignments currently include <code>vary_by_round</code> but no equivalent <code>vary_by_topic</code> flag, and assignment questionnaire records do not have a topic foreign key.

== Proposed Data Model ==

This design introduces minimal extensions to the existing schema while reusing the current rubric infrastructure. A new <code>vary_by_topic</code> boolean field is added to the assignments table with a default value of false. This ensures that existing assignments continue to function without any changes, and instructors can explicitly enable topic-based rubric variation only when needed.

In addition, an optional <code>project_topic_id</code> foreign key is added to the <code>assignments_questionnaires</code> table. A null <code>project_topic_id</code> represents the default review rubric for the assignment. This approach allows topic-specific mappings to coexist naturally with round-based mappings through <code>used_in_round</code>.

Indexes should be added on the new topic field and on the fields used for rubric lookup so that Expertiza can efficiently find the correct rubric for an assignment, topic, and round.

The <code>assignments_questionnaires</code> table already indicates which rubrics are used for an assignment. Adding <code>project_topic_id</code> allows one of those rubric records to apply to a specific topic instead of applying to the whole assignment. A project topic will normally have at most one review-rubric mapping, or one per round if round-based variation is enabled.

No new questionnaire type is introduced. The new <code>project_topic_id</code> field only connects an existing assignment-rubric record to a topic, and the rubric itself remains the existing questionnaire referenced by <code>questionnaire_id</code>.

== Rubric Resolution Behavior ==

No separate selection service is required, and rubric selection remains within existing Expertiza code paths. The workflow proceeds by resolving the assignment from the map, determining whether topic variation is enabled, and, if so, resolving the reviewee team's topic. The system then looks for the best matching assignment questionnaire for that assignment, topic, and round context.

When both round and topic variation are active, matching is deterministic. The precedence follows assignment and topic with round first, then assignment and topic with default round, followed by assignment with default topic and round, and finally assignment with both default topic and default round. In this precedence rule, the default topic corresponds to <code>project_topic_id</code> being null, and the default round corresponds to <code>used_in_round</code> being null.

If no topic-specific mapping is found, the system falls back according to the defined precedence, including assignment-level round-specific rubrics when applicable, before using the assignment-wide default rubric.

Only the reviewee team's confirmed topic signup for the current assignment is considered, and waitlisted topic signups do not determine rubric selection.

== UI and Instructor Workflow ==

Instructors configure topic-based rubrics directly in assignment setup on the Topics tab. For each topic, the UI exposes dropdowns for selecting the review rubric, and when <code>vary_by_round</code> is enabled, round-specific rubric choices are also supported. This approach keeps all configuration within the standard assignment authoring flow and avoids introducing any external components.

The existing assignment-level review rubric remains available as the default fallback when a topic-specific rubric is not configured. Reviewers do not choose the rubric themselves; instead, they receive the questionnaire resolved automatically for the team they are reviewing.

== API and Validation ==

Assignment create and update flows persist <code>vary_by_topic</code> cleanly, and parameter naming is normalized so that there is a single canonical internal field. Assignment questionnaire updates accept an optional <code>project_topic_id</code> and validate that the topic belongs to the same assignment.

Validation ensures that <code>project_topic_id</code> references an existing topic and does not cross assignment boundaries. The system also prevents duplicate review-rubric mappings for the same assignment, topic, and round combination, ensuring that rubric resolution always produces a single correct result.

Only instructors or administrators who can edit the assignment are allowed to enable topic-based rubric variation or modify topic-rubric mappings.

== Migration and Rollout ==

Migration first adds <code>vary_by_topic</code> to assignments, followed by adding a nullable <code>project_topic_id</code> to assignment questionnaires along with the necessary foreign key and indexes. Existing rows remain valid with a null topic id and continue to represent assignment-level defaults.

After deployment, instructors can enable topic variation gradually on a per-assignment basis using the existing user interface.

== Testing Strategy ==

Tests verify that assignments without topic variation remain unchanged, while topic-enabled assignments correctly select topic-specific rubrics when configured. They also ensure that fallback behavior functions properly when a team has no topic or no topic-specific rubric mapping.

Additional tests confirm deterministic precedence when both topic and round variation are enabled and ensure that scoring behavior remains consistent with the selected questionnaire.

== Risks ==

The main risk is ambiguity in precedence when both topic and round variation are present. This is mitigated by defining a strict lookup order and enforcing it through testing. Another risk is partial configuration, where topic variation is enabled but mappings are incomplete; this is mitigated by safe fallback behavior and validation in configuration paths.

== Open Questions ==

If an instructor changes a topic-rubric mapping after reviews have begun, existing responses could resolve to a different rubric unless the selected mapping is preserved. The implementation should either store the selected assignment questionnaire on the response or prevent mapping changes once reviews are underway.

== Implementation Notes ==

Implementation focuses on schema changes, shared internal rubric lookup logic, integration at current rubric selection call sites, user interface support on the Topics tab, and regression coverage for response and scoring paths.

Known integration points include response questionnaire lookup, scoring maximum-score calculation, and any review form initialization path that currently assumes assignment-level rubric selection.