Reports and Course Summary

Reports and Course Summary

Overview

The reporting subsystem was ported from the original Expertiza codebase (referred to as Repo X) into the reimplemented Rails API back-end (referred to as Repo Y) and redesigned in the process.

Repo X used a Rails helper module (ReportFormatterHelper) that assigned instance variables, such as @reviewers and @review_scores, for ERB views.

Since Repo Y is a JSON API with no views, instance variables are not applicable. The architecture was therefore redesigned around a streaming reduce pipeline that returns plain Ruby hashes rendered as JSON.

Design Goals

• Avoid loading entire result sets into memory at once.
• Keep domain-specific computation out of the base class.
• Make each report type independently composable and testable.
• Fix N+1 query patterns present in Repo X.

Anti-Patterns Addressed

The design directly addresses two anti-patterns identified during architectural review of Repo X.

The fetch_responses Anti-Pattern

Repo X loaded all response records into an unnamed, ad-hoc array before processing:

# WRONG
responses = fetch_responses   # full memory load
grouped   = group(responses)
metrics   = compute_metrics(grouped)

This forces the entire result set into Ruby memory, prevents streaming, and makes the intermediate structure implicit.

The fix is to never materialise all rows at once. Instead, use find_each to stream records in batches, so memory usage scales with the number of groups, not the number of raw rows.

Default Metrics in the Base Class

Repo X placed domain-specific math directly in the base class:

# WRONG — in BaseReport
def compute_metrics(grouped)
  grouped.transform_values do |responses|
    {
      count:     responses.size,
      avg_score: responses.map(&:score).sum / responses.size.to_f
    }
  end
end

This ties every subclass to one particular shape of computation. The fix is to move all domain math into per-report accumulator logic. The base class contains zero domain math.

What "Domain Math" Means

Domain math refers to the business-logic calculations specific to a particular report type.

Report	Its domain math
Review scores	(raw_score / max_score) * 100 — percentage score per reviewer per round
Avg/ranges	max, min, sum / size — score aggregates across a team's reviewers
Feedback	Bucket response IDs into round_1, round_2, round_3 arrays
Bookmark rating	Collect distinct bookmark IDs into a Set

Rule: BaseReport only defines how to run the pipeline — stream, group, fold, finalize. The what to compute belongs entirely inside each report's own accumulate and finalize methods.

Architecture: The Streaming Pipeline

Pipeline Shape

All reports are built on a single pipeline template defined in Reports::BaseReport:

def run
  state = initial_state
  source.find_each(batch_size: 500) do |row|
    accumulate(state, grouper.call(row), row)
  end
  finalize(state)
end

The pipeline consists of four concerns:

Concern	Responsibility
Source	ActiveRecord relation streamed via find_each. Subclasses use includes(...) to eagerly load associations and prevent N+1 queries.
Grouper	A lambda (row) -> key that determines how rows are bucketed in the accumulator state.
Accumulate	Folds one row into the state in place. Contains all domain-specific math for that report.
Finalize	Post-processes the finished state into the output hash. Default implementation returns state unchanged.

Memory Model

State grows proportional to the number of groups (such as distinct reviewers), not the number of raw rows.

For example, a dataset with 10,000 responses across 20 reviewers keeps only 20 entries in the accumulator state at any point.

Base Class Definition

module Reports
  class BaseReport
    def initialize(assignment)
      @assignment = assignment
    end

    def run
      state = initial_state
      source.find_each(batch_size: 500) do |row|
        accumulate(state, grouper.call(row), row)
      end
      finalize(state)
    end

    private

    def source        = raise NotImplementedError
    def grouper       = raise NotImplementedError
    def initial_state = raise NotImplementedError

    def accumulate(_state, _key, _row)
      raise NotImplementedError
    end

    def finalize(state) = state
  end
end

Controller

Dispatch Mechanism

The controller uses a constant hash to map type strings to report classes, replacing the send(type) meta-programming pattern used in Repo X.

REPORT_CLASSES = {
  'review_response_map'          => Reports::ReviewReport,
  'feedback_response_map'        => Reports::FeedbackReport, # To be implemented
  'teammate_review_response_map' => Reports::TeammateReviewReport,
  'bookmark_rating_response_map' => Reports::BookmarkRatingReport, # To be implemented
  'basic'                        => Reports::BasicReport
}.freeze

def response_report
  type         = params.dig(:report, :type) || params[:type] || 'basic'
  report_class = REPORT_CLASSES[type]

  unless report_class
    return render json: { error: "Unknown report type: #{type}" },
                  status: :unprocessable_entity
  end

  assignment = Assignment.find(params[:assignment_id] || params[:id])
  data = report_class.new(assignment).run
  render json: { type: type, assignment_id: assignment.id }.merge(data)
end

Routes

GET  /reports/response_report?assignment_id=<id>&type=<type>
POST /reports/response_report
PATCH /review_reports/:id/update_grade

Report Implementations

Review Report (review_response_map)

The review report is the most complex. It is implemented as a coordinator class (ReviewReport) that runs three independent inner pipelines and merges their results.

Pipeline	Source	Groups by	Produces
ReviewersPipeline	ReviewResponseMap	reviewer_id	Sorted reviewer list
ScoresPipeline	Response JOIN map	reviewer_id	Score percentage per round/reviewee
AvgRangesPipeline	Response JOIN map	[reviewee_id, round]	Max/min/avg per team/round

N+1 Fix: Precomputed Max Question Score

Repo X called response.maximum_score inside the accumulation loop, resulting in one query per response. In Repo Y, both score pipelines precompute a round -> max_question_score map with a single query before the pipeline runs:

def precompute_max_q_scores
  AssignmentQuestionnaire
    .joins(:questionnaire)
    .where(assignment_id: @assignment.id)
    .pluck(:used_in_round, 'questionnaires.max_question_score')
    .to_h
end

# Used inside accumulate:
max_score = total_wt * (@max_q_score[round] || @max_q_score[nil] || 1)

Sample Response

{
  "type": "review_response_map",
  "assignment_id": 1,
  "reviewers": [
    { "id": 5, "user_id": 2, "name": "alice",
      "full_name": "Alice Smith", "handle": "alice" }
  ],
  "review_scores": { "5": { "1": { "3": 87.5 } } },
  "avg_and_ranges": { "3": { "1": { "max": 92.0, "min": 75.0, "avg": 83.5 } } }
}

Feedback Report (feedback_response_map)

Produces the list of authors and the IDs of review responses that received author feedback, bucketed by round for varying-rubric assignments.

Deduplication uses a Set (O(1) lookup), rather than the array-based seen_map_round_keys.include? from Repo X (O(n) lookup). Authors are fetched once in finalize, not inside the stream.

Sample Response (varying rubrics)

{
  "type": "feedback_response_map",
  "authors": [{ "id": 7, "name": "bob", "full_name": "Bob Jones" }],
  "review_response_ids": {
    "round_1": [12, 15], "round_2": [18], "round_3": []
  }
}

Teammate Review Report (teammate_review_response_map)

Streams TeammateReviewResponseMap records grouped by reviewer_id. The first occurrence per reviewer is kept using deduplication via early return. Reviewer associations are eagerly loaded.

See the TeammateReviewReportPage section for the corresponding frontend.

Sample Response

{
  "type": "teammate_review_response_map",
  "reviewers": [
    { "reviewer_id": 5, "user_id": 2,
      "name": "alice", "full_name": "Alice Smith" }
  ]
}

Bookmark Rating Report (bookmark_rating_response_map)

To be implemented.

Basic Report (basic)

Returns minimal assignment metadata. No streaming is required since all data comes from the already-loaded Assignment object. Used as the default when no type parameter is provided.

Sample Response

{
  "type": "basic",
  "assignment_id": 1,
  "assignment": {
    "id": 1, "name": "Project 1",
    "num_review_rounds": 2,
    "varying_rubrics_by_round": true
  }
}

Course Grade Summary

Instructor-facing endpoint that aggregates peer scores and instructor grades per student per assignment within a course. Lives in CourseReportsController, which includes PenaltyHelper.

Route:

GET /courses/:id/course_report/grade_summary

Key computations:

• Peer score — precompute_peer_scores bulk-loads all ReviewResponseMap records for the course in one query, computing weighted average percentage scores per (assignment, team) pair. Reviewer weight comes from ReviewGrade#grade_for_reviewer; defaults to 1.0 if absent.
• Instructor grade — team.grade_for_submission minus the late penalty from PenaltyHelper#get_penalty(participant_id)[:submission]. Returns nil when no grade has been set.
• Penalty guard — get_penalty returns { submission: 0, review: 0, meta_review: 0 } immediately when the assignment has no late policy (@penalty_per_unit is nil), avoiding constant-lookup errors for MetareviewResponseMap.
• Calibrated assignments excluded — assignments where is_calibrated: true are filtered out.
• Empty assignments excluded — assignments with no participants are rejected from the response.

def precompute_peer_scores(assignment_ids, team_ids)
  return {} if team_ids.empty?

  maps = ReviewResponseMap
    .where(reviewed_object_id: assignment_ids, reviewee_id: team_ids)
    .includes(responses: :scores)

  reviewer_grades = ReviewGrade
    .where(participant_id: maps.map(&:reviewer_id).uniq)
    .index_by(&:participant_id)

  weighted = Hash.new { |h, k| h[k] = { sum: 0.0, weight: 0.0 } }
  maps.each do |map|
    w = reviewer_grades[map.reviewer_id]&.grade_for_reviewer || 1.0
    map.responses.select(&:is_submitted).each do |resp|
      max = resp.maximum_score
      next if max.zero?
      pct = resp.aggregate_questionnaire_score.to_f / max * 100
      weighted[[map.reviewed_object_id, map.reviewee_id]][:sum]    += pct * w
      weighted[[map.reviewed_object_id, map.reviewee_id]][:weight] += w
    end
  end

  weighted.transform_values { |v| (v[:sum] / v[:weight]).round(2) }
end

Sample Response:

{
  "course_id": 1,
  "course_name": "CSC 517",
  "assignments": [{ "id": 10, "name": "Project 1", "has_topics": true }],
  "rows": [
    {
      "user_id": 5,
      "user_name": "alice",
      "assignments": [
        {
          "assignment_id": 10,
          "assignment_name": "Project 1",
          "topic": "Machine Learning",
          "peer_score": 87.5,
          "instructor_grade": 85.0
        }
      ],
      "final_grade": 85.0
    }
  ]
}

Course All Reviews

Aggregates teammate review scores received per student per assignment within a course. Companion endpoint to grade summary.

Route:

GET /courses/:id/course_report/all_reviews

Key computations:

• Teammate scores — precompute_teammate_scores bulk-loads TeammateReviewResponseMap records and returns { participant_id => "avg%" }.
• Teammate count — precompute_teammate_counts counts distinct teammates per user across all assignments in the course.

Sample Response:

{
  "course_id": 1,
  "course_name": "CSC 517",
  "assignments": [{ "id": 10, "name": "Project 1" }],
  "rows": [
    {
      "user_id": 5,
      "user_name": "alice",
      "teammate_count": 2,
      "assignments": [
        { "assignment_id": 10, "assignment_name": "Project 1", "teammate_review": "83%" }
      ],
      "aggregate": "83%"
    }
  ]
}

ReviewGrade Model

Stores an instructor's grade and comment for a reviewer participant. Created alongside the Review Report frontend to support the grade/comment save workflow.

Migrations:

# 20260810135129_create_review_grades.rb
create_table :review_grades, if_not_exists: true do |t|
  t.integer  :participant_id, null: false, index: { unique: true }
  t.float    :grade_for_reviewer
  t.text     :comment_for_reviewer
  t.integer  :grader_id
  t.timestamps
end

# 20260813000001_add_instructor_grade_scores_to_assignments.rb
add_column :assignments, :instructor_grade_min_score, :integer
add_column :assignments, :instructor_grade_max_score, :integer

These two columns back the instructor_grade_min_score and instructor_grade_max_score fields exposed in assignment_params. The frontend's GradeCommentCell uses them to determine the valid grade range and warn instructors when a saved grade would fall outside it.

Endpoint:

PATCH /review_reports/:id/update_grade
# params: { grade_for_reviewer:, comment_for_reviewer: }
# :id is the participant_id (AssignmentParticipant)

Known limitation: ReviewGrade is per-participant (one row per reviewer), not per response-map. This means the grade/comment field appears on every row for a reviewer in the UI. This matches the original Expertiza schema and is deferred.

Frontend Architecture and Renders

The following React/TypeScript pages implement the instructor-facing report views. All pages are gated behind instructor privileges.

Assignments

Few changes have been done to the assignment pages to include the reports and instructor grading scales.

• 

  This is the Assignments tab for an instructor/admin.

• 

  Review the strategy tab for an assignment where min and max scores for the reviews can be set.

• 

  The implemented reports have been listed in the View Reports button.

ReviewReportPage

File: src/pages/ReviewReportPage/ReviewReportPage.tsx

• Rounds 1–5 visibility toggled per reviewer row
• MetricsChart — per-round averages with a Total bar; rounds where a reviewer did not participate are suppressed from the chart entirely
• Status color coding per reviewer, including brown pre-population on reload from reviewer_grades in the fetch response
• Scores awarded and average score display
• Grade/comment save via PATCH /review_reports/:id/update_grade
• CSV export, column sorting, search filter
• Summary modal per reviewer
• Color cutoffs at 90/80/70/60 (A/B/C/D scale) via getColorClass
• Dynamic color bands via scoreToColor() — k = min(maxScore, 10) HSL bands, green → red spectrum
• Coloring relative to observed data range (dataMin/dataMax per round), not absolute rubric max
• Review metrics column narrowed; barGap=1; slotWidth=75px between groups

• 

  The Review Report page showing reviewer scores and reviewer grade for an assignment with a collapsible legend.

• 

  The Review Report page (with sticky headers) showing reviewer scores and visual metrics comparing the reviewer and average.

• 

  The Review Report page summary button for a reviewer redirects to a new tab with the reviews done segregated by team and round with heatgrid styling enabled.

TeammateReviewReportPage

File: src/pages/TeammateReviewReportPage/TeammateReviewReportPage.tsx

• Team grouping column
• View modal showing scores per reviewee
• Assignment name heading
• Compact layout: p-3 container, line-height: 1.4, padding 6px 24px 6px 8px on th/td
• tableStyle=Template:Width: "fit-content" — table shrinks to content, no full-width stretch
• "Reviews" header on the view-link column
• CSS scoped under .teammate-review-report-page (does not affect Review Report)
• Status color #dc3545 for below-threshold scores

• 

  The Teammate Review Report page showing the status of reviews required by teammates about each other.

• 

  Detailed teammate reviews page redirected from the previous Teammate Review Report page view button

Courses

Few changes have been done to the courses page to redirect to the new course reports. Two new buttons have been made for that. Each assignment in a course has a button to redirect to the ReviewReport page of that assignment.

• 

  This is the courses tab accessed by instructors/admin.

Course Report Pages

Accessible from the course list: expand a course → assignment row → "View Review Report" button.

Page	Frontend Route	Backend Endpoint
CourseGradeSummaryPage	courses/:courseId/course-report/grade-summary	GET /courses/:id/course_report/grade_summary
CourseAllReviewsPage	courses/:courseId/course-report/all-reviews	GET /courses/:id/course_report/all_reviews

Both pages render a heat-grid layout where each cell's color is determined by the value's position in the observed data range (not a fixed rubric max).

• 

  A horizontal scroll and heatstyle enabled table which shows detailed grade distribution for each students in the course per assignment.

• 

  A horizontal scroll and heatstyle enabled table which shows detailed teammate reviews given and received by each teammate per assignment.

• 

  The Review Report page summary button for a reviewer redirects to a new tab with the reviews done segregated by team and round with heatgrid styling enabled.

Shared Utilities & Heat Grid

src/utils/heatgridUtils.ts

• getHeatColorClass(value, dataMin, dataMax) — relative coloring, maps value to a CSS class based on its position between dataMin and dataMax.
• getColorClass(score) — absolute grade-scale coloring (90/80/70/60 cutoffs).

src/utils/reviewTypes.ts

• Exports ReviewData and SectionHeaderData interfaces. Re-exported from ViewTeamGrades/App.tsx.

Heat grid CSS (in custom.scss, global, after Bootstrap import, with !important):

.c5 { background: #1a7a4a; color: #fff; }  /* A — high */
.c4 { background: #5cb85c; }               /* B */
.c3 { background: #f0ad4e; }               /* C */
.c2 { background: #d9534f; color: #fff; }  /* D */
.c1 { background: #8b1a1a; color: #fff; }  /* F — low */
.c0 { background: #e0e0e0; }               /* no data */
.cf { background: #f8f8f8; }               /* empty */

Frontend Routes (App.tsx)

courses/:courseId/course-report/grade-summary  →  CourseGradeSummaryPage
courses/:courseId/course-report/all-reviews    →  CourseAllReviewsPage

Dropdown on the Review Report page: Review Report | Teammate Review Report. (Per Team report removed — no counterpart in Repo X.)

Authorization Changes

Controller	Change
AssignmentsController	Added action_allowed? → current_user_has_instructor_privileges?. Previously defaulted to true — any user could CRUD assignments.
CoursesController#index	Scoped to Course.where(instructor_id: current_user.id). Admins still see Course.all.
AssignmentsController#index	Scoped to instructor's own assignments plus assignments under their courses. Admins see Assignment.all.
ReportsController	action_allowed? now checks current_user_has_admin_privileges?	current_user_teaching_staff_of_assignment?. Removed overly broad instructor_privileges?.
CourseReportsController	New controller; auth: current_user_has_instructor_privileges?.

RSpec Test Coverage

Spec file	Covers
spec/requests/api/v1/review_reports_controller_spec.rb	Response report fetch, update_grade endpoint
spec/requests/api/v1/review_grade_conflicts_spec.rb	ReviewGrade conflict detection endpoint
spec/requests/api/v1/course_reports_controller_spec.rb	grade_summary (incl. has_topics flag, weighted peer scores, penalty integration), all_reviews
spec/requests/api/v1/teammate_review_report_spec.rb	Teammate review report endpoint
spec/models/response_map_spec.rb	Added .compute_average_reviewer_score describe block (8 cases, uses instance_double)
spec/requests/api/v1/assignment_controller_spec.rb	show/update with AQ data, instructor_grade_min/max fields

Pending Roadmap

Items identified in the 2026-08-19 team meeting, not yet implemented:

• Info button on the heat grid explaining relative coloring (displayed max ≠ 100)
• Toggle to show/hide the metrics/chart column
• "Volume" column header label derived from active metrics (not hardwired)
• Move rubric scale indicator (e.g. /5, −2 to 2) to column header — not repeated per cell
• Partial review display: exclude incomplete rounds from averages; use purple; show timestamp / em-dash per round
• Grade change warning banner when instructor lowers max points below an already-assigned score

Previously completed from the same meeting:

• Color cutoffs changed to 90/80/70/60 (A/B/C/D scale)
• Dynamic color bands via scoreToColor()
• "Question" column header renamed to "Item" in heatgrid and feedback table
• /100 hardcoded scale label removed from GradeCommentCell
• Avg bar suppressed for rounds where reviewer did not participate

File Structure

app/
  controllers/
    reports_controller.rb               Entry point, REPORT_CLASSES dispatch
    course_reports_controller.rb        Grade summary & all-reviews endpoints  [NEW]
    assignments_controller.rb           show/update with render body: fix
  helpers/
    report_formatter_helper.rb          Empty namespace (logic moved to services)
    penalty_helper.rb                   get_penalty with nil-guard early return
  services/
    reports/
      base_report.rb                    Abstract pipeline template
      review_report.rb                  3-pipeline coordinator
      feedback_report.rb                Single pipeline, round bucketing
      teammate_review_report.rb         Single pipeline
      bookmark_rating_report.rb         Single pipeline
      basic_report.rb                   Simple struct
  models/
    review_grade.rb                     Grade & comment per reviewer  [NEW]
    response.rb                         maximum_score nil guard
    review_response_map.rb
    feedback_response_map.rb
    teammate_review_response_map.rb
    bookmark_rating_response_map.rb

db/
  migrate/
    20260810135129_create_review_grades.rb                      [NEW]
    20260813000001_add_instructor_grade_scores_to_assignments.rb  [NEW]

spec/
  requests/api/v1/
    review_reports_controller_spec.rb   [NEW]
    review_grade_conflicts_spec.rb      [NEW]
    course_reports_controller_spec.rb   [NEW]
    teammate_review_report_spec.rb      [NEW]
    assignment_controller_spec.rb       Updated
  models/
    response_map_spec.rb                compute_average_reviewer_score block  [NEW]

# Frontend (reimplementation-front-end repo)
src/
  pages/
    ReviewReportPage/ReviewReportPage.tsx         [NEW]
    TeammateReviewReportPage/TeammateReviewReportPage.tsx  [NEW]
    CourseGradeSummaryPage/CourseGradeSummaryPage.tsx      [NEW]
    CourseAllReviewsPage/CourseAllReviewsPage.tsx           [NEW]
  utils/
    heatgridUtils.ts                    getColorClass, getHeatColorClass  [NEW]
    reviewTypes.ts                      ReviewData, SectionHeaderData  [NEW]
  styles/
    custom.scss                         .c0–.c5, .cf heat-grid classes

Blocked Report Types

The following report types exist in Repo X but cannot yet be implemented in Repo Y due to missing database tables or models. Each is ready to be added once its dependency is ported.

Report Type	Missing Dependency	Repo X Location
calibration	calibrate_to column on response_maps	report_formatter_helper.rb
self_review	SelfReviewResponseMap model	self_review_response_map.rb
survey	survey_deployments table	survey_response_map.rb
quiz	quiz_responses table	quiz_response_map.rb
answer_tagging	tag_prompt_deployments, answer_tags tables	tag_prompt_deployment.rb

To add a blocked report once its dependencies are available:

1. Create app/services/reports/<name>_report.rb inheriting BaseReport.
2. Define source, grouper, initial_state, accumulate, and finalize.
3. Add an entry to ReportsController::REPORT_CLASSES.

Comparison with Repo X

Concern	Repo X	Repo Y
Output format	ERB instance variables (@reviewers, @review_scores)	JSON hash from report.run
Loading strategy	All records loaded into arrays at once	find_each batched streaming
Metrics location	compute_metrics in helper base	Each report owns accumulate and finalize
Dispatch	send(@type.underscore, params, session)	REPORT_CLASSES[type].new(assignment).run
N+1 on scores	response.maximum_score per row — questionnaire lookup each time	Precomputed round→max_score map, one query before pipeline
Deduplication	Array#include? — O(n) per check	Set#include? — O(1) per check
Course reports	Grade book & review summary in ERB views	CourseReportsController with JSON API endpoints
Reviewer grading	ReviewGrade on Assignment model	Dedicated ReviewGrade model, update_grade endpoint
Additional features	LLM evaluation, CSV export, calibration, self-review, survey, quiz, answer tagging	CSV export ✓; others blocked on schema

Author

Name	Role
Aanand Sreekumaran Nair Jayakumari	Project contributor / developer — Report Generation Framework
Bestin Lalu	Project contributor / developer — Course Reports, ReviewGrade model, Frontend