Expertiza_Wiki - Recent changes [en]

CSC/ECE 517 Spring 2026 - E2617. Testing Questionnaire and Course Models

Zobrook2 — Mon, 04 May 2026 19:42:08 GMT

CSC/ECE 517 Spring 2026 - E2605. Specialized Rubrics for Different Topic Types

Vagarwa3 — Sun, 03 May 2026 01:40:01 GMT

Links

← Older revision		Revision as of 01:40, 3 May 2026
Line 737:		Line 737:

	Demo Video:		Demo Video:
	* ~~Add demo video link here~~		* https://youtu.be/TRxBHXo7GQQ

	== References ==		== References ==

Report Generation Framework (WIP)

Asreeku — Sat, 02 May 2026 21:51:39 GMT

Created page with "= Report Generation = == Design and Implementation Documentation == === Expertiza Reimplementation Back-End === ---- == Overview == The reporting subsystem was ported from the original Expertiza codebase (referred to as '''Repo X''') into the reimplemented Rails API back-end (referred to as '''Repo Y''') and redesigned in the process. Repo X used a Rails helper module (<code>ReportFormatterHelper</code>) that assigned instance variables, such as <code>@reviewers</cod..."

New page

= Report Generation =
== Design and Implementation Documentation ==
=== Expertiza Reimplementation Back-End ===

----

== Overview ==

The reporting subsystem was ported from the original Expertiza codebase
(referred to as '''Repo X''') into the reimplemented Rails API back-end
(referred to as '''Repo Y''') and redesigned in the process.

Repo X used a Rails helper module (<code>ReportFormatterHelper</code>) that
assigned instance variables, such as <code>@reviewers</code> and
<code>@review_scores</code>, for ERB views.

Since Repo Y is a JSON API with no views, instance variables are not applicable.
The architecture was therefore redesigned around a '''streaming reduce pipeline'''
that returns plain Ruby hashes rendered as JSON.

=== Design Goals ===

* Avoid loading entire result sets into memory at once.
* Keep domain-specific computation out of the base class.
* Make each report type independently composable and testable.
* Fix N+1 query patterns present in Repo X.

----

== Anti-Patterns Addressed ==

The design directly addresses two anti-patterns identified during architectural
review of Repo X.

=== The <code>fetch_responses</code> Anti-Pattern ===

Repo X loaded all response records into an unnamed, ad-hoc array before
processing:

<syntaxhighlight lang="ruby">
# WRONG
responses = fetch_responses # full memory load
grouped = group(responses)
metrics = compute_metrics(grouped)
</syntaxhighlight>

This forces the entire result set into Ruby memory, prevents streaming, and
makes the intermediate structure implicit.

The fix is to '''never materialise all rows at once'''. Instead, use
<code>find_each</code> to stream records in batches, so memory usage scales
with the number of '''groups''', not the number of raw rows.

=== Default Metrics in the Base Class ===

Repo X placed domain-specific math directly in the base class:

<syntaxhighlight lang="ruby">
# WRONG — in BaseReport
def compute_metrics(grouped)
grouped.transform_values do |responses|
{
count: responses.size,
avg_score: responses.map(&:score).sum / responses.size.to_f
}
end
end
</syntaxhighlight>

This ties every subclass to one particular shape of computation. The fix is
to move all domain math into per-report accumulator logic. The base class
contains '''zero''' domain math.

==== What "Domain Math" Means ====

'''Domain math''' refers to the business-logic calculations specific to a
particular report type — the actual formulas and aggregations that answer what
the report is trying to show.

It is called "domain" math because it belongs to the problem domain
(peer assessment), not to the generic pipeline machinery.

Each report type has its own domain math:

{| class="wikitable"
! Report !! Its domain math
|-
| Review scores
| <code>(raw_score / max_score) * 100</code> — percentage score per reviewer per round
|-
| Avg/ranges
| <code>max</code>, <code>min</code>, <code>sum / size</code> — score aggregates across a team's reviewers
|-
| Feedback
| Bucket response IDs into <code>round_1</code>, <code>round_2</code>, <code>round_3</code> arrays
|-
| Bookmark rating
| Collect distinct bookmark IDs into a <code>Set</code>
|}

Notice that these are completely different in shape: one computes a percentage,
another computes min/max/avg, and another just collects IDs.

If <code>avg_score</code> lived in <code>BaseReport</code>, every subclass
would either inherit math it does not need — for example, a bookmark report has
no scores — or be forced to override the method just to suppress it.

The rule is therefore:

> <code>BaseReport</code> only defines '''how to run the pipeline''' — stream, group, fold, finalize.
> The '''what to compute''' belongs entirely inside each report's own <code>accumulate</code> and <code>finalize</code> methods.

----

== System Diagrams [To be updated] ==

=== Overall Request Flow ===

The diagram below shows how an incoming HTTP request travels through the
system from the controller down to the pipeline and back out as JSON.

Since MediaWiki does not render TikZ directly, this diagram is represented
as a text-based flow.

<syntaxhighlight lang="text">
HTTP Client (Front-end)
|
| GET /reports/response_report?assignment_id=&type=
v
ReportsController#response_report
|
| params[:type]
v
REPORT_CLASSES[type]
look up concrete class
|
| .new(assignment).run
v
Concrete Report
for example, FeedbackReport
|
| inherits run
v
BaseReport#run
inherited find_each streaming loop
|
| calls subclass methods
v
source -> grouper -> accumulate -> finalize
|
| output hash
v
render json: { ... }
|
| JSON response
v
HTTP Client (Front-end)
</syntaxhighlight>

''Figure: End-to-end request flow for report generation''

=== Pipeline Internals ===

This diagram shows the four stages inside <code>BaseReport#run</code>.
The stages are defined by each concrete subclass; the pipeline loop itself
never changes.

<syntaxhighlight lang="text">
+------------------+ +------------------+
| 1. Source | rows | 2. Grouper |
| AR relation | -----> | lambda: row->key |
| streamed via | | e.g. reviewer_id |
| find_each | +------------------+
+------------------+ |
| key, row
v
+------------------+ +------------------+
| 4. Finalize | state | 3. Accumulate |
| shape state into | <----- | fold row into |
| output hash | | state |
+------------------+ | domain math here |
| +------------------+
|
v
Hash -> JSON
</syntaxhighlight>

Additional notes:

* <code>source</code> uses <code>includes(...)</code> where needed to avoid N+1 queries.
* <code>accumulate</code> handles scores, deduplication, bucketing, and counting depending on the report type.

''Figure: The four stages every report passes through inside BaseReport#run''

=== Class Hierarchy ===

This diagram shows how concrete report classes relate to <code>BaseReport</code>.

Solid inheritance from <code>BaseReport</code> is represented by indentation.
Composition from <code>ReviewReport</code> to its inner pipelines is represented
under the coordinator.

<syntaxhighlight lang="text">
BaseReport
|
|-- ReviewReport (coordinator)
| |
| |-- ReviewersPipeline
| |-- ScoresPipeline
| |-- AvgRangesPipeline
|
|-- FeedbackReport
|-- TeammateReviewReport
|-- BookmarkRatingReport
|-- BasicReport
</syntaxhighlight>

Legend:

* Direct child under <code>BaseReport</code> = inherits <code>BaseReport</code>
* Pipelines under <code>ReviewReport</code> = coordinator runs inner pipeline
* The inner pipelines also inherit <code>BaseReport</code>

''Figure: Class hierarchy for the report generation subsystem''

----

== Architecture: The Streaming Pipeline ==

=== Pipeline Shape ===

All reports are built on a single pipeline template defined in
<code>Reports::BaseReport</code>:

<syntaxhighlight lang="ruby">
def run
state = initial_state
source.find_each(batch_size: 500) do |row|
accumulate(state, grouper.call(row), row)
end
finalize(state)
end
</syntaxhighlight>

The pipeline consists of four concerns:

{| class="wikitable"
! Concern !! Responsibility
|-
| '''Source'''
| ActiveRecord relation streamed via <code>find_each</code>. Subclasses use <code>includes(...)</code> to eagerly load associations and prevent N+1 queries.
|-
| '''Grouper'''
| A lambda <code>(row) -> key</code> that determines how rows are bucketed in the accumulator state.
|-
| '''Accumulate'''
| Folds one row into the state in place. Contains all domain-specific math for that report.
|-
| '''Finalize'''
| Post-processes the finished state into the output hash. Default implementation returns state unchanged.
|}

=== Memory Model ===

State grows proportional to the number of '''groups''', such as the number of
distinct reviewers, not the number of raw rows.

For example, a dataset with 10,000 responses across 20 reviewers keeps only
20 entries in the accumulator state at any point.

=== Base Class Definition ===

<syntaxhighlight lang="ruby">
module Reports
class BaseReport
def initialize(assignment)
@assignment = assignment
end

def run
state = initial_state
source.find_each(batch_size: 500) do |row|
accumulate(state, grouper.call(row), row)
end
finalize(state)
end

private

def source = raise NotImplementedError
def grouper = raise NotImplementedError
def initial_state = raise NotImplementedError

def accumulate(_state, _key, _row)
raise NotImplementedError
end

def finalize(state) = state
end
end
</syntaxhighlight>

----

== Controller ==

=== Dispatch Mechanism ===

The controller uses a constant hash to map type strings to report classes,
replacing the <code>send(type)</code> meta-programming pattern used in Repo X.

<syntaxhighlight lang="ruby">
REPORT_CLASSES = {
'review_response_map' => Reports::ReviewReport,
'feedback_response_map' => Reports::FeedbackReport,
'teammate_review_response_map' => Reports::TeammateReviewReport,
'bookmark_rating_response_map' => Reports::BookmarkRatingReport,
'basic' => Reports::BasicReport
}.freeze

def response_report
type = params.dig(:report, :type) || params[:type] || 'basic'
report_class = REPORT_CLASSES[type]

unless report_class
return render json: { error: "Unknown report type: #{type}" },
status: :unprocessable_entity
end

assignment = Assignment.find(params[:assignment_id] || params[:id])
data = report_class.new(assignment).run
render json: { type: type, assignment_id: assignment.id }.merge(data)
end
</syntaxhighlight>

=== Route ===

<syntaxhighlight lang="text">
GET /reports/response_report?assignment_id=<id>&type=<type>
POST /reports/response_report
</syntaxhighlight>

----

== Report Implementations ==

=== Review Report (<code>review_response_map</code>) ===

The review report is the most complex. It is implemented as a coordinator
class (<code>ReviewReport</code>) that runs three independent inner pipelines
and merges their results.

{| class="wikitable"
! Pipeline !! Source !! Groups by !! Produces
|-
| <code>ReviewersPipeline</code>
| <code>ReviewResponseMap</code>
| <code>reviewer_id</code>
| Sorted reviewer list
|-
| <code>ScoresPipeline</code>
| <code>Response</code> JOIN map
| <code>reviewer_id</code>
| Score percentage per round/reviewee
|-
| <code>AvgRangesPipeline</code>
| <code>Response</code> JOIN map
| <code>[reviewee_id, round]</code>
| Max/min/avg per team/round
|}

==== N+1 Fix: Precomputed Max Question Score ====

Repo X called <code>response.maximum_score</code> inside the accumulation loop.

This method internally calls:

<syntaxhighlight lang="ruby">
response_assignment.assignment_questionnaires
.find_by(used_in_round: round)
.questionnaire
</syntaxhighlight>

This resulted in one query per response.

In Repo Y, both score pipelines precompute a
<code>round -> max_question_score</code> map with a single query before the
pipeline runs:

<syntaxhighlight lang="ruby">
def precompute_max_q_scores
AssignmentQuestionnaire
.joins(:questionnaire)
.where(assignment_id: @assignment.id)
.pluck(:used_in_round, 'questionnaires.max_question_score')
.to_h
end
</syntaxhighlight>

The result, for example <code>{nil => 10, 1 => 10, 2 => 5}</code>, is stored in
<code>@max_q_score</code> and used as a lookup inside <code>accumulate</code>:

<syntaxhighlight lang="ruby">
max_score = total_wt * (@max_q_score[round] || @max_q_score[nil] || 1)
</syntaxhighlight>

==== Sample Response ====

<syntaxhighlight lang="json">
{
"type": "review_response_map",
"assignment_id": 1,
"reviewers": [
{ "id": 5, "user_id": 2, "name": "alice",
"full_name": "Alice Smith", "handle": "alice" }
],
"review_scores": { "5": { "1": { "3": 87.5 } } },
"avg_and_ranges": { "3": { "1": { "max": 92.0,
"min": 75.0,
"avg": 83.5 } } }
}
</syntaxhighlight>

----

=== Feedback Report (<code>feedback_response_map</code>) ===

Produces the list of authors and the IDs of review responses that received
author feedback, bucketed by round for varying-rubric assignments.

<syntaxhighlight lang="ruby">
def source
Response
.joins(:response_map)
.where(
response_maps: { type: 'ReviewResponseMap',
reviewed_object_id: @assignment.id }
)
.order(created_at: :desc)
end

def grouper = ->(r) { [r.map_id, r.round] }
def initial_state = { seen: Set.new, round_1: [],
round_2: [], round_3: [], all: [] }

def accumulate(state, key, response)
return if state[:seen].include?(key)
state[:seen].add(key)
if @assignment.varying_rubrics_by_round?
case response.round
when 1 then state[:round_1] << response.id
when 2 then state[:round_2] << response.id
when 3 then state[:round_3] << response.id
end
else
state[:all] << response.id
end
end
</syntaxhighlight>

Deduplication uses a <code>Set</code>, which gives O(1) lookup, rather than the
array-based <code>seen_map_round_keys.include?</code> from Repo X, which gives
O(n) lookup.

Authors are fetched once in <code>finalize</code>, not inside the stream.

==== End-to-End Execution Flow ====

The <code>run</code> method is '''inherited from <code>BaseReport</code>'''.
<code>FeedbackReport</code> never defines it.

Calling <code>FeedbackReport.new(assignment).run</code> triggers the following
sequence:

<syntaxhighlight lang="text">
ReportsController
REPORT_CLASSES['feedback_response_map'].new(assignment).run
|
| (inherited from BaseReport)
v
BaseReport#run
|
|-- Step 1: state = initial_state
| => { seen: Set.new,
| round_1: [], round_2: [], round_3: [], all: [] }
|
|-- Step 2: source.find_each(batch_size: 500)
| => Response.joins(:response_map)
| .where(type: 'ReviewResponseMap',
| reviewed_object_id: assignment.id)
| .order(created_at: :desc)
| streams Response records newest-first, in batches
|
|-- Step 3: for each Response row:
| key = grouper.call(row)
| => [row.map_id, row.round] e.g. [42, 1]
|
| accumulate(state, key, row)
| => skip if state[:seen] already has [map_id, round]
| (keeps only the latest response per map per round
| because source is ordered newest-first)
| => otherwise: add key to :seen, then bucket row.id:
| round == 1 => state[:round_1] << row.id
| round == 2 => state[:round_2] << row.id
| round == 3 => state[:round_3] << row.id
| (or state[:all] if single-rubric assignment)
|
|-- Step 4: finalize(state)
=> fetch_authors (one query: teams -> users -> participants)
=> if varying_rubrics_by_round?
return { authors: [...],
review_response_ids: {
round_1: [...], round_2: [...], round_3: [...] } }
else
return { authors: [...],
review_response_ids: [...] }
</syntaxhighlight>

The key point is that <code>FeedbackReport</code> only defines the four pieces
the pipeline needs:

* <code>source</code>
* <code>grouper</code>
* <code>initial_state</code>
* <code>accumulate</code>
* <code>finalize</code>

Ruby's inheritance mechanism means calling <code>.run</code> on a
<code>FeedbackReport</code> instance automatically executes
<code>BaseReport#run</code>, which calls back into <code>FeedbackReport</code>'s
implementations of those methods.

==== Sample Response (varying rubrics) ====

<syntaxhighlight lang="json">
{
"type": "feedback_response_map",
"authors": [{ "id": 7, "name": "bob", "full_name": "Bob Jones" }],
"review_response_ids": {
"round_1": [12, 15], "round_2": [18], "round_3": []
}
}
</syntaxhighlight>

----

=== Teammate Review Report (<code>teammate_review_response_map</code>) ===

Streams <code>TeammateReviewResponseMap</code> records grouped by
<code>reviewer_id</code>.

The first occurrence per reviewer is kept using deduplication via early return
if the key already exists in state. Reviewer associations are eagerly loaded.

==== Sample Response ====

<syntaxhighlight lang="json">
{
"type": "teammate_review_response_map",
"reviewers": [
{ "reviewer_id": 5, "user_id": 2,
"name": "alice", "full_name": "Alice Smith" }
]
}
</syntaxhighlight>

----

=== Bookmark Rating Report (<code>bookmark_rating_response_map</code>) ===

Streams <code>BookmarkRatingResponseMap</code> records, accumulating distinct
bookmark IDs into a <code>Set</code>.

Project topics are fetched once in <code>finalize</code>.

==== Bug Fixed During Port ====

The model's <code>bookmark_response_report</code> in Repo Y was incorrectly
calling:

<syntaxhighlight lang="ruby">
.pluck(:reviewed_object_id)
</syntaxhighlight>

This returns assignment IDs, since <code>reviewed_object_id</code> is the
foreign key to <code>Assignment</code>.

Bookmark IDs are stored in <code>reviewee_id</code>. Therefore, this was fixed
to:

<syntaxhighlight lang="ruby">
.pluck(:reviewee_id)
</syntaxhighlight>

==== Sample Response ====

<syntaxhighlight lang="json">
{
"type": "bookmark_rating_response_map",
"bookmark_ids": [10, 14, 22],
"topics": [{ "id": 3, "topic_name": "Machine Learning" }]
}
</syntaxhighlight>

----

=== Basic Report (<code>basic</code>) ===

Returns minimal assignment metadata.

No streaming is required since all data comes from the already-loaded
<code>Assignment</code> object.

This report is used as the default when no type parameter is provided.

==== Sample Response ====

<syntaxhighlight lang="json">
{
"type": "basic",
"assignment_id": 1,
"assignment": {
"id": 1, "name": "Project 1",
"num_review_rounds": 2,
"varying_rubrics_by_round": true
}
}
</syntaxhighlight>

----

== File Structure ==

<syntaxhighlight lang="text">
app/
controllers/
reports_controller.rb Entry point, REPORT_CLASSES dispatch
helpers/
report_formatter_helper.rb Empty namespace (logic moved to services)
services/
reports/
base_report.rb Abstract pipeline template
review_report.rb 3-pipeline coordinator
feedback_report.rb Single pipeline, round bucketing
teammate_review_report.rb Single pipeline
bookmark_rating_report.rb Single pipeline
basic_report.rb Simple struct
models/
review_response_map.rb
.review_response_report class method
feedback_response_map.rb
.feedback_response_report class method
teammate_review_response_map.rb
.teammate_response_report class method
bookmark_rating_response_map.rb
.bookmark_response_report (bug fixed)
</syntaxhighlight>

----

== Blocked Report Types ==

The following report types exist in Repo X but cannot yet be implemented in
Repo Y due to missing database tables or models.

Each is ready to be added once its dependency is ported.

{| class="wikitable"
! Report Type !! Missing Dependency !! Repo X Location
|-
| <code>calibration</code>
| <code>calibrate_to</code> column on <code>response_maps</code>
| <code>report_formatter_helper.rb</code>
|-
| <code>self_review</code>
| <code>SelfReviewResponseMap</code> model
| <code>self_review_response_map.rb</code>
|-
| <code>survey</code>
| <code>survey_deployments</code> table
| <code>survey_response_map.rb</code>
|-
| <code>quiz</code>
| <code>quiz_responses</code> table
| <code>quiz_response_map.rb</code>
|-
| <code>answer_tagging</code>
| <code>tag_prompt_deployments</code>, <code>answer_tags</code> tables
| <code>tag_prompt_deployment.rb</code>
|}

To add a blocked report once its dependencies are available:

# Create <code>app/services/reports/<name>_report.rb</code> inheriting <code>BaseReport</code>.
# Define <code>source</code>, <code>grouper</code>, <code>initial_state</code>, <code>accumulate</code>, and <code>finalize</code>.
# Add an entry to <code>ReportsController::REPORT_CLASSES</code>.

----

== Comparison with Repo X ==

{| class="wikitable"
! Concern !! Repo X !! Repo Y
|-
| Output format
| ERB instance variables, such as <code>@reviewers</code> and <code>@review_scores</code>
| JSON hash from <code>report.run</code>
|-
| Loading strategy
| All records loaded into arrays at once
| <code>find_each</code> batched streaming
|-
| Metrics location
| <code>compute_metrics</code> in helper base
| Each report owns <code>accumulate</code> and <code>finalize</code>
|-
| Dispatch
| <code>send(@type.underscore, params, session)</code>
| <code>REPORT_CLASSES[type].new(assignment).run</code>
|-
| N+1 on scores
| <code>response.maximum_score</code> per row — questionnaire lookup each time
| Precomputed <code>round->max_score</code> map, one query before pipeline
|-
| Deduplication
| <code>Array#include?</code> — O(n) per check
| <code>Set#include?</code> — O(1) per check
|-
| Additional features
| LLM evaluation, CSV export, calibration, self-review, survey, quiz, answer tagging
| Not yet ported; blocked on schema
|}

----
== Author ==

{| class="wikitable"
! Name !! Role
|-
| '''Aanand Sreekumaran Nair Jayakumari'''
| Project contributor / developer
|}

Main Page

Asreeku — Sat, 02 May 2026 21:49:59 GMT

← Older revision		Revision as of 21:49, 2 May 2026
Line 41:		Line 41:
	* [[ReactJs Frontend]]		* [[ReactJs Frontend]]
	* [[Front-End/Back-End]]		* [[Front-End/Back-End]]
	* [[Report ~~generation framework~~(WIP)]]		* [[Report Generation Framework (WIP)]]

	==Application Behavior==		==Application Behavior==

Report generation framework(WIP)

Asreeku — Sat, 02 May 2026 21:49:32 GMT

New page

= Report Generation =
== Design and Implementation Documentation ==
=== Expertiza Reimplementation Back-End ===

----

== Overview ==

The reporting subsystem was ported from the original Expertiza codebase
(referred to as '''Repo X''') into the reimplemented Rails API back-end
(referred to as '''Repo Y''') and redesigned in the process.

Repo X used a Rails helper module (<code>ReportFormatterHelper</code>) that
assigned instance variables, such as <code>@reviewers</code> and
<code>@review_scores</code>, for ERB views.

Since Repo Y is a JSON API with no views, instance variables are not applicable.
The architecture was therefore redesigned around a '''streaming reduce pipeline'''
that returns plain Ruby hashes rendered as JSON.

=== Design Goals ===

* Avoid loading entire result sets into memory at once.
* Keep domain-specific computation out of the base class.
* Make each report type independently composable and testable.
* Fix N+1 query patterns present in Repo X.

----

== Anti-Patterns Addressed ==

The design directly addresses two anti-patterns identified during architectural
review of Repo X.

=== The <code>fetch_responses</code> Anti-Pattern ===

Repo X loaded all response records into an unnamed, ad-hoc array before
processing:

<syntaxhighlight lang="ruby">
# WRONG
responses = fetch_responses # full memory load
grouped = group(responses)
metrics = compute_metrics(grouped)
</syntaxhighlight>

This forces the entire result set into Ruby memory, prevents streaming, and
makes the intermediate structure implicit.

The fix is to '''never materialise all rows at once'''. Instead, use
<code>find_each</code> to stream records in batches, so memory usage scales
with the number of '''groups''', not the number of raw rows.

=== Default Metrics in the Base Class ===

Repo X placed domain-specific math directly in the base class:

<syntaxhighlight lang="ruby">
# WRONG — in BaseReport
def compute_metrics(grouped)
grouped.transform_values do |responses|
{
count: responses.size,
avg_score: responses.map(&:score).sum / responses.size.to_f
}
end
end
</syntaxhighlight>

This ties every subclass to one particular shape of computation. The fix is
to move all domain math into per-report accumulator logic. The base class
contains '''zero''' domain math.

==== What "Domain Math" Means ====

'''Domain math''' refers to the business-logic calculations specific to a
particular report type — the actual formulas and aggregations that answer what
the report is trying to show.

It is called "domain" math because it belongs to the problem domain
(peer assessment), not to the generic pipeline machinery.

Each report type has its own domain math:

{| class="wikitable"
! Report !! Its domain math
|-
| Review scores
| <code>(raw_score / max_score) * 100</code> — percentage score per reviewer per round
|-
| Avg/ranges
| <code>max</code>, <code>min</code>, <code>sum / size</code> — score aggregates across a team's reviewers
|-
| Feedback
| Bucket response IDs into <code>round_1</code>, <code>round_2</code>, <code>round_3</code> arrays
|-
| Bookmark rating
| Collect distinct bookmark IDs into a <code>Set</code>
|}

Notice that these are completely different in shape: one computes a percentage,
another computes min/max/avg, and another just collects IDs.

If <code>avg_score</code> lived in <code>BaseReport</code>, every subclass
would either inherit math it does not need — for example, a bookmark report has
no scores — or be forced to override the method just to suppress it.

The rule is therefore:

> <code>BaseReport</code> only defines '''how to run the pipeline''' — stream, group, fold, finalize.
> The '''what to compute''' belongs entirely inside each report's own <code>accumulate</code> and <code>finalize</code> methods.

----

== System Diagrams [To be updated] ==

=== Overall Request Flow ===

The diagram below shows how an incoming HTTP request travels through the
system from the controller down to the pipeline and back out as JSON.

Since MediaWiki does not render TikZ directly, this diagram is represented
as a text-based flow.

<syntaxhighlight lang="text">
HTTP Client (Front-end)
|
| GET /reports/response_report?assignment_id=&type=
v
ReportsController#response_report
|
| params[:type]
v
REPORT_CLASSES[type]
look up concrete class
|
| .new(assignment).run
v
Concrete Report
for example, FeedbackReport
|
| inherits run
v
BaseReport#run
inherited find_each streaming loop
|
| calls subclass methods
v
source -> grouper -> accumulate -> finalize
|
| output hash
v
render json: { ... }
|
| JSON response
v
HTTP Client (Front-end)
</syntaxhighlight>

''Figure: End-to-end request flow for report generation''

=== Pipeline Internals ===

This diagram shows the four stages inside <code>BaseReport#run</code>.
The stages are defined by each concrete subclass; the pipeline loop itself
never changes.

<syntaxhighlight lang="text">
+------------------+ +------------------+
| 1. Source | rows | 2. Grouper |
| AR relation | -----> | lambda: row->key |
| streamed via | | e.g. reviewer_id |
| find_each | +------------------+
+------------------+ |
| key, row
v
+------------------+ +------------------+
| 4. Finalize | state | 3. Accumulate |
| shape state into | <----- | fold row into |
| output hash | | state |
+------------------+ | domain math here |
| +------------------+
|
v
Hash -> JSON
</syntaxhighlight>

Additional notes:

* <code>source</code> uses <code>includes(...)</code> where needed to avoid N+1 queries.
* <code>accumulate</code> handles scores, deduplication, bucketing, and counting depending on the report type.

''Figure: The four stages every report passes through inside BaseReport#run''

=== Class Hierarchy ===

This diagram shows how concrete report classes relate to <code>BaseReport</code>.

Solid inheritance from <code>BaseReport</code> is represented by indentation.
Composition from <code>ReviewReport</code> to its inner pipelines is represented
under the coordinator.

<syntaxhighlight lang="text">
BaseReport
|
|-- ReviewReport (coordinator)
| |
| |-- ReviewersPipeline
| |-- ScoresPipeline
| |-- AvgRangesPipeline
|
|-- FeedbackReport
|-- TeammateReviewReport
|-- BookmarkRatingReport
|-- BasicReport
</syntaxhighlight>

Legend:

* Direct child under <code>BaseReport</code> = inherits <code>BaseReport</code>
* Pipelines under <code>ReviewReport</code> = coordinator runs inner pipeline
* The inner pipelines also inherit <code>BaseReport</code>

''Figure: Class hierarchy for the report generation subsystem''

----

== Architecture: The Streaming Pipeline ==

=== Pipeline Shape ===

All reports are built on a single pipeline template defined in
<code>Reports::BaseReport</code>:

<syntaxhighlight lang="ruby">
def run
state = initial_state
source.find_each(batch_size: 500) do |row|
accumulate(state, grouper.call(row), row)
end
finalize(state)
end
</syntaxhighlight>

The pipeline consists of four concerns:

{| class="wikitable"
! Concern !! Responsibility
|-
| '''Source'''
| ActiveRecord relation streamed via <code>find_each</code>. Subclasses use <code>includes(...)</code> to eagerly load associations and prevent N+1 queries.
|-
| '''Grouper'''
| A lambda <code>(row) -> key</code> that determines how rows are bucketed in the accumulator state.
|-
| '''Accumulate'''
| Folds one row into the state in place. Contains all domain-specific math for that report.
|-
| '''Finalize'''
| Post-processes the finished state into the output hash. Default implementation returns state unchanged.
|}

=== Memory Model ===

State grows proportional to the number of '''groups''', such as the number of
distinct reviewers, not the number of raw rows.

For example, a dataset with 10,000 responses across 20 reviewers keeps only
20 entries in the accumulator state at any point.

=== Base Class Definition ===

<syntaxhighlight lang="ruby">
module Reports
class BaseReport
def initialize(assignment)
@assignment = assignment
end

def run
state = initial_state
source.find_each(batch_size: 500) do |row|
accumulate(state, grouper.call(row), row)
end
finalize(state)
end

private

def source = raise NotImplementedError
def grouper = raise NotImplementedError
def initial_state = raise NotImplementedError

def accumulate(_state, _key, _row)
raise NotImplementedError
end

def finalize(state) = state
end
end
</syntaxhighlight>

----

== Controller ==

=== Dispatch Mechanism ===

The controller uses a constant hash to map type strings to report classes,
replacing the <code>send(type)</code> meta-programming pattern used in Repo X.

<syntaxhighlight lang="ruby">
REPORT_CLASSES = {
'review_response_map' => Reports::ReviewReport,
'feedback_response_map' => Reports::FeedbackReport,
'teammate_review_response_map' => Reports::TeammateReviewReport,
'bookmark_rating_response_map' => Reports::BookmarkRatingReport,
'basic' => Reports::BasicReport
}.freeze

def response_report
type = params.dig(:report, :type) || params[:type] || 'basic'
report_class = REPORT_CLASSES[type]

unless report_class
return render json: { error: "Unknown report type: #{type}" },
status: :unprocessable_entity
end

assignment = Assignment.find(params[:assignment_id] || params[:id])
data = report_class.new(assignment).run
render json: { type: type, assignment_id: assignment.id }.merge(data)
end
</syntaxhighlight>

=== Route ===

<syntaxhighlight lang="text">
GET /reports/response_report?assignment_id=<id>&type=<type>
POST /reports/response_report
</syntaxhighlight>

----

== Report Implementations ==

=== Review Report (<code>review_response_map</code>) ===

The review report is the most complex. It is implemented as a coordinator
class (<code>ReviewReport</code>) that runs three independent inner pipelines
and merges their results.

{| class="wikitable"
! Pipeline !! Source !! Groups by !! Produces
|-
| <code>ReviewersPipeline</code>
| <code>ReviewResponseMap</code>
| <code>reviewer_id</code>
| Sorted reviewer list
|-
| <code>ScoresPipeline</code>
| <code>Response</code> JOIN map
| <code>reviewer_id</code>
| Score percentage per round/reviewee
|-
| <code>AvgRangesPipeline</code>
| <code>Response</code> JOIN map
| <code>[reviewee_id, round]</code>
| Max/min/avg per team/round
|}

==== N+1 Fix: Precomputed Max Question Score ====

Repo X called <code>response.maximum_score</code> inside the accumulation loop.

This method internally calls:

<syntaxhighlight lang="ruby">
response_assignment.assignment_questionnaires
.find_by(used_in_round: round)
.questionnaire
</syntaxhighlight>

This resulted in one query per response.

In Repo Y, both score pipelines precompute a
<code>round -> max_question_score</code> map with a single query before the
pipeline runs:

<syntaxhighlight lang="ruby">
def precompute_max_q_scores
AssignmentQuestionnaire
.joins(:questionnaire)
.where(assignment_id: @assignment.id)
.pluck(:used_in_round, 'questionnaires.max_question_score')
.to_h
end
</syntaxhighlight>

The result, for example <code>{nil => 10, 1 => 10, 2 => 5}</code>, is stored in
<code>@max_q_score</code> and used as a lookup inside <code>accumulate</code>:

<syntaxhighlight lang="ruby">
max_score = total_wt * (@max_q_score[round] || @max_q_score[nil] || 1)
</syntaxhighlight>

==== Sample Response ====

<syntaxhighlight lang="json">
{
"type": "review_response_map",
"assignment_id": 1,
"reviewers": [
{ "id": 5, "user_id": 2, "name": "alice",
"full_name": "Alice Smith", "handle": "alice" }
],
"review_scores": { "5": { "1": { "3": 87.5 } } },
"avg_and_ranges": { "3": { "1": { "max": 92.0,
"min": 75.0,
"avg": 83.5 } } }
}
</syntaxhighlight>

----

=== Feedback Report (<code>feedback_response_map</code>) ===

Produces the list of authors and the IDs of review responses that received
author feedback, bucketed by round for varying-rubric assignments.

<syntaxhighlight lang="ruby">
def source
Response
.joins(:response_map)
.where(
response_maps: { type: 'ReviewResponseMap',
reviewed_object_id: @assignment.id }
)
.order(created_at: :desc)
end

def grouper = ->(r) { [r.map_id, r.round] }
def initial_state = { seen: Set.new, round_1: [],
round_2: [], round_3: [], all: [] }

def accumulate(state, key, response)
return if state[:seen].include?(key)
state[:seen].add(key)
if @assignment.varying_rubrics_by_round?
case response.round
when 1 then state[:round_1] << response.id
when 2 then state[:round_2] << response.id
when 3 then state[:round_3] << response.id
end
else
state[:all] << response.id
end
end
</syntaxhighlight>

Deduplication uses a <code>Set</code>, which gives O(1) lookup, rather than the
array-based <code>seen_map_round_keys.include?</code> from Repo X, which gives
O(n) lookup.

Authors are fetched once in <code>finalize</code>, not inside the stream.

==== End-to-End Execution Flow ====

The <code>run</code> method is '''inherited from <code>BaseReport</code>'''.
<code>FeedbackReport</code> never defines it.

Calling <code>FeedbackReport.new(assignment).run</code> triggers the following
sequence:

<syntaxhighlight lang="text">
ReportsController
REPORT_CLASSES['feedback_response_map'].new(assignment).run
|
| (inherited from BaseReport)
v
BaseReport#run
|
|-- Step 1: state = initial_state
| => { seen: Set.new,
| round_1: [], round_2: [], round_3: [], all: [] }
|
|-- Step 2: source.find_each(batch_size: 500)
| => Response.joins(:response_map)
| .where(type: 'ReviewResponseMap',
| reviewed_object_id: assignment.id)
| .order(created_at: :desc)
| streams Response records newest-first, in batches
|
|-- Step 3: for each Response row:
| key = grouper.call(row)
| => [row.map_id, row.round] e.g. [42, 1]
|
| accumulate(state, key, row)
| => skip if state[:seen] already has [map_id, round]
| (keeps only the latest response per map per round
| because source is ordered newest-first)
| => otherwise: add key to :seen, then bucket row.id:
| round == 1 => state[:round_1] << row.id
| round == 2 => state[:round_2] << row.id
| round == 3 => state[:round_3] << row.id
| (or state[:all] if single-rubric assignment)
|
|-- Step 4: finalize(state)
=> fetch_authors (one query: teams -> users -> participants)
=> if varying_rubrics_by_round?
return { authors: [...],
review_response_ids: {
round_1: [...], round_2: [...], round_3: [...] } }
else
return { authors: [...],
review_response_ids: [...] }
</syntaxhighlight>

The key point is that <code>FeedbackReport</code> only defines the four pieces
the pipeline needs:

* <code>source</code>
* <code>grouper</code>
* <code>initial_state</code>
* <code>accumulate</code>
* <code>finalize</code>

Ruby's inheritance mechanism means calling <code>.run</code> on a
<code>FeedbackReport</code> instance automatically executes
<code>BaseReport#run</code>, which calls back into <code>FeedbackReport</code>'s
implementations of those methods.

==== Sample Response (varying rubrics) ====

<syntaxhighlight lang="json">
{
"type": "feedback_response_map",
"authors": [{ "id": 7, "name": "bob", "full_name": "Bob Jones" }],
"review_response_ids": {
"round_1": [12, 15], "round_2": [18], "round_3": []
}
}
</syntaxhighlight>

----

=== Teammate Review Report (<code>teammate_review_response_map</code>) ===

Streams <code>TeammateReviewResponseMap</code> records grouped by
<code>reviewer_id</code>.

The first occurrence per reviewer is kept using deduplication via early return
if the key already exists in state. Reviewer associations are eagerly loaded.

==== Sample Response ====

<syntaxhighlight lang="json">
{
"type": "teammate_review_response_map",
"reviewers": [
{ "reviewer_id": 5, "user_id": 2,
"name": "alice", "full_name": "Alice Smith" }
]
}
</syntaxhighlight>

----

=== Bookmark Rating Report (<code>bookmark_rating_response_map</code>) ===

Streams <code>BookmarkRatingResponseMap</code> records, accumulating distinct
bookmark IDs into a <code>Set</code>.

Project topics are fetched once in <code>finalize</code>.

==== Bug Fixed During Port ====

The model's <code>bookmark_response_report</code> in Repo Y was incorrectly
calling:

<syntaxhighlight lang="ruby">
.pluck(:reviewed_object_id)
</syntaxhighlight>

This returns assignment IDs, since <code>reviewed_object_id</code> is the
foreign key to <code>Assignment</code>.

Bookmark IDs are stored in <code>reviewee_id</code>. Therefore, this was fixed
to:

<syntaxhighlight lang="ruby">
.pluck(:reviewee_id)
</syntaxhighlight>

==== Sample Response ====

<syntaxhighlight lang="json">
{
"type": "bookmark_rating_response_map",
"bookmark_ids": [10, 14, 22],
"topics": [{ "id": 3, "topic_name": "Machine Learning" }]
}
</syntaxhighlight>

----

=== Basic Report (<code>basic</code>) ===

Returns minimal assignment metadata.

No streaming is required since all data comes from the already-loaded
<code>Assignment</code> object.

This report is used as the default when no type parameter is provided.

==== Sample Response ====

<syntaxhighlight lang="json">
{
"type": "basic",
"assignment_id": 1,
"assignment": {
"id": 1, "name": "Project 1",
"num_review_rounds": 2,
"varying_rubrics_by_round": true
}
}
</syntaxhighlight>

----

== File Structure ==

<syntaxhighlight lang="text">
app/
controllers/
reports_controller.rb Entry point, REPORT_CLASSES dispatch
helpers/
report_formatter_helper.rb Empty namespace (logic moved to services)
services/
reports/
base_report.rb Abstract pipeline template
review_report.rb 3-pipeline coordinator
feedback_report.rb Single pipeline, round bucketing
teammate_review_report.rb Single pipeline
bookmark_rating_report.rb Single pipeline
basic_report.rb Simple struct
models/
review_response_map.rb
.review_response_report class method
feedback_response_map.rb
.feedback_response_report class method
teammate_review_response_map.rb
.teammate_response_report class method
bookmark_rating_response_map.rb
.bookmark_response_report (bug fixed)
</syntaxhighlight>

----

== Blocked Report Types ==

The following report types exist in Repo X but cannot yet be implemented in
Repo Y due to missing database tables or models.

Each is ready to be added once its dependency is ported.

{| class="wikitable"
! Report Type !! Missing Dependency !! Repo X Location
|-
| <code>calibration</code>
| <code>calibrate_to</code> column on <code>response_maps</code>
| <code>report_formatter_helper.rb</code>
|-
| <code>self_review</code>
| <code>SelfReviewResponseMap</code> model
| <code>self_review_response_map.rb</code>
|-
| <code>survey</code>
| <code>survey_deployments</code> table
| <code>survey_response_map.rb</code>
|-
| <code>quiz</code>
| <code>quiz_responses</code> table
| <code>quiz_response_map.rb</code>
|-
| <code>answer_tagging</code>
| <code>tag_prompt_deployments</code>, <code>answer_tags</code> tables
| <code>tag_prompt_deployment.rb</code>
|}

To add a blocked report once its dependencies are available:

# Create <code>app/services/reports/<name>_report.rb</code> inheriting <code>BaseReport</code>.
# Define <code>source</code>, <code>grouper</code>, <code>initial_state</code>, <code>accumulate</code>, and <code>finalize</code>.
# Add an entry to <code>ReportsController::REPORT_CLASSES</code>.

----

== Comparison with Repo X ==

{| class="wikitable"
! Concern !! Repo X !! Repo Y
|-
| Output format
| ERB instance variables, such as <code>@reviewers</code> and <code>@review_scores</code>
| JSON hash from <code>report.run</code>
|-
| Loading strategy
| All records loaded into arrays at once
| <code>find_each</code> batched streaming
|-
| Metrics location
| <code>compute_metrics</code> in helper base
| Each report owns <code>accumulate</code> and <code>finalize</code>
|-
| Dispatch
| <code>send(@type.underscore, params, session)</code>
| <code>REPORT_CLASSES[type].new(assignment).run</code>
|-
| N+1 on scores
| <code>response.maximum_score</code> per row — questionnaire lookup each time
| Precomputed <code>round->max_score</code> map, one query before pipeline
|-
| Deduplication
| <code>Array#include?</code> — O(n) per check
| <code>Set#include?</code> — O(1) per check
|-
| Additional features
| LLM evaluation, CSV export, calibration, self-review, survey, quiz, answer tagging
| Not yet ported; blocked on schema
|}

----

== Author ==

{| class="wikitable"
! Name !! Role
|-
| '''Aanand Sreekumaran Nair Jayakumari'''
| Project contributor / developer
|}

Main Page

Asreeku — Sat, 02 May 2026 21:43:54 GMT

← Older revision		Revision as of 21:43, 2 May 2026
Line 41:		Line 41:
	* [[ReactJs Frontend]]		* [[ReactJs Frontend]]
	* [[Front-End/Back-End]]		* [[Front-End/Back-End]]
	* [[Report generation framework]]		* [[Report generation framework(WIP)]]

	==Application Behavior==		==Application Behavior==

Report generation framework

Asreeku — Sat, 02 May 2026 21:43:05 GMT

Created page with "# Report Generation"

New page

# Report Generation

CSC/ECE 517 Spring 2026 - E2612. Course-based reports

Rsgopara — Sat, 02 May 2026 21:34:13 GMT

8. Reuse Decisions

← Older revision		Revision as of 21:34, 2 May 2026
Line 298:		Line 298:
	\| Legacy Rails report views \|\| Not reused \|\| Clean reimplementation only		\| Legacy Rails report views \|\| Not reused \|\| Clean reimplementation only
	\|}		\|}

			The project was designed to be minimally intrusive by reusing existing, well-tested code whenever possible instead of rebuilding functionality from scratch. The biggest reuse decision was the shared Table.tsx component, whose TanStack Table foundation already supported grouped headers, sorting, and column visibility, covering most of the report’s interactive needs with only a small addition: disablePaginationRowModel. We also followed the existing patterns in gradesService.ts and Course.tsx for the service and routing layers, keeping the new code consistent and easy to understand within the existing codebase.

	== 9. Tests ==		== 9. Tests ==

Main Page

Asreeku — Sat, 02 May 2026 21:30:20 GMT

← Older revision		Revision as of 21:30, 2 May 2026
Line 41:		Line 41:
	* [[ReactJs Frontend]]		* [[ReactJs Frontend]]
	* [[Front-End/Back-End]]		* [[Front-End/Back-End]]
			* [[Report generation framework]]

	==Application Behavior==		==Application Behavior==

CSC/ECE 517 Spring 2026 - E2612. Course-based reports

Rsgopara — Sat, 02 May 2026 21:28:12 GMT

6.2 Response Shape

← Older revision		Revision as of 21:28, 2 May 2026
(5 intermediate revisions by the same user not shown)
Line 25:		Line 25:

	This project consolidates both into a single unified table, accessible from the Courses list page. The table displays every enrolled student alongside their grades, topics, teammate scores, and author feedback scores for every assignment in the course, with instructor-controlled column visibility to keep the view manageable. The implementation targets Expertiza's reimplemented React/TypeScript frontend. The legacy Rails views serve only as a reference for the underlying data model; no legacy code was reused.		This project consolidates both into a single unified table, accessible from the Courses list page. The table displays every enrolled student alongside their grades, topics, teammate scores, and author feedback scores for every assignment in the course, with instructor-controlled column visibility to keep the view manageable. The implementation targets Expertiza's reimplemented React/TypeScript frontend. The legacy Rails views serve only as a reference for the underlying data model; no legacy code was reused.

			[[File:E2612-FinalFrontend1.png\|1100px]]

	== 2. Requirements ==		== 2. Requirements ==
Line 143:		Line 145:
	=== 6.2 Response Shape ===		=== 6.2 Response Shape ===

	<~~pre~~>		<syntaxhighlight lang="ruby">
	{		{
	"course_id": 44,		"course_id": 44,
Line 183:		Line 185:
	]		]
	}		}
	</~~pre~~>		</syntaxhighlight>

	Assignments are sorted by their final review deadline before building the report. The <code>topic</code> field is present only when <code>has_topics</code> is true for that assignment.		Assignments are sorted by their final review deadline before building the report. The <code>topic</code> field is present only when <code>has_topics</code> is true for that assignment.
Line 236:		Line 238:
	\|}		\|}

	=== 7.3 ~~Report Page Screenshot~~ ===		=== 7.3 Frontend Type Definitions ===

	~~[[File:E2612-FinalFrontend1.png\|800px]]~~		<syntaxhighlight lang="ruby">

	=== ~~7.4 Frontend Type Definitions ===~~

	~~<pre~~>
	interface AssignmentMetadata {		interface AssignmentMetadata {
	assignment_id: number;		assignment_id: number;
Line 278:		Line 276:
	avgAuthorFeedbackScore: boolean;		avgAuthorFeedbackScore: boolean;
	};		};
	</~~pre~~>		</syntaxhighlight>

	== 8. Reuse Decisions ==		== 8. Reuse Decisions ==

CSC/ECE 517 Spring 2026 - E2612. Course-based reports

Mskamat — Sat, 02 May 2026 19:19:07 GMT

@@ Line 301: / Line 301: @@
 |}
-== 9. Out of Scope ==
+== 9. Tests ==
 The following are not part of this implementation: