View source for Report Generation Framework (WIP)

= Report Generation =
== Design and Implementation Documentation ==
=== Expertiza Reimplementation Back-End ===

----

== Overview ==

The reporting subsystem was ported from the original Expertiza codebase
(referred to as '''Repo X''') into the reimplemented Rails API back-end
(referred to as '''Repo Y''') and redesigned in the process.

Repo X used a Rails helper module (<code>ReportFormatterHelper</code>) that
assigned instance variables, such as <code>@reviewers</code> and
<code>@review_scores</code>, for ERB views.

Since Repo Y is a JSON API with no views, instance variables are not applicable.
The architecture was therefore redesigned around a '''streaming reduce pipeline'''
that returns plain Ruby hashes rendered as JSON.

=== Design Goals ===

* Avoid loading entire result sets into memory at once.
* Keep domain-specific computation out of the base class.
* Make each report type independently composable and testable.
* Fix N+1 query patterns present in Repo X.

----

== Anti-Patterns Addressed ==

The design directly addresses two anti-patterns identified during architectural
review of Repo X.

=== The <code>fetch_responses</code> Anti-Pattern ===

Repo X loaded all response records into an unnamed, ad-hoc array before
processing:

<syntaxhighlight lang="ruby">
# WRONG
responses = fetch_responses   # full memory load
grouped   = group(responses)
metrics   = compute_metrics(grouped)
</syntaxhighlight>

This forces the entire result set into Ruby memory, prevents streaming, and
makes the intermediate structure implicit.

The fix is to '''never materialise all rows at once'''. Instead, use
<code>find_each</code> to stream records in batches, so memory usage scales
with the number of '''groups''', not the number of raw rows.

=== Default Metrics in the Base Class ===

Repo X placed domain-specific math directly in the base class:

<syntaxhighlight lang="ruby">
# WRONG — in BaseReport
def compute_metrics(grouped)
  grouped.transform_values do |responses|
    {
      count:     responses.size,
      avg_score: responses.map(&:score).sum / responses.size.to_f
    }
  end
end
</syntaxhighlight>

This ties every subclass to one particular shape of computation. The fix is
to move all domain math into per-report accumulator logic. The base class
contains '''zero''' domain math.

==== What "Domain Math" Means ====

'''Domain math''' refers to the business-logic calculations specific to a
particular report type — the actual formulas and aggregations that answer what
the report is trying to show.

It is called "domain" math because it belongs to the problem domain
(peer assessment), not to the generic pipeline machinery.

Each report type has its own domain math:

{| class="wikitable"
! Report !! Its domain math
|-
| Review scores
| <code>(raw_score / max_score) * 100</code> — percentage score per reviewer per round
|-
| Avg/ranges
| <code>max</code>, <code>min</code>, <code>sum / size</code> — score aggregates across a team's reviewers
|-
| Feedback
| Bucket response IDs into <code>round_1</code>, <code>round_2</code>, <code>round_3</code> arrays
|-
| Bookmark rating
| Collect distinct bookmark IDs into a <code>Set</code>
|}

Notice that these are completely different in shape: one computes a percentage,
another computes min/max/avg, and another just collects IDs.

If <code>avg_score</code> lived in <code>BaseReport</code>, every subclass
would either inherit math it does not need — for example, a bookmark report has
no scores — or be forced to override the method just to suppress it.

The rule is therefore:

> <code>BaseReport</code> only defines '''how to run the pipeline''' — stream, group, fold, finalize.  
> The '''what to compute''' belongs entirely inside each report's own <code>accumulate</code> and <code>finalize</code> methods.

----

== System Diagrams [To be updated] ==

=== Overall Request Flow ===

The diagram below shows how an incoming HTTP request travels through the
system from the controller down to the pipeline and back out as JSON.

Since MediaWiki does not render TikZ directly, this diagram is represented
as a text-based flow.

<syntaxhighlight lang="text">
HTTP Client (Front-end)
        |
        | GET /reports/response_report?assignment_id=&type=
        v
ReportsController#response_report
        |
        | params[:type]
        v
REPORT_CLASSES[type]
look up concrete class
        |
        | .new(assignment).run
        v
Concrete Report
for example, FeedbackReport
        |
        | inherits run
        v
BaseReport#run
inherited find_each streaming loop
        |
        | calls subclass methods
        v
source -> grouper -> accumulate -> finalize
        |
        | output hash
        v
render json: { ... }
        |
        | JSON response
        v
HTTP Client (Front-end)
</syntaxhighlight>

''Figure: End-to-end request flow for report generation''

=== Pipeline Internals ===

This diagram shows the four stages inside <code>BaseReport#run</code>.
The stages are defined by each concrete subclass; the pipeline loop itself
never changes.

<syntaxhighlight lang="text">
+------------------+        +------------------+
| 1. Source        | rows   | 2. Grouper       |
| AR relation      | -----> | lambda: row->key |
| streamed via     |        | e.g. reviewer_id |
| find_each        |        +------------------+
+------------------+                 |
                                     | key, row
                                     v
+------------------+        +------------------+
| 4. Finalize      | state  | 3. Accumulate    |
| shape state into | <----- | fold row into    |
| output hash      |        | state            |
+------------------+        | domain math here |
        |                   +------------------+
        |
        v
Hash -> JSON
</syntaxhighlight>

Additional notes:

* <code>source</code> uses <code>includes(...)</code> where needed to avoid N+1 queries.
* <code>accumulate</code> handles scores, deduplication, bucketing, and counting depending on the report type.

''Figure: The four stages every report passes through inside BaseReport#run''

=== Class Hierarchy ===

This diagram shows how concrete report classes relate to <code>BaseReport</code>.

Solid inheritance from <code>BaseReport</code> is represented by indentation.
Composition from <code>ReviewReport</code> to its inner pipelines is represented
under the coordinator.

<syntaxhighlight lang="text">
BaseReport
|
|-- ReviewReport (coordinator)
|     |
|     |-- ReviewersPipeline
|     |-- ScoresPipeline
|     |-- AvgRangesPipeline
|
|-- FeedbackReport
|-- TeammateReviewReport
|-- BookmarkRatingReport
|-- BasicReport
</syntaxhighlight>

Legend:

* Direct child under <code>BaseReport</code> = inherits <code>BaseReport</code>
* Pipelines under <code>ReviewReport</code> = coordinator runs inner pipeline
* The inner pipelines also inherit <code>BaseReport</code>

''Figure: Class hierarchy for the report generation subsystem''

----

== Architecture: The Streaming Pipeline ==

=== Pipeline Shape ===

All reports are built on a single pipeline template defined in
<code>Reports::BaseReport</code>:

<syntaxhighlight lang="ruby">
def run
  state = initial_state
  source.find_each(batch_size: 500) do |row|
    accumulate(state, grouper.call(row), row)
  end
  finalize(state)
end
</syntaxhighlight>

The pipeline consists of four concerns:

{| class="wikitable"
! Concern !! Responsibility
|-
| '''Source'''
| ActiveRecord relation streamed via <code>find_each</code>. Subclasses use <code>includes(...)</code> to eagerly load associations and prevent N+1 queries.
|-
| '''Grouper'''
| A lambda <code>(row) -> key</code> that determines how rows are bucketed in the accumulator state.
|-
| '''Accumulate'''
| Folds one row into the state in place. Contains all domain-specific math for that report.
|-
| '''Finalize'''
| Post-processes the finished state into the output hash. Default implementation returns state unchanged.
|}

=== Memory Model ===

State grows proportional to the number of '''groups''', such as the number of
distinct reviewers, not the number of raw rows.

For example, a dataset with 10,000 responses across 20 reviewers keeps only
20 entries in the accumulator state at any point.

=== Base Class Definition ===

<syntaxhighlight lang="ruby">
module Reports
  class BaseReport
    def initialize(assignment)
      @assignment = assignment
    end

    def run
      state = initial_state
      source.find_each(batch_size: 500) do |row|
        accumulate(state, grouper.call(row), row)
      end
      finalize(state)
    end

    private

    def source        = raise NotImplementedError
    def grouper       = raise NotImplementedError
    def initial_state = raise NotImplementedError

    def accumulate(_state, _key, _row)
      raise NotImplementedError
    end

    def finalize(state) = state
  end
end
</syntaxhighlight>

----

== Controller ==

=== Dispatch Mechanism ===

The controller uses a constant hash to map type strings to report classes,
replacing the <code>send(type)</code> meta-programming pattern used in Repo X.

<syntaxhighlight lang="ruby">
REPORT_CLASSES = {
  'review_response_map'          => Reports::ReviewReport,
  'feedback_response_map'        => Reports::FeedbackReport,
  'teammate_review_response_map' => Reports::TeammateReviewReport,
  'bookmark_rating_response_map' => Reports::BookmarkRatingReport,
  'basic'                        => Reports::BasicReport
}.freeze

def response_report
  type         = params.dig(:report, :type) || params[:type] || 'basic'
  report_class = REPORT_CLASSES[type]

  unless report_class
    return render json: { error: "Unknown report type: #{type}" },
                  status: :unprocessable_entity
  end

  assignment = Assignment.find(params[:assignment_id] || params[:id])
  data = report_class.new(assignment).run
  render json: { type: type, assignment_id: assignment.id }.merge(data)
end
</syntaxhighlight>

=== Route ===

<syntaxhighlight lang="text">
GET  /reports/response_report?assignment_id=<id>&type=<type>
POST /reports/response_report
</syntaxhighlight>

----

== Report Implementations ==

=== Review Report (<code>review_response_map</code>) ===

The review report is the most complex. It is implemented as a coordinator
class (<code>ReviewReport</code>) that runs three independent inner pipelines
and merges their results.

{| class="wikitable"
! Pipeline !! Source !! Groups by !! Produces
|-
| <code>ReviewersPipeline</code>
| <code>ReviewResponseMap</code>
| <code>reviewer_id</code>
| Sorted reviewer list
|-
| <code>ScoresPipeline</code>
| <code>Response</code> JOIN map
| <code>reviewer_id</code>
| Score percentage per round/reviewee
|-
| <code>AvgRangesPipeline</code>
| <code>Response</code> JOIN map
| <code>[reviewee_id, round]</code>
| Max/min/avg per team/round
|}

==== N+1 Fix: Precomputed Max Question Score ====

Repo X called <code>response.maximum_score</code> inside the accumulation loop.

This method internally calls:

<syntaxhighlight lang="ruby">
response_assignment.assignment_questionnaires
  .find_by(used_in_round: round)
  .questionnaire
</syntaxhighlight>

This resulted in one query per response.

In Repo Y, both score pipelines precompute a
<code>round -> max_question_score</code> map with a single query before the
pipeline runs:

<syntaxhighlight lang="ruby">
def precompute_max_q_scores
  AssignmentQuestionnaire
    .joins(:questionnaire)
    .where(assignment_id: @assignment.id)
    .pluck(:used_in_round, 'questionnaires.max_question_score')
    .to_h
end
</syntaxhighlight>

The result, for example <code>{nil => 10, 1 => 10, 2 => 5}</code>, is stored in
<code>@max_q_score</code> and used as a lookup inside <code>accumulate</code>:

<syntaxhighlight lang="ruby">
max_score = total_wt * (@max_q_score[round] || @max_q_score[nil] || 1)
</syntaxhighlight>

==== Sample Response ====

<syntaxhighlight lang="json">
{
  "type": "review_response_map",
  "assignment_id": 1,
  "reviewers": [
    { "id": 5, "user_id": 2, "name": "alice",
      "full_name": "Alice Smith", "handle": "alice" }
  ],
  "review_scores": { "5": { "1": { "3": 87.5 } } },
  "avg_and_ranges": { "3": { "1": { "max": 92.0,
                                    "min": 75.0,
                                    "avg": 83.5 } } }
}
</syntaxhighlight>

----

=== Feedback Report (<code>feedback_response_map</code>) ===

Produces the list of authors and the IDs of review responses that received
author feedback, bucketed by round for varying-rubric assignments.

<syntaxhighlight lang="ruby">
def source
  Response
    .joins(:response_map)
    .where(
      response_maps: { type: 'ReviewResponseMap',
                       reviewed_object_id: @assignment.id }
    )
    .order(created_at: :desc)
end

def grouper       = ->(r) { [r.map_id, r.round] }
def initial_state = { seen: Set.new, round_1: [],
                      round_2: [], round_3: [], all: [] }

def accumulate(state, key, response)
  return if state[:seen].include?(key)
  state[:seen].add(key)
  if @assignment.varying_rubrics_by_round?
    case response.round
    when 1 then state[:round_1] << response.id
    when 2 then state[:round_2] << response.id
    when 3 then state[:round_3] << response.id
    end
  else
    state[:all] << response.id
  end
end
</syntaxhighlight>

Deduplication uses a <code>Set</code>, which gives O(1) lookup, rather than the
array-based <code>seen_map_round_keys.include?</code> from Repo X, which gives
O(n) lookup.

Authors are fetched once in <code>finalize</code>, not inside the stream.

==== End-to-End Execution Flow ====

The <code>run</code> method is '''inherited from <code>BaseReport</code>'''.
<code>FeedbackReport</code> never defines it.

Calling <code>FeedbackReport.new(assignment).run</code> triggers the following
sequence:

<syntaxhighlight lang="text">
ReportsController
  REPORT_CLASSES['feedback_response_map'].new(assignment).run
    |
    |  (inherited from BaseReport)
    v
  BaseReport#run
    |
    |-- Step 1: state = initial_state
    |       => { seen: Set.new,
    |            round_1: [], round_2: [], round_3: [], all: [] }
    |
    |-- Step 2: source.find_each(batch_size: 500)
    |       => Response.joins(:response_map)
    |                  .where(type: 'ReviewResponseMap',
    |                         reviewed_object_id: assignment.id)
    |                  .order(created_at: :desc)
    |          streams Response records newest-first, in batches
    |
    |-- Step 3: for each Response row:
    |       key = grouper.call(row)
    |           => [row.map_id, row.round]   e.g. [42, 1]
    |
    |       accumulate(state, key, row)
    |           => skip if state[:seen] already has [map_id, round]
    |              (keeps only the latest response per map per round
    |               because source is ordered newest-first)
    |           => otherwise: add key to :seen, then bucket row.id:
    |                round == 1  =>  state[:round_1] << row.id
    |                round == 2  =>  state[:round_2] << row.id
    |                round == 3  =>  state[:round_3] << row.id
    |                (or state[:all] if single-rubric assignment)
    |
    |-- Step 4: finalize(state)
            => fetch_authors  (one query: teams -> users -> participants)
            => if varying_rubrics_by_round?
                 return { authors: [...],
                          review_response_ids: {
                            round_1: [...], round_2: [...], round_3: [...] } }
               else
                 return { authors: [...],
                          review_response_ids: [...] }
</syntaxhighlight>

The key point is that <code>FeedbackReport</code> only defines the four pieces
the pipeline needs:

* <code>source</code>
* <code>grouper</code>
* <code>initial_state</code>
* <code>accumulate</code>
* <code>finalize</code>

Ruby's inheritance mechanism means calling <code>.run</code> on a
<code>FeedbackReport</code> instance automatically executes
<code>BaseReport#run</code>, which calls back into <code>FeedbackReport</code>'s
implementations of those methods.

==== Sample Response (varying rubrics) ====

<syntaxhighlight lang="json">
{
  "type": "feedback_response_map",
  "authors": [{ "id": 7, "name": "bob", "full_name": "Bob Jones" }],
  "review_response_ids": {
    "round_1": [12, 15], "round_2": [18], "round_3": []
  }
}
</syntaxhighlight>

----

=== Teammate Review Report (<code>teammate_review_response_map</code>) ===

Streams <code>TeammateReviewResponseMap</code> records grouped by
<code>reviewer_id</code>.

The first occurrence per reviewer is kept using deduplication via early return
if the key already exists in state. Reviewer associations are eagerly loaded.

==== Sample Response ====

<syntaxhighlight lang="json">
{
  "type": "teammate_review_response_map",
  "reviewers": [
    { "reviewer_id": 5, "user_id": 2,
      "name": "alice", "full_name": "Alice Smith" }
  ]
}
</syntaxhighlight>

----

=== Bookmark Rating Report (<code>bookmark_rating_response_map</code>) ===

Streams <code>BookmarkRatingResponseMap</code> records, accumulating distinct
bookmark IDs into a <code>Set</code>.

Project topics are fetched once in <code>finalize</code>.

==== Bug Fixed During Port ====

The model's <code>bookmark_response_report</code> in Repo Y was incorrectly
calling:

<syntaxhighlight lang="ruby">
.pluck(:reviewed_object_id)
</syntaxhighlight>

This returns assignment IDs, since <code>reviewed_object_id</code> is the
foreign key to <code>Assignment</code>.

Bookmark IDs are stored in <code>reviewee_id</code>. Therefore, this was fixed
to:

<syntaxhighlight lang="ruby">
.pluck(:reviewee_id)
</syntaxhighlight>

==== Sample Response ====

<syntaxhighlight lang="json">
{
  "type": "bookmark_rating_response_map",
  "bookmark_ids": [10, 14, 22],
  "topics": [{ "id": 3, "topic_name": "Machine Learning" }]
}
</syntaxhighlight>

----

=== Basic Report (<code>basic</code>) ===

Returns minimal assignment metadata.

No streaming is required since all data comes from the already-loaded
<code>Assignment</code> object.

This report is used as the default when no type parameter is provided.

==== Sample Response ====

<syntaxhighlight lang="json">
{
  "type": "basic",
  "assignment_id": 1,
  "assignment": {
    "id": 1, "name": "Project 1",
    "num_review_rounds": 2,
    "varying_rubrics_by_round": true
  }
}
</syntaxhighlight>

----

== File Structure ==

<syntaxhighlight lang="text">
app/
  controllers/
    reports_controller.rb        Entry point, REPORT_CLASSES dispatch
  helpers/
    report_formatter_helper.rb   Empty namespace (logic moved to services)
  services/
    reports/
      base_report.rb             Abstract pipeline template
      review_report.rb           3-pipeline coordinator
      feedback_report.rb         Single pipeline, round bucketing
      teammate_review_report.rb  Single pipeline
      bookmark_rating_report.rb  Single pipeline
      basic_report.rb            Simple struct
  models/
    review_response_map.rb
      .review_response_report class method
    feedback_response_map.rb
      .feedback_response_report class method
    teammate_review_response_map.rb
      .teammate_response_report class method
    bookmark_rating_response_map.rb
      .bookmark_response_report (bug fixed)
</syntaxhighlight>

----

== Blocked Report Types ==

The following report types exist in Repo X but cannot yet be implemented in
Repo Y due to missing database tables or models.

Each is ready to be added once its dependency is ported.

{| class="wikitable"
! Report Type !! Missing Dependency !! Repo X Location
|-
| <code>calibration</code>
| <code>calibrate_to</code> column on <code>response_maps</code>
| <code>report_formatter_helper.rb</code>
|-
| <code>self_review</code>
| <code>SelfReviewResponseMap</code> model
| <code>self_review_response_map.rb</code>
|-
| <code>survey</code>
| <code>survey_deployments</code> table
| <code>survey_response_map.rb</code>
|-
| <code>quiz</code>
| <code>quiz_responses</code> table
| <code>quiz_response_map.rb</code>
|-
| <code>answer_tagging</code>
| <code>tag_prompt_deployments</code>, <code>answer_tags</code> tables
| <code>tag_prompt_deployment.rb</code>
|}

To add a blocked report once its dependencies are available:

# Create <code>app/services/reports/&lt;name&gt;_report.rb</code> inheriting <code>BaseReport</code>.
# Define <code>source</code>, <code>grouper</code>, <code>initial_state</code>, <code>accumulate</code>, and <code>finalize</code>.
# Add an entry to <code>ReportsController::REPORT_CLASSES</code>.

----

== Comparison with Repo X ==

{| class="wikitable"
! Concern !! Repo X !! Repo Y
|-
| Output format
| ERB instance variables, such as <code>@reviewers</code> and <code>@review_scores</code>
| JSON hash from <code>report.run</code>
|-
| Loading strategy
| All records loaded into arrays at once
| <code>find_each</code> batched streaming
|-
| Metrics location
| <code>compute_metrics</code> in helper base
| Each report owns <code>accumulate</code> and <code>finalize</code>
|-
| Dispatch
| <code>send(@type.underscore, params, session)</code>
| <code>REPORT_CLASSES[type].new(assignment).run</code>
|-
| N+1 on scores
| <code>response.maximum_score</code> per row — questionnaire lookup each time
| Precomputed <code>round->max_score</code> map, one query before pipeline
|-
| Deduplication
| <code>Array#include?</code> — O(n) per check
| <code>Set#include?</code> — O(1) per check
|-
| Additional features
| LLM evaluation, CSV export, calibration, self-review, survey, quiz, answer tagging
| Not yet ported; blocked on schema
|}

----
== Author ==

{| class="wikitable"
! Name !! Role
|-
| '''Aanand Sreekumaran Nair Jayakumari'''
| Project contributor / developer
|}