CSC/ECE 517 Fall 2021 - E2161. Merge code for role based reviewing with code for topic specific rubrics

This wiki page contains description of changes designed and implemented for E2161. Merge code for role-based reviewing with code for topic-specific rubrics, a Final Project for CSC/ECE 517, Fall 2021.

Purpose

In CSC/ECE 517, there are Expertiza-based course projects, Mozilla-based course projects, etc. However, currently, we can only specify one kind of rubric for all kinds of course projects. This means that refactoring projects, testing projects, and Mozilla projects need to use the same rubric. We hope we could specify different rubrics to be used with different kinds of course projects.

This project was implemented by another team in Spring 2020. We will be implementing our project on top of what was implemented by the previous team and fix the issues that were found in their implementation. Information on what the previous team had worked on is described in the Previous Implementation section.

Previous Implementation

All aspects of previous implementation were good, and their design was well appreciated. Their changes were also merged to expertiza:beta branch, but were later reverted since the specialized rubrics weren’t saved in the database. This github issue provides a detailed explanation to the problem.

For us to get started, we were provided with following links from previous implementation:

• http://wiki.expertiza.ncsu.edu/index.php/E1936_Specialized_Rubrics
• https://github.com/gabalmat/expertiza/tree/beta
• https://github.com/expertiza/expertiza/pull/1444
• https://www.youtube.com/watch?v=F7nQsIUspQM

Design Strategy

This feature was previously implemented and was detailed here. The feature concluded with allowing 4 rubric scenarios for an assignment:

• Rubric does not vary by round or by topic.
• Rubric varies by round, but not by topic.
• Rubric varies by topic, but not by round.
• Rubric varies by both round and topic.

However, there were two issues with the implementation:

• Rubrics in the dropdown were only those created by the logged-in instructor, so the TA wouldn’t be able to see them
• Rubrics would not be saved after selecting them and saving.

In light of these issues, changes to expertiza were made as follows:

• Two additional columns are added into the Assignment table that determines whether Rubrics varies by either Round or Topic with default values False
• update_assignment_questionnaires method is re-implemented
  • Having extra column in the assignment questionnaire table topic_id, no need for deleting all the data and re-writing it again every single time in the DB (this caused the previous implementation to have a delay when selecting **rubrics)
  • The only varying value is questionnaire_id, the rest values may not change from Topics or Rubrics tabs, but can be added
• There are 4 (four) possible cases for saving and updating data:
  • used_in_round = null and topic_id = null
  • used_in_round = integer and topic_id = null
  • used_in_round = null and topic_id = integer
  • used_in_round = integer and topic_id = integer

This solved the issue of having rubrics save

Problem

The feature we have to implement was not fully committed due to the previously mentioned problems, so we have to reincorporate the missing code. However, a resulting issue is the inability for an instructor or TA to use rubrics that they did not create. The current problem can be broken down into the following parts:

1. Integrate the changes made from the original implementation into the current version of Expertiza
2. Allow an instructor/TA to chose rubrics that aren’t only theirs
3. Create and update tests to reflect the changes as needed

Proposed Solution

Our proposed solution keeps the previous implementation, but changes rubric filtering to allow instructors/TAs to use filters that are not theirs. Alongside the changes in the original implementation we propose:

• Allow an instructor to choose different rubrics for different topics
  • PROBLEM: The drop-downs for selecting rubrics show only those rubrics created by the currently-logged in instructor (per project mentor).
    • SOLUTION: Change the filtering in the questionnaire_options method to reflect the desired filtering.
    • FILE: app/helpers/assignment_helper.rb

UML Diagram

Following the previous implementations's footsteps, this diagram depicts the interactions between an instructor and an assignment. The instructor may edit, delete, copy, and other stuff (already existing in Expertiza). Alongside editing topics and due dates, the instructor can edit what rubrics are assigned to an assignment. The highlighted portions are of interest. The topics tab allows instructors to specify which rubric associates with each topic while the rubrics tab lets the instructor determine if the assignment will vary by topic or not.

Files to be modified

Since our project is to improve upon a previous implementation that was slightly flawed, we will be modifying all of the same files that they previously modified (even if we don’t choose to alter their implementation in that file).

The major modified files from the previous implementation include:

• Controllers
  • assignments_controller.rb: To refresh the topics list when changing tabs
  • popup_controller.rb: To add a potential error message to the rubric view scores popup

• Models
  • assignment.rb: Add methods to determine if an assignment varies by rubric/topic. (will be refactored as part of our change, however)
  • assignment_form.rb: Add topic ids to created assignment questionnaires
  • assignment_questionnaire.rb: Add topic id to assignment questionnaire model
  • review_response_map.rb: To allow finding review questionnaires by topic id
  • sign_up_topic.rb: To allow a topic to have many assignment questionnaires attach to it (via topic id)

• Views
  • assignments/edit.html.erb: To move topic editing view to its own file that is rendered as part of assignment edit
  • edit/_rubrics.html.erb: Factor out common code into a function, update to use topic id
  • edit/_topics.html.erb: Topic editing view that was moved from assignment edit view
  • popup/view_review_scores_popup.html.erb: Assignments that vary by topic should not be displayed, instead getting error
  • sign_up_sheet/_table_line.html.erb: Add questionnaires to signup sheet table if assignment varies by topic

• Helpers
  • assignment_helper.rb: To add a topic id to the searchable fields for a questionnaire

• DB Migrate
  • XXXXXXXXXXX_add_topic_id_to_assignment_questionnaires.rb: Migration to add topic id to assignment questionnaire schema

• All of the related test files to accommodate the above changes

To address the issues brought up with the previous implementation, we will also make the following major modifications:

• Controllers
  • assignments_controller.rb: Refactor to use the persisted assignment fields for varying by topic/round instead of using methods
  • grades_controller.rb: Refactor to use the persisted assignment fields for varying by topic/round instead of using methods
  • popup_controller.rb: Refactor to use the persisted assignment fields for varying by topic/round instead of using methods

• Models
  • assignment.rb: Add persisted fields to the assignment for varying by topic/round instead of using methods to determine it
  • assignment_form.rb: No longer delete all existing questionnaires on update, update them instead. Now find questionnaire by assignment questionnaire and type rather than assignment/type/round_number/topic_id.
  • assignment_participant.rb: Refactor to use the persisted assignment fields for varying by topic/round instead of using methods
  • feedback_response_map.rb: Refactor to use the persisted assignment fields for varying by topic/round instead of using methods
  • on_the_fly_calc.rb: Refactor to use the persisted assignment fields for varying by topic/round instead of using methods
  • self_review_response_map.rb: Refactor to use the persisted assignment fields for varying by topic/round instead of using methods
  • tag_prompt_deployment.rb: Refactor to use the persisted assignment fields for varying by topic/round instead of using methods

• Views
  • edit/_rubrics.html.erb: Modify to set assignment vary by round/topic fields instead of non persisted flags

• Helpers
  • assignment_helper.rb: Refactor by moving function to find questionnaire / assignment questionnaire to assignment_form.rb. Remove filters that only allow instructors to see rubrics.
  • grades_helper.rb: Refactor to use the persisted assignment fields for varying by topic/round instead of using methods
  • summary_helper.rb: Refactor to use the persisted assignment fields for varying by topic/round instead of using methods

• DB Migrate
  • XXXXXXXXX_add_vary_by_topic_to_assignments.rb: Migration to add “vary by topic” field to assignment
  • XXXXXXXXX_add_vary_by_round_to_assignments.rb: Migration to add “vary by round” field to assignment

• All of the related test files to accommodate the above changes

Database Flow

We will be re-adding the database flow that was added in the previous implementation, linking sign_up_topic to assignment_questionnaire via a topic_id field.

In addition to that, we will be adding two additional boolean fields to the assignment schema: vary_by_round and vary_by_topic. As discussed earlier, in the previous implementation, these were methods that were called to determine if an assignment varied by round/topic rather than a persisted value.

Only a subset of the fields for each table is shown in the diagram because most of the fields are not relevant to these changes and would only serve to distract from the relevant changes. Additions are shown in bold.

New Implementation

While trying to integrate the previous team's implementation of this feature, we discovered that their method for querying AssignmentQuestionnaires (AQs) was flawed.

It would always query for the AQs by assignment id, current round number, and current topic id.

However, if the assignment did not have reviews that vary by round or topic, these values would be nil on the AQ, and the previous query would fail.

To fix this, we modified the assignment_questionnaire function in the AssignmentForm model to check the Assignment's vary_by_round and vary_by_topic flags, and then add the corresponding fields to the query.

This means that:

• If an assignment does not vary by round or by topic, the query for AQs will be by assignment id only.
• If the assignment varies by round but not topic, the query for AQs will be by assignment id and round number.

Testing Plan

As part of our implementation, we modified existing code as well as added new code. To ensure that existing functionality was not broken, and new functionality worked as expected, we used the following Test Strategy (which was also used by previous team):

Run and pass existing RSpec Tests

• The following existing RSpec test files have been modified and they pass as part of testing:
  • spec/controllers/assignments_controller_spec.rb
  • spec/controllers/questionnaires_controller_spec.rb
  • spec/controllers/response_controller_spec.rb
  • spec/factories/factories.rb
  • spec/features/assignment_creation_spec.rb
  • spec/features/quiz_spec.rb
  • spec/features/staggered_deadline_spec.rb
  • spec/models/assignment_form_spec.rb
  • spec/models/assignment_spec.rb
  • spec/models/on_the_fly_calc_spec.rb
  • spec/models/response_spec.rb
  • spec/models/review_response_map_spec.rb

Develop New RSpec Tests

• As part of previous implementation, the team had introduced the following new RSpec test files. We have retained those file.
  • spec/helpers/assignment_helper_spec.rb
  • spec/models/self_review_response_map_spec.rb
• All these rspec tests passed.

Manual Testing in Expertiza UI

Here we describe manual UI Testing steps to edit an existing assignment to allow it have to specialized rubrics for different topic types. These steps are also shown in recorded demo video.

• Login to Expertiza using instructor account (For testing, username: instructor6, password: password)
• Click on Manage > Assignments
• Click on Edit option for any assignment, you should get following view. Make sure Has topics? box is checked.

• Click on Rubrics tab. You will see 2 checkboxes (Review rubric varies by round?, Review rubric varies by topic?)
• Check the box for Review rubric varies by topic?
• Go to Topics tab and verify that there is dropdown menu beside each Topic.
• Select a rubric from dropdown menu, and click Save

• Go back to Home, and select the same assignment to edit. When you click on Topics tab, you should see the rubric you had selected.

Coverage

All rspec tests part of Travis CI build has passed. The code coverage has increased by 10.9% to 41.425%.

Future Work

After merging the previous team's work with expertiza, several RSpec tests failed.

• This was due to how questionnaires were retrieved/found for an assignment. Staggered deadlines also rely on finding questionnaires using topic id, which isn't handled until after an assignment is made.
• Resolved

However, undoing filtering results in even more errors across models, controllers, and forms. This feature, however is a quality of life improvement and is less of a priority.

• Unresolved

Another quality of life improvement is that staggered deadlines and topics cannot be added to an assignment until after the assignment has been created and saved.

• This issue exists on expertiza's beta branch itself, separate from our changes.
• Unresolved

Useful Links

• Github: https://github.com/abhishekupadhyaya/expertiza
• Pull Request: https://github.com/expertiza/expertiza/pull/1720
• Demo Recording: https://www.youtube.com/watch?v=3Ns3DYGtJdQ

Team Information

• Naman Shrimali
• Kanchan Rawat
• Subodh Thota
• Andrew Shipman

Mentor : Prof. Edward F. Gehringer