CSC/ECE 517 Fall 2020 - E2083. Revision planning tool E2016

This page provides a description of the Expertiza based OSS project.

Introduction

Rounds of peer reviews may be implemented between submissions for assignments on Expertiza. In order to better track the implementation of reviewer's suggestions, a Revision Planning Tool should be implemented.

Problem Statement

In the first round of Expertiza reviews, we ask reviewers to give authors some guidance on how to improve their work. Then in the second round, reviewers rate how well authors have followed their suggestions. We could carry the interaction one step further if we asked authors to make up a revision plan based on the first-round reviews. That is, authors would say what they were planning to do to improve their work. Then second-round reviewers would assess how well they did it. In essence, this means that authors would be adding criteria to the second-round rubric that applied only to their submission. We are interested in having this implemented and used in a class so that we can study its effect.

Previous Implementations

Revision planning has been implemented twice before, once in E1875 and once in E2016. While the functionality worked and effectively minimized changes to the code, they also had the following problems:

• Hardcoded “round” numbers in many places of the code.
• Documentation does not reflect the new changes they made.
• Revision planning responses and responses to the other items are not distinguished in heatgrid view.
• The idea of adding a team_id field to each question is intuitive. However, they failed to come up with a clean implementation of this idea. Specifically, they had passed some trailing parameters several methods down before reaching the place that needs them.

Rationale

To implement revision planning, E2016 added EnableRevisionPlan (boolean) to Assignment and TeamID to Question. Adding a boolean to enable revision planning for an assignment makes sense, but having certain questions belong to a team is not a clean implementation. Instead, we will create a new questionnaire that will belong to each team and keep track of the set of questions. We will use this new questionnaire to separate the revision plan responses and responses to other items in heatgrid view by creating a different table for each.

Files Modified

This is a list of files requiring modification and will be updated as the implementation progresses.

Requiring Change:
controllers/questionnaires_controller.rb
controllers/response_controller.rb
controllers/grades_controller.rb
models/response.rb
models/assignment_team.rb
views/questionnaires/_questioinnaire.html.erb
views/student_task/view.html.erb
views/assignments/edit/_review_strategy.html.erb
views/response/view.html.erb
views/response/response.html.erb
views/grades/_review.html.erb

To be Added:
controllers/revision_plan_questionnaire.rb
models/revision_plan_questionnaire.rb
views/questionnaires/edit_revision_plan.html.erb

Design

Database Design

Items in green are additions.

In the assignment table we will add is_revision_enabled? column to indicate whether the assignment accepts a revision plan along with review rubric.

RevisionPlanQuestionnaire maps a questionnaire to an assignment team and round. This will map to a questionnaire that will be created by the reviewee.

Proposed Changes

1. A team would be allowed to create a RevisionPlanQuestionnaire after completion of round 1 of review. This revision plan questionnaire would be linked to the next round and to the team. Team would be able to add questions to this revision plan questionnaire.
2. Reviewer would be displayed questions of both the original assignment review rubric and reviewee created revision plan. To achieve this we will create a CompositeQuestionnaire by combining the two questionnaires. This composite questionnaire would not have any revision plan or review rubric based code.
   • Composite questionnaire will be created in the following:
   1. When revision plan is enabled and a questionnaire is requested from ReviewResponseMap.
   2. When revision plan is enabled and questionnaire is requested from response.
   • Composite questionnaire would have a function questions to return questions of contained questionnaires.
3. The Response currently displays section headings using SectionHeader. SectionHeader is a child class of QuestionnaireHeader which itself derives from Question.
   A new QuestionnaireNameHeader class which derives from QuestionnaireHeader will be created to display name of Questionnaire (Improvement Plan/Assignment) in a way which is similar to how we show section headings in UI.

User Interface

Enable Revision Planning

In order to enable Revision Planning, the setting must be enabled when creating or editing an assignment under the General tab. The wireframe below demonstrates creating an assignment, and editing the assignment functions similarly. Improvement plan? should be checked to enable this option.

Wireframe of Enabling Assignment's Revision Planning

Assignment Overview Page (contains the Link to the revision planning page)

The Revision Planning link is available to students during every submission period (except the first round submission) and not available during every review period. As shown in the wireframe, by clicking Revision Planning students would be redirected to a page explained under the ‘Revision planning page’ subsection.

Wireframe of Assignment Overview Page

Editing the Revision Plan Questionnaire

After creating the Revision Plan Questionnaire, it must be edited. Questions can be added by specifying the amount of questions and their type. Questions can be removed by clicking Remove in the leftmost column. Once the questionnaire is complete, it can be saved. This page will be visible during each submission period after the first and will be unavailable during all review periods.

Wireframe of Editing an Assignment's Revision Plan

Reviewing an Assignment

Once the Revision Plan Questionnaire and its resubmission round have been finished, the Revision Plan Questionnaire will be appended to the Assignment Questionnaire for that around.

Wireframe of Completing a Review for an Assignment

Summary Report Page

When a project has been reviewed at least once, a participant will be able to view their team's score. The wireframe below shows what this will look like after the second round of reviews. For the second and all subsequent reviews, the results of questions that were created by the instructor will be shown under Assignment Questionnaire. The results of the questions created by the team will be shown under Improvement Plan.

Wireframe of Summary Report for an Assignment

Control Flow Diagram

Control Flow of the Revision Planning Function

Test Plan

RSpec Testing

The RSpec tests would test both controller and models.

Controllers

• spec/controllers/questionnaires_controller_spec.rb: test revision plan questionnaire can be created and edited.
• spec/controllers/assignments_controller_spec.rb: create and validate an assignment with revision planning enabled.

Models Test validations and logic present in revision plan and composite questionnaire models.

• spec/models/revision_plan_questionnaire_spec.rb
• spec/models/composite_questionnaire_spec.rb

Manual Testing

Manual testing will aim to verify the following:

• Create an assignment with revision planning enabled.
• Are participants allowed to create/edit revision plan when round 1+ (1 or greater than 1) reviews have finished.
• Is revision plan editing disabled when assignment is in review stage.
• Are reviewers shown questions created by reviewees.
• Are participants shown summary of score for revision plan after review deadline has expired.

Team Members

Chaitanya Mehta (cmehta)
Darby Madewell (demadewe)
Dongni Yang (dyang23)
Sidharth Mehta (smehta22)
Mentor: Yulin Zhang (yzhan114)

References

1. Previous Implementation: E1875 wiki, E2016 wiki
2. Forked Repository: E2083 github
3. Pull Request: E2083 pull request