CSC/ECE 517 Fall 2021 - E2152. Revision planning tool

This page provides a description of the Expertiza based OSS project.

Introduction

Rounds of peer reviews may be implemented between submissions for assignments on Expertiza. In order to better track the implementation of reviewer's suggestions, a Revision Planning Tool should be implemented.

Problem Statement

In the first round of Expertiza reviews, we ask reviewers to give authors some guidance on how to improve their work. Then in the second round, reviewers rate how well authors have followed their suggestions. We could carry the interaction one step further if we asked authors to make up a revision plan based on the first-round reviews. That is, authors would say what they were planning to do to improve their work. Then the second round reviewers would assess how well they did it. In essence, this means that authors would be adding criteria to the second round rubric that applied only to their submission. We are interested in having this implemented and used in a class so that we can study its effect.

Previous Implementations

In the first round of Expertiza reviews, we ask reviewers to give authors some guidance on how to improve their work. Then in the second round, reviewers rate how well authors have followed their suggestions. We could carry the interaction one step further if we asked authors to make up a revision plan based on the first-round reviews. That is, authors would say what they were planning to do to improve their work. Then second-round reviewers would assess how well they did it. In essence, this means that authors would be adding criteria to the second-round rubric that applied only to their submission. We are interested in having this implemented and used in a class so that we can study its effect.

Revision planning has been implemented three times before, once in E1875, once in E2016 and another one in E2083. While the functionality worked and effectively minimized changes to the code, they also had the following problems:

• Hardcoded “round” numbers in many places of the code.
• Documentation does not reflect the new changes they made.
• Revision planning responses and responses to the other items are not distinguished in heatgrid view.
• The idea of adding a team_id field to each question is intuitive. However, they failed to come up with a clean implementation of this idea. Specifically, they had passed some trailing parameters several methods down before reaching the place that needs them.

Problems with Previous Implementation

Students cannot edit the revision plan in the review phase so the review doesn't get changed while other students are reviewing. However, it assumes that reviews cannot be done during the submission phase. Code needs to be written to take care of this part. Set up an assignment with 2 rounds of review. Allow editing the revision-planning rubric just like editing a normal rubric, using a shared template .Instructors and students can view the review report with the revision plan placed in a separate section. In RevisionPlanQuestionnairesController, the method for determining who's on a team would be better located in team.rb. The current_round method duplicates a method elsewhere in the system. The generate_heatgrid has too many conditional statements. It would be better to split it into smaller methods. A lot of code deals with score calculations, which shouldn't be a concern for this project. Too many files are involved, although they seem to make reasonable decisions about their changes. The code should have more comments. The team had a good initial design but took as twice much as LoC compared to the previous teams.

Goals

• Let the authors be able to edit the revision plan during the review phase.
• Set up an assignment with 2 rounds of review.
• Allow editing the revision-planning rubric just like editing a normal rubric, using a shared template.
• Instructors and students can view the review report with the revision plan placed in a separate section.
• Relocate the method for determining who's on a team in RevisionPlanQuestionnairesController to team.rb.
• Delete the duplicated current_round method.
• Split the conditional statements of generate_heatgrid into smaller subclass methods.

Proposed Solution

Implementation

• 1. Enable Revision Planning during the review phase.

Let the author make a choice to decide whether to enable revision planning during the review phase. In order to enable Revision Planning, the setting must be enabled when creating or editing an assignment under the General tab. The wireframe below demonstrates creating an assignment, and editing the assignment functions similarly. Revision plan? should be checked to enable this option.

• 2. Set up an assignment with 2 rounds of review.

• 3. Allow editing the revision-planning rubric just like editing a normal rubric, using a shared template.

• 4. Instructors and students can view the review report with the revision plan placed in a separate section.

• 5. Relocate the method for determining who's on a team in RevisionPlanQuestionnairesController to team.rb.

• 6. Delete the duplicated current_round method.

• 7. Split the conditional statements of generate_heatgrid into smaller subclass methods.

Rationale

The general workflow is shown bellow.

Design

Database Design

Items in green are additions.

Changes:

• In the assignment table we have added is_revision_planning_enabled? column to indicate whether the assignment accepts a revision plan along with review rubric.
• RevisionPlanTeamMap maps a questionnaire to an assignment team and round. This will map to a questionnaire of type revision plan that will be created by the revewee.
• RevisionPlanQuestionnaire extends Questionnaire using single table inheritance.

Test Plan

• There should be 2 rounds of reviews. And reviews should only be available at the review phase, not at the submission phase. Also, students should not be able to edit the revision plan during the review phase.
• After implementation, only a reasonable number of the most important and directly related files have been modified.
• There will be concise comments for new code to indicate the purpose.
• Compared to the previous teams, there will only be a reasonable amount of new code.
• The revision-planning rubric can be edited.
• The revision plan will be placed in a separate section.
• The method for determining who's on a team would be located in team.rb.
• Delete the redundant functions and test the system's integrity.
• Break down the long functions into small functions and reduce conditional cases, then test the system's integrity.
• Reduce the score calculation and test the system's integrity.

Team Members

Kai Gao (kgao2@ncsu.edu)
Huangxing Chen (hchen63)
Yi Li (yli273)
Shengjie Guo (sguo25)
Mentor: Nicholas Himes (nnhimes@ncsu.edu)