CSC/ECE 517 Fall 2020 - E2083. Revision planning tool E2016

This page provides a description of the Expertiza based OSS project.

Introduction

Rounds of peer reviews may be implemented between submissions for assignments on Expertiza. In order to better track the implementation of reviewer's suggestions, a Revision Planning Tool should be implemented.

Problem Statement

In the first round of Expertiza reviews, we ask reviewers to give authors some guidance on how to improve their work. Then in the second round, reviewers rate how well authors have followed their suggestions. We could carry the interaction one step further if we asked authors to make up a revision plan based on the first-round reviews. That is, authors would say what they were planning to do to improve their work. Then second-round reviewers would assess how well they did it. In essence, this means that authors would be adding criteria to the second-round rubric that applied only to their submission. We are interested in having this implemented and used in a class so that we can study its effect.

Previous Implementations

Revision planning has been implemented twice before, once in E1875 and once in E2016. While the functionality worked and effectively minimized changes to the code, they also had the following problems:

• Hardcoded “round” numbers in many places of the code.
• Documentation does not reflect the new changes they made.
• Revision planning responses and responses to the other items are not distinguished in heatgrid view.
• The idea of adding a team_id field to each question is intuitive. However, they failed to come up with a clean implementation of this idea. Specifically, they had passed some trailing parameters several methods down before reaching the place that needs them.

Rationale

To implement revision planning, E2016 added is_revision_planning_enabled (Boolean) to Assignment and team_id to Question. Adding a Boolean to enable revision planning for an assignment makes sense, but having certain questions belong to a team is not a clean implementation. Instead, we will create a new questionnaire that will belong to each team and keep track of the set of questions. We will use this new questionnaire to separate the revision plan responses and responses to other items in heat-grid view by creating a different table for each.

Files Modified - Under progress

This is a list of files modified and added. The list will be updated as the implementation progresses.
Changed:

• app/controllers/advice_controller.rb
• app/controllers/grades_controller.rb
• app/controllers/questionnaires_controller.rb
• app/controllers/questions_controller.rb
• app/controllers/response_controller.rb
• app/controllers/student_task_controller.rb
• app/helpers/summary_helper.rb
• app/models/questionnaire.rb
• app/models/response.rb
• app/models/student_task.rb
• app/models/vm_question_response.rb
• app/views/advice/edit_advice.html.erb
• app/views/assignments/edit/_general.html.erb
• app/views/grades/view_team.html.erb
• app/views/questionnaires/_questionnaire.html.erb
• app/views/response/response.html.erb
• app/views/student_task/view.html.erb
• config/routes.rb
• db/schema.rb
• spec/controllers/response_controller_spec.rb
• spec/models/response_spec.rb
• spec/models/review_response_map_spec.rb

Added:

• app/controllers/revision_plan_questionnaires_controller.rb
• app/models/revision_plan_questionnaire.rb
• app/models/revision_plan_team_map.rb
• app/views/questionnaires/_questions.html.erb
• app/views/revision_plan_questionnaires/edit.html.erb
• db/migrate/20201106020642_create_revision_plan_team_maps.rb /
• db/migrate/20201105023711_add_is_revison_planning_enabled_to_assignments.rb

Design

Database Design

Items in green are additions.

Changes:

• In the assignment table we have added is_revision_planning_enabled? column to indicate whether the assignment accepts a revision plan along with review rubric.
• RevisionPlanTeamMap maps a questionnaire to an assignment team and round. This will map to a questionnaire of type revision plan that will be created by the revewee.
• RevisionPlanQuestionnaire extends Questionnaire using single table inheritance.

User Interface

Enable Revision Planning

In order to enable Revision Planning, the setting must be enabled when creating or editing an assignment under the General tab. The wireframe below demonstrates creating an assignment, and editing the assignment functions similarly. Revision plan? should be checked to enable this option.

Initial Wireframe

Wireframe of Enabling Assignment's Revision Planning

Final UI

Create New Assignment page in the UI, with added Revision Planning checkbox

Implementation

• Added a Boolean field is_revision_planning_enabled to Assignment that saves whether revision planning is enabled.

db/migrate/20201105023711_add_is_revison_planning_enabled_to_assignments.rb

   class AddIsRevisonPlanningEnabledToAssignments < ActiveRecord::Migration
     def change
       add_column :assignments, :is_revision_planning_enabled, :boolean, default: false
     end
   end

• Add checkbox to enable revision planning in edit assignment view.

app/views/assignments/edit/_general.html.erb

     <input name="assignment_form[assignment][is_revision_planning_enabled]" type="hidden" value="false"/>
     <%= check_box_tag('assignment_form[assignment][is_revision_planning_enabled]', 'true', @assignment_form.assignment.is_revision_planning_enabled) %>
     <%= label_tag('assignment_form[assignment][is_revision_planning_enabled]', 'Revision Planning?') %>

Assignment Overview Page (contains the Link to the revision planning page)

The Revision Planning link is available to students during every submission period (except the first round submission) and not available during every review period. As shown in the wireframe, by clicking Revision Planning students would be redirected to the ‘Revision planning page’.

Initial Wireframe

Wireframe of Assignment Overview Page

Final UI

Submit or Review Work page demonstrating added Revision Planning link

Implementation

• Show link to revision planning page when revision planning is enabled for an assignment, round 1 has passed and current stage is submission.

In order to check if a team is allowed to submit a revision plan @can_submit_revision_plan field is added to StudentTaskController's view action. If a revision plan questionnaire (@revision_plan_questionnaire_id) exists for the current round then clicking the link performs RevisionPlanQuestionnairesController's edit action else new action is performed.

app/views/student_task/view.html.erb

 <% if @team && @assignment.is_revision_planning_enabled %>
   <li>
     <% if @can_submit_revision_plan %>
       <% if @revision_plan_questionnaire_id %>
         <%= link_to "Revision Planning", controller: 'revision_plan_questionnaires', action: 'edit', id: @revision_plan_questionnaire_id %> 
       <% else %>
         <%= link_to "Revision Planning", controller: 'revision_plan_questionnaires', action: 'new', team_id: @team %>
       <% end %>
       (Add new questions to next-round rubric)
     <% else %>
       <font color="gray">Revision Planning</font> (You are not allowed to submit your revison plan right now)
     <% end %>
   </li>
 <%end%>

app/controllers/student_task_controller.rb

 def view
   student_task = StudentTask.from_participant_id params[:id]
   ...
   ...
   # Revision plan feature
   @can_submit_revision_plan = student_task.can_submit_revision_plan?
   @revision_plan_questionnaire_id = student_task.revision_plan_questionnaire_id
 end

app/models/student_task.rb

 def current_round
   assignment.number_of_current_round(@topic.try(:id))
 end
 def can_submit_revision_plan?
   current_stage == 'submission' && current_round > 1
 end
 def revision_plan_questionnaire_id
   RevisionPlanTeamMap.find_by(team: @participant.team, used_in_round: current_round).try(:questionnaire_id)
 end

Editing the Revision Plan Questionnaire

After creating the Revision Plan Questionnaire, it must be edited. Questions can be added by specifying the amount of questions and their type. Questions can be removed by clicking Remove in the leftmost column. Once the questionnaire is complete, it can be saved. This page will be visible during each submission period after the first and will be unavailable during all review periods.

Initial Wireframe

Wireframe of Editing an Assignment's Revision Plan

Final UI

Edit Revision Plan page demonstrating added view

Implementation

• Create a model RevisionPlanTeamMap that maps a revision plan to a team and stores information of the round it is used in.

/app/models/revision_plan_team_map.rb

 class RevisionPlanTeamMap < ActiveRecord::Base
   belongs_to :team
   belongs_to :questionnaire
 end

/db/migrate/20201106020642_create_revision_plan_team_maps.rb

 class CreateRevisionPlanTeamMaps < ActiveRecord::Migration
   def change
     create_table :revision_plan_team_maps do |t|
       t.references :team, index: true, foreign_key: true
       t.references :questionnaire, index: true, foreign_key: true
       t.integer :used_in_round
       t.timestamps null: false
     end
   end
 end

• Create a Revision Plan Questionnaire Model and Controller

app/models/revision_plan_questionnaire.rb

Added a model for revision plan questionnaire. The code provides method to get questionnaire for a current round based on team.

 def self.get_questionnaire_for_current_round(team_id)
   assignment_team = Team.find(team_id)
   assignment = assignment_team.assignment
   current_round = assignment.number_of_current_round(assignment_team.topic)
   questionnaire = RevisionPlanTeamMap.find_by(team: assignment_team, used_in_round: current_round).try(:questionnaire)
   unless questionnaire
     questionnaire = RevisionPlanQuestionnaire.new
     questionnaire.name = 'Revision Plan Questionnaire'
     questionnaire.instructor_id = assignment.instructor_id
     questionnaire.max_question_score = 5
     questionnaire.save
     questionnaire_team_map = RevisionPlanTeamMap.create(team_id: assignment_team.id, used_in_round: current_round, questionnaire_id: questionnaire.id)
   end
   return questionnaire
 end

This method returns the team associated to a revision plan questionnaire.

 def team
   revision_plan_team_map.team
 end

app/models/questionnaire.rb

Added a new type of questionnaire 'RevisionPlanQuestionnaire' to QUESTIONNAIRE_TYPES

 QUESTIONNAIRE_TYPES = ['ReviewQuestionnaire',
                       ...
                       ...
                        'RevisionPlanQuestionnaire'].freeze

app/controllers/revision_plan_questionnaires_controller.rb

Added a controller RevisionPlanQuestionnairesController. The controller derives from QuestionnairesController, it overrides the new action to allow creation of a RevisionPlanQuestionnaire for a team. It also updates permissions for controller.

 def action_allowed?
   case params[:action]
   when 'edit'
     questionnaire = Questionnaire.find(params[:id])
     questionnaire.team.users.collect { |u| u.id }.include? session[:user].id || super
     (user_logged_in? && questionnaire.team.users.collect { |u| u.id }.include?(session[:user].id)) || super
   else
     super
   end
 end

 def new
   begin
     questionnaire = RevisionPlanQuestionnaire.get_questionnaire_for_current_round(params[:team_id])
     redirect_to action: 'edit', id: questionnaire.id
   rescue StandardError
     flash[:error] = $ERROR_INFO
   end
 end

config/routes.rb

Add routes for RevisionPlanQuestionnairesController.

 resources :revision_plan_questionnaires, controller: :revision_plan_questionnaires, only: %i[new edit update] do
   collection do
     post :add_new_questions
     post :save_all_questions
   end
 end

• Add a view to edit Revision Plan

app/views/revision_plan_questionnaires/edit.html.erb

 <%= render :partial => 'question_weight' %>
 <h1>Edit <%= @questionnaire.display_type %></h1>
 <%= render :partial => 'questions' %>
 <br />
 <a href="javascript:window.history.back()">Back</a>

• Refactor code to display questions into a partial. Allows for code reuse.

app/controllers/questionnaires_controller.rb

Updated redirect_to links for add_new_questions and save_all_questions methods.

 redirect_to action: 'edit', id: questionnaire_id.to_sym

app/views/questionnaires/_questionnaire.html.erb

This file generates a view to edit/create a new questionnaire. It has functionality to display questions and some functions that are not needed for revision plan questionnaire. To enable code reuse the functionality to add/edit/remove/save questions was moved to a new partial app/views/questionnaires/_questions.html.erb.

app/views/questionnaires/_questions.html.erb

• Add code to allow users to delete questions from revision plan questionnaire.

app/controllers/questions_controller.rb

Update authorization code to allow a user to remove questions from his teams revision plan.

 def action_allowed?
   if ['destroy'].include?(params[:action])
     question = Question.find(params[:id])
     if(user_logged_in? && question.questionnaire.owner?(session[:user].id))
       return true
     end
   end
   current_user_has_ta_privileges?
 end

 def destroy
   ...
   ...
   if(question.questionnaire.type == 'RevisionPlanQuestionnaire')
     redirect_to edit_revision_plan_questionnaire_path(questionnaire_id.to_s.to_sym)
   else
     redirect_to edit_questionnaire_path(questionnaire_id.to_s.to_sym)
   end
 end

app/models/questionnaire.rb

Method returns true if user owns questionnaire.

 def owner?(user_id)
   instructor_id == user_id
 end

app/models/revision_plan_questionnaire.rb

Overrides implementation for owner? defined in Questionnaire to return true if a team owns the questionnaire.

 def owner?(user_id)
   team.users.map{ |u| u.id }.include?(user_id) || super
 end

• Add code to allow users to access show/edit advice functionality for revision plan questions.

app/controllers/advice_controller.rb

Update authorization code to allow a user to update advice his teams revision plan.

 def action_allowed?
   questionnaire = Questionnaire.find(params[:id])
   if(user_logged_in? && questionnaire.owner?(session[:user].id))
     return true
   end
   current_user_has_ta_privileges?
 end

app/views/advice/edit_advice.html.erb

Update 'Back to questionnaire' link.

 <% if @questionnaire.type == 'RevisionPlanQuestionnaire'%>
   <%= link_to 'Back to questionnaire', :controller => 'revision_plan_questionnaires', :action => 'edit', :id => @questionnaire.id %>
 <% else %>
   <%= link_to 'Back to questionnaire', :controller => 'questionnaires', :action => 'edit', :id => @questionnaire.id %>
 <% end %>

Reviewing an Assignment

Once the Revision Plan Questionnaire and its resubmission round have been finished, the Revision Plan Questionnaire is appended to the Assignment Questionnaire for that around.

Wireframe of Completing a Review for an Assignment

Implementation

• Add Revision Plan questions to Response

app/controllers/response_controller.rb

This methods gets questions sorted by sequence from review and revision questionnaire and merges them in a single list.

 def set_questions_for_new_response
   @questions = sort_questions(@questionnaire.questions)
   if(@assignment && @assignment.is_revision_planning_enabled)
     reviewees_topic = SignedUpTeam.topic_id_by_team_id(@contributor.id)
     current_round = @assignment.number_of_current_round(reviewees_topic)
     @revision_plan_questionnaire = RevisionPlanTeamMap.find_by(team_id: @map.reviewee_id, used_in_round: current_round).try(:questionnaire)
     if(@revision_plan_questionnaire)
       @questions += sort_questions(@revision_plan_questionnaire.questions)
     end
   end
   return @questions
 end

This method gets questions from a response instead of questionnaire. We use this when response has already been created.

 def set_questions
   @questions = @response.get_questions
 end

app/models/response.rb

Added a method to get all the questionnaires for a response, response is a collection of answers. Answers contain reference to their question, and question have a reference to questionnaire.

 def questionnaires_by_answers(answers)
   answers_with_questionnaires = answers.select{ |ans|  ans && ans.question && ans.question.questionnaire }
   questionnaires = answers_with_questionnaires.collect{ |ans| ans.question.questionnaire }.uniq
   unless(questionnaires.any?)
     questionnaires = []
     questionnaires << questionnaire_by_answer(answers.first)
   end
   questionnaires
 end

This method gets all the questions for a response.

 def get_questions
   @questions = []
   questionnaires = questionnaires_by_answers(scores)
   questionnaires.each {|questionnaire| @questions += questionnaire.questions }
   return @questions
 end

• Separating questions by their questionnaire type

app/models/response.rb

construct_review_response method has been updated to create separate tables for review questions, revision questions and additional comment.

app/models/questionnaire.rb

Add a method to check if heading is to be displayed for the questionnaire.

 def display_heading?
   return false
 end

app/models/revision_plan_questionnaire.rb

Overriding the method defined in questionnaire.rb to display heading for revision plan questionnaire.

 def display_heading?
   return true
 end

app/views/response/response.html.erb

View groups questions by questionnaire type and then adds a heading for questionnaires that return true for display_heading?.

Bug fix: The existing implementation has a bug where the first question in a questionnaire is displayed last in response. This was fixed as well.

Summary Report Page - Under progress

When a project has been reviewed at least once, a participant is able to view their team's score. The UI below shows what this looks like after the second round of reviews. For the second and all subsequent reviews, the results of questions that were created by the instructor are shown under Assignment Questionnaire. The results of the questions created by the team are shown under Improvement Plan.

Initial Wireframe

Wireframe of Summary Report for an Assignment

Final UI

Summary Report page demonstrating added Revision Plan heatgrid

Implementation

• Show revision plan questions in heat grid.

app/controllers/grades_controller.rb

app/helpers/summary_helper.rb

app/models/vm_question_response.rb

app/views/grades/view_team.html.erb

• Show grading for revision plan questions.

Control Flow Diagram

Control Flow of the Revision Planning Function

Test Plan - Under progress

RSpec Testing

The RSpec tests are written to test both controller and models.

Controllers

• spec/controllers/grades_controller_spec.rb:

• spec/controllers/revision_plan_questionnaires_controller_spec.rb: test revision plan questionnaire can be created and edited.

• spec/controllers/assignments_controller_spec.rb: create and validate an assignment with revision planning enabled.

• spec/controllers/response_controller_spec.rb: Updated spec to pass build by stubbing get_questions method.

Models

• spec/models/revision_plan_questionnaire_spec.rb:

• spec/models/revision_plan_team_map_spec.rb:

• spec/models/response_spec.rb: Updated spec to pass build by updating expected html output.

• spec/models/review_response_map_spec.rb: Updated spec to pass build by updating expected html output.

Manual Testing

Manual testing aims to verify the following:

• Create an assignment with revision planning enabled.
• Are participants allowed to create/edit revision plan when round 1+ (1 or greater than 1) reviews have finished.
• Is revision plan editing disabled when assignment is in review stage.
• Are reviewers shown questions created by reviewees.
• Are participants shown summary of score for revision plan after review deadline has expired.

Team Members

Chaitanya Mehta (cmehta)
Darby Madewell (demadewe)
Dongni Yang (dyang23)
Sidharth Mehta (smehta22)
Mentor: Yulin Zhang (yzhan114)

References

1. Previous Implementation: E1875 wiki, E2016 wiki
2. Forked Repository: E2083 github
3. Pull Request: E2083 pull request
4. Deployment: Deployment Link