CSC/ECE 517 Fall 2020 - E2083. Revision planning tool E2016: Difference between revisions
Line 105: | Line 105: | ||
[[File:Revision_Planning_Link.png|600px|thumb|center|Submit or Review Work page demonstrating added Revision Planning link]] | [[File:Revision_Planning_Link.png|600px|thumb|center|Submit or Review Work page demonstrating added Revision Planning link]] | ||
=====Implementation===== | |||
* Show link to revision planning page when revision planning is enabled for an assignment, round 1 has passed and current stage is submission. | * Show link to revision planning page when revision planning is enabled for an assignment, round 1 has passed and current stage is submission. | ||
Revision as of 02:10, 17 November 2020
This page provides a description of the Expertiza based OSS project.
Introduction
Rounds of peer reviews may be implemented between submissions for assignments on Expertiza. In order to better track the implementation of reviewer's suggestions, a Revision Planning Tool should be implemented.
Problem Statement
In the first round of Expertiza reviews, we ask reviewers to give authors some guidance on how to improve their work. Then in the second round, reviewers rate how well authors have followed their suggestions. We could carry the interaction one step further if we asked authors to make up a revision plan based on the first-round reviews. That is, authors would say what they were planning to do to improve their work. Then second-round reviewers would assess how well they did it. In essence, this means that authors would be adding criteria to the second-round rubric that applied only to their submission. We are interested in having this implemented and used in a class so that we can study its effect.
Previous Implementations
Revision planning has been implemented twice before, once in E1875 and once in E2016. While the functionality worked and effectively minimized changes to the code, they also had the following problems:
- Hardcoded “round” numbers in many places of the code.
- Documentation does not reflect the new changes they made.
- Revision planning responses and responses to the other items are not distinguished in heatgrid view.
- The idea of adding a
team_id
field to each question is intuitive. However, they failed to come up with a clean implementation of this idea. Specifically, they had passed some trailing parameters several methods down before reaching the place that needs them.
Rationale
To implement revision planning, E2016 added is_revision_planning_enabled
(Boolean) to Assignment
and team_id
to Question
. Adding a Boolean to enable revision planning for an assignment makes sense, but having certain questions belong to a team is not a clean implementation. Instead, we will create a new questionnaire that will belong to each team and keep track of the set of questions. We will use this new questionnaire to separate the revision plan responses and responses to other items in heat-grid view by creating a different table for each.
Files Modified - Under progress
This is a list of files modified and added. The list will be updated as the implementation progresses.
Changed:
- app/controllers/advice_controller.rb
- app/controllers/grades_controller.rb
- app/controllers/questionnaires_controller.rb
- app/controllers/questions_controller.rb
- app/controllers/response_controller.rb
- app/controllers/student_task_controller.rb
- app/helpers/summary_helper.rb
- app/models/questionnaire.rb
- app/models/response.rb
- app/models/student_task.rb
- app/models/vm_question_response.rb
- app/views/advice/edit_advice.html.erb
- app/views/assignments/edit/_general.html.erb
- app/views/grades/view_team.html.erb
- app/views/questionnaires/_questionnaire.html.erb
- app/views/response/response.html.erb
- app/views/student_task/view.html.erb
- config/routes.rb
- db/schema.rb
- spec/controllers/response_controller_spec.rb
- spec/models/response_spec.rb
- spec/models/review_response_map_spec.rb
Added:
- app/controllers/revision_plan_questionnaires_controller.rb
- app/models/revision_plan_questionnaire.rb
- app/models/revision_plan_team_map.rb
- app/views/questionnaires/_questions.html.erb
- app/views/revision_plan_questionnaires/edit.html.erb
- db/migrate/20201106020642_create_revision_plan_team_maps.rb /
- db/migrate/20201105023711_add_is_revison_planning_enabled_to_assignments.rb
Design
Database Design
Items in green are additions.
Changes:
- In the assignment table we have added
is_revision_planning_enabled?
column to indicate whether the assignment accepts a revision plan along with review rubric. RevisionPlanTeamMap
maps a questionnaire to an assignment team and round. This will map to a questionnaire of type revision plan that will be created by the revewee.RevisionPlanQuestionnaire
extendsQuestionnaire
using single table inheritance.
User Interface
Enable Revision Planning
In order to enable Revision Planning, the setting must be enabled when creating or editing an assignment under the General tab. The wireframe below demonstrates creating an assignment, and editing the assignment functions similarly. Revision plan? should be checked to enable this option.
Initial Wireframe
Final UI
Implementation
- Added a Boolean field
is_revision_planning_enabled
toAssignment
that saves whether revision planning is enabled.
db/migrate/20201105023711_add_is_revison_planning_enabled_to_assignments.rb
class AddIsRevisonPlanningEnabledToAssignments < ActiveRecord::Migration def change add_column :assignments, :is_revision_planning_enabled, :boolean, default: false end end
- Add checkbox to enable revision planning in edit assignment view.
app/views/assignments/edit/_general.html.erb
<input name="assignment_form[assignment][is_revision_planning_enabled]" type="hidden" value="false"/> <%= check_box_tag('assignment_form[assignment][is_revision_planning_enabled]', 'true', @assignment_form.assignment.is_revision_planning_enabled) %> <%= label_tag('assignment_form[assignment][is_revision_planning_enabled]', 'Revision Planning?') %>
Assignment Overview Page (contains the Link to the revision planning page)
The Revision Planning link is available to students during every submission period (except the first round submission) and not available during every review period. As shown in the wireframe, by clicking Revision Planning students would be redirected to the ‘Revision planning page’.
Initial Wireframe
Final UI
Implementation
- Show link to revision planning page when revision planning is enabled for an assignment, round 1 has passed and current stage is submission.
In order to check if a team is allowed to submit a revision plan @can_submit_revision_plan
field is added to StudentTaskController
's view action. If a revision plan questionnaire (@revision_plan_questionnaire_id
) exists for the current round then clicking the link performs RevisionPlanQuestionnairesController
's edit action else new action is performed.
app/views/student_task/view.html.erb
<% if @team && @assignment.is_revision_planning_enabled %> <li> <% if @can_submit_revision_plan %> <% if @revision_plan_questionnaire_id %> <%= link_to "Revision Planning", controller: 'revision_plan_questionnaires', action: 'edit', id: @revision_plan_questionnaire_id %> <% else %> <%= link_to "Revision Planning", controller: 'revision_plan_questionnaires', action: 'new', team_id: @team %> <% end %> (Add new questions to next-round rubric) <% else %> <font color="gray">Revision Planning</font> (You are not allowed to submit your revison plan right now) <% end %> </li> <%end%>
app/controllers/student_task_controller.rb
def view student_task = StudentTask.from_participant_id params[:id] ... ... # Revision plan feature @can_submit_revision_plan = student_task.can_submit_revision_plan? @revision_plan_questionnaire_id = student_task.revision_plan_questionnaire_id end
app/models/student_task.rb
def current_round assignment.number_of_current_round(@topic.try(:id)) end def can_submit_revision_plan? current_stage == 'submission' && current_round > 1 end def revision_plan_questionnaire_id RevisionPlanTeamMap.find_by(team: @participant.team, used_in_round: current_round).try(:questionnaire_id) end
Editing the Revision Plan Questionnaire
After creating the Revision Plan Questionnaire, it must be edited. Questions can be added by specifying the amount of questions and their type. Questions can be removed by clicking Remove in the leftmost column. Once the questionnaire is complete, it can be saved. This page will be visible during each submission period after the first and will be unavailable during all review periods.
Initial Wireframe
Final UI
Implementation
- Create a model RevisionPlanTeamMap that maps a revision plan to a team and stores information of the round it is used in.
/app/models/revision_plan_team_map.rb
class RevisionPlanTeamMap < ActiveRecord::Base belongs_to :team belongs_to :questionnaire end
/db/migrate/20201106020642_create_revision_plan_team_maps.rb
class CreateRevisionPlanTeamMaps < ActiveRecord::Migration def change create_table :revision_plan_team_maps do |t| t.references :team, index: true, foreign_key: true t.references :questionnaire, index: true, foreign_key: true t.integer :used_in_round t.timestamps null: false end end end
- Create a Revision Plan Questionnaire Model and Controller
app/models/revision_plan_questionnaire.rb
Added a model for revision plan questionnaire. The code provides method to get questionnaire for a current round based on team.
def self.get_questionnaire_for_current_round(team_id) assignment_team = Team.find(team_id) assignment = assignment_team.assignment current_round = assignment.number_of_current_round(assignment_team.topic) questionnaire = RevisionPlanTeamMap.find_by(team: assignment_team, used_in_round: current_round).try(:questionnaire) unless questionnaire questionnaire = RevisionPlanQuestionnaire.new questionnaire.name = 'Revision Plan Questionnaire' questionnaire.instructor_id = assignment.instructor_id questionnaire.max_question_score = 5 questionnaire.save questionnaire_team_map = RevisionPlanTeamMap.create(team_id: assignment_team.id, used_in_round: current_round, questionnaire_id: questionnaire.id) end return questionnaire end
This method returns the team associated to a revision plan questionnaire.
def team revision_plan_team_map.team end
app/models/questionnaire.rb
Added a new type of questionnaire 'RevisionPlanQuestionnaire'
to QUESTIONNAIRE_TYPES
QUESTIONNAIRE_TYPES = ['ReviewQuestionnaire', ... ... 'RevisionPlanQuestionnaire'].freeze
app/controllers/revision_plan_questionnaires_controller.rb
Added a controller RevisionPlanQuestionnairesController
. The controller derives from QuestionnairesController
, it overrides the new
action to allow creation of a RevisionPlanQuestionnaire
for a team
. It also updates permissions for controller.
def action_allowed? case params[:action] when 'edit' questionnaire = Questionnaire.find(params[:id]) questionnaire.team.users.collect { |u| u.id }.include? session[:user].id || super (user_logged_in? && questionnaire.team.users.collect { |u| u.id }.include?(session[:user].id)) || super else super end end
def new begin questionnaire = RevisionPlanQuestionnaire.get_questionnaire_for_current_round(params[:team_id]) redirect_to action: 'edit', id: questionnaire.id rescue StandardError flash[:error] = $ERROR_INFO end end
config/routes.rb
Add routes for RevisionPlanQuestionnairesController
.
resources :revision_plan_questionnaires, controller: :revision_plan_questionnaires, only: %i[new edit update] do collection do post :add_new_questions post :save_all_questions end end
- Add a view to edit Revision Plan
app/views/revision_plan_questionnaires/edit.html.erb
<%= render :partial => 'question_weight' %> <h1>Edit <%= @questionnaire.display_type %></h1> <%= render :partial => 'questions' %> <br /> <a href="javascript:window.history.back()">Back</a>
- Refactor code to display questions into a partial. Allows for code reuse.
app/controllers/questionnaires_controller.rb
Updated redirect_to
links for add_new_questions
and save_all_questions
methods.
redirect_to action: 'edit', id: questionnaire_id.to_sym
app/views/questionnaires/_questionnaire.html.erb
This file generates a view to edit/create a new questionnaire. It has functionality to display questions and some functions that are not needed for revision plan questionnaire. To enable code reuse the functionality to add/edit/remove/save questions was moved to a new partial app/views/questionnaires/_questions.html.erb.
app/views/questionnaires/_questions.html.erb
- Add code to allow users to delete questions from revision plan questionnaire.
app/controllers/questions_controller.rb
Update authorization code to allow a user to remove questions from his teams revision plan.
def action_allowed? if ['destroy'].include?(params[:action]) question = Question.find(params[:id]) if(user_logged_in? && question.questionnaire.owner?(session[:user].id)) return true end end current_user_has_ta_privileges? end
def destroy ... ... if(question.questionnaire.type == 'RevisionPlanQuestionnaire') redirect_to edit_revision_plan_questionnaire_path(questionnaire_id.to_s.to_sym) else redirect_to edit_questionnaire_path(questionnaire_id.to_s.to_sym) end end
app/models/questionnaire.rb
Method returns true if user owns questionnaire.
def owner?(user_id) instructor_id == user_id end
app/models/revision_plan_questionnaire.rb
Overrides implementation for owner?
defined in Questionnaire
to return true if a team owns the questionnaire.
def owner?(user_id) team.users.map{ |u| u.id }.include?(user_id) || super end
- Add code to allow users to access show/edit advice functionality for revision plan questions.
app/controllers/advice_controller.rb
Update authorization code to allow a user to update advice his teams revision plan.
def action_allowed? questionnaire = Questionnaire.find(params[:id]) if(user_logged_in? && questionnaire.owner?(session[:user].id)) return true end current_user_has_ta_privileges? end
app/views/advice/edit_advice.html.erb
Update 'Back to questionnaire' link.
<% if @questionnaire.type == 'RevisionPlanQuestionnaire'%> <%= link_to 'Back to questionnaire', :controller => 'revision_plan_questionnaires', :action => 'edit', :id => @questionnaire.id %> <% else %> <%= link_to 'Back to questionnaire', :controller => 'questionnaires', :action => 'edit', :id => @questionnaire.id %> <% end %>
Reviewing an Assignment
Once the Revision Plan Questionnaire and its resubmission round have been finished, the Revision Plan Questionnaire is appended to the Assignment Questionnaire for that round.
Initial Wireframe
Final UI
Implementation
- Add Revision Plan questions to Response
app/controllers/response_controller.rb
This methods gets questions sorted by sequence from review and revision questionnaire and merges them in a single list.
def set_questions_for_new_response @questions = sort_questions(@questionnaire.questions) if(@assignment && @assignment.is_revision_planning_enabled) reviewees_topic = SignedUpTeam.topic_id_by_team_id(@contributor.id) current_round = @assignment.number_of_current_round(reviewees_topic) @revision_plan_questionnaire = RevisionPlanTeamMap.find_by(team_id: @map.reviewee_id, used_in_round: current_round).try(:questionnaire) if(@revision_plan_questionnaire) @questions += sort_questions(@revision_plan_questionnaire.questions) end end return @questions end
This method gets questions from a response instead of questionnaire. We use this when response has already been created.
def set_questions @questions = @response.get_questions end
app/models/response.rb
Added a method to get all the questionnaires for a response, response is a collection of answers. Answers contain reference to their question, and question have a reference to questionnaire.
def questionnaires_by_answers(answers) answers_with_questionnaires = answers.select{ |ans| ans && ans.question && ans.question.questionnaire } questionnaires = answers_with_questionnaires.collect{ |ans| ans.question.questionnaire }.uniq unless(questionnaires.any?) questionnaires = [] questionnaires << questionnaire_by_answer(answers.first) end questionnaires end
This method gets all the questions for a response.
def get_questions @questions = [] questionnaires = questionnaires_by_answers(scores) questionnaires.each {|questionnaire| @questions += questionnaire.questions } return @questions end
- Separating questions by their questionnaire type
app/models/response.rb
construct_review_response
method has been updated to create separate tables for review questions, revision questions and additional comment.
app/models/questionnaire.rb
Add a method to check if heading is to be displayed for the questionnaire.
def display_heading? return false end
app/models/revision_plan_questionnaire.rb
Overriding the method defined in questionnaire.rb to display heading for revision plan questionnaire.
def display_heading? return true end
app/views/response/response.html.erb
View groups questions by questionnaire type and then adds a heading for questionnaires that return true for display_heading?
.
Bug fix: The existing implementation has a bug where the first question in a questionnaire is displayed last in response. This was fixed as well.
Summary Report Page - Under progress
When a project has been reviewed at least once, a participant is able to view their team's score. The UI below shows what this looks like after the second round of reviews. For the second and all subsequent reviews, the results of questions that were created by the instructor are shown under Assignment Questionnaire. The results of the questions created by the team are shown under Improvement Plan.
Initial Wireframe
Final UI
Implementation
- Show revision plan questions in heat grid.
app/controllers/grades_controller.rb
app/helpers/summary_helper.rb
app/models/vm_question_response.rb
app/views/grades/view_team.html.erb
- Show grading for revision plan questions.
Control Flow Diagram
Test Plan - Under progress
RSpec Testing
The RSpec tests are written to test both controller and models.
Controllers
- spec/controllers/grades_controller_spec.rb:
- spec/controllers/revision_plan_questionnaires_controller_spec.rb: test revision plan questionnaire can be created and edited.
- spec/controllers/assignments_controller_spec.rb: create and validate an assignment with revision planning enabled.
- spec/controllers/response_controller_spec.rb: Updated spec to pass build by stubbing
get_questions
method.
Models
- spec/models/revision_plan_questionnaire_spec.rb:
- spec/models/revision_plan_team_map_spec.rb:
- spec/models/response_spec.rb: Updated spec to pass build by updating expected html output.
- spec/models/review_response_map_spec.rb: Updated spec to pass build by updating expected html output.
Manual Testing
Manual testing aims to verify the following:
- Create an assignment with revision planning enabled.
- Are participants allowed to create/edit revision plan when round 1+ (1 or greater than 1) reviews have finished.
- Is revision plan editing disabled when assignment is in review stage.
- Are reviewers shown questions created by reviewees.
- Are participants shown summary of score for revision plan after review deadline has expired.
Team Members
Chaitanya Mehta (cmehta)
Darby Madewell (demadewe)
Dongni Yang (dyang23)
Sidharth Mehta (smehta22)
Mentor: Yulin Zhang (yzhan114)
References
- Previous Implementation: E1875 wiki, E2016 wiki
- Forked Repository: E2083 github
- Pull Request: E2083 pull request
- Deployment: Deployment Link