CSC/ECE 517 Fall 2018/E1879 Student Generated Questions Added To Rubric
Introduction
The rubrics in expertiza are created by instructors. These Rubrics only contain questions that are related to the existed topics. Now, Expertiza only supports students to pull questionnaires is specific rubrics to get some help. Even that they can ask about anything that is relevant to all the projects that will be submitted But when students encounter difficult problems which are not in the existed topics, and they want to get special advice on that new field, the demand for creating supplementary review questionnaire raises. This project (E1879) aims to solve this problem by allowing students to add questions to the standard instructor generated rubric so that they can get specific feedback on from the reviewers.
Proposed solution
We will add the Supplementary Review Questions to the current Review Questions, and show these student-generated questions under the rubric given by instructor. Even though these questions won't be graded, they will eventually provide diversity for feedback. It will increase the benefit that each team gets because they can get feedback that is specific to their project.
Implementation
Design Strategy
In Expertiza, all kinds of rubrics and surveys are subclasses of Questionnaire. A Questionnaire can contain “questions” of several types (e.g., checkboxes, dropdowns, text boxes). So that we will add a new subclass of Questionnaire named SupplementaryReviewQuestionnaire to achieve our goal.
First, we want the professor to control if the students can create their own supplementary questions by using a checkbox to indicate. If the professor allow students to create their own questions, there will be a button in the students "Your work" page which can let the students jump to the same page that an instructor lands on when creating a new rubric. Then we will add the questionnaire id of this new SupplementaryReviewQuestionnaire to the Team table when students create their own questionnaire in order to show the corresponding supplementary questions to the reviewers. And when showing review questions to the reviewers we will look up the questionnaire id in Team table and modify the view to add the supplementary questions to review page. Finally, after the reviewers finish the review and submit their responds, we will change the view and let the team member can see the feedback of their supplementary questions.
Changes in the User Interface
1. Assignment Page
We add a checkbox for the professor to indicate if student can add their own questions.
2. A button called "Create Supplementary Rubric" will appear in the student's "Your Work" section when the Instructor has allowed students to create supplementary review questions. So when a student clicks the button he/she can add the desired questions.
3. The page where student will be directed to when he/she clicks the Create Supplementary Rubric button. Students can add Questions to the created Supplementary Review questionnaire here.
4. Review Page
The students own supplementary questions are leading by a heading "Supplementary Review Questions".
5. Review Results Page
The supplementary review results will be showed to the team like other results.
Use Cases
1. Use case of creating a Supplementary Review Questionnaire
2. Use case of reviewing questions for reviewers.
3. Use case of reviewing responses of questions.
Test Plan
Actually, we need to add tests for the following:
1. To check the link for "Supplementary Review Questionnaire" appears in the "Your Work" section of a student.
2. To check if the link for "Supplementary Review Questionnaire" redirects to page which allows to create questionnaire.
3. To check if the questionnaire ID is stored in the Team table.
4. To check if the reviewers can see the supplementary questions that were added by the team as part of the review questions.
5. To check if the responses of the Supplementary Review Questions have been added to the responses of the existing review questions.
File which are added/modified
1. Models
app/models/team.rb
Look up the questionnaire ID of a team.
def self.get_supplementary_review_questionnaire_id_of_team(team_id)
team = Team.find(team_id)
if team.blank?
nil
else
team.supplementary_review_questionnaire_id
end
end
app/models/supplementary_review_questionnaire.rb
Add the SupplementaryReviewQuestionnaire class which is the subclass of Questionnaire.
# This is one type of Questionnaire and as intuitively expected this model class # derives from Questionnaire. class SupplementaryReviewQuestionnaire < Questionnaire # Make me better. attr_accessible end
app/models/response.rb
Add supplementary review responses to "Show Review" section.
def construct_review_response code, self_id, show_tags = nil, current_user = nil
code += '<table id="review_' + self_id + '" style="display: none;" class="table table-bordered">'
answers = Answer.where(response_id: self.response_id)
team_id = response_map.reviewee_id
supplementary_review_questionnaire_id = Team.get_supplementary_review_questionnaire_id_of_team(team_id)
unless answers.empty?
questionnaire = self.questionnaire_by_answer(answers.first)
unless supplementary_review_questionnaire_id.nil?
supplementary_review_questionnaire = Questionnaire.find(supplementary_review_questionnaire_id)
supplementary_review_questions = supplementary_review_questionnaire.questions.sort_by(&:seq)
end
questionnaire_max = questionnaire.max_question_score
questions = questionnaire.questions.sort_by(&:seq)
unless supplementary_review_questionnaire.nil?
questions += supplementary_review_questions
end
# get the tag settings this questionnaire
tag_prompt_deployments = show_tags ? TagPromptDeployment.where(questionnaire_id: questionnaire.id, assignment_id: self.map.assignment.id) : nil
code = add_table_rows questionnaire_max, questions, answers, code, tag_prompt_deployments, current_user
end
comment = if !self.additional_comment.nil?
self.additional_comment.gsub('^p', '').gsub(/\n/, '<BR/>')
else
''
end
code += '<tr><td><b>Additional Comment: </b>' + comment + '</td></tr>'
code += '</table>'
end
2. Views:
app/views/assignments/edit/_rubrics.html.erb
Add checkbox which professor used to indicate whether students can create their own questions.
<!--Add a allow studuent-generated questionnaires checkbox. -->
<input name="assignment_form[assignment][supplementary_review_questionnaire]" type="hidden" value="false"/>
<%= check_box_tag('assignment_form[assignment][supplementary_review_questionnaire]', 'true', @assignment_form.assignment.supplementary_review_questionnaire) %>
<%= label_tag('assignment_form[assignment][supplementary_review_questionnaire]', 'Allow student-generated questions added to rubric?') %>
app/views/response/response.html.erb
Add code to show the supplementary questions to the reviewers.
<br>
<% unless @supplementary_review_questions.nil? %>
<tr>
<td>
<h2>Supplementary Review Questions</h2>
</td>
</tr>
<% @supplementary_review_questions.each do |question| %>
<tr>
<td>
<% answer = Answer.where(question_id: question.id, response_id: @response.id).first if !@response.nil? %>
<% if question.instance_of? Criterion %>
<%= question.complete(i, answer, @questionnaire.min_question_score, @questionnaire.max_question_score, @dropdown_or_scale) %>
<%= tinymce %>
<% elsif question.instance_of? Scale %>
<%= question.complete(i, answer, @questionnaire.min_question_score, @questionnaire.max_question_score) %>
<%= tinymce %>
<% elsif question.instance_of? UploadFile %>
<% else %>
<%= question.complete(i, answer) %>
<% end %>
</td>
<td width="25%">
<% if [Criterion, TextArea].include? question.class %>
<div id="analysis_<%= i.to_s %>_comments"></div>
<% end %>
</td>
</tr>
<% i += 1 %>
<% end %>
<% end %>
app/views/submitted_content/_main.html.erb
Add button for "your work" page to let students create their own questions.
<% if stage != "Finished" and controller.controller_name == 'submitted_content' and @can_submit %>
<h2>Add Supplementary Review Questions:</h2>
<% if @assignment.supplementary_review_questionnaire? %>
<%= button_to "Create Supplementary Rubric", {:controller => "questionnaires", :action => "create_supplementary_review_questionnaire", :id => participant.id } %>
<% else %>
The instructor hasn't enabled this feature for this assignment
<% end %>
<% end %>
3. Controllers:
app/controllers/response_controller.rb
Add supplementary review questionnaires part to the create, edit, update methods and some other methods which getting the questionnaires.
# Prepare the parameters when student clicks "Edit"
def edit
assign_instance_vars
get_all_responses
if @prev.present?
@sorted = @review_scores.sort {|m1, m2| m1.version_num.to_i && m2.version_num.to_i ? m2.version_num.to_i <=> m1.version_num.to_i : (m1.version_num ? -1 : 1) }
@largest_version_num = @sorted[0]
end
@modified_object = @response.response_id
# set more handy variables for the view
set_content
@review_scores = []
@questions.each do |question|
@review_scores << Answer.where(response_id: @response.response_id, question_id: question.id).first
end
@supplementary_review_questions.each do |question|
@review_scores << Answer.where(response_id: @response.response_id, question_id: question.id).first
end
@questionnaire = set_questionnaire
render action: 'response'
end
# Update the response and answers when student "edit" existing response
def update
render nothing: true unless action_allowed?
# the response to be updated
@response = Response.find(params[:id])
msg = ""
begin
@map = @response.map
@response.update_attribute('additional_comment', params[:review][:comments])
set_questionnaire
questions = sort_questions(@questionnaire.questions)
supplementary_review_questions = sort_questions(@supplementary_review_questionnaire.questions)
unless @supplementary_review_questionnaire.nil?
questions += supplementary_review_questions
end
create_answers(params, questions) unless params[:responses].nil?
@response.update_attribute('is_submitted', true) if params['isSubmit'] && params['isSubmit'] == 'Yes'
@response.notify_instructor_on_difference if (@map.is_a? ReviewResponseMap) && @response.is_submitted && @response.significant_difference?
rescue StandardError
msg = "Your response was not saved. Cause:189 #{$ERROR_INFO}"
end
ExpertizaLogger.info LoggerMessage.new(controller_name, session[:user].name, "Your response was submitted: #{@response.is_submitted}", request)
redirect_to controller: 'response', action: 'save', id: @map.map_id,
return: params[:return], msg: msg, review: params[:review], save_options: params[:save_options]
end
def create
map_id = params[:id]
map_id = params[:map_id] unless params[:map_id].nil? # pass map_id as a hidden field in the review form
@map = ResponseMap.find(map_id)
@team_id = @map.reviewee_id
get_all_responses
if params[:review][:questionnaire_id]
@questionnaire = Questionnaire.find(params[:review][:questionnaire_id])
@round = params[:review][:round]
@supplementary_review_questionnaire_id = Team.get_supplementary_review_questionnaire_id_of_team(@team_id)
unless @supplementary_review_questionnaire_id.nil?
@supplementary_review_questionnaire = Questionnaire.find(@supplementary_review_questionnaire_id)
end
else
@round = nil
end
is_submitted = (params[:isSubmit] == 'Yes')
was_submitted = false
@response = Response.where(map_id: @map.id, round: @round.to_i).first
if @response.nil?
@response = Response.create(
map_id: @map.id,
additional_comment: params[:review][:comments],
round: @round.to_i,
is_submitted: is_submitted
)
end
was_submitted = @response.is_submitted
@response.update(additional_comment: params[:review][:comments], is_submitted: is_submitted) # ignore if autoupdate try to save when the response object is not yet created.
# ,:version_num=>@version)
# Change the order for displaying questions for editing response views.
questions = sort_questions(@questionnaire.questions)
unless @supplementary_review_questionnaire.nil?
supplementary_review_questions = sort_questions(@supplementary_review_questionnaire.questions)
questions += supplementary_review_questions
end
create_answers(params, questions) if params[:responses]
msg = "Your response was successfully saved."
error_msg = ""
# only notify if is_submitted changes from false to true
if (@map.is_a? ReviewResponseMap) && (was_submitted == false && @response.is_submitted) && @response.significant_difference?
@response.notify_instructor_on_difference
@response.email
end
redirect_to controller: 'response', action: 'save', id: @map.map_id,
return: params[:return], msg: msg, error_msg: error_msg, review: params[:review], save_options: params[:save_options]
end
# new_response if a flag parameter indicating that if user is requesting a new rubric to fill
# if true: we figure out which questionnaire to use based on current time and records in assignment_questionnaires table
# e.g. student click "Begin" or "Update" to start filling out a rubric for others' work
# if false: we figure out which questionnaire to display base on @response object
# e.g. student click "Edit" or "View"
def set_content(new_response = false)
@title = @map.get_title
if @map.survey?
@survey_parent = @map.survey_parent
else
@assignment = @map.assignment
end
@participant = @map.reviewer
@contributor = @map.contributor
new_response ? set_questionnaire_for_new_response : set_questionnaire
set_dropdown_or_scale
@questions = sort_questions(@questionnaire.questions)
unless @supplementary_review_questionnaire.nil?
@supplementary_review_questions = sort_questions(@supplementary_review_questionnaire.questions)
end
@min = @questionnaire.min_question_score
@max = @questionnaire.max_question_score
end
def set_questionnaire_for_new_response
case @map.type
when "ReviewResponseMap", "SelfReviewResponseMap"
reviewees_topic = SignedUpTeam.topic_id_by_team_id(@contributor.id)
@current_round = @assignment.number_of_current_round(reviewees_topic)
@questionnaire = @map.questionnaire(@current_round)
@supplementary_review_questionnaire_id = Team.get_supplementary_review_questionnaire_id_of_team(@contributor.id)
unless @supplementary_review_questionnaire_id.nil?
@supplementary_review_questionnaire = Questionnaire.find(@supplementary_review_questionnaire_id)
end
when
"MetareviewResponseMap",
"TeammateReviewResponseMap",
"FeedbackResponseMap",
"CourseSurveyResponseMap",
"AssignmentSurveyResponseMap",
"GlobalSurveyResponseMap"
@questionnaire = @map.questionnaire
end
end
def set_questionnaire
# if user is not filling a new rubric, the @response object should be available.
# we can find the questionnaire from the question_id in answers
answer = @response.scores.first
@questionnaire = @response.questionnaire_by_answer(answer)
supplementary_review_answer = @response.scores.last
@supplementary_review_questionnaire = @response.questionnaire_by_answer(supplementary_review_answer)
end
app/controllers/grades_controller.rb
Add supplementary review responses to "Your Scores" section.
def view_team
@participant = AssignmentParticipant.find(params[:id])
@assignment = @participant.assignment
@team = @participant.team
@team_id = @team.id
@questions = {}
questionnaires = @assignment.questionnaires
retrieve_questions questionnaires
@pscore = @participant.scores(@questions)
@vmlist = []
# loop through each questionnaire, and populate the view model for all data necessary
# to render the html tables.
counter_for_same_rubric = 0
questionnaires.each do |questionnaire|
@round = nil
if @assignment.varying_rubrics_by_round? && questionnaire.type == "ReviewQuestionnaire"
questionnaires = AssignmentQuestionnaire.where(assignment_id: @assignment.id, questionnaire_id: questionnaire.id)
if questionnaires.count > 1
@round = questionnaires[counter_for_same_rubric].used_in_round
counter_for_same_rubric += 1
else
@round = questionnaires[0].used_in_round
counter_for_same_rubric = 0
end
end
vm = VmQuestionResponse.new(questionnaire, @assignment, @round)
supplementary_review_questionnaire_id = Team.get_supplementary_review_questionnaire_id_of_team(@team_id)
unless supplementary_review_questionnaire_id.nil?
supplementary_review_questionnaire = Questionnaire.find(supplementary_review_questionnaire_id)
unless supplementary_review_questionnaire.nil?
supplementary_review_questions = supplementary_review_questionnaire.questions
questionnaire.questions += supplementary_review_questions
end
end
vmquestions = questionnaire.questions
vm.add_questions(vmquestions)
vm.add_team_members(@team)
vm.add_reviews(@participant, @team, @assignment.varying_rubrics_by_round?)
vm.get_number_of_comments_greater_than_10_words
@vmlist << vm
end
@current_role_name = current_role_name
end
app/controllers/questionnaires_controller.rb
Method for creating Supplementary Review Questionnaire.
def create_supplementary_review_questionnaire
@participant = AssignmentParticipant.find(params[:id])
@team = Team.find(@participant.team.id)
if @team.supplementary_review_questionnaire_id.nil?
@questionnaire = Questionnaire.create(privete: false, name: "supplementary_review_questionnaire_" + @team.id.to_s,
instructor_id: @team_id, min_question_score: 0, max_question_score: 5, type: "Questionnaire", display_type: "Review"
instruction_loc: Questionnaire::DEFAULT_QUESTIONNAIRE_URL)
if @questionnaire.save
@team.supplementary_review_questionnaire_id = @questionnaire.id
@team.save
flash[:success] = 'You have successfully created a rubric!'
else
flash[:error] = $ERROR_INFO
end
else
@questionnaire = Questionnaire.find(@team.supplementary_review_questionnaire_id)
end
redirect_to controller: 'questionnaires', action: 'edit', id: @questionnaire.id
end
4. Config:
routes.rb
resources :questionnaires, only: %i[new create edit update] do
collection do
get :copy
get :list
post :list_questionnaires
get :new_quiz
get :select_questionnaire_type
post :select_questionnaire_type
get :toggle_access
get :view
post :create_quiz_questionnaire
post :update_quiz
post :add_new_questions
post :save_all_questions
post :create_supplementary_review_questionnaire
end
end
resources :author_feedback_questionnaires, controller: :questionnaires
resources :review_questionnaires, controller: :questionnaires
resources :metareview_questionnaires, controller: :questionnaires
resources :teammate_review_questionnaires, controller: :questionnaires
resources :survey_questionnaires, controller: :questionnaires
resources :assignment_survey_questionnaires, controller: :questionnaires
resources :global_survey_questionnaires, controller: :questionnaires
resources :course_survey_questionnaires, controller: :questionnaires
resources :bookmarkrating_questionnaires, controller: :questionnaires
resources :supplementary_review_questionnaire, controller: :questionnaires
Team members
Yanchen Zhao
Pingping Chen
Jianshu Zhang
Zirun Han
Links
Expertiza on Github:https://github.com/zyczyh/expertiza
Pull Request:https://github.com/expertiza/expertiza/pull/1325
Expertiza YouTube channel:https://youtu.be/3PUNknSbU-k