Expertiza_Wiki - User contributions [en]

CSC/ECE 517 Fall 2018/E1879 Student Generated Questions Added To Rubric

2018-12-18T20:56:38Z

Zhan23:

'''Introduction'''
----
The rubrics in expertiza are created by instructors. These Rubrics only contain questions that are related to the existed topics. Now, Expertiza only supports students to pull questionnaires is specific rubrics to get some help. Even that they can ask about anything that is relevant to all the projects that will be submitted But when students encounter difficult problems which are not in the existed topics, and they want to get special advice on that new field, the demand for creating supplementary review questionnaire raises. This project (E1879) aims to solve this problem by allowing students to add questions to the standard instructor generated rubric so that they can get specific feedback on from the reviewers.

'''Proposed solution'''
----

We will add the Supplementary Review Questions to the current Review Questions, and show these student-generated questions under the rubric given by instructor. Even though these questions won't be graded, they will eventually provide diversity for feedback. It will increase the benefit that each team gets because they can get feedback that is specific to their project.

'''Implementation'''
----

''Design Strategy''

In Expertiza, all kinds of rubrics and surveys are subclasses of Questionnaire. A Questionnaire can contain “questions” of several types (e.g., checkboxes, dropdowns, text boxes). So that we will add a new subclass of Questionnaire named SupplementaryReviewQuestionnaire to achieve our goal.

First, we want the professor to control if the students can create their own supplementary questions by using a checkbox to indicate. If the professor allow students to create their own questions, there will be a button in the students "Your work" page which can let the students jump to the same page that an instructor lands on when creating a new rubric. Then we will add the questionnaire id of this new SupplementaryReviewQuestionnaire to the Team table when students create their own questionnaire in order to show the corresponding supplementary questions to the reviewers. And when showing review questions to the reviewers we will look up the questionnaire id in Team table and modify the view to add the supplementary questions to review page. Finally, after the reviewers finish the review and submit their responds, we will change the view and let the team member can see the feedback of their supplementary questions.

''Changes in the User Interface''

1. Assignment Page

We add a checkbox for the professor to indicate if student can add their own questions.

[[File:Yzhao47_1.png]]

2. A button called "Create Supplementary Rubric" will appear in the student's "Your Work" section when the Instructor has allowed students to create supplementary review questions. So when a student clicks the button he/she can add the desired questions.

[[File:Yzhao47_2.png]]

3. The page where student will be directed to when he/she clicks the Create Supplementary Rubric button. Students can add Questions to the created Supplementary Review questionnaire here.

[[File:Yzhao47_3.png]]

4. Review Page

The students own supplementary questions are leading by a heading "Supplementary Review Questions".

[[File:Yzhao47_4.png]]

5. Review Results Page

The supplementary review results will be showed to the team like other results.

[[File:Yzhao47_5.png]]

'''Use Cases'''
----

1. Use case of creating a Supplementary Review Questionnaire

[[File:case1.jpg]]

2. Use case of reviewing questions for reviewers.

[[File:case2.jpg]]

3. Use case of reviewing responses of questions.

[[File:case3.jpg]]

'''Test Plan'''
----
Actually, we need to add tests for the following:

1. To check the link for "Supplementary Review Questionnaire" appears in the "Your Work" section of a student.

2. To check if the link for "Supplementary Review Questionnaire" redirects to page which allows to create questionnaire.

3. To check if the questionnaire ID is stored in the Team table.

4. To check if the reviewers can see the supplementary questions that were added by the team as part of the review questions.

5. To check if the responses of the Supplementary Review Questions have been added to the responses of the existing review questions.

'''File which are added/modified'''
----

Note: Because the pull request contains some code irrelevant to this project which make it confusing, we post our code here for you to read clearly.

1. Models

app/models/team.rb

Look up the questionnaire ID of a team.

<pre>
def self.get_supplementary_review_questionnaire_id_of_team(team_id)
team = Team.find(team_id)
if team.blank?
nil
else
team.supplementary_review_questionnaire_id
end
end
</pre>

app/models/supplementary_review_questionnaire.rb

Add the SupplementaryReviewQuestionnaire class which is the subclass of Questionnaire.

<pre>

# This is one type of Questionnaire and as intuitively expected this model class
# derives from Questionnaire.
class SupplementaryReviewQuestionnaire < Questionnaire
# Make me better.
attr_accessible
end
</pre>

app/models/response.rb

Add supplementary review responses to "Show Review" section.

<pre>
def construct_review_response code, self_id, show_tags = nil, current_user = nil
code += '<table id="review_' + self_id + '" style="display: none;" class="table table-bordered">'
answers = Answer.where(response_id: self.response_id)
team_id = response_map.reviewee_id
supplementary_review_questionnaire_id = Team.get_supplementary_review_questionnaire_id_of_team(team_id)
unless answers.empty?
questionnaire = self.questionnaire_by_answer(answers.first)

unless supplementary_review_questionnaire_id.nil?
supplementary_review_questionnaire = Questionnaire.find(supplementary_review_questionnaire_id)
supplementary_review_questions = supplementary_review_questionnaire.questions.sort_by(&:seq)
end

questionnaire_max = questionnaire.max_question_score
questions = questionnaire.questions.sort_by(&:seq)

unless supplementary_review_questionnaire.nil?
questions += supplementary_review_questions
end

# get the tag settings this questionnaire
tag_prompt_deployments = show_tags ? TagPromptDeployment.where(questionnaire_id: questionnaire.id, assignment_id: self.map.assignment.id) : nil
code = add_table_rows questionnaire_max, questions, answers, code, tag_prompt_deployments, current_user
end
comment = if !self.additional_comment.nil?
self.additional_comment.gsub('^p', '').gsub(/\n/, '<BR/>')
else
''
end
code += '<tr><td><b>Additional Comment: </b>' + comment + '</td></tr>'
code += '</table>'
end
</pre>

2. Views:

app/views/assignments/edit/_rubrics.html.erb

Add checkbox which professor used to indicate whether students can create their own questions.

<pre>

<input name="assignment_form[assignment][supplementary_review_questionnaire]" type="hidden" value="false"/>
<%= check_box_tag('assignment_form[assignment][supplementary_review_questionnaire]', 'true', @assignment_form.assignment.supplementary_review_questionnaire) %>
<%= label_tag('assignment_form[assignment][supplementary_review_questionnaire]', 'Allow student-generated questions added to rubric?') %>
</pre>

app/views/response/response.html.erb

Add code to show the supplementary questions to the reviewers.

<pre>
<br>
<% unless @supplementary_review_questions.nil? %>
<tr>
<td>
<h2>Supplementary Review Questions</h2>
</td>
</tr>
<% @supplementary_review_questions.each do |question| %>
<tr>
<td>
<% answer = Answer.where(question_id: question.id, response_id: @response.id).first if !@response.nil? %>
<% if question.instance_of? Criterion %>
<%= question.complete(i, answer, @questionnaire.min_question_score, @questionnaire.max_question_score, @dropdown_or_scale) %>
<%= tinymce %>
<% elsif question.instance_of? Scale %>
<%= question.complete(i, answer, @questionnaire.min_question_score, @questionnaire.max_question_score) %>
<%= tinymce %>
<% elsif question.instance_of? UploadFile %>
<% else %>
<%= question.complete(i, answer) %>
<% end %>
</td>
<td width="25%">
<% if [Criterion, TextArea].include? question.class %>
<div id="analysis_<%= i.to_s %>_comments"></div>
<% end %>
</td>
</tr>
<% i += 1 %>
<% end %>
<% end %>
</pre>

app/views/submitted_content/_main.html.erb

Add button for "your work" page to let students create their own questions.

<pre>
<% if stage != "Finished" and controller.controller_name == 'submitted_content' and @can_submit %>
<h2>Add Supplementary Review Questions:</h2>
<% if @assignment.supplementary_review_questionnaire? %>
<%= button_to "Create Supplementary Rubric", {:controller => "questionnaires", :action => "create_supplementary_review_questionnaire", :id => participant.id } %>
<% else %>
The instructor hasn't enabled this feature for this assignment
<% end %>
<% end %>
</pre>

3. Controllers:

app/controllers/response_controller.rb

Add supplementary review questionnaires part to the create, edit, update methods and some other methods which getting the questionnaires.

<pre>
# Prepare the parameters when student clicks "Edit"
def edit
assign_instance_vars
get_all_responses
if @prev.present?
@sorted = @review_scores.sort {|m1, m2| m1.version_num.to_i && m2.version_num.to_i ? m2.version_num.to_i <=> m1.version_num.to_i : (m1.version_num ? -1 : 1) }
@largest_version_num = @sorted[0]
end
@modified_object = @response.response_id
# set more handy variables for the view
set_content
@review_scores = []
@questions.each do |question|
@review_scores << Answer.where(response_id: @response.response_id, question_id: question.id).first
end

@supplementary_review_questions.each do |question|
@review_scores << Answer.where(response_id: @response.response_id, question_id: question.id).first
end

@questionnaire = set_questionnaire
render action: 'response'
end

# Update the response and answers when student "edit" existing response
def update
render nothing: true unless action_allowed?
# the response to be updated
@response = Response.find(params[:id])
msg = ""
begin
@map = @response.map
@response.update_attribute('additional_comment', params[:review][:comments])

set_questionnaire
questions = sort_questions(@questionnaire.questions)
supplementary_review_questions = sort_questions(@supplementary_review_questionnaire.questions)
unless @supplementary_review_questionnaire.nil?
questions += supplementary_review_questions
end
create_answers(params, questions) unless params[:responses].nil?

@response.update_attribute('is_submitted', true) if params['isSubmit'] && params['isSubmit'] == 'Yes'
@response.notify_instructor_on_difference if (@map.is_a? ReviewResponseMap) && @response.is_submitted && @response.significant_difference?
rescue StandardError
msg = "Your response was not saved. Cause:189 #{$ERROR_INFO}"
end
ExpertizaLogger.info LoggerMessage.new(controller_name, session[:user].name, "Your response was submitted: #{@response.is_submitted}", request)
redirect_to controller: 'response', action: 'save', id: @map.map_id,
return: params[:return], msg: msg, review: params[:review], save_options: params[:save_options]
end

def create
map_id = params[:id]
map_id = params[:map_id] unless params[:map_id].nil? # pass map_id as a hidden field in the review form
@map = ResponseMap.find(map_id)
@team_id = @map.reviewee_id

get_all_responses
if params[:review][:questionnaire_id]
@questionnaire = Questionnaire.find(params[:review][:questionnaire_id])
@round = params[:review][:round]
@supplementary_review_questionnaire_id = Team.get_supplementary_review_questionnaire_id_of_team(@team_id)
unless @supplementary_review_questionnaire_id.nil?
@supplementary_review_questionnaire = Questionnaire.find(@supplementary_review_questionnaire_id)
end
else
@round = nil
end
is_submitted = (params[:isSubmit] == 'Yes')
was_submitted = false
@response = Response.where(map_id: @map.id, round: @round.to_i).first
if @response.nil?
@response = Response.create(
map_id: @map.id,
additional_comment: params[:review][:comments],
round: @round.to_i,
is_submitted: is_submitted
)
end
was_submitted = @response.is_submitted
@response.update(additional_comment: params[:review][:comments], is_submitted: is_submitted) # ignore if autoupdate try to save when the response object is not yet created.

# ,:version_num=>@version)
# Change the order for displaying questions for editing response views.
questions = sort_questions(@questionnaire.questions)

unless @supplementary_review_questionnaire.nil?
supplementary_review_questions = sort_questions(@supplementary_review_questionnaire.questions)
questions += supplementary_review_questions
end

create_answers(params, questions) if params[:responses]
msg = "Your response was successfully saved."
error_msg = ""
# only notify if is_submitted changes from false to true
if (@map.is_a? ReviewResponseMap) && (was_submitted == false && @response.is_submitted) && @response.significant_difference?
@response.notify_instructor_on_difference
@response.email
end
redirect_to controller: 'response', action: 'save', id: @map.map_id,
return: params[:return], msg: msg, error_msg: error_msg, review: params[:review], save_options: params[:save_options]
end

# new_response if a flag parameter indicating that if user is requesting a new rubric to fill
# if true: we figure out which questionnaire to use based on current time and records in assignment_questionnaires table
# e.g. student click "Begin" or "Update" to start filling out a rubric for others' work
# if false: we figure out which questionnaire to display base on @response object
# e.g. student click "Edit" or "View"
def set_content(new_response = false)
@title = @map.get_title
if @map.survey?
@survey_parent = @map.survey_parent
else
@assignment = @map.assignment
end
@participant = @map.reviewer
@contributor = @map.contributor
new_response ? set_questionnaire_for_new_response : set_questionnaire
set_dropdown_or_scale
@questions = sort_questions(@questionnaire.questions)

unless @supplementary_review_questionnaire.nil?
@supplementary_review_questions = sort_questions(@supplementary_review_questionnaire.questions)
end

@min = @questionnaire.min_question_score
@max = @questionnaire.max_question_score
end

def set_questionnaire_for_new_response
case @map.type
when "ReviewResponseMap", "SelfReviewResponseMap"
reviewees_topic = SignedUpTeam.topic_id_by_team_id(@contributor.id)
@current_round = @assignment.number_of_current_round(reviewees_topic)
@questionnaire = @map.questionnaire(@current_round)

@supplementary_review_questionnaire_id = Team.get_supplementary_review_questionnaire_id_of_team(@contributor.id)
unless @supplementary_review_questionnaire_id.nil?
@supplementary_review_questionnaire = Questionnaire.find(@supplementary_review_questionnaire_id)
end

when
"MetareviewResponseMap",
"TeammateReviewResponseMap",
"FeedbackResponseMap",
"CourseSurveyResponseMap",
"AssignmentSurveyResponseMap",
"GlobalSurveyResponseMap"
@questionnaire = @map.questionnaire
end
end

def set_questionnaire
# if user is not filling a new rubric, the @response object should be available.
# we can find the questionnaire from the question_id in answers
answer = @response.scores.first
@questionnaire = @response.questionnaire_by_answer(answer)

supplementary_review_answer = @response.scores.last
@supplementary_review_questionnaire = @response.questionnaire_by_answer(supplementary_review_answer)

end

</pre>

app/controllers/grades_controller.rb

Add supplementary review responses to "Your Scores" section.

<pre>
def view_team
@participant = AssignmentParticipant.find(params[:id])
@assignment = @participant.assignment
@team = @participant.team
@team_id = @team.id
@questions = {}
questionnaires = @assignment.questionnaires
retrieve_questions questionnaires
@pscore = @participant.scores(@questions)
@vmlist = []

# loop through each questionnaire, and populate the view model for all data necessary
# to render the html tables.
counter_for_same_rubric = 0
questionnaires.each do |questionnaire|
@round = nil
if @assignment.varying_rubrics_by_round? && questionnaire.type == "ReviewQuestionnaire"
questionnaires = AssignmentQuestionnaire.where(assignment_id: @assignment.id, questionnaire_id: questionnaire.id)
if questionnaires.count > 1
@round = questionnaires[counter_for_same_rubric].used_in_round
counter_for_same_rubric += 1
else
@round = questionnaires[0].used_in_round
counter_for_same_rubric = 0
end
end
vm = VmQuestionResponse.new(questionnaire, @assignment, @round)
supplementary_review_questionnaire_id = Team.get_supplementary_review_questionnaire_id_of_team(@team_id)
unless supplementary_review_questionnaire_id.nil?
supplementary_review_questionnaire = Questionnaire.find(supplementary_review_questionnaire_id)
unless supplementary_review_questionnaire.nil?
supplementary_review_questions = supplementary_review_questionnaire.questions
questionnaire.questions += supplementary_review_questions
end
end
vmquestions = questionnaire.questions
vm.add_questions(vmquestions)
vm.add_team_members(@team)
vm.add_reviews(@participant, @team, @assignment.varying_rubrics_by_round?)
vm.get_number_of_comments_greater_than_10_words
@vmlist << vm
end
@current_role_name = current_role_name
end
</pre>

app/controllers/questionnaires_controller.rb

Method for creating Supplementary Review Questionnaire.

<pre>
def create_supplementary_review_questionnaire
@participant = AssignmentParticipant.find(params[:id])
@team = Team.find(@participant.team.id)
if @team.supplementary_review_questionnaire_id.nil?
@questionnaire = Questionnaire.create(privete: false, name: "supplementary_review_questionnaire_" + @team.id.to_s,
instructor_id: @team_id, min_question_score: 0, max_question_score: 5, type: "Questionnaire", display_type: "Review"
instruction_loc: Questionnaire::DEFAULT_QUESTIONNAIRE_URL)
if @questionnaire.save
@team.supplementary_review_questionnaire_id = @questionnaire.id
@team.save
flash[:success] = 'You have successfully created a rubric!'
else
flash[:error] = $ERROR_INFO
end
else
@questionnaire = Questionnaire.find(@team.supplementary_review_questionnaire_id)
end
redirect_to controller: 'questionnaires', action: 'edit', id: @questionnaire.id
end
</pre>

4. Config:

routes.rb
<pre>

resources :questionnaires, only: %i[new create edit update] do
collection do
get :copy
get :list
post :list_questionnaires
get :new_quiz
get :select_questionnaire_type
post :select_questionnaire_type
get :toggle_access
get :view
post :create_quiz_questionnaire
post :update_quiz
post :add_new_questions
post :save_all_questions
post :create_supplementary_review_questionnaire
end
end

resources :author_feedback_questionnaires, controller: :questionnaires
resources :review_questionnaires, controller: :questionnaires
resources :metareview_questionnaires, controller: :questionnaires
resources :teammate_review_questionnaires, controller: :questionnaires
resources :survey_questionnaires, controller: :questionnaires
resources :assignment_survey_questionnaires, controller: :questionnaires
resources :global_survey_questionnaires, controller: :questionnaires
resources :course_survey_questionnaires, controller: :questionnaires
resources :bookmarkrating_questionnaires, controller: :questionnaires
resources :supplementary_review_questionnaire, controller: :questionnaires
</pre>

'''Team members'''
----
Yanchen Zhao

Pingping Chen

Jianshu Zhang

Zirun Han

'''Links'''
----
Expertiza on Github:https://github.com/zyczyh/expertiza

Pull Request:https://github.com/expertiza/expertiza/pull/1325

Expertiza YouTube channel:https://youtu.be/3PUNknSbU-k

CSC/ECE 517 Fall 2018/E1879 Student Generated Questions Added To Rubric

2018-12-18T20:28:50Z

Zhan23:

'''Introduction'''
----
The rubrics in expertiza are created by instructors. These Rubrics only contain questions that are related to the existed topics. Now, Expertiza only supports students to pull questionnaires is specific rubrics to get some help. Even that they can ask about anything that is relevant to all the projects that will be submitted But when students encounter difficult problems which are not in the existed topics, and they want to get special advice on that new field, the demand for creating supplementary review questionnaire raises. This project (E1879) aims to solve this problem by allowing students to add questions to the standard instructor generated rubric so that they can get specific feedback on from the reviewers.

'''Proposed solution'''
----

We will add the Supplementary Review Questions to the current Review Questions, and show these student-generated questions under the rubric given by instructor. Even though these questions won't be graded, they will eventually provide diversity for feedback. It will increase the benefit that each team gets because they can get feedback that is specific to their project.

'''Implementation'''
----

''Design Strategy''

In Expertiza, all kinds of rubrics and surveys are subclasses of Questionnaire. A Questionnaire can contain “questions” of several types (e.g., checkboxes, dropdowns, text boxes). So that we will add a new subclass of Questionnaire named SupplementaryReviewQuestionnaire to achieve our goal.

First, we want the professor to control if the students can create their own supplementary questions by using a checkbox to indicate. If the professor allow students to create their own questions, there will be a button in the students "Your work" page which can let the students jump to the same page that an instructor lands on when creating a new rubric. Then we will add the questionnaire id of this new SupplementaryReviewQuestionnaire to the Team table when students create their own questionnaire in order to show the corresponding supplementary questions to the reviewers. And when showing review questions to the reviewers we will look up the questionnaire id in Team table and modify the view to add the supplementary questions to review page. Finally, after the reviewers finish the review and submit their responds, we will change the view and let the team member can see the feedback of their supplementary questions.

''Changes in the User Interface''

1. Assignment Page

We add a checkbox for the professor to indicate if student can add their own questions.

[[File:Yzhao47_1.png]]

2. A button called "Create Supplementary Rubric" will appear in the student's "Your Work" section when the Instructor has allowed students to create supplementary review questions. So when a student clicks the button he/she can add the desired questions.

[[File:Yzhao47_2.png]]

3. The page where student will be directed to when he/she clicks the Create Supplementary Rubric button. Students can add Questions to the created Supplementary Review questionnaire here.

[[File:Yzhao47_3.png]]

4. Review Page

The students own supplementary questions are leading by a heading "Supplementary Review Questions".

[[File:Yzhao47_4.png]]

5. Review Results Page

The supplementary review results will be showed to the team like other results.

[[File:Yzhao47_5.png]]

'''Use Cases'''
----

1. Use case of creating a Supplementary Review Questionnaire

[[File:case1.jpg]]

2. Use case of reviewing questions for reviewers.

[[File:case2.jpg]]

3. Use case of reviewing responses of questions.

[[File:case3.jpg]]

'''Test Plan'''
----
Actually, we need to add tests for the following:

1. To check the link for "Supplementary Review Questionnaire" appears in the "Your Work" section of a student.

2. To check if the link for "Supplementary Review Questionnaire" redirects to page which allows to create questionnaire.

3. To check if the questionnaire ID is stored in the Team table.

4. To check if the reviewers can see the supplementary questions that were added by the team as part of the review questions.

5. To check if the responses of the Supplementary Review Questions have been added to the responses of the existing review questions.

'''File which are added/modified'''
----

1. Models

app/models/team.rb

Look up the questionnaire ID of a team.

<pre>
def self.get_supplementary_review_questionnaire_id_of_team(team_id)
team = Team.find(team_id)
if team.blank?
nil
else
team.supplementary_review_questionnaire_id
end
end
</pre>

app/models/supplementary_review_questionnaire.rb

Add the SupplementaryReviewQuestionnaire class which is the subclass of Questionnaire.

<pre>

# This is one type of Questionnaire and as intuitively expected this model class
# derives from Questionnaire.
class SupplementaryReviewQuestionnaire < Questionnaire
# Make me better.
attr_accessible
end
</pre>

app/models/response.rb

Add supplementary review responses to "Show Review" section.

<pre>
def construct_review_response code, self_id, show_tags = nil, current_user = nil
code += '<table id="review_' + self_id + '" style="display: none;" class="table table-bordered">'
answers = Answer.where(response_id: self.response_id)
team_id = response_map.reviewee_id
supplementary_review_questionnaire_id = Team.get_supplementary_review_questionnaire_id_of_team(team_id)
unless answers.empty?
questionnaire = self.questionnaire_by_answer(answers.first)

unless supplementary_review_questionnaire_id.nil?
supplementary_review_questionnaire = Questionnaire.find(supplementary_review_questionnaire_id)
supplementary_review_questions = supplementary_review_questionnaire.questions.sort_by(&:seq)
end

questionnaire_max = questionnaire.max_question_score
questions = questionnaire.questions.sort_by(&:seq)

unless supplementary_review_questionnaire.nil?
questions += supplementary_review_questions
end

# get the tag settings this questionnaire
tag_prompt_deployments = show_tags ? TagPromptDeployment.where(questionnaire_id: questionnaire.id, assignment_id: self.map.assignment.id) : nil
code = add_table_rows questionnaire_max, questions, answers, code, tag_prompt_deployments, current_user
end
comment = if !self.additional_comment.nil?
self.additional_comment.gsub('^p', '').gsub(/\n/, '<BR/>')
else
''
end
code += '<tr><td><b>Additional Comment: </b>' + comment + '</td></tr>'
code += '</table>'
end
</pre>

2. Views:

app/views/assignments/edit/_rubrics.html.erb

Add checkbox which professor used to indicate whether students can create their own questions.

<pre>

<input name="assignment_form[assignment][supplementary_review_questionnaire]" type="hidden" value="false"/>
<%= check_box_tag('assignment_form[assignment][supplementary_review_questionnaire]', 'true', @assignment_form.assignment.supplementary_review_questionnaire) %>
<%= label_tag('assignment_form[assignment][supplementary_review_questionnaire]', 'Allow student-generated questions added to rubric?') %>
</pre>

app/views/response/response.html.erb

Add code to show the supplementary questions to the reviewers.

<pre>
<br>
<% unless @supplementary_review_questions.nil? %>
<tr>
<td>
<h2>Supplementary Review Questions</h2>
</td>
</tr>
<% @supplementary_review_questions.each do |question| %>
<tr>
<td>
<% answer = Answer.where(question_id: question.id, response_id: @response.id).first if !@response.nil? %>
<% if question.instance_of? Criterion %>
<%= question.complete(i, answer, @questionnaire.min_question_score, @questionnaire.max_question_score, @dropdown_or_scale) %>
<%= tinymce %>
<% elsif question.instance_of? Scale %>
<%= question.complete(i, answer, @questionnaire.min_question_score, @questionnaire.max_question_score) %>
<%= tinymce %>
<% elsif question.instance_of? UploadFile %>
<% else %>
<%= question.complete(i, answer) %>
<% end %>
</td>
<td width="25%">
<% if [Criterion, TextArea].include? question.class %>
<div id="analysis_<%= i.to_s %>_comments"></div>
<% end %>
</td>
</tr>
<% i += 1 %>
<% end %>
<% end %>
</pre>

app/views/submitted_content/_main.html.erb

Add button for "your work" page to let students create their own questions.

<pre>
<% if stage != "Finished" and controller.controller_name == 'submitted_content' and @can_submit %>
<h2>Add Supplementary Review Questions:</h2>
<% if @assignment.supplementary_review_questionnaire? %>
<%= button_to "Create Supplementary Rubric", {:controller => "questionnaires", :action => "create_supplementary_review_questionnaire", :id => participant.id } %>
<% else %>
The instructor hasn't enabled this feature for this assignment
<% end %>
<% end %>
</pre>

3. Controllers:

app/controllers/response_controller.rb

Add supplementary review questionnaires part to the create, edit, update methods and some other methods which getting the questionnaires.

<pre>
# Prepare the parameters when student clicks "Edit"
def edit
assign_instance_vars
get_all_responses
if @prev.present?
@sorted = @review_scores.sort {|m1, m2| m1.version_num.to_i && m2.version_num.to_i ? m2.version_num.to_i <=> m1.version_num.to_i : (m1.version_num ? -1 : 1) }
@largest_version_num = @sorted[0]
end
@modified_object = @response.response_id
# set more handy variables for the view
set_content
@review_scores = []
@questions.each do |question|
@review_scores << Answer.where(response_id: @response.response_id, question_id: question.id).first
end

@supplementary_review_questions.each do |question|
@review_scores << Answer.where(response_id: @response.response_id, question_id: question.id).first
end

@questionnaire = set_questionnaire
render action: 'response'
end

# Update the response and answers when student "edit" existing response
def update
render nothing: true unless action_allowed?
# the response to be updated
@response = Response.find(params[:id])
msg = ""
begin
@map = @response.map
@response.update_attribute('additional_comment', params[:review][:comments])

set_questionnaire
questions = sort_questions(@questionnaire.questions)
supplementary_review_questions = sort_questions(@supplementary_review_questionnaire.questions)
unless @supplementary_review_questionnaire.nil?
questions += supplementary_review_questions
end
create_answers(params, questions) unless params[:responses].nil?

@response.update_attribute('is_submitted', true) if params['isSubmit'] && params['isSubmit'] == 'Yes'
@response.notify_instructor_on_difference if (@map.is_a? ReviewResponseMap) && @response.is_submitted && @response.significant_difference?
rescue StandardError
msg = "Your response was not saved. Cause:189 #{$ERROR_INFO}"
end
ExpertizaLogger.info LoggerMessage.new(controller_name, session[:user].name, "Your response was submitted: #{@response.is_submitted}", request)
redirect_to controller: 'response', action: 'save', id: @map.map_id,
return: params[:return], msg: msg, review: params[:review], save_options: params[:save_options]
end

def create
map_id = params[:id]
map_id = params[:map_id] unless params[:map_id].nil? # pass map_id as a hidden field in the review form
@map = ResponseMap.find(map_id)
@team_id = @map.reviewee_id

get_all_responses
if params[:review][:questionnaire_id]
@questionnaire = Questionnaire.find(params[:review][:questionnaire_id])
@round = params[:review][:round]
@supplementary_review_questionnaire_id = Team.get_supplementary_review_questionnaire_id_of_team(@team_id)
unless @supplementary_review_questionnaire_id.nil?
@supplementary_review_questionnaire = Questionnaire.find(@supplementary_review_questionnaire_id)
end
else
@round = nil
end
is_submitted = (params[:isSubmit] == 'Yes')
was_submitted = false
@response = Response.where(map_id: @map.id, round: @round.to_i).first
if @response.nil?
@response = Response.create(
map_id: @map.id,
additional_comment: params[:review][:comments],
round: @round.to_i,
is_submitted: is_submitted
)
end
was_submitted = @response.is_submitted
@response.update(additional_comment: params[:review][:comments], is_submitted: is_submitted) # ignore if autoupdate try to save when the response object is not yet created.

# ,:version_num=>@version)
# Change the order for displaying questions for editing response views.
questions = sort_questions(@questionnaire.questions)

unless @supplementary_review_questionnaire.nil?
supplementary_review_questions = sort_questions(@supplementary_review_questionnaire.questions)
questions += supplementary_review_questions
end

create_answers(params, questions) if params[:responses]
msg = "Your response was successfully saved."
error_msg = ""
# only notify if is_submitted changes from false to true
if (@map.is_a? ReviewResponseMap) && (was_submitted == false && @response.is_submitted) && @response.significant_difference?
@response.notify_instructor_on_difference
@response.email
end
redirect_to controller: 'response', action: 'save', id: @map.map_id,
return: params[:return], msg: msg, error_msg: error_msg, review: params[:review], save_options: params[:save_options]
end

# new_response if a flag parameter indicating that if user is requesting a new rubric to fill
# if true: we figure out which questionnaire to use based on current time and records in assignment_questionnaires table
# e.g. student click "Begin" or "Update" to start filling out a rubric for others' work
# if false: we figure out which questionnaire to display base on @response object
# e.g. student click "Edit" or "View"
def set_content(new_response = false)
@title = @map.get_title
if @map.survey?
@survey_parent = @map.survey_parent
else
@assignment = @map.assignment
end
@participant = @map.reviewer
@contributor = @map.contributor
new_response ? set_questionnaire_for_new_response : set_questionnaire
set_dropdown_or_scale
@questions = sort_questions(@questionnaire.questions)

unless @supplementary_review_questionnaire.nil?
@supplementary_review_questions = sort_questions(@supplementary_review_questionnaire.questions)
end

@min = @questionnaire.min_question_score
@max = @questionnaire.max_question_score
end

def set_questionnaire_for_new_response
case @map.type
when "ReviewResponseMap", "SelfReviewResponseMap"
reviewees_topic = SignedUpTeam.topic_id_by_team_id(@contributor.id)
@current_round = @assignment.number_of_current_round(reviewees_topic)
@questionnaire = @map.questionnaire(@current_round)

@supplementary_review_questionnaire_id = Team.get_supplementary_review_questionnaire_id_of_team(@contributor.id)
unless @supplementary_review_questionnaire_id.nil?
@supplementary_review_questionnaire = Questionnaire.find(@supplementary_review_questionnaire_id)
end

when
"MetareviewResponseMap",
"TeammateReviewResponseMap",
"FeedbackResponseMap",
"CourseSurveyResponseMap",
"AssignmentSurveyResponseMap",
"GlobalSurveyResponseMap"
@questionnaire = @map.questionnaire
end
end

def set_questionnaire
# if user is not filling a new rubric, the @response object should be available.
# we can find the questionnaire from the question_id in answers
answer = @response.scores.first
@questionnaire = @response.questionnaire_by_answer(answer)

supplementary_review_answer = @response.scores.last
@supplementary_review_questionnaire = @response.questionnaire_by_answer(supplementary_review_answer)

end

</pre>

app/controllers/grades_controller.rb

Add supplementary review responses to "Your Scores" section.

<pre>
def view_team
@participant = AssignmentParticipant.find(params[:id])
@assignment = @participant.assignment
@team = @participant.team
@team_id = @team.id
@questions = {}
questionnaires = @assignment.questionnaires
retrieve_questions questionnaires
@pscore = @participant.scores(@questions)
@vmlist = []

# loop through each questionnaire, and populate the view model for all data necessary
# to render the html tables.
counter_for_same_rubric = 0
questionnaires.each do |questionnaire|
@round = nil
if @assignment.varying_rubrics_by_round? && questionnaire.type == "ReviewQuestionnaire"
questionnaires = AssignmentQuestionnaire.where(assignment_id: @assignment.id, questionnaire_id: questionnaire.id)
if questionnaires.count > 1
@round = questionnaires[counter_for_same_rubric].used_in_round
counter_for_same_rubric += 1
else
@round = questionnaires[0].used_in_round
counter_for_same_rubric = 0
end
end
vm = VmQuestionResponse.new(questionnaire, @assignment, @round)
supplementary_review_questionnaire_id = Team.get_supplementary_review_questionnaire_id_of_team(@team_id)
unless supplementary_review_questionnaire_id.nil?
supplementary_review_questionnaire = Questionnaire.find(supplementary_review_questionnaire_id)
unless supplementary_review_questionnaire.nil?
supplementary_review_questions = supplementary_review_questionnaire.questions
questionnaire.questions += supplementary_review_questions
end
end
vmquestions = questionnaire.questions
vm.add_questions(vmquestions)
vm.add_team_members(@team)
vm.add_reviews(@participant, @team, @assignment.varying_rubrics_by_round?)
vm.get_number_of_comments_greater_than_10_words
@vmlist << vm
end
@current_role_name = current_role_name
end
</pre>

app/controllers/questionnaires_controller.rb

Method for creating Supplementary Review Questionnaire.

<pre>
def create_supplementary_review_questionnaire
@participant = AssignmentParticipant.find(params[:id])
@team = Team.find(@participant.team.id)
if @team.supplementary_review_questionnaire_id.nil?
@questionnaire = Questionnaire.create(privete: false, name: "supplementary_review_questionnaire_" + @team.id.to_s,
instructor_id: @team_id, min_question_score: 0, max_question_score: 5, type: "Questionnaire", display_type: "Review"
instruction_loc: Questionnaire::DEFAULT_QUESTIONNAIRE_URL)
if @questionnaire.save
@team.supplementary_review_questionnaire_id = @questionnaire.id
@team.save
flash[:success] = 'You have successfully created a rubric!'
else
flash[:error] = $ERROR_INFO
end
else
@questionnaire = Questionnaire.find(@team.supplementary_review_questionnaire_id)
end
redirect_to controller: 'questionnaires', action: 'edit', id: @questionnaire.id
end
</pre>

4. Config:

routes.rb
<pre>

resources :questionnaires, only: %i[new create edit update] do
collection do
get :copy
get :list
post :list_questionnaires
get :new_quiz
get :select_questionnaire_type
post :select_questionnaire_type
get :toggle_access
get :view
post :create_quiz_questionnaire
post :update_quiz
post :add_new_questions
post :save_all_questions
post :create_supplementary_review_questionnaire
end
end

resources :author_feedback_questionnaires, controller: :questionnaires
resources :review_questionnaires, controller: :questionnaires
resources :metareview_questionnaires, controller: :questionnaires
resources :teammate_review_questionnaires, controller: :questionnaires
resources :survey_questionnaires, controller: :questionnaires
resources :assignment_survey_questionnaires, controller: :questionnaires
resources :global_survey_questionnaires, controller: :questionnaires
resources :course_survey_questionnaires, controller: :questionnaires
resources :bookmarkrating_questionnaires, controller: :questionnaires
resources :supplementary_review_questionnaire, controller: :questionnaires
</pre>

'''Team members'''
----
Yanchen Zhao

Pingping Chen

Jianshu Zhang

Zirun Han

'''Links'''
----
Expertiza on Github:https://github.com/zyczyh/expertiza

Pull Request:https://github.com/expertiza/expertiza/pull/1325

Expertiza YouTube channel:https://youtu.be/3PUNknSbU-k

CSC/ECE 517 Fall 2018/E1879 Student Generated Questions Added To Rubric

2018-12-18T20:27:08Z

Zhan23:

'''Introduction'''
----
The rubrics in expertiza are created by instructors. These Rubrics only contain questions that are related to the existed topics. Now, Expertiza only supports students to pull questionnaires is specific rubrics to get some help. Even that they can ask about anything that is relevant to all the projects that will be submitted But when students encounter difficult problems which are not in the existed topics, and they want to get special advice on that new field, the demand for creating supplementary review questionnaire raises. This project (E1879) aims to solve this problem by allowing students to add questions to the standard instructor generated rubric so that they can get specific feedback on from the reviewers.

'''Proposed solution'''
----

We will add the Supplementary Review Questions to the current Review Questions, and show these student-generated questions under the rubric given by instructor. Even though these questions won't be graded, they will eventually provide diversity for feedback. It will increase the benefit that each team gets because they can get feedback that is specific to their project.

'''Implementation'''
----

''Design Strategy''

In Expertiza, all kinds of rubrics and surveys are subclasses of Questionnaire. A Questionnaire can contain “questions” of several types (e.g., checkboxes, dropdowns, text boxes). So that we will add a new subclass of Questionnaire named SupplementaryReviewQuestionnaire to achieve our goal.

First, we want the professor to control if the students can create their own supplementary questions by using a checkbox to indicate. If the professor allow students to create their own questions, there will be a button in the students "Your work" page which can let the students jump to the same page that an instructor lands on when creating a new rubric. Then we will add the questionnaire id of this new SupplementaryReviewQuestionnaire to the Team table when students create their own questionnaire in order to show the corresponding supplementary questions to the reviewers. And when showing review questions to the reviewers we will look up the questionnaire id in Team table and modify the view to add the supplementary questions to review page. Finally, after the reviewers finish the review and submit their responds, we will change the view and let the team member can see the feedback of their supplementary questions.

''Changes in the User Interface''

1. Assignment Page

We add a checkbox for the professor to indicate if student can add their own questions.

[[File:Yzhao47_1.png]]

2. A button called "Create Supplementary Rubric" will appear in the student's "Your Work" section when the Instructor has allowed students to create supplementary review questions. So when a student clicks the button he/she can add the desired questions.

[[File:Yzhao47_2.png]]

3. The page where student will be directed to when he/she clicks the Create Supplementary Rubric button. Students can add Questions to the created Supplementary Review questionnaire here.

[[File:Yzhao47_3.png]]

4. Review Page

The students own supplementary questions are leading by a heading "Supplementary Review Questions".

[[File:Yzhao47_4.png]]

5. Review Results Page

The supplementary review results will be showed to the team like other results.

[[File:Yzhao47_5.png]]

'''Use Cases'''
----

1. Use case of creating a Supplementary Review Questionnaire

[[File:case1.jpg]]

2. Use case of reviewing questions for reviewers.

[[File:case2.jpg]]

3. Use case of reviewing responses of questions.

[[File:case3.jpg]]

'''Test Plan'''
----
Actually, we need to add tests for the following:

1. To check the link for "Supplementary Review Questionnaire" appears in the "Your Work" section of a student.

2. To check if the link for "Supplementary Review Questionnaire" redirects to page which allows to create questionnaire.

3. To check if the questionnaire ID is stored in the Team table.

4. To check if the reviewers can see the supplementary questions that were added by the team as part of the review questions.

5. To check if the responses of the Supplementary Review Questions have been added to the responses of the existing review questions.

'''File which are added/modified'''
----

1. Models

app/models/team.rb

Look up the questionnaire ID of a team.

<pre>
def self.get_supplementary_review_questionnaire_id_of_team(team_id)
team = Team.find(team_id)
if team.blank?
nil
else
team.supplementary_review_questionnaire_id
end
end
</pre>

app/models/supplementary_review_questionnaire.rb

Add the SupplementaryReviewQuestionnaire class which is the subclass of Questionnaire.

<pre>

# This is one type of Questionnaire and as intuitively expected this model class
# derives from Questionnaire.
class SupplementaryReviewQuestionnaire < Questionnaire
# Make me better.
attr_accessible
end
</pre>

app/models/response.rb

Added supplementary review responses to "Show Review" section.

<pre>
def construct_review_response code, self_id, show_tags = nil, current_user = nil
code += '<table id="review_' + self_id + '" style="display: none;" class="table table-bordered">'
answers = Answer.where(response_id: self.response_id)
team_id = response_map.reviewee_id
supplementary_review_questionnaire_id = Team.get_supplementary_review_questionnaire_id_of_team(team_id)
unless answers.empty?
questionnaire = self.questionnaire_by_answer(answers.first)

unless supplementary_review_questionnaire_id.nil?
supplementary_review_questionnaire = Questionnaire.find(supplementary_review_questionnaire_id)
supplementary_review_questions = supplementary_review_questionnaire.questions.sort_by(&:seq)
end

questionnaire_max = questionnaire.max_question_score
questions = questionnaire.questions.sort_by(&:seq)

unless supplementary_review_questionnaire.nil?
questions += supplementary_review_questions
end

# get the tag settings this questionnaire
tag_prompt_deployments = show_tags ? TagPromptDeployment.where(questionnaire_id: questionnaire.id, assignment_id: self.map.assignment.id) : nil
code = add_table_rows questionnaire_max, questions, answers, code, tag_prompt_deployments, current_user
end
comment = if !self.additional_comment.nil?
self.additional_comment.gsub('^p', '').gsub(/\n/, '<BR/>')
else
''
end
code += '<tr><td><b>Additional Comment: </b>' + comment + '</td></tr>'
code += '</table>'
end
</pre>

2. Views:

app/views/assignments/edit/_rubrics.html.erb

Add checkbox which professor used to indicate whether students can create their own questions.

<pre>

<input name="assignment_form[assignment][supplementary_review_questionnaire]" type="hidden" value="false"/>
<%= check_box_tag('assignment_form[assignment][supplementary_review_questionnaire]', 'true', @assignment_form.assignment.supplementary_review_questionnaire) %>
<%= label_tag('assignment_form[assignment][supplementary_review_questionnaire]', 'Allow student-generated questions added to rubric?') %>
</pre>

app/views/response/response.html.erb

Add code to show the supplementary questions to the reviewers.

<pre>
<br>
<% unless @supplementary_review_questions.nil? %>
<tr>
<td>
<h2>Supplementary Review Questions</h2>
</td>
</tr>
<% @supplementary_review_questions.each do |question| %>
<tr>
<td>
<% answer = Answer.where(question_id: question.id, response_id: @response.id).first if !@response.nil? %>
<% if question.instance_of? Criterion %>
<%= question.complete(i, answer, @questionnaire.min_question_score, @questionnaire.max_question_score, @dropdown_or_scale) %>
<%= tinymce %>
<% elsif question.instance_of? Scale %>
<%= question.complete(i, answer, @questionnaire.min_question_score, @questionnaire.max_question_score) %>
<%= tinymce %>
<% elsif question.instance_of? UploadFile %>
<% else %>
<%= question.complete(i, answer) %>
<% end %>
</td>
<td width="25%">
<% if [Criterion, TextArea].include? question.class %>
<div id="analysis_<%= i.to_s %>_comments"></div>
<% end %>
</td>
</tr>
<% i += 1 %>
<% end %>
<% end %>
</pre>

app/views/submitted_content/_main.html.erb

Add button for "your work" page to let students create their own questions.

<pre>
<% if stage != "Finished" and controller.controller_name == 'submitted_content' and @can_submit %>
<h2>Add Supplementary Review Questions:</h2>
<% if @assignment.supplementary_review_questionnaire? %>
<%= button_to "Create Supplementary Rubric", {:controller => "questionnaires", :action => "create_supplementary_review_questionnaire", :id => participant.id } %>
<% else %>
The instructor hasn't enabled this feature for this assignment
<% end %>
<% end %>
</pre>

3. Controllers:

app/controllers/response_controller.rb

Add supplementary review questionnaires part to the create, edit, update methods and some other methods which getting the questionnaires.

<pre>
# Prepare the parameters when student clicks "Edit"
def edit
assign_instance_vars
get_all_responses
if @prev.present?
@sorted = @review_scores.sort {|m1, m2| m1.version_num.to_i && m2.version_num.to_i ? m2.version_num.to_i <=> m1.version_num.to_i : (m1.version_num ? -1 : 1) }
@largest_version_num = @sorted[0]
end
@modified_object = @response.response_id
# set more handy variables for the view
set_content
@review_scores = []
@questions.each do |question|
@review_scores << Answer.where(response_id: @response.response_id, question_id: question.id).first
end

@supplementary_review_questions.each do |question|
@review_scores << Answer.where(response_id: @response.response_id, question_id: question.id).first
end

@questionnaire = set_questionnaire
render action: 'response'
end

# Update the response and answers when student "edit" existing response
def update
render nothing: true unless action_allowed?
# the response to be updated
@response = Response.find(params[:id])
msg = ""
begin
@map = @response.map
@response.update_attribute('additional_comment', params[:review][:comments])

set_questionnaire
questions = sort_questions(@questionnaire.questions)
supplementary_review_questions = sort_questions(@supplementary_review_questionnaire.questions)
unless @supplementary_review_questionnaire.nil?
questions += supplementary_review_questions
end
create_answers(params, questions) unless params[:responses].nil?

@response.update_attribute('is_submitted', true) if params['isSubmit'] && params['isSubmit'] == 'Yes'
@response.notify_instructor_on_difference if (@map.is_a? ReviewResponseMap) && @response.is_submitted && @response.significant_difference?
rescue StandardError
msg = "Your response was not saved. Cause:189 #{$ERROR_INFO}"
end
ExpertizaLogger.info LoggerMessage.new(controller_name, session[:user].name, "Your response was submitted: #{@response.is_submitted}", request)
redirect_to controller: 'response', action: 'save', id: @map.map_id,
return: params[:return], msg: msg, review: params[:review], save_options: params[:save_options]
end

def create
map_id = params[:id]
map_id = params[:map_id] unless params[:map_id].nil? # pass map_id as a hidden field in the review form
@map = ResponseMap.find(map_id)
@team_id = @map.reviewee_id

get_all_responses
if params[:review][:questionnaire_id]
@questionnaire = Questionnaire.find(params[:review][:questionnaire_id])
@round = params[:review][:round]
@supplementary_review_questionnaire_id = Team.get_supplementary_review_questionnaire_id_of_team(@team_id)
unless @supplementary_review_questionnaire_id.nil?
@supplementary_review_questionnaire = Questionnaire.find(@supplementary_review_questionnaire_id)
end
else
@round = nil
end
is_submitted = (params[:isSubmit] == 'Yes')
was_submitted = false
@response = Response.where(map_id: @map.id, round: @round.to_i).first
if @response.nil?
@response = Response.create(
map_id: @map.id,
additional_comment: params[:review][:comments],
round: @round.to_i,
is_submitted: is_submitted
)
end
was_submitted = @response.is_submitted
@response.update(additional_comment: params[:review][:comments], is_submitted: is_submitted) # ignore if autoupdate try to save when the response object is not yet created.

# ,:version_num=>@version)
# Change the order for displaying questions for editing response views.
questions = sort_questions(@questionnaire.questions)

unless @supplementary_review_questionnaire.nil?
supplementary_review_questions = sort_questions(@supplementary_review_questionnaire.questions)
questions += supplementary_review_questions
end

create_answers(params, questions) if params[:responses]
msg = "Your response was successfully saved."
error_msg = ""
# only notify if is_submitted changes from false to true
if (@map.is_a? ReviewResponseMap) && (was_submitted == false && @response.is_submitted) && @response.significant_difference?
@response.notify_instructor_on_difference
@response.email
end
redirect_to controller: 'response', action: 'save', id: @map.map_id,
return: params[:return], msg: msg, error_msg: error_msg, review: params[:review], save_options: params[:save_options]
end

# new_response if a flag parameter indicating that if user is requesting a new rubric to fill
# if true: we figure out which questionnaire to use based on current time and records in assignment_questionnaires table
# e.g. student click "Begin" or "Update" to start filling out a rubric for others' work
# if false: we figure out which questionnaire to display base on @response object
# e.g. student click "Edit" or "View"
def set_content(new_response = false)
@title = @map.get_title
if @map.survey?
@survey_parent = @map.survey_parent
else
@assignment = @map.assignment
end
@participant = @map.reviewer
@contributor = @map.contributor
new_response ? set_questionnaire_for_new_response : set_questionnaire
set_dropdown_or_scale
@questions = sort_questions(@questionnaire.questions)

unless @supplementary_review_questionnaire.nil?
@supplementary_review_questions = sort_questions(@supplementary_review_questionnaire.questions)
end

@min = @questionnaire.min_question_score
@max = @questionnaire.max_question_score
end

def set_questionnaire_for_new_response
case @map.type
when "ReviewResponseMap", "SelfReviewResponseMap"
reviewees_topic = SignedUpTeam.topic_id_by_team_id(@contributor.id)
@current_round = @assignment.number_of_current_round(reviewees_topic)
@questionnaire = @map.questionnaire(@current_round)

@supplementary_review_questionnaire_id = Team.get_supplementary_review_questionnaire_id_of_team(@contributor.id)
unless @supplementary_review_questionnaire_id.nil?
@supplementary_review_questionnaire = Questionnaire.find(@supplementary_review_questionnaire_id)
end

when
"MetareviewResponseMap",
"TeammateReviewResponseMap",
"FeedbackResponseMap",
"CourseSurveyResponseMap",
"AssignmentSurveyResponseMap",
"GlobalSurveyResponseMap"
@questionnaire = @map.questionnaire
end
end

def set_questionnaire
# if user is not filling a new rubric, the @response object should be available.
# we can find the questionnaire from the question_id in answers
answer = @response.scores.first
@questionnaire = @response.questionnaire_by_answer(answer)

supplementary_review_answer = @response.scores.last
@supplementary_review_questionnaire = @response.questionnaire_by_answer(supplementary_review_answer)

end

</pre>

app/controllers/grades_controller.rb

Add supplementary review responses to "Your Scores" section.

<pre>
def view_team
@participant = AssignmentParticipant.find(params[:id])
@assignment = @participant.assignment
@team = @participant.team
@team_id = @team.id
@questions = {}
questionnaires = @assignment.questionnaires
retrieve_questions questionnaires
@pscore = @participant.scores(@questions)
@vmlist = []

# loop through each questionnaire, and populate the view model for all data necessary
# to render the html tables.
counter_for_same_rubric = 0
questionnaires.each do |questionnaire|
@round = nil
if @assignment.varying_rubrics_by_round? && questionnaire.type == "ReviewQuestionnaire"
questionnaires = AssignmentQuestionnaire.where(assignment_id: @assignment.id, questionnaire_id: questionnaire.id)
if questionnaires.count > 1
@round = questionnaires[counter_for_same_rubric].used_in_round
counter_for_same_rubric += 1
else
@round = questionnaires[0].used_in_round
counter_for_same_rubric = 0
end
end
vm = VmQuestionResponse.new(questionnaire, @assignment, @round)
supplementary_review_questionnaire_id = Team.get_supplementary_review_questionnaire_id_of_team(@team_id)
unless supplementary_review_questionnaire_id.nil?
supplementary_review_questionnaire = Questionnaire.find(supplementary_review_questionnaire_id)
unless supplementary_review_questionnaire.nil?
supplementary_review_questions = supplementary_review_questionnaire.questions
questionnaire.questions += supplementary_review_questions
end
end
vmquestions = questionnaire.questions
vm.add_questions(vmquestions)
vm.add_team_members(@team)
vm.add_reviews(@participant, @team, @assignment.varying_rubrics_by_round?)
vm.get_number_of_comments_greater_than_10_words
@vmlist << vm
end
@current_role_name = current_role_name
end
</pre>

app/controllers/questionnaires_controller.rb

Method for creating Supplementary Review Questionnaire.

<pre>
def create_supplementary_review_questionnaire
@participant = AssignmentParticipant.find(params[:id])
@team = Team.find(@participant.team.id)
if @team.supplementary_review_questionnaire_id.nil?
@questionnaire = Questionnaire.create(privete: false, name: "supplementary_review_questionnaire_" + @team.id.to_s,
instructor_id: @team_id, min_question_score: 0, max_question_score: 5, type: "Questionnaire", display_type: "Review"
instruction_loc: Questionnaire::DEFAULT_QUESTIONNAIRE_URL)
if @questionnaire.save
@team.supplementary_review_questionnaire_id = @questionnaire.id
@team.save
flash[:success] = 'You have successfully created a rubric!'
else
flash[:error] = $ERROR_INFO
end
else
@questionnaire = Questionnaire.find(@team.supplementary_review_questionnaire_id)
end
redirect_to controller: 'questionnaires', action: 'edit', id: @questionnaire.id
end
</pre>

4. Config:

routes.rb
<pre>

resources :questionnaires, only: %i[new create edit update] do
collection do
get :copy
get :list
post :list_questionnaires
get :new_quiz
get :select_questionnaire_type
post :select_questionnaire_type
get :toggle_access
get :view
post :create_quiz_questionnaire
post :update_quiz
post :add_new_questions
post :save_all_questions
post :create_supplementary_review_questionnaire
end
end

resources :author_feedback_questionnaires, controller: :questionnaires
resources :review_questionnaires, controller: :questionnaires
resources :metareview_questionnaires, controller: :questionnaires
resources :teammate_review_questionnaires, controller: :questionnaires
resources :survey_questionnaires, controller: :questionnaires
resources :assignment_survey_questionnaires, controller: :questionnaires
resources :global_survey_questionnaires, controller: :questionnaires
resources :course_survey_questionnaires, controller: :questionnaires
resources :bookmarkrating_questionnaires, controller: :questionnaires
resources :supplementary_review_questionnaire, controller: :questionnaires
</pre>

'''Team members'''
----
Yanchen Zhao

Pingping Chen

Jianshu Zhang

Zirun Han

'''Links'''
----
Expertiza on Github:https://github.com/zyczyh/expertiza

Pull Request:https://github.com/expertiza/expertiza/pull/1325

Expertiza YouTube channel:https://youtu.be/3PUNknSbU-k

CSC/ECE 517 Fall 2018/E1879 Student Generated Questions Added To Rubric

2018-12-18T18:38:07Z

Zhan23:

'''Introduction'''
----
The rubrics in expertiza are created by instructors. These Rubrics only contain questions that are related to the existed topics. Now, Expertiza only supports students to pull questionnaires is specific rubrics to get some help. Even that they can ask about anything that is relevant to all the projects that will be submitted But when students encounter difficult problems which are not in the existed topics, and they want to get special advice on that new field, the demand for creating supplementary review questionnaire raises. This project (E1879) aims to solve this problem by allowing students to add questions to the standard instructor generated rubric so that they can get specific feedback on from the reviewers.

'''Proposed solution'''
----

We will add the Supplementary Review Questions to the current Review Questions, and show these student-generated questions under the rubric given by instructor. Even though these questions won't be graded, they will eventually provide diversity for feedback. It will increase the benefit that each team gets because they can get feedback that is specific to their project.

'''Implementation'''
----

''Design Strategy''

In Expertiza, all kinds of rubrics and surveys are subclasses of Questionnaire. A Questionnaire can contain “questions” of several types (e.g., checkboxes, dropdowns, text boxes). So that we will add a new subclass of Questionnaire named SupplementaryReviewQuestionnaire to achieve our goal.

First, we want the professor to control if the students can create their own supplementary questions by using a checkbox to indicate. If the professor allow students to create their own questions, there will be a button in the students "Your work" page which can let the students jump to the same page that an instructor lands on when creating a new rubric. Then we will add the questionnaire id of this new SupplementaryReviewQuestionnaire to the Team table when students create their own questionnaire in order to show the corresponding supplementary questions to the reviewers. And when showing review questions to the reviewers we will look up the questionnaire id in Team table and modify the view to add the supplementary questions to review page. Finally, after the reviewers finish the review and submit their responds, we will change the view and let the team member can see the feedback of their supplementary questions.

''Changes in the User Interface''

1. Assignment Page

We add a checkbox for the professor to indicate if student can add their own questions.

[[File:Yzhao47_1.png]]

2. A button called "Create Supplementary Rubric" will appear in the student's "Your Work" section when the Instructor has allowed students to create supplementary review questions. So when a student clicks the button he/she can add the desired questions.

[[File:Yzhao47_2.png]]

3. The page where student will be directed to when he/she clicks the Create Supplementary Rubric button. Students can add Questions to the created Supplementary Review questionnaire here.

[[File:Yzhao47_3.png]]

4. Review Page

The students own supplementary questions are leading by a heading "Supplementary Review Questions".

[[File:Yzhao47_4.png]]

5. Review Results Page

The supplementary review results will be showed to the team like other results.

[[File:Yzhao47_5.png]]

'''Use Cases'''
----

1. Use case of creating a Supplementary Review Questionnaire

[[File:case1.jpg]]

2. Use case of reviewing questions for reviewers.

[[File:case2.jpg]]

3. Use case of reviewing responses of questions.

[[File:case3.jpg]]

'''Test Plan'''
----
Actually, we need to add tests for the following:

1. To check the link for "Supplementary Review Questionnaire" appears in the "Your Work" section of a student.

2. To check if the link for "Supplementary Review Questionnaire" redirects to page which allows to create questionnaire.

3. To check if the questionnaire ID is stored in the Team table.

4. To check if the reviewers can see the supplementary questions that were added by the team as part of the review questions.

5. To check if the responses of the Supplementary Review Questions have been added to the responses of the existing review questions.

'''File which are added/modified'''
----

1. Models

app/models/team.rb

<pre>
def self.get_supplementary_review_questionnaire_id_of_team(team_id)
team = Team.find(team_id)
if team.blank?
nil
else
team.supplementary_review_questionnaire_id
end
end
</pre>

app/models/response.rb

<pre>
def construct_review_response code, self_id, show_tags = nil, current_user = nil
code += '<table id="review_' + self_id + '" style="display: none;" class="table table-bordered">'
answers = Answer.where(response_id: self.response_id)
team_id = response_map.reviewee_id
supplementary_review_questionnaire_id = Team.get_supplementary_review_questionnaire_id_of_team(team_id)
unless answers.empty?
questionnaire = self.questionnaire_by_answer(answers.first)

unless supplementary_review_questionnaire_id.nil?
supplementary_review_questionnaire = Questionnaire.find(supplementary_review_questionnaire_id)
supplementary_review_questions = supplementary_review_questionnaire.questions.sort_by(&:seq)
end

questionnaire_max = questionnaire.max_question_score
questions = questionnaire.questions.sort_by(&:seq)

unless supplementary_review_questionnaire.nil?
questions += supplementary_review_questions
end

# get the tag settings this questionnaire
tag_prompt_deployments = show_tags ? TagPromptDeployment.where(questionnaire_id: questionnaire.id, assignment_id: self.map.assignment.id) : nil
code = add_table_rows questionnaire_max, questions, answers, code, tag_prompt_deployments, current_user
end
comment = if !self.additional_comment.nil?
self.additional_comment.gsub('^p', '').gsub(/\n/, '<BR/>')
else
''
end
code += '<tr><td><b>Additional Comment: </b>' + comment + '</td></tr>'
code += '</table>'
end
</pre>

2. Viewers:

app/views/grades/view_team.html.erb

app/views/response/response.html.erb
<pre>
<br>
<% unless @supplementary_review_questions.nil? %>
<tr>
<td>
<h2>Supplementary Review Questions</h2>
</td>
</tr>
<% @supplementary_review_questions.each do |question| %>
<tr>
<td>
<% answer = Answer.where(question_id: question.id, response_id: @response.id).first if !@response.nil? %>
<% if question.instance_of? Criterion %>
<%= question.complete(i, answer, @questionnaire.min_question_score, @questionnaire.max_question_score, @dropdown_or_scale) %>
<%= tinymce %>
<% elsif question.instance_of? Scale %>
<%= question.complete(i, answer, @questionnaire.min_question_score, @questionnaire.max_question_score) %>
<%= tinymce %>
<% elsif question.instance_of? UploadFile %>
<% else %>
<%= question.complete(i, answer) %>
<% end %>
</td>
<td width="25%">
<% if [Criterion, TextArea].include? question.class %>
<div id="analysis_<%= i.to_s %>_comments"></div>
<% end %>
</td>
</tr>
<% i += 1 %>
<% end %>
<% end %>
</pre>

app/views/submitted_content/_main.html.erb
<pre>
<% if stage != "Finished" and controller.controller_name == 'submitted_content' and @can_submit %>
<h2>Add Supplementary Review Questions:</h2>
<% if @assignment.supplementary_review_questionnaire? %>
<%= button_to "Create Supplementary Rubric", {:controller => "questionnaires", :action => "create_supplementary_review_questionnaire", :id => participant.id } %>
<% else %>
The instructor hasn't enabled this feature for this assignment
<% end %>
<% end %>
</pre>

3. Controllers:

app/controllers/response_controller.rb
<pre>
# Prepare the parameters when student clicks "Edit"
def edit
assign_instance_vars
get_all_responses
if @prev.present?
@sorted = @review_scores.sort {|m1, m2| m1.version_num.to_i && m2.version_num.to_i ? m2.version_num.to_i <=> m1.version_num.to_i : (m1.version_num ? -1 : 1) }
@largest_version_num = @sorted[0]
end
@modified_object = @response.response_id
# set more handy variables for the view
set_content
@review_scores = []
@questions.each do |question|
@review_scores << Answer.where(response_id: @response.response_id, question_id: question.id).first
end

@supplementary_review_questions.each do |question|
@review_scores << Answer.where(response_id: @response.response_id, question_id: question.id).first
end

@questionnaire = set_questionnaire
render action: 'response'
end

# Update the response and answers when student "edit" existing response
def update
render nothing: true unless action_allowed?
# the response to be updated
@response = Response.find(params[:id])
msg = ""
begin
@map = @response.map
@response.update_attribute('additional_comment', params[:review][:comments])

set_questionnaire
questions = sort_questions(@questionnaire.questions)
supplementary_review_questions = sort_questions(@supplementary_review_questionnaire.questions)
unless @supplementary_review_questionnaire.nil?
questions += supplementary_review_questions
end
create_answers(params, questions) unless params[:responses].nil?

@response.update_attribute('is_submitted', true) if params['isSubmit'] && params['isSubmit'] == 'Yes'
@response.notify_instructor_on_difference if (@map.is_a? ReviewResponseMap) && @response.is_submitted && @response.significant_difference?
rescue StandardError
msg = "Your response was not saved. Cause:189 #{$ERROR_INFO}"
end
ExpertizaLogger.info LoggerMessage.new(controller_name, session[:user].name, "Your response was submitted: #{@response.is_submitted}", request)
redirect_to controller: 'response', action: 'save', id: @map.map_id,
return: params[:return], msg: msg, review: params[:review], save_options: params[:save_options]
end

def create
map_id = params[:id]
map_id = params[:map_id] unless params[:map_id].nil? # pass map_id as a hidden field in the review form
@map = ResponseMap.find(map_id)
@team_id = @map.reviewee_id

get_all_responses
if params[:review][:questionnaire_id]
@questionnaire = Questionnaire.find(params[:review][:questionnaire_id])
@round = params[:review][:round]
@supplementary_review_questionnaire_id = Team.get_supplementary_review_questionnaire_id_of_team(@team_id)
unless @supplementary_review_questionnaire_id.nil?
@supplementary_review_questionnaire = Questionnaire.find(@supplementary_review_questionnaire_id)
end
else
@round = nil
end
is_submitted = (params[:isSubmit] == 'Yes')
was_submitted = false
@response = Response.where(map_id: @map.id, round: @round.to_i).first
if @response.nil?
@response = Response.create(
map_id: @map.id,
additional_comment: params[:review][:comments],
round: @round.to_i,
is_submitted: is_submitted
)
end
was_submitted = @response.is_submitted
@response.update(additional_comment: params[:review][:comments], is_submitted: is_submitted) # ignore if autoupdate try to save when the response object is not yet created.

# ,:version_num=>@version)
# Change the order for displaying questions for editing response views.
questions = sort_questions(@questionnaire.questions)

unless @supplementary_review_questionnaire.nil?
supplementary_review_questions = sort_questions(@supplementary_review_questionnaire.questions)
questions += supplementary_review_questions
end

create_answers(params, questions) if params[:responses]
msg = "Your response was successfully saved."
error_msg = ""
# only notify if is_submitted changes from false to true
if (@map.is_a? ReviewResponseMap) && (was_submitted == false && @response.is_submitted) && @response.significant_difference?
@response.notify_instructor_on_difference
@response.email
end
redirect_to controller: 'response', action: 'save', id: @map.map_id,
return: params[:return], msg: msg, error_msg: error_msg, review: params[:review], save_options: params[:save_options]
end

# new_response if a flag parameter indicating that if user is requesting a new rubric to fill
# if true: we figure out which questionnaire to use based on current time and records in assignment_questionnaires table
# e.g. student click "Begin" or "Update" to start filling out a rubric for others' work
# if false: we figure out which questionnaire to display base on @response object
# e.g. student click "Edit" or "View"
def set_content(new_response = false)
@title = @map.get_title
if @map.survey?
@survey_parent = @map.survey_parent
else
@assignment = @map.assignment
end
@participant = @map.reviewer
@contributor = @map.contributor
new_response ? set_questionnaire_for_new_response : set_questionnaire
set_dropdown_or_scale
@questions = sort_questions(@questionnaire.questions)

unless @supplementary_review_questionnaire.nil?
@supplementary_review_questions = sort_questions(@supplementary_review_questionnaire.questions)
end

@min = @questionnaire.min_question_score
@max = @questionnaire.max_question_score
end

def set_questionnaire_for_new_response
case @map.type
when "ReviewResponseMap", "SelfReviewResponseMap"
reviewees_topic = SignedUpTeam.topic_id_by_team_id(@contributor.id)
@current_round = @assignment.number_of_current_round(reviewees_topic)
@questionnaire = @map.questionnaire(@current_round)

@supplementary_review_questionnaire_id = Team.get_supplementary_review_questionnaire_id_of_team(@contributor.id)
unless @supplementary_review_questionnaire_id.nil?
@supplementary_review_questionnaire = Questionnaire.find(@supplementary_review_questionnaire_id)
end

when
"MetareviewResponseMap",
"TeammateReviewResponseMap",
"FeedbackResponseMap",
"CourseSurveyResponseMap",
"AssignmentSurveyResponseMap",
"GlobalSurveyResponseMap"
@questionnaire = @map.questionnaire
end
end

def set_questionnaire
# if user is not filling a new rubric, the @response object should be available.
# we can find the questionnaire from the question_id in answers
answer = @response.scores.first
@questionnaire = @response.questionnaire_by_answer(answer)

supplementary_review_answer = @response.scores.last
@supplementary_review_questionnaire = @response.questionnaire_by_answer(supplementary_review_answer)

end

</pre>

app/controllers/grades_controller.rb
<pre>
def view_team
@participant = AssignmentParticipant.find(params[:id])
@assignment = @participant.assignment
@team = @participant.team
@team_id = @team.id
@questions = {}
questionnaires = @assignment.questionnaires
retrieve_questions questionnaires
@pscore = @participant.scores(@questions)
@vmlist = []

# loop through each questionnaire, and populate the view model for all data necessary
# to render the html tables.
counter_for_same_rubric = 0
questionnaires.each do |questionnaire|
@round = nil
if @assignment.varying_rubrics_by_round? && questionnaire.type == "ReviewQuestionnaire"
questionnaires = AssignmentQuestionnaire.where(assignment_id: @assignment.id, questionnaire_id: questionnaire.id)
if questionnaires.count > 1
@round = questionnaires[counter_for_same_rubric].used_in_round
counter_for_same_rubric += 1
else
@round = questionnaires[0].used_in_round
counter_for_same_rubric = 0
end
end
vm = VmQuestionResponse.new(questionnaire, @assignment, @round)
supplementary_review_questionnaire_id = Team.get_supplementary_review_questionnaire_id_of_team(@team_id)
unless supplementary_review_questionnaire_id.nil?
supplementary_review_questionnaire = Questionnaire.find(supplementary_review_questionnaire_id)
unless supplementary_review_questionnaire.nil?
supplementary_review_questions = supplementary_review_questionnaire.questions
questionnaire.questions += supplementary_review_questions
end
end
vmquestions = questionnaire.questions
vm.add_questions(vmquestions)
vm.add_team_members(@team)
vm.add_reviews(@participant, @team, @assignment.varying_rubrics_by_round?)
vm.get_number_of_comments_greater_than_10_words
@vmlist << vm
end
@current_role_name = current_role_name
end
</pre>

app/controllers/questionnaires_controller.rb
<pre>
def create_supplementary_review_questionnaire
@participant = AssignmentParticipant.find(params[:id])
@team = Team.find(@participant.team.id)
if @team.supplementary_review_questionnaire_id.nil?
@questionnaire = Questionnaire.create(privete: false, name: "supplementary_review_questionnaire_" + @team.id.to_s,
instructor_id: @team_id, min_question_score: 0, max_question_score: 5, type: "Questionnaire", display_type: "Review"
instruction_loc: Questionnaire::DEFAULT_QUESTIONNAIRE_URL)
if @questionnaire.save
@team.supplementary_review_questionnaire_id = @questionnaire.id
@team.save
flash[:success] = 'You have successfully created a rubric!'
else
flash[:error] = $ERROR_INFO
end
else
@questionnaire = Questionnaire.find(@team.supplementary_review_questionnaire_id)
end
redirect_to controller: 'questionnaires', action: 'edit', id: @questionnaire.id
end
</pre>

4. Config:

routes.rb
<pre>

resources :questionnaires, only: %i[new create edit update] do
collection do
get :copy
get :list
post :list_questionnaires
get :new_quiz
get :select_questionnaire_type
post :select_questionnaire_type
get :toggle_access
get :view
post :create_quiz_questionnaire
post :update_quiz
post :add_new_questions
post :save_all_questions
post :create_supplementary_review_questionnaire
end
end

resources :author_feedback_questionnaires, controller: :questionnaires
resources :review_questionnaires, controller: :questionnaires
resources :metareview_questionnaires, controller: :questionnaires
resources :teammate_review_questionnaires, controller: :questionnaires
resources :survey_questionnaires, controller: :questionnaires
resources :assignment_survey_questionnaires, controller: :questionnaires
resources :global_survey_questionnaires, controller: :questionnaires
resources :course_survey_questionnaires, controller: :questionnaires
resources :bookmarkrating_questionnaires, controller: :questionnaires
resources :supplementary_review_questionnaire, controller: :questionnaires
</pre>

'''Team members'''
----
Yanchen Zhao

Pingping Chen

Jianshu Zhang

Zirun Han

'''Links'''
----
Expertiza on Github:https://github.com/zyczyh/expertiza

Pull Request:https://github.com/expertiza/expertiza/pull/1325

Expertiza YouTube channel:https://youtu.be/3PUNknSbU-k

CSC/ECE 517 Fall 2018/E1879 Student Generated Questions Added To Rubric

2018-12-18T18:31:11Z

Zhan23:

'''Introduction'''
----
The rubrics in expertiza are created by instructors. These Rubrics only contain questions that are related to the existed topics. Now, Expertiza only supports students to pull questionnaires is specific rubrics to get some help. Even that they can ask about anything that is relevant to all the projects that will be submitted But when students encounter difficult problems which are not in the existed topics, and they want to get special advice on that new field, the demand for creating supplementary review questionnaire raises. This project (E1879) aims to solve this problem by allowing students to add questions to the standard instructor generated rubric so that they can get specific feedback on from the reviewers.

'''Proposed solution'''
----

We will add the Supplementary Review Questions to the current Review Questions, and show these student-generated questions under the rubric given by instructor. Even though these questions won't be graded, they will eventually provide diversity for feedback. It will increase the benefit that each team gets because they can get feedback that is specific to their project.

'''Implementation'''
----

''Design Strategy''

In Expertiza, all kinds of rubrics and surveys are subclasses of Questionnaire. A Questionnaire can contain “questions” of several types (e.g., checkboxes, dropdowns, text boxes). So that we will add a new subclass of Questionnaire named SupplementaryReviewQuestionnaire to achieve our goal.

First, we want the professor to control if the students can create their own supplementary questions by using a checkbox to indicate. If the professor allow students to create their own questions, there will be a button in the students "Your work" page which can let the students jump to the same page that an instructor lands on when creating a new rubric. Then we will add the questionnaire id of this new SupplementaryReviewQuestionnaire to the Team table when students create their own questionnaire in order to show the corresponding supplementary questions to the reviewers. And when showing review questions to the reviewers we will look up the questionnaire id in Team table and modify the view to add the supplementary questions to review page. Finally, after the reviewers finish the review and submit their responds, we will change the view and let the team member can see the feedback of their supplementary questions.

''Changes in the User Interface''

1. Assignment Page

We add a checkbox for the professor to indicate if student can add their own questions.

[[File:Yzhao47_1.png]]

2. A button called "Create Supplementary Rubric" will appear in the student's "Your Work" section when the Instructor has allowed students to create supplementary review questions. So when a student clicks the button he/she can add the desired questions.

[[File:Yzhao47_2.png]]

3. The page where student will be directed to when he/she clicks the Create Supplementary Rubric button. Students can add Questions to the created Supplementary Review questionnaire here.

[[File:Yzhao47_3.png]]

4. Review Page

The students own supplementary questions are leading by a heading "Supplementary Review Questions".

[[File:Yzhao47_4.png]]

5. Review Results Page

The supplementary review results will be showed to the team like other results.

[[File:Yzhao47_5.png]]

'''Use Cases'''
----

1. Use case of creating a Supplementary Review Questionnaire

[[File:case1.jpg]]

2. Use case of reviewing questions for reviewers.

[[File:case2.jpg]]

3. Use case of reviewing responses of questions.

[[File:case3.jpg]]

'''Test Plan'''
----
Actually, we need to add tests for the following:

1. To check the link for "Supplementary Review Questionnaire" appears in the "Your Work" section of a student.

2. To check if the link for "Supplementary Review Questionnaire" redirects to page which allows to create questionnaire.

3. To check if the questionnaire ID is stored in the Team table.

4. To check if the reviewers can see the supplementary questions that were added by the team as part of the review questions.

5. To check if the responses of the Supplementary Review Questions have been added to the responses of the existing review questions.

'''File which are added/modified'''
----

1. Models

app/models/team.rb

<pre>
def self.get_supplementary_review_questionnaire_id_of_team(team_id)
team = Team.find(team_id)
if team.blank?
nil
else
team.supplementary_review_questionnaire_id
end
end
</pre>

app/models/response.rb

2. Viewers:

app/viewers/grades/view_team.html.erb

app/viewers/response/response.html.erb

app/viewers/submitted_content/_main.html.erb

3. Controllers:

app/controllers/response_controller.rb
<pre>
# Prepare the parameters when student clicks "Edit"
def edit
assign_instance_vars
get_all_responses
if @prev.present?
@sorted = @review_scores.sort {|m1, m2| m1.version_num.to_i && m2.version_num.to_i ? m2.version_num.to_i <=> m1.version_num.to_i : (m1.version_num ? -1 : 1) }
@largest_version_num = @sorted[0]
end
@modified_object = @response.response_id
# set more handy variables for the view
set_content
@review_scores = []
@questions.each do |question|
@review_scores << Answer.where(response_id: @response.response_id, question_id: question.id).first
end

@supplementary_review_questions.each do |question|
@review_scores << Answer.where(response_id: @response.response_id, question_id: question.id).first
end

@questionnaire = set_questionnaire
render action: 'response'
end

# Update the response and answers when student "edit" existing response
def update
render nothing: true unless action_allowed?
# the response to be updated
@response = Response.find(params[:id])
msg = ""
begin
@map = @response.map
@response.update_attribute('additional_comment', params[:review][:comments])

set_questionnaire
questions = sort_questions(@questionnaire.questions)
supplementary_review_questions = sort_questions(@supplementary_review_questionnaire.questions)
unless @supplementary_review_questionnaire.nil?
questions += supplementary_review_questions
end
create_answers(params, questions) unless params[:responses].nil?

@response.update_attribute('is_submitted', true) if params['isSubmit'] && params['isSubmit'] == 'Yes'
@response.notify_instructor_on_difference if (@map.is_a? ReviewResponseMap) && @response.is_submitted && @response.significant_difference?
rescue StandardError
msg = "Your response was not saved. Cause:189 #{$ERROR_INFO}"
end
ExpertizaLogger.info LoggerMessage.new(controller_name, session[:user].name, "Your response was submitted: #{@response.is_submitted}", request)
redirect_to controller: 'response', action: 'save', id: @map.map_id,
return: params[:return], msg: msg, review: params[:review], save_options: params[:save_options]
end

def create
map_id = params[:id]
map_id = params[:map_id] unless params[:map_id].nil? # pass map_id as a hidden field in the review form
@map = ResponseMap.find(map_id)
@team_id = @map.reviewee_id

get_all_responses
if params[:review][:questionnaire_id]
@questionnaire = Questionnaire.find(params[:review][:questionnaire_id])
@round = params[:review][:round]
@supplementary_review_questionnaire_id = Team.get_supplementary_review_questionnaire_id_of_team(@team_id)
unless @supplementary_review_questionnaire_id.nil?
@supplementary_review_questionnaire = Questionnaire.find(@supplementary_review_questionnaire_id)
end
else
@round = nil
end
is_submitted = (params[:isSubmit] == 'Yes')
was_submitted = false
@response = Response.where(map_id: @map.id, round: @round.to_i).first
if @response.nil?
@response = Response.create(
map_id: @map.id,
additional_comment: params[:review][:comments],
round: @round.to_i,
is_submitted: is_submitted
)
end
was_submitted = @response.is_submitted
@response.update(additional_comment: params[:review][:comments], is_submitted: is_submitted) # ignore if autoupdate try to save when the response object is not yet created.

# ,:version_num=>@version)
# Change the order for displaying questions for editing response views.
questions = sort_questions(@questionnaire.questions)

unless @supplementary_review_questionnaire.nil?
supplementary_review_questions = sort_questions(@supplementary_review_questionnaire.questions)
questions += supplementary_review_questions
end

create_answers(params, questions) if params[:responses]
msg = "Your response was successfully saved."
error_msg = ""
# only notify if is_submitted changes from false to true
if (@map.is_a? ReviewResponseMap) && (was_submitted == false && @response.is_submitted) && @response.significant_difference?
@response.notify_instructor_on_difference
@response.email
end
redirect_to controller: 'response', action: 'save', id: @map.map_id,
return: params[:return], msg: msg, error_msg: error_msg, review: params[:review], save_options: params[:save_options]
end

# new_response if a flag parameter indicating that if user is requesting a new rubric to fill
# if true: we figure out which questionnaire to use based on current time and records in assignment_questionnaires table
# e.g. student click "Begin" or "Update" to start filling out a rubric for others' work
# if false: we figure out which questionnaire to display base on @response object
# e.g. student click "Edit" or "View"
def set_content(new_response = false)
@title = @map.get_title
if @map.survey?
@survey_parent = @map.survey_parent
else
@assignment = @map.assignment
end
@participant = @map.reviewer
@contributor = @map.contributor
new_response ? set_questionnaire_for_new_response : set_questionnaire
set_dropdown_or_scale
@questions = sort_questions(@questionnaire.questions)

unless @supplementary_review_questionnaire.nil?
@supplementary_review_questions = sort_questions(@supplementary_review_questionnaire.questions)
end

@min = @questionnaire.min_question_score
@max = @questionnaire.max_question_score
end

def set_questionnaire_for_new_response
case @map.type
when "ReviewResponseMap", "SelfReviewResponseMap"
reviewees_topic = SignedUpTeam.topic_id_by_team_id(@contributor.id)
@current_round = @assignment.number_of_current_round(reviewees_topic)
@questionnaire = @map.questionnaire(@current_round)

@supplementary_review_questionnaire_id = Team.get_supplementary_review_questionnaire_id_of_team(@contributor.id)
unless @supplementary_review_questionnaire_id.nil?
@supplementary_review_questionnaire = Questionnaire.find(@supplementary_review_questionnaire_id)
end

when
"MetareviewResponseMap",
"TeammateReviewResponseMap",
"FeedbackResponseMap",
"CourseSurveyResponseMap",
"AssignmentSurveyResponseMap",
"GlobalSurveyResponseMap"
@questionnaire = @map.questionnaire
end
end

def set_questionnaire
# if user is not filling a new rubric, the @response object should be available.
# we can find the questionnaire from the question_id in answers
answer = @response.scores.first
@questionnaire = @response.questionnaire_by_answer(answer)

supplementary_review_answer = @response.scores.last
@supplementary_review_questionnaire = @response.questionnaire_by_answer(supplementary_review_answer)

end

</pre>

app/controllers/grades_controller.rb
<pre>
def view_team
@participant = AssignmentParticipant.find(params[:id])
@assignment = @participant.assignment
@team = @participant.team
@team_id = @team.id
@questions = {}
questionnaires = @assignment.questionnaires
retrieve_questions questionnaires
@pscore = @participant.scores(@questions)
@vmlist = []

# loop through each questionnaire, and populate the view model for all data necessary
# to render the html tables.
counter_for_same_rubric = 0
questionnaires.each do |questionnaire|
@round = nil
if @assignment.varying_rubrics_by_round? && questionnaire.type == "ReviewQuestionnaire"
questionnaires = AssignmentQuestionnaire.where(assignment_id: @assignment.id, questionnaire_id: questionnaire.id)
if questionnaires.count > 1
@round = questionnaires[counter_for_same_rubric].used_in_round
counter_for_same_rubric += 1
else
@round = questionnaires[0].used_in_round
counter_for_same_rubric = 0
end
end
vm = VmQuestionResponse.new(questionnaire, @assignment, @round)
supplementary_review_questionnaire_id = Team.get_supplementary_review_questionnaire_id_of_team(@team_id)
unless supplementary_review_questionnaire_id.nil?
supplementary_review_questionnaire = Questionnaire.find(supplementary_review_questionnaire_id)
unless supplementary_review_questionnaire.nil?
supplementary_review_questions = supplementary_review_questionnaire.questions
questionnaire.questions += supplementary_review_questions
end
end
vmquestions = questionnaire.questions
vm.add_questions(vmquestions)
vm.add_team_members(@team)
vm.add_reviews(@participant, @team, @assignment.varying_rubrics_by_round?)
vm.get_number_of_comments_greater_than_10_words
@vmlist << vm
end
@current_role_name = current_role_name
end
</pre>

4. Config:

routes.rb
<pre>

resources :questionnaires, only: %i[new create edit update] do
collection do
get :copy
get :list
post :list_questionnaires
get :new_quiz
get :select_questionnaire_type
post :select_questionnaire_type
get :toggle_access
get :view
post :create_quiz_questionnaire
post :update_quiz
post :add_new_questions
post :save_all_questions
post :create_supplementary_review_questionnaire
end
end

resources :author_feedback_questionnaires, controller: :questionnaires
resources :review_questionnaires, controller: :questionnaires
resources :metareview_questionnaires, controller: :questionnaires
resources :teammate_review_questionnaires, controller: :questionnaires
resources :survey_questionnaires, controller: :questionnaires
resources :assignment_survey_questionnaires, controller: :questionnaires
resources :global_survey_questionnaires, controller: :questionnaires
resources :course_survey_questionnaires, controller: :questionnaires
resources :bookmarkrating_questionnaires, controller: :questionnaires
resources :supplementary_review_questionnaire, controller: :questionnaires
</pre>

'''Team members'''
----
Yanchen Zhao

Pingping Chen

Jianshu Zhang

Zirun Han

'''Links'''
----
Expertiza on Github:https://github.com/zyczyh/expertiza

Pull Request:https://github.com/expertiza/expertiza/pull/1325

Expertiza YouTube channel:https://youtu.be/3PUNknSbU-k

CSC/ECE 517 Fall 2018/E1879 Student Generated Questions Added To Rubric

2018-12-17T22:17:27Z

Zhan23:

CSC/ECE 517 Fall 2018/E1879 Student Generated Questions Added To Rubric

2018-12-17T22:03:57Z

Zhan23:

CSC/ECE 517 Fall 2018/E1879 Student Generated Questions Added To Rubric

2018-12-17T19:05:00Z

Zhan23:

CSC/ECE 517 Fall 2018/E1879 Student Generated Questions Added To Rubric

2018-12-17T18:59:52Z

Zhan23:

File:Yzhao47 5.png

2018-12-17T18:58:39Z

Zhan23:

File:Yzhao47 4.png

2018-12-17T18:58:22Z

Zhan23:

File:Yzhao47 3.png

2018-12-17T18:58:03Z

Zhan23:

File:Yzhao47 2.png

2018-12-17T18:57:48Z

Zhan23:

File:Yzhao47 1.png

2018-12-17T18:57:17Z

Zhan23:

CSC/ECE 517 Fall 2018/E1879 Student Generated Questions Added To Rubric

2018-12-17T18:20:40Z

Zhan23:

CSC/ECE 517 Fall 2018/E1879 Student Generated Questions Added To Rubric

2018-12-17T18:15:06Z

Zhan23:

File:Capture.PNG

2018-12-17T18:12:10Z

Zhan23: uploaded a new version of "File:Capture.PNG"

CSC/ECE 517 Fall 2018/E1879 Student Generated Questions Added To Rubric

2018-12-17T16:52:15Z

Zhan23:

CSC/ECE 517 Fall 2018/E1879 Student Generated Questions Added To Rubric

2018-12-17T16:45:19Z

Zhan23:

CSC/ECE 517 Fall 2018/E1879 Student Generated Questions Added To Rubric

2018-12-17T16:44:33Z

Zhan23:

CSC/ECE 517 Fall 2018/E1879 Student Generated Questions Added To Rubric

2018-12-17T16:28:16Z

Zhan23:

CSC/ECE 517 Fall 2018/E1879 Student Generated Questions Added To Rubric

2018-12-17T06:13:50Z

Zhan23:

CSC/ECE 517 Fall 2018/E1879 Student Generated Questions Added To Rubric

2018-12-17T05:52:55Z

Zhan23:

CSC/ECE 517 Fall 2018/E1879 Student Generated Questions Added To Rubric

2018-11-20T21:55:59Z

Zhan23:

CSC/ECE 517 Fall 2018/E1879 Student Generated Questions Added To Rubric

2018-11-14T00:33:01Z

Zhan23:

File:Case3.jpg

2018-11-14T00:32:34Z

Zhan23:

CSC/ECE 517 Fall 2018/E1879 Student Generated Questions Added To Rubric

2018-11-14T00:32:23Z

Zhan23:

CSC/ECE 517 Fall 2018/E1879 Student Generated Questions Added To Rubric

2018-11-14T00:30:09Z

Zhan23:

File:Case2.jpg

2018-11-14T00:25:43Z

Zhan23:

File:Case2.PNG

2018-11-14T00:24:59Z

Zhan23:

CSC/ECE 517 Fall 2018/E1879 Student Generated Questions Added To Rubric

2018-11-14T00:24:36Z

Zhan23:

CSC/ECE 517 Fall 2018/E1879 Student Generated Questions Added To Rubric

2018-11-14T00:21:01Z

Zhan23:

File:Case1.jpg

2018-11-14T00:20:16Z

Zhan23: uploaded a new version of "File:Case1.jpg"

File:Case1.jpg

2018-11-14T00:19:33Z

Zhan23:

File:UseCase1.jpg

2018-11-14T00:10:05Z

Zhan23:

File:Case1.PNG

2018-11-14T00:00:26Z

Zhan23:

CSC/ECE 517 Fall 2018/E1879 Student Generated Questions Added To Rubric

2018-11-13T23:58:55Z

Zhan23:

CSC/ECE 517 Fall 2018/E1879 Student Generated Questions Added To Rubric

2018-11-13T17:31:57Z

Zhan23:

CSC/ECE 517 Fall 2018/E1879 Student Generated Questions Added To Rubric

2018-11-13T17:28:27Z

Zhan23:

CSC/ECE 517 Fall 2018/E1879 Student Generated Questions Added To Rubric

2018-11-13T17:20:10Z

Zhan23:

CSC/ECE 517 Fall 2018/E1879 Student Generated Questions Added To Rubric

2018-11-13T15:27:17Z

Zhan23:

CSC/ECE 517 Fall 2018/E1879 Student Generated Questions Added To Rubric

2018-11-13T08:22:50Z

Zhan23:

CSC/ECE 517 Fall 2018/E1879 Student Generated Questions Added To Rubric

2018-11-13T08:22:25Z

Zhan23:

CSC/ECE 517 Fall 2018/E1879 Student Generated Questions Added To Rubric

2018-11-13T08:21:38Z

Zhan23:

Introduction
----
The rubrics in expertiza are created by instructors. These Rubrics only contain questions that are related to the existed topics.Now, Expertiza only supports students to pull questionnaires is specific rubrics to get some help. Even that they can ask about anything that is relevant to all the projects that will be submitted But when students encounter difficult problems which are not in the existed topics, and they want to get special advice on that new field, the demand for creating supplementary review questionnaire raises.

'''Implementation'''
----

In Expertiza, all kinds of rubrics and surveys are subclasses of Questionnaire. A Questionnaire can contain “questions” of several types (e.g., checkboxes, dropdowns, text boxes). So that we will add a new subclass of Questionnaire named SupplementaryReviewQuestionnaire to achieve our goal.

Test Plan
----

File which are added/modified
----

Team members
----
Yanchen Zhao

Pingping Chen

Jianshu Zhang

Zirun Han

Links
----

CSC/ECE 517 Fall 2018/E1879 Student Generated Questions Added To Rubric

2018-11-13T08:21:24Z

Zhan23:

Introduction
----
The rubrics in expertiza are created by instructors. These Rubrics only contain questions that are related to the existed topics.Now, Expertiza only supports students to pull questionnaires is specific rubrics to get some help. Even that they can ask about anything that is relevant to all the projects that will be submitted But when students encounter difficult problems which are not in the existed topics, and they want to get special advice on that new field, the demand for creating supplementary review questionnaire raises.

Implementation
----

In Expertiza, all kinds of rubrics and surveys are subclasses of Questionnaire. A Questionnaire can contain “questions” of several types (e.g., checkboxes, dropdowns, text boxes). So that we will add a new subclass of Questionnaire named SupplementaryReviewQuestionnaire to achieve our goal.

Test Plan
----

File which are added/modified
----

Team members
----
Yanchen Zhao

Pingping Chen

Jianshu Zhang

Zirun Han

Links
----

CSC/ECE 517 Fall 2018/E1879 Student Generated Questions Added To Rubric

2018-11-13T08:18:15Z

Zhan23:

Introduction
----
The rubrics in expertiza are created by instructors. These Rubrics only contain questions that are related to the existed topics.Now, Expertiza only supports students to pull questionnaires is specific rubrics to get some help. Even that they can ask about anything that is relevant to all the projects that will be submitted But when students encounter difficult problems which are not in the existed topics, and they want to get special advice on that new field, the demand for creating supplementary review questionnaire raises.

Implementation
----

Test Plan
----

File which are added/modified
----

Team members
----
Yanchen Zhao

Pingping Chen

Jianshu Zhang

Zirun Han

Links
----

CSC/ECE 517 Fall 2018/E1879 Student Generated Questions Added To Rubric

2018-11-13T08:17:57Z

Zhan23:

Introduction
----
The rubrics in expertiza are created by instructors. These Rubrics only contain questions that are related to the existed topics.Now, Expertiza only supports students to pull questionnaires is specific rubrics to get some help. Even that they can ask about anything that is relevant to all the projects that will be submitted But when students encounter difficult problems which are not in the existed topics, and they want to get special advice on that new field, the demand for creating supplementary review questionnaire raises.

Implementation
----

Test Plan
----

File which are added/modified
----

Team members
----
Yanchen Zhao

Pingping Chen

Jianshu Zhang

Zirun Han

Links
----

CSC/ECE 517 Fall 2018/E1879 Student Generated Questions Added To Rubric

2018-11-13T06:33:01Z

Zhan23:

Introduction
----

Implementation
----

Test Plan
----

File which are added/modified
----

Team members
----
Yanchen Zhao

Pingping Chen

Jianshu Zhang

Zirun Han

Links
----

CSC/ECE 517 Fall 2018/E1879 Student Generated Questions Added To Rubric

2018-11-13T06:32:28Z

Zhan23:

[[Introduction]]

Introduction
----

Implementation
----

Test Plan
----

File which are added/modified
----

Team members
----
Yanchen Zhao

Pingping Chen

Jianshu Zhang

Zirun Han

Links
----

CSC/ECE 517 Fall 2018/E1879 Student Generated Questions Added To Rubric

2018-11-13T06:29:35Z

Zhan23:

Introduction
----

Implementation
----

Test Plan
----

File which are added/modified
----

Team members
----
'''Bold text'''Yanchen Zhao

Pingping Chen

Jianshu Zhang

Zirun Han

Links
----