CSC/ECE 517 Spring 2018- Project E1817: Adding Student-generated Questions to Rubric: Difference between revisions
Line 55: | Line 55: | ||
*Review Page | *Review Page | ||
**When a reviewer fills out a rubric, the ResponseController will display a set of rubrics, in order, on the same page. This set would normally consist of just a review rubric, which is a Response object. However, the set would now have a review rubric and a supplementary review rubric (two items in the set instead of one) if the Supplementary Review rubric has been added. Thus, now the reviewer will be able to see the Supplementary Review questions along with the existing review questions. | **When a reviewer fills out a rubric, the ResponseController will display a set of rubrics, in order, on the same page. This set would normally consist of just a review rubric, which is a Response object. However, the set would now have a review rubric and a supplementary review rubric (two items in the set instead of one) if the Supplementary Review rubric has been added. Thus, now the reviewer will be able to see the Supplementary Review questions along with the existing review questions. | ||
:[[File:rsz_screenshot_51.png]] | |||
*Review Results Page | *Review Results Page |
Revision as of 23:48, 29 April 2018
Introduction
Overview of Project
Expertiza contains Peer-Review where in all students are asked to review the work done by their fellow classmates. The rubric for the same are created by instructors. These Rubrics contain questions that are related to the submitted topics. But sometimes students have questions which may or may not be related to the work other students are working on. So there is no option currently available to ask these questions to their peers. This project (E1817) aims to solve this problem by allowing students to add questions to the standard instructor generated rubric so that they can get specific feedback on from the reviewers.
Proposed Solution
In order to solve the above problem, we intend to add Supplementary Review Questions to the existing Review Questions already added by the instructor. These “extra” questions will not be graded. However, they will increase the benefit that each person gets because they can get feedback that is specific to their project. The rest of this document describes the design and approach for project E1817.
Implementation
In Expertiza, all rubrics and surveys are subclasses of Questionnaire. A Questionnaire has “questions” that make use of checkboxes, dropdowns, text boxes, etc. We want to add a new subclass of Questionnaire called SupplementaryReviewQuestionnaire to help us implement this project.
Design Strategy
The flowcharts below describes the design that we chose to implement for this project.
- When a student wants to create/edit Supplementary Review Questionnaire
- If the instructor has enabled a particular assignment to have supplementary review questionnaire only then a student can create a supplementary review questionnaire. A student can click on the "Your Work" tab and see a link to the add/edit Supplementary Review Questionnaire. This link directs the student to the create review questionnaire(same as instructor).
- When a student creates a Supplementary Review Questionnaire (SRQ), the entry in the SRQ column added to the Teams table will contain the questionnaire id to link the SRQ with the team that generated it. If the SRQ column of a team is empty it means no supplementary review questionnaire was created. If its not empty, it will indicate that a supplementary review questionnaire was created.
- When Reviewer wants to access the Review Questions.
- When a reviewer requests for a review, first the entry of the SRQ column in the Teams table is checked to see if supplementary review questionnaire was added. If the field is not empty, the supplementary review questionnaire is appended to the existing review questionnaire created by the instructor. Otherwise the reviewer only sees the default review questionnaire.
- When responses of questionnaire need to be displayed
- When the responses of the reviewer is recorded, the student can click on "View Scores" tab for a particular assignment which contains the responses of the reviewer. This view should contain the responses for both the default review questions as well the supplementary review questions added by the student. So if the entry of the SRQ column in the Teams table is not empty the responses of the supplementary review questions is appended to the responses of the default review questions.
NOTE: The responses of the Supplementary Review Questions will not be added to the review scores of the project. It will just be shown to the team that has asked these questions so that they can see what suggestions does the reviewer has for the questions.
Changes in the User Interface
The following changes need to be made to the User Interface.
- Assignment Page
- An assignment will have a checkbox that the instructor can ‘check’ to enable a student to add supplementary review rubric. Once ‘checked’ a button or link will appear in the student's "Your Work" section. This link will redirect the student to the same page that an instructor lands on when creating a new rubric, which will allow the student to create a review rubric just like an instructor does.
- A link called "Supplemantary Review Questionnaire" will appear in the student's "Your Work" section when the Instructor has allowed students to create supplementary review questions. So when a student clicks the link he/she can add the desired questions.
- The page where student will be directed to when he/she clicks the Supplementary Review Questionnaire link. Students can add Questions to the created Supplementary Review questionnaire here.
- Review Page
- When a reviewer fills out a rubric, the ResponseController will display a set of rubrics, in order, on the same page. This set would normally consist of just a review rubric, which is a Response object. However, the set would now have a review rubric and a supplementary review rubric (two items in the set instead of one) if the Supplementary Review rubric has been added. Thus, now the reviewer will be able to see the Supplementary Review questions along with the existing review questions.
- Review Results Page
- The “View” function for a rubric will display answers submitted for the SupplementaryReviewQuestionnaire as well as the ReviewQuestionnaire. We will add another column to the “View scores” page (for both instructor and students) so that the responses of the Supplementary Review Questions are also shown to student.
NOTE:The student-generated questions (Supplementary Review Questions) will not be graded to avoid encouraging students to ask “easy” questions so that their reviewers would give them a high scores.
Test Plan
Firstly, we need to add tests for the following:
1. To check the link for "Supplementary Review Questionnaire" appears in the "Your Work" section of a student.
2. To check if the link for "Supplementary Review Questionnaire" redirects to page which allows to create questionnaire.
3. To check if the reviewers can see the supplementary questions that were added by the team as part of the review questions.
4. To check if the responses of the Supplementary Review Questions have been added to the responses of the existing review questions.
Supplementary Review Questionnaire Testing : supplementary_review_rubric_spec.rb :
- In this test, we are testing the addition and editing of the supplementary questionnaire. Furthermore, we are also checking whether the questionnaire is being displayed or not to the reviewer. Also a test has been added to check if the questionnaire gets added to the model.
include TopicHelper describe "supplementary rubric testing" do before(:each) do # create assignment and topic create(:assignment, name: "TestAssignment", directory_path: "TestAssignment") create_list(:participant, 3) create(:assignment_team) create(:team_user, user: User.where(role_id: 2).first, team: AssignmentTeam.first) create(:topic, topic_name: "TestTopic") create(:deadline_type, name: "submission") create(:deadline_type, name: "review") create(:deadline_type, name: "metareview") create(:deadline_type, name: "drop_topic") create(:deadline_type, name: "signup") create(:deadline_type, name: "team_formation") create(:deadline_right) create(:deadline_right, name: 'Late') create(:deadline_right, name: 'OK') create(:assignment_questionnaire) create(:questionnaire) create(:question) create(:assignment_due_date, deadline_type: DeadlineType.where(name: "submission").first, due_at: DateTime.now.in_time_zone + 1.day) end it "has srq to add/edit supplementary questionnaire" do signup_topic expect(page).to have_content "Create Supplementary Review Questionnaire" end it "add supplementary questionnaire to teams model" do signup_topic click_link "Create Supplementary Review Questionnaire" click_button "Add" click_button "Save supplementaryreview questionnaire" # assert !Team.supplementary_rubric_by_team_id(2).nil? end it "should display Supplementary Questionnaire to assigned student" do submit_to_topic click_link "Create Supplementary Review Questionnaire" click_button "Add" click_button "Save supplementaryreview questionnaire" user = User.find_by(name: "student2066") stub_current_user(user, user.role.name, user.role) visit '/student_task/list' click_link "TestAssignment" click_link "Others' work" find(:css, "#i_dont_care").set(true) click_button "Request a new submission to review" click_link "Begin" #expect(page).to have_content "Supplementary Reviewee Generated Questions" end end
Topic Helper : topic_helper.rb :
- This file is included in the helper folder inside spec/features. This helper is used because the given methods are used in more than one place and to keep the code "DRY". In the first method we are stubbing the user so that that user signs up for an assignment. In the second method more testing is being performed to test the upload link.
module TopicHelper def signup_topic user = User.find_by(name: "student2064") stub_current_user(user, user.role.name, user.role) visit '/student_task/list' visit '/sign_up_sheet/sign_up?id=1&topic_id=1' # signup topic visit '/student_task/list' click_link "TestAssignment" click_link "Your work" end def submit_to_topic signup_topic fill_in 'submission', with: "https://www.ncsu.edu" click_on 'Upload link' expect(page).to have_content "https://www.ncsu.edu" end end
File which are added/modified
1. app/models/supplementary_review_questionnaire.rb
- Added a migration to create a subclass "SupplementaryReviewQuestionnaire".
class SupplementaryReviewQuestionnaire < ReviewQuestionnaire after_initialize :post_initialization def post_initialization self.display_type = 'SupplementaryReview' end def symbol "supplementary".to_sym end end
2. db/migrate/20180419210958_add_srq_id_to_teams.rb
- Added a migration which will add the Supplementary Review Questionnaire id column to the "Teams" table. This is to link the team which created the supplementary review questionnaire to the created questionnaire.
class AddSrqIdToTeams < ActiveRecord::Migration def change add_column :teams, :srq_id, :integer end end
3. config/routes.rb
4. app/controllers/response_controller.rb
- Obtained supplementary review questionnaire from the questionnaires table for new response.
def set_questionnaire_for_new_response case @map.type when "ReviewResponseMap", "SelfReviewResponseMap" reviewees_topic = SignedUpTeam.topic_id_by_team_id(@contributor.id) @current_round = @assignment.number_of_current_round(reviewees_topic) @questionnaire = @map.questionnaire(@current_round) # added for srq @sr_questionnaire_id = Team.get_srq_id_of_team(@contributor.id) unless @sr_questionnaire_id.nil? @sr_questionnaire = Questionnaire.find(@sr_questionnaire_id) end when "MetareviewResponseMap", "TeammateReviewResponseMap", "FeedbackResponseMap", "CourseSurveyResponseMap", "AssignmentSurveyResponseMap", "GlobalSurveyResponseMap" @questionnaire = @map.questionnaire end end
- Obtained answers for the supplementary review questionnaire from the responses.
def set_questionnaire # if user is not filling a new rubric, the @response object should be available. # we can find the questionnaire from the question_id in answers answer = @response.scores.first @questionnaire = @response.questionnaire_by_answer(answer) # added for srq sr_answer = @response.scores.last @sr_questionnaire = @response.questionnaire_by_answer(sr_answer) end
- Appended responses of the supplementary review questionnaire in the edit method.
def edit @header = "Edit" @next_action = "update" @return = params[:return] @response = Response.find(params[:id]) @map = @response.map @contributor = @map.contributor set_all_responses if @prev.present? @sorted = @review_scores.sort {|m1, m2| m1.version_num.to_i and m2.version_num.to_i ? m2.version_num.to_i <=> m1.version_num.to_i : (m1.version_num ? -1 : 1) } @largest_version_num = @sorted[0] end @modified_object = @response.response_id # set more handy variables for the view set_content(new_response = true) @review_scores = [] @questions.each do |question| @review_scores << Answer.where(response_id: @response.response_id, question_id: question.id).first end # added for srq @sr_questions.each do |question| @review_scores << Answer.where(response_id: @response.response_id, question_id: question.id).first end render action: 'response' end
- Append questions of supplementary review questionnaire in create method and set content method.
def create map_id = params[:id] map_id = params[:map_id] if !params[:map_id].nil?# pass map_id as a hidden field in the review form @map = ResponseMap.find(map_id) # added for srq @team_id = @map.reviewee_id set_all_responses if params[:review][:questionnaire_id] @questionnaire = Questionnaire.find(params[:review][:questionnaire_id]) @round = params[:review][:round] # added for srq @sr_questionnaire_id = Team.get_srq_id_of_team(@team_id) unless @sr_questionnaire_id.nil? @sr_questionnaire = Questionnaire.find(@sr_questionnaire_id) end else @round = nil end is_submitted = (params[:isSubmit] == 'Yes') if params[:saved].nil? || params[:saved] == "0" # a flag so the autosave doesn't create different versions. The value's changed by the javascript in response.js @response = Response.create( map_id: @map.id, additional_comment: params[:review][:comments], round: @round, is_submitted: is_submitted ) else @response = Response.find_by(map_id: @map.id, round: @round) if !@response.nil? @response.update(additional_comment: params[:review][:comments]) # ignore if autoupdate try to save when the response object is not yet created. else logger.error("Can't find response with '#{@map.id}' and round '#{@round}' to update, even though params[:saved] = #{params[:saved]}") end end # ,:version_num=>@version) # Change the order for displaying questions for editing response views. questions = sort_questions(@questionnaire.questions) # added for srq unless @sr_questionnaire.nil? sr_questions = sort_questions(@sr_questionnaire.questions) questions += sr_questions end create_answers(params, questions) if params[:responses] msg = "Your response was successfully saved." error_msg = "" # @response.notify_instructor_on_difference if (@map.is_a? ReviewResponseMap) && @response.is_submitted && @response.significant_difference? # @response.email redirect_to controller: 'response', action: 'saving', id: @map.map_id, return: params[:return], msg: msg, error_msg: error_msg, save_options: params[:save_options] end
def set_content(new_response = false) @title = @map.get_title if @map.survey? @survey_parent = @map.survey_parent else @assignment = @map.assignment end @participant = @map.reviewer @contributor = @map.contributor new_response ? set_questionnaire_for_new_response : set_questionnaire set_dropdown_or_scale @questions = sort_questions(@questionnaire.questions) # added for srq unless @sr_questionnaire.nil? @sr_questions = sort_questions(@sr_questionnaire.questions) end @min = @questionnaire.min_question_score @max = @questionnaire.max_question_score end
5. app/controllers/grades_controller.rb
- Added supplementary review responses to "Your Scores" section.
def view_team @participant = AssignmentParticipant.find(params[:id]) @assignment = @participant.assignment @team = @participant.team @team_id = @team.id @questions = {} questionnaires = @assignment.questionnaires retrieve_questions questionnaires @pscore = @participant.scores(@questions) @vmlist = [] # loop through each questionnaire, and populate the view model for all data necessary # to render the html tables. questionnaires.each do |questionnaire| @round = if @assignment.varying_rubrics_by_round? && questionnaire.type == "ReviewQuestionnaire" AssignmentQuestionnaire.find_by(assignment_id: @assignment.id, questionnaire_id: questionnaire.id).used_in_round end vm = VmQuestionResponse.new(questionnaire, @assignment) vmquestions = questionnaire.questions # added for srq @sr_questionnaire_id = Team.get_srq_id_of_team(@team.id) unless @sr_questionnaire_id.nil? @sr_questionnaire = Questionnaire.find(@sr_questionnaire_id) unless @sr_questionnaire.nil? sr_questions = @sr_questionnaire.questions end end vmquestions += sr_questions vm.add_questions(vmquestions) vm.add_team_members(@team) vm.add_reviews(@participant, @team, @assignment.varying_rubrics_by_round?) vm.get_number_of_comments_greater_than_10_words @vmlist << vm end @current_role_name = current_role_name end
6. app/views/assignments/edit/_rubrics.html.erb
- Add a checkbox called "Allow students to create supplementary review questionnaire".
<input name="assignment_form[assignment][srq]" type="hidden" value="false"/> <%= check_box_tag('assignment_form[assignment][srq]', 'true', @assignment_form.assignment.srq) %> <%= label_tag('assignment_form[assignment][srq]', 'Allow students to add Supplementary Review Questionnaire?') %>
7. app/views/submitted_content/_main.html.erb
- Created a link " Create Supplementary Review Questionnaire" to appear in the student's "Your Work" section if the instructor has enabled student generated Questions for this assignment.
<% if stage != "Finished" and controller.controller_name == 'submitted_content' and @can_submit %> <h2>Add Supplementary Review Questions:</h2> <% if @assignment.srq? %> <%= link_to "Create Supplementary Review Questionnaire", {:controller => "questionnaires", :action => "create_srq", :id => participant.id } %> <% else %> The instructor has not enabled this feature for this assignment <% end %> <% end %>
8. app/controllers/questionnaires_controller
- Added a method to create Supplementary Review Questionnaire.
def create_srq @participant = AssignmentParticipant.find(params[:id]) @team = Team.find(@participant.team.id) if @team.srq_id.nil? then @questionnaire = Questionnaire.new @questionnaire.private = false @questionnaire.name = "SRQ_" + @team.id.to_s @questionnaire.instructor_id = @team.id @questionnaire.min_question_score = 0 @questionnaire.max_question_score = 5 @questionnaire.type = "SupplementaryReviewQuestionnaire" @questionnaire.display_type = "SupplementaryReviewQuestionnaire" @questionnaire.instruction_loc = Questionnaire::DEFAULT_QUESTIONNAIRE_URL @questionnaire.save @team.srq_id = @questionnaire.id @team.save flash[:success] = 'You have successfully created a questionnaire!' else @questionnaire = Questionnaire.find(@team.srq_id) end redirect_to controller: 'questionnaires', action: 'edit', id: @questionnaire.id end
9. app/models/questionnaire.rb
- Added "SupplementaryReviewQuestionnaire" to QUESTIONNAIRE_TYPES.
10. app/models/response.rb
- Added supplementary review responses to be visible in the "Show Review" section.
def construct_review_response code, self_id code += '<table id="review_' + self_id + '" style="display: none;" class="table table-bordered">' answers = Answer.where(response_id: self.response_id) unless answers.empty? questionnaire = self.questionnaire_by_answer(answers.first) # added for srq sr_questionnaire = self.questionnaire_by_answer(answers.last) questionnaire_max = questionnaire.max_question_score questions = questionnaire.questions.sort_by(&:seq) # added for srq sr_questions = sr_questionnaire.questions.sort_by(&:seq) unless sr_questionnaire.nil? questions += sr_questions end # get the tag settings this questionnaire tag_prompt_deployments = TagPromptDeployment.where(questionnaire_id: questionnaire.id, assignment_id: self.map.assignment.id) code = add_table_rows questionnaire_max, questions, answers, code end comment = if !self.additional_comment.nil? self.additional_comment.gsub('^p', '').gsub(/\n/, '<BR/>') else '' end code += '<tr><td><b>Additional Comment: </b>' + comment + '</td></tr>' code += '</table>' end
11. app/models/team.rb
- Added a method to obtain srq_id of a team.
def self.get_srq_id_of_team(team_id) t = Team.find(team_id) if t.blank? nil else t.srq_id end end
12. app/views/response/response.html.erb
- Added the following code section to append supplementary review to existing review.
<br> <% unless @sr_questions.nil? %> <% @sr_questions.each do |question| %> <% answer = Answer.where(question_id: question.id, response_id: @response.id).first if !@response.nil? %> <% if question.instance_of? Criterion %> <%= question.complete(i, answer, @questionnaire.min_question_score, @questionnaire.max_question_score, @dropdown_or_scale) %> <%= tinymce %> <% elsif question.instance_of? Scale %> <%= question.complete(i, answer, @questionnaire.min_question_score, @questionnaire.max_question_score) %> <%= tinymce %> <% elsif question.instance_of? UploadFile %> <!--One form had better not in another form--> <!--Zhewei: I move the Uploadfile question to the end of questionnaire. So multiple forms will not been overlapped.--> <% else %> <%= question.complete(i, answer) %> <% end %> <% i += 1 %> <% end %> <% end %>
13. db/migrate/20180428233726_add_srq_to_assignments.rb
- . Added a migration to add a column "srq" to the "assignments" table.
class AddSrqToAssignments < ActiveRecord::Migration def change add_column :assignments, :srq, :boolean end end
Team members
- Shrinath Cheriyana
- Sanika Sabnis
- Pratik Keny
- Praneet Mocherla
Links
- Expertiza on Github:https://github.com/Pratik2611/expertiza/tree/shrinathedit
- Pull Request: https://github.com/expertiza/expertiza/pull/1178
- Expertiza YouTube channel: https://youtu.be/p2zz_WN8lLs