CSC/ECE 517 Spring 2018- Project E1817: Adding Student-generated Questions to Rubric

From Expertiza_Wiki
Jump to navigation Jump to search

Introduction

Overview of Project

Expertiza contains Peer-Review where in all students are asked to review the work done by their fellow classmates. The rubric for the same are created by instructors. These Rubrics contain questions that are related to the submitted topics. But sometimes students have questions which may or may not be related to the work other students are working on. So there is no option currently available to ask these questions to their peers. This project (E1817) aims to solve this problem by allowing students to add questions to the standard instructor generated rubric so that they can get specific feedback on from the reviewers.

Proposed Solution

In order to solve the above problem, we intend to add Supplementary Review Questions to the existing Review Questions already added by the instructor. These “extra” questions will not be graded. However, they will increase the benefit that each person gets because they can get feedback that is specific to their project. The rest of this document describes the design and approach for project E1817.

Implementation

In Expertiza, all rubrics and surveys are subclasses of Questionnaire. A Questionnaire has “questions” that make use of checkboxes, dropdowns, text boxes, etc. We want to add a new subclass of Questionnaire called SupplementaryReviewQuestionnaire to help us implement this project.

Design Strategy

The flowcharts below describes the design that we chose to implement for this project.

  • When a student wants to create/edit Supplementary Review Questionnaire
    • If the instructor has enabled a particular assignment to have supplementary review questionnaire only then a student can create a supplementary review questionnaire. A student can click on the "Your Work" tab and see a link to the add/edit Supplementary Review Questionnaire. This link directs the student to the create review questionnaire(same as instructor).
    • When a student creates a Supplementary Review Questionnaire (SRQ), the entry in the SRQ column added to the Teams table will contain the questionnaire id to link the SRQ with the team that generated it. If the SRQ column of a team is empty it means no supplementary review questionnaire was created. If its not empty, it will indicate that a supplementary review questionnaire was created.


  • When Reviewer wants to access the Review Questions.
    • When a reviewer requests for a review, first the entry of the SRQ column in the Teams table is checked to see if supplementary review questionnaire was added. If the field is not empty, the supplementary review questionnaire is appended to the existing review questionnaire created by the instructor. Otherwise the reviewer only sees the default review questionnaire.

  • When responses of questionnaire need to be displayed
    • When the responses of the reviewer is recorded, the student can click on "View Scores" tab for a particular assignment which contains the responses of the reviewer. This view should contain the responses for both the default review questions as well the supplementary review questions added by the student. So if the entry of the SRQ column in the Teams table is not empty the responses of the supplementary review questions is appended to the responses of the default review questions.

NOTE: The responses of the Supplementary Review Questions will not be added to the review scores of the project. It will just be shown to the team that has asked these questions so that they can see what suggestions does the reviewer has for the questions.

Changes in the User Interface

The following changes need to be made to the User Interface.

  • Assignment Page
    • An assignment will have a checkbox that the instructor can ‘check’ to enable a student to add supplementary review rubric. Once ‘checked’ a button or link will appear in the student's "Your Work" section. This link will redirect the student to the same page that an instructor lands on when creating a new rubric, which will allow the student to create a review rubric just like an instructor does.


  • A link called "Supplemantary Review Questionnaire" will appear in the student's "Your Work" section when the Instructor has allowed students to create supplementary review questions. So when a student clicks the link he/she can add the desired questions.


  • The page where student will be directed to when he/she clicks the Supplementary Review Questionnaire link. Students can add Questions to the created Supplementary Review questionnaire here.


  • Review Page
    • When a reviewer fills out a rubric, the ResponseController will display a set of rubrics, in order, on the same page. This set would normally consist of just a review rubric, which is a Response object. However, the set would now have a review rubric and a supplementary review rubric (two items in the set instead of one) if the Supplementary Review rubric has been added. Thus, now the reviewer will be able to see the Supplementary Review questions along with the existing review questions.
  • Review Results Page
    • The “View” function for a rubric will display answers submitted for the SupplementaryReviewQuestionnaire as well as the ReviewQuestionnaire.

NOTE:The student-generated questions (Supplementary Review Questions) should not be graded to avoid encouraging students to ask “easy” questions so that their reviewers would give them a high scores. In order to do that, the students must explicitly set the weight of these questions to 0 while creating the supplementary review questionnaire.

Test Plan

Firstly, we need to add tests for the following:

1. To check the link for "Supplementary Review Questionnaire" appears in the "Your Work" section of a student.

2. To check if the link for "Supplementary Review Questionnaire" redirects to page which allows to create questionnaire.

3. To check if the reviewers can see the supplementary questions that were added by the team as part of the review questions.

4. To check if the responses of the Supplementary Review Questions have been added to the responses of the existing review questions.


Supplementary Review Questionnaire Testing : supplementary_review_rubric_spec.rb :

In this test, we are testing the addition and editing of the supplementary questionnaire. Furthermore, we are also checking whether the questionnaire is being displayed or not to the reviewer. Also a test has been added to check if the questionnaire gets added to the model.
include TopicHelper
describe "supplementary rubric testing" do
  before(:each) do
      # create assignment and topic
      create(:assignment, name: "TestAssignment", directory_path: "TestAssignment")
      create_list(:participant, 3)
      create(:assignment_team)
      create(:team_user, user: User.where(role_id: 2).first, team: AssignmentTeam.first)
      create(:topic, topic_name: "TestTopic")
      create(:deadline_type, name: "submission")
      create(:deadline_type, name: "review")
      create(:deadline_type, name: "metareview")
      create(:deadline_type, name: "drop_topic")
      create(:deadline_type, name: "signup")
      create(:deadline_type, name: "team_formation")
      create(:deadline_right)
      create(:deadline_right, name: 'Late')
      create(:deadline_right, name: 'OK')
      create(:assignment_questionnaire)
      create(:questionnaire)
      create(:question)
      create(:assignment_due_date, deadline_type: DeadlineType.where(name: "submission").first, due_at: DateTime.now.in_time_zone + 1.day)
    end

  it "has srq to add/edit supplementary questionnaire" do
    signup_topic
    expect(page).to have_content "Create Supplementary Review Questionnaire"
  end

  it "add supplementary questionnaire to teams model" do
    signup_topic
    click_link "Create Supplementary Review Questionnaire"
    click_button "Add"
    click_button "Save supplementaryreview questionnaire"
   # assert !Team.supplementary_rubric_by_team_id(2).nil?
  end

  it "should display Supplementary Questionnaire to assigned student" do
    submit_to_topic
    click_link "Create Supplementary Review Questionnaire"
    click_button "Add"
    click_button "Save supplementaryreview questionnaire"
    user = User.find_by(name: "student2066")
    stub_current_user(user, user.role.name, user.role)
    visit '/student_task/list'
    click_link "TestAssignment"
    click_link "Others' work"
    find(:css, "#i_dont_care").set(true)
    click_button "Request a new submission to review"
    click_link "Begin"
    #expect(page).to have_content "Supplementary Reviewee Generated Questions"
  end
end

Topic Helper : topic_helper.rb :

This file is included in the helper folder inside spec/features. This helper is used because the given methods are used in more than one place and to keep the code "DRY". In the first method we are stubbing the user so that that user signs up for an assignment. In the second method more testing is being performed to test the upload link.
module TopicHelper
    def signup_topic
        user = User.find_by(name: "student2064")
        stub_current_user(user, user.role.name, user.role)
        visit '/student_task/list'
        visit '/sign_up_sheet/sign_up?id=1&topic_id=1' # signup topic
        visit '/student_task/list'
        click_link "TestAssignment"
        click_link "Your work"
      end
  
    def submit_to_topic
        signup_topic
        fill_in 'submission', with: "https://www.ncsu.edu"
        click_on 'Upload link'
        expect(page).to have_content "https://www.ncsu.edu"

      end
  end

File which are added/modified

1. app/models/supplementary_review_questionnaire.rb

  • Added a migration to create a subclass "SupplementaryReviewQuestionnaire".
class SupplementaryReviewQuestionnaire < ReviewQuestionnaire
  after_initialize :post_initialization
  def post_initialization
    self.display_type = 'SupplementaryReview'
  end

  def symbol
    "supplementary".to_sym
  end

end

2. db/migrate/20180419210958_add_srq_id_to_teams.rb

  • Added a migration which will add the Supplementary Review Questionnaire id column to the "Teams" table. This is to link the team which created the supplementary review questionnaire to the created questionnaire.
class AddSrqIdToTeams < ActiveRecord::Migration
  def change
    add_column :teams, :srq_id, :integer
  end
end

3. config/routes.rb

4. app/controllers/response_controller.rb

  • Obtained supplementary review questionnaire from the questionnaires table for new response.
def set_questionnaire_for_new_response
    case @map.type
    when "ReviewResponseMap", "SelfReviewResponseMap"
      reviewees_topic = SignedUpTeam.topic_id_by_team_id(@contributor.id)
      @current_round = @assignment.number_of_current_round(reviewees_topic)
      @questionnaire = @map.questionnaire(@current_round)
      # added for srq
      @sr_questionnaire_id = Team.get_srq_id_of_team(@contributor.id)
      unless @sr_questionnaire_id.nil?
        @sr_questionnaire = Questionnaire.find(@sr_questionnaire_id)
      end
    when
      "MetareviewResponseMap",
      "TeammateReviewResponseMap",
      "FeedbackResponseMap",
      "CourseSurveyResponseMap",
      "AssignmentSurveyResponseMap",
      "GlobalSurveyResponseMap"
      @questionnaire = @map.questionnaire
    end
  end
  • Obtained answers for the supplementary review questionnaire from the responses.
def set_questionnaire
    # if user is not filling a new rubric, the @response object should be available.
    # we can find the questionnaire from the question_id in answers
    answer = @response.scores.first
    @questionnaire = @response.questionnaire_by_answer(answer)
    # added for srq
    sr_answer = @response.scores.last
    @sr_questionnaire = @response.questionnaire_by_answer(sr_answer)
  end
  • Appended responses of the supplementary review questionnaire in the edit method.
def edit
    @header = "Edit"
    @next_action = "update"
    @return = params[:return]
    @response = Response.find(params[:id])
    @map = @response.map
    @contributor = @map.contributor
    set_all_responses
    if @prev.present?
      @sorted = @review_scores.sort {|m1, m2| m1.version_num.to_i and m2.version_num.to_i ? m2.version_num.to_i <=> m1.version_num.to_i : (m1.version_num ? -1 : 1) }
      @largest_version_num = @sorted[0]
    end
    @modified_object = @response.response_id
    # set more handy variables for the view
    set_content(new_response = true)
    @review_scores = []
    @questions.each do |question|
      @review_scores << Answer.where(response_id: @response.response_id, question_id: question.id).first
    end
    # added for srq
    @sr_questions.each do |question|
      @review_scores << Answer.where(response_id: @response.response_id, question_id: question.id).first
    end
    render action: 'response'
  end
  • Append questions of supplementary review questionnaire in create method and set content method.
def create
    map_id = params[:id]
    map_id = params[:map_id] if !params[:map_id].nil?# pass map_id as a hidden field in the review form
    @map = ResponseMap.find(map_id)
    # added for srq
    @team_id = @map.reviewee_id
    set_all_responses
    if params[:review][:questionnaire_id]
      @questionnaire = Questionnaire.find(params[:review][:questionnaire_id])
      @round = params[:review][:round]
      # added for srq
      @sr_questionnaire_id = Team.get_srq_id_of_team(@team_id)
      unless @sr_questionnaire_id.nil?
        @sr_questionnaire = Questionnaire.find(@sr_questionnaire_id)
      end
    else
      @round = nil
    end
    is_submitted = (params[:isSubmit] == 'Yes')

    if params[:saved].nil? || params[:saved] == "0" # a flag so the autosave doesn't create different versions. The value's changed by the javascript in response.js
      @response = Response.create(
          map_id: @map.id,
          additional_comment: params[:review][:comments],
          round: @round,
          is_submitted: is_submitted
      )
    else
      @response = Response.find_by(map_id: @map.id, round: @round)
      if !@response.nil?
        @response.update(additional_comment: params[:review][:comments]) # ignore if autoupdate try to save when the response object is not yet created.
      else
        logger.error("Can't find response with '#{@map.id}' and round '#{@round}' to update, even though params[:saved] = #{params[:saved]}")
      end
    end

    # ,:version_num=>@version)
    # Change the order for displaying questions for editing response views.
    questions = sort_questions(@questionnaire.questions)
    # added for srq
    unless @sr_questionnaire.nil?
      sr_questions = sort_questions(@sr_questionnaire.questions)
      questions += sr_questions
    end
    create_answers(params, questions) if params[:responses]

    msg = "Your response was successfully saved."
    error_msg = ""
    # @response.notify_instructor_on_difference if (@map.is_a? ReviewResponseMap) && @response.is_submitted && @response.significant_difference?
    # @response.email
    redirect_to controller: 'response', action: 'saving', id: @map.map_id, return: params[:return], msg: msg, error_msg: error_msg, save_options: params[:save_options]
  end

def set_content(new_response = false)
    @title = @map.get_title
    if @map.survey?
      @survey_parent = @map.survey_parent
    else
      @assignment = @map.assignment
    end
    @participant = @map.reviewer
    @contributor = @map.contributor
    new_response ? set_questionnaire_for_new_response : set_questionnaire
    set_dropdown_or_scale
    @questions = sort_questions(@questionnaire.questions)
    # added for srq
    unless @sr_questionnaire.nil?
      @sr_questions = sort_questions(@sr_questionnaire.questions)
    end
    @min = @questionnaire.min_question_score
    @max = @questionnaire.max_question_score
  end

5. app/controllers/grades_controller.rb

Added supplementary review responses to "Your Scores" section.
def view_team
    @participant = AssignmentParticipant.find(params[:id])
    @assignment = @participant.assignment
    @team = @participant.team
    @team_id = @team.id
    @questions = {}
    questionnaires = @assignment.questionnaires
    retrieve_questions questionnaires
    @pscore = @participant.scores(@questions)
    @vmlist = []

    # loop through each questionnaire, and populate the view model for all data necessary
    # to render the html tables.
    questionnaires.each do |questionnaire|
      @round = if @assignment.varying_rubrics_by_round? && questionnaire.type == "ReviewQuestionnaire"
                 AssignmentQuestionnaire.find_by(assignment_id: @assignment.id, questionnaire_id: questionnaire.id).used_in_round
               end
      vm = VmQuestionResponse.new(questionnaire, @assignment)
      vmquestions = questionnaire.questions
      # added for srq
      @sr_questionnaire_id = Team.get_srq_id_of_team(@team.id)
      unless @sr_questionnaire_id.nil?
        @sr_questionnaire = Questionnaire.find(@sr_questionnaire_id)
        unless @sr_questionnaire.nil?
          sr_questions = @sr_questionnaire.questions
        end
      end
      vmquestions += sr_questions
      vm.add_questions(vmquestions)
      vm.add_team_members(@team)
      vm.add_reviews(@participant, @team, @assignment.varying_rubrics_by_round?)
      vm.get_number_of_comments_greater_than_10_words
      @vmlist << vm
    end
    @current_role_name = current_role_name
  end

6. app/views/assignments/edit/_rubrics.html.erb

  • Add a checkbox called "Allow students to create supplementary review questionnaire".
<input name="assignment_form[assignment][srq]" type="hidden" value="false"/>
<%= check_box_tag('assignment_form[assignment][srq]', 'true', @assignment_form.assignment.srq) %>
<%= label_tag('assignment_form[assignment][srq]', 'Allow students to add Supplementary Review Questionnaire?') %>

7. app/views/submitted_content/_main.html.erb

  • Created a link " Create Supplementary Review Questionnaire" to appear in the student's "Your Work" section if the instructor has enabled student generated Questions for this assignment.
<% if stage != "Finished" and controller.controller_name == 'submitted_content' and @can_submit %>
  <h2>Add Supplementary Review Questions:</h2>
    <% if @assignment.srq? %>
      <%= link_to "Create Supplementary Review Questionnaire", {:controller => "questionnaires", :action => "create_srq", :id => participant.id } %>
    <% else %>
      The instructor has not enabled this feature for this assignment
    <% end %>
<% end %>

8. app/controllers/questionnaires_controller

  • Added a method to create Supplementary Review Questionnaire.
def create_srq
    @participant = AssignmentParticipant.find(params[:id])
    @team = Team.find(@participant.team.id)
    if @team.srq_id.nil? then
      @questionnaire = Questionnaire.new
      @questionnaire.private = false
      @questionnaire.name = "SRQ_" + @team.id.to_s
      @questionnaire.instructor_id = @team.id
      @questionnaire.min_question_score = 0
      @questionnaire.max_question_score = 5
      @questionnaire.type = "SupplementaryReviewQuestionnaire"
      @questionnaire.display_type = "SupplementaryReviewQuestionnaire"
      @questionnaire.instruction_loc = Questionnaire::DEFAULT_QUESTIONNAIRE_URL
      @questionnaire.save
      @team.srq_id = @questionnaire.id
      @team.save
      flash[:success] = 'You have successfully created a questionnaire!'
    else
      @questionnaire = Questionnaire.find(@team.srq_id)
    end
    redirect_to controller: 'questionnaires', action: 'edit', id: @questionnaire.id
  end

9. app/models/questionnaire.rb

  • Added "SupplementaryReviewQuestionnaire" to QUESTIONNAIRE_TYPES.

10. app/models/response.rb

  • Added supplementary review responses to be visible in the "Show Review" section.
def construct_review_response code, self_id
    code += '<table id="review_' + self_id + '" style="display: none;" class="table table-bordered">'
    answers = Answer.where(response_id: self.response_id)
    unless answers.empty?
      questionnaire = self.questionnaire_by_answer(answers.first)
      # added for srq
      sr_questionnaire = self.questionnaire_by_answer(answers.last)
      questionnaire_max = questionnaire.max_question_score
      questions = questionnaire.questions.sort_by(&:seq)
      # added for srq
      sr_questions = sr_questionnaire.questions.sort_by(&:seq)
      unless sr_questionnaire.nil?
        questions += sr_questions
      end
      # get the tag settings this questionnaire
      tag_prompt_deployments = TagPromptDeployment.where(questionnaire_id: questionnaire.id, assignment_id: self.map.assignment.id)
      code = add_table_rows questionnaire_max, questions, answers, code
    end
    comment = if !self.additional_comment.nil?
                self.additional_comment.gsub('^p', '').gsub(/\n/, '<BR/>')
              else
                ''
              end
    code += '<tr><td><b>Additional Comment: </b>' + comment + '</td></tr>'
    code += '</table>'
  end

11. app/models/team.rb

  • Added a method to obtain srq_id of a team.
def self.get_srq_id_of_team(team_id)
    t = Team.find(team_id)
    if t.blank?
      nil
    else
      t.srq_id
    end
  end

12. app/views/response/response.html.erb

  • Added the following code section to append supplementary review to existing review.
<br>
    <% unless @sr_questions.nil? %>
      <% @sr_questions.each do |question| %>
        <% answer = Answer.where(question_id: question.id, response_id: @response.id).first if !@response.nil? %>
        <% if question.instance_of? Criterion %>
          <%= question.complete(i, answer, @questionnaire.min_question_score, @questionnaire.max_question_score, @dropdown_or_scale) %>
          <%= tinymce %>
        <% elsif question.instance_of? Scale %>
          <%= question.complete(i, answer, @questionnaire.min_question_score, @questionnaire.max_question_score) %>
          <%= tinymce %>
        <% elsif question.instance_of? UploadFile %>
          <!--One form had better not in another form-->
          <!--Zhewei: I move the Uploadfile question to the end of questionnaire. So multiple forms will not been overlapped.-->
        <% else %>
          <%= question.complete(i, answer) %>
        <% end %>
        <% i += 1 %>
      <% end %>
    <% end %>

13. db/migrate/20180428233726_add_srq_to_assignments.rb

  • . Added a migration to add a column "srq" to the "assignments" table.
class AddSrqToAssignments < ActiveRecord::Migration
  def change
    add_column :assignments, :srq, :boolean
  end
end

Team members

  • Shrinath Cheriyana
  • Sanika Sabnis
  • Pratik Keny
  • Praneet Mocherla

Links

Retrieved from "https://wiki.expertiza.ncsu.edu/index.php?title=CSC/ECE_517_Spring_2018-_Project_E1817:_Adding_Student-generated_Questions_to_Rubric&oldid=117506"