CSC/ECE 517 Spring 2018- Project E1817: Adding Student-generated Questions to Rubric

Introduction

Overview of Project

Expertiza contains Peer-Review where in all students are asked to review the work done by their fellow classmates. The rubric for the same are created by instructors. These Rubrics contain questions that are related to the submitted topics. But sometimes students have questions which may or may not be related to the work other students are working on. So there is no option currently available to ask these questions to their peers. This project (E1817) aims to solve this problem by allowing students to add questions to the standard instructor generated rubric so that they can get specific feedback on from the reviewers.

Proposed Solution

In order to solve the above problem, we intend to add Supplementary Review Questions to the existing Review Questions already added by the instructor. These “extra” questions will not be graded. However, they will increase the benefit that each person gets because they can get feedback that is specific to their project. The rest of this document describes the design and approach for project E1817.

Implementation

In Expertiza, all rubrics and surveys are subclasses of Questionnaire. A Questionnaire has “questions” that make use of checkboxes, dropdowns, text boxes, etc. We want to add a new subclass of Questionnaire called SupplementaryReviewQuestionnaire to help us implement this project.

Design Strategy

The flowcharts below describes the design that we chose to implement for this project.

• When a student wants to create/edit Supplementary Review Questionnaire
  • If the instructor has enabled a particular assignment to have supplementary review questionnaire only then a student can create a supplementary review questionnaire. A student can click on the "Your Work" tab and see a link to the add/edit Supplementary Review Questionnaire. This link directs the student to the create review questionnaire(same as instructor).
  • When a student creates a Supplementary Review Questionnaire (SRQ), the entry in the SRQ column added to the Teams table will contain the questionnaire id to link the SRQ with the team that generated it. If the SRQ column of a team is empty it means no supplementary review questionnaire was created. If its not empty, it will indicate that a supplementary review questionnaire was created.

• When Reviewer wants to access the Review Questions.
  • When a reviewer requests for a review, first the entry of the SRQ column in the Teams table is checked to see if supplementary review questionnaire was added. If the field is not empty, the supplementary review questionnaire is appended to the existing review questionnaire created by the instructor. Otherwise the reviewer only sees the default review questionnaire.

• When responses of questionnaire need to be displayed
  • When the responses of the reviewer is recorded, the student can click on "View Scores" tab for a particular assignment which contains the responses of the reviewer. This view should contain the responses for both the default review questions as well the supplementary review questions added by the student. So if the entry of the SRQ column in the Teams table is not empty the responses of the supplementary review questions is appended to the responses of the default review questions.

NOTE: The responses of the Supplementary Review Questions will not be added to the review scores of the project. It will just be shown to the team that has asked these questions so that they can see what suggestions does the reviewer has for the questions.

Changes in the User Interface

The following changes need to be made to the User Interface.

• Assignment Page
  • An assignment will have a checkbox that the instructor can ‘check’ to enable a student to add supplementary review rubric. Once ‘checked’ a button or link will appear in the student's "Your Work" section. This link will redirect the student to the same page that an instructor lands on when creating a new rubric, which will allow the student to create a review rubric just like an instructor does.

• A link called "Supplemantary Review Questionnaire" will appear in the student's "Your Work" section when the Instructor has allowed students to create supplementary review questions. So when a student clicks the link he/she can add the desired questions.

• The page where student will be directed to when he/she clicks the Supplementary Review Questionnaire link.

• The page which allows students to add different types of questions to the Supplementary Review Questionnaire.

• Review Page
  • When a reviewer fills out a rubric, the ResponseController will display a set of rubrics, in order, on the same page. This set would normally consist of just a review rubric, which is a Response object. However, the set would now have a review rubric and a supplementary review rubric (two items in the set instead of one) if the Supplementary Review rubric has been added. Thus, now the reviewer will be able to see the Supplementary Review questions along with the existing review questions.

• Review Results Page
  • The “View” function for a rubric will display answers submitted for the SupplementaryReviewQuestionnaire as well as the ReviewQuestionnaire. We will add another column to the “View scores” page (for both instructor and students) so that the responses of the Supplementary Review Questions are also shown to student.

NOTE:The student-generated questions (Supplementary Review Questions) will not be graded to avoid encouraging students to ask “easy” questions so that their reviewers would give them a high scores.

Test Plan

Firstly, we need to add tests for the following:

1. To check the link for "Supplementary Review Questionnaire" appears in the "Your Work" section of a student.

2. To check if the link for "Supplementary Review Questionnaire" redirects to page which allows to create questionnaire.

3. To check if after creation of the supplementary questionnaire, the supplementary questionnaire column of the Teams table has a correct value (i.e not NIL).

4. To check if the reviewers can see the supplementary questions that were added by the team as part of the review questions.

5. To check if the responses of the Supplementary Review Questions have been added to the responses of the existing review questions.

Supplementary Review Questionnaire Testing : supplementary_review_rubric_spec.rb :

In this test, we are testing the addition and editing of the supplementary questionnaire. Furthermore, we are also checking whether the questionnaire is being displayed or not to the reviewer. Also a test has been added to check if the questionnaire gets added to the model.

include TopicHelper
describe "supplementary rubric testing" do
  before(:each) do
      # create assignment and topic
      create(:assignment, name: "TestAssignment", directory_path: "TestAssignment")
      create_list(:participant, 3)
      create(:assignment_team)
      create(:team_user, user: User.where(role_id: 2).first, team: AssignmentTeam.first)
      create(:topic, topic_name: "TestTopic")
      create(:deadline_type, name: "submission")
      create(:deadline_type, name: "review")
      create(:deadline_type, name: "metareview")
      create(:deadline_type, name: "drop_topic")
      create(:deadline_type, name: "signup")
      create(:deadline_type, name: "team_formation")
      create(:deadline_right)
      create(:deadline_right, name: 'Late')
      create(:deadline_right, name: 'OK')
      create(:assignment_questionnaire)
      create(:questionnaire)
      create(:question)
      create(:assignment_due_date, deadline_type: DeadlineType.where(name: "submission").first, due_at: DateTime.now.in_time_zone + 1.day)
    end

  it "has srq to add/edit supplementary questionnaire" do
    signup_topic
    expect(page).to have_content "Create Supplementary Review Questionnaire"
  end

  it "add supplementary questionnaire to teams model" do
    signup_topic
    click_link "Create Supplementary Review Questionnaire"
    click_button "Add"
    click_button "Save supplementaryreview questionnaire"
   # assert !Team.supplementary_rubric_by_team_id(2).nil?
  end

  it "should display Supplementary Questionnaire to assigned student" do
    submit_to_topic
    click_link "Create Supplementary Review Questionnaire"
    click_button "Add"
    click_button "Save supplementaryreview questionnaire"
    user = User.find_by(name: "student2066")
    stub_current_user(user, user.role.name, user.role)
    visit '/student_task/list'
    click_link "TestAssignment"
    click_link "Others' work"
    find(:css, "#i_dont_care").set(true)
    click_button "Request a new submission to review"
    click_link "Begin"
    #expect(page).to have_content "Supplementary Reviewee Generated Questions"
  end
end

Topic Helper : topic_helper.rb :

This file is included in the helper folder inside spec/features. This helper is used because the given methods are used in more than one place and to keep the code "DRY". In the first method we are stubbing the user so that that user signs up for an assignment. In the second method more testing is being performed to test the upload link.

module TopicHelper
    def signup_topic
        user = User.find_by(name: "student2064")
        stub_current_user(user, user.role.name, user.role)
        visit '/student_task/list'
        visit '/sign_up_sheet/sign_up?id=1&topic_id=1' # signup topic
        visit '/student_task/list'
        click_link "TestAssignment"
        click_link "Your work"
      end
  
    def submit_to_topic
        signup_topic
        fill_in 'submission', with: "https://www.ncsu.edu"
        click_on 'Upload link'
        expect(page).to have_content "https://www.ncsu.edu"

      end
  end

File which were added/modified

As of now, we have figured out that the following files need to be added/modified:

1. app/models/supplementary_review_questionnaire.rb

Added a migration to create a subclass "SupplementaryReviewQuestionnaire".

2. db/migrate/add_srq_id_to_teams.rb

Added a migration which will add the Supplementary Review Questionnaire id column to the "Teams" table. This is to link the team which created the supplementary review questionnaire to the created questionnaire.

3. config/routes.rb

To create Supplementary Review Questionnaire

4. app/controllers/response_controller.rb

• Obtained supplementary review questionnaire from the questionnaires table for new response

def set_questionnaire_for_new_response
    case @map.type
    when "ReviewResponseMap", "SelfReviewResponseMap"
      reviewees_topic = SignedUpTeam.topic_id_by_team_id(@contributor.id)
      @current_round = @assignment.number_of_current_round(reviewees_topic)
      @questionnaire = @map.questionnaire(@current_round)
      # added for srq
      @sr_questionnaire_id = Team.get_srq_id_of_team(@contributor.id)
      unless @sr_questionnaire_id.nil?
        @sr_questionnaire = Questionnaire.find(@sr_questionnaire_id)
      end
    when
      "MetareviewResponseMap",
      "TeammateReviewResponseMap",
      "FeedbackResponseMap",
      "CourseSurveyResponseMap",
      "AssignmentSurveyResponseMap",
      "GlobalSurveyResponseMap"
      @questionnaire = @map.questionnaire
    end
  end

• Obtained answers for the supplementary review questionnaire from the responses

def set_questionnaire
    # if user is not filling a new rubric, the @response object should be available.
    # we can find the questionnaire from the question_id in answers
    answer = @response.scores.first
    @questionnaire = @response.questionnaire_by_answer(answer)
    # added for srq
    sr_answer = @response.scores.last
    @sr_questionnaire = @response.questionnaire_by_answer(sr_answer)
  end

• Appended responses of the supplementary review questionnaire in the edit method

def edit
    @header = "Edit"
    @next_action = "update"
    @return = params[:return]
    @response = Response.find(params[:id])
    @map = @response.map
    @contributor = @map.contributor
    set_all_responses
    if @prev.present?
      @sorted = @review_scores.sort {|m1, m2| m1.version_num.to_i and m2.version_num.to_i ? m2.version_num.to_i <=> m1.version_num.to_i : (m1.version_num ? -1 : 1) }
      @largest_version_num = @sorted[0]
    end
    @modified_object = @response.response_id
    # set more handy variables for the view
    set_content(new_response = true)
    @review_scores = []
    @questions.each do |question|
      @review_scores << Answer.where(response_id: @response.response_id, question_id: question.id).first
    end
    # added for srq
    @sr_questions.each do |question|
      @review_scores << Answer.where(response_id: @response.response_id, question_id: question.id).first
    end
    render action: 'response'
  end

• Append questions of supplementary review questionnaire in create method and set content method

def create
    map_id = params[:id]
    map_id = params[:map_id] if !params[:map_id].nil?# pass map_id as a hidden field in the review form
    @map = ResponseMap.find(map_id)
    # added for srq
    @team_id = @map.reviewee_id
    set_all_responses
    if params[:review][:questionnaire_id]
      @questionnaire = Questionnaire.find(params[:review][:questionnaire_id])
      @round = params[:review][:round]
      # added for srq
      @sr_questionnaire_id = Team.get_srq_id_of_team(@team_id)
      unless @sr_questionnaire_id.nil?
        @sr_questionnaire = Questionnaire.find(@sr_questionnaire_id)
      end
    else
      @round = nil
    end
    is_submitted = (params[:isSubmit] == 'Yes')

    if params[:saved].nil? || params[:saved] == "0" # a flag so the autosave doesn't create different versions. The value's changed by the javascript in response.js
      @response = Response.create(
          map_id: @map.id,
          additional_comment: params[:review][:comments],
          round: @round,
          is_submitted: is_submitted
      )
    else
      @response = Response.find_by(map_id: @map.id, round: @round)
      if !@response.nil?
        @response.update(additional_comment: params[:review][:comments]) # ignore if autoupdate try to save when the response object is not yet created.
      else
        logger.error("Can't find response with '#{@map.id}' and round '#{@round}' to update, even though params[:saved] = #{params[:saved]}")
      end
    end

    # ,:version_num=>@version)
    # Change the order for displaying questions for editing response views.
    questions = sort_questions(@questionnaire.questions)
    # added for srq
    unless @sr_questionnaire.nil?
      sr_questions = sort_questions(@sr_questionnaire.questions)
      questions += sr_questions
    end
    create_answers(params, questions) if params[:responses]

    msg = "Your response was successfully saved."
    error_msg = ""
    # @response.notify_instructor_on_difference if (@map.is_a? ReviewResponseMap) && @response.is_submitted && @response.significant_difference?
    # @response.email
    redirect_to controller: 'response', action: 'saving', id: @map.map_id, return: params[:return], msg: msg, error_msg: error_msg, save_options: params[:save_options]
  end

def set_content(new_response = false)
    @title = @map.get_title
    if @map.survey?
      @survey_parent = @map.survey_parent
    else
      @assignment = @map.assignment
    end
    @participant = @map.reviewer
    @contributor = @map.contributor
    new_response ? set_questionnaire_for_new_response : set_questionnaire
    set_dropdown_or_scale
    @questions = sort_questions(@questionnaire.questions)
    # added for srq
    unless @sr_questionnaire.nil?
      @sr_questions = sort_questions(@sr_questionnaire.questions)
    end
    @min = @questionnaire.min_question_score
    @max = @questionnaire.max_question_score
  end

5. spec/models/assignment_spec.rb

Add additional test case to test the new functionality.

6. app/views/assignments/edit/_general.html.erb

Add a checkbox called "Allow students to create supplementary review questionnaire" .

7. app/views/submitted_content/_main.html.erb

Allow a link "Supplementary Review Questionnaire" to appear in the student's "Your Work" section.

Links

• Expertiza on Github:https://github.com/Pratik2611/expertiza/tree/shrinathedit
• Expertiza YouTube channel: https://www.youtube.com/channel/UCdKXzox7hrWjfOMML6FzTWg/videos