CSC/ECE 517 Fall 2021 - E2164. Heatgrid fixes and improvements

Project Background

Heatgrid is the page where an instructor can visit and view the review scores, author feedback scores, and meta-review scores of the student. This web page displays all the evaluations done by the peers of the participant for his/her project. The scores of each questionnaire for a review are shown in a tabular format.

Overview of Issues

Heatgrid page can be navigated through the instructor's "Assign Grades" button or the student's "View Scores" button.

There are two issues assigned to us:-

• Issue 1869: The first is that the column name "metric-1" in the heatgrid is not self-explanatory and needs to be changed. Also, we need to change it in such a way that in later stages if the instructor wants to add a new metric for the assignment, he/she should be able to add it with just minor changes.
• Issue 2019: The problem in the second issue is when an instructor tries to Assign Grades to a student 9277 for Madeup Problem 3, he/she is not able to see the Round-1 Reviews. However, the database (old database) we were provided with does not have any student with Student id: 9277, and the assignment Madeup Problem 3 doesn't exist.

Issue 1869

Problem Definition

Note that there is a “metric-1” column in the review report, but it doesn’t say what the metric represents. The column header in the heatgrid review scores should be changed accordingly. However, the intent was that eventually an instructor would be allowed to select a metric to be shown on the heatgrid. So, somewhere in the UI for creating/editing an assignment, there should be a way for the instructor to select a metric from a dropdown list. The mechanism for adding a metric in the future has to be created.

Current Implementation

Overview

In the current implementation, the columns in the heatgrid table are static. The "metric-1" column in the table represents the count of comments for the respective question which have a word count greater than 10. On hovering over the column name, we can see the following message: "A count of comments, for the respective question, which have word count > 10. The purpose of this metric is to represent how many comments for the question are of a substantial length to provide quality feedback.". The information displayed is correct but the name itself does not make the information apparent at first glance.

UI Screenshots

The following image captures how the heatgrid is currently being displayed:

Plan of work

Implementation Detail

• The _general.html.erb file is used for choosing the general properties of an assignment while creating and editing. For example:- Reputation Algorithm can be chosen through the general view of an assignment. In this, we have added a dropdown for selecting the metric to be displayed in the heatgrid. The default for this dropdown is set to "Verbose-Comment-Count" which was initially metric-1.
• The view_html.html.erb file is responsible for rendering the heatgrid. In the view_team.html.erb, instead of hard-coded metric-1 column header, a switch case is defined which will select the metric chosen by the instructor.
• Now, if the instructor wants to add any new metric, he/she just needs to add the method for calculating the new metric in the vm_question_response.rb file. In _general.html.erb, the instructor needs to add a new dropdown option for selecting this metric. Then the heatgrid_metric variable will be updated according to the instructor's selection. Now, in the view_team.html.erb, a new switch case needs to added to incorporate the newly added metric.

Implementation Control Flow

Proposed Control Flow For Adding a Metric

Detailed Implementation

• In the database in the assignments table, one column named "heatgrid_metric" is added which will store the value of the dropdown menu.
• app/views/assignments/edit/_general.html.erb:- In this file we have added the dropdown which takes the metric from the instructor and sets the metric accordingly. The code we added is mentioned below:-

<tr>
    <td style='padding:5px' id='assignment_metrics_field'>
    <%= label_tag('assignment_form[assignment][heatgrid_metric]', 'Select a Metric to be displayed in the Report Heatgrid:') %>
    <%= select('assignment_form[assignment]', 'heatgrid_metric', [['--', ''], %w[Verbose-Comment-Count countofcomments]], 
    { :selected => @assignment_form.assignment.heatgrid_metric}, { :class => 'form-control', :style => 'width: 100px'}) %>
    </td>
</tr>

We have added assigned two values to the dropdown:- '--' and 'Verbose-Comment-Count'. This is how it looks after adding this dropdown on the expertiza:-

• app/views/grades/view_team.html.erb:- In this file, there is a view for the heatgrid, so when the instructor clicks on Assign Grades, the view from this file is rendered. So, the column name is changed here from "metric-1" to "countofcomments" variable, which will take the store the value of the dropdown selected.

The code that we have changed is:-

<% case @assignment.heatgrid_metric %>
      <% when 'countofcomments' %>
      <th class="sorter-false">
      <span  data-toggle="tooltip" data-placement="right" title="A count of comments, for the respective question, 
       which have word count > 10. The purpose of this metric is to represent how many comments for the question are 
       of a substantial length to provide quality feedback.">Verbose Comment Count</span>
       </th>
       <%end%>

This is how it now looks on expertiza:

Test Plan

• Capybara Test cases for the heat grid view are added to select the value from the dropdown menu. Both the 'Verbose-Comment-Count' and '--' are added.

it "is able to create a public assignment with countofcomments metric" do
		login_as("instructor6")
		visit "/assignments/new?private=0"

		fill_in 'assignment_form_assignment_name', with: 'public assignment for test'
		select('Course 2', from: 'assignment_form_assignment_course_id')
		fill_in 'assignment_form_assignment_directory_path', with: 'testDirectory'
		fill_in 'assignment_form_assignment_spec_location', with: 'testLocation'
		check("assignment_form_assignment_microtask")
		check("assignment_form_assignment_reviews_visible_to_all")
		check("assignment_form_assignment_is_calibrated")
		uncheck("assignment_form_assignment_availability_flag")
		expect(page).to have_select("assignment_form[assignment][reputation_algorithm]", options: %w[-- Hamer Lauw])
		select "Verbose-Comment-Count", from: 'countsofcomment'

		click_button 'Create'
		assignment = Assignment.where(name: 'public assignment for test').first
		expect(assignment).to have_attributes(
			name: 'public assignment for test',
			course_id: Course.find_by(name: 'Course 2').id,
			directory_path: 'testDirectory',
			spec_location: 'testLocation',
			microtask: true,
			is_calibrated: true,
			availability_flag: false,
			heatgrid_metric: 'countofcomments'
		)
	end

	it "is able to create a public assignment with no metrics expected" do
		login_as("instructor6")
		visit "/assignments/new?private=0"

		fill_in 'assignment_form_assignment_name', with: 'public assignment for test'
		select('Course 2', from: 'assignment_form_assignment_course_id')
		fill_in 'assignment_form_assignment_directory_path', with: 'testDirectory'
		fill_in 'assignment_form_assignment_spec_location', with: 'testLocation'
		check("assignment_form_assignment_microtask")
		check("assignment_form_assignment_reviews_visible_to_all")
		check("assignment_form_assignment_is_calibrated")
		uncheck("assignment_form_assignment_availability_flag")
		expect(page).to have_select("assignment_form[assignment][reputation_algorithm]", options: %w[-- Hamer Lauw])
		select "--", from: 'countsofcomment'

		click_button 'Create'
		assignment = Assignment.where(name: 'public assignment for test').first
		expect(assignment).to have_attributes(
			name: 'public assignment for test',
			course_id: Course.find_by(name: 'Course 2').id,
			directory_path: 'testDirectory',
			spec_location: 'testLocation',
			microtask: true,
			is_calibrated: true,
			availability_flag: false,
			heatgrid_metric: ''
		)
	end

Spec test cases are added for the grades controller should be added to verify the responses returned from the controller.

describe '#create with countofcomments' do
    before(:each) do
      allow(AssignmentForm).to receive(:new).with(any_args).and_return(assignment_form)
      @params = {
        button: '',
        assignment_form: {
          assignment_questionnaire: [{"assignment_id" => "1", "questionnaire_id" => "666", "dropdown" => "true",
                                      "questionnaire_weight" => "100", "notification_limit" => "15", "used_in_round" => "1"}],
          due_date: [{"id" => "", "parent_id" => "", "round" => "1", "deadline_type_id" => "1", "due_at" => "2017/12/05 00:00", "submission_allowed_id" => "3", "review_allowed_id" => "1", "teammate_review_allowed_id" => "3", "review_of_review_allowed_id" => "1", "threshold" => "1"},
                     {"id" => "", "parent_id" => "", "round" => "1", "deadline_type_id" => "2", "due_at" => "2017/12/02 00:00", "submission_allowed_id" => "1", "review_allowed_id" => "3", "teammate_review_allowed_id" => "3", "review_of_review_allowed_id" => "1", "threshold" => "1"}],
          assignment: {
            instructor_id: 2,
            course_id: 1,
            max_team_size: 1,
            id: 1,
            name: 'test assignment',
            directory_path: '/test',
            spec_location: '',
            private: false,
            show_teammate_reviews: false,
            require_quiz: false,
            num_quiz_questions: 0,
            staggered_deadline: false,
            microtask: false,
            reviews_visible_to_all: false,
            is_calibrated: false,
            availability_flag: true,
            reputation_algorithm: 'Lauw',
            heatgrid_metric: 'countofcomments',
            simicheck: -1,
            simicheck_threshold: 100
          }
        }
      }
    end
    context 'when assignment_form is saved successfully' do
      it 'redirects to assignment#edit page' do
        allow(assignment_form).to receive(:assignment).and_return(assignment)
        allow(assignment_form).to receive(:save).and_return(true)
        allow(assignment_form).to receive(:update).with(any_args).and_return(true)
        allow(assignment_form).to receive(:create_assignment_node).and_return(double('node'))
        allow(assignment).to receive(:id).and_return(1)
        allow(Assignment).to receive(:find_by).with(id: 1).and_return(assignment)
        allow_any_instance_of(AssignmentsController).to receive(:undo_link)
          .with('Assignment "test assignment" has been created successfully. ').and_return(true)
        post :create, @params
        expect(response).to redirect_to('/assignments/1/edit')
      end
    end

    context 'when assignment_form is not saved successfully' do
      it 'renders assignment#new page' do
        allow(assignment_form).to receive(:save).and_return(false)
        post :create, @params
        expect(response).to render_template(:new)
      end
    end
  end

• Unit test cases are added to verify the logic of the number_of_comments_greater_than_10_words method of the vm_question_response model.

describe '#heatgrid_metric' do
    it 'returns countofcomments by default' do      
      assignment = create(:assignment)
      expect(assignment.heatgrid_metric).to eq('countofcomments')
    end

    it 'none value can be selected' do
      assignment = build(:assignment, heatgrid_metric: '')
      expect(assignment.heatgrid_metric).to eq('')
    end
  end

Issue 2019

Observations

The problem in this issue is when an instructor tries to Assign Grades to a student 9277 for an assignment Madeup Problem 3, he/she can only see the reviews for the last Review Round. However, the database (old database) we were provided with does not have any student with Student id: 9277, and the assignment Madeup Problem 3 doesn't exist.

In the current review system implementation, the instructor has the ability to select or deselect the "Review rubric varies by round?" checkbox from the "Rubrics" tab of the Edit Assignment page. On selecting this checkbox, the reviewer is able to see all Rounds of Reviews. However, when the checkbox is not selected, he/she can only see the review of the last round. Currently, while creating an assignment, the default value of this checkbox is set to false.

UI Screenshots

The following images show how this functionality works currently:

As you can see, the instructor is able to see both the Round of reviews.

Now, if we deselect it, then the heatgrid metric view page looks as follows:-

Implementation

So, as mentioned above, it works as intended, so if the instructor wants to see all the rounds of reviews, he/she have to mark the checkbox "Review rubric varies by round?" to true, then only it can be done. So, for that, we have added a workaround, where we have set the value of the assignment to be true. For that, we created one migration that will set the value to true. Code looks as below:-

class UpdateVaryByRoundToAssignments < ActiveRecord::Migration
  def change
    change_column :assignments, :vary_by_round, :boolean, :default => true
  end
end

Important Links

• GitHub repository link: https://github.com/ShlokSayani/expertiza
• GitHub repository PR link: https://github.com/expertiza/expertiza/pull/2125/files
• Project Description Document: https://docs.google.com/document/d/1slx4HPIbgTH-psIKMSCF-HDF9brxf-FuYhzVT9ZiIrM/edit#heading=h.tqdrrd12xs4x
• GitHub Issue #1869: https://github.com/expertiza/expertiza/issues/1869
• Video Link: https://youtu.be/7EqmFPLwFLE
• GitHub Issue #2019: https://github.com/expertiza/expertiza/issues/2019

Team

• Shlok Sayani (sdsayani)

• Hardik Udeshi (hvudeshi)

• Isha Gupta (igupta)

• Manish Shinde (msshinde)