CSC/ECE 517 Spring 2019 - Project E1931. Conflict notification

This wiki page is for the description of changes made under ECE517 SPRING 2019 PROJECT E1931 CONFLICT NOTIFICATIONS

Problem Statement

Current conflict notification sends an email to the instructor whenever two reviews for the same submission differ “significantly” on their scoring (the threshold is specified in the “Notification limit” on the Rubrics tab of assignment creation). Right now an email is sent at any such time one of these conflicts happen, and it links to the review or submission which initiated the notification. It gives the link in raw HTML, but it should give it as a link associated with text, such as the text “this new review” linking to the page that shows the review.The email send to the instructor should also link to page that shows data on each of the reviews done on this team.

Existing Feature Description

Currently, this feature works as follows:

Whenever a new review is submitted, it is compared with the average of the previously submitted reviews. If the difference between the new submission and existing average is more than the threshold limit set, then a mail is sent to the instructor.With every review submitted for an assignment of a particular student, the average is updated.The mail sent to the instructor contains the links to the conflicted review and a summary of the reviews.

Flaws with the existing Implementation

No view exists to analyze the conflict report

Currently whenever the conflict happens,a summary link is sent to the instructor which contains the score of all the reviews but there is no view where the instructor can see all the conflicts and analyze them.

Incorrect email message links

The existing email body uses hardcoded URLs mentioned in models/response.rb file in the method notify_instructor_on_difference. Being hardcoded, these links wouldn't work on other servers where Expertiza is running. For Example, if the setup is done on localhost, the links will not be functional.

• Conflicting Review which triggered the mail

What needs to be done

The scope of this project is to send an email notification to instructor which will contain links to the conflicting review, summary link, and a link to report which can be used for analyzing. The new report will have the information like the Team(having conflicts), the standard deviation for the team review score and pictorial representation of all the review scores.

The email sent to the instructor should link to a page that shows the data on each of the reviews done on this team:

• The name of the team.
• The members (user-IDs and/or names)
• One line for each review received.In this line would be

1. The name of the person (or team) that submitted the review, which would be hyperlinked to the review itself
2. The score given for the review
3. A bar chart showing pictorial view

• The average score received from reviewers.
• The standard deviation of the score received from reviewers.
• Review scores that are outside some “tolerance” limit should be highlighted, e.g., boldfaced or shown in a different color.Probably the tolerance should be, perhaps, one or two standard deviations from the mean.

Implementation

• Feature 1: New page to show the conflict report

A new view page of the report which will have the following information:-

• The name of the team.
• The members of the team.
• The name of the reviewer which would be hyperlinked to the review itself.
• The score given for the review.
• A bar chart showing pictorial view of all the reviews for a particular team.
• The average score received from reviewers.
• The standard deviation of the score received from reviewers.
• Conflicting reviewers are highlighted.These are those which are two standard deviations away from the mean.

This view have a report of all the teams on one page.This report link is added to the mail that is already being sent whenever some conflict happens.This report page will be also accessible from the reports drop-down in the instructor view.

• A new partial is created for this view -

<div style="color:green;text-align:adjust">
  Note: <br>
  X- axis represents precentage score of the team<br>
  Y- axis represents the reviewers <br>
  Highlighted reviewers are outside the tolerance limit<br>
</div><br><br>
<table id="myTable" border="1px solid black" margin-top="-9px"  width="100%" cellspacing='0' cellpadding='2' border='0'>
  <thead>
  <tr bgcolor='#CCCCCC'>
    <!-- class value decides whether the column should be sortable or not  -->
    <th width="30%" class="sorter-false" border="1px black solid" style="text-align:center">Team <span></span></th>
    <th width="70%" class="sorter-false" border="1px black solid" style="text-align:center" >Reviewers/Metrics <span></span></th>
  </tr>
  </thead>
  <tbody>
  <% max_scores= max_assignment_review_score_per_round(@id) %>
  <%@reviewers.keys.each do |team_name| %>
    <tr>
      <td style="padding-left: 50px;" >
        <% members = team_members(team_name, @id)%>
        <strong>Team Name: </strong><%= team_name %><br>
        <strong>Team Members: </strong><br>
	<% members.each do |member|%>
          <%=User.find(member.user_id).fullname %><br>
        <% end %>
      </td>
      <td>
      <%= render 'review_conflict_metric' , team_name: team_name , max_scores: max_scores %>
  </td>
  </tr>
  <% end %>
  </tbody>
</table>

• Sample view of the new report -

• Feature 2: No hardcoded URLs

Hardcoded URLs are be managed by the config file in order to make these links functional on every server running Expertiza and raw URLs will be given some meaningful names.

Files changed in this Project

• mailer.rb
• Config File
• response.rb
• Response Controller
• Report Format Helper
• Reports View
• review_response_map.rb
• review_mapping_helper.rb

A new option will be added to view the conflict report in the _searchbox.html.erb.A new view will be added to the same.
A new view partial _review_conflict_report.html.erb will be added in report views.

Test Plan

To test code implementation and correctness of the modified feature, we plan to do the following tests:-

• Run and pass existing RSpec Tests
• Develop New RSpec Tests for the new code
• UI testing on the deployed project

Rspec Testing

Rspec test files to be updated -

• Reports Controller Spec
• Response Controller Spec
• Response spec
• mailer spec

[Screenshots to be populated after coding is complete]

UI Testing

[Screenshots/video to be populated after coding is complete]

References

• expertiza
• Previous Work
• Previous Pull Request
• Standard-Deviation gem
• descriptive-statistics gem

Team

Abhishek Arya
Bhargav Deshpande
Ramandeep Kaur
Udita Chattopadhyay