CSC/ECE 517 Spring 2020 - E2015. Conflict notification

Abstract

Expertiza is an open-source web application implemented using Ruby on Rails. It is used for management of courses and the assignments for respective courses, by the faculties and the students. Students can form teams in Expertiza to work on an assignment in a group. A student team can submit their work through multiple means such as file uploads and embedded links. Submissions made on Expertiza can be peer-reviewed by students within the course based on a rubric. Results of the peer review can be accessed by instructors and respective student teams. During the peer evaluation, Expertiza sends appropriate email notifications to the instructor and also provides a complete statistical report of the evaluation.

Problem Statement

In the current implementation, during the peer review phase of an assignment, email notifications are sent out to the instructor whenever a submitted review score differs “significantly” from the average score of other submitted reviews for that submission. The threshold to trigger a notification is specified in the “Notification limit” on the Rubrics tab of assignment creation. The email that gets sent to instructors contains three links: a link the the conflicting review, a link the the summary page of grades for the reviewee, and a link to edit the assignment. Although the links sent in the email are helpful for reference, the process of examining a conflicting review could be made more efficient if we provide an easy to understand report of the conflict. The information sent in the email should be updated to contain a link to a report page which should contain more details about the conflict. In addition to creating a new conflict report page, we also want to change the algorithm to detect and trigger a notification if there is a significant difference between a submitted review and any previous submitted review for that assignment. Rather than compare the review score to the average of all review scores, an email should be sent whenever two review scores differ significantly for the same submission.

Design

Proposed Solution

What we need to modify:

• Related to Email:
  • App/mailers/mailer.rb in method notify_grade_conflict_message to make the link to the conflict report available in the email.
  • App/models/response.rb in method notify_instructor_on_difference to add a link to the conflict report that gets sent to the mailer.
  • App/views/mailer/notify_grade_conflict_message.html.erb to add to the message a link to view the conflict report.

• Related to the conflict report view:
  • App/views/reports/_searchbox.html.erb to add the option to view a conflict report.
  • App/views/reports/response_report.html.haml to render the conflict report partial if the option to view a conflict report was selected.

What we need to create:

• Related to the conflict report view:
  • App/views/reports/_conflict_report.html.erb to make the view for a conflict report

The conflict report view should:

• For each team that had a submission with a review that was in conflict:
  • List the team name and team members
  • For each round where there was a review in conflict:
    • List the students who's review caused a conflict and the score they gave
    • List the threshold for conflict review score, max review score, average review score, and std dev.

Useful Links

Github: https://github.com/sid189/expertiza/tree/beta
Pull Request: https://github.com/expertiza/expertiza/pull/1714

Team Information

Project Mentor:
Pratik Abhyankar

Project Members:
Sahil Papalkar
Sahil Mehta
Siddharth Deshpande
Carl Klier