CSC/ECE 517 Spring 2020 - E2015. Conflict notification

From Expertiza_Wiki
Jump to navigation Jump to search

Abstract

Expertiza is an open-source web application implemented using Ruby on Rails. It is used for management of courses and the assignments for respective courses, by the faculties and the students. Students can form teams in Expertiza to work on an assignment in a group. A student team can submit their work through multiple means such as file uploads and embedded links. Submissions made on Expertiza can be peer-reviewed by students within the course based on a rubric. Results of the peer review can be accessed by instructors and respective student teams. During the peer evaluation, Expertiza sends appropriate email notifications to the instructor and also provides a complete statistical report of the evaluation.

Problem Statement

In the current implementation, during the peer review phase of an assignment, email notifications are sent out to the instructor whenever a submitted review score differs “significantly” from the average score of other submitted reviews for that submission. The threshold to trigger a notification is specified in the “Notification limit” on the Rubrics tab of assignment creation. The email that gets sent to instructors contains three links: a link the the conflicting review, a link the the summary page of grades for the reviewee, and a link to edit the assignment. Although the links sent in the email are helpful for reference, the process of examining a conflicting review could be made more efficient if we provide an easy to understand report of the conflict. The information sent in the email should be updated to contain a link to a report page which should contain more details about the conflict. In addition to creating a new conflict report page, we also want to change the algorithm to detect and trigger a notification if there is a significant difference between a submitted review and any previous submitted review for that assignment. Rather than compare the review score to the average of all review scores, an email should be sent whenever two review scores differ significantly for the same submission.

Existing Implementation

On submitting a new review for a particular artifact, the submitted score is compared to the average score of the previously submitted reviews. If the difference between the average score and the newly submitted score is more than the notification threshold specified for the assignment an email is sent out to the instructor. The email sent to the instructor contains the names of the reviewer and reviewee and a link to the conflicting review. The flowchart below delineates the entire process:

Problems

The email sent out to the instructor contains the following details:

  • Name of both the students - reviewer and reviewee
  • Newly submitted conflicting score
  • Link to view the conflicting review
  • Link to view the summary of all reviews
  • Link to change the notification threshold

Design

Proposed Solution

What we need to modify:

  • Related to Email:
    • App/mailers/mailer.rb in method notify_grade_conflict_message to make the link to the conflict report available in the email.
    • App/models/response.rb in method notify_instructor_on_difference to add a link to the conflict report that gets sent to the mailer.
    • App/views/mailer/notify_grade_conflict_message.html.erb to add to the message a link to view the conflict report.
  • Related to the conflict report view:
    • App/views/reports/_searchbox.html.erb to add the option to view a conflict report.
    • App/views/reports/response_report.html.haml to render the conflict report partial if the option to view a conflict report was selected.


What we need to create:

  • Related to the conflict report view:
    • App/views/reports/_conflict_report.html.erb to make the view for a conflict report


The conflict report view should:

  • For each team that had a submission with a review that was in conflict:
    • List the team name and team members
    • Provide a graphical view of the statistics displayed in the report
    • For each round where there was a review in conflict:
      • List the students who's review caused a conflict and the score they gave
      • List the threshold for conflict review score, max review score, average review score, and standard deviation.

Why we chose this approach?

Use Case Diagram

Actors participating:

  • Student: In this use case, fills out the reviews for different projects submitted by the peers.
  • TA/Instructor: In this use case, goes over the conflicting review submitted by the students for the same project.

Testing

Test Plan

We intend to break down our test plan into 3 phases:

1. Run existing RSpec Tests, make sure those pass.

2. Create new RSpec Test to check if there exists any significant difference between the old and the new code.

3. Manually test the application to see if the code breaks anywhere after the new code is written.

Useful Links

Github: https://github.com/sid189/expertiza/tree/beta
Pull Request: https://github.com/expertiza/expertiza/pull/1714

Team Information

Project Mentor:
Pratik Abhyankar

Project Members:
Sahil Papalkar
Sahil Mehta
Siddharth Deshpande
Carl Klier

Retrieved from "https://wiki.expertiza.ncsu.edu/index.php?title=CSC/ECE_517_Spring_2020_-_E2015._Conflict_notification&oldid=133047"