CSC/ECE 517 Spring 2018- Project E1820 Review-Comment Tone Analysis Report

Introduction

Team

Peter Shore (pnshore)

Vishy Makthal (vmaktha)

Cameron Nelson (cenelso3)

Robert Montgomery (rjmontgo)

Background Information

Expertiza

Expertiza is an open source project that can be used by instructors to create assignments and assign them to students. Students can manage teams for the assignment and submit work for each of the assignments. After the students have submitted their work they can review other student's work to facilitate grading. The project is based on the Ruby on Rails framework. You can find our GitHub Repo.

Tone Analysis

Tone analysis is a tool that is used to measure the sentiment or tone of a sentence. This will be used in expertiza to help determine the tone of the review comments to determine if the feedback is useful. Since implementing a working tone analysis tool would be a project on its own we will use one that is already implemented through an NC State University project called Peer Logic.

Peer logic is going to be used to implement tone analysis throughout this project. The peer logic web service is able to provide colored HTML text based on the tone by sending a POST request of a JSON {“review”: “review comments….”} to an API endpoint[1]. The JSON data that is returned will be used to determine coloration of the cells that are displayed in the popup views as well as in the table. For reference, Peer logic assigns green to a positive tone, yellow to a moderate tone, and red to a negative tone.

Problem Statement

The basis of this project was to clean up the table in view_review_scores_popup_.html.rb. Currently, the table is very simple and unorganized, reducing its readability. There are incomplete table borders, misaligned cells, and HTML tags littered throughout the text. In addition, tabs were added to separate each review round. Another feature that was added was tone analysis of the comments. A colored tone analysis is retrieved from the Peer Logic web service [2] and displayed accordingly. This view was implemented as a pop up as well, showing a tile char visualization depicting points of review on the y-axis, and a color analysis of their review on the x-axis.

Files Modified

view/popup/view_review_scores_popup.html.erb

view/review_mapping/_answer_tagging_report.html.erb

Design Document

UML

The UML diagram regarding the scope of our project is included below. Overall, an existing view will be modified, and a new one will be created. Both of these views are associated with the popup_controller, which, as a result, will also be modified accordingly.

Wireframes

These wireframes show the flow of the user to each of the updated views. You can see the implementation of the colored grid showing the tone analysis report. You can also see that clicking on one of the grid squares shows the full text of the comment.

You can also see that clicking on the summary link under the 'reviews done' column will take you to a summary of the Review Scores page which shows the highlighted text based on the tone analysis report.

Test Plan

Test 1: PopulateToneAnalysisComments

Feature: As an Admin, Super Admin, or Instructor
                 I want to view the tone of every comment a student reviewer has given a team per
    question
    So that I can grade the student reviewer based on the tone of their comments

Scenario: Populate Tone Analysis Comments Instructor Given the user logs in as an Instructor with Username instructor6 and Password password When the instructor clicks Assignments under Manage Notifications And clicks the View Review Report icon in the assignment OSS project/Writing assignment 2 And clicks Summary under the Reviews Done column for student reviewer student5061 Then a pop up page will show all the review scores and comments that student5061 has contributed and it will be ordered by review round Scenario: Populate Tone Analysis Comments Instructor Given the user logs in as an Admin with Username admin1 and Password password When the instructor clicks Assignments under Manage Notifications And clicks the View Review Report icon in the assignment Expertiza Demo And clicks Summary under the Reviews Done column for student reviewer student5402 Then a pop up page will show all the review scores and comments that student5402 has contributed and it will be ordered by review round Scenario: Populate Tone Analysis Comments Instructor Given the user logs in as a Super Admin with Username super_administrator2 and Password password When the instructor clicks Assignments under Manage Notifications And clicks the View Review Report icon in the assignment Expertiza Demo And clicks Summary under the Reviews Done column for student reviewer student4542 Then a pop up page will show all the review scores and comments that student4542 has contributed and it will be ordered by review round Test 2: ViewPositiveHeatMapMatrixChart Feature: As an Admin, Super Admin, or Instructor

I want to view the visualization of the tone of every comment a student reviewer has
given a team for every question per assignment
So that I give the student reviewer a good grade for their positive comments

Scenario: View Heat Map Matrix Chart Instructor Given the user logs in as an Instructor with Username instructor6 and Password password When the instructor clicks Assignments under Manage Notifications And clicks the View Review Report icon in the assignment OSS project/Writing assignment 2 And clicks View Tone Analysis under the Metrics column for student reviewer student5061 Then a pop up page will show the mapping of all comments the student reviewer gave to each team for every question per assignment and it will be grouped by review rounds Scenario: View Heat Map Matrix Chart Admin Given the user logs in as an Admin with Username admin1 and Password password When the instructor clicks Assignments under Manage Notifications And clicks the View Review Report icon in the assignment Expertiza Demo And clicks View Tone Analysis under the Metrics column for student reviewer student5402 Then a pop up page will show the mapping of all comments the student reviewer gave to each team for every question per assignment and it will be grouped by review rounds Scenario: View Heat Map Matrix Chart Super Admin Given the user logs in as a Super Admin with Username super_administrator2 and Password password When the instructor clicks Assignments under Manage Notifications And clicks the View Review Report icon in the assignment Expertiza Demo And clicks View Tone Analysis under the Metrics column for student reviewer Student4542 Then a pop up page will show the mapping of all comments the student reviewer gave to each team for every question per assignment and it will be grouped by review rounds Test 3: ViewNegativeHeatMapMatrixChart Feature: As an Admin, Super Admin, or Instructor

I want to view the visualization of the tone of every comment a student reviewer has
given a team for every question per assignment
So that I give the student reviewer a bad grade for their negative comments

Scenario: View Heat Map Matrix Chart Instructor Given the user logs in as an Instructor with Username instructor6 and Password password When the instructor clicks Assignments under Manage Notifications And clicks the View Review Report icon in the assignment OSS project/Writing assignment 2 And clicks View Tone Analysis under the Metrics column for student reviewer student5061 Then a pop up page will show the mapping of all comments the student reviewer gave to each team for every question per assignment and it will be grouped by review rounds Scenario: View Heat Map Matrix Chart Admin Given the user logs in as an Admin with Username admin1 and Password password When the instructor clicks Assignments under Manage Notifications And clicks the View Review Report icon in the assignment Expertiza Demo And clicks View Tone Analysis under the Metrics column for student reviewer student5402 Then a pop up page will show the mapping of all comments the student reviewer gave to each team for every question per assignment and it will be grouped by review rounds Scenario: View Heat Map Matrix Chart Super Admin Given the user logs in as a Super Admin with Username super_administrator2 and Password password When the instructor clicks Assignments under Manage Notifications And clicks the View Review Report icon in the assignment Expertiza Demo And clicks View Tone Analysis under the Metrics column for student reviewer Student4542 Then a pop up page will show the mapping of all comments the student reviewer gave to each team for every question per assignment and it will be grouped by review rounds Test 4: ViewNeutralHeatMapMatrixChart Feature: As an Admin, Super Admin, or Instructor

I want to view the visualization of the tone of every comment a student reviewer has
given a team for every question per assignment
So that I give the student reviewer a fair grade for their positive/negative comments

Scenario: View Heat Map Matrix Chart Instructor Given the user logs in as an Instructor with Username instructor6 and Password password When the instructor clicks Assignments under Manage Notifications And clicks the View Review Report icon in the assignment OSS project/Writing assignment 2 And clicks View Tone Analysis under the Metrics column for student reviewer student5061 Then a pop up page will show the mapping of all comments the student reviewer gave to each team for every question per assignment and it will be grouped by review rounds Scenario: View Heat Map Matrix Chart Admin Given the user logs in as an Admin with Username admin1 and Password password When the instructor clicks Assignments under Manage Notifications And clicks the View Review Report icon in the assignment Expertiza Demo And clicks View Tone Analysis under the Metrics column for student reviewer student5402 Then a pop up page will show the mapping of all comments the student reviewer gave to each team for every question per assignment and it will be grouped by review rounds Scenario: View Heat Map Matrix Chart Super Admin Given the user logs in as a Super Admin with Username super_administrator2 and Password password When the instructor clicks Assignments under Manage Notifications And clicks the View Review Report icon in the assignment Expertiza Demo And clicks View Tone Analysis under the Metrics column for student reviewer Student4542 Then a pop up page will show the mapping of all comments the student reviewer gave to each team for every question per assignment and it will be grouped by review rounds Test 5: ViewBlankReviewScoresPopupPage Feature: As an Admin, Super Admin, or Instructor

I want to view the review scores popup page of a student reviewer
So that I can give the student reviewer a zero for not doing any peer reviews for an
assignment

Scenario: Populate Tone Analysis Comments Instructor Given the user logs in as an Instructor with Username instructor6 and Password password When the instructor clicks Assignments under Manage Notifications And clicks the View Review Report icon in the assignment WCAE 2017 And clicks Summary under the Reviews Done column for student reviewer student515 Then a pop up page will show nothing that student515 contributed to Scenario: Populate Tone Analysis Comments Admin Given the user logs in as an Admin with Username admin1 and Password password When the instructor clicks Assignments under Manage Notifications And clicks the View Review Report icon in the assignment UWOU demo And clicks Summary under the Reviews Done column for student reviewer student4304 Then a pop up page will show nothing that student4304 contributed to Scenario: Populate Tone Analysis Comments Super Admin Given the user logs in as a Super Admin with Username super_administrator2 and Password password When the instructor clicks Assignments under Manage Notifications And clicks the View Review Report icon in the assignment UWOU Demo And clicks Summary under the Reviews Done column for student reviewer student4304 Then a pop up page will show nothing that student4304 contributed to Test 6: ViewBlankHeatMapMatrixChart Feature: As an Admin, Super Admin, or Instructor

I want to view a gray colored visualization of the tone analysis for a student reviewer
So that I know that the student did not contribute to peer reviewing for an assignment

Scenario: View Heat Map Matrix Chart Instructor Given the user logs in as an Instructor with Username instructor6 and Password password When the instructor clicks Assignments under Manage Notifications And clicks the View Review Report icon in the assignment Madeup problem And clicks View Tone Analysis under the Metrics column for student reviewer Student7350 Then a pop up page will show the heat map matrix with grayed out blocks specifying that the student reviewer did not comment on some or all questions for the assignment and it will be grouped by review rounds Scenario: View Heat Map Matrix Chart Admin Given the user logs in as an Admin with Username admin1 and Password password When the instructor clicks Assignments under Manage Notifications And clicks the View Review Report icon in the assignment Team test 2 And clicks View Tone Analysis under the Metrics column for student reviewer Student431 Then a pop up page will show the heat map matrix with grayed out blocks specifying that the student reviewer did not comment on some or all questions for the assignment and it will be grouped by review rounds Scenario: View Heat Map Matrix Chart Super Admin Given the user logs in as a Super Admin with Username super_administrator2 and Password password When the instructor clicks Assignments under Manage Notifications And clicks the View Review Report icon in the assignment Team test 2 And clicks View Tone Analysis under the Metrics column for student reviewer Student431 Then a pop up page will show the heat map matrix with grayed out blocks specifying that the student reviewer did not comment on some or all questions for the assignment and it will be grouped by review rounds