CSC/ECE 517 Fall 2021 - E2150. Integrate suggestion detection algorithm

Problem Definition

Peer-review systems like Expertiza utilize a lot of students’ input to determine each other’s performance. At the same time, students learn from the reviews they receive to improve their own performance. In order to make this happen, it would be good to have everyone give quality reviews instead of generic ones. Currently, Expertiza has a few classifiers that can detect useful features of review comments, such as whether they contain suggestions. The suggestion-detection algorithm has been coded as a web service, and other detection algorithms, such as problem detection and sentiment analysis, also exist as newer web services. We need to make the UI more intuitive by allowing users to view the feedback of specific review comments and the code needs to be refactored to remove redundancy to follow the DRY principle.

Previous Implementation

Overview

The previous implementation added the following features:

1. Setup a config file 'review_metric.yml' where the instructor can select what review metric to display for the current assignments
2. API calls (sentiment, problem, suggestion) are made and a table is rendered below, displaying the feedback of the review comments.
3. The total time taken by the API calls was also displayed.

UI Screenshots

The following image shows how a reviewer interacts with the system to get feedback on the review comments.

Control Flow

Issues with Previous Work

With the previous implementation of this project, students can write comments and request feedback for the comments. There are certain issues with the previous implementation that needs to be addressed.

1. The criteria are numbered in the view, and those numbers do not correspond to anything on the rubric form. So if the rubric is long, it would be quite difficult for the reviewer to figure out what automated feedback referred to which comment.
2. Too much specific information on metrics is encoded into the text. While some of the info is in configuration files, the help text for the info buttons is in the code.
3. Currently there are many repetitive blocks of code. For Ex.; in _response_analysis.html for the getReviewFeedback(), the API call for each type of tag (sentiment, suggestion, etc) is being repeated. Only the API link differs there.

Proposed Solution

Refactoring

previous Implementation

1. We need to improve the UI so that if there are many rubric items, users can easily check the feedback for each comment given to each rubric item. We propose the following solution
   1. Each comment on hover shows the user's respective comment on the specific rubric review question so that the feedback is easily accessible.

LOFI Design of Proposed Solution

1. Another issue with the current implementation is that code is quite repetitive for making different API calls (Sentiment, Suggestion, etc). We propose to store the API link in the config file and use the variable in the getReviewFeedback()
2. We also plan to remove a global variable response_general, which is being used to store the response of API calls. We will refactor the makeRequest function to directly return the response which can be used in various places. This will resolve implicit coupling issues in the code and make it more easily extendible.
3. The previous implementation has hardcoded configuration information like the help text button. We propose to save this information in config files which can be easily modified and will remove unnecessary information from the code.

Possible files to be edited

1. review_metrics.yml
2. response.html.erb
3. _response_analysis.html.erb
4. response_controller.rb
5. response_controller_spec.rb

Sample API Input/Output

• Input Text is passed in the following JSON format

 { 
     "text": "This is an excellent project. Keep up the great work" 
 }

• Output is returned in the following JSON format:

 {          
     "sentiment_score": 0.9,
     "sentiment_tone": "Positive",
     "suggestions": "absent",
     "suggestions_chances": 10.17,
     "text": "This is an excellent project. Keep up the great work",
     "total_volume": 10,
     "volume_without_stopwords": 6
  }

Testing Plan

This project mainly addresses the UI issues and some code refactoring in views. Therefore, we will be focussing on view tests as all the automated tests have been previously implemented and new code won't affect those.

View Tests

1. The functionality was written on the client side in javascript solely in _response_analysis.html.erb
2. To test this view, any type of review must be accessible as a student.
3. There is a button at the bottom of the review called 'Get Review Feedback'.
4. When pressing button, API calls are issued and the metrics will show up within the table (a sample of which is shown in above screenshot).
5. As API calls will take time, 'Loading..' text will appear until the API calls are complete
6. All the review feedback for the comments will be displayed in a colorful table.
7. In the feedback table, upon hovering on the comment number, we will be able to see the rubric item and review comments associated with that particular rubric item

Important Links

1. https://docs.google.com/document/d/1slx4HPIbgTH-psIKMSCF-HDF9brxf-FuYhzVT9ZiIrM/edit#heading=h.fxfungdw1d5r
2. https://github.com/expertiza/expertiza/pull/1952 (Previous implementation pull request)
3. https://www.youtube.com/watch?v=OYZvNVa3GZ8&feature=youtu.be (Youtube video for previous implementation)

Team

• Prashan Sengo (psengo)

• Griffin Brookshire (glbrook2)

• Divyang Doshi (ddoshi2)

• Srujan (sponnur)