CSC/ECE 517 Spring 2021 - E2112. Integrate Suggestion Detection Algorithm

Introduction

Team

Dr. Gehringer (mentor)

• Harsh Kachhadia (hmkachha)
• Parimal Mehta (pmehta3)
• Jatin Chinchkar (jchinch)
• Jordan Farthing (mjfarthi)

Project plan

Problem statement

Peer-review systems like Expertiza utilize a lot of students’ input to determine each other’s performance. At the same time, we hope students learn from the reviews they receive to improve their own performance. In order to make this happen, we would like to have everyone give quality reviews instead of generic ones. Currently we have a few classifiers that can detect useful features of review comments, such as how many suggestions they contain. The suggestion-detection algorithm has been coded as a web service, and other detection algorithms will be so coded in the future.

How it will work

In order to make the API call, the response_controller.rb will be responsible for sending a JSON input to the web service. The input will contain the review comment submitted by the user in the following format:

Below is a sample input

 Sample Input:
 {
 	"reviews":[{
 		"text":"review text here",
 		"metrics":["problem","suggestion","sentiment"]
 	},{
 		"text":"another review here",
 		"metrics":["problem"]
 	},{
 		"text":"more text, maybe a large para or whatever you like",
 		"metrics":["sentiment", "suggestion"]
 	},{
 		...
 	}]
 }

Once the request is send, we expect the output to be in the following format:

Sample Output: (as 'suggestion' and 'volume' as you might expect)

{ "sentiment_score": 0.9,

 "sentiment_tone": "Positive",
 "suggestions": "absent",
 "suggestions_chances": 10.17,
 "text": "This is an excellent project. Keep up the great work",
  "total_volume": 10,
  "volume_without_stopwords": 6}

The output (which is a JSON) will be parsed and the suggestion metrics such as the tone and presence of suggestion will be extracted so the user will be able to view a summarized result of how well their review comments were. In addition, an average score will be computed based on the scores they received for each comment section, and the result will be presented in a colorful format to the user after they hit the submit button.

Previous work and our Plan of Action

Spring 2019 pull request

• They had a functional suggestion detection API call that successfully communicated with the PeerLogic Server and retrieved the output, but as PeerLogic Server is no longer functioning, it needs to be re-deployed to new server(Peer Reviews NLP Server) and corresponding codes changes have to made.
• They included their API call in response_controller.rb using JavaScript, which need to be Refactored.
• They were able to display the cumulative output for the reviews on a new page. They displayed all of the information returned from the endpoint. But the display was not proper nor did it record or contain the time taken for to receive back the response from the Peer Reviews NLP Server.
• Since the code is added in the controller, testing needs to be performed too.
• Since the new files they added are too clunky, badly named, files are too long, etc. The previous work needs to be refactored and reworked upon to have a smooth and lag-free implementation.
• Expertiza team also has a new web service for problem detection. That needs to be integrated too.

Here are the various endpoints for the deployment of Suggestion Detection Algorithm.

• https://peer-reviews-nlp.herokuapp.com/all for all metrics at once
• https://peer-reviews-nlp.herokuapp.com/volume for volume metrics only
• https://peer-reviews-nlp.herokuapp.com/sentiment for sentiment metrics only
• https://peer-reviews-nlp.herokuapp.com/suggestions for suggestions metrics only

Flowchart to Describe Plan of Action