CSC/ECE 517 Fall 2021 - E2150. Integrate suggestion detection algorithm

From Expertiza_Wiki
Revision as of 23:33, 2 November 2021 by Ddoshi2 (talk | contribs)
Jump to navigation Jump to search

Problem Defination

Peer-review systems like Expertiza utilize a lot of students’ input to determine each other’s performance. At the same time, students learn from the reviews they receive to improve their own performance. In order to make this happen,it would be good to have everyone give quality reviews instead of generic ones. Currently, Expertiza has a few classifiers that can detect useful features of review comments, such as whether they contain suggestions. The suggestion-detection algorithm has been coded as a web service, and other detection algorithms, such as problem detection and sentiment analysis, also exist as newer web services. With the previous implementation of this project, student can write comments and request the fee


Tasks

There are Several tasks that we are looking to address by the end of the project which are seen below:

  1. When a student submits a review (in the response_controller), we would like to call this web service with the student’s review as the input. We would then want to tell the student whether their reviews contain suggestions or not, so they can make improvements based on the results of the webservice.
    • The response_controller should pass a text string (the comments field of each Answer object) to the web service.
    • In fact, the response_controller could pass this string to a whole set of metrics (volume, sentiment, problem detection, suggestion counter).
  2. We would also like to evaluate how much time this API is taking and if possible work a way out to improve it. We don’t want the system to be terribly slow.
  3. We also have a web service for problem detection. That should be integrated too.

Issues to be Addressed

  1. Update how current implementation gets data from the web service.
    • Yulin's code calls another module that grabs the data from web services in bulk, and divvies it out one item at a time.
    • How would you call it to get suggestion confidence and problem-detection confidence level, etc.?
  2. Reading a data structure that tells which metrics we are interested in now, and therefore which need to be shown in the view.
    • For example, the data structure may say, For this assignment, we are interested in
      • problem detection
      • suggestions, and
      • sentiment analysis.
    • For another assignment, it might be
      • localization and
      • helpfulness
  3. Displaying the information in an appropriate view.


Comments on Prior Teams Implementation

  • The criteria are numbered in the view, and those numbers do not correspond to anything on the rubric form.
    • So if the rubric is long, it would be quite difficult for the reviewer to figure out what automated feedback referred to which comment
  • There is too much specific information on metrics that is encoded into the text.
    • While some of the info is in configuration files, the help text for the info buttons is in the code. Also, calls for each metric are programmed into the code, though, depending on how diverse they are, this is perhaps unavoidable
  • The code can be more modularized.
    • Currently there are many repetitive blocks of code. For ex; in _response_analysis.html for the getReviewFeedback(), the API call for each type of tag (sentiment, suggestion, etc) is being repeated. Only the API link differs there. So it might be more beneficial to store the API link in the config file and use the variable in the function

API Endpoints

JSON Formatting

  • Input Text is passed in the following JSON format
 { 
     "text" : "This is an excellent project. Keep up the great work" 
 }
  • Output is returned in the following JSON format:
 {          
     "sentiment_score": 0.9,
     "sentiment_tone": "Positive",
     "suggestions": "absent",
     "suggestions_chances": 10.17,
     "text": "This is an excellent project. Keep up the great work",
     "total_volume": 10,
     "volume_without_stopwords": 6
  }

Team

  • Prashan Sengo (psengo)
  • Griffin Brookshire ()
  • Divyang Doshi ()
  • Srujan ()


Relevant Links

Retrieved from "https://wiki.expertiza.ncsu.edu/index.php?title=CSC/ECE_517_Fall_2021_-_E2150._Integrate_suggestion_detection_algorithm&oldid=140486"