CSC/ECE 517 Spring 2021 - E2112. Integrate Suggestion Detection Algorithm

From Expertiza_Wiki
Revision as of 17:21, 29 April 2021 by Hmkachha (talk | contribs)
Jump to navigation Jump to search

Introduction

Team

Dr. Gehringer (mentor) John Bumgardner (mentor)

  • Harsh Kachhadia (hmkachha)
  • Parimal Mehta (pmehta3)
  • Jatin Chinchkar (jchinch)
  • Jordan Farthing (mjfarthi)

Project Background

Peer-review systems like Expertiza utilize a lot of students’ input to determine each other’s performance. At the same time, we hope students learn from the reviews they receive to improve their own performance. In order to make this happen, we would like to have everyone give quality reviews instead of generic ones. Currently we have a few classifiers that can detect useful features of review comments, such as whether they contain suggestions. The suggestion-detection algorithm has been coded as a web service, and other detection algorithms, such as problem detection and sentiment analysis, also exist as newer web services..but they need to be integrated properly using API calls in expertiza code.

How it works

In order to make the API call, the partial view "_response_analysis.html.erb" is rendered in "response.html.erb" view file which will be responsible for sending a JSON input to the web service. The input will contain the review comment written by the user and when the student hits the "Get review feedback button" the comments will be sent to these api calls in the following json format:

Below is a sample input 

 Sample Input:

   {"reviews" : [
   { 
         "id" : 1,
         "text" : "this is a sample test. I am writing a long test just to make sure api if working as expected. Earlier had some issue processing the test due to size of the test used in the request api."
   },
   {
         "id" : 2,        
         "text" : "this is a sample test. I am writing a long test just to make sure api if working as expected. Earlier had some issue processing the test due to size of the test used in the request api."
   } 
               ]}

Once the request is sent, we expect the output to be in the following format:

Sample Output: (for sentiment analysis API call) 

   {"sentiments": [
       {
           "id": 1,
           "neg": "0.00",
           "neu": "0.95",
           "pos": "0.05",
           "sentiment": "0.11",
           "text": "this is a sample test. I am writing a long test just to make sure api if working as expected. Earlier had some issue processing the test due to size of the test used in the request api."
       },
       {
           "id": 2,
           "neg": "0.00",
           "neu": "0.95",
           "pos": "0.05",
           "sentiment": "0.11",
           "text": "this is a sample test. I am writing a long test just to make sure api if working as expected. Earlier had some issue processing the test due to size of the test used in the request api."
       }
   ]}

Sample Output: (for problem detection API call) 

   {"reviews": [
       {
           "id": 1,
           "problems": "Present",
           "text": "this is a sample test. I am writing a long test just to make sure api if working as expected. Earlier had some issue processing the test due to size of the test used in the request api."
       },
       {
           "id": 2,
           "problems": "Present",
           "text": "this is a sample test. I am writing a long test just to make sure api if working as expected. Earlier had some issue processing the test due to size of the test used in the request api."
       }
   ]}

Sample Output: (for suggestion detection API call) 

   {"reviews": [
       {
           "id": 1,
           "suggestions": "Present",
           "text": "this is a sample test. I am writing a long test just to make sure api if working as expected. Earlier had some issue processing the test due to size of the test used in the request api."
       },
       {
           "id": 2,
           "suggestions": "Present",
           "text": "this is a sample test. I am writing a long test just to make sure api if working as expected. Earlier had some issue processing the test due to size of the test used in the request api."
       }
   ]}

These outputs (which is a JSON) will be parsed and the concerned metrics such as the sentiment, problem and suggestion will be extracted so the user will be able to view a summarized result of how well their review comments are. In addition, the result will be presented in a colorful tabular format to the user after they hit the "Get Review Feedback" button. Also, the total time taken for making these API calls will be displayed below the table.

Flowchart to Describe the Flow of control


Overview of how we achieved this

Initially, we had planned to work on the work already done by the previous students Spring 2019 pull request. But later on we faced many issues such as: 1) The API call links were outdated 2) The new link that we found were incompatible with the previous work 3) The planned task of adding new API calls as per Carl Colglaizer's Framework turned out to be irrelevant or not required for this project, so we dropped it.

So we decided to start with integrating these API Calls from scratch.


Features we added in this project

  • Setting up a config file 'review_metric.yml' where instructor can select what review metric to display for the current assignments
  • Based on the selection made by professor, API calls(sentiment, problem, sugegstion) are made and a colorful table is displayed below the review form for student to review
  • The total time taken for making these API calls will be displayed below the table.

Files that are modified or added in this project

  • review_metrics.yml
  • response.html.erb
  • _rseponse_analysis.html.erb
  • response_controller.rb
  • load_config.rb

New Javascript Functions that we added

In the partial view file _response_analysis.html.erb file, we added new javascript functions to make, process and display output of API calls.

  • Frontend: This image shows the flow of control for a student reviewer.

  • Frontend: The image shows the information regarding the reviews sent by the APIs to the student reviewer

API Details/Endpoints Appendix I

Here are the various endpoints for the deployment of Suggestion Detection Algorithm.

(We can't make the API links unclickable for this design doc, but clicking on them won't lead you anywhere. They are just endpoints and are mentioned here for reference only.)

Testing plan

We aims to perform extensive testing for this project in order to achieve better reliability for this implementation.

Projects files that will be changed(projected) and will be tested:

  • _response_analysis.html.erb

Tests

  • The functionality was written client side in javascript solely in _response_analysis.html.erb
  • To test this view, any type of review must be accessible as a student
  • There is a button at the bottom of the review called Get Review Feedback.
  • When pressing button, API calls are issued and the metrics will show up within the table.
  • API calls are slow and will take time to process on click
  • Save review will allow for a new metric calculation to occur
  • All functionality associated with this table will be displayed

View Tests

Retrieved from "https://wiki.expertiza.ncsu.edu/index.php?title=CSC/ECE_517_Spring_2021_-_E2112._Integrate_Suggestion_Detection_Algorithm&oldid=138735"