CSC/ECE 517 Spring 2021 - E2112. Integrate Suggestion Detection Algorithm

Introduction

Team

Dr. Gehringer (mentor), John Bumgardner (mentor)

• Harsh Kachhadia (hmkachha)
• Parimal Mehta (pmehta3)
• Jatin Chinchkar (jchinch)
• Jordan Farthing (mjfarthi)

Project Background

Peer-review systems like Expertiza utilize a lot of students’ input to determine each other’s performance. At the same time, we hope students learn from the reviews they receive to improve their own performance. In order to make this happen, we would like to have everyone give quality reviews instead of generic ones. Currently we have a few classifiers that can detect useful features of review comments, such as whether they contain suggestions. The suggestion-detection algorithm has been coded as a web service, and other detection algorithms, such as problem detection and sentiment analysis, also exist as newer web services..but they need to be integrated properly using API calls in expertiza code.

Implementation Overview

Initially, we had planned to work on the work already done by the previous students Spring 2019 pull request. But later on we faced many issues such as: 1) The API call links were outdated 2) The new link that we found were incompatible with the previous work 3) The planned task of adding new API calls as per Carl Colglaizer's Framework turned out to be irrelevant or not required for this project, so we dropped it.

So we decided to start with integrating these API Calls from scratch.

Features we added in this project

• Setting up a config file 'review_metric.yml' where instructor can select what review metric to display for the current assignments
• Based on the selection made by professor, API calls (sentiment, problem, sugegstion) are made and a colorful table is displayed below the review form for student to review
• The total time taken for making these API calls will be displayed below the table.

Files that are modified or added in this project

• review_metrics.yml
• response.html.erb
• _response_analysis.html.erb
• response_controller.rb
• load_config.rb
• response_controller_spec.rb

Design Pattern

In order to achieve the primary tasks of integrating the API along with making the application more extensible, the team implemented a more extensive application of the Facade design pattern to decouple the details of the calling the APIs from the caller method (here - makeArequest method). This design pattern helped us achieve the decoupling and abstraction of implementation code base (makeARequest Function) of API call from the calling function (getReviewFeedback function). Later, refactoring of _response_analysis.html.erb partial further decoupled the implementation. Thus, in a nutshell, application of facade pattern along with some refactoring lead to a decoupled implementation of integration of all 3 API calls.

UI Screenshots

• Frontend: This image shows the flow of control for a reviewer.

Control Flow

To inspect implementation in detail, check out the 'Javascript Functionality'

Javascript Functionality

In the partial view file _response_analysis.html.erb file, we added new javascript functions to make, process and display output of API calls.

• review_metric.yml config file

In this file, the configuration for this functionality on which metrics to display in the table, can be set by the
instructor. Instructor can change which metric to display or not display in feedback by changing the value for that
particular metric as 'true' or 'false'.

• response_controller.rb

In this file, we added a method 'fetch_review_metric' which fetches the configurations set by instructor in the review_metric.yml file by 
the instructor and passes those fetched values to the response.html.erb view using the 'new' or 'edit' method.

• response.html.rb

Here, we added a line that renders the partial _response_analysis.html.erb after the Save button to show the 'get review feedback' button and 
display the metrics table. Also, we fetch the passed configuration data here.

• _response_analysis.html.erb - fetch_response_comments() function

This function fetches the comments written by the user in the review form and formats the collected comments as per the expectation of the API
 calls.

• _response_analysis.html.erb - getReviewFeedback() function

This function makes calls to respective APIs and get their output based on the configuration received from the review_metrics.yml file.

• _response_analysis.html.erb - makeARequest() function

This is the driver function for making API call request to any url and data that is passed to it.

• _response_analysis.html.erb - combine_api_output() function

This function combines the output received from different API responses, combines them and formats the combined data to one that is suitable for 
table generation.

• _response_analysis.html.erb - generateTable() function

This code generates a colorful table dynamically on the UI so that the reviewer can get a visual feedback on the comments that he/she wrote.

API Details/Endpoints Appendix I

Here are the various endpoints for the deployment of Suggestion Detection Algorithm.

(We can't make the API links unclickable for this design doc, but clicking on them won't lead you anywhere. They are just endpoints and are mentioned here for reference only.)

• http://152.7.99.200:5000/problem for problem metrics only
• https://peerlogic.csc.ncsu.edu/sentiment/analyze_reviews_bulk for sentiment metrics only
• http://152.7.99.200:5000/suggestions for suggestions metrics only

API Action

In order to make the API call, the partial view "_response_analysis.html.erb" is rendered in "response.html.erb" view file which will be responsible for sending a JSON input to the web service. The input will contain the review comment written by the user and when the student hits the "Get review feedback button" the comments will be sent to these api calls in the following json format:

Below is a sample input

 Sample Input:

   {"reviews" : [
   { 
         "id" : 1,
         "text" : "Comment 1."
   },
   {
         "id" : 2,        
         "text" : "Comment 2."
   } 
               ]}

Once the request is sent, we expect the output to be in the following format:

Sample Output: (for sentiment analysis API call)

   {"sentiments": [
       {
           "id": 1,
           "neg": "0.00",
           "neu": "0.95",
           "pos": "0.05",
           "sentiment": "0.11",
           "text": "Comment 1."
       },
       {
           "id": 2,
           "neg": "0.00",
           "neu": "0.95",
           "pos": "0.05",
           "sentiment": "0.11",
           "text": "Comment 2."
       }
   ]}

Sample Output: (for problem detection API call)

   {"reviews": [
       {
           "id": 1,
           "problems": "Present",
           "text": "Comment 1."
       },
       {
           "id": 2,
           "problems": "Present",
           "text": "Comment 2."
       }
   ]}

Sample Output: (for suggestion detection API call)

   {"reviews": [
       {
           "id": 1,
           "suggestions": "Present",
           "text": "Comment 1."
       },
       {
           "id": 2,
           "suggestions": "Present",
           "text": "Comment 2."
       }
   ]}

These outputs (which is a JSON) will be parsed and the concerned metrics such as the sentiment, problem and suggestion will be extracted so the user will be able to view a summarized result of how well their review comments are. In addition, the result will be presented in a colorful tabular format to the user after they hit the "Get Review Feedback" button. Also, the total time taken for making these API calls will be displayed below the table.

Testing plan

We aim to perform automatic and manual testing for this project in order to achieve better reliability for this implementation.

As for this project, very few lines of code have been written in ruby (fetch_review_metric method in response_controller.rb) we will be testing that method in response controller using rspec tests.

Rspec tests for the same have been written in response_controller_spec.rb

View Tests

• The functionality was written on the client side in javascript solely in _response_analysis.html.erb
• To test this view, any type of review must be accessible as a student.
• There is a button at the bottom of the review called 'Get Review Feedback'.
• When pressing button, API calls are issued and the metrics will show up within the table (a sample of which is displayed below).
• API calls are slow and will take time to process until the 'Loading...' text disappears.
• You can modify the comments and click the 'Get Review Feedback' button again to get new feedback, that too can be achieved without the need of saving the review, but still saving the review first is a better option to approach this.
• All the review feedback for the comments will be displayed in a colorful table.

Sample image of the table

Important Links

• Github Pull Request
• Expertiza wiki page link (the link is mentioned here for records. It redirects to this same page.)
• Google Doc Project Description
• Github Repository Link for this expertiza fork (make sure you change branch to 'beta' branch if the page doesn't load by default to beta branch)
• Youtube Video Link