CSC/ECE 517 Fall 2018/E1878 Integrate Suggestion Detection Algorithm

CSC-517 Fall 2018 - E1878 Integrate suggestion detection algorithm in Expertiza Project. This Wiki page explains the integration of the suggestion detection algorithm in Expertiza project.

Introduction

Team

1. Ameya Dhavalikar
2. Komal Kangutkar
3. Prashanthi Kanniappan Murthy
4. Vibhav Nandavaram Abbai Srivaths

Background information

Expertiza

Expertiza is an Open Source project based on the Ruby on Rails framework, supported by National Science Foundation. It is the software to create reusable learning objects through peer review. It is a project where students can submit and peer review learning objects(articles, code, websites, etc). The users of this software include students and professors. Expertiza is used by select professors and students in North Carolina State University, for example. It supports team projects, reviews of projects/teammates, submission URLs, Wiki pages and certain document types. Instructors can create projects and the students can bid for the projects. The students can be assigned teams for a particular project or they may form their own team with fellow classmates.

Suggestion detection algorithm

Peer-review systems like Expertiza utilize a lot of students’ input to determine each other’s performance. In the same time, we hope students could also gain knowledge from the reviews received thus improve their own performance.

Currently, we have a few classifiers that could catch useful components of review comments, such as if it contains suggestions, etc. These classifiers are already ported into web services that we’d like to be integrated into Expertiza. As stated in the Problem statement, we get the response from the REST endpoints as given and integrate it with Expertiza. The suggestion detection algorithm from which we are getting the metrics is added in the "References" section in case more details are needed on it. According to the problem statement, we will be considering the "Suggestion Detection Algorithm" to be a black box, which take a json with text as input and returns a json with few metrics as given below.

Input JSON to "Suggestion Detection Algorithm"

 { "text" : "This is an excellent project. Keep up the great work" }

Output JSON the "Suggestion Detection Algorithm"

  {"sentiment_score": 0.9,
 "sentiment_tone": "Positive",
 "suggestions": "absent",
 "suggestions_chances": 10.17,
 "text": "This is an excellent project. Keep up the great work",
  "total_volume": 10,
  "volume_without_stopwords": 6}

Problem statement

1. When a student submits a review, we call this web service with the student’s review as the input. We then tell the student whether their reviews contain suggestions or not, so they can make improvements based on the results of the web service.
2. We evaluate how much time this API is taking. We don’t want the system to be terribly slow.

Files modified

1. app/views/response/response.html.erb

Wireframes

Current State

In its current state, the student directly writes their review in segmented text boxes under instructor specified questions with no further checks about the relevance of the review they have written as shown below:

UML Diagrams

We are not modifying any models / adding any logic that requires an UML diagram. Most of our changes are on HTML. Our logic to be implemented is pre-processing the review and sending it as JSON to the Suggestion Detection API, getting the JSON response from API, post-processing the metrics to output the Suggestion Analysis.

The overall flow of the logic is explained as follows:

Implementation

Pre-requisites

Following are the necessary steps needed to be set up in order to have the feature running :

1. Browser: Google Chrome
2. Cores plug-in (link in references)

Before the JSON response is displayed, certain fields are first filtered out or formatted. For instance, the field "text" is filtered out as it is redundant to display it again. The field "sentiment_tone" is formatted with the help of another font color JSON depending on its value. Following is the code snippet of our implementation :

//Suggestion detection API call
      tinyMCE.on('AddEditor', function(e) {
      e.editor.on('blur', function (e) {
        ed = e.target;
        if (ed.id.includes("comments")){
          json_review = JSON.stringify({ "text": ed.getContent({format : 'text'}) })
          $.ajax({
            url: "https://peer-reviews-nlp.herokuapp.com/all",
            type: "POST",
            crossDomain: true,
            contentHeader: "application/javascript",
            contentType: "application/json",
            data: json_review,
            dataType: "json",
            //Response JSON fields are filtered and then displayed
            
            success: function (response) {
	    	console.log("success");
		var APIresultbox ="";
		var colors = {"Negative": ":</b> <font style=\"color:red;\">", 
				"Positive": ":</b> <font style=\"color:green;\">", 
				"Neutral": ":</b> <font style=\"color:saddlebrown;\">", 
				"Mixed": ":</b> <font style=\"color:saddlebrown;\">"};
		APIresultbox += "<table border='1'>"
                //Add the response to the API result box
        	for (x in response) {
                        if(x == "text"){
                                continue;
                        }
			if (x == "sentiment_tone")
				APIresultbox += "<tr><td><b> " + x + colors[response[x]] + response[x] + "</font></td></tr>";
			else
           			APIresultbox += "<tr><td><b> " + x + ":</b> " +response[x] + "</td></tr>";		
       		}
        APIresultbox += "</table>"
	console.log(APIresultbox);
              console.log(response);
             if(ed.id.includes("responses"))
                  $('#' + ed.id.replace("responses", "analysis")).html(APIresultbox);
              else if(ed.id.includes('review_comments'))
                 $('#review_comments_analysis').html(APIresultbox);
            },
            error: function (xhr, status) {
	      console.log(status)
              $('#save_progress').html('Unable to reach suggestion detection service at the moment')
            }
          });
        }
      });
    }

Screenshot with integrated API

Test plan

Test: ViewSuggestionMetricsForReviewSegment

Feature: As a Student,
         I want to view a popup displaying the suggestion metrics for the segment of the review I am writing
         So that I can use it to improve the relevance of the review segment provided

Scenario: Display Suggestion Metrics on Empty Review Segment
          Given the Student has not entered any text in the review segment
          When the Student clicks outside the textbox
          No suggestions are displayed

Scenario: Display Suggestion Metrics on Review Segment with Text
          Given the Student has entered any form of text in the review segment
          When the Student clicks outside the text box for the segment
          Then a table will show the user the results of the suggestion metrics analysis

Scenario: Display Suggestion Metrics on Review Segment with Text
          Given the Student has entered positive text and suggestions in the review segment
          When the Student clicks outside the text box for the segment
          Then a table will show the user the results of the suggestion metrics analysis

Scenario: Display Suggestion Metrics on Review Segment with Text
          Given the Student has entered negative text and suggestions in the review segment
          When the Student clicks outside the text box for the segment
          Then a table will show the user the results of the suggestion metrics analysis

References

1. NLP Project for suggestions algorithm
2. The live Expertiza website
3. Expertiza on GitHub
4. Expertiza project documentation wiki
5. Plug-In: Cores