CSC/ECE 517 Fall 2018/E1878 Integrate Suggestion Detection Algorithm

CSC-517 Fall 2018 - E1878 Integrate suggestion detection algorithm in Expertiza Project. This Wiki page explains the integration of the suggestion detection algorithm in Expertiza project.

Introduction

Team

1. Ameya Dhavalikar
2. Komal Kangutkar
3. Prashanthi Kanniappan Murthy
4. Vibhav Nandavaram Abbai Srivaths

Background information

Expertiza

Expertiza is an Open Source project based on the Ruby on Rails framework, supported by National Science Foundation. It is the software to create reusable learning objects through peer review. It is a project where students can submit and peer review learning objects(articles, code, websites, etc). The users of this software include students and professors. Expertiza is used by select professors and students in North Carolina State University, for example. It supports team projects, reviews of projects/teammates, submission URLs, Wiki pages and certain document types. Instructors can create projects and the students can bid for the projects. The students can be assigned teams for a particular project or they may form their own team with fellow classmates.

Suggestion detection algorithm

Peer-review systems like Expertiza utilize a lot of students’ input to determine each other’s performance. In the same time, we hope students could also gain knowledge from the reviews received thus improve their own performance. In order to make this happen, we would like to have everyone give quality reviews instead of generic ones. Currently, we have a few classifiers that could catch useful components of review comments, such as if it contains suggestions, etc. These classifiers are already ported into web services that we’d like to be integrated into Expertiza. As stated in the Problem statement, we will be getting the response from the REST endpoints as given and integrate it with Expertiza. The suggestion detection algorithm from which we are getting the metrics is added in the "References" section in case more details are needed on it. According to the problem statement, we will be considering the "Suggestion Detection Algorithm" to be a black box, which take a json with text as input and returns a json with few metrics as given below.

Input JSON to "Suggestion Detection Algorithm"

 { "text" : "This is an excellent project. Keep up the great work" }

Output JSON the "Suggestion Detection Algorithm"

  {"sentiment_score": 0.9,
 "sentiment_tone": "Positive",
 "suggestions": "absent",
 "suggestions_chances": 10.17,
 "text": "This is an excellent project. Keep up the great work",
  "total_volume": 10,
  "volume_without_stopwords": 6}

Problem statement

1. When a student submits a review, we call this web service with the student’s review as the input. We then tell the student whether their reviews contain suggestions or not, so they can make improvements based on the results of the web service.
2. We evaluate how much time this API is taking. We don’t want the system to be terribly slow.

Files modified

1. app/views/response/response.html.erb

Wireframes

Current State

In its current state, the student directly writes their review in segmented text boxes under instructor specified questions with no further checks about the relevance of the review they have written as shown below:

Expected State

We plan to implement buttons next to each text box so that the student can check their review for volume, sentiment, suggestion, praise and criticism metrics. The rationale for including buttons right next to each text box, is so that the student gets to test only the written matter that they wish to check, and not the entirety of their review.

UML Diagrams

We are not modifying any models / adding any logic that requires an UML diagram. Most of our changes are on HTML. Our logic to be implemented is pre-processing the review and sending it as JSON to the Suggestion Detection API, getting the JSON response from API, post-processing the metrics to output the Suggestion Analysis.

The overall flow of the logic is explained as follows:

Implementation

Following is the code snippet of our implementation :

//Suggestion detection API call
      tinyMCE.on('AddEditor', function(e) {
      e.editor.on('blur', function (e) {
        ed = e.target;
        if (ed.id.includes("comments")){
          json_review = JSON.stringify({ "text": ed.getContent({format : 'text'}) })
          $.ajax({
            url: "https://peer-reviews-nlp.herokuapp.com/all",
            type: "POST",
            crossDomain: true,
            contentHeader: "application/javascript",
            contentType: "application/json",
            data: json_review,
            dataType: "json",
            success: function (response) {
	    	console.log("success");
		var APIresultbox ="";
		var colors = {"Negative": ":</b> <font style=\"color:red;\">", 
				"Positive": ":</b> <font style=\"color:green;\">", 
				"Neutral": ":</b> <font style=\"color:saddlebrown;\">", 
				"Mixed": ":</b> <font style=\"color:saddlebrown;\">"};
		APIresultbox += "<table border='1'>"
        	for (x in response) {
			if (x == "sentiment_tone")
				APIresultbox += "<tr><td><b> " + x + colors[response[x]] + response[x] + "</font></td></tr>";
			else
           			APIresultbox += "<tr><td><b> " + x + ":</b> " +response[x] + "</td></tr>";		
       		}
	//API result box
        APIresultbox += "</table>"
	console.log(APIresultbox);
              console.log(response);
             if(ed.id.includes("responses"))
                  $('#' + ed.id.replace("responses", "analysis")).html(APIresultbox);
              else if(ed.id.includes('review_comments'))
                 $('#review_comments_analysis').html(APIresultbox);
            },
            error: function (xhr, status) {
		console.log(status)
              $('#save_progress').html('Unable to reach suggestion detection service at the moment')
            }
          });
        }
      });
    }

Test plan

Test: ViewSuggestionMetricsForReviewSegment

Feature: As a Student,
         I want to view a popup displaying the suggestion metrics for the segment of the review I am writing
         So that I can use it to improve the relevance of the review segment provided
Scenario: Display Suggestion Metrics on Empty Review Segment
          Given the Student has not entered any text in the review segment
          When the Student clicks the Analyze button for the segment
          Then a popup will show that the user has not entered any text for suggestion metrics analysis.

Scenario: Display Suggestion Metrics on Review Segment with Text
          Given the Student has entered any form of text in the review segment
          When the Student clicks the Analyze button for the segment
          Then a popup will show the user the results of the suggestion metrics analysis

References

1. NLP Project for suggestions algorithm
2. The live Expertiza website
3. Expertiza on GitHub
4. Expertiza project documentation wiki