CSC/ECE 517 Fall 2018/E1878 Integrate Suggestion Detection Algorithm

CSC-517 Fall 2018 - E1878 Integrate suggestion detection algorithm in Expertiza Project. This Wiki page explains how we are planning to integrate the suggestion detection algorithm in Expertiza project.

Introduction

Team

1. Ameya Dhavalikar
2. Komal Kangutkar
3. Prashanthi Kanniappan Murthy
4. Vibhav Nandavaram Abbai Srivaths

Background information

Expertiza

Expertiza is an Open Source project based on the Ruby on Rails framework, supported by National Science Foundation. It is the software to create reusable learning objects through peer review. It is a project where students can submit and peer review learning objects(articles, code, websites, etc). The users of this software include students and professors. Expertiza is used by select professors and students in North Carolina State University, for example. It supports team projects, reviews of projects/teammates, submission URLs, Wiki pages and certain document types. Instructors can create projects and the students can bid for the projects. The students can be assigned teams for a particular project or they may form their own team with fellow classmates.

Suggestion detection algorithm

Peer-review systems like Expertiza utilize a lot of students’ input to determine each other’s performance. In the same time, we hope students could also gain knowledge from the reviews received thus improve their own performance. In order to make this happen, we would like to have everyone give quality reviews instead of generic ones.

Currently, we have a few classifiers that could catch useful components of review comments, such as if it contains suggestions, etc. These classifiers are already ported into web services that we’d like to be integrated into Expertiza. As stated in the Problem statement, we will be getting the response from the REST endpoints as given and integrate it with Expertiza. According to the problem statement, we will be considering the "Suggestion Detection Algorithm" to be a black box, which take a json with text as input and returns a json with few metrics as given below.

Input to the "Suggestion Detection Algorithm" Input text is to be passed in the following JSON format to the "Suggestion Detection Algorithm" { "text" : "This is an excellent project. Keep up the great work" }

Output to the "Suggestion Detection Algorithm" Output is returned in the following JSON format:

  {"sentiment_score": 0.9,
 "sentiment_tone": "Positive",
 "suggestions": "absent",
 "suggestions_chances": 10.17,
 "text": "This is an excellent project. Keep up the great work",
  "total_volume": 10,
  "volume_without_stopwords": 6}

Problem statement

1. When a student submits a review, we would like to call this web service with the student’s review as the input. We would then want to tell the student whether their reviews contain suggestions or not, so they can make improvements based on the results of the webservice.
2. We would also like to evaluate how much time this API is taking and if possible work a way out to improve it. We don’t want the system to be terribly slow.

Files modified

1. app/views/response/response.html.erb
2. app/views/submitted_content/_self_review.html.erb

Wireframes

Current State

In its current state, the student directly writes their review in segmented text boxes under instructor specified questions with no further checks about the relevance of the review they have written as shown below:

Expected State

We plan to implement buttons next to each text box so that the student can check their review for volume, sentiment, suggestion, praise and criticism metrics. The rationale for including buttons right next to each text box, is so that the student gets to test only the written matter that they wish to check, and not the entirety of their review.

Test plan

Test: ViewSuggestionMetricsForReviewSegment

Feature: As a Student,
         I want to view a popup displaying the suggestion metrics for the segment of the review I am writing
         So that I can use it to improve the relevance of the review segment provided
Scenario: Display Suggestion Metrics on Empty Review Segment
          Given the Student has not entered any text in the review segment
          When the Student clicks the Analyze button for the segment
          Then a popup will show that the user has not entered any text for suggestion metrics analysis.

Scenario: Display Suggestion Metrics on Review Segment with Text
          Given the Student has entered any form of text in the review segment
          When the Student clicks the Analyze button for the segment
          Then a popup will show the user the results of the suggestion metrics analysis

References

1. NLP Project for suggestions algorithm
2. The live Expertiza website
3. Expertiza on GitHub
4. Expertiza project documentation wiki