CSC/ECE 517 Fall 2018/E1878 Integrate Suggestion Detection Algorithm: Difference between revisions
| Line 15: | Line 15: | ||
'''Currently, we have a few classifiers that could catch useful components of review comments, such as if it contains suggestions, etc. These classifiers are already ported into web services that we’d like to be integrated into Expertiza.''' As stated in the Problem statement, we will be getting the response from the REST endpoints as given and integrate it with Expertiza. '''According to the problem statement, we will be considering the "Suggestion Detection Algorithm" to be a black box, which take a json with text as input and returns a json with few metrics as given below.''' | '''Currently, we have a few classifiers that could catch useful components of review comments, such as if it contains suggestions, etc. These classifiers are already ported into web services that we’d like to be integrated into Expertiza.''' As stated in the Problem statement, we will be getting the response from the REST endpoints as given and integrate it with Expertiza. '''According to the problem statement, we will be considering the "Suggestion Detection Algorithm" to be a black box, which take a json with text as input and returns a json with few metrics as given below.''' | ||
''Input to | ''Input JSON to "Suggestion Detection Algorithm"'' | ||
{ "text" : "This is an excellent project. Keep up the great work" } | { "text" : "This is an excellent project. Keep up the great work" } | ||
''Output | ''Output JSON the "Suggestion Detection Algorithm"'' | ||
{"sentiment_score": 0.9, | {"sentiment_score": 0.9, | ||
"sentiment_tone": "Positive", | "sentiment_tone": "Positive", | ||
Revision as of 20:29, 20 November 2018
CSC-517 Fall 2018 - E1878 Integrate suggestion detection algorithm in Expertiza Project. This Wiki page explains how we are planning to integrate the suggestion detection algorithm in Expertiza project.
Introduction
Team
- Ameya Dhavalikar
- Komal Kangutkar
- Prashanthi Kanniappan Murthy
- Vibhav Nandavaram Abbai Srivaths
Background information
Expertiza
Expertiza is an Open Source project based on the Ruby on Rails framework, supported by National Science Foundation. It is the software to create reusable learning objects through peer review. It is a project where students can submit and peer review learning objects(articles, code, websites, etc). The users of this software include students and professors. Expertiza is used by select professors and students in North Carolina State University, for example. It supports team projects, reviews of projects/teammates, submission URLs, Wiki pages and certain document types. Instructors can create projects and the students can bid for the projects. The students can be assigned teams for a particular project or they may form their own team with fellow classmates.
Suggestion detection algorithm
Peer-review systems like Expertiza utilize a lot of students’ input to determine each other’s performance. In the same time, we hope students could also gain knowledge from the reviews received thus improve their own performance. In order to make this happen, we would like to have everyone give quality reviews instead of generic ones. Currently, we have a few classifiers that could catch useful components of review comments, such as if it contains suggestions, etc. These classifiers are already ported into web services that we’d like to be integrated into Expertiza. As stated in the Problem statement, we will be getting the response from the REST endpoints as given and integrate it with Expertiza. According to the problem statement, we will be considering the "Suggestion Detection Algorithm" to be a black box, which take a json with text as input and returns a json with few metrics as given below.
Input JSON to "Suggestion Detection Algorithm"
{ "text" : "This is an excellent project. Keep up the great work" }
Output JSON the "Suggestion Detection Algorithm"
{"sentiment_score": 0.9,
"sentiment_tone": "Positive",
"suggestions": "absent",
"suggestions_chances": 10.17,
"text": "This is an excellent project. Keep up the great work",
"total_volume": 10,
"volume_without_stopwords": 6}
Problem statement
- When a student submits a review, we would like to call this web service with the student’s review as the input. We would then want to tell the student whether their reviews contain suggestions or not, so they can make improvements based on the results of the webservice.
- We would also like to evaluate how much time this API is taking and if possible work a way out to improve it. We don’t want the system to be terribly slow.
Files modified
- app/views/response/response.html.erb
- app/views/submitted_content/_self_review.html.erb
Design document
Wireframes
Current State
In its current state, the student directly writes their review in segmented text boxes under instructor specified questions with no further checks about the relevance of the review they have written as shown below:
Expected State
We plan to implement buttons next to each text box so that the student can check their review for volume, sentiment, suggestion, praise and criticism metrics. The rationale for including buttons right next to each text box, is so that the student gets to test only the written matter that they wish to check, and not the entirety of their review.
Test plan
Test: ViewSuggestionMetricsForReviewSegment
Feature: As a Student,
I want to view a popup displaying the suggestion metrics for the segment of the review I am writing
So that I can use it to improve the relevance of the review segment provided
Scenario: Display Suggestion Metrics on Empty Review Segment
Given the Student has not entered any text in the review segment
When the Student clicks the Analyze button for the segment
Then a popup will show that the user has not entered any text for suggestion metrics analysis.
Scenario: Display Suggestion Metrics on Review Segment with Text
Given the Student has entered any form of text in the review segment
When the Student clicks the Analyze button for the segment
Then a popup will show the user the results of the suggestion metrics analysis