CSC/ECE 517 Spring 2019 - Project E1937. Integrate suggestion detection algorithm.rb

From Expertiza_Wiki
Jump to navigation Jump to search

Introduction

Team

Dr. Gehringer (mentor)

  • Sushan Basnet (sbasnet2)
  • Jasmine Wang (jfwang2)
  • Bill Mwaniki (bnmwanik)
  • Pratik Kumar Kundanmal Jain (pjain22)

Project plan

Problem statement

Peer-review systems like Expertiza utilize a lot of students’ input to determine each other’s performance. At the same time, we hope students learn from the reviews they receive to improve their own performance. In order to make this happen, we would like to have everyone give quality reviews instead of generic ones. Currently we have a few classifiers that can detect useful features of review comments, such as how many suggestions they contain. The suggestion-detection algorithm has been coded as a web service, and other detection algorithms will be so coded in the future.


How it will work

In order to make the API call, the response_controller.rb will be responsible for sending a JSON input to the web service. The input will contain the review comment submitted by the user in the following format:

Below is a sample input 

 Sample Input:
 {
 	"reviews":[{
 		"text":"review text here",
 		"metrics":["problem","suggestion","sentiment"]
 	},{
 		"text":"another review here",
 		"metrics":["problem"]
 	},{
 		"text":"more text, maybe a large para or whatever you like",
 		"metrics":["sentiment", "suggestion"]
 	},{
 		...
 	}]
 }

Once the request is send, we expect the output to be in the following format: 

Sample Output: (as 'suggestion' and 'volume' as you might expect)
{
	"metrics":[{
		"suggestion":{
			"suggestions_chances":30,
			"suggestions":"absent"
		},
		"volume":{
			"total_volume":10,
			"volume_without_stopwords":3
		}
	},{
		...
	},{
		...
	}]
}

The output (which is a JSON) will be parsed and the suggestion metrics such as the tone and presence of suggestion will be extracted so the user will be able to view a summarized result of how well their review comments were. In addition, an average score will be computed based on the scores they received for each comment section, and the result will be presented in a colorful format to the user after they hit the submit button.

Previous work

Fall 2018 pull request

  • They had a functional suggestion detection API call that successfully communicated with the PeerLogic Server and retrieved the output.
  • They included their API call in response.html.erb using JavaScript.
  • They were able to display the output for each review beside the review. They displayed all of the information returned from the endpoint, which can look clunky.
  • Since the code is added in the views, they performed manual testing in the views.

More information about the suggestion detection service can be found at https://www.peerlogic.org

Flowchart to Describe Plan of Action

Changes Made

  • Move API calls of suggestion-detection algorithm from view to response_controller.rb
  • Change default review view from displaying analysis for each comment to summarized analysis for all comments
    • Do not include comment text in analysis view
    • Focus on sentiment_score, suggestions, suggestions_chances returned from API
  • Include displaying analysis for each review as a "debug" option
  • Ensure that CORS does not need to be enabled for API call to work
  • Write unit tests for our method(s) in response_controller.rb
  • Fix grammar issues in response.html.erb
  • Evaluate how much time this API is taking and, if possible, work a way out to improve it.


Files that we changed

  • app/views/response/response.html.erb - Refactored and reworked JS methods for the autosave and form submits, added a new hidden button to the response form
  • app/controllers/response_controller.rb - Refactored and reworked methods for the autosave and form submits, added an API method to pass to view
  • app/helpers/review_mapping_helper.rb - Adjusted the bar thickness according to desired specifications
  • config/routes.rb - Added another route to support the new review confirmation page listed below

New files we added

  • app/models/metric.rb - New ActiveRecord model added to act as a template for future API usage
  • db/migrate/20190425194052_create_metrics.rb - New table added for permanent storage of the Metric class
  • app/views/response/show_confirmation_page.erb - The new page added for review confirmation
  • test/fixtures/metrics.yml - In support of the new ActiveRecord model called Metric
  • test/models/metric_test.rb - In support of the new ActiveRecord model called Metric
  • app/assets/images/lightbulb.png - Lightbulb picture for the new review confirmation page
  • app/assets/images/music_note.png - A music note picture for the new review confirmation page

Response Controller

Lines 359 and 360 are where we will retrieve the input from each review comment found in the form.

  • The Answer model represents the review provided by the reviewer.
  • On line 360, answer: v[:score] is the review comment from a given textarea HTML element
  • params[:responses] stores the aggregate review data provided by the user

Confirmation Page View

Student View

The program flow is as follows:

  1. After the reviewer has filled out their review, they have 2 choices: See an analysis of their review or Save the review.
  2. The reviewer selects to view their analysis by clicking on "Confirm Submission" button
  3. The program directs the reviewer to the new confirmation page which displays an analysis of their review.
  4. The reviewer has 2 options in the confirmation page: Submit the review or go back and modify their review.
  5. If the reviewer decides to Submit their review, the program directs them back to where they started (Create or edit review).

The image below shows the above steps in a visual depiction.

Detailed Popup Information

Below we showcase the available information from the API if a reviewer is curious to find more details

Conditional Formatting

To make the interface more user-friendly, we used conditional formatting to color our text. For the suggestions metrics, anything >50% means suggestions are present while anything <50% means suggestions are absent. Our conditional formatting is set so "Yes (>50%)" is green, 50% is yellow, and "No (<50%)" is red. The formatting is the same for the class average of suggestions metrics.

For tone, there are three options in the API: positive, neutral, and negative. We created conditional formatting in our view file so positive is green text, neutral is yellow text, and negative is red text.

Testing plan

Travis CI

Since we changed the controller and UI in our project to add the redirect view, the current Travis CI test build is failing.

  • For TESTFOLDER=features, in the response.hmtl.erb view, we changed the submit button text from "Submit" to "Confirm Submission" in order to accurately reflect the new redirect page.
  • For TESTFOLDER=controllers, we modified the save method inside response_controller.rb to redirect according to logic defined above.

Tests

  • We tested the new model added i.e. Metric model in metric.rb.
  • The RSpec file is added to the rspec/models folder with the name metric_spec.rb.
  • The testing tests the calculation of the average of suggestions chances from each response for a particular assignment.
  • This also tests the output text of the sentiment that is received based on an input numerical value.

View Tests

We have...

  • Manually tested response.html.erb to verify the Review page outputs the correct message
  • Provided sample inputs and shown expected outputs using screenshots

Links to our work

Retrieved from "https://wiki.expertiza.ncsu.edu/index.php?title=CSC/ECE_517_Spring_2019_-_Project_E1937._Integrate_suggestion_detection_algorithm.rb&oldid=124624"