CSC/ECE 517 Spring 2017/E1730. Override-review-grader pre-sort
E1730
Please note that our problem statement has changed! The 'Problem Statement' section in this article describes the new one.
Deployment Link
About Expertiza
Expertiza is an open source peer review system. It allows students to upload their work which can then be reviewed by other students using a rubric. Instructors and graders can view the reviews and they can then assign grades to the reviewers as well the reviewee.
Problem Statement
While grading students, a grader might want to organize reviews by sorting them by a particular metric. The only available metric was ‘Average Volume’ (number of words in the review). A new metric called ‘Overall Sentiment’ is added which is a gauge of the sentiment of the review, that is, whether the review was positive or negative. The task was to allow a grader to select a particular metric, and the view displays that metric for each review. In addition to that, the grader should be able to sort the reviews by that selected metric.
Design
1) For each review, send a POST request to the Sentiment Analysis URL for sentiment analysis with the review as parameters.
2) Store the sentiment values in a list.
3) In the view, allow the grader to select the metric(average volume or overall sentiment) from the drop down menu.
4) The view is rendered accordingly to either show overall sentiment in the metric or the average volume
5) Using the tablesorter (jQuery) addParser method, the Metric column is sorted by the appropriate metric.
Current Implementation
Only a single metric for sorting reviews called ‘Average Volume’ was present. The table was sortable only by that metric.
New Implementation
The ‘Overall Sentiment’ metric is added to the system. The grader can choose between two metrics (Average volume, Overall Sentiment) and sort reviews by that metric.
Gem
A gem called HTTParty is added which facilitates the sending of HTTP requests (GET, POST, etc.) to the web service using Ruby code.
Code Snippets
The method constructs the query to be sent to the sentiment generation service.
def construct_sentiment_query(id, review_text) review = {} review["id"] = id.to_s review["text"] = review_text review end
The method retrieves the response from the sentiment generation service.
def retrieve_sentiment_response(review, first_try) if first_try response = HTTParty.post( 'http://peerlogic.csc.ncsu.edu/sentiment/analyze_reviews_bulk', body: {"reviews" => [review]}.to_json, headers: {'Content-Type' => 'application/json'} ) else # Send only the first sentence of the review for sentiment analysis in case of a failure first_sentence = review["text"].split('.')[0] reconstructed_review = construct_sentiment_query(review["id"], first_sentence) response = HTTParty.post( 'http://peerlogic.csc.ncsu.edu/sentiment/analyze_reviews_bulk', body: {"reviews" => [reconstructed_review]}.to_json, headers: {'Content-Type' => 'application/json'} ) end response end
The method creates a sentiment hash with the id of the review and the results from the sentiment generation service.
def create_sentiment(id, sentiment_value) sentiment = {} sentiment["id"] = id sentiment["sentiment"] = sentiment_value sentiment end
The method handles error scenarios on getting a response from the sentiment server
def handle_sentiment_generation_retry(response, review) sentiment = case response.code when 200 create_sentiment(response.parsed_response["sentiments"][0]["id"], response.parsed_response["sentiments"][0]["sentiment"]) else # Instead of checking for individual error response codes, have a generic code set for any server related error # For now the value representing any server error will be -500 create_sentiment(review["id"], "-500") end sentiment end
The method generates the sentiment list for all the reviews.
def generate_sentiment_list @sentiment_list = [] @reviewers.each do |r| review = construct_sentiment_query(r.id, Response.concatenate_all_review_comments(@id, r).join(" ")) response = retrieve_sentiment_response(review, true) # Insert the extracted sentiment into the sentiment list @sentiment_list << case response.code when 200 # Extract the sentiment from the response in case of a successful response create_sentiment(response.parsed_response["sentiments"][0]["id"], response.parsed_response["sentiments"][0]["sentiment"]) else # Retry to get a sentiment response once again in case of a failure handle_sentiment_generation_retry(retrieve_sentiment_response(review, false), review) end end @sentiment_list end
The method displays the value of Sentiment.
def display_sentiment_metric(id) hashed_sentiment = @sentiment_list.select {|sentiment| sentiment["id"] == id.to_s } sentiment_value = hashed_sentiment[0]["sentiment"].to_f.round(2) metric = "Overall Sentiment: #{sentiment_value}
" metric.html_safe end
Test Plan
Scenario 1:
Testing Tool:
RSpec
Description:
Check whether the sentiment query is not nil
require 'rails_helper'
describe 'ReviewMappingHelper', :type => :helper do describe "#construct_sentiment_query" do it "should not return nil" do expect(helper.construct_sentiment_query(1,"Text")).not_to eq(nil) end end
Scenario 2:
Testing Tool:
RSpec
Description:
Test whether the webservice does not return nil
describe "#get_sentiment" do it "should not return nil" do review = helper.construct_sentiment_query(1,"Test Review") # Test first try to get sentiment from the sentiment analysis web service expect(helper.get_sentiment(review, true)).not_to eq(nil) # Test a retry to get sentiment from the sentiment analysis web service expect(helper.get_sentiment(review, false)).not_to eq(nil) end end
Scenario 3:
Testing tool:
RSpec
Description:
Check whether the sentiment_list is populated and not nil
describe "#get_sentiment_list" do it "should not return nil" do @id=1 @assignment = Assignment.where(id: @id) @reviewers = ReviewResponseMap.review_response_report(@id, @assignment, "ReviewResponseMap", nil) expect(helper.get_sentiment_list).not_to eq(nil) end end
end
Scenario 4:
Testing tool:
RSpec
Description:
Check whether the web does not respond with a 404 error
it "should not get web response with 404 error" do review = helper.construct_sentiment_query(1,"Test Reviews to check if our Rest Client is able to reach the sentiment analysis service.") # Test first try to get sentiment from the sentiment analysis web service expect(helper.get_sentiment(review, true).code).not_to eq(404) # Test a retry to get sentiment from the sentiment analysis web service expect(helper.get_sentiment(review, true).code).not_to eq(404) end
Scenario 5:
Testing tool:
RSpec
Description:
Check whether the web service does not respond with a 500 error
it "should not get web response with 500 error" do review = helper.construct_sentiment_query(1,"Test Reviews to check if our Rest Client is able to reach the sentiment analysis service.") # Test first try to get sentiment from the sentiment analysis web service expect(helper.get_sentiment(review, true).code).not_to eq(500) # Test a retry to get sentiment from the sentiment analysis web service expect(helper.get_sentiment(review, true).code).not_to eq(500) end
Scenario 6:
Testing tool:
RSpec
Description:
Check whether the web service returns a response in JSON format
it "should return json response" do review = helper.construct_sentiment_query(1,"Test Review") # Test first try to get sentiment from the sentiment analysis web service expect(helper.get_sentiment(review, true).header['Content-Type']).to include 'application/json' # Test a retry to get sentiment from the sentiment analysis web service expect(helper.get_sentiment(review, false).header['Content-Type']).to include 'application/json' end end
Manual Testing
Steps:
1) cd into the expertiza directory
2) Run the command: rspec spec/helpers/review_mapping_helper_spec.rb