CSC/ECE 517 Spring 2018- Project E1812: on the fly calc.rb
E1812: Unit tests for on_the_fly_calc.rb
This wiki page gives a description of changes made under Expertiza OSS project E1812 assignment for Spring 2018, CSC/ECE 517.
NOTE to reviewers
Because this project is just a unit test, it cannot be tested via UI. Per Dr. Gehringer's instructions we have posted ScreenCast to prove the tests work. In the video, the unit tests are run from the terminal, and then we check the coverage. The goal of our project was to have at least 90% coverage, which we achieved.
Expertiza Background
Expertiza is an open source web application developed with Ruby on Rails framework. This application allows students to access assignments posted by an instructor, submit their work (articles, codes, websites), review others work submissions. Expertiza gives a chance to improve students’ work based on their peer-reviews, and it can also help in grading students’ submissions based on that feedback. Students can form their teams for an assignment, otherwise, teams can be assigned automatically.
Problem Statement
The OnTheFlyCalc model does not have any test cases corresponding to it. Thus, the task of this assignment is to write unit tests for on_the_fly_calc.rb using RSpec. The test cases should be fast, effective, and achieve the maximum code coverage, i.e. > 90%.
OnTheFlyCalc
On_the_fly_calc is a module that is included in assignment.rb. This module calculates the score for both students and instructors. E.g., when a student clicks “view my score”, Expertiza calculates the review scores for each review. This is called “on the fly” because expertiza stores the review scores based on each question (in answers table), but does not store the total score that a reviewer gives to a reviewee. It directly calculates the total review score for each review, using the score values in the answer table during run-time.
Created/Modified Files
As a part of the project the file listed below was created and saved under spec/models folder:
- on_the_fly_calc_spec.rb
The file listed below was modified due to some code issues in it
- on_the_fly_calc.rb (path: app/models)
1. scores(question) function returned scores which was being considered as recursive calls and thus we changed the return variable to score which is being modified in the functions. 2. score_team[:scores] = Answer.compute_scores(assessments, questions[:review]) line in the scores was changed to score_team[:scores] = Answer.compute_scores(assessments, questions) this was done because the compute scores function in Answers was throwing an error while trying to deference the index of questions using a symbol. 3. grades_by_rounds[round_sym] = Answer.compute_scores(assessments, questions[round_sym]) to rades_by_rounds[round_sym] = Answer.compute_scores(assessments, questions) same reason as above.
Thoughts
What we need to do is to write tests for four public methods listed in the module OnTheFLyCalc. However, the on_the_fly_calc itself has some code issues in #scores, and IDE reports some errors when we run the test case for it. Thus, we need to modify the module itself a little.
Tools
RSpec
For this project we use RSpec which is a testing tool for Ruby, created for behavior-driven development (BDD).
Plan of Work
In order to accomplish this task, that is to build tests for the OnTheFlyCalc model, we need to implement the following plan:
1. Set up the Expertiza environment;
2. Understand the functionality of the model;
3. Understand the relative methods and objects involved with each function of the model;
4. Create test objects;
5. Write ‘it block’ test scenarios with appropriate contexts for all public functions of the model, and check if all expectations pass.
Expertiza Environment
We installed VM VirtualBox, and imported Ubuntu-Expertiza image into VirtualBox. In Git, Expertiza master brunch was forked, in the VM terminal we cloned it and then run all required commands to finish setting up the environment.
Testing
To DRY up our tests, we use "before" and "let". The following set-up lines create variables that are common across tests:
let(:on_the_fly_calc) { Class.new { extend OnTheFlyCalc } }
let(:questionnaire) { create(:questionnaire, id: 1) }
let(:question1) { create(:question, questionnaire: questionnaire, weight: 1, id: 1) }
let(:response) { build(:response, id: 1, map_id: 1, scores: [answer]) }
let(:answer) { Answer.new(answer: 1, comments: 'Answer text', question_id: 1) }
let(:team) { build(:assignment_team) }
let(:assignment) { build(:assignment, id: 1, name: 'Test Assgt') }
let(:questionnaire1) {build(:questionnaire, name: "abc", private: 0, min_question_score: 0, max_question_score: 10, instructor_id: 1234)}
let(:contributor) { build(:assignment_team, id:1) }
Listed below are the functions in on_the_fly_calc.rb and the RSpec unit tests corresponding to the function names along with a list of scenarios tested.
Function: compute_total_score
The function compute total score totals the scores of each questionnaire in an assignment.
def compute_total_score(scores)
total = 0
self.questionnaires.each {|questionnaire| total += questionnaire.get_weighted_score(self, scores) }
total
end
To test this function
We mocked the calls made in the Questionnaire class. For example, to mock the call:
round = AssignmentQuestionnaire.find_by(assignment_id: assignment.id, questionnaire_id: self.id).used_in_round
we used the rspec line which looks like this
allow(AssignmentQuestionnaire).to receive(:find_by).with(assignment_id: 1, questionnaire_id: nil).and_return(double('AssignmentQuestionnaire', used_in_round: 1))
similarly, other calls made to
self.assignment_questionnaires.find_by(assignment_id: assignment.id)
was mocked using calls that look like :-
allow(ReviewQuestionnaire).to receive_message_chain(:assignment_questionnaires,:find_by).with(no_args).with(assignment_id: 1).and_return(double('AssignmentQuestionnaire', id: 1))
cases for consideration :
1. when avg score equals nil
2. when avg score is not equal to nil
Function: compute_reviews_hash
Below is most of the code used in the test case. It is relatively simple. The first test case is for when assignments vary by rubrics. We check that the function returns {}. This is correct because we set reviewer to nil so that reviewer will be set to {} in the "reviewer = {} if reviewer.nil?" statement. In the second test case we check that it returns {1=>{1=>50}, 2=>{1=>30}}. This is correct because we create 2 response maps and set their value to 50 and 30.
context 'when current assignment varys rubrics by round' do
it 'scores varying rubrics and returns review scores' do
allow(assignment).to receive(:varying_rubrics_by_round?).and_return(TRUE)
allow(assignment).to receive(:rounds_of_reviews).and_return(1)
temp = assignment.compute_reviews_hash()
expect(temp).to eql({})
end
end
context 'when current assignment does not vary rubrics by round' do
it 'scores varying rubrics and returns review scores' do
temp = assignment.compute_reviews_hash()
expect(temp).to eql({1=>{1=>50}, 2=>{1=>30}})
end
end
Function: compute_avg_and_ranges_hash
This function calculates the average score and score range for each reviewee (team).
def compute_avg_and_ranges_hash
scores = {}
contributors = self.contributors # assignment_teams
if self.varying_rubrics_by_round?
rounds = self.rounds_of_reviews
(1..rounds).each do |round|
review_questionnaire_id = review_questionnaire_id(round)
questions = Question.where('questionnaire_id = ?', review_questionnaire_id)
contributors.each do |contributor|
assessments = ReviewResponseMap.get_assessments_for(contributor)
assessments = assessments.select {|assessment| assessment.round == round }
scores[contributor.id] = {} if round == 1
scores[contributor.id][round] = {}
scores[contributor.id][round] = Answer.compute_scores(assessments, questions)
end
end
else
review_questionnaire_id = review_questionnaire_id()
questions = Question.where('questionnaire_id = ?', review_questionnaire_id)
contributors.each do |contributor|
assessments = ReviewResponseMap.get_assessments_for(contributor)
scores[contributor.id] = {}
scores[contributor.id] = Answer.compute_scores(assessments, questions)
end
end
scores
end
We consider 2 scenarios to test for this function:
1. When current assignment varies rubrics by round: function computes avg score and score range for each team in each round and return scores
context 'when current assignment varys rubrics by round' do
it 'computes avg score and score range for each team in each round and return scores' do
allow(on_the_fly_calc).to receive(:varying_rubrics_by_round?).and_return(TRUE)
allow(on_the_fly_calc).to receive(:rounds_of_reviews).and_return(1)
expect(on_the_fly_calc.compute_avg_and_ranges_hash).to eq({1=>{1=>{:min=>50.0, :max=>50.0, :avg=>50.0}}})
end
end
2. When current assignment does not varies rubrics by round: function computes avg score and score range for each team and return scores
context 'when current assignment does not vary rubrics by round' do
it 'computes avg score and score range for each team and return scores' do
allow(on_the_fly_calc).to receive(:varying_rubrics_by_round?).and_return(FALSE)
expect(on_the_fly_calc.compute_avg_and_ranges_hash).to eq({1=>{:max=>50, :min=>50, :avg=>50}})
end
end
Function: scores
def scores(questions)
index = 0
score_assignment
self.teams.each do |team|
score_team = {}
score_team[:team] = team
if self.varying_rubrics_by_round?
calculate_rounds
calculate_score
calculate_assessment
else
assessments = ReviewResponseMap.get_assessments_for(team)
score_team[:scores] = Answer.compute_scores(assessments, questions)
end
index += 1
end
score
end
end
This method calls a lot of private methods to calculate a score hash. To test this method we had to mock calls made in these private methods.
When varying_rubrics_by_round? is true this method calls
1. calculate_rounds
2. calculate_score
3. calculate_assessment
we mocked the call to 1. so it is not covered in our coverage as it just initializes our hashes.
to mock the call to 2. and 3. calls such as the ones given below can be seen in our test cases.
allow(on_the_fly_calc).to receive(:index).and_return(0)
allow(on_the_fly_calc).to receive(:num_review_rounds).and_return([1,2])
allow(on_the_fly_calc).to receive(:team).and_return(double('AssignmentTeam'))
allow(ReviewResponseMap).to receive(:get_responses_for_team_round).with(any_args).and_return([])
allow(on_the_fly_calc).to receive(:questions).and_return(questions)
allow(Answer).to receive(:compute_scores).with([],[question1]).and_return({})
allow(on_the_fly_calc).to receive(:round_sym).and_return(:review1)
allow(on_the_fly_calc).to receive(:grades_by_rounds).and_return(0)
allow(on_the_fly_calc).to receive(:score_assignment).and_return('')
we are basically mocking calls to run through all lines of the methods and only calls that is returning something or are assigning value to a variable need to be mocked.
for example the call
allow(Answer).to receive(:compute_scores).with([],[question1]).and_return({})
is to mock the call made to compute_score function in the Answer class we made it to return an empty hash.
cases for consideration :
1. when current assignment varys rubrics by round and number of assessments is 0.in this case
allow(on_the_fly_calc).to receive(:total_num_of_assessments).and_return(0)
this call is made to mask the value of total_num_of_assessments value, similarly for others.
2. when current assignment varys rubrics by round and number of assessments is non-zero.
3. when current assignment does not vary rubrics by round
Running RSpec
- To run RSpec for a particular file (on_the_fly_calc_spec.rb):
$ rspec spec/models/on_the_fly_calc_spec.rb
Coverage Results
We have a 92.74% coverage. Below is a picture of the results from the coverage.
References
1. RSpec Wiki
2. RSpec.info
3. youtube.com TDD: RSpec Test Structure
4. github/expertiza response_spec example
5. RSpec Documentation