CSC/ECE 517 Fall 2016/E1670. Unit tests for answers.rb

E1670 . Unit tests for answers.rb

This wiki page is for the description of changes made under E1670 OSS assignment for Fall 2016, CSC/ECE 517.

NOTE to reviewers

Mocks and factories are used in the unit tests so they can't be tested by UI.

Background

Expertiza is an Open Source web application developed on Ruby On Rails platform. It is a platform which allows students to access assignments posted by the course instructor. Expertiza also allows students to select assignment topics, form teams and submit their work. It also allows them to review other students' submissions and improve their work based on the feedback provided. The instructor can look at the feedback provided by students and rate the student based on feedback provided for others work. It helps organize and upload assignments and reducing the manual review process and helps students provide a peer feedback and also learn from reviewed work. Teams can be chosen by the student or can be assigned automatically based on a priority list inputted alongside each topic available for the assignment.

Project Description

In Expertiza, each questionnaire contains many questions, those question may have different types (e.g. checkbox, criterion, etc). When a user fills in a rubric, the responses for each question will become an answer record. The responses of rubrics are stored in answers table in DB. The answer.rb is the model for the answers table in the Database. The Answer.rb model does not have any test cases corresponding to it and the aim of the project is to write fast, effective and flexible unit test cases that offer maximum code coverage.

Files Created/Modified

As a part of the project the files mentioned below were the ones, created/modified as needed.

• answer_spec.rb (path /spec/models/answer_spec.rb)
• factories.rb (path /spec/factory/)

RSpec

RSpec Introduction

RSpec is a testing framework for Ruby for behavior-driven development (BDD) licensed under MIT. It is inspired by JBehave and contains fully integrated JMock based framework which has a very rich and powerful DSL (domain-specific language) which resembles a natural language specification. Composed of multiple libraries structured to work together, RSpec provides encapsulated testing via the describe block to specify the behavior of the class and the context for the unit test case.

Why RSpec?

RSpec is easy to learn and implement and can be used with other testing tools like Cucumber and Minitest independently. It is extremely powerful for testing states with complicated setup and also helps in tearing down complex code to access the objects required for testing RSpec semantics encourage agile thinking and practice and it structures the tests in a more intuitive way.

Functions in answers.rb

Listed below are the functionalities and the Rspec unit tests corresponding to the function names along with a list of scenarios tested.

Function Name : Compute_scores

Compute_scores is a function in model answer.rb which gives the average, maximum and minimum scores obtained in a series a responses/assessments. It has two input parameters: List of assessments and questions. The function iterates over each assessment to get the total score of that assessment. If the response or review is invalid, that assessment won’t be considered in the calculation of the score average. After iterating over all assessments, Max score, and Min scores are calculated along with the average score based on the number of valid assessments. Their scores are returned by the function.

Rspec Unit Test : Compute_scores

Following scenarios are tested:

• To return scores as nil if the input list of assessments is nil. This is to make sure no Null Pointer exceptions are thrown by the code
• To return a particular score when a single valid assessment is given as an input.
• To return a particular score when multiple valid assessments are given as an input. This is to test the looping functionality of the method.
• To return a particular score when invalid assessments are given as an input. The validity and total scores are returned by a mock and not by the actual functions. This is to make the test cases less rigid so that failure of dependent functions do not disrupt the functionality of interface level tests.
• To return a particular score when invalid flag is nil. Invalid flag can either be 0,1 or nil. Nil situation is tested to prevent NullPointer Exceptions
• To check if the method get_total_score is called with the right parameters.

This unit test uses a stub and returns a mock value.

Answer.stub(:get_total_score)

This is to eliminate tight coupling between compute_scores and get_total_scores. Failure in get_total_scores wouldn't break the compute_scores test cases.

Function Name : Get_total_scores

This function is called by the Compute_scores method of the answer.rb model to compute the total score of an assessment. The input consists of assessment for which the total score is being calculated and the list of questions being evaluated in the assessment. The method uses questionnaire id from the questions and response id to obtain a score view questionnaire data. This score view is a read-only record. The questionnaire data mainly consists of q1_max_question_score, sum_of_weights and weighted_score. Before calculating the score, the function performs two crucial tasks.

• Checks if the answer for a scored question is nil. If it is nil or unanswered, the question will be ignored and not counted towards the score of this response.
• Calls the submission_valid function by passing the response record to set the @invalid flag based on the validity of the response.

The total score is calculated using the below formula

(weighted_score / (sum_of_weights * max_question_score)) * 100

Any edge cases would return -1 indicating no score

Rspec Unit Test : Get_total_scores

Following scenarios are tested:

• To return an anticipated total score of a single response without any edge cases. Since this function will be called only on one response at a time, there is no need to test for multiple responses at the same time.
• To return an anticipated total score of a response where nil answer is for a scored question. This is to check if its weight gets removed from the sum_of_weights.
• To return -1 when the sum of weights becomes 0. This can happen when all the scored questions are unanswered. Return value of -1 is checked at the calling function to ensure if a score is returned or not.
• To return -1 when weighted_score of questionnaireData is nil
• To check if submission_valid is called. This method is called to set invalid flag to indicate whether the response entered is valid or not. The validity criteria is explained in the submission_valid? Function.

This unit test uses two stubs and returns mock results

ScoreView.stub(:find_by_sql)

Answer.stub(:where)

This is to reduce the outcome of the test to depend on DB calls. For example, in case the connection to DB fails, this unit test would still pass making it less rigid.

Function Name : submission_valid?

This purpose of this function is to verify the validity of a review based on a deadline. This function obtains a list of AssignmentDueDate objects in descending order which is then compared against the current time to determine which deadlines are valid and which ones are not. The flag variable is used to represent whether or not a deadline was available in the previous iteration of the loop. If the flag is set to TRUE and the deadline is less than the current date, then latest_review_phase_start_time is set to this deadline. A list of ResubmissionTime objects is also retrieved from the controller. These objects are then compared against the then latest_review_phase_start_time variable to determine a response.

Observations: The current implementation of the function is bugged. It retrieves a list of sorted AssignmentDueDate objects and proceeds to check if this list is empty. However, instead of exiting if the function if the list is empty, it carries on execution and raises an exception on hitting the for loop.

• Checks if a review is valid or not by comparing its date with a list of deadlines.
• Returns 1 or 0 depending on validity
• Current implementation is bugged. Throws an exception if no AssignmentDueDate objects are passed, returns nil if any objects are passed.

Rspec Unit Test : submission_valid?

Following scenarios are tested:

• Passing valid AssignmentDueDate objects
• Not passing any AssignmentDueDate objects

When valid AssignmentDueDate objects are passed, stubs are used to create fake AssignmentDueDate Objects and ResubmissionTime objects. These objects are then populated with valid deadline dates and deadline_type values. These values are then supplied when requested by the submission_valid? Function rather than actually calling the function. The current implementation of this test case expects the program to throw an error when it reaches the for loop. Once this bug is fixed, this test case may be re-written to test a more legitimate test case.

In case an empty list of AssignmentDueDate objects is passed back to the submission_valid?() method. This would cause the function to return nil. When this function is fixed, the following test case may be re-written to test for a more legitimate test-case.

This unit test uses two stubs and returns mock results

AssignmentDueDate.stub_chain(:where, :order)

The above stub returns two AssignmentDueDate objects whenever the where() and order() method are chained on AssignmentDueDate.

ResubmissionTime.stub_chain(:where, :order)

The above stub returns two ResubmissionTime objects whenever the where() and order() method are chained on ResubmissionTime.

AssignmentDueDate.stub_chain(:where, :order)

This stub is used in the second test case to return nil objects.

the stub_chain method can be used to create stubs where chained methods are expected.

Function Name : answers_by_question, answers_by_question_for_reviewee,answers_by_question_for_reviewee_in_round

These three functions are sql queries that hit the DB to get an output record after a series of selections and joins. SQL queries that hit DB often do not have a high priority when it comes to testing, so the project does not aggressively test these functions. However, these functions are tested to make sure the queries are able to find the right column names and tables successfully. Any change in the table schema would be detected in these test cases. The functionality of these queries are not tested but instead the existence of an output is tested. This ensures that answer.rb is able to make a successful db connection and is sync with the latest db schema. Since the function actually hits the DB, mocks can no longer be used, instead active records were created and saved in test db using FactoryGirl gem from the factories.rb. These records are cleared after every test case.

Following factories were used to create records in the table

• question
• response_record
• response_map
• answer

Rspec Unit Test : answers_by_question, answers_by_question_for_reviewee,answers_by_question_for_reviewee_in_round

The factories are created such a way that the query would return an output and the tests will pass if the query returns a non nil record. The functionality of these functions would be tested by the integration tests of the controller

describe "#test sql queries in answer.rb" do

    it "returns answer by question record from db which is not empty" do
      assignment_id = 1
      q_id = 1
      expect(Answer.answers_by_question(assignment_id,q_id)).not_to be_empty
    end

    it "returns answers by question for reviewee from the db which is not empty" do
      assignment_id = 1
      reviewee_id = 1
      q_id = 1	
      expect(Answer.answers_by_question_for_reviewee(assignment_id,reviewee_id,q_id)).not_to be_empty
    end

    it "returns answers by question for reviewee in round from db which is not empty" do
      assignment_id = 1
      reviewee_id = 1
      q_id = 1
      round = 1
      expect(Answer.answers_by_question_for_reviewee_in_round(assignment_id,reviewee_id,q_id,round)).not_to be_empty
    end
  end

Validation and Dependency Rspec Test

Apart from testing functions, the models are also tested for validations and external dependencies. Model answer.rb does not have any validations but does have a dependency on question.rb. Answer belongs_to question and this was tested using a simple dependency rspec test.

it { should belong_to(:question) }

Running Rspec

• To run the test suite for a particular file only( answer_spec.rb):

root@ubuntu:~/expertiza$ rspec spec/models/answer_spec.rb

• To run the entire suite of test cases:

root@ubuntu:~/expertiza$ rspec spec

Rspec Test Results

1 deprecation warning total

Finished in 12.39 seconds (files took 11.25 seconds to load)
17 examples, 0 failures

Randomized with seed 37447

Coverage report generated for RSpec to /home/root/expertiza/coverage. 998 / 3919 LOC (25.47%) covered.
[Coveralls] Outside the CI environment, not sending data.

References

1. wikipedia.org/wiki/RSpec
2. rspec.info/
3. semaphoreci.com/community/tutorials/getting-started-with-rspec
4. github.com/rspec/rspec
5. Expertiza Wiki
6. Expertiza Github
7. Expertiza
8. Research Papers on Expertiza