CSC/ECE 517 Fall 2010/ch1 1f vn

Introduction

Unit testing is a method by which we can isolate and test a unit functionality of the program, typically individual methods during and long after the code is written. It helps to identify errors in the program even without running the entire program. Unit testing frameworks provides us with constructs which simplifies the process of unit testing. This chapter walks through three different unit testing frameworks available for Ruby, explains how to use them with examples, and compares them with one another. The three commonly used unit testing frameworks for ruby are

1. Test::Unit
2. Shoulda
3. RSpec

Test::Unit

Background

Ruby comes with an in-built, ready to use unit testing framework called Test::Unit. It is a XUnit type framework and typically have a setup method for initialization, a teardown method for cleanup and the actual test methods itself. The tests themselves are bundled separately in a test class from the code it is testing.

Test Fixture

Test fixture represents the initial environment setup(eg. initialization data) and/or the expected outcome of the tests for that environment. This is typically done in the setup()/teardown() methods and it helps to separate test initialization/cleanup from the actual tests. It also helps to reuse the same fixture for more than one tests.

For example, consider a method prime_check(num) which takes an integer number as input and outputs whether it is prime number or not. In order to unit test this method we can create the following fixture containing a 2-dimensional array with a number and the expected output of whether it is prime or not.

 def setup
   @NUMBERS = [[3,true], [4,false], [7,true], [10,false]]    
 end

Assertions

The core part of test::unit framework is the ability to assert a statement of expected outcome. If an assert statement is correct then the test will proceed, otherwise the test will fail. This feature helps us to verify the method under test with different types of inputs and track the results. Test::unit provides a bunch of assert methods for this purpose:

assert( boolean, [message] )	True if boolean
assert_equal( expected, actual, [message] ) assert_not_equal( expected, actual, [message] )	True if expected == actual
assert_raise( Exception,... ) {block} assert_nothing_raised( Exception,...) {block}	True if the block raises (or doesn't) one of the listed exceptions.

For the full list of assertion methods provided by test::unit refer to test::unit assertions

Example

The test case class BinarySearchTest subclasses the Test::Unit::TestCase class and overrides setup and teardown methods. The test methods should start with 'test' prefix. This helps in isolating the test methods from the helper methods if any. The Test::Unit::TestCase class takes care of making the test methods into tests, wrapping them into a suite and running the individual tests. The test results are collected into Test::Unit::TestResult object.

   require 'test/unit'
   require 'binarysearch'

   class BinarySearchTest < Test::Unit::TestCase

     def setup
       @input_array = [1,2,3,4,5]
     end
     
     def test_success_left_half
       assert_equal(binary_search(@input_array,1),true)
       assert_equal(binary_search(@input_array,2),true)
     end

     def test_success_right_half
       assert_equal(binary_search(@input_array,5),true)
       assert_equal(binary_search(@input_array,4),true)
     end

     def test_success_middle
       assert_equal(binary_search(@input_array,3),true)
     end

     def test_failure
       assert_equal(binary_search(@input_array,6),false)
     end

     def teardown
       #nothing to do here
     end

   end

Here we have four test methods testing different logical paths of the binary search algorithm. Each test method can have one or more assert statements to test whether conditions are correct in each situation. To run the tests we simply have to run the file binary_search_test.rb

 Loaded suite C:/Users/vivake/RubymineProjects/Example/binarysearchtest
 Started
 ....
 Finished in 0.002000 seconds.

 4 tests, 4 assertions, 0 failures, 0 errors, 0 skips

 Test run options: --seed 48516

 Process finished with exit code 0

Shoulda

Background

Shoulda is not a testing framework by itself. It extends the Test::Unit framework with the idea of matchers, helpers and assertions

RSpec

Now let us consider about RSpec Testing Framework in detail.

Background

Behaviour Driven Development (BDD) is an Agile development process that comprises aspects of Acceptance Test Driven Planning, Domain Driven Design and Test Driven Development (TDD). RSpec is a Behavioural Driven Development (BDD) tool aimed at Test Driven Development, originally created by Dave Astels and Steven Baker. However David Chelimsky is really the gatekeeper of the RSpec project.

Traditionally tests have been written using a Unit Test framework like JUnit, NUnit or RUnit. However it is easy to spending a lot of time writing tests that test every unit of code in your software system, what RSPec does is to provide a subtle shift of focus from Unit testing to Behaviour testing or Behaviour Driven Development (BDD). By focusing on the behaviour of the system it helps clarify in our minds what the system should actually be doing. It also means that our energy is directed at more ‘useful’ tests. Useful tests, cover what the system should be doing and build in enough redundancy so that it should be easy to refactor our code without having to re-write every test.

RSpec is really two projects merged into one. The RSpec project pages describes these merged projects as:

1. a Story Framework for describing behaviour at the application level
   
2. a Spec Framework for describing behaviour at the object level
   
   

Dan North created rbehave which is the Story Framework and David Chelimsky created the Spec Framework. By encompassing two frameworks RSpec equips a programmer with a thorough set of testing tools, allowing you to think about your software problem from a number of perspectives.

Prerequisites

The prerequisites are

1. Ruby 1.8.4 or later
2. RSpec Gem (latest)

To install Ruby, please visit http://www.ruby-lang.org
To install RSpec, open a command shell, go to /bin folder in Ruby directory and type

> gem install rspec

Terms & Definitions

Here are some terms which are used frequently while working with RSpec.

1. subject code - The code whose behavior is specified using RSpec
2. expectation - The expected behavior of subject code is expressed using expectation (Similar to 'Assertions' statements used in Test::Unit or other tools in other languages)
3. code example - An executable example containing the subject code and the expectations (Similar to 'Test Method' terminology used elsewhere)
4. example group - A group of code examples (Similar to 'Test Case' terminology used elsewhere)
5. spec file - A file which contains one or more example groups

Example

Let us go through an example to be clear on the usage of RSpec.

 Example to be inserted

describe() method

The describe() method can take an arbitrary number of arguments and a block and returns a sub-class of Spec::Example::ExampleGroup. We generally use only one or two arguments which is used to describe the behavior. The first argument can be a reference to a Class or module or a string. The second argument is optional and should be a string when used.

it() method

Similar to the describe() method, the it() method takes a single String, an optional Hash and an optional block. The String expression within the it() should be such that it informs the behavior of the code within the block.

Expectations in RSpec

There are two methods available for checking expectations: should() and should_not(). Both the methods accept either an expression matcher or a Ruby expression using a specific subset of Ruby operators. An expression matcher is an objects that matches an expression.

Built-in Matchers

There are several matchers that can be used with should and should_not, which are divided into well-separated categories.

Equality

subject.should == ece517
subject.should === ece517
subject.should eql(subject)
subject.should equal(subject)

The == method is used to express equivalence and equal is used when you want the receiver and the argument to be the same object. Instead of using !=, you should use the should_not method!

Floating Point Calculations

piValue.should be_close(3.14, 0.001593)

Sometimes the values generated might be correct upto some fixed decimal positions, after that they may have slight variations. To avoid the test beings failed, we provide the (value, delta) to be_close method which passes the test if the obtained value lies within the range (value+delta).

Regular Expressions

resultExpression.should match(/this regular expression/)
resultExpression.should =~ /this regular expression/

This can be very useful when dealing with multiple-line expectations, instead of using the open file technique to compare contents.