Unit-testing frameworks for Ruby

Unit Testing

A unit is the smallest building block of a software. Such a unit can be: a class, a method, an interface etc. Unit testing is the process of validating such units of code.

Benefits

Some of the benefits are:

Proof of your code.
Better design - Thinking about the tests can help us to create small design elements, thereby improving the modularity and reusability of units.
Safety net on bugs - Unit tests will confirm that while refactoring no additional errors were introduced.
Relative cost - It helps to detect and remove defects in a more cost effective manner compared to the other stages of testing.
Individual tests - Enables us to test parts of a source code in isolation.
Less compile-build-debug cycles - Makes debugging more efficient by searching for bugs in the probable code areas.
Documentation - Designers can look at the unit test for a particular method and learn about its functionality.

Unit-testing frameworks

Unit Test Framework is a software tool to support writing and running unit test.

The above diagram shows the relationships of unit tests to production code. Application is built from objects linked together. The unit test uses this application's objects but exists inside the unit test framework.

List of most popular unit testing frameworks for Ruby

Test::Unit
RSpec
Shoulda
Cucumber

Simple Application Program in Ruby

Let us write a Calculator class in a file called calculator.rb. Its a simple Calculator class having four methods addition, subtraction, multiplication and division. Later in the chapter we shall write tests in each of the above frameworks respectively.

class Calculator 
 
 attr_writer :number1 
 attr_writer :number2
 
 def initialize(number1,number2)
   @number1 = number1
   @number2 = number2
 end
#-----------Addition of two numbers----------------#
 def addition
   result = @number1 + @number2
   return result
 end
#----------Subtraction of two numbers--------------#
  def subtraction
    result= @number1 - @number2
    return result
  end
#----------Multiplication of two numbers------------#
  def multiplication
    result= @number1 * @number2
    return result
  end
#-----------Division of two numbers-------------------#
 def division
   result = @number1 / @number2
   return result
 end
end

Test::Unit

Features of Test::Unit

It provides assertions which return pass or fail result for each unit test.
Each of the assertions has to be called from a context which is also called Test Method

Usage

The general idea behind unit testing is that you write a test method that makes certain assertions about your code, working against a test fixture. The results of a run are gathered in a test result and displayed to the user through some UI.Basically we do following things:

Create require ‘test/unit’ in our test script.
Create a class that subclasses Test::Unit::TestCase.
Optionally define setup and/or teardown to set up and/or tear down your common test fixture.

So lets analyse the above things and find out the different components of Test::Unit

Assertions

An assertion statement specifies a condition that you expect to hold true at some particular point in your program. If that condition does not hold true, the assertion fails. An error is propagated with pertinent information so that you can go back and make your assertion succeed.

Test Fixture

A test fixture is all the things we need to have in place in order to run a test and expect a particular outcome. Sometimes it is also knows as the test context.

Test Method

A test method is a definitive procedure that produces a test result. Add a method that begins with "test" to your class.

Test Runners

To run the test class and view any failures that occur during the run we need a Test Runner. Examples -Test::Unit::UI::Console::TestRunner, Test::Unit::UI::GTK::TestRunner, etc.

Test Suite

A collection of tests which can be run.

Example of a test class

So already we have a Calculator Class given above.Now we want to perform some test for each method of the class.So lets start by creating a class called "TC_Calculator" which inherits "Test::Unit::TestCase".We have created four addition test methods following a naming convention by starting the method name with "test".We have used here assertion API for each unit test.

 require "calculator"
 require "test/unit"

 class TC_Calculator < Test::Unit::TestCase
  
  
  def test_addition
    assert_equal(8,Calculator.new(3,4).addition)
   
  end
 
  def test_subtraction
   assert_same(1,Calculator.new(4,3).subtraction)
  end
 
  def test_multiplication
   assert_not_same(12,Calculator.new(3,4).multiplication)
  end
 
  def test_division
     assert_not_equal(5,Calculator.new(8,2).division)
  end
 end

If we run it as "Ruby Application " we will get the following output.Let us analyze the following output.Here it says amon

Loaded suite tc__calculator
Started
F.F.
Finished in 0.046 seconds.
 1) Failure:
  test_addition(TC_Calculator) [tc__calculator.rb:8]:
  <8> expected but was
  <7>.
 2) Failure:
  test_multiplication(TC_Calculator) [tc__calculator.rb:17]:
  <12>
  with id <25> expected to not be equal? to
  <12>
  with id <25>.
 4 tests, 4 assertions, 2 failures, 0 errors

Shoulda

Features of Shoulda

It allows to write code with more clarity for ruby application
It allows to run test with the help of Test::Unit Configurations
It provides context to group tests for a particular requirement

Installing Shoulda

If your Environment path is set to " ...\Ruby\bin" then open command prompt and run gem install shoulda.

Usage

Shoulda makes it easy to write elegant, understandable, and maintainable tests. Shoulda consists of matchers, test helpers, and assertions. It’s fully compatible with your existing tests in Test::Unit or RSpec, and requires no retooling to use.

Matchers: Test::Unit- and RSpec-compatible one-liners that test common Rails functionality. These tests would otherwise be much longer, more complex, and error-prone.
Helpers: #context and #should give you RSpec like test blocks in Test::Unit. In addition, you get nested contexts and a much more readable syntax.
Assertions: Many common Rails testing idioms have been distilled into a set of useful assertions

Let us create a test class to understand how shoulda unit testing frameworks works.Here we start writing the program by describing the context what we want to do.As we want to perform calculation we have taken context here as "Calculate".We can also give setup block as below to do some initialization.

setup do
//Initialization ...
end

Also we can provide nested Context .We can specify different context and create different environment for our test cases.

 context "calculate" do
   setup do
   end
   should "" do
   end
   context "addition" do
     setup do
     end
     should "" do
     end
   end
 end

So here is our class "TC_Calculator_Shoulda" which inherits "Test::Unit::TestCase".We provide assertion inside should block.At least we can avoid giving names like "test_<method name>".

 require "rubygems"
 require "calculator"
 require "test/unit"
 require "shoulda"

 class TC_Calculator_Shoulda < Test::Unit::TestCase

  context "Calculate" do

   should "addition of two numbers " do
     assert_equal 8,Calculator.new(3,4).addition 
   end

   should "subtraction of two numbers " do
    assert_equal 1,Calculator.new(4,3).subtraction
   end

   should "multiplication of two numbers" do
    assert_equal 12,Calculator.new(3,4).multiplication
   end

   should "division of two numbers" do
    assert_equal 4,Calculator.new(12,3).division
   end

  end

 end

If we run above test as "Ruby Application" we will get the output as follows.If we analyze the error below we will find that we have recived a single failure in case of addition of two numbers.The sum should have been 7 rather than 8.Rest all the tests are successful.

Loaded suite tc__calculator__shoulda
Started
F...
Finished in 0.064 seconds.
 1) Failure:
    test: Calculate should addition of two numbers . (TC_Calculator_Shoulda)
    <8> expected but was
    <7>.
4 tests, 4 assertions, 1 failures, 0 errors

RSpec

Features of RSpec

Use RSpec to independently specify models, views, controllers and helpers.
Integrated fixture loading.
Special generators for models and controllers that generate specs instead of tests.
Special RSpec matchers for even more readable specs.

Installing RSpec

If your Environment path is set to " ...\Ruby\bin" then open command prompt and run gem install rspec.

Usage

Initial spec

You start by describing what your application, method or class should behave like. Let us create our first specification file:

require "calculator"

describe "TheCalculator's", "basic calculating" do

  it "should add two numbers correctly" do

  it "should subtract two numbers correctly" do

  it "should multiply two numbers correctly" do

  it "should divide two numbers correctly" do

  #...

end

What do we have here? First, at line 2 we defined a context for our test, using RSpec's describe block. The file contains a description of an aspect of the calculator class. Furthermore we have four behaviors/expectations (it "should ..." -blocks), defining what we expect our system to behave like.

We can run this specification using the spec command:

$ spec ts_spec.rb

produces:

****

Pending:

TheCalculator's basic calculating should add two numbers correctly (Not Yet Impl
emented)
./21file.rb:5

TheCalculator's basic calculating should subtract two numbers correctly (Not Yet
 Implemented)
./21file.rb:7

TheCalculator's basic calculating should multiply two numbers correctly (Not Yet
 Implemented)
./21file.rb:9

TheCalculator's basic calculating should divide two numbers correctly (Not Yet I
mplemented)
./21file.rb:11

Finished in 0.021001 seconds

4 examples, 0 failures, 4 pending

Establishing behavior

Isn't it cool? Executing the tests echoes our expectations back at us, telling us that each has yet to be implemented. This is forcing/Establishing behavior. I know that all calculator's must perform the above functions. I don't know how I'm going to design this yet, but the tests will derive my design.

Writing the application code

At the very end, you write the application code, and fire up the tests to check, if it behaves the way you wanted while writing the tests… but I‘m getting ahead of myself.

Developing the code

Let's go a step further and associate code blocks with our expectations.

require "calculator"

 describe "TheCalculator's", "basic calculating" do
  before(:each) do
    @cal = Calculator.new(6,2)
  end

  it "should add two numbers correctly" do
    @cal.addition.should == 8
  end

  it "should subtract two numbers correctly" do
    @cal.subtraction.should == 4
  end

  it "should multiply two numbers correctly" do
    @cal.multiplication.should == 11
  end

  it "should divide two numbers correctly" do
    @cal.division.should == 3
  end

end

Note that creating a Calculator object for each of our expectations will create duplication in the specification. This can be fixed using a before stanza in the specification. It allows us to run code before expectations are executed.

Let us run it:

produces:

Spec::Expectations::ExpectationNotMetError:
expected: 11,
     got: 12 (using ==)
./unit_test.rb:17

Uh-oh! The third expectation did not meet. See how the error message uses the fact that the expectation knows both the expected and actual values.

RSpec is about Behavior Driven Development. It's about encouraging conversation about testing and looking at it in different ways. It's about illuminating the design, specification, collaboration and documentation aspects of tests, and thinking of them as executable examples of behavior.

Cucumber

Features of Cucumber

Its a Behavior Driven Development(BDD) tool for Ruby.
It focuses on story styling plain English Text.
It follows GWT(Given,When,Then) pattern

Installing Cucumber

If your Environment path is set to " ...\Ruby\bin" then open command prompt and run gem install cucumber.

Usage

Let us first create a feature file to describe about our requirements

Feature: Addition
  In order perform addition of two numbers
  As a user
  I want the sum of two numbers

  
Scenario: Add two numbers
   Given I have entered <input1>
   And   I have entered <input2>
   When  I give add
   Then  The result should be <output>

Then we will create a ruby file to give a code implementation

require 'calculator'

Before do
 @input1=3
 @input2=4
end

Given /^I have entered <input(\d+)>$/ do |arg1|
   @calc = Calculator.new(@input1,@input2)
end

When /^I give add$/ do
  @result = @calc.addition
end

Then /^The result should be <output>$/ do
 puts @result
end

Now if we run the feature file we will get the following

Feature: Addition
  In order perform addition of two numbers
  As a user
  I want the sum of two numbers

  Scenario: Add two numbers            # calculator.feature:7
    Given I have entered <input1>      # addition.rb:8
    And I have entered <input2>        # addition.rb:8
    When I give add                    # addition.rb:12
7
    Then The result should be <output> # addition.rb:16

1 scenario (1 passed)
4 steps (4 passed)
0m0.010s

Comparison of Test::Unit, Shoulda, RSpec

Test::Unit	Shoulda	RSpec
Test::Unit is a testing framework provided by Ruby.It is based on Test Driven Development.	Shoulda is an extension to Test::Unit. It is basically Test::Unit with more capabilities and simpler readable syntax.	RSpec is a Behavior Driven Development framework provided by Ruby.
Test::Unit provides assertions,test methods like setup,teardown,test fixtures,test runners and test suites	Shoulda provide macros,helpers like context and should,matchers	RSpec provides a Domain Specific Language to express the behavior of code.Basically RSpec is TDD embraced with documentation.
Test::Unit does not allow nested setup and nested teardown though multiple setup methods and teardown methods are provided in Test::Unit 2.0	Shoulda allows nested contexts which helps to create different environment for different set of tests.	RSpec allows nested describe structure.

Conclusion

I like RSpec best since I find the output to be most readable. I love the pending keyword, which allows me to set up the tests to be written later on. I find it helps focus on exactly one test and one failure. Shoulda would be our second choice because the tests are just as readable as RSpec, even if the output takes some learning to read.

Comparison of run times

The following are the run times for these three frameworks testing the same calculator class with 4 succesful tests -

RSpec

Finished in 0.023002 seconds

4 examples, 0 failures

Shoulda

Finished in 0.002 seconds.

4 tests, 4 assertions, 0 failures, 0 errors

Test::unit

Finished in 0.001 seconds.

4 tests, 4 assertions, 0 failures, 0 errors

External Links

RSpec
RSpec Documentation

CSC/ECE 517 Fall 2010/ch1 1f TU

Contents

Unit-testing frameworks for Ruby

Unit Testing

Benefits

Unit-testing frameworks

List of most popular unit testing frameworks for Ruby

Test::Unit

Features of Test::Unit

Usage

Assertions

Test Fixture

Test Method

Test Runners

Test Suite

Example of a test class

Shoulda

Features of Shoulda

Installing Shoulda

Usage

RSpec

Features of RSpec

Installing RSpec

Usage

Initial spec

Establishing behavior

Writing the application code

Developing the code

Cucumber

Features of Cucumber

Installing Cucumber

Usage

Comparison of Test::Unit, Shoulda, RSpec

Conclusion

Comparison of run times

External Links

Navigation menu

CSC/ECE 517 Fall 2010/ch1 1f TU

Unit-testing frameworks for Ruby

Unit Testing

Benefits

Unit-testing frameworks

List of most popular unit testing frameworks for Ruby

Test::Unit

Features of Test::Unit

Usage

Assertions

Test Fixture

Test Method

Test Runners

Test Suite

Example of a test class

Shoulda

Features of Shoulda

Installing Shoulda

Usage

RSpec

Features of RSpec

Installing RSpec

Usage

Initial spec

Establishing behavior

Writing the application code

Developing the code

Cucumber

Features of Cucumber

Installing Cucumber

Usage

Comparison of Test::Unit, Shoulda, RSpec

Conclusion

Comparison of run times

External Links

Navigation menu

Search