CSC/ECE 517 Fall 2011/ch2 2e ps

Testing Frameworks For RUBY

Introduction to Testing Frameworks

A dynamically-typed language such as Ruby relies heavily on extensive testing to ensure all objects operate as they are expected to. Ruby has got a pool of testing frameworks.

This article goes through different testing frameworks available for Ruby and explains how to use them with examples.

The Frameworks discussed for Ruby are

 Test::Unit

 Shoulda

 RSpec

 Cucumber

 RIOT

1. The Concept of UNIT TESTING

1.1 Unit

A unit is the building block of a code. It can be a class, typically individual methods or lines within methods, an interface etc.

1.2 What does Unit Testing mean?

Unit testing is the process of testing a code by focusing on a small chunk (unit) of code at a time, checking whether it functions properly, to give the desired results.

1.3 Why Unit Testing?

1.3.1 Benefits of Unit Testing:

 Fast – It gives instant feedback on the accuracy of your code.

 Design – For testing, the code is designed as small modules facilitating the reusability of the units.

 Individuality – Each unit is designed independent of each other and tested individually making error identification easy.

 Efficiency - Bugs can be scanned in the probable bug prone code areas making debugging efficient.

 Understanding – Helps any reader, easily understand the methods’ functionality.

1.3.2 Limitations of Unit Testing:

 Tedious Job for Complex Codes – Implementing and running test cases for complex codes is time consuming. Moreover, coding a unit test is as tedious as coding the target program itself.

 Impact of Change in the Code Requirements - In an agile project that is accompanied with periodic changes in requirements will result in constant updating of test cases to match the latest developments.

Concept of UNIT TESTING:

1 Unit:

The smallest building block of a code is known as Unit. It can be a class, typically individual methods or lines within methods, an interface etc.

2.Unit Testing:

Unit testing focuses on small chunks (units) of code checking whether they functions to give the desired results.

3.Advantages of Unit Testing:

Below are the some of the benefits of Unit testing,

• Fast – It gives instant feedback on the accuracy of your code.

• Reusability –For testing,the code is designed as small modules, which facilitates the reusability of units.

• Individuality – Each unit is tested individually making error identification easy.

• Efficiency - Bugs can be scanned in the probable code areas making debugging efficient.

• Understanding –Helps others easily understand the methods’ functionality.

4.Disadvantages of Unit Testing :

Though it provides several benefits it has got few cons too.Below are some of the disadvantages

• Time Consuming – Its consumes lots of time for implementation of complex cases and it augments the complexity too .

• Change in requirement Impact- In agile project, which is accompanied with periodic change in requirements results in rewriting the previous test cases to match the latest requiremnts.

Testing Techniques:

Test Driven Development (TDD):

Test Driven Development (TDD)is a type of development where we write test cases before the actual code is written. And these test cases are followed by the required coding, which passes the test cases. To be concise it can be defined as below-

• Test cases are written first.
• These tests determine what code we need to write.
• Only the Code with associated tests goes into production.
• Exhaustive suite of Programmer tests can be maintained..

Development Cycle of TTD

Behavior driven development (BDD):

Behavior driven development (BDD)is a software development technique that relies on the use of simple vocabulary. Tests are designed using natural language so that even a non-technical participant will be able to follow. BDD is an evolution in the thinking behind Test Driven Development but stresses extensively on the behavior aspect of the methods used in the code.

Unit Testing Frameworks for Ruby:

Frameworks: A framework is a tool to support writing and running of a program.Unit Test framework is a set of reusable libraries or classes for the software system. They are capable of running and checking the test code by themselves. Some of the extensively used Unit testing frameworks for Ruby are,

• Test::Unit

• Shoulda

• RSpec

• Cucumber

• Riot

Sample Application Code :

class Point 

attr_writer :xcoordinate 
attr_writer :ycoordinate

def initialize(xcoordinate,ycoordinate)
  @xcoordinate = xcoordinate
  @ycoordinate = ycoordinate
end
#-----------Location of point on X-axis ----------------#
def isOnXaxis?
   if @ycoordinate == 0
    return true   	
  end
end
#---------- Location of point on Y-axis --------------#
 def isOnYaxis?
   if @xcoordinate == 0
     return true
  end
 end
#---------- Location of point on Origin------------#
 def isOnOrigin?
   if @xcoordinate == 0 && @ycoordinate == 0
   return true
  end
 end
end

1.Test ::Unit :

Overview:

a.It is based on the Test Driven Development Technique.

b. The Test::Unit framework integrates the following facilities:

1. Way of expressing individual tests.

2. Provides a framework for structuring the tests.

3. Has flexible ways of invoking the tests.

a.Test Case:

It is a main class, where test methods are coded. All the units to be tested are inherited to this class.

b.Assertions (Test::Unit::Assertions):

An assertion is the main condition that when held true would result in successful culmination of the test. If the assertion fails then the test would generate an error message with pertinent information so that the programmer can make necessary changes.

c.Test Fixture:

A test fixture is used to clean up methods for conducting more than one test, eliminating duplication of methods.

d.Test Method:

It is a process to handle individual units referred for testing.

e.Test Runners:

Test Runners GUI’s that are used to conduct testing and provide with the results.Test runners such as Test::Unit::UI::Console::TestRunner and GTK test runners are unsed.

f.Test Suite:

It is a collection of tests.

Installation:

Test::Unit framework is pre-installed on all Ruby versions.

Algorithm:

• We use a UI is used to run the Test code and display the gathered results. Here, we have used the web based UI .

• . Structuring the tests: Inherit the required classes.

a. Here, “point” class is inherited to call the target units which have to be tested.

b. The “test/unit” class is inherited for methods required to conduct testing.

• Write the test methods.

• Test methods are prefixed with test. Eg: test_examp where examp is the unit being tested.

• In the methods write Assertions comparing them with expected results for the target units.

• Text fixtures are optional; they can be used to clean the methods.

• Run the tests as Test::Unit test class name.rb

• To run only a particular method we can its name as classname.rb --name method

• If there are bugs, refactor the code.

• Repeat testing until all the bugs are fixed.

Flowchart:

Development Cycle of TTD

CODE Test::Unit

require "Point"
require "test/unit"
class TestCase_Point < Test::Unit::TestCase 
 def test_isOnXaxis
   assert(Point.new(3,0).isOnXaxis?)
   end
  def test_isOnYaxis
      assert(Point.new(0,4).isOnYaxis?)
 end
  def test_isOnOrigin
      assert(Point.new(3,4).isOnOrigin?)
 end
end

Result Test::Unit

Below are the test results that are obtained after running the above test cases,

Failed output

This is how the failed test cases are shown once we run the test cases,

Development Cycle of TTD

Passed output

Development Cycle of TTD

Storing Test Files:

Test files can be stored in the same directory as the target file but eventually the directory gets bloated storing 2 files for the same target code. Instead we can store it in a separate test directory for better organization. Here, one must take care to clearly specify the path of the target code for inheritance purpose in the test file. Eg: If point.rb is stored in the directory lib and test_point.rb is stored in the test directory then to inherit point.rb we specify the path. require “../lib/point”

2.Shoulda:

Shoulda is a UnitTestFramework , which allows us to write better and easily understandable test cases for the ruby application when compared to Test::Unit. And it allows us to context test with similar feature or property of the application. In other words it is improvised form of Test::Unit and Rspec.

Installation:

This can be installed by running “gem install shoulda” in command prompt after setting the environment path as “…\Ruby\bin”

Algorithm:

Below is our class " TestCase_Point_Shoulda " which inherits "Test::Unit::TestCase".

• Add require"rubygems", require”Point” and require "shoulda" to the script.

• Then setup up a context “Point” (context is nothing but a region of code which delas with the funcationality you are interested in.)

• Then define a method starting with should “...” do and give a description to the method.

• Make assertions in your method.

Code for shoulda:

require "rubygems"
require “Point”
require "test/unit"
require "shoulda"
class TestCase_Point_Shoulda < Test::Unit::TestCase
 context "Point" do
  should "lie on the X-axis " do
     assert(Point.new(3,0).isOnXaxis?)
  end
  should "lie on the Y-axis " do
     assert(Point.new(0,4).isOnYaxis?)
  end
  should "lie on the Origin" do
   assert(Point.new(0,4).isOnOrigin?)
  end	
 end
end

Result :

Below are the test results that are obtained after running the above test cases,

Failed Output:

Development Cycle of TTD

Passed output :

Development Cycle of TTD

3.RSpec:

Features of RSpec

• RSpec is a Behavior Driven Development Unit test tool for ruby .It is the evolution from Test-driven development and Domain Driven Design.
• It gives a Domain Specific Language with which you can express executable examples of the expected behaviour of your code.
• It allows us to write a initial requirement (spec) document, which thereby allows to write the code based on it. These requirement (spec) document helps us understanding the test case better compared to shoulda

Installing RSpec

This can be installed by running “gem install rspec” in command prompt after setting the environment path as “…\Ruby\bin”

Initial Requirement(spec)

In the below code describe() method returns an ExampleGroup class(similar to TestCase in Test::Unit). The it()method returns an instance of the ExampleGroup in which that example is run.

require "Point"
describe "The location of a point" do
 it "should check whether point lies on X-axis correctly"                        
 it " should check whether point lies on Y-axis correctly " 
 it " should check whether point lies on Origin correctly " 
end

The are two methods available to check the expectations and they are should and should_not. The result obtained by running the initial requirement doc by using $ spec rspec_test.rb is

****
Pending:
The location of a point should check whether point lies on X-axis correctly (Not Yet Implemented)
./02file.rb:2
The location of a point should check whether point lies on Y-axis correctly (Not Yet Implemented)
./02file.rb:3
The location of a point should check whether point lies on Origin correctly (Not Yet Implemented)
./02file.rb:5
Finished in 0.021001 seconds
4 examples, 0 failures, 4 pending

Hence it clearly says that the above methods are yet to be implemented. This helps us in developing the code more efficiently such that they pass the above tests Code block with expectations :

require “Point”
require "rubygems"
describe "The location of a point" do
 before(:each) do
   @point = Point.new(0,2)
 end
 it "should check whether point lies on X-axis correctly" do
   @point.isOnXaxis?.should be_true
 end
 it "should check whether point lies on Y-axis correctly" do
   @point.isOnYaxis?.should be_true
 end
 it "should check whether point lies Origin correctly" do
   @point.isOnOrigin?.should be_true
 end
end

Failed output:

Below are the test results that are obtained after running the above test cases,

Development Cycle of TTD

4.Cucumber:

BDD focuses heavily on the behavior of the unit. Taking another step further towards making testing non-technical user friendly, cucumber addresses a lot of these issues. Here, basically the tests are written in plain English (or any other language). Cucumber will then parse the features described and generate test templates which the developer to execute.

• It is of GWT (Given, When,Then) pattern.
• Cucumber itself is written in Ruby.

Installing Cucumber

This can be installed by running “gem install cucumber” in command prompt after setting the environment path as “…\Ruby\bin”

Test Code:

The requirement(spec) file for the cucumber is given below,

Feature: Check the point location
 In order perform check
 As a user
 I want the two coordinates of a point
 Scenario: check the location on X-axis
  Given I have entered <xcoordinate>
  And   I have entered <ycoordinate>
  When  I give check
  Then  The result should be <output>

The code implementation based on the above Requirement for Cucumber is as shown below,

require “Point”
require "rubygems"
Before do
@xcoordinate=3
@ycoordinate=0
end
Given /^I have entered <input(\d+)>$/ do |arg1|
  @point = Point.new(@xcoordinate,@ycoordinate)
end
When /^I give check$/ do
 @result = @calc.isOnXaxis?
end
Then /^The result should be <output>$/ do
puts @result
end

5.Riot:

Crisp and more expressive test cases can be obtained using Riot. It is a very fast unit test framework compared to other unit test frameworks. Unlike shoulda it is not depended on test::unit Before running the assertions in the context the setup block is executed.

Installing Riot

This can be installed by running “gem install riot” in command prompt after setting the environment path as “…\Ruby\bin”

Test Code:

require “Point”
require "rubygems"
context "The point locater" do
  setup{Point.new(3,0)}
  asserts("Location on Xaxis") {topic.isOnXaxis?}.nil
end

Conclusion:

Test::unit is the traditionally used unit testing framework as it comes default by ruby but based on the readability of the test cases Rspec is preferred by majority of the people. Below is the plot showing the comparison of the times taken to execute the test cases in various unit test frameworks,

Development Cycle of TTD

Hence we can infer from the above tabulation that riot is faster compared to majority of the unit test Frameworks

See also

• Unit Testing Frameworks for Ruby
• Community Preferred Unit Testing Framework

External Links:

• Test::Unit
• Ruby-doc
• Shoulda
• RSpec
• RSpec Documentation
• Shoulda
• Cucumber
• Unit testing
• Test-driven development
• Riot

References:

R.Venkat Rajendran, White Paper on Unit Testing

Benchmark Conclusions