CSC/ECE 517 Fall 2011/ch2 2e ps: Difference between revisions

From Expertiza_Wiki
Jump to navigation Jump to search
 
(92 intermediate revisions by 2 users not shown)
Line 9: Line 9:
=====The Frameworks discussed for Ruby are=====
=====The Frameworks discussed for Ruby are=====


 Test::Unit
[http://pg-server.csc.ncsu.edu/mediawiki/index.php/CSC/ECE_517_Fall_2010/ch1_1f_TU#Test::Unit Test::Unit]


 Shoulda
[http://pg-server.csc.ncsu.edu/mediawiki/index.php/CSC/ECE_517_Fall_2010/ch1_1f_TU#Shoulda Shoulda]


 RSpec
[http://pg-server.csc.ncsu.edu/mediawiki/index.php/CSC/ECE_517_Fall_2010/ch1_1f_TU#RSpec RSpec]


 Cucumber
[http://pg-server.csc.ncsu.edu/mediawiki/index.php/CSC/ECE_517_Fall_2010/ch1_1f_TU#Cucumber Cucumber]


 RIOT
[http://thumblemonks.github.com/riot/ RIOT]


=='''1. The Concept of Unit Testing'''==
=='''The Concept of Unit Testing'''==


 
====Unit====
====1.1 Unit====
A unit is the building block of a code. It can be a class, typically individual methods or lines within methods, an interface etc.
A unit is the building block of a code. It can be a class, typically individual methods or lines within methods, an interface etc.


====1.2 What does Unit Testing mean?====
====What does Unit Testing mean?====
Unit testing is the process of testing a code by focusing on a small chunk (unit) of code at a time, checking whether it functions properly, to give the desired results.
[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_517_Fall_2010/ch1_1f_vn Unit testing] is the process of testing a code by focusing on a small chunk (unit) of code at a time, checking whether it functions properly, to give the desired results.


====1.3 Why Only Unit Testing?====
====Why Unit Testing?====


=====1.3.1 Benefits of Unit Testing:=====
=====Benefits of Unit Testing:=====


 Fast – It gives instant feedback on the accuracy of your code.
 Fast – It gives instant feedback on the accuracy of your code.
Line 42: Line 41:
 Understanding – Helps any reader, easily understand the methods’ functionality.
 Understanding – Helps any reader, easily understand the methods’ functionality.


=====1.3.2 Limitations of Unit Testing:=====
=====Limitations of Unit Testing:=====


 Tedious Job for Complex Codes – Implementing and running test cases for complex codes is time consuming. Moreover, coding a unit test is as tedious as coding the target program itself.
 Tedious Job for Complex Codes – Implementing and running test cases for complex codes is time consuming. Moreover, coding a unit test is as tedious as coding the target program itself.
Line 48: Line 47:
 Impact of Change in the Code Requirements - In an agile project that is accompanied with periodic changes in requirements will result in constant updating of test cases to match the latest developments.
 Impact of Change in the Code Requirements - In an agile project that is accompanied with periodic changes in requirements will result in constant updating of test cases to match the latest developments.


==Testing Techniques:==


===Test Driven Development (TDD):===  
=='''Testing Techniques'''==
[http://en.wikipedia.org/wiki/Test-driven_development Test Driven Development (TDD)]is a type of development where we write test cases before the actual code is written. And these test cases are followed by the required coding, which passes the test cases. To be concise it can be defined as below-
 
*Test cases are written first.
====Test-Driven Development(TDD)====  
*These tests determine what code we need to write.
[http://en.wikipedia.org/wiki/Test-driven_development Test Driven Development (TDD)]is an advanced technique of using automated unit tests to drive the design of software. The result of using this practice is a comprehensive suite of unit tests that can be run at any time to keep track of the code development.  
*Only the Code with associated tests goes into production.
 
*Exhaustive suite of Programmer tests can be maintained..
Here
[[File:TTD development cycle.png|thumb|400px|center|Development Cycle of TTD]]
Test cases are written initially keeping in mind the final result.
 
 The initial test is conducted which eventually fails because of absence of a target code.  
 
 The test suggests further steps to build/refactor an error free code and only those programs associated with the tests go into production.
 
 Tests are then re-conducted until satisfactory results are achieved.
 
[[File:TDD development cycle.png|thumb|400px|center|Development Cycle of TDD]]
 
====Behavior driven development (BDD)====


==='''Behavior driven development (BDD):'''===
[http://en.wikipedia.org/wiki/Behavior_Driven_Development Behavior driven development (BDD)]is a software development technique that relies on the use of simple vocabulary. Tests are designed using natural language so that even a non-technical participant will be able to follow. BDD is an evolution in the thinking behind Test Driven Development but stresses extensively on the behavior aspect of the methods used in the code.
[http://en.wikipedia.org/wiki/Behavior_Driven_Development Behavior driven development (BDD)]is a software development technique that relies on the use of simple vocabulary. Tests are designed using natural language so that even a non-technical participant will be able to follow. BDD is an evolution in the thinking behind Test Driven Development but stresses extensively on the behavior aspect of the methods used in the code.


== Unit Testing Frameworks for Ruby:==
'''Frameworks:''' A framework is a tool to support writing and running of a program.Unit Test framework is a set of reusable libraries or classes for the software system. They are capable of running and checking the test code by themselves.
Some of the extensively used Unit testing frameworks for Ruby are,
*'''Test::Unit'''


*'''Shoulda'''
Eg: Tests result such as “the point cannot be created without code description” is more understandable than something like Test_category_ThrowsArgumentNullException .
 
 
=='''Frameworks'''==
 
A framework is a tool to support writing and running of a program. A Unit Test framework is a set of reusable libraries or classes for a software system. Code is written in these frameworks, they are capable of running and checking the test code by themselves.
 
Here, a unit test framework will evaluate the code based on a variety of inputs provided in the test case and check for the accuracy of results.
 
====Testing Framework Parlance:====
 
a. '''Test Case:'''  It is a main class where test methods are coded. All the units to be tested are inherited to this class.
 
 
b. [http://ruby-doc.org/stdlib/libdoc/test/unit/rdoc/classes/Test/Unit/Assertions.html '''Assertions (Test::Unit::Assertions):''']  An assertion is the main condition that when held true would result in successful culmination of the test. If the assertion fails then the test would generate an error message with pertinent information so that the programmer can make necessary changes.
 
 
c. '''Test Fixture:'''  A test fixture is used to clean up methods for conducting more than one test, eliminating duplication of methods.
 
 
d. '''Test Method:'''  It is a process to handle individual units referred for testing.
 
 
e. '''Test Runners:'''  Test Runners GUI’s that are used to conduct testing and provide with the results. Test runners such as Test::Unit::UI::Console::Test Runner and GTK test runners are used.
 
 
f. '''Test Suite:'''    It is a collection of tests.
 
 


*'''RSpec'''
We need a target code to conduct tests.


*'''Cucumber'''
====Example Problem Statement for Target Code:====
To find whether a given point lies on the x-axis, y-axis or the origin.


*'''Riot'''
Solution:  We write a class to define the position (x-axis, y-axis or origin) of the point.


== Sample Application Code :==  
====Code : Position of a Point====  
  class Point  
  class Point  
   
   
Line 90: Line 123:
   end
   end
  end
  end
  #---------- Location of point on Y-axis --------------#
  #---------- Location of point on Y-axis ----------------#
   def isOnYaxis?
   def isOnYaxis?
     if @xcoordinate == 0
     if @xcoordinate == 0
Line 96: Line 129:
   end
   end
   end
   end
  #---------- Location of point on Origin------------#
  #---------- Location of point on Origin-----------------#
   def isOnOrigin?
   def isOnOrigin?
     if @xcoordinate == 0 && @ycoordinate == 0
     if @xcoordinate == 0 && @ycoordinate == 0
Line 104: Line 137:
  end
  end


=='''1.Test ::Unit :'''==
=='''Testing Frameworks'''==
 
====Test::Unit====


==='''Overview:'''===
=====Overview=====
a.It is based on the Test Driven Development Technique.


b. The Test::Unit framework integrates the following facilities:
a. It is based on the Test Driven Development Technique (TDD).


b. The Test::Unit framework integrates the following features:
1. Way of expressing individual tests.
1. Way of expressing individual tests.


Line 117: Line 153:
3. Has flexible ways of invoking the tests.
3. Has flexible ways of invoking the tests.


==='''Test::Unit Framework Features:'''===
=====Process of Testing=====


==='''a.Test Case: '''===  
=====a. Installation=====
It is a main class, where test methods are coded. All the units to be tested are inherited to this class.


==='''b.Assertions (Test::Unit::Assertions):'''===
Test::Unit framework is pre-installed on all Ruby versions.
An assertion is the main condition that when held true would result in successful culmination of the test. If the assertion fails then the test would generate an error message with pertinent information so that the programmer can make necessary changes.
 
==='''c.Test Fixture:'''===
A test fixture is used to clean up methods for conducting more than one test, eliminating duplication of methods.
 
==='''d.Test Method:'''===
It is a process to handle individual units referred for testing.


==='''e.Test Runners:''' ===  
=====b. Algorithm=====
Test Runners GUI’s that are used to conduct testing and provide with the results.Test runners such as Test::Unit::UI::Console::TestRunner and GTK test runners are unsed.


==='''f.Test Suite:'''===
We use a UI is used to run the Test code and display the gathered results. Here, we have used the web based UI.
It is a collection of tests.
==='''Process of Testing:'''===
==='''Installation:'''===
Test::Unit framework is pre-installed on all Ruby versions.
==='''Algorithm:'''===
* We use a UI is used to run the Test code and display the gathered results. Here, we have used the web based UI .


*. Structuring the tests: Inherit the required classes.
Structuring the tests: Inherit the required classes.


a. Here, “point” class is inherited to call the target units which have to be tested.
a. Here, “point” class is inherited to call the target units which have to be tested.


b. The “test/unit” class is inherited for methods required to conduct testing.
b. The “test/unit” class is inherited for methods required to conduct testing.


* Write the test methods.
Write the test methods.


* Test methods are prefixed with test. Eg: test_examp where examp is the unit being tested.
Test methods are prefixed with test. Eg: test_examp where examp is the unit being tested.


* In the methods write Assertions comparing them with expected results for the target units.
In the methods write Assertions comparing them with expected results for the target units.


* Text fixtures are optional; they can be used to clean the methods.
Text fixtures are optional; they can be used to clean the methods.


* Run the tests as Test::Unit test class name.rb  
Run the tests as Test::Unit test class name.rb  


* To run only a particular method we can its name as classname.rb --name method
To run only a particular method we can its name as classname.rb --name method


* If there are bugs, refactor the code.
If there are bugs, refactor the code.
* Repeat testing until all the bugs are fixed.


==='''Flowchart:'''===
 Repeat testing until all the bugs are fixed.
[[File:Test unit flowchart.png|thumb|400px|center|Development Cycle of TTD]]


==='''CODE Test::Unit'''===
=====Flowchart=====
[[File:Test unit flowchart.png|thumb|400px|center|Development Cycle of TDD]]


=====CODE Test::Unit=====
  require "Point"
  require "Point"
  require "test/unit"
  require "test/unit"
Line 182: Line 202:
   end
   end
  end
  end
==='''Result Test::Unit'''===
Below are the test results that are obtained after running the above test cases,


==='''Failed output'''===
=====Result Test::Unit=====
Below are the test results that are obtained after running the above test cases.
 
=====Failed output=====
This is how the failed test cases are shown once we run the test cases,
This is how the failed test cases are shown once we run the test cases,
[[File:Testunit failed.png|center|700px|center|Development Cycle of TTD]]
[[File:Testunit failed.png|center|700px|center|Development Cycle of TTD]]


==='''Passed output'''===
=====Passed output=====
[[File:Testunit passed.png|center|800px|center|Development Cycle of TTD]]
[[File:Testunit passed.png|center|800px|center|Development Cycle of TTD]]


==='''Storing Test Files:'''===
=====Storing Test Files=====
Test files can be stored in the same directory as the target file but eventually the directory gets bloated storing 2 files for the same target code. Instead we can store it in a separate test directory for better organization. Here, one must take care to clearly specify the path of the target code for inheritance purpose in the test file.
Test files can be stored in the same directory as the target file but eventually the directory gets bloated storing 2 files for the same target code. Instead we can store it in a separate test directory for better organization. Here, one must take care to clearly specify the path of the target code for inheritance purpose in the test file.
Eg: If point.rb is stored in the directory lib and test_point.rb is stored in the test directory then to inherit point.rb we specify the path.  
Eg: If point.rb is stored in the directory lib and test_point.rb is stored in the test directory then to inherit point.rb we specify the path.  
require “../lib/point”
require “../lib/point”
----
====Shoulda====
=====Overview=====
a. Shoulda is a library that allows us to write better and easily understandable test cases for the ruby applications compared to Test::Unit.
b. It allows us to use the "it"-blocks from RSpec to define nice error-messages if a test fails.
c. In other words it is improvised form of Test::Unit and Rspec. It works inside the Test::Unit — you can even mix Shoulda tests with regular Test::Unit test methods.
=====Process of Testing=====
=====a. Installation=====


==2.Shoulda:==
Shoulda is installed by running “gem install shoulda” in command prompt after setting the environment path as “…\Ruby\bin”
Shoulda is a UnitTestFramework , which allows us to write better and easily understandable test cases for the ruby application when compared to Test::Unit. And it allows us to context test with similar feature or property of the application. In other words it is improvised form of Test::Unit and Rspec.


==='''Installation:'''===
=====b. Algorithm=====
This can be installed by running “gem install shoulda”  in command prompt after setting the environment path as “…\Ruby\bin”


==='''Algorithm:'''===
 Inherit all the classes involved in testing i.e. the target class and the shoulda library class. To achieve this add require "rubygems", require ”Point” and require "shoulda" to the test class.
Below is our class " TestCase_Point_Shoulda " which inherits "Test::Unit::TestCase".
*Add require"rubygems", require”Point” and require "shoulda" to the script.


*Then setup up a context “Point” (context is nothing but a region of code which delas with the funcationality you are interested in.)
Then setup up a context “Point” (context is nothing but a region of code which deals with the functionality you are interested in.)


*Then define a method starting with should “...” do and give a description to the method.
Then define a method starting with should “method name” do and then describe the entire method.


*Make assertions in your method.
Make proper assertions to test the functionality in your method.


==='''Code for shoulda:'''===
=====Code shoulda=====
  require "rubygems"
  require "rubygems"
  require “Point”
  require “Point”
Line 232: Line 267:
  end
  end


==='''Result :'''===
=====Result=====
Below are the test results that are obtained after running the above test cases,
Below are the test results that are obtained after running the above test cases,


==='''Failed Output:'''===
=====Failed Output=====
[[File:Shoulda failed.png|center|800px|center|Development Cycle of TTD]]
[[File:Shoulda failed.png|center|800px|center|Development Cycle of TTD]]


==='''Passed output :'''===
=====Passed output=====
[[File:Shoulda passed.png|center|600px|center|Development Cycle of TTD]]
[[File:Shoulda passed.png|center|600px|center|Development Cycle of TTD]]
----


==3.RSpec:==
====RSpec====
==='''Features of RSpec'''===
 
*RSpec is a [http://en.wikipedia.org/wiki/Behavior_Driven_Development Behavior Driven Development] Unit test tool for ruby .It is the evolution from Test-driven development and Domain Driven Design.
=====Overview=====
*It gives a Domain Specific Language with which you can express executable examples of the expected behaviour of your code.
a. RSpec is Behavior Driven Development Unit test tool for ruby.
*It allows us to write a initial requirement (spec) document, which  thereby allows to write the code based on it. These requirement (spec) document helps us understanding the test case better compared to shoulda
 
b. It allows us to write an initial code specifications (spec) document. The spec document talks about the program’s function.
 
c. Thereby, we can write the code based on the specifications (spec file).
 
d. The (spec) document is written in Domain Specific Language which makes it simple for the programmer to handle.
 
e. Better understanding of the test cases compared to shoulda as the cases and notifications are more descriptive.
 
f. In other words it is improvised form of Test::Unit and Rspec. It works inside the Test::Unit — you can even mix Shoulda tests with regular Test::Unit test methods.
 
=====Process of Testing=====
 
=====a. Installation=====
RSpec is installed by running “gem install rspec” in command prompt after setti ng the environment path as “…\Ruby\bin”
 
=====b. Algorithm=====
 
 We start by describing what our application behaves like. So we write down a specifications file.
 
 Using RSpec's “describe” block we describe the basic functions of the application.
 
 We then write the behaviors/expectations defining what we expect our system to behave like (expectations are similar to assertions in Test::Unit framework).
 
 There are two methods available for checking expectations: should() and should_not(). Spec::Expectations - Object#should(matcher) and Object#should_not(matcher).
 
 The code returns list of all the methods to be implemented.
 
 Main code implementing all the specified methods is written.
 
 Then, a recheck is done for fulfillment of all the expectations.
 
=====Code RSpec=====
 
======Initial Requirement(spec)======


==='''Installing RSpec'''===
This can be installed by running “gem install rspec”  in command prompt after setting the environment path as “…\Ruby\bin”
==='''Test Code :'''===
==='''Initial Requirement(spec)'''===
In the below code describe() method returns an ExampleGroup class(similar to TestCase in Test::Unit).
In the below code describe() method returns an ExampleGroup class(similar to TestCase in Test::Unit).
The it()method returns an instance of the ExampleGroup in which that example is run.
The it()method returns an instance of the ExampleGroup in which that example is run.
Line 295: Line 362:
  end
  end


==='''Failed output:'''===
=====Failed output=====
Below are the test results that are obtained after running the above test cases,
Below are the test results that are obtained after running the above test cases,
[[File:Rspec failed.png|center|600px|center|Development Cycle of TTD]]
[[File:Rspec failed.png|center|600px|center|Development Cycle of TTD]]
----
====Cucumber====
Cucumber is becoming highly popular because of its easy coding features.
=====Overview=====
a. Cucumber is a Behavior Driven Development Unit test tool for ruby.
b. It follows a similar pattern of RSPEC with some added features.


==4.Cucumber:==
c. It allows the behavior of a system to be written in the native language of the programmer as either specs or functional tests.
BDD focuses heavily on the behavior of the unit. Taking another step further towards making testing non-technical user friendly, cucumber addresses a lot of these issues. Here, basically the tests are written in plain English (or any other language). Cucumber will then parse the features described and generate test templates which the developer to execute.
 
*It is of GWT (Given, When,Then) pattern.
d. It implements the GWT (Given, When, Then) pattern. Given is a behavior, accordingly coding is done. When all the specifications are met, then the tests are deemed successful.
*Cucumber itself is written in Ruby.
 
==='''Installing Cucumber'''===
e. Cucumber itself is written in Ruby.
This can be installed by running “gem install cucumber”  in command prompt after setting the environment path as “…\Ruby\bin”
 
=====Process of Testing=====
 
=====a. Installation=====
If your Environment path is set to " ...\Ruby\bin" then open command prompt and run gem install cucumber.
 
=====b. Algorithm=====
 We first create a feature file to describe about our requirements.
 
 Then we will create a ruby file to give a code to implement the function.
 
 Then, a recheck is done for fulfillment of all the expectations.
 
=====Code Cucumber=====


==='''Test Code:'''===
The requirement(spec) file for the cucumber is given below,
The requirement(spec) file for the cucumber is given below,
  Feature: Check the point location
  Feature: Check the point location
Line 329: Line 420:
  end
  end
  When /^I give check$/ do
  When /^I give check$/ do
   @result = @calc.isOnXaxis?
   @result = @point.isOnXaxis?
  end
  end
  Then /^The result should be <output>$/ do
  Then /^The result should be <output>$/ do
  puts @result
  puts @result
  end
  end
----
====RIOT====
RIOT aids in speedy execution of test cases.
=====Overview=====
a. Riot does not run a setup and teardown functions before and after each test like in case of Test::Unit.


==5.Riot:==
b. This is what speeds up the test execution.  
Crisp and more expressive test cases can be obtained using Riot.
It is a very fast unit test framework compared to other unit test frameworks.
Unlike shoulda it is not depended on test::unit
Before running the assertions in the context the setup block is executed.


==='''Installing Riot'''===
c. Tests are written differently in RIOT. We only receive a Boolean output for our test case.
This can be installed by running “gem install riot”  in command prompt after setting the environment path as “…\Ruby\bin”


==='''Test Code:'''===
c. Here, assertions are done using a generalized attribute instead of specific variables.
 
=====Process of Testing=====
 
=====a. Installation=====
 
This can be installed by running “gem install riot” in command prompt after setting the environment path as “…\Ruby\bin”
 
=====b. Algorithm=====
 
 We inherit all the classes that are required.
 
 A “Context” block is used to specify behavior of the class.
 
 Now, a setup block does not use specific variables but instead uses a generalized attribute “topic” to access objects in assertions.
 
 Assertions only return a boolean value. When using asserts, true indicates a pass while false indicates a fail.
 
=====Code RIOT=====
  require “Point”
  require “Point”
  require "rubygems"
  require "rubygems"
Line 351: Line 464:
   asserts("Location on Xaxis") {topic.isOnXaxis?}.nil
   asserts("Location on Xaxis") {topic.isOnXaxis?}.nil
  end
  end
----


==Conclusion:==
 
Test::unit is the traditionally used unit testing framework as it comes default by ruby but based on the readability of the test cases Rspec is preferred by majority of the people.
=='''Comparision between Frameworks'''==
[[File:Comparision.jpg]]
 
 
=='''Conclusion'''==
Test::unit is the traditionally used unit testing framework as it comes by default with ruby but based on the readability of the test cases Rspec is preferred by majority.
Below is the plot showing the comparison of the times taken to execute the test cases in various unit test frameworks,  
Below is the plot showing the comparison of the times taken to execute the test cases in various unit test frameworks,  


Line 359: Line 478:


Hence we can infer from the above tabulation that riot is faster compared to majority of the unit test Frameworks
Hence we can infer from the above tabulation that riot is faster compared to majority of the unit test Frameworks
Each of the frameworks have their specific advantages and are used depending upon the situation.


==See also==
==See also==

Latest revision as of 21:03, 4 October 2011

Testing Frameworks For RUBY


Introduction to Testing Frameworks

A dynamically-typed language such as Ruby relies heavily on extensive testing to ensure all objects operate as they are expected to. Ruby has got a pool of testing frameworks.

This article goes through different testing frameworks available for Ruby and explains how to use them with examples.

The Frameworks discussed for Ruby are

Test::Unit

Shoulda

RSpec

Cucumber

RIOT

The Concept of Unit Testing

Unit

A unit is the building block of a code. It can be a class, typically individual methods or lines within methods, an interface etc.

What does Unit Testing mean?

Unit testing is the process of testing a code by focusing on a small chunk (unit) of code at a time, checking whether it functions properly, to give the desired results.

Why Unit Testing?

Benefits of Unit Testing:

 Fast – It gives instant feedback on the accuracy of your code.

 Design – For testing, the code is designed as small modules facilitating the reusability of the units.

 Individuality – Each unit is designed independent of each other and tested individually making error identification easy.

 Efficiency - Bugs can be scanned in the probable bug prone code areas making debugging efficient.

 Understanding – Helps any reader, easily understand the methods’ functionality.

Limitations of Unit Testing:

 Tedious Job for Complex Codes – Implementing and running test cases for complex codes is time consuming. Moreover, coding a unit test is as tedious as coding the target program itself.

 Impact of Change in the Code Requirements - In an agile project that is accompanied with periodic changes in requirements will result in constant updating of test cases to match the latest developments.


Testing Techniques

Test-Driven Development(TDD)

Test Driven Development (TDD)is an advanced technique of using automated unit tests to drive the design of software. The result of using this practice is a comprehensive suite of unit tests that can be run at any time to keep track of the code development.

Here

 Test cases are written initially keeping in mind the final result.

 The initial test is conducted which eventually fails because of absence of a target code.

 The test suggests further steps to build/refactor an error free code and only those programs associated with the tests go into production.

 Tests are then re-conducted until satisfactory results are achieved.

Development Cycle of TDD

Behavior driven development (BDD)

Behavior driven development (BDD)is a software development technique that relies on the use of simple vocabulary. Tests are designed using natural language so that even a non-technical participant will be able to follow. BDD is an evolution in the thinking behind Test Driven Development but stresses extensively on the behavior aspect of the methods used in the code.


Eg: Tests result such as “the point cannot be created without code description” is more understandable than something like Test_category_ThrowsArgumentNullException .


Frameworks

A framework is a tool to support writing and running of a program. A Unit Test framework is a set of reusable libraries or classes for a software system. Code is written in these frameworks, they are capable of running and checking the test code by themselves.

Here, a unit test framework will evaluate the code based on a variety of inputs provided in the test case and check for the accuracy of results.

Testing Framework Parlance:

a. Test Case: It is a main class where test methods are coded. All the units to be tested are inherited to this class.


b. Assertions (Test::Unit::Assertions): An assertion is the main condition that when held true would result in successful culmination of the test. If the assertion fails then the test would generate an error message with pertinent information so that the programmer can make necessary changes.


c. Test Fixture: A test fixture is used to clean up methods for conducting more than one test, eliminating duplication of methods.


d. Test Method: It is a process to handle individual units referred for testing.


e. Test Runners: Test Runners GUI’s that are used to conduct testing and provide with the results. Test runners such as Test::Unit::UI::Console::Test Runner and GTK test runners are used.


f. Test Suite: It is a collection of tests.


We need a target code to conduct tests.

Example Problem Statement for Target Code:

To find whether a given point lies on the x-axis, y-axis or the origin.

Solution: We write a class to define the position (x-axis, y-axis or origin) of the point.

Code : Position of a Point

class Point 

attr_writer :xcoordinate 
attr_writer :ycoordinate

def initialize(xcoordinate,ycoordinate)
  @xcoordinate = xcoordinate
  @ycoordinate = ycoordinate
end
#-----------Location of point on X-axis ----------------#
def isOnXaxis?
   if @ycoordinate == 0
    return true   	
  end
end
#---------- Location of point on Y-axis ----------------#
 def isOnYaxis?
   if @xcoordinate == 0
     return true
  end
 end
#---------- Location of point on Origin-----------------#
 def isOnOrigin?
   if @xcoordinate == 0 && @ycoordinate == 0
   return true
  end
 end
end

Testing Frameworks

Test::Unit

Overview

a. It is based on the Test Driven Development Technique (TDD).

b. The Test::Unit framework integrates the following features:

1. Way of expressing individual tests.

2. Provides a framework for structuring the tests.

3. Has flexible ways of invoking the tests.

Process of Testing
a. Installation

Test::Unit framework is pre-installed on all Ruby versions.

b. Algorithm

 We use a UI is used to run the Test code and display the gathered results. Here, we have used the web based UI.

 Structuring the tests: Inherit the required classes.

a. Here, “point” class is inherited to call the target units which have to be tested.

b. The “test/unit” class is inherited for methods required to conduct testing.

 Write the test methods.

 Test methods are prefixed with test. Eg: test_examp where examp is the unit being tested.

 In the methods write Assertions comparing them with expected results for the target units.

 Text fixtures are optional; they can be used to clean the methods.

 Run the tests as Test::Unit test class name.rb

 To run only a particular method we can its name as classname.rb --name method

 If there are bugs, refactor the code.

 Repeat testing until all the bugs are fixed.

Flowchart
Development Cycle of TDD
CODE Test::Unit
require "Point"
require "test/unit"
class TestCase_Point < Test::Unit::TestCase 
 def test_isOnXaxis
   assert(Point.new(3,0).isOnXaxis?)
   end
  def test_isOnYaxis
      assert(Point.new(0,4).isOnYaxis?)
 end
  def test_isOnOrigin
      assert(Point.new(3,4).isOnOrigin?)
 end
end
Result Test::Unit

Below are the test results that are obtained after running the above test cases.

Failed output

This is how the failed test cases are shown once we run the test cases,

Development Cycle of TTD
Development Cycle of TTD
Passed output
Development Cycle of TTD
Development Cycle of TTD
Storing Test Files

Test files can be stored in the same directory as the target file but eventually the directory gets bloated storing 2 files for the same target code. Instead we can store it in a separate test directory for better organization. Here, one must take care to clearly specify the path of the target code for inheritance purpose in the test file.

Eg: If point.rb is stored in the directory lib and test_point.rb is stored in the test directory then to inherit point.rb we specify the path. require “../lib/point”



Shoulda

Overview

a. Shoulda is a library that allows us to write better and easily understandable test cases for the ruby applications compared to Test::Unit.

b. It allows us to use the "it"-blocks from RSpec to define nice error-messages if a test fails.

c. In other words it is improvised form of Test::Unit and Rspec. It works inside the Test::Unit — you can even mix Shoulda tests with regular Test::Unit test methods.

Process of Testing
a. Installation

Shoulda is installed by running “gem install shoulda” in command prompt after setting the environment path as “…\Ruby\bin”

b. Algorithm

 Inherit all the classes involved in testing i.e. the target class and the shoulda library class. To achieve this add require "rubygems", require ”Point” and require "shoulda" to the test class.

 Then setup up a context “Point” (context is nothing but a region of code which deals with the functionality you are interested in.)

 Then define a method starting with should “method name” do and then describe the entire method.

 Make proper assertions to test the functionality in your method.

Code shoulda
require "rubygems"
require “Point”
require "test/unit"
require "shoulda"
class TestCase_Point_Shoulda < Test::Unit::TestCase
 context "Point" do
  should "lie on the X-axis " do
     assert(Point.new(3,0).isOnXaxis?)
  end
  should "lie on the Y-axis " do
     assert(Point.new(0,4).isOnYaxis?)
  end
  should "lie on the Origin" do
   assert(Point.new(0,4).isOnOrigin?)
  end	
 end
end
Result

Below are the test results that are obtained after running the above test cases,

Failed Output
Development Cycle of TTD
Development Cycle of TTD
Passed output
Development Cycle of TTD
Development Cycle of TTD


RSpec

Overview

a. RSpec is Behavior Driven Development Unit test tool for ruby.

b. It allows us to write an initial code specifications (spec) document. The spec document talks about the program’s function.

c. Thereby, we can write the code based on the specifications (spec file).

d. The (spec) document is written in Domain Specific Language which makes it simple for the programmer to handle.

e. Better understanding of the test cases compared to shoulda as the cases and notifications are more descriptive.

f. In other words it is improvised form of Test::Unit and Rspec. It works inside the Test::Unit — you can even mix Shoulda tests with regular Test::Unit test methods.

Process of Testing
a. Installation

RSpec is installed by running “gem install rspec” in command prompt after setti ng the environment path as “…\Ruby\bin”

b. Algorithm

 We start by describing what our application behaves like. So we write down a specifications file.

 Using RSpec's “describe” block we describe the basic functions of the application.

 We then write the behaviors/expectations defining what we expect our system to behave like (expectations are similar to assertions in Test::Unit framework).

 There are two methods available for checking expectations: should() and should_not(). Spec::Expectations - Object#should(matcher) and Object#should_not(matcher).

 The code returns list of all the methods to be implemented.

 Main code implementing all the specified methods is written.

 Then, a recheck is done for fulfillment of all the expectations.

Code RSpec
Initial Requirement(spec)

In the below code describe() method returns an ExampleGroup class(similar to TestCase in Test::Unit). The it()method returns an instance of the ExampleGroup in which that example is run.

require "Point"
describe "The location of a point" do
 it "should check whether point lies on X-axis correctly"                        
 it " should check whether point lies on Y-axis correctly " 
 it " should check whether point lies on Origin correctly " 
end

The are two methods available to check the expectations and they are should and should_not. The result obtained by running the initial requirement doc by using $ spec rspec_test.rb is

****
Pending:
The location of a point should check whether point lies on X-axis correctly (Not Yet Implemented)
./02file.rb:2
The location of a point should check whether point lies on Y-axis correctly (Not Yet Implemented)
./02file.rb:3
The location of a point should check whether point lies on Origin correctly (Not Yet Implemented)
./02file.rb:5
Finished in 0.021001 seconds
4 examples, 0 failures, 4 pending

Hence it clearly says that the above methods are yet to be implemented. This helps us in developing the code more efficiently such that they pass the above tests Code block with expectations :

require “Point”
require "rubygems"
describe "The location of a point" do
 before(:each) do
   @point = Point.new(0,2)
 end
 it "should check whether point lies on X-axis correctly" do
   @point.isOnXaxis?.should be_true
 end
 it "should check whether point lies on Y-axis correctly" do
   @point.isOnYaxis?.should be_true
 end
 it "should check whether point lies Origin correctly" do
   @point.isOnOrigin?.should be_true
 end
end
Failed output

Below are the test results that are obtained after running the above test cases,

Development Cycle of TTD
Development Cycle of TTD


Cucumber

Cucumber is becoming highly popular because of its easy coding features.

Overview

a. Cucumber is a Behavior Driven Development Unit test tool for ruby.

b. It follows a similar pattern of RSPEC with some added features.

c. It allows the behavior of a system to be written in the native language of the programmer as either specs or functional tests.

d. It implements the GWT (Given, When, Then) pattern. Given is a behavior, accordingly coding is done. When all the specifications are met, then the tests are deemed successful.

e. Cucumber itself is written in Ruby.

Process of Testing
a. Installation

If your Environment path is set to " ...\Ruby\bin" then open command prompt and run gem install cucumber.

b. Algorithm

 We first create a feature file to describe about our requirements.

 Then we will create a ruby file to give a code to implement the function.

 Then, a recheck is done for fulfillment of all the expectations.

Code Cucumber

The requirement(spec) file for the cucumber is given below,

Feature: Check the point location
 In order perform check
 As a user
 I want the two coordinates of a point
 Scenario: check the location on X-axis
  Given I have entered <xcoordinate>
  And   I have entered <ycoordinate>
  When  I give check
  Then  The result should be <output>

The code implementation based on the above Requirement for Cucumber is as shown below,

require “Point”
require "rubygems"
Before do
@xcoordinate=3
@ycoordinate=0
end
Given /^I have entered <input(\d+)>$/ do |arg1|
  @point = Point.new(@xcoordinate,@ycoordinate)
end
When /^I give check$/ do
 @result = @point.isOnXaxis?
end
Then /^The result should be <output>$/ do
puts @result
end

RIOT

RIOT aids in speedy execution of test cases.

Overview

a. Riot does not run a setup and teardown functions before and after each test like in case of Test::Unit.

b. This is what speeds up the test execution.

c. Tests are written differently in RIOT. We only receive a Boolean output for our test case.

c. Here, assertions are done using a generalized attribute instead of specific variables.

Process of Testing
a. Installation

This can be installed by running “gem install riot” in command prompt after setting the environment path as “…\Ruby\bin”

b. Algorithm

 We inherit all the classes that are required.

 A “Context” block is used to specify behavior of the class.

 Now, a setup block does not use specific variables but instead uses a generalized attribute “topic” to access objects in assertions.

 Assertions only return a boolean value. When using asserts, true indicates a pass while false indicates a fail.

Code RIOT
require “Point”
require "rubygems"
context "The point locater" do
  setup{Point.new(3,0)}
  asserts("Location on Xaxis") {topic.isOnXaxis?}.nil
end


Comparision between Frameworks


Conclusion

Test::unit is the traditionally used unit testing framework as it comes by default with ruby but based on the readability of the test cases Rspec is preferred by majority. Below is the plot showing the comparison of the times taken to execute the test cases in various unit test frameworks,

Development Cycle of TTD
Development Cycle of TTD

Hence we can infer from the above tabulation that riot is faster compared to majority of the unit test Frameworks

Each of the frameworks have their specific advantages and are used depending upon the situation.

See also

External Links:

References:

R.Venkat Rajendran, White Paper on Unit Testing

Benchmark Conclusions