CSC/ECE 517 Fall 2011/ch2 2e kt

Ruby Testing Frameworks

Mini::Test

Evolution of Mini::Test

Although Ruby’s Test::Unit has been used for years and is a favorite (mostly due to its inclusion with the Ruby standard library), many Ruby developers felt the need for a more modern test infrastructure. This caused them to abandon Test::Unit and pull in additional test gems (e.g. rspec, shoulda, cucumber, etc.). With the new standard Mini::Test, however, this may be a thing of the past. Mini::Test was created to be small, clean and fast. Test::Unit could be rather slow and contained little-used features, such as test cases, GUI runners and some assertions. Mini::Test provides 90% of the functionality of Test::Unit that people were actually using, as well as some additional features.

Most of the assertions in Mini::Test are the same as those in its predecessor. The major difference is in the negative assertions. In Test::Unit where you have a assert_not_something method, Mini::Test provides a refute_something method. (assert_not_raise and assert_not_throws are no longer available.) Mini::Test provides the following assertions:

assert	assert_block	assert_empty	refute	refute_empty
assert_equal	assert_in_delta	assert_in_epsilon	refute_equal	refute_in_delta	refute_in_epsilon
assert_includes	assert_instance_of	assert_kind_of	refute_includes	refute_instance_of	refute_kind_of
assert_match	assert_nil	assert_operator	refute_match	refute_nil	refute_operator
assert_respond_to	assert_same	assert_output	refute_respond_to	refute_same
assert_raises	assert_send	assert_silent	assert_throws

Additional Features

Aside from the API improvements, Mini::Test also provides some additional features such as test randomization. In unit testing, tests should run independent from each other (i.e. the outcome or resulting state(s) of one test should not affect another). By randomizing the order, Mini::Test prevents tests from becoming order-dependent. Should you need to repeat the order to test for such issues, Mini::Test provides the current seed as part of the output and gives you the option to run the test using this same seed.

Mini::Test also gives you the ability to skip tests that are not working correctly (for debug at a later time). It also provides additional options for determining the performance of your test suite. But perhaps one of the best improvements is mini/spec - a BDD framework like RSpec for those programmers who prefer to use spec expectations over test assertions. Mini::Test contains the following expectations:

must_be	must_be_close_to	must_be_empty	wont_be	wont_be_close_to	wont_be_empty
must_be_instance_of	must_be_kind_of	must_be_nil	wont_be_instance_of	wont_be_kind_of	wont_be_nil
must_be_same_as	must_be_silent	must_be_within_delta	wont_be_same_as	wont_be_within_delta
must_be_within_epsilon	must_equal	must_include	wont_be_within_epsilon	wont_equal	wont_include
must_match	must_output	must_raise	wont_match
must_respond_to	must_send	must_throw	wont_respond_to

Code Example - TDD

require 'minitest/autorun'
require_relative 'account.rb'

class AccountTest < MiniTest::Unit::TestCase

  def setup
         @a = Account.new(100)
  end

  def test_deposit
    assert_equal(200, @a.deposit(100))
  end
 
  def test_withdrawal
    assert_equal(50, @a.withdrawal(50))
  end
 
  def test_name
    @a.name = "Checking"
    refute_nil(@a.name())
    assert_match(@a.name, "Checking")
  end

  def test_interest
    assert_in_delta(@a.addinterest(0.333), 130, 5)
  end

  def test_fail
    assert_equal(@a.balance(), 200)
  end
 
  def test_whatru
      assert_instance_of(Account, @a)
  end
     
end

Gives the following results:

Loaded suite C:/Users/Tracy2/Desktop/NCSU/CSC 541/Workspace/Account/test_account2
Started
.F....
Finished in 0.001000 seconds.

  1) Failure:
test_fail(AccountTest) [C:/Users/Tracy2/Desktop/NCSU/CSC 541/Workspace/Account/test_account2.rb:30]:
Expected 100, not 200.

6 tests, 8 assertions, 1 failures, 0 errors, 0 skips

Test run options: --seed 13360

Code Example - BDD

 require 'minitest/autorun'
 require_relative 'account.rb'

 describe Account do

   before do
      @a = Account.new(100)
   end

   describe "deposit" do
     it "should add amount to balance" do
       @a.deposit(100).must_equal 200
     end
   end
 
   describe "withdraw" do
     it "should subtract amount from balance" do
       @a.withdrawal(50).must_equal 50
     end
   end
 
   describe "set name" do
     it "should set account name" do
       @a.name = "Checking"
       @a.name.wont_be_nil
       @a.name.must_match "Checking"
     end
   end
 
   describe "interest" do
     it "should add interest to balance" do
       @a.addinterest(0.333).must_be_within_delta(130, 5)
     end
   end

   describe "fail" do
     it "should fail" do
       @a.balance.must_equal 200
     end
   end

   describe "what are you" do
     it "should be an instance of Account" do
       @a.must_be_instance_of Account
     end
   end

 end

Gives the following results:

Loaded suite C:/Users/Tracy2/Desktop/NCSU/CSC 541/Workspace/Account/test_account3
Started
.F....
Finished in 0.003000 seconds.

  1) Failure:
test_0001_should_fail(AccountSpec::FailSpec) [C:/Users/Tracy2/Desktop/NCSU/CSC 541/Workspace/Account/test_account3.rb:39]:
Expected 200, not 100.

6 tests, 8 assertions, 1 failures, 0 errors, 0 skips

Test run options: --seed 21728

Install and Use

One of the best features of Mini::Test is that it is included the Ruby 1.9 standard library (No install required!!). If you are running Ruby 1.8, you can run gem install minitest to get Mini::Test.

Using Mini::Test is relatively intuitive. As creator, Ryan Davis says, "There is no magic". (View Ryan's presentation on Mini::Test at the Cascadia Ruby 2011 conference.) Mini::Test is clean, simple, straightforward and with the addition of mini/spec there is something for everyone!

Criteria

• IDE integration
• Test output detail
• Testing constructs available
• TDD or BDD
• Documentation
• Learning curve and ease of use

Framework Matrix

Framework	Website	Documentation	IDE Integration	Type	Ease of Use
Unit::Test
MiniTest::Unit	GitHub	RubyDoc

Demo Code

 class Account
   @balance
   @name
 
   attr_accessor :balance
   attr_accessor :name
 
   def initialize(amount)
     @balance = amount
   end
 
   def deposit(amount)
     @balance += amount
   end
 
   def addinterest(rate)
     @balance *= (1 + rate)
   end
 
   def withdrawal(amount)
     @balance -= amount
   end
 end

Test::Unit

 require "test/unit"
 require_relative("../Account.rb")
 
 class AccountTest < Test::Unit::TestCase
 
   def test_balance
     a = Account.new(100)
     assert_equal(100, a.balance())
   end
 
   def test_deposit 
     a = Account.new(100)
     assert_equal(200, a.deposit(100))
   end
   
   def test_withdrawal
     a = Account.new(100)
     assert_equal(50, a.withdrawal(50))
   end
   
   def test_name
     a = Account.new(100)
     a.name = "Checking"
     assert_not_nil(a.name())
   end
 
   def test_interest
     a = Account.new(100)
     assert_equal(a.addinterest(0.5), 150)
   end
 
   def test_fail
     a = Account.new(100)
     assert_equal(a.balance(), 200)
   end
 end

References

[1] http://ruby-toolbox.com/categories/testing_frameworks.html