CSC/ECE 517 Fall 2012/ch1 1w3 pl

Unit-Testing Frameworks for Ruby: Cucumber

This wiki-page serves as a knowledge source for understanding Unit-Testing Frameworks available for Ruby particularly Cucumber .

Introduction

Unit testing is a software development process for testing individual units of source code independently for proper operation ^[1]. An unit testing framework helps the process of unit testing by automating the test cases which are written to ensure the correctness of the system . Ruby provides a framework in its standard library for setting up, organizing, and running tests called Test::Unit ^[2]. Other testing framework available for ruby are Shoulda, RSpec, Cucumber.

Cucumber

Cucumber is one of the latest unit test frameworks to have come for Ruby as part of the RSpec family of tools. Cucumber is written in the Ruby programming language and adheres to behaviour driven development(BDD). Behaviour Driven Development is an Agile Development process that comprises aspects of Acceptance Test Driven Planning , Domain Driven Design and Test Driven Development (TDD). Cucumber is designed specifically to ensure the acceptance tests can easily be read and written by anyone.

Behaviour Driven Development(BDD)

BDD preserves the basic iterative (fail-pass) workflow of TDD, but stresses on specifying behaviors that are understandable to people (say from non programming background). In this approach we write tests in a natural language such that even a non programmer can understand.

Acceptance Test

Acceptance Test Drive Planning is practice that stresses on the importance of the identifying the functions and requirements of the software beforehand so that we exactly get to know when our development phase is over. This practice results in fewer bugs , shorter delivery times and great customer satisfaction.

Cucumber Acceptance Test

     Feature: Sign up
     Sign up should be quick and friendly.

     Scenario: Successful sign up
     New users should get a confirmation email and be greeted personally by the site once signed in.
     Given I have chosen to sign up
     When I sign up with valid details
     Then I should receive a confirmation email
     And I should see a personalized greeting message

     Scenario: Duplicate email
     Where someone tries to create an account for an email address that already exists.
     Given I have chosen to sign up
     But I enter an email address that has already registered
     Then I should be told that the email is already registered
     And I should be offered the option to recover my password

Acceptance tests written in this style are more than just tests; they are executable specifications. As of January 2012, Cucumber was the second most popular testing framework after RSpec for Ruby

How Cucumber works

Cucumber is a command-line tool. It reads in specifications from plain-language text files called features, examines them for scenarios to test, and runs the scenarios against your system. The feature files should be written according to specific syntax rules called Gherkin. Along with the features, you give Cucumber a set of step definitions, which map the business-readable language of each step into Ruby code to carry out whatever action is being described by the step

Cucumber takes customer understandable user stories as input and does integration and acceptance tests on the system by using the user stories as its input. Cucumber acts as the bridge between the customer and the developer. User stories are customer understandable but at the same time can be used to create test cases from it. This does pose some restrictions on the user stories that are written.

Feature

A high level statement of the what the test case actually does and there is only one feature per user story.

Scenario

A Scenario consists of one or more steps which describe about the various actions performed by the user with respect to this particular feature. There can be one or more scenarios per feature in a user story. A feature usually contains 3 to 8 steps in it. The Steps of the scenario always begin with one of the below mentioned key words :

1. Given : Steps that begin with given represent the state of the world before an event.(preconditions)
2. When : Steps that represent the event.
3. Then : Steps that represent the expected outcome.
4. And and But : Steps that extend the previous step.

Step Definition

/* Content */

Cucumber Testing Stack

/* Some Content */

Cucumber Testing Stack

Advantages of Cucumber

Advantages of Cucumber

• Specifications can be written in more than forty different spoken languages
• Cucumber helps build bridges between the technical and nontechnical members of a software team
• Cucumber tests can be easily read and written by business stakeholders
• Writing Cucumber tests are quick , easy , efficient and understable.

Cucumber Installation

To install cucumber we should make sure that the Environment path is set to “..../Ruby/bin”. Then open the command prompt and run

    gem install cucumber 

Once cucumber gem is installed then we would be able to check whether it is installed by issuing the following command:

    cucumber --version

this should give a version number like “1.2.1”

Cucumber uses Rspec for assertions and hence we need Rspec installed to use Cucumber. Use the following command to install rspec

    gem install rspec

Cucumber depends on a particular gherkin version which will be downloaded automatically when we install Cucumber. Suppose we have a later version of gherkin installed in our system then RubyGems get confused and hence the required version of gherkin will not be downloaded and used. In order to resolve this issue we might have to give this command :

    gem install gherkin --version 2.2.9

Testing a Sample Ruby Application using Cucumber

Step 1 : Create a new directory

    mkdir HelloCucumber 

Step 2 : Install Cucumber and Rspec

    gem install cucumber 
    gem instal rspec

Step 3 : Create a directory features and create a file basic.feature in the features directory mkdir features

     mkdir features

Content of features/basic.feature

    Feature: Hello World Feature

    Scenario: Hello World Scenario
    Given The Action is Hello
    When The Subject is World
    Then The Greeting is Hello, World

Step 4: Create step_definitions directory inside features directory. Create basic_step.rb file inside step_difinitions.

     mkdir features/step_definitions

Content of features/step_definitions/basic_step.rb

     require 'rspec/expectations'

     Given /The Action is ([A-z]*)/ do |action|
        @action = action
     end

     When /The Subject is ([A-z]*)/ do |subject|
       @subject = subject
     end

     Then /The Greeting is (.*)/ do |greeting|
       greeting.should == "#{@action}, #{@subject}"
     end

Cucumber uses regular expression after the keywords Given, When and Then to map lines in a feature file to step definitions.

Step 5: Running cucumber command you will see the following output

    Feature: Hello World Feature

    Scenario: Hello World Scenario    # features\basic.feature:3
    Given The Action is Hello         # features/step_definitions/basic_steps.rb:3
    When The Subject is World         # features/step_definitions/basic_steps.rb:7
    Then The Greeting is Hello, World # features/step_definitions/basic_steps.rb:11

    
    1 scenario (1 passed)
    3 steps (3 passed)
    0m0.016s
    Step 6 : Modify the basic.features file to include 2 scenario

Step 6 : Modify the basic.features file to include 2 scenarios

    Feature: Hello World Feature

    Scenario: Hello World Scenario
    Given The Action is Hello
    When The Subject is World
    Then The Greeting is Hello, World

    Scenario: Bye World Scenario
    Given The Action is Bye
    When The Subject is World
    Then The Greeting is Hello, World

Step 7 : Running the cucumber command you will see the following output

   Scenario: Hello World Scenario      # features\basic.feature:3
   Given The Action is Hello         # features/step_definitions/basic_steps.rb:3
   When The Subject is World         # features/step_definitions/basic_steps.rb:7
   Then The Greeting is Hello, World # features/step_definitions/basic_steps.rb:11

   Scenario: Bye World Scenario       # features\basic.feature:8
   Given The Action is Bye          # features/step_definitions/basic_steps.rb:3
   When The Subject is World        # features/step_definitions/basic_steps.rb:7
   Then The Greeting is Hello, World # features/step_definitions/basic_steps.rb:11
     expected: "Bye, World"
          got: "Hello, World" (using ==) (RSpec::Expectations::ExpectationNotMetError)
     ./features/step_definitions/basic_steps.rb:12:in `/The Greeting is (.*)/'
     features\basic.feature:11:in `Then The Greeting is Hello, World'

   Failing Scenarios:
   cucumber features\basic.feature:8 # Scenario: Bye World Scenario

   2 scenarios (1 failed, 1 passed)
   6 steps (1 failed, 5 passed)
   0m0.033s

The Bye Scenario fails since the expected output is “Hello, World” but the resulting output is “Bye,World”.

Comparison of Unit Test Frameworks : Rspec, Shoulda and Cucumber

Rspec	Shoulda	Cucumber
RSpec is based on the Behaviour driven development model provided by Ruby. It has its origins in Test Driven Development and Domain Driven Design.	Shoulda is based on the Test:Unit which is another Ruby Test Framework. It is based on Test Driven Development.	Cucumber is a combination of Test Driven Development and Behaviour driven development.