CSC/ECE 517 Fall 2012/ch1b 1w39 sn

Introduction

This article is a summary of Lecture 10 "Testing in Rails"<ref>http://guides.rubyonrails.org/testing.html</ref> and it basically describes in detail the various types of tests in rails which one might encounter while developing a typical rails application. There are five components central to testing in rails: Fixtures, Unit tests, Functional tests, Integration tests and Performance tests. These have been described below.

Software Testing

In the simplest terms, software testing can be summarized as follows. We provide some test inputs to the software and we get some test outputs from the software. Then we check if the output is acceptable or not. If the output is acceptable then the test case has passed, otherwise it has failed and we have to debug it. The hard part of doing software testing is selecting a good set of test inputs and designing good acceptability tests.

But while testing the software, we have to keep the following things in mind. We have to find bugs as early as possible. The earlier we find the bug, the cheaper it is to fix it. Also, more testing is not always better. We may write a lot of test cases but they still may not cover every functionality of our software.

Setup test environment in Rails

In-Memory Databases

Since all tests involve a high amount of database interaction, it is highly recommended to install the gem 'memory_test_fix'<ref>http://agilewebdevelopment.com/plugins/memory_test_fix</ref> which basically (monkey) patches all tests in rails. This gem allows your tests to mock up a database within the memory, so that all reads/writes to the database executed by the test (when they run) are done to memory instead of the disk. This helps run all the unit tests a lot faster than what they would, if they were to read/write all their results to files (on the disk). It eliminates file locking issues on the test database when running on Windows. This is not a requirement, but it improves the speed of testing and development which is ultimately desirable. Most importantly it is good for testing because one usually does not need the data after the test is done, but only needs it during the lifetime of the test.

Make the following change to the 'config/database.yml' file:

test:
  adapter: sqlite3
  database: ":memory:"

The change is that the 'database:' field has been changed from:

db/development.sqlite3 to ":memory:"

This now ensures that for all the tests, the database used will be the one in memory and not in an actual Sqlite database.

Database Setup

Rails provides a basic boiler plate to create tests.There are three environments provided by Rails - production,development and testing.As the names suggest they are used for different purposes.This prevents developers from messing with their development environments.Inside the rails app directory there will be a directory called test.This directory contains folders-unit,functional,integration and fixtures.The unit folder holds tests for the models, the functional folder is meant to hold tests for your controllers, and the integration folder contains tests that involve any number of controllers interacting.Fixtures contain the sample test data.Rails has the Test::Unit included by default but there are other frameworks also available like RSpec<ref>http://rspec.info/</ref>,Cucumber(for behavior driven development),Shoulda <ref>https://github.com/thoughtbot/shoulda#readme</ref>.When we create the rails scaffold for a particular model then it creates the directories unit,functional,integration which contains the different tests for the respective models.After the test cases have been written we need to prepare the test db.

 rake db:migrate
 rake db:test:load

This two commands should suffice but a complete reference of rake commands for testing purpose is mentioned in <ref>http://guides.rubyonrails.org/testing.html#preparing-your-application-for-testing</ref> After preparing everything we are now ready to run our test.If you are using a Integrated Development Environment(IDE) like RubyMine then you need not worry anything and just do right click on the unit test folder->Select Run->All tests in unit.The figure provided below presents a better picture.

If you are using command line then you can use the following options

ruby -Itest test/unit/post_test.rb
Loaded suite unit/post_test
Started
.
Finished in 0.023513 seconds.
 
2 tests, 2 assertions, 0 failures, 0 errors

Fixtures

Rails tests are data-driven, which means that all of its tests need some sort of sample data to run on. Fixtures<ref>http://ar.rubyonrails.org/classes/Fixtures.html</ref> allow the tester to populate the testing database before any of the tests in the test folder can run. Fixtures have a file format which describes data structures in a human readable format and can be found under the 'test/fixtures' directory. When the rails generate model is executed to create a new model, fixture stubs are automatically created and placed in that directory. YAML fixtures are stored in a single file per model i.e. for every model there is a corresponding fixture. Each record is given a name and is followed by an indented list of key/value pairs in the "key: value" format. When you create a fixture, it generates an internal hash table. Fixtures are hash objects which can be accessed directly because it is automatically setup as a local variable for the test case. The good thing about this is that we can reference these objects using symbolic names. So if we were to declare a fixture called ':cookie' (see example below), we could reference the entire cookie record simply by:

categories(:cookie)

This will return the hash for the fixture named cookie which corresponds to a row in the recipe table describing the recipe for that cookie.

On creating the model, the default fixtures generated are of the form:

one:
  title: MyString
  description: MyString
  instructions: MyText
  
two:
  title: MyString
  description: MyString
  instructions: MyText

We spoke of the :cookie fixture which would be defined as:

cookie:
  Title: Biscuit
  Description: Round and Small 
  Instructions: Buy and bake them 

This allows us to access this entire record using the symbolic name ':cookie' which hashes to this particular fixture.

An important feature of YAML fixtures is that it supports Embedded Ruby i.e. we can embed ruby code into fixtures to generate a large set of sample data. For example:

<% (1..1000).each do |i| %>
fix_<%= i %>:
  name: category_<%= i %>
<% end %>

This would create a thousand fixtures having symbolic names fix_1, fix_2 up to fix_1000, each one of them having a corresponding name attribute category_1, category_2 etc. This is a much better alternative than having to copy-paste the fixture fixture a thousand times.

A very important thing to remember about fixtures is that the ones which are generated by default by the scaffolds do not factor in for any foreign-key relationships that might be present in the models. Thus, such references have to be explicitly added to the fixture manually in order to reflect any 'has-many' or 'belongs-to' relationships across models.

Unit Testing

If the application was created using the scaffold command then it should create a stub in test/unit directory.The initial code would look something like this

require 'test_helper'
 
class PostTest < ActiveSupport::TestCase
  # Replace this with your real tests.
  test "the truth" do
    assert true
  end
end

Now if we wanted to add real tests to it then let us take two scenarios 1.Post with empty entries. 2.Post with actual entries The code for these two test cases would look something like this

require 'test_helper'

class PostTest < ActiveSupport::TestCase

  test "Post new empty" do
    p = Post.new
    assert !p.save, "Saved post without title, content, user, or category"
    assert p.invalid?
  end

  test "Post new correct" do
    p = Post.new
    #Post has following fields title,email,content
    p.title = 'General title'
    p.content = 'A new content'
    p.email = 'Azrael@ncsu.edu'
    #place an assert .so as to find out whether this statement is valid or not
    assert p.valid?
  end

end

test_helper.rb contains the default configuration to run the tests,ActiveSupport::TestCase defines the basic methods for defining a test case.The test cases must begin with the name "test". The statement that actually determines whether the test has passed or not is the assert statement.An assertion is a line of code that evaluates an object (or expression) for expected results.It can check a variety of things like is the expression true or false,is it valid etc. In this example, in the first test case we are checking whether p is an invalid object,if yes then the test has passed because that is the expected thing.Whereas the second test checks whether p is an valid object or not,if its not then the test fails as the expected output in this case is that p should be a valid object.

Here is another example where we try to test the functionality where a user tries to register with an already existing username

test "username exists" do
    user = User.new(:username => "abcdef", :password => "abcdef", :password_confirmation => "abcdef")
    user.save
    user1 =User.new(:username => "abcdef", :password => "abcdef", :password_confirmation => "abcdef")
    assert_false user1.save
end

Here, a user tries to add a post with a valid title but he leaves the content field blank.

test "empty content test" do
    post = Post.new( :title => "No content for this post",:content => nil  )
    post.User_id=1;
    assert_false post.save
end

Functional Testing

If unit tests covered models then functional tests took care of the controllers.The basic purpose of writing functional tests is to check if all the methods of a controller are working correctly. Since the controllers often influence the content of the web page (which is rendered by the corresponding method of a controller) functional tests are typically written to check if the controller’s method is rendering/redirecting to the correct page, whether or not the users are getting authenticated correctly, validating the correctness of the content displayed on the page,etc.Lets say we have a application where users are allowed to post and then comment on those posts.After the user has made a comment then he has to get redirected to that particular post page.Here is how the create method of the comment controller looks like

def create
    #@comment = Comment.new(params[:comment])
    if(session[:email] == nil)
      redirect_to :root
      return
    end
    @comment = Comment.new
    @comment.post_id = params[:id]
    @comment.content = params[:content_new]
    @comment.email = session[:email]
    @comment.vote_count = 0

    @post = Post.find(@comment.post_id)

    dateTime = Time.new
    timestamp = dateTime.to_time
    @post.update_attributes(:updated_at => timestamp)

    respond_to do |format|
      if @comment.save
        format.html { redirect_to :back }
        format.json { render json: @comment, status: :created, location: @comment }
      else
        format.html { render action: "new" }
        format.json { render json: @comment.errors, status: :unprocessable_entity }
      end
    end
  end

As it can be seen if no there is no session then no one can comment.If a user is successfully able to comment then he is redirected to the specific post page for which the comment was made.The functional test for this piece of code would look like this

class CommentsControllerTest < ActionController::TestCase
  setup do
    @comment_new = Comment.new(:content => "Comment to create", :email => "test@gm.com", :post_id => 1)
    @post = Post.find(@comment_new.post_id)
    @comment = comments(:one)#The fixtures contain a row named one
  end

test "should create comment" do
    assert_difference('Comment.count') do
      post :create,  { content: @comment_new.content, email: @comment_new.email, post_id: @comment_new.post_id } #parameters that goes with the post request
    end

    assert_redirected_to post_path(assigns(:post))
  end
end

As we see from the code that it is important to set the session variable and also we need to know before hand for which post are we commenting so we set those variables in the setup method itself.Inside the test method we attempt to create a new comment and after that we check in the assert statement whether it has been redirected to the correct path which in this case is the post page for which the comment has been made.

Testing the response to your request by asserting the presence of key HTML elements and their content is a useful way to test the views of your application. The assert_select assertion allows you to do this by using a simple yet powerful syntax.

In the example below, we test the functionality upon deleting a user.

test "should destroy user" do
    assert_difference('User.count', -1) do
      delete :destroy, id: @user
    end
    assert_redirected_to users_path
end

Integration Tests

Typically in software development, different modules of a project are worked on by different teams/developers. Each team might ensure that the model works correctly in-itself, but this might not necessarily be the case when all the modules are coupled together as a single unit. This is where Integration tests come into play. They test the interaction between multiple controllers and all the components in a sequence, end-to-end. An example of it would be that of a shopping cart application. Even though different phases of the application may work correctly, while running integration tests, one might realize that the 'add to cart' button is absent in the product-catalog, even though the add to cart functionality has been correctly implemented.

The default integration tests framework included in Test-Unit are very low level i.e. they deal with HTTP GET, POST requests responses, session objects, cookies, redirects etc. In Behavioral-Driven-Development we want to deal with the system on a higher level – similar to a user’s interaction with the system i.e. we want to deal only with clicks, with typing etc. Hence, we can use some of the popular Integration Testing frameworks like Capybara which is a GUI testing framework and allows one to specify - within a test - various actions like 'click' to click on a button, 'fill_in' to fill some text into a designated text-box etc. We can see that this is at a high level and somewhat analogous to actions an end-user might go through while using the application. So the rule of thumb while writing integration tests is to identify the end-users requirements and scope of interaction with the system, walk through the steps that they would take and mimic those in the form of tests. It is clearly evident how such Behavioral-Driven-Development goes hand in hand with Test-Driven-Development and helps in removing the ambiguities which are often associated with Customer Requirements.

The following example shows how the test framework CapyBara<ref>http://opinionated-programmer.com/2011/02/capybara-and-selenium-with-rspec-and-rails-3/</ref> is used for Integration Testing:

test “create category from main page” do
  visit categories_path
  click_link “New category”
  fill_in “category_name”, :with => “Sample Category”
  click_button “Create Category”
end

Here we have simulated a user-action (for the CookBook example) where the user would carry out the following steps:

1. Visit the Categories Home Page (whose url is specified as categories_path by the routes.rb file)
2. Click on the Link which says "New Category", which would lead to another page.
3. On this new page, fill the text-box with some text, say "Sample Category"
4. Click on the button that says "Create Category".

One can easily identify these actions from the code which is highly intuitive and self-explanatory. Capybara thus provides us with these convenient methods which greatly expedites the whole Integration Testing process.

To use the framework, simply include the corresponding gem in the Gemfile, and the following lines to the end of the test_helper.rb file.

# Add more helper methods ...
require ‘capybara/rails’

class ActionDispatch::IntegrationTest
  include Capybara::DSL
end

The easy-to-use commands mentioned before are created by Capybara using a Domain Specific Language (DSL) and in order to be able to use it, every Integration test written must require 'test_helper' . This is basically a mixin, so one still has the capability to access all the low-level GET/POST commands in Test-Unit in addition to all the methods offered by Capybara.

Another example of Integration Testing with Test-Framework RSpec Version 1.3.2<ref>https://github.com/rspec/rspec#readme</ref>:

describe "Recipes" do

  before(:all) do
    @recipe = Recipe.new
  end

  it "should not accept empty recipe" do
    @user.should_not_be_valid
    @user.title = "Cookie"
    @user.description = "Chocolate Chip Cookie"
    @user.instructions = "Bake in Oven"
    @user.category = 3
    @user.should_be_valid
  end

end

In the above example we basically test that an empty recipe is invalid and that a recipe with fields filled out is valid. This is a very primitive example of using RSpec and it is just to showcase the difference between RSpec and Capybara, and is in no way a comprehensive example. A thing to note is the before(:all) method, which is similar to the setup() method in Java's JUnit Framework i.e. this method is called before every test in the describe block gets executed.

In sum, Integration tests are vital and are carried out in the final stages of testing to ensure that the system works as a cohesive and complete unit.

Performance Tests

Performance tests as the name indicates are used to gauge the performance of the system and play a very important role in software development for the simple reason that as a developer, one does not want the end user to have a poor experience while using the application. Users do not want to wait long for pages to load and elements on the page to respond. They are not - and should not - be concerned with the capability of the system to handle large loads, scale to accommodate increased volumes of traffic etc. Such details are abstracted away from the user, but they do have a significant impact on user's interaction with the system.

Rails Performance test can be categorized as a special type of integration tests, which are designed for bench-marking and profiling the test code<ref>http://guides.rubyonrails.org/performance_testing.html#modes</ref>. In these tests, one can mention how many connections are to be simulated to the server etc. at the outcome of which it would be possible to identify the performance bottlenecks and hopefully pinpoint the source of speed and/or memory problems.

Detailed examples can be found Here.

Conclusion

Testing is an indispensable and an inevitable part of development in rails.It must be fully exploited to avail the benefits associated with Test-Driven-Development, for the simple reason that rails provides an excellent in-built framework upon which writing tests is a highly natural and intuitive process. There are many advantages to testing and many articles<ref>http://www.learn.geekinterview.com/programming/ruby/ruby-on-rails-application-testing.html</ref> have been written that emphasize this point.

References

<references/>

Most of the content for this article has been obtained from the Lecture taught in class which has been the primary resource. The video of the lecture can be found here.

The old wiki can be found here.