CSC/ECE 517 Fall 2011/ch4 4e gs

Introduction

This article is a summary of Lecture 10 "Testing in Rails"<ref>http://guides.rubyonrails.org/testing.html</ref> and it basically describes in detail the various types of tests in rails which one might encounter while developing a typical rails application. There are five components central to testing in rails: Fixtures, Unit tests, Functional tests, Integration tests and Performance tests. These have been described below.

Setup prior to Testing

In-Memory Databases:

Since all tests involve a high amount of database interaction, it is highly recommended to install the gem 'memory_test_fix'<ref>http://agilewebdevelopment.com/plugins/memory_test_fix</ref> which basically (monkey) patches all tests in rails. This gem allows your tests to mock up a database within the memory, so that all reads/writes to the database executed by the test (when they run) are done to memory instead of the disk. This helps run all the unit tests a lot faster than what they would, if they were to read/write all their results to files (on the disk). It eliminates file locking issues on the test database when running on Windows. This is not a requirement, but it improves the speed of testing and development which is ultimately desirable. Most importantly it is good for testing because one usually does not need the data after the test is done, but only needs it during the lifetime of the test.

Make the following change to the 'config/database.yml' file:

test:
  adapter: sqlite3
  database: ":memory:"

The change is that the 'database:' field has been changed from:

db/development.sqlite3 to ":memory:"

This now ensures that for all the tests, the database used will be the one in memory and not in an actual Sqlite database.

Fixtures

Rails tests are data-driven, which means that all of its tests need some sort of sample data to run on. Fixtures<ref>http://ar.rubyonrails.org/classes/Fixtures.html</ref> allow the tester to populate the testing database before any of the tests in the test folder can run. Fixtures have a file format which describes data structures in a human readable format and can be found under the 'test/fixtures' directory. When the rails generate model is executed to create a new model, fixture stubs are automatically created and placed in that directory. YAML fixtures are stored in a single file per model i.e. for every model there is a corresponding fixture. Each record is given a name and is followed by an indented list of key/value pairs in the "key: value" format. When you create a fixture, it generates an internal hash table. Fixtures are hash objects which can be accessed directly because it is automatically setup as a local variable for the test case. The good thing about this is that we can reference these objects using symbolic names. So if we were to declare a fixture called ':cookie' (see example below), we could reference the entire cookie record simply by:

categories(:cookie)

This will return the hash for the fixture named cookie which corresponds to a row in the recipe table describing the recipe for that cookie.

On creating the model, the default fixtures generated are of the form:

one:
  title: MyString
  description: MyString
  instructions: MyText
  
two:
  title: MyString
  description: MyString
  instructions: MyText

We spoke of the :cookie fixture which would be defined as:

cookie:
  Title: Biscuit
  Description: Round and Small 
  Instructions: Buy and bake them 

This allows us to access this entire record using the symbolic name ':cookie' which hashes to this particular fixture.

An important feature of YAML fixtures is that it supports Embedded Ruby i.e. we can embed ruby code into fixtures to generate a large set of sample data. For example:

<% (1..1000).each do |i| %>
fix_<%= i %>:
  name: category_<%= i %>
<% end %>

This would create a thousand fixtures having symbolic names fix_1, fix_2 up to fix_1000, each one of them having a corresponding name attribute category_1, category_2 etc. This is a much better alternative than having to copy-paste the fixture fixture a thousand times.

A very important thing to remember about fixtures is that the ones which are generated by default by the scaffolds do not factor in for any foreign-key relationships that might be present in the models. Thus, such references have to be explicitly added to the fixture manually in order to reflect any 'has-many' or 'belongs-to' relationships across models.

Unit Tests

Unit tests are used to test the models. Thus any tests that deal with the validation of data in a model are to be done in Unit tests. It is always a good practice to have one test for every kind of validation present so that everything that could potentially break has goes through tests before production. Models contain most of the business logic, hence the unit tests need to test all the individual methods present in the models too.

When models are generated (either using the 'generate' command or by using scaffolds), default Unit test are generated alongside. These tests by no means contain any functionality by themselves, but are merely placeholders that provide a framework upon which one can write their own tests. Such test stubs are created in the test/unit folder. Almost all tests, be it Unit tests or Integration tests require 'test_helper' which specifies the default configuration of our tests.

Consider the CookBook example in which the model 'recipe' validated the presence of its attributes 'title', 'description', 'instructions' and 'category'. Such a validation ensures that none of the fields can be empty and therefore creation of an empty recipe should be invalid. A Unit test that does just this is given below:

test 'should require all fields' do
  r = recipe.new
  assert_false r.valid?
end

A blank recipe should be invalidated by the model which is what assert_false checks for. This test would pass if r.valid? is false i.e. model does not accept an empty recipe.

Functional Tests

Fundamentally, these tests are used for testing the 'functionality' of the various components of a controller. So typically, one has as many functional tests as there are controllers. The basic purpose of writing functional tests is to check if all the methods of a controller are working correctly. Since the controllers often influence the content of the web page (which is rendered by the corresponding method of a controller) functional tests are typically written to check if the controller’s method is rendering/redirecting to the correct page, whether or not the users are getting authenticated correctly, validating the correctness of the content displayed on the page,etc. Functional tests are also used to test the Views because controllers and views are tightly coupled in rails. This tight coupling is evident by the fact that instance variables in the controller are available to the view. Hence, view related functionalities can also be tested in functional tests e.g. the most common kind of view test would be checking that the title of a web page is displayed correctly or not.

An example of a functional test:

test “should use layout” do
  get :index
  assert_response :success
  assert_select ‘title’, ‘Online Cookbook’
end

In rails the title of the test gives you a basic idea of what the test is all about.In this case, "should use layout" implies that this test checks the layout of the page. The gist of this test is that it gets the index page (corresponding to view of this controller),and ensures that the page is rendered correctly with the mothod:

 assert_response 

Finally it checks that the title of the page is "Online CookBook".

 assert_select 

This method allows you to select the value of a particular tag from html - in this case the title tag.

Integration Tests

Typically in software development, different modules of a project are worked on by different teams/developers. Each team might ensure that the model works correctly in-itself, but this might not necessarily be the case when all the modules are coupled together as a single unit. This is where Integration tests come into play. They test the interaction between multiple controllers and all the components in a sequence, end-to-end. An example of it would be that of a shopping cart application. Even though different phases of the application may work correctly, while running integration tests, one might realize that the 'add to cart' button is absent in the product-catalog, even though the add to cart functionality has been correctly implemented.

The default integration tests framework included in Test-Unit are very low level i.e. they deal with HTTP GET, POST requests responses, session objects, cookies, redirects etc. In Behavioral-Driven-Development we want to deal with the system on a higher level – similar to a user’s interaction with the system i.e. we want to deal only with clicks, with typing etc. Hence, we can use some of the popular Integration Testing frameworks like Capybara which is a GUI testing framework and allows one to specify - within a test - various actions like 'click' to click on a button, 'fill_in' to fill some text into a designated text-box etc. We can see that this is at a high level and somewhat analogous to actions an end-user might go through while using the application. So the rule of thumb while writing integration tests is to identify the end-users requirements and scope of interaction with the system, walk through the steps that they would take and mimic those in the form of tests. It is clearly evident how such Behavioral-Driven-Development goes hand in hand with Test-Driven-Development and helps in removing the ambiguities which are often associated with Customer Requirements.

The following example shows how the test framework CapyBara<ref>http://opinionated-programmer.com/2011/02/capybara-and-selenium-with-rspec-and-rails-3/</ref> is used for Integration Testing:

test “create category from main page” do
  visit categories_path
  click_link “New category”
  fill_in “category_name”, :with => “Sample Category”
  click_button “Create Category”
end

Here we have simulated a user-action (for the CookBook example) where the user would carry out the following steps:

1. Visit the Categories Home Page (whose url is specified as categories_path by the routes.rb file)
2. Click on the Link which says "New Category", which would lead to another page.
3. On this new page, fill the text-box with some text, say "Sample Category"
4. Click on the button that says "Create Category".

One can easily identify these actions from the code which is highly intuitive and self-explanatory. Capybara thus provides us with these convenient methods which greatly expedites the whole Integration Testing process.

To use the framework, simply include the corresponding gem in the Gemfile, and the following lines to the end of the test_helper.rb file.

# Add more helper methods ...
require ‘capybara/rails’

class ActionDispatch::IntegrationTest
  include Capybara::DSL
end

The easy-to-use commands mentioned before are created by Capybara using a Domain Specific Language (DSL) and in order to be able to use it, every Integration test written must require 'test_helper' . This is basically a mixin, so one still has the capability to access all the low-level GET/POST commands in Test-Unit in addition to all the methods offered by Capybara.

Another example of Integration Testing with Test-Framework RSpec Version 1.3.2<ref>http://rspec.info/documentation/</ref>:

describe "Recipes" do

  before(:all) do
    @recipe = Recipe.new
  end

  it "should not accept empty recipe" do
    @user.should_not_be_valid
    @user.title = "Cookie"
    @user.description = "Chocolate Chip Cookie"
    @user.instructions = "Bake in Oven"
    @user.category = 3
    @user.should_be_valid
  end

end

In the above example we basically test that an empty recipe is invalid and that a recipe with fields filled out is valid. This is a very primitive example of using RSpec and it is just to showcase the difference between RSpec and Capybara, and is in no way a comprehensive example. A thing to note is the before(:all) method, which is similar to the setup() method in Java's JUnit Framework i.e. this method is called before every test in the describe block gets executed.

In sum, Integration tests are vital and are carried out in the final stages of testing to ensure that the system works as a cohesive and complete unit.

Performance Tests

Performance tests as the name indicates are used to gauge the performance of the system and play a very important role in software development for the simple reason that as a developer, one does not want the end user to have a poor experience while using the application. Users do not want to wait long for pages to load and elements on the page to respond. They are not - and should not - be concerned with the capability of the system to handle large loads, scale to accommodate increased volumes of traffic etc. Such details are abstracted away from the user, but they do have a significant impact on user's interaction with the system.

Rails Performance test can be categorized as a special type of integration tests, which are designed for bench-marking and profiling the test code<ref>http://guides.rubyonrails.org/performance_testing.html#modes</ref>. In these tests, one can mention how many connections are to be simulated to the server etc. at the outcome of which it would be possible to identify the performance bottlenecks and hopefully pinpoint the source of speed and/or memory problems.

Detailed examples can be found Here.

Conclusion

Testing is an indispensable and an inevitable part of development in rails.It must be fully exploited to avail the benefits associated with Test-Driven-Development, for the simple reason that rails provides an excellent in-built framework upon which writing tests is a highly natural and intuitive process. There are many advantages to testing and many articles<ref>http://www.learn.geekinterview.com/programming/ruby/ruby-on-rails-application-testing.html</ref> have been written that emphasize this point.

References

<references/>

Most of the content for this article has been obtained from the Lecture taught in class which has been the primary resource. The video of the lecture can be found here.