CSC/ECE 517 Fall 2011/ch4 4e cl
CSC/ECE 517 Fall 2010/ch1 4e cl
Introduction
In this Wiki, we introduce the testing in Ruby Rails. We give brief introduction of different kinds of tests and give the procedure about each kind.
What is Software Test?
Software testing is an investigation conducted to provide stakeholders with information about the quality of the product or service under test.[1] Software testing can also provide an objective, independent view of the software to allow the business to appreciate and understand the risks of software implementation. Test techniques include, but are not limited to, the process of executing a program or application with the intent of finding software bugs (errors or other defects). Software testing can be stated as the process of validating and verifying that a software program/application/product: 1.Meets the requirements that guided its design and development; 2.Works as expected; and 3.Can be implemented with the same characteristics.
Software testing, depending on the testing method employed, can be implemented at any time in the development process. However, most of the test effort occurs after the requirements have been defined and the coding process has been completed. As such, the methodology of the test is governed by the software development methodology adopted.<ref>http://en.wikipedia.org/wiki/Software_testing</ref>
What is Ruby on Rails?
Ruby on Rails, also called Rails or RoR, is an open source web application framework for the Ruby programming language. It uses the Model-View-Controller(MVC) architecture pattern to organize the application programming. Ruby on Rails is separated into various packages, namely ActiveRecord, ActiveResource, ActionPack, ActiveSupport and ActionMailer. It is often installed using RubyGem, which is a package manager including with current versions of Ruby.<ref>http://rubyonrails.org/</ref>
The MVC Architecture
Model-View-Controller(MVC) is a software architecture, currently considered anarchitectural pattern used in software engineering. The model manages the behaviour and data of the application domain, responds to requests for information about its state (usually from the view), and responds to instructions to change state (usually from the controller). In event-driven systems, the model notifies observers (usually views) when the information changes so that they can react.<ref>http://msdn.microsoft.com/en-us/library/ff649643.aspx</ref> The view renders the model into a form suitable for interaction, typically a user interface element. Multiple views can exist for a single model for different purposes. A viewport typically has a one to one correspondence with a display surface and knows how to render to it. The controller receives user input and initiates a response by making calls on model objects. A controller accepts input from the user and instructs the model and a viewport to perform actions based on that input. <ref>http://en.wikipedia.org/wiki/Model%E2%80%93view%E2%80%93controller</ref>
Why need Software Test for Rails Application?
Ruby on Rails, often shortened to Rails or RoR, is an open source web application framework for the Ruby programming language. Rails makes it super easy to write your tests. It starts by producing skeleton test code in the background while you are creating your models and controllers.<ref> http://effectif.com/articles/testing-rails-with-rack-test</ref> By simply running your Rails tests you can ensure your code adheres to the desired functionality even after some major code refactoring. Rails tests can also simulate browser requests and thus you can test your application’s response without having to test it through your browser. <ref>http://guides.rubyonrails.org/testing.html</ref>
Flow of Testing in Rails
The Three Environments is Production, Development and Test.
To Set Up the Environment, we may follow the next: There is a folder named "test" which is generate when create the Rail projects. To list contents of this folder $ ls -F test/ We can see: fixtures/ functional/ integration/ test_helper.rb unit/
The Low-Down on Fixtures: Test fixture refers to the fixed state used as a baseline for running tests in software testing. The purpose of a test fixture is to ensure that there is a well known and fixed environment in which tests are run so that results are repeatable.<ref>http://en.wikipedia.org/wiki/Test_fixture</ref>For good tests, you’ll need to give some thought to setting up test data. In Rails, you can handle this by defining and customizing fixtures.
Unit Testing for Models
In Rails, unit tests aims to test the models of the application. We also called it as model tests. The code you write up for model tests are placed in the directory of test/unit. What the unit testing in Rails contains is listed as follow: • Load data from fixtures before each unit test • Multiple block syntax for setup and teardown. • The test_helper.rb file, which is a important file in your application and includes the default configuration for all tests. You can put common setup and assertion methods for all tests here. It injects some additional methods into ActiveSupport::TestCase. • A series of assertion methods. Each assertion performs the examination to make sure that things are going as expected. We will show the list of assertions later.
What is A Good Unit Testing?
Our goal for unit testing is to cover almost 100% of the code in models. We recommend writing test file for each individual model. To ensure the difference between a valid and invalid object is as expected, you need to write test to cover validations. The test-driven development advocates that any new logic should be driven by a failing test. Conversely, any new test should fail and trigger a new logic code. If you want your tests to be fewer for each method, please make sure to keep your method simpler and smaller. The following is some tips for writing a good unit testing:
• Keep each of your tests to be small.
• Write at least one test for some known errors. For example, without passing arguments.
• Write one test for the normal case.
• For the case with different branches, write one test for each branch.
There is a tradeoff about the alignment of assertions. If you put all the assertions in single one test method, your test looks more cohesive but it does not allow the tests to run independently. If you put all the assertions in different tests, you can run the test independently but you can not observe the relationships among the tests.
Preparation for A Unit Testing
The scaffolding in Rails will create the model, controller, view and migration. The corresponding full test suite is created at the same time. For example, when you use rails to generate scaffold of “user”, the default test stub for UserTest is created in test/unit/user_post.rb file:
require "test_helper"
class UserTest < ActiveSupport::TestCase
test "the truth" do
assert true
end
end
require 'test_helper' The statement is included in all test files. It means that all the methods in this file will be available to your tests. It declare the default configuration to run the tests. class UserTest < ActiveSupport::TestCase: The UserTest class defines a test case because it inherits from ActiveSupport::TestCase.UserTest thus has all the methods available from ActiveSupport::TestCase. The naming convention for test method is starting with test.
test "the truth" do
assert true
end
The method name will be generated by replacing spaces with underscores when you run the test. So the above test method can also be written in the form of :
def test_the_truth assert true end
Before running your test, you need to use the following rake commands to make sure that the test database structure is current.
| rake db:migrate | runs any pending migrations on the development environment and updates db/schema.rb. |
| rake db:test:load | recreates the test database from the currentdb/schema.rb |
| rake db:test:clone | Recreate the test database from the current environment’s database schema |
| rake db:test:clone_structure | Recreate the test database from the development structure |
| rake db:test:load: | Recreate the test database from the current schema.rb |
| rake db:test:prepare | Check for pending migrations and load the test schema |
| rake db:test:purge | rake db:test:purge |
Assertion Methods
The following shows a complete list assertion methods for basic unit test[1]. The left side shows the assertion method names and their corresponding parameters while the right side displays the purpose of each assertion method. The parameter of [msg] is an optional string message where you could place the detailed test failure messages if you want.
| Assertion | Objective |
|---|---|
| assert_send( array, [msg] ) | Ensures that executing the method listed in array[1] on the object in array[0] with the parameters of array[2 and up] is true. |
| assert_nil( obj, [msg] ) | Ensures that obj.nil? is true. |
| assert_not_nil( obj, [msg] ) | Ensures that obj.nil? is false. |
| assert_match( regexp, string, [msg] ) | Ensures that a string matches the regular expression. |
| assert_no_match( regexp, string, [msg] ) | Ensures that a string doesn’t match the regular expression. |
| assert( boolean, [msg] ) | Ensures that the object/expression is true. |
| assert_equal( A, B, [msg] ) | Ensures that A == B is true. A/B is a object or an expression. |
| assert_not_equal( A, B, [msg] ) | Ensures that A == B is false. A/B is a object or an expression. |
| assert_same( A, B, [msg] ) | Ensures that A.equal?(B) is true. A/B is a object or an expression. |
| assert_not_same( A, B, [msg] ) | Ensures that A.equal?(B) is false. A/B is a object or an expression. |
| assert_in_delta( expecting, actual, delta, [msg] ) | Ensures that the numbers expecting and actual are within delta of each other. |
| assert_instance_of( class, obj, [msg] ) | Ensures that obj is of the class type. |
| assert_kind_of( class, obj, [msg] ) | Ensures that obj is or descends from class. |
| assert_respond_to( obj, symbol, [msg] ) | Ensures that obj has a method called symbol. |
| assert_operator( obj1, operator, obj2, [msg] ) | Ensures that obj1.operator(obj2) is true. |
| assert_operator( obj1, operator, obj2, [msg] ) | Ensures that obj1.operator(obj2) is true. |
| assert_throws( str_msg, [msg] ) { block } | Ensures that the given block throws the str_msg. |
| assert_raise( exception_1, exception_2, exception_n) { block } | Ensures that the given block raises one of the given exceptions. |
| assert_nothing_raised exception_1, exception_2, exception_n) { block } | Ensures that the given block doesn’t raise one of the given exceptions. |
| flunk( [msg] ) | Ensures failure. This is useful to explicitly mark a test that isn’t finished yet. |
Below are some custom assertions that Rails adds to the basic unit test. <ref>http://guides.rubyonrails.org/testing.html</ref>
| Assertion | Objective |
|---|---|
| assert_difference(expressions, difference = 1, message = nil) {...} | Test numeric difference between the return value of an expression as a result of what is evaluated in the yielded block. |
| assert_no_difference(expressions, message = nil, &block) | Asserts that the numeric result of evaluating an expression is not changed before and after invoking the passed in block. |
| assert(record.valid?) | Ensures that the passed record is valid by Active Record standards and returns any error messages if it is not. |
| assert_recognizes(expected_options, path, extras={}, message=nil) | Asserts that the routing of the given path was handled correctly and that the parsed options (given in the expected_options hash) match path. Basically, it asserts that Rails recognizes the route given by expected_options. |
| assert_generates(expected_path, options, defaults={}, extras = {}, message=nil) | Asserts that the provided options can be used to generate the provided path. This is the inverse of assert_recognizes. The extras parameter is used to tell the request the names and values of additional request parameters that would be in a query string. The message parameter allows you to specify a custom error message for assertion failures. |
| assert_response(type, message = nil) | Asserts that the response comes with a specific status code. You can specify :success to indicate 200, :redirect to indicate 300-399, :missing to indicate 404, or :error to match the 500-599 range |
| assert_redirected_to(options = {}, message=nil) | Assert that the redirection options passed in match those of the redirect called in the latest action. This match can be partial, such thatassert_redirected_to(:controller => "weblog") will also match the redirection ofredirect_to(:controller => "weblog", :action => "show") and so on. |
| assert_template(expected = nil, message=nil) | Asserts that the request was rendered with the appropriate template file. |
Examples of Unit Testing
We can write our unit testing cases now. We will use the live question website as the example. 1 Ensure the username and password are not empty. Either one is missing, the test case will throw the error message.
test "should not be saved without username and password" do
user = User.new
assert !user.save
assert user.new
end
2) Ensure the post title and content are not empty. Either one is missing, the test case will throw the error message.
test "post attributes must not be empty" do
post = Post.new
assert post.invalid?
assert post.errors[:title].any?
assert post.errors[:content].any?
end
3) The following test case is used to test any method that extracts a set of records from the database. First, in line 3-4, it extracts the model class being tested and calls the find method on that model class, resulting in a set of instances of that model. Then in lines 5–8, each instance in the list of matching objects is tested against the block and must return true for the test to pass. Just as importantly, lines 11–14 run the block against all the instances that weren’t returned by the method and assert that the block is false for each one.<ref> http://www.jetbrains.com/ruby/features/ruby_unit_testing.html</ref>
def self.should_match_find_method(named_scope, *args, &block)
should "match a find method #{named_scope}" do
ar_class = self.class.model_class
found_objects = ar_class.send(named_scope, *args)
assert !found_objects.blank?
found_objects.each do |obj|
assert block.call(obj)
end
unfound_objects = ar_class.all - found_objects
assert !unfound_objects.blank?
unfound_objects.each do |obj|
assert !block.call(obj)
end
end
end
<ref>http://en.wikipedia.org/wiki/Unit_testing</ref>
Functional Testing for Controllers
Functional testing is a type of black box testing that bases its test cases on the specifications of the software component under test. Functions are tested by feeding them input and examining the output, and internal program structure is rarely considered. <ref>http://en.wikipedia.org/wiki/Functional_testing</ref> In rails, controller testing depends on Rails-specific structures and methods. Controller testing aims to simulate various actions of a specific controller method and test whether the results are come out as expected. It is up to you to include or not include the assertions about the view output. The test of controller and views are included in the standard rails functional testing, but some of the third party add-ons provide the separate testing of views and partials.
What is Included by Functional Testing?
All the features included by unit testing are still present in functional testing. What is more functional testing include the following unique features: • In functional tests, you have access to the three instance variables: @controller(i.e.,the controller processing the request), @request and @response.
• Rails functional tests support 5 request types: get(), post(), put(), head(), delete(). They are used to simulate each HTTP verb for the purpose of pretending to call a controller. Out of the 5 request types, the first two get() and post() are the frequently used ones.
• After a request has been made by using one of the 5 methods (get, post, etc.) and processed, you will have 4 Hash objects ready for use: assign, session, cookies, and flash, each of which represents the Rails construct of the same name. Assigns allow access to any instance variable set in controller method. For example, @user in the controller method can be verified in the test by using assigns(:user). Flash is any object living in the flash. Session is any object living in session variable. As is the case with normal Hash objects, you can access the values by referencing the keys by string. You can also reference them by symbol name, except for assigns. <ref>Noel Rappin, edited by Colleen Toporek: Rails Test Prescriptions: Keeping Your Application Healthy, 2011.</ref>For example:
flash["taste"] flash[:taste] session["userid"] session[:userid] cookies["display_post"] cookies[:display_post]
• A series of assertions aimed at the specifics of controller and view testing. Testing the response to your request by asserting the presence of key HTML elements and their content is a useful way to test the views of your application. The assert_select assertion allows you to do this by using a simple yet powerful syntax. The following table list the series of frequently used assertion methods in functional testing.
| Assertion | Objective |
|---|---|
| assert_select(selector, [equality], [message]) | ensures that the equality condition is met on the selected elements through the selector. |
| assert_select(element, selector, [equality], [message]) | ensures that the equality condition is met on all the selected elements through the selector starting from the element (instance ofHTML::Node) and its descendants. |
| assert_select_email | Allows you to make assertions on the body of an e-mail. |
| assert_select_encoded | Allows you to make assertions on encoded HTML. It does this by un-encoding the contents of each element and then calling the block with all the un-encoded elements. |
| css_select(selector)orcss_select(element, selector) | Returns an array of all the elements selected by the selector. In the second variant it first matches the base element and tries to match the selector expression on any of its children. If there are no matches both variants return an empty array. |
What to Test in Functional Testing?
The goal of the functional testing includes: • To ensure that a normal user request can trigger the ActiveRecord calls as expected and pass the correct data to the view. • To ensure that an invalid user request is properly handled • To ensure a series of security check. For example, requiring logins for the web application; only the admin have the grant to delete other users; For the users who enter URL for a resource, the page returned by the application should not be blocked or diverted.
Functional Testing Examples
Starting a Controller Call
The first step to initialize the functional testing is to simulate a controller call. Let us look a simple example:
setup :generic_setup
def generic_setup
@task = Task.create
login_as :admin
end
test "should show a task" do
get :show, :id => @task.id.to_s
assert_equal(@task.id, assigns(:task).id)
assert_response :success
assert_template :show
end
The test methods for each HTTP verb (get( ), post( ), put( ), and delete( )) is provided by Rails. They works in the same way. The first argument to the simulated call is the controller method to invoke. The second argument contains the key/value pairs that become the params of the call. The Rails holds the convention of placing complex data types into parameter names. Therefore, :task => {:project => {:id => "3"}} creates params[:task][:project][:id] = "3".
If a session is getting a user ID and current project, and the flash is getting a notice:
get :show, {:id => @task.id.to_s},
{:user_id => "3", :current_project => @project.id.to_s},
{:notice => "flash test"}
The xhr( ) method simulates an Ajax call to the controller. The signature of the method is a little different; the first argument is the HTTP verb, the second is the controller method, and the remaining arguments match the order of the other HTTP mimic methods:
test "my ajax call" do
xhr :post, :create, :task => {:id => "3"}
end
The controller will respond to a test call made with the xhr( ) exactly the way it would respond to an actual Ajax request. That is to say, if you use the more modern respond_to blocks, then the request will match the format.j s block. Alternately, the controller method xhr? will return true for the action being tested.
Testing Controller Response
After you submit a request, you expect the web application returns the HTTP status code as you expected and that appropriate Rails template is placed in charge of returning the response. Rails provides three assertion methods to help: assert_redirected_to( ), assert_response( ), and assert_template( ). The following code exemplifies using assert_response( ) and assert_template( ) to verify whether a normal HTTP response is return and not directed.
test "successful index request" do
get :index
assert_response :success
assert_template "index"
end
The assert_response( ) actually verifies the response code sent by Rails to the browser. The assert_template( ) verifies which template Rails uses to generate the response. The template name is specified exactly as it would be in the controller using render :action—the template name can be a string or a symbol. If the argument is just a single string or symbol, then it is checked against the name of the main template that rendered the action.
For a redirect, Rails provides assert_redirected_to( ), which takes as an argument any object that can be resolved into a URL.
assert_redirected_to :controller => :task, :action => :show, :id =>3
Testing Returned Data
The four collections provided by Rails (assigns, session, cookies, and flash) can be used to verify the data generated by the controller method. The assigns, a hash of instance variables created in the controller, is the most commonly used. A typical use might look like this, with a common use of assigns, and a frankly contrived use of session:
test "should show task" do
get :show, :id => @task_1.id
assert_response :success
assert_equal @task_1.id, assigns(:task).id
assert_equal "task/show", session[:last_page]
end
Integration Testing
Integration tests are used to test the interaction among any number of controllers. They are generally used to test important work flows within your application.<ref>http://en.wikipedia.org/wiki/Integration_testing</ref> Unlike Unit and Functional tests, integration tests have to be explicitly created under the ‘test/integration’ folder within your application. Rails provides a generator to create an integration test skeleton for you.<ref>rspec-rails-1.3.4, http://rspec.info/rails/</ref> $ rails generate integration_test user_flows
exists test/integration/
create test/integration/user_flows_test.rb
Here’s what a freshly-generated integration test looks like:
require 'test_helper'
class UserFlowsTest < ActionDispatch::IntegrationTest
fixtures :all
# Replace this with your real tests.
test "the truth" do
assert true
end
end
| Helper | Purpose |
|---|---|
| https? | Returns true if the session is mimicking a secure HTTPS request. |
| https! | Allows you to mimic a secure HTTPS request. |
| host! | Allows you to set the host name to use in the next request. |
| redirect? | Returns true if the last request was a redirect. |
| follow_redirect! | Follows a single redirect response. |
| request_via_redirect(http_method, path, [parameters], [headers]) | Allows you to make an HTTP request and follow any subsequent redirects. |
| post_via_redirect(path, [parameters], [headers]) | Allows you to make an HTTP POST request and follow any subsequent redirects. |
| get_via_redirect(path, [parameters], [headers]) | Allows you to make an HTTP GET request and follow any subsequent redirects. |
| put_via_redirect(path, [parameters], [headers]) | Allows you to make an HTTP PUT request and follow any subsequent redirects. |
| delete_via_redirect(path, [parameters], [headers]) | Allows you to make an HTTP DELETE request and follow any subsequent redirects. |
| open_session | Opens a new session instance. |
Example:
require 'test_helper'
class UserFlowsTest < ActionDispatch::IntegrationTest
fixtures :users
test "login and browse site" do
# login via https
https!
get "/login"
assert_response :success
post_via_redirect "/login", :username => users(:avs).username, :password => users(:avs).password
assert_equal '/welcome', path
assert_equal 'Welcome avs!', flash[:notice]
https!(false)
get "/posts/all"
assert_response :success
assert assigns(:products)
end
end
<ref>http://weblog.rubyonrails.org/2006/3/1/new-for-rails-1-1-integration-tests</ref>
Running Your Tests
You don’t need to set up and run your tests by hand on a test-by-test basis. Rails comes with a number of rake tasks to help in testing. The table below lists all rake tasks that come along in the default Rakefile when you initiate a Rails project.<ref>Performance Testing Rails Applications — How To? http://rubylearning.com/blog/2011/08/14/performance-testing-rails-applications-how-to/</ref>
| Tasks | Description |
|---|---|
| rake test | Runs all unit, functional and integration tests. You can also simply runrake as the test target is the default. |
| rake test:benchmark | Benchmark the performance tests |
| rake test:functionals | Runs all the functional tests from test/functional |
| rake test:integration | Runs all the integration tests from test/integration |
| rake test:plugins | Run all the plugin tests from vendor/plugins/*/**/test (or specify with PLUGIN=_name_) |
| rake test:profile | Profile the performance tests |
| rake test:recent | Tests recent changes |
| rake test:uncommitted | Runs all the tests which are uncommitted. Supports Subversion and Git |
| rake test:units | Runs all the unit tests from test/unit |
Testing Routes
To set a testing route, we can write the relative codes in the controller.rb file like following:
test "should route to post" do
assert_routing '/posts/1', { :controller => "posts", :action => "show", :id => "1" }
end
Conclusion
As we can see above, to build and run a test for rails application is fairly easy because it always provide some existing framework to do that. Also we should know that it is very important of testing the rails application. There are also some other approaches and aids for testing such as NullDB, Factory Girl, Machinist, Shoulda, and RSpec, etc.<ref>Ruby on Rails 3 Testing: http://jonathanhui.com/ruby-rails-3-testing</ref>
References
<references/>