CSC/ECE 517 Fall 2012/ch2a 2w13 sm
Introduction
Test stubs are programs which simulate the behaviors of software components (or modules) on which other modules that are being tested are dependent upon. Stubs are replacements for missing components that the components being tested will call as part of the test. While doing an Integration, if we don't have all the modules ready and need to test a particular module which is ready then we use Stubs and Drivers. Stubs are used in Integration testing for a Top Down Integration testing and Bottom Up Integration Testing.
In the above figure, we can see that the Modules 2 and 3 are not yet implemented. However, if we need to test the modules 4,5 and 6 which are dependent on modules 2 and 3, we can simple use some dummy code in the form of Stubs 1 and 2 in place of the modules that return the values as expected by the modules beneath them. This piece of Dummy code is Called a Stub in a Top Down Integration. Hence Stubs are called Functions in Top Down Integration tests.
Stubs in different phases of testing
Unit Testing
Unit testing with Stubs:
In developing a huge software if we are following top down approach of development there may be some functionality which may not have been implemented. This can be overcome by implementing all the required functions. However, then we would be testing a huge chunk of code without actually testing it. This will lead to bugs and lot of time will be wasted on debugging. Consider the following example:
Suppose we want to evaluate an expression factorial(x)/( sin(x) + tan(x) ). Assuming that we don't have library support available for calculating factorial,sin and tan of the given input we would have to write those functions. Using stubs we can write functions which would return appropriate values. Thus, assuming that factorial(x) is implemented then we can write stubs for sin and cos functions returning floating point values. We can than verify if our main function gives expected results for those values. This shows us that our main function, factorial are working correct without actually implementing sin and cos functions
code:
int factorial(x)
{
//implmentation of factorial
}
float sin(int x)
{
return 0.5;
}
float cos(int x)
{
return 0.3;
}
Functional Testing
The idea of stub remains the same in different forms of testing. In unit testing we are dealing with working inside a module and hence we overwrite the dependent implementations by stub. This allows us to test the calling code without the actual implementation. In functional testing we will replace other modules or collaborators of a class. The client class might be interested in some service from some other class ( collaborator class ). However, again instead of implementing the actual class we will have a stub implementation. In ruby the mocha gem provides support for functional test using stubbing. This helps in avoiding overlapping test cases. We can see the use of stubs in functional testing using mocha in ruby in the programming languages section.
Integration Testing
Integration testing is a logical extension of unit testing. In its simplest form, two units that have already been tested are combined into a component and the interface between them is tested. A component, in this sense, refers to an integrated aggregate of more than one unit. In a realistic scenario, many units are combined into components, which are in turn aggregated into even larger parts of the program. The idea is to test combinations of pieces and eventually expand the process to test your modules with those of other groups. Eventually all the modules making up a process are tested together. Beyond that, if the program is composed of more than one process, they should be tested in pairs rather than all at once. Integration testing identifies problems that occur when units are combined. By using a test plan that requires you to test each unit and ensure the viability of each before combining units, you know that any errors discovered when combining units are likely related to the interface between units. This method reduces the number of possibilities to a far simpler level of analysis. The top-down approach to integration testing requires the highest-level modules be test and integrated first. This allows high-level logic and data flow to be tested early in the process and it tends to minimize the need for drivers. However, the need for stubs complicates test management and low-level utilities are tested relatively late in the development cycle. Another disadvantage of top-down integration testing is its poor support for early release of limited functionality.
Regression Testing
Regression testing is any type of software testing that seeks to uncover new software bugs, or regressions, in existing functional and non-functional areas of a system after changes, such as enhancements, patches or configuration changes, have been made to them. The intent of regression testing is to ensure that a change, such as a bugfix, did not introduce new faults.[1] One of the main reasons for regression testing is to determine whether a change in one part of the software affects other parts of the software.[2] Common methods of regression testing include rerunning previously run tests and checking whether program behavior has changed and whether previously fixed faults have re-emerged. Regression testing can be used to test a system efficiently by systematically selecting the appropriate minimum set of tests needed to adequately cover a particular change. In Regression testing the use of test stubs is minimized as most of the system is already implemented and the main goal of the regression test is to see if there are any bugs that might have been ‘regressed’ from the previous stages of integrations. Ideally there will not be unimplemented modules by the time a system is tested for regression. However, in case of mutually exclusive modules of a system, a test stub for one of them can be used to test the regression of the other.
Stubs in software development methodologies
Agile Development Methodology
Agile software development is a group of software development methods based on iterative and incremental development, where requirements and solutions evolve through collaboration between self-organizing, cross-functional teams. It promotes adaptive planning, evolutionary development and delivery, a time-boxed iterative approach, and encourages rapid and flexible response to change. It is a conceptual framework that promotes foreseen interactions throughout the development cycle. In such a development system, it recognizes that testing is not a separate phase, but an integral part of software development, along with coding. Agile testing is a software testing practice that follows the principles of agile software development. Agile testing involves all members of a cross-functional agile team, with special expertise contributed by testers, to ensure delivering the business value desired by the customer at frequent intervals, working at a sustainable pace. Specification by example, also known as acceptance test-driven development, is used to capture examples of desired and undesired behavior and guide coding. In the agile testing toolkit, a stub is a piece of code that behaves in a predefined way, in order to simplify the testing process. Here are the kind of simplifications that a stub can bring:
- If the code under test never returns the same value (usage of a timestamp or of a random value), a stub can be used to fix the root of the undeterminism.
- If the code depends on some other class that has complex behavior that you don’t want to take into account in this test, using a stub for this other class can simplify its behavior and clarify the intent of the test
- If the code depends on some other code that is not yet written or released, it is possible to start building the rest of the code before the dependency is actually finished.
Examples of Stubs
Java
In Java, the basic technique is to implement the stub collaborators as concrete classes which only exhibit the small part of the overall behaviour of the collaborator which is needed by the class under test. As an example consider the case where a service implementation is under test. The implementation has a collaborator:
public class SimpleService implements Service {
private Collaborator collaborator;
public void setCollaborator(Collaborator collaborator) {
this.collaborator = collaborator;
}
// part of Service interface
public boolean isActive() {
return collaborator.isActive();
}
}
We could have a stub collaborator for testing as defined below:
public class StubCollaboratorAdapter implements Collaborator {
public boolean isActive() {
return false;
}
}
Now, a test case can be written as something like this:
public void testActiveWhenCollaboratorIsActive() throws Exception {
Service service = new SimpleService();
service.setCollaborator(new StubCollaboratorAdapter() {
public boolean isActive() {
return true;
}
});
assertTrue(service.isActive());
}
Ruby
While writing stubs, one might decide to replace a whole object or simply change the results of a single method. When you're replacing a whole object, it's easy—just write the new object and replace it within your test case. Methods might be tougher in some languages, but using dynamic languages such as Ruby, stubbing is easy because you're simply redefining a method. Imagine you have a user interface that prints a document. You want to test code that handles a print failure. You really don't care if the print method gets called. You just want to verify that if your code does call print, that code will handle a failed print effectively. Think of what a pseudo-code sentence would look like. On an instance of the document object, stub the print method, returning false. You can break that sentence down into two parts. The first part identifies what you want to stub. The second part defines what the stub should do. Below is an example of the Mocha framework that can be used to build stubs in Ruby:
class Document
def print
# doesn't matter -- we are stubbing it out
end
end
class View
attr :document
def initialize(document)
@document = document
end
def print()
if document.print
puts "Excellent!"
true
else
puts "Bummer."
false
end
end
end
Test code can be written as follows:
require 'test/unit'
require 'rubygems'
require 'mocha'
require 'document'
class ViewTest < Test::Unit::TestCase
def test_should_return_false_for_failed_print
document = stub("my document")
document.stubs(:print).returns(false)
ui = View.new(document)
assert_equal false, ui.print
end
end
You can see how this works. I build a simple, named stub object with the statement stub("my document"). Then, I define the behavior for print within the stub with the line of code document.stubs(:print).returns(false). If you look carefully, this line of code looks remarkably similar to the pseudo-code from earlier: On an instance of the document object, stub the print method, returning false.
Python
IDE support for Stubs
Eclipse
Eclipse provides support to create test stub methods to test the methods developed during development. We can create the junit test case on a java class. We can then select the methods in the class which we would like to test. This will automatically create test stubs to test those methods.
RubyMine
The most basic way to create test templates in RubyMine is to use the regular procedure of creating files in the project. This means is both available for both plain Ruby projets, and for the Rails applications, and allows creating stub tests for Test::Unit, RSpec and Test-Spec.
For RSpec and Test-Spec, RubyMine provides a number of generators. These generators are context sensitive and become available depending on the selected location, and on the gems activated for your project. For example, ir rspec-rails is activated, the usual generators are hidden, and the list of generators includes only those specific for RSpec.
Visual Studio
Visual Studio is a fine IDE that provides good support for automatically generating stubs for methods. Generate Method Stub is an IntelliSense Automatic Code Generation feature that provides an easy way to have Visual Studio create a new method declaration at the time you are writing a method call. Visual Studio infers the declaration from the call. Some programming styles, such as test-driven development, suggest that you should consume before you define. That way, it is easier to figure out the form of the API that you are developing. You can use IntelliSense to program in that style. Using the Generate Method Stub operation, you avoid defining everything before you consume it.
The Generate Method Stub IntelliSense operation can also increase productivity because you do not need to move from the calling code, your present focus, to the defining code, a separate focus, in order to generate a new method. You can instead write a method call, and then invoke the Generate Method Stub operation without dividing your attention.
Conclusion
We need tests that “run fast, and help us localize problems.” This can be hard to accomplish when your code accesses a database, hits another server, is time-dependent, etc. By substituting custom objects such as stubs for some of your module's dependencies, you can thoroughly test your code, increase your coverage, and still run in less than a second. You can even simulate rare scenarios like database failures and test your error handling code. In general, Stubs greatly increase the speed of running unit tests and provide a means of testing code independently.