CSC/ECE 517 Fall 2009/wiki1b 4 xy

Functional and integration testing and beyond

Recognition of software test engineering as a specialty area has increased during the last twenty years. Testing is one of the key focus areas in software engineering. It is an essential activity that observes the functionality of a software system validates its intended behavior and identifies the potential malfunctions. Testing is also the primary way to improve software reliability.

The test cases should be carefully designed to cover many aspects in the SDLC such as possible aberrations from user’s requirements, system behavior upon malicious inputs, and robustness to extensive load conditions and so on. As more and more software is used in critical applications, the time and cost involved in testing is also increasing owing to the higher quality required. Although test automation saves a lot of time and can be used for regression load tests, manual testing is still widely used in the industry.

Terms

1. Driver: Supporting code or data used for testing the whole or part of the software system
2. Sub-system: Component or Module of a software system
3. System Under Test: The software system which has to be tested
4. Stub: Dummy procedure or module that can be used to simulate or substitute an actual portion of a system
5. Test: Sequence of steps to validate the system design and implementation
6. Usecase :Scenario that describing an interaction between a user and a system

Integration testing

Integration is a systematic approach to build the complete software structure specified in the design from unit-tested modules.Integration testing is the next phase of unit testing. Generally in software development complete software is divided into various sub-systems mostly developed by different teams. More often than not each sub-system works very well individually but when they are integrated then lot of problems. So the main focus of integration tests is not the functionality of individual modules but the interaction between these modules like interaction with operating system, interaction between client and server

There are many approaches that can be followed for Integration Testing

• Big Bang Testing Approach As the name suggests all the components are integrated at once. After all the integration is completed a set of tests are run which not only validate a single component of sub-system but also test and validate the interaction between different components. This approach is more useful in smaller systems because integration of smaller systems are easier.
  • Advantages:
    • Since all the modules are combined and all are tested at the same time, this approach of integration is less time consuming
  • Disadvantages:
    • Debugging and solving a fault is not that trivial. Because in this type of testing many modules are integrated one cannot be sure whether the bug found was present in any sub-system or interaction between different sub-systems.
    • Testing process cannot be started until all the sub-systems have finished their development.
    • Writing test cases are also non-trivial.

• Incremental Testing Approach Testing is done after integration of one module at a time. After finishing the unit testing of each module or sub-system they are integrated and tested. Stubs are used for modules in the system which are not integrated till now.
  • Advantages:
    • Problems detection is early compared to the Big Bang approach
    • No need to wait till the unit testing of all the modules have finished
    • Debugging is much simpler
  • Disadvantages:
    • This approach is a time taking process because the integration tests have to run multiple times
    • Stubs have to be developed to substitute the missing modules

Various Approach can be taken for Incremental Testing

• Top-Down Approach: In this approach the system is tested according to the work flow or code flow. The Top level module is tested first then each lower level module is integrated and tested.Top level modules are modules which call other modules and lower level modules are the ones which are called by other modules. Terminal modules are modules which do not call any other module.

In the above figure Module A is the top most module in the system then B,C,D are lower level modules. Generally module A is the user interface and stubs for each interface are used. Then each module is integrated and the tested.

• Advantages:
  • The testing flow follows the architecture of the system so any bug in the top level design or implementation could be found at early stages
  • If the top level modules call multiple lower modules then the testing can be done in various combinations

For example in the figure A calls B and C. The testing can be done as follows
A-B-C-D 
A-C-B-D

• Disadvantage :
  • The main problem with this approach is the use of stub. Mostly stubs are written to print out some trace statement based on the output of module A after running the test. Now if multiple scenarios have to be tested then different version of Stub B have to written, which is an extra overhead

• Bottom-Up Approach: The bottom-up approach is the opposite of the top-down approach. In this approach the terminal approaches are tested first, then the higher modules and finally the top-most one. Drivers are used in this approach.

Bottom Up Approach In the above figure E,D are the terminal modules which are tested first. Then the higher level modules B,C are tested and finally the topmost module is tested

• Advantages:
  • No need to write stubs
  • If the terminal modules does the main work in the system then they will be tested first
• Disadvantages:
  • Need to write drivers
  • The testing is not according to the architecture of the system

Functional testing

Functional testing is designed to ensure the system requirements and specifications are achieved. It can start with testing of the requirements and whether the application compiles with the organizational policies and procedures.

Regression testing is another form of functional testing where you test the earlier features and functionality of the module which were working on the desired lines but have to be tested currently after some changes have been in the system. The changes might not be in the particular module but it can be affected due to inter dependencies. During the design phase, after knowing the functionality of the module/modules and the system, some errors can be predicted in advance. A functional test should incorporate such type of scenarios too. In this case the erroneous transactions/ applications are deliberately introduced and the system should be able to correctly find them. In general, all the use cases in the user’s requirements become the functional test cases.

As an example, suppose we are to develop the software for an elevator. For designing the test cases for this software, all the scenarios and operations that can take place in the elevator need to be analyzed. To begin from the basic operations, the elevator should be bidirectional and it should stop at each floor (unless we have two elevators in the requirement for even and odd floors separately).The elevator should stop at the desired floor.