CSC/ECE 517 Fall 2009/wiki1b 4 mv

Introduction

Unit Testing is given a lot of importance in Software Testing. But functional and integration testing are essential too. Hence Functional and Integration Testing is discussed here. This page provides an introduction to these testing methods; also including the good practices to write functional and integration tests. Later additional types of Software Tests are mentioned, which could be used to gain further confidence that one's code is correct.

Functional Testing

Functional testing is the process of testing of all the functions of a system, ensuring that the system conforms to the user requirements and specifications. It is sometimes referred to as black-box testing, as it does not require understanding of the internal structure of the implementation.

Functional testing can be done by using manual or automated methods[1]. In both the methods, a series of tests have to be run marking the interaction between the user and the system, ensuring that the system performs as it was supposed to. Since Use cases mainly depict the interaction between the user and system, functional test suites are created from requirement use cases, with each scenario becoming a functional test[2].

Functional Test Plan

"The functional test plan measures the quality of the functional components of the system[3]". The test plan is written by the Author. The Author lists a number of test cases, containing the input, expected results, and pre and post-test setup. After a module is unit tested, the Tester executes the functional test cases and records the actual results. The Reviewer compares the Actual results with the expected results and reports whether each test case passed or failed the test.

Guidelines to write good Functional Tests

1. The test plan must be written by persons who have extensive knowledge of the business requirements. It is advisable that the developers do not write the test plans. This is because developers tend to unknowingly write test cases whose functionality is based on the implementation rather than the initial functional specifications.
2. The functional test plans should be created as early as possible, probably as soon as the functional specifications are ready.
3. The functional test plan should be updated when there are changes in the business requirements.
4. For each functionality, a list of test cases should be defined so that each aspect of the functionality is covered.
5. The Author should clearly include the pre-test setup for the test cases so that the Tester can easily execute the test cases.
6. Usually the same test cases are run a number of times. Hence after executing a test case, any data added to the system should be cleared so as to not affect further tests or rerunning of the same test case. Hence the Author should also give post-test setup details.
7. As a module is built and unit tested, the Tester should execute the particular test cases and record the results.
8. Reviewer should review the results and report any bugs to the developers.

Examples

Examples of pre-test setup, test cases and post-test setup are given below. The complete reference for the example can be found at Functional Test Plans

The pre-test setup for Test Case: Create new user profile can be given as:

      [UserName: "A": --> username:     "A"]
      [UserName: "A": --> first name:   "John"]
      [UserName: "A": --> last name:    "Smith"]
      [UserName: "A": --> access level: "read-only"]
      [UserName: "A": --> password:     "profile1"]

The test cases for the function:Verify that only valid users can access the system can be given as:

Test Case #	Required Input	Expected Results	Results	Pass/Fail
Verify that only valid users can access the system. Verify that valid users cannot access the system without entering a valid password.
1	Navigate to the login screen	Login screen with release number A is loaded.
2	Enter user name "INVALIDUSER", password "INVALIDUSER", and click on login button	Error message: "User name is not found"
3	Enter user name for user [UserName: "A":--> username] and password "test"	Error Message: "Password for user [UserName: "A": --> username] is invalid"
4	Enter user name [UserName: "A": --> username] and password [UserName: "A": --> password]	The main page is loaded with a message: "Welcome [UserName: "A": --> first name] [UserName: "A": --> last name]

The post-test setup for the above scenario can be given as:

      Remove [UserName: "A": --> username: "A"] from the LDAP Server.

Another Example of Test Cases can be found at Initial Functional Test Cases for an ATM system

Integration Testing

Integration Testing is a process of testing the application on the whole. After going through the unit testing of the modules, these modules need to be integrated, and during this process things may go wrong, Integration process is the phase wherein these errors are captured and rectified. It is very common for modules after having successful passed all the unit test cases to fail when they are integrated with other modules. Integration testing is an important phase and sometimes there are separate teams which handle integration testing.
DEFINITION:Integration testing is a systematic technique for constructing the program structure while at the same time conducting tests to uncover errors associated with interfacing. The objective is to take unit tested components and build a program structure that has been dictated by design.[4]

Integration Test approaches

Integration testing can be done in a variety of ways

Top Down Approach

1. Top-Down Approach. In this approach the modules higher up in the control hierarchy are integrated first and low-level modules are added later. The advantage of this approach is that the modules with high level logic are tested first and hence bugs in the crucial logic are caught early. Further this approach does not require drivers (dummy control modules used instead of the actual modules for the purpose of testing) to be written. Although a number of stubs have to be written and managed
2. Bottom Up Approach. In this approach, the low level modules are integrated first and high level logic is tested late. While integrating the low level modules, the control modules are replaced by dummy modules called as drivers. Thus driver codes have to be written and managed while using this approach.

   

   Bottom Up Approach

3. Umbrella approach. Another approach is called the umbrella approach. It is a hybrid of the top down and the botom up approach, ie both the bottom up and top down approaches are used.In this approach we first test along the functional and control flow paths, for this we use the bottom up approach. The results thus gathered are later integrated in a top down manner. This kind of testing allows creating early prototypes of the system with has limited functionality thereby helping is the creations of the product in an incremental manner. This types of testing also minimizes the use of stubs and drivers which are needed in the top-down and bottom up approaches.

Prerequisites before Integration Testing

The system that is being tested, all of its modules should be rigorously tested as a component, i.e they should have gone through unit tests.

Steps to write good Integration Test Cases

1. Create a Test Plan
2. Create test cases and the sample data against which they would be tested.
3. If automated tools are being used, scripts should be created.
4. After the components have been integrated, start executing the test cases created in point 2.
5. Log the bugs found and rectify the same.
6. Retest the code.

How To write good Integration Test Cases and do effective Integration Testing

The main focus of integration testing is that it needs to check whether the different components are working smoothly and are interacting with each other in the proper manner. The Integration test cases specifically focus on the flow of data/information/control from one component to the other. A few thing that needs to be kept in mind for effective integration testing are

1. Ensure that the correct versions of the components are integrated together.
2. Automate the integration of components so that there is lesser chance for human error.
3. Log all the errors that have been found and document them with the solutions as well as the version number so that we could keep track of which build was being tested when the error was discovered.

Examples

Sample Integration test case Templates can be found at Integration test cases

Integration test plan Templates can be found at Integration test plans

Beyond Functional and Integration Testing

Test Phases

Integration and Functional testing are not the only testing that are done for checking the validity of the system. Other test cases includes:
Unit Testing: This is the first stage where the developer tests the component he/she is creating. More can be found at Unit Testing
Validation Testing: After integration Testing, validation testing takes place wherein checks are made to ensure that the functional requirements are satisfied.
Alpha and Beta Testing:When the product about to be released, it is first exposed to some end users who test the application. This is alpha beta testing. More can be found at Alpha and Beta Testing
Recovery Testing: In this phase the system is made to crash and is used to check to see if it recovers in the proper manner. More can be found at Recovery Testing
Security Testing: Systems that store and record sensitive information need to be protected from threats, in this phase the systems protection mechanisms are tested. More can be found at Security Testing
Stress Testing:The system is overloaded with data and it is checked to whether it crashes. More can be found at Stress Testing
Performance Testing: It tests the run time performance of software within the context of the system. More can be found at Performance Testing
Smoke Testing: smoke testing is a preliminary to further testing, which should reveal simple failures severe enough to reject a prospective software release. More can be found at Smoke Test
Acceptance Testing: acceptance testing performed by the customer is known as user acceptance testing (UAT). This is also known as end-user testing, site (acceptance) testing, or field (acceptance) testing. More can be found at Acceptance Testing

References

1. Automated Functional Testing
2. Functional Tests to ensure Software Quality
3. Functional Test Plans
4. Software Engineering: A Practitioner's Approach by Roger S. Pressman ISBN 0073655783

External Links

1. Initial Functional Test Cases for an ATM system
2. Integration test cases
3. Integration test plans
4. Unit Testing
5. Alpha Beta Testing
6. Recovery Testing
7. Security Testing
8. Stress Testing
9. Performance Testing
10. Smoke Test
11. Acceptance Testing