CSC/ECE 517 Fall 2009/wiki1b 4 mv

Problem Statement

We have focused mainly on unit tests in this course. But functional and integration testing are also important. How does one write good functional and integration tests? Are there any kinds of tests beyond unit, functional, and integration that one should implement to gain confidence that one's code is correct?

Different stages of Software Testing

http://www.aptest.com/testtypes.html (either use reference to this place or type in our own words)

Functional Testing

Functional testing is the process of testing of all features and functions of a system [software, hardware, etc.] to ensure requirements and specifications are met[1]. Functional testing ensures that the system conforms to the user requirements and specifications. It is sometimes referred to as black box testing[2], as it does not require understanding of the internal structure of the implementation.
 
Functional testing can involve either manual or automated methods [3]. In both the methods a series of tests have to be run marking the interaction between the user and the, system ensuring that the system performs as it was supposed to. Since Use cases mainly depict the interaction between the user and system functional test suites are created from requirement use cases, with each scenario becoming a functional test[4].

Use cases are generally developed from the user specifications. These use cases are useful in providing the guidelines for functional testing.

Prerequisites before Functional Testing

1. Build Functional test cases from uses cases, with each use case scenario becoming a functional test.
2. As a module is built and unit tested apply the particular test case.
3. Prioritize the test cases.
4. it should be geared towards a non-technical audience. Therefore, references to underlying programming languages must not be present in the final document
5. The author of the functional test plan must be a person who has in-depth knowledge of the business requirements. It is preferable that application developers do not compose functional test plans. The reason behind such an approach is simple: The developer's interpretation of the requirements might differ from those of the person who initially created them.

Integration Testing

Integration Testing is a process of testing the application on the whole. After going through the unit testing of the modules, these modules need to be integrated, and during this process things may go wrong, Integration process is the phase wherein these errors are captured and rectified. It is very common for modules after having successful passed all the unit test cases to fail when they are integrated with other modules. Integration testing is an important phase and sometimes there are separate teams which handle integration testing.
DEFINITION:Integration testing is a systematic technique for constructing the program structure while at the same time conducting tests to uncover errors associated with interfacing. The objective is to take unit tested components and build a program structure that has been dictated by design.

Integration Test approaches

Integration testing can be done in a variety of ways

Top Down Approach

1. Top-Down Approach. In this approach the modules higher up in the control hierarchy are integrated first and low-level modules are added later. The advantage of this approach is that the modules with high level logic are tested first and hence bugs in the crucial logic are caught early. Further this approach does not require drivers to be written. Although a number of stubs have to be written and managed
2. Bottom Up Approach. In this approach, the low level modules are integrated first and high level logic is tested late. While integrating the low level modules, the control modules are replaced by dummy modules called as drivers. Thus driver codes have to be written and managed while using this approach.

   

   Bottom Up Approach

3. Umbrella approach. Another approach is called the umbrella approach. It is a hybrid of the top down and the botom up approach, ie both the bottom up and top down approaches are used.In this approach we first test along the functional and control flow paths, for this we use the bottom up approach. The results thus gathered are later integrated in a top down manner. This kind of testing allows creating early prototypes of the system with has limited functionality thereby helping is the creations of the product in an incremental manner. This types of testing also minimizes the use of stubs and drivers which are needed in the top-down and bottom up approaches.

Prerequisites before Integration Testing

The system that is being tested, all of its modules should be rigorously tested as a component, i.e they should have gone through unit tests.

Steps in writing good Integration Test Cases

1. Create a Test Plan
2. Create test cases and the sample data against which they would be tested.
3. If automated tools are being used, scripts should be created.
4. After the components have been integrated, start executing the test cases created in point 2.
5. Log the bugs found and rectify the same.
6. Retest the code.

How To write Good Integration Test Cases and do effective Integration Testing

The main focus of integration testing is that it needs to check whether the different components are working smoothly and are interacting with each other in the proper manner. The Integration test cases specifically focus on the flow of data/information/control from one component to the other. A few thing that needs to be kept in mind for effective integration testing are

1. Ensure that the correct versions of the components are integrated together.
2. Automate the integration of components so that there is lesser chance for human error.
3. Log all the errors that have been found and document them with the solutions as well as the version number so that we could keep track of which build was being tested when the error was discovered.

Beyond

Integration and Functional testing are not the only testing that are done for checking the validity of the system. Other test cases includes:
Unit Testing:[5] This is the first stage where the developer tests the component he/she is creating.
Validation Testing: After integration Testing, validation testing takes place wherein checks are made to ensure that the functional requirements are satisfied.
Alpha and Beta Testing:[6] When the product about to be released, it is first exposed to some end users who test the application. This is alpha beta testing.
Recovery Testing:[7] In this phase the system is made to crash and is used to check to see if it recovers in the proper manner.
Security Testing[8] Systems that store and record sensitive information need to be protected from threats, in this phase the systems protection mechanisms are tested.
Stress Testing:[9] The system is overloaded with data and it is checked to whether it crashes.
Performance Testing:[10] It tests the run time performance of software within the context of the system.
Smoke Testing [11] smoke testing is a preliminary to further testing, which should reveal simple failures severe enough to reject a prospective software release.
Acceptance Testing:[12] acceptance testing performed by the customer is known as user acceptance testing (UAT). This is also known as end-user testing, site (acceptance) testing, or field (acceptance) testing.

http://www.davenicolette.net/articles/ftdd.pdf