CSC/ECE 517 Fall 2009/wiki2 10 28

GUI testing frameworks

GUI is a graphical user interface [1] that allows users to interact with the system. The user has more natural approach to the applications and feels more in control of the system. Today, every application provides GUI support and its easy to use feature is making it popular among users. The process of testing the user interface to test if it meets its specification is called GUI testing [2].

Challenges in GUI testing

Testing Graphical User Interfaces (GUI) is a difficult task because GUI possesses a large number of states that are to be tested; the input space is extremely large due to different permutations of inputs and events and also there are complex GUI dependencies, which may exist.

Input domain:Testing graphical user interface requires different permutations of input. This not only increases the complexity but also the input size domain. These inputs are users interactions or events while output could be an event or some graphical object [8].Also, one state of GUI may depend on many other states. This further increases the difficulty in GUI testing.
Sequences: -
Regression testing:
Maintainability
Event driven nature: Users have full control while using graphical user interface. This leads to many user inputs and multiple windows to be opened. Also, there could be many events like user clicking textbox field or a button and since user has many available options, large number of actions and inputs can be expected. The framework has to take care of these events. Thus, it’s tough for tester to write test case for every possible event [3].

Current GUI testing frameworks

Capture/Replay framework

Capture/Replay is the most popular technique in GUI testing domain. In this technique the capture tool generates a script that records every interaction of tester with the application GUI. This file is later replayed by the replay component of the software [6]. However, this process of GUI testing is resource intensive and requires significant human intervention [6].

Pitfalls of Capture/Replay technique

Capture/Replay technique relies on test engineer to provide all possible GUI interactions. Automation of this technique requires programmer to code to cover all possible decision points in the GUI [7]. Thus, it’s a very time consuming, requires human intervention and there are high chances of missing important decision points while supporting automation [7].

Test Monkeys

Monkey test Manual testing may not detect all the bugs in an application. Even if automation of manual testing is done by generating test scripts, it will still lack human common sense; making it difficult to cover high risk part of an application. Test Monkeys are automated tools that randomly assign test cases, leading to large range input size. Thus, working along with the test scripts , with no human intervention and large input size combination, makes it possible to test an application effectively. The other advantage of using Test Monkeys is that its user interface independent, thus any change in GUI will not affect its performance.

IBM Rational Test Tools

IBM’s Rational software branch produce large number of testing tools for every stage of development.

Rational Robot

The user needs to activate the capture/replay recorder in Microsoft Visual Studio 6.It captures the user interactions and generate script written in visual basic type language called SQABasic. The lacks effective capturing and verification of objects. For more details, please visit http://www-01.ibm.com/software/awdtools/tester/robot/index.html

Rational Visual Test

This tool allows testers either to write test cases or record test sessions. Applications built on Microsoft visual studio are tested using this tool. Testers can also provide test data while performing recording sessions.For more details, please visit http://www-01.ibm.com/software/awdtools/visualtest/support/index.html

Mercury Interactive tools

Mercury Interactive toolsprovides tool for testing internet applications and customized packages.

WinRunner

WinRunner is a testing tool for windows and web applications based on Capture/Replay technique. The recorded scripts are written in language called Test Script Language (TSL). After generating test scripts, the testers can add or remove GUI components, insert test data etc.

LoadRunner

LoadRunner acts as a virtual user to simulate the real user actions on the scripts generated by other tools. Its main feature is to perform stress testing, measure responsiveness, identify bottlenecks and other performance related issues.

References

[1] http://en.wikipedia.org/wiki/Graphical_user_interface
[2] http://en.wikipedia.org/wiki/GUI_software_testing
[3] http://www.gerrardconsulting.com/GUI/TestGui.html
[4] http://en.wikipedia.org/wiki/List_of_GUI_testing_tools
[5] http://www.geocities.com/xtremetesting/TestingDictionary.html#S
[6] GUI Ripping: Reverse Engineering of Graphical User Interfaces for Testing, Ishan Banerjee, Adithya Nagarajan, and Atif Memon
[7] http://www.cs.umd.edu/~atif/papers/MemonIEEEComputer2002.pdf
[8] Lee J. White, "Regression Testing of GUI Event Interactions," icsm, pp.350, 12th International Conference on Software Maintenance (ICSM'96), 1996