CSC/ECE 517 Fall 2009/wiki2 10 28: Difference between revisions
| Line 4: | Line 4: | ||
==Challenges in GUI testing== | ==Challenges in GUI testing== | ||
Testing Graphical User Interfaces (GUI) is a difficult task because GUI possesses a large number of states that are to be tested; the input space is extremely large due to different permutations of inputs and events | Testing Graphical User Interfaces (GUI) is a difficult task because GUI possesses a large number of states that are to be tested; the input space is extremely large due to different permutations of inputs and events. | ||
'''Input domain:'''Testing graphical user interface requires different permutations of input. This not only increases the complexity but also the input size domain. These inputs are users interactions or events while output could be an event or some graphical object [8].Also, one state of GUI may depend on many other states. This further increases the difficulty in GUI testing. | '''Input domain :'''Testing graphical user interface requires different permutations of input. This not only increases the complexity but also the input size domain. These inputs are users interactions or events while output could be an event or some graphical object [8].Also, one state of GUI may depend on many other states. This further increases the difficulty in GUI testing. | ||
<br> | <br> | ||
''' | '''Regression testing :''' | ||
<br> | <br> | ||
''' | '''Maintainability :''' | ||
<br> | <br> | ||
'''Event driven nature :''' Users have full control while using graphical user interface. This leads to many user inputs and multiple windows to be opened. Also, there could be many events like user clicking textbox field or a button and since user has many available options, large number of actions and inputs can be expected. The framework has to take care of these events. Thus, it’s tough for tester to write test case for every possible event [3]. | |||
'''Event driven nature:''' Users have full control while using graphical user interface. This leads to many user inputs and multiple windows to be opened. Also, there could be many events like user clicking textbox field or a button and since user has many available options, large number of actions and inputs can be expected. The framework has to take care of these events. Thus, it’s tough for tester to write test case for every possible event [3]. | |||
== Current GUI testing frameworks == | == Current GUI testing frameworks == | ||
=== Capture/Replay framework === | === Capture/Replay framework === | ||
Revision as of 01:37, 10 October 2009
GUI testing frameworks
GUI is a graphical user interface [1] that allows users to interact with the system. The user has more natural approach to the applications and feels more in control of the system. Today, every application provides GUI support and its easy to use feature is making it popular among users. The process of testing the user interface to test if it meets its specification is called GUI testing [2].
Challenges in GUI testing
Testing Graphical User Interfaces (GUI) is a difficult task because GUI possesses a large number of states that are to be tested; the input space is extremely large due to different permutations of inputs and events.
Input domain :Testing graphical user interface requires different permutations of input. This not only increases the complexity but also the input size domain. These inputs are users interactions or events while output could be an event or some graphical object [8].Also, one state of GUI may depend on many other states. This further increases the difficulty in GUI testing.
Regression testing :
Maintainability :
Event driven nature : Users have full control while using graphical user interface. This leads to many user inputs and multiple windows to be opened. Also, there could be many events like user clicking textbox field or a button and since user has many available options, large number of actions and inputs can be expected. The framework has to take care of these events. Thus, it’s tough for tester to write test case for every possible event [3].
Current GUI testing frameworks
Capture/Replay framework
Capture/Replay is the most popular technique in GUI testing domain. In this technique the capture tool generates a script that records every interaction of tester with the application GUI. This file is later replayed by the replay component of the software [6]. However, this process of GUI testing is resource intensive and requires significant human intervention [6].
Pitfalls of Capture/Replay technique
Capture/Replay technique relies on test engineer to provide all possible GUI interactions. Automation of this technique requires programmer to code to cover all possible decision points in the GUI [7]. Thus, it’s a very time consuming, requires human intervention and there are high chances of missing important decision points while supporting automation [7].
Test Monkeys
Monkey test Manual testing may not detect all the bugs in an application. Even if automation of manual testing is done by generating test scripts, it will still lack human common sense; making it difficult to cover high risk part of an application[9]. Test Monkeys are automated tools that randomly assign test cases, leading to large range input size. Thus, working along with the test scripts , with no human intervention and large input size combination, makes it possible to test an application effectively. The other advantage of using Test Monkeys is that its user interface independent, thus any change in GUI will not affect its performance[9].
IBM Rational Test Tools
IBM’s Rational software branch produce large number of testing tools for every stage of development.
Rational Robot
The user needs to activate the capture/replay recorder in Microsoft Visual Studio 6.It captures the user interactions and generate script written in visual basic type language called SQABasic. The lacks effective capturing and verification of objects. For more details, please visit http://www-01.ibm.com/software/awdtools/tester/robot/index.html
Rational Visual Test
This tool allows testers either to write test cases or record test sessions. Applications built on Microsoft visual studio are tested using this tool. Testers can also provide test data while performing recording sessions.For more details, please visit http://www-01.ibm.com/software/awdtools/visualtest/support/index.html
Mercury Interactive tools
Mercury Interactive toolsprovides tool for testing internet applications and customized packages.
WinRunner
WinRunner is a testing tool for windows and web applications based on Capture/Replay technique. The recorded scripts are written in language called Test Script Language (TSL). After generating test scripts, the testers can add or remove GUI components, insert test data etc.
LoadRunner
LoadRunner acts as a virtual user to simulate the real user actions on the scripts generated by other tools. Its main feature is to perform stress testing, measure responsiveness, identify bottlenecks and other performance related issues.
Segue’s Silk Test
Segue’s Silk testing tool is used for testing GUI components built using Microsoft foundation Class library.It comes with capture/replay component and a interactive wizard that allow testers to select the GUI components to be tested.The testers actions are recorded into test scripts an language used is 4Test.
Open Source Testing tools
Most of the open source GUI testing tool are for java platform.
Abbot
Abbot is a testing tool for Java GUI test framework and performs unit and functional testing. User actions are reproduced and the state of GUI components are examined. Java code or scripts can be used to invoke this tool.
GUIdancer
GUIdancer an automatic testing tool for generation of java and swing test programs. No programming is required, easy to create tests and reusable.
Jacareto
Jacareto is a testing tool for java application and applets. This open source capture/replay tool has 3 parts: Jacareto: event record and playback, Picorder: command line interface and CleverPHL: graphical user interface. It allows customization i.e. testers can design their own capture and replay modules. It can also be used to perform quantitative analysis.
Pounder
Poundera utility that does dynamic loading of Java GUI and recorded scripts are used in test harnessing.
References
[1] http://en.wikipedia.org/wiki/Graphical_user_interface
[2] http://en.wikipedia.org/wiki/GUI_software_testing
[3] http://www.gerrardconsulting.com/GUI/TestGui.html
[4] http://en.wikipedia.org/wiki/List_of_GUI_testing_tools
[5] http://www.geocities.com/xtremetesting/TestingDictionary.html#S
[6] GUI Ripping: Reverse Engineering of Graphical User Interfaces for Testing, Ishan Banerjee, Adithya Nagarajan, and Atif Memon
[7] http://www.cs.umd.edu/~atif/papers/MemonIEEEComputer2002.pdf
[8] Lee J. White, "Regression Testing of GUI Event Interactions," icsm, pp.350, 12th International Conference on Software
Maintenance (ICSM'96), 1996
[9] Effective GUI test automation: developing an automated GUI testing tool, by Kanglin Li, Mengqi Wu.