CSC/ECE 517 Fall 2009/wiki2 10 mk

GUI Testing Frameworks

Problem Statement

Most software developed in recent years has a graphical user interface (GUI). The only way for the end-user to interact with the software application is through the GUI. Hence, acceptance and system testing of the software requires GUI testing. In this wiki we aim at covering the the different approaches, including patterns and tools for GUI testing.

Some problems of GUI testing

• GUIs are tested manually, often by the developers themselves. This is very unreliable and expensive. For new GUIs or those being significantly changed, quality is low, and failures at integration time or during user acceptance tests are common.
• ScreenScrapper based GUI test does a nice job but to a certain extent. Even though they are cheap, the problem with these tests are that if you change the screen layout all existing tests become useless, which means you have no regression tests. Another problem here is that test creators can't start writing tests ill the GUIs are finished. Example: test harnesses, capture/replay tools, and model-based methods
• The user has an extremely wide choice of actions. The user could click on any pixel on the screen using manual tools to mimic the usage of the GUI, only provides limited testing.
• There are tools which try to capture GUI widgets rather than mouse coordinates. These tools, however, require a significant amount of manual effort to be effective, including developing test scripts and manually detecting failures.Modifications to the GUI require changes to the scripts as well. Example: Winrunner, Abbot, and Rational Robot

Approaches for GUI testing

1. An Ontology-Based Approach for GUI Testing

In the approach an GUI testing ontology is is established by analyzing the source code with reverse engineering techniques. Then from the user experience the generation rules are extracted to create test cases. GUI testing is proposed for the purpose of making use of the knowledge provided by GUI systems and testers’ experience. GUI ontology is used to store potential information in a GUI system, while test case generation rules extract useful information from testers’ experience. In a word, ontology based GUI testing is a new branch of software testing, which not only takes the knowledge intensive features of GUI testing into account, but also sufficiently make use of them.

3. Plan Generation GUI testing

This is based on the AI techniques, for partially automating GUI testing.In this method of the testing the tester specifies the initial and the final goal states for the users of the system.The automated system produces a set of sequences or plans which will start with the initial state and end with the final state specified by the user. Each of the plans generated will represent a test case of a user of the system

4. A practical approach to testing GUI systems

In this approach, GUI is divided into two tires. One the component and other the system. Flow graphs will be created for each GUI component. The flow graph represents a set of preconditions, event sequences and post conditions of the corresponding component. On the system tire we build a viewpoint by integrating the components of the system. This will ensure that the components are working fine and are interacting as required. This is a simple, effective and practical method of performing GUI testing.

5 A Dynamic Partitioning Approach for GUI Testing

The above approaches specify how to generate the test cases without actually specifying how to run them. This approach specifies how the test cases have to run in order to make GUI testing effective and useful. Th GUI primitive actions are partitioned into two classes. They are prioritized primitive actions and non-prioritized primitive actions. This further divides the testing into two stages which contains two feed back loops.The first stage prioritizes primitive actions and the second stage selects and performs prioritized primitive actions. The first feedback loop is local and occurs in the second stage, which adjusts the memberships of primitive actions after they are performed. The second feedback loop is global and occurs between the first and second stages. It switches GUI testing from the second stage to the first stage upon no prioritized primitive actions are available. The two testing experiments with real GUI applications show that the proposed dynamic partitioning approach can really work in practice and may significantly outperform the random testing approach.

Tools for GUI testing

1. GUITAR

The GUITAR (GUI Testing frAmewoRk) project which helps in simplifying GUI testing by automatically creating test cases that intelligently challenge a GUI's functionality. It currently contains a rich collection of plug-ins that may be used to test an application through its graphical user interface. For example, the “test case generator” plug-in, a tester can automatically generate various types of test cases for the Application Under Test (AUT); the “replayer” plug-in may be used to execute these test cases on the AUT automatically; during the various development phases of the AUT, the “regression tester” plug-in can be used to efficiently perform regression testing on the AUT.

figure 1:Guitar Framework

In order to test the GUI of the AUT, the tester has to perform a certain set of steps. These steps are detailed below

• Initialize configurations in GUITAR for the AUT. This can be done using the below window

figure 2:Preferences window for initializing the application type

• Replay the Testcases

figure 3:Testcase execution

• Coverage Evaluation

Execute the coverage evaluator to analyze the coverage generated when the testcases were executed on the instrumented AUT. A coverage report is generated by the instrumented code, when the testcases are replayed on it. The coverage evaluator analyzes this report and a summary report is generated.

figure 4:Running the coverage evaluator

2. Planning Assisted Tester for grapHical user interface Systems (PATHS)

This is based on the event interaction sequences. This tests the GUI software using interactions which are mostly likely to be used in actual scenarios. This accepts an operator, initial state and a final state, with which the planning sequence produces a series of sequences which transforms the system form the initial state to the final state. The GUI tester can use this to generate interactions sequences by specifying the final state.

3. GUIdancer

GUIdancer is an Eclipse-based automated GUI test-tool which runs as a standalone application or as an Eclipse Plugin.

GUIdancer is different from other GUI test-tools because automated tests can be written before the Application Under Test (AUT) is ready. This means that GUIdancer is not a tool which tests an application by recording user actions and replaying them. Tests can be created from the requirements without access to the AUT, and involve no programming, script or code. GUIdancer tests can be created, run and maintained without support from automation experts.

Each Test Step (the smallest unit in GUIdancer) consists of three pieces of information chosen from interactive dialogs: the GUI-component to be tested, the action to execute on this component, and the parameters (or data) the action requires. A Test Step to enter “hello” into a text field would look like this:

* GUI-component: Text field
* Action: Enter Text
* Parameter: Hello

Conclusion

In summary, GUI testing is a complicated task. Systematic test design helps us to focus on the important tests and gives us an objective way of addressing risks. Tools are appropriate for many but not all tests and a staged approach to testing enables us to identify which tests to automate much more easily. Tools can therefore be used to detect errors pro-actively as well as to execute regression tests.

References

[1] http://c2.com/cgi/wiki?GuiTesting
[2] http://chandlerproject.org/Journal/AutomatedGuiTestingProject
[3] http://en.wikipedia.org/wiki/GUI_software_testing
[4] http://www.open-xchange.com/wiki/index.php?title=Automated_GUI_Tests
[5] http://agilistas.org/presentations/codecamp06/
[6] http://www.ranorex.com/?gclid=CNvA_YnesJ0CFchW2godpT5YrQ
[7] http://www.gerrardconsulting.com/GUI/TestGui.html
[8] http://www.slideshare.net/rpires/GUI-Test-Patterns
[9] http://en.wikipedia.org/wiki/List_of_GUI_testing_tools
[10] http://www.junit.org/taxonomy/term/6
[11] http://www.cs.umd.edu/~atif/papers/MemonSQW2000.pdf
[12] http://www.testingfaqs.org/t-gui.html
[13] http://www.springerlink.com/content/d08681k5081553r7/
[14] http://www.cs.umd.edu/~atif/GUITAR-distribution/manuals/Perform%20Tests%20on%20the%20AUT.htm#Rip
[15] "A Planning-Based Approach to GUI Testing" by Atif M. Memon, Martha E. Pollack, and Mary Lou Soffa. In Proceedings of The 13th International Software/Internet Quality Week, May 2000.