CSC/ECE 517 Fall 2009/wiki2 10 Gui Testing Frameworks aa
GUI TESTING FRAMEWORKS
Introduction
A Graphical User Interface is a program interface that takes advantage of the computer's graphics capabilities to make the program easier to use.Ideally, user interfaces should only be thin shells on top of the next layer in an application displaying information in a useful fashion, and passing user input as quickly as possible to some code that knows what to do with it.However due to its nature there are many difficulties that arise with GUI testing.
GUIs are the assumed user interface for virtually all systems development using modern technologies. Graphical User Interface deliver the system functionality for the clients in the client/server system. Testing systems become more complex with GUI because of the event-driven nature of GUI, unsolicited events and a never ending input domain. Thus there aren’t any developed or implemented testing patterns for GUI testing.
The article here describes the problems associated with GUI testing and several possible patterns and tools used for GUI Testing.
GUI Testing Difficulties
As GUI hides the complexity from the user and from the programmers too on using development frameworks but the hidden issues become really obvious when it comes to testing. GUI testing is much more difficult comparatively because
1. There is an infinite number of unnoticed events that occur when a user is using the application making. This makes it really difficult for a code to include all of them and to complicate the issue further special test drivers are needed in the Operating system to generate such events. For instance when a printer in a LAN is switched off, and the operating system puts up a dialog box inviting the user to feed more paper into the printer.
2. A major problem involved with GUI testing is a large number of input spaces wherein user can click on any point on the screen at any time and there may be an undefined number of applications running simultaneously. This means that the application code needs to check the next event irrespective of whether it’s meant for it. This event driven approach makes testing an extremely complicated issue.
3. GUIs are better suited to object oriented paradigm. A large number of methods and attributes are associated with each object that deals with GUI events thus making it even more complex. For example even for a simple text box there may be around thirty attributes which means a larger set of test cases to deal with.
4. GUI environment deals with windows and user take the standard features of window management (re-size, move, maximize and minimize) for granted. These events do affect the application and thus needs to be taken care of. For instance testing backward, forward, refresh etc makes testing a challenging task.
5. GUI tests can’t be written until you finish the implementation phase. This is an issue considering the fact that there may be contractors developing the GUI who may not be available for testing thus forcing the tester to reverse engineer making life miserable for him.
6. GUIs involve complex dependencies thereby resulting in a synchronization problem. For example if an application involves two windows one showing a portfolio of your stock market investments and the other making transactions then as soon as you buy a stock the user would like to see it in the portfolio, now for that to happen you need the two windows to be synchronized. With a large number of windows involved with the application the dependencies are extremely complex. Now a user may also want his transaction history window, the order book window etc be synchronized too with the transaction window. This shows the problem of a large number of dependencies which makes synchronization complicated.
7. There can be many ways a user can reach a point in the application which may be a keyboard shortcut, function keys or mouse movements creating a never ending list of test cases
8. An infinite input domain problem is an issue wherein a user has an option of clicking anywhere on the window in view and entering data in any order he wishes too. This may result in a n factorial ways of inputting data where n is the number of fields in the GUI application. Greater the n larger the input domain and more the complexity for a tester.
Different Approaches used for GUI Testing
MVC Design Pattern
It is typically more difficult to perform automated tests on code residing in the user interface, and the more logic contained there, the greater the need for automated testing. The solution is the Model View Controller pattern. The basic idea behind this approach is to add a new level of indirection behind the user interface and the domain model with the intention of isolating UI changes and prevent them from requiring changes to the domain logic of the application. MVC divides an application into three concerns:
- Model - Encapsulates core application data and functionality domain logic.
- View - obtains data from the model and presents it to the user.
- Controller - receives and translates input to requests on the model or the view.
It pulls the complex part of the user interface code into a library that can be unit-tested, thus making GUI testing simpler.
Test Stub
A test stub is a dummy software component or object used (during development and testing) to simulate the behavior of a real component. More specifically it is programs which simulates the behavior of software components (or modules) that are the dependent modules of a under test module. It is a pattern used for testing interfaces in an application if the real components have not been implemented or are too expensive to use for testing.
A pattern is a recurring solution to a recurring problem in a context.
First, a test-specific implementation of an interface on which the System under testing(SUT) depends is defined. This implementation is configured to respond to calls from the SUT with the values (or exceptions) that will exercise the Untested Code within the SUT. Before exercising the SUT, we install the Test Stub so that the SUT uses it instead of the real implementation. When called by the SUT during test execution, the Test Stub returns the previously defined values. The test can then verify the expected outcome in the normal way.
Capture and Replay Tool
Guitar GUI Testing Framework
GUITAR is an extensible framework for testing software applications. It presents a unified solution to the software testing problem. GUITAR’s architecture allows testers to create new “process plug-ins” (i.e., tools/techniques/algorithms) to perform different testing activities. It currently contains a rich collection of plug-ins that may be used to test an application through its graphical user interface. For example, the “test case generator” plug-in, a tester can automatically generate various types of test cases for the Application Under Test (AUT); the “replayer” plug-in may be used to execute these test cases on the AUT automatically; during the various development phases of the AUT, the “regression tester” plug-in can be used to efficiently perform regression testing on the AUT.
Cucumber
Cucumber is a tool that can execute plain-text functional descriptions as automated tests. The language that Cucumber understands is called Gherkin. Here is an example:
Feature: Search courses
In order to ensure better utilization of courses
Potential students should be able to search for courses
Scenario: Search by topic
Given there are 240 courses which do not have the topic "biology"
And there are 2 courses A001, B205 that each have "biology" as one of the topics
When I search for "biology"
Then I should see the following courses:
| Course code |
| A001 |
| B205 |
While Cucumber can be thought of as a “testing” tool, the intent of the tool is to support BDD. This means that the “tests” (plain text feature descriptions with scenarios) are typically written before anything else and verified by business analysts, domain experts, etc. non technical stakeholders. The production code is then written outside-in, to make the stories pass.