CSC/ECE 517 Fall 2009/wiki2 10 Gui Testing Frameworks aa
GUI TESTING FRAMEWORKS
Introduction
A Graphical User Interface is a program interface that takes advantage of the computer's graphics capabilities to make the program easier to use.Ideally, user interfaces should only be thin shells on top of the next layer in an application displaying information in a useful fashion, and passing user input as quickly as possible to some code that knows what to do with it.However due to its nature there are many difficulties that arise with GUI testing.
GUIs are the assumed user interface for virtually all systems development using modern technologies. Graphical User Interface deliver the system functionality for the clients in the client/server system. Testing systems become more complex with GUI because of the event-driven nature of GUI, unsolicited events and a never ending input domain. Thus there aren’t any developed or implemented testing patterns for GUI testing.
The article here describes the problems associated with GUI testing and several possible patterns and tools used for GUI Testing.
GUI Testing Difficulties
As GUI hides the complexity from the user and from the programmers too on using development frameworks but the hidden issues become really obvious when it comes to testing. GUI testing is much more difficult comparatively because
1. There is an infinite number of unnoticed events that occur when a user is using the application making. This makes it really difficult for a code to include all of them and to complicate the issue further special test drivers are needed in the Operating system to generate such events. For instance when a printer in a LAN is switched off, and the operating system puts up a dialog box inviting the user to feed more paper into the printer.
2. A major problem involved with GUI testing is a large number of input spaces wherein user can click on any point on the screen at any time and there may be an undefined number of applications running simultaneously. This means that the application code needs to check the next event irrespective of whether it’s meant for it. This event driven approach makes testing an extremely complicated issue.
3. GUIs are better suited to object oriented paradigm. A large number of methods and attributes are associated with each object that deals with GUI events thus making it even more complex. For example even for a simple text box there may be around thirty attributes which means a larger set of test cases to deal with.
4. GUI environment deals with windows and user take the standard features of window management (re-size, move, maximize and minimize) for granted. These events do affect the application and thus needs to be taken care of. For instance testing backward, forward, refresh etc makes testing a challenging task.
5. GUI tests can’t be written until you finish the implementation phase. This is an issue considering the fact that there may be contractors developing the GUI who may not be available for testing thus forcing the tester to reverse engineer making life miserable for him.
6. GUIs involve complex dependencies thereby resulting in a synchronization problem. For example if an application involves two windows one showing a portfolio of your stock market investments and the other making transactions then as soon as you buy a stock the user would like to see it in the portfolio, now for that to happen you need the two windows to be synchronized. With a large number of windows involved with the application the dependencies are extremely complex. Now a user may also want his transaction history window, the order book window etc be synchronized too with the transaction window. This shows the problem of a large number of dependencies which makes synchronization complicated.
7. There can be many ways a user can reach a point in the application which may be a keyboard shortcut, function keys or mouse movements creating a never ending list of test cases
8. An infinite input domain problem is an issue wherein a user has an option of clicking anywhere on the window in view and entering data in any order he wishes too. This may result in a n factorial ways of inputting data where n is the number of fields in the GUI application. Greater the n larger the input domain and more the complexity for a tester.
Different Approaches used for GUI Testing
The different approaches used for GUI testing are as follows
Capture and Replay Tool
In this approach, as the name suggests, the user starts testing the web applications manually thereby invoking the events and the test software starts recording all the actions. These actions identify the respective GUI widgets and are collected in the form of events. The recording forms the basis of automating the test for web applications which are replayed via the GUI.
A simple workflow explaining the test approach is shown below
PLZ INSERT IMAGE
figure 1 testing web applications with Capture & Replay – workflow
Drawbacks
• As mentioned in the problems before, this can be tested only after the development has finished.
• A layout may change over the time and hence the test cases may become invalid owing to the changes in the physical interface.
• It lacks the necessary support to design the test cases and coverage criteria evaluation.
MVC Design Pattern
It is typically more difficult to perform automated tests on code residing in the user interface, and the more logic contained there, the greater the need for automated testing. The solution is the Model View Controller pattern. The basic idea behind this approach is to add a new level of indirection behind the user interface and the domain model with the intention of isolating UI changes and prevent them from requiring changes to the domain logic of the application. MVC divides an application into three concerns:
- Model - Encapsulates core application data and functionality domain logic.
- View - obtains data from the model and presents it to the user.
- Controller - receives and translates input to requests on the model or the view.
It pulls the complex part of the user interface code into a library that can be unit-tested, thus making GUI testing simpler.
Test Stubs and Drivers
A test stub is a dummy software component or object used (during development and testing) to simulate the behavior of a real component. More specifically while intergrating all the modules in top to bottom approach whenever any mandatory module is missing that module is replaced by a temperoary program which is called as stub.
If the function A you are testing, calls another function B,then use a simplified version of function B, called a stub.
void function_under_test(int& x, int& y) {
...
p = price(x);
...
}
double price(int x) {return 10.00;}
The value returned by function price is good enough for testing.The real price() function may not yet have been tested, or even
written.
Similarly, while integrating all the modules in bottom to top approach whenever any mandatory module is missing that module is replaced by a temperoary program which is called as test driver.
//Driver program for the function get_input.
#include <iostream.h>
void get_input(x& cost, int& y);
//Precondition: User is ready to enters values correctly.Postconditions: The value of cost has been set to the.
//wholesale cost of one item. The value of turnover has been set to the expected number of days until the item is sold.
int main( )
{ double a;
int b;
char ans;
do {
get_input(a, b);
cout.setf(ios::fixed);
cout.setf(ios::showpoint);
cout.precision(2);
cout << "a is " << a << endl;
cout << "b is " << b << endl;
cout << "Test again?"
<< " (Type y for yes or n for no): ";
cin >> ans;
cout << endl;
} while (ans == 'y' || ans == 'Y');
return 0;
}
Capture and Replay Tool
Guitar GUI Testing Framework
GUITAR is an extensible framework for testing software applications. It presents a unified solution to the software testing problem. GUITAR’s architecture allows testers to create new “process plug-ins” (i.e., tools/techniques/algorithms) to perform different testing activities. It currently contains a rich collection of plug-ins that may be used to test an application through its graphical user interface. For example, the “test case generator” plug-in, a tester can automatically generate various types of test cases for the Application Under Test (AUT); the “replayer” plug-in may be used to execute these test cases on the AUT automatically; during the various development phases of the AUT, the “regression tester” plug-in can be used to efficiently perform regression testing on the AUT.
Cucumber
Cucumber is a tool that can execute plain-text functional descriptions as automated tests. The language that Cucumber understands is called Gherkin. Here is an example:
Feature: Search courses
In order to ensure better utilization of courses
Potential students should be able to search for courses
Scenario: Search by topic
Given there are 240 courses which do not have the topic "biology"
And there are 2 courses A001, B205 that each have "biology" as one of the topics
When I search for "biology"
Then I should see the following courses:
| Course code |
| A001 |
| B205 |
While Cucumber can be thought of as a “testing” tool, the intent of the tool is to support BDD. This means that the “tests” (plain text feature descriptions with scenarios) are typically written before anything else and verified by business analysts, domain experts, etc. non technical stakeholders. The production code is then written outside-in, to make the stories pass.