CSC/ECE 517 Spring 2016/Write automated tests for WebDriver

Write automated tests for the WebDriver server

Servo is an open source prototype web browser layout engine being developed by Mozilla, and it is written in Rust language. It is currently developed on 64bit OS X, 64bit Linux, Android, and Gonk (Firefox OS). The main idea is to create a highly parallel environment, in which different components can be handled by fine grained, isolated tasks. The different components can be rendering, HTML parsing, etc.

Background information:

Python:

Python is a widely used high-level, general-purpose, interpreted, dynamic programming language. Python supports multiple programming paradigms, including object oriented, imperative and functional programming or procedural styles. It features a dynamic type system and automatic memory management and has a large and comprehensive standard library. Modules for creating graphical user interfaces, connecting to relational databases, pseudorandom number generators, arithmetic with arbitrary precision decimals,[77] manipulating regular expressions, and doing unit testing are also included.

Servo:

Servo is a project to develop a new Web browser engine to help mitigate vulnerabilities associated with incorrect memory management and data races. Servo is written in Rust, a new language designed specifically with Servo's requirements in mind. Rust provides a task-parallel infrastructure and a strong type system that enforces memory safety and data race freedom.

Servo is focused both on supporting a full Web browser, through the use of the purely HTML-based user interface Browser.html and on creating a solid, embeddable engine. Although Servo was originally a research project, it was implemented with the goal of having production-quality code and is in the process of shipping several of its components in the Firefox browser.

The above diagram denotes several of the tasks that are associated with building the Servo web browser. Each of the boxes represents a task that is implemented in Rust and the lines indicates relationships between the tasks.

WebDriver:

The WebDriver specification defines a communication protocol for automating interactions with a web browser that would traditionally require a human being (eg. clicking the mouse on a particular link; typing text into an input field; etc.) This protocol is being implemented by major browsers such as Firefox, Edge, and Chrome, and Servo also provides a WebDriver server. The goal of this project is both to submit a broad range of automated client tests using the Python WebDriver API that can be run against multiple browsers, and also to report on how many of the tests Servo currently passes.

Scope

Initial Steps

• Compile Servo and ensure that it runs on tests/html/about-mozilla.html
• To run a simple webdriver script using the python client against Servo.
• Create amach command (test-webdriver) in python/servo/testing_commands.py that will import each python file in the directory and execute the test method contained within it by invoking the Servo process before running each test, and killing it after each test finishes.
• Create a webdriver directory under tests where the forthcoming tests will reside.
• Writing an automated test for loading a particular URL, asserting that the resulting page is the expected url.

Subsequent Steps

• Writing tests that exercise the lower-level command API (eg. self.send_command("POST", "url", body)) to test server capabilities:
• Writing a test with a simple get to a url that exists on the server, ensure that we get the right response type
• Sending a non-string as the URL and ensuring that we get the right kind of error message.
• Trying other kinds of non-conforming messages e.g. ones with extra fields (should be ignored) or no url field (should return the correct type of error)
• Testing that we are actually loading the correct page. This will require the use of some other webdriver functions e.g. to get an element's text, or take a screenshot
• Testing that Get always navigates the top level browsing context (by navigating to a page with an iframe, setting the current bc to that iframe, and ensuring that it is not just the iframe that is navigated).
• Trying Get with pages that return non-200 responses.
• Loading a page that loads slower than the timeout, and ensure we get the correct error message

Design

Design patterns are not applicable as our task involved writing automated tests in python. However, the Implementation section below provides details of the steps the way it was implemented.

We have implemented the test cases based on the input from the Mozilla team and there were several major tasks that were implemented.

1. Execute tests on a standalone HTTP server: In this we wanted to test the functionality if a web page opens as expected on the wed driver engine. We have implemented this by giving a URL which opens a web page whose actual URL is different when the page opens. We matched the web page expected with the one that is opened to successfully execute the test case.

Implementation

Initial Steps

The following steps were followed to meet the project requirements as per this github page.

Step 1:

We created a new mach command (test-webdriver) in python/servo/testing_commands.py file. It imports each python file in the directory and executes the test method contained within it.

Step 2:

We created tests in test.py and test1.py for loading a particular URL, asserting that the resulting page is the expected url.

Step 1:

We would create 3 different test.py files for each of the different tests that would reside in tests/webdriver folder.

Step 2:

Creating a ServoProcess that would be used by the test.py files to start a servo session.

Step 3:

Catching exceptions when the tests fail and displaying them in proper format based on the kind of different error.

Testing

Following are the steps to run the automated tests for WebDriver:

git clone https://github.com/krunal3103/servo 

Note: Follow the steps listed in the Readme.md file of this link to clone and build servo on your local machine.

cd servo 

./mach test-webdriver 

You will see that all tests pass as expected.

Pull Request

Here is our pull request. In the link you can see all code snippets and commits as well as integration test progression information.