CSC/ECE 517 Spring 2015 M1503 EDTS

Extending Developer Tools for Servo

Introduction

Rust

Rust is a general purpose, multi-paradigm, compiled programming language developed by Mozilla Research. It is designed to be a "safe, concurrent, practical language", supporting pure-functional, concurrent-actor, imperative-procedural, and object-oriented styles.<ref>http://en.wikipedia.org/wiki/Rust_%28programming_language%29</ref> Being a modern systems programming language focusing on safety and speed, it accomplishes these goals by being memory safe without using garbage collection.<ref>http://doc.rust-lang.org/nightly/intro.html</ref>

Servo

Servo is an experimental project to build a Web browser engine for a new generation of hardware: mobile devices, multi-core processors and high-performance GPUs. With Servo, we are rethinking the browser at every level of the technology stack — from input parsing to page layout to graphics rendering — to optimize for power efficiency and maximum parallelism. Servo builds on top of Rust to provide a secure and reliable foundation. Memory safety at the core of the platform ensures a high degree of assurance in the browser’s trusted computing base. Rust’s lightweight task mechanism also promises to allow fine-grained isolation between browser components, such as tabs and extensions, without the need for expensive runtime protection schemes, like operating system process isolation.<ref>https://www.mozilla.org/en-US/research/projects/</ref>

Background

Remote Developer Tools

Firefox supports remote developer tools - ie. communicating with an arbitrary server that implements a protocol for exposing information about web content. You can use the Firefox developer tools on your desktop to debug Web sites and Web apps running in other browsers or runtimes. The other browser might be on the same device as the tools themselves or on a different device, such as a phone connected over USB.

Project Description

Servo implements a very basic developer tools server that currently supports executing JS remotely and investigating the DOM tree in the document inspector. We want to expand these capabilities by completing previous work that enables remote logging from web content, and add new capabilities to log HTTP requests and responses to allow for easier debugging of network-related problems in Servo.<ref>https://github.com/servo/servo/wiki/More-developer-tools-student-project</ref>

The initial step included changes for enabling remote logging from the Web Console using 'console.log'. This was completed as part of the OSS project (Pull request here).

In continuation, the objective of the project is to add support for logging HTTP requests and responses in the Web Console.

Requirement Analysis

To configure Firefox for remote debugging, please follow the instructions on Setting up Firefox

Alternatively, to just view the Web Console, follow instructions here.

As mentioned above, Firefox provides the ability to debug a web page running on a remote server with its Developer Tools. The Message Display pane of the Web Console displays various kinds of messages:

• HTTP requests
• JavaScript warnings and errors
• CSS warnings, errors, and reflow events
• Security warnings and errors
• console API calls
• Input/output messages

The Message Display pane looks like this:

Open the Web Console from the Developer menu (or Ctrl+Shift+K) from a version of Firefox with Developer Tools extensions. Navigate to any web page and observe messages on the Web Console (Console tab). You may have to check the Log option under "Net" (the first option in black). The log is cleared on every redirection/reload of a page. To avoid this, you can select the "Enable Persistent Logs" option in Settings.

HTTP Requests are logged with lines that looks like this on the Web Console:

Clicking on the message brings up a new window that gives more details about the HTTP request and the response.

Versions of Firefox in use today use Gecko as the browser engine. The Developer Edition of Firefox includes complete support for the latest Firefox Developer Tools

Servo so far implements partial support for Developer Tools, which does not include the logging of HTTP requests as shown above. The goal of this project is to implement capability for logging HTTP requests and responses on the Web Console. When we run Servo with our changes on a webpage with the --devtools argument, and connect from an instance of Firefox, we should be able to see the HTTP requests and responses from the server appear on the log in the Web Console.

Implementation

The following is a rough list of changes that will be required in the code to enable logging of HTTP messages. It includes adding types for HTTP request and response messages to the enum that represents messages exchanged between Servo and the debugger client. An "actor" is created to store the information that will be sent in these messages.

• Add an HTTRequest variant to the DevtoolsControlMsg enum in devtools_traits/lib.rs, containing fields for the target url, the method, headers, and the request body. Use the types that are present in the LoadData struct in components/net/resource_task.rs.
• Add an HTTPResponse variant to the DevtoolsControlMsg enum containing fields for the response headers, status, and body. Use the same types that are present in the Metadata struct in components/net/resource_task.rs.
• Make the HTTP loader's load function take an optional Sender<DevtoolsControlMsg>. Use the cookies_chan argument as a model. Send HTTPRequest and HTTPResponse messages using this sender at the appropriate times.
• Create a NetworkEventActor actor in the devtools crate that stores the request and response information transmitted in the new messages. Add a String field to HTTPRequest and HTTPResponse which contains a unique ID that joins them - see Node::summarize for an example of creating this. Associate the unique IDs with the corresponding NetworkEventActor names via a hashtable.
• Send the networkEvent message when the HTTPRequest and HTTPResponse messages are received. Use the Firefox devtools code (onNetworkEvent) as a reference.
• Implement the getRequestHeaders, getRequestCookies, getRequestPostData, getReponseHeaders, getReponseCookies, and getResponseContent messages for NetworkEventActor.

Architecture

The conceptual architecture of Mozilla Firefox running Gecko is represented in the following figure. This is from Grosskurth and Godfrey, A Reference Architecture for Web Browsers <ref>http://grosskurth.ca/papers/browser-refarch.pdf</ref>

Component Diagrams

Servo is designed to be highly parallel, with many components (rendering, layout, HTML parsing, image decoding, etc.) handled by fine-grained, isolated tasks.

The following figures represent the browser architecture with all these components, and the Servo architecture <ref>http://www.joshmatthews.net/fosdemservo/</ref>

Browser:

Servo Architecture:

Design

enum in Rust

Enums are datatypes with several alternate representations. A simple enum defines one or more constants with the same data type. In Rust, an enum can have complex variants though, like a struct. For example, consider an enum 'Shape' with variants 'Circle' and 'Triangle' each of which is a struct.

   enum Shape {
       Circle { center: Point, radius: f64 },
       Triangle{ vert1: Point, vert2: Point, vert3: Point }
   }

A variable of type Shape can be resolved to its appropriate variant by using a 'match'.

   fn area(sh: Shape) -> f64 {
       match sh {
           Circle(_, size) => f64::consts::PI * size * size,
           Rectangle(Point { x, y }, Point { x: x2, y: y2 }) => (x2 - x) * (y2 - y)
       }
   }

The Servo Developers Tools project has a 'DevtoolsControlMsg' enum which is used to instruct the devtools server to update its known actors/state according to changes in the browser. The current project requires the addition of two new variants to the 'DevtoolsControlMsg' enum, namely 'HTTRequest' and 'HTTResponse'.

Sender<DevtoolsControlMsg>

In Rust, a pipe is used for communication between tasks. A pipe is simply a pair of endpoints: one for sending messages and another for receiving messages(Sender and Receiver). The simplest way to create a pipe is to use the channel function to create a (Sender, Receiver) pair. In Rust parlance, a sender is a sending endpoint of a pipe, and a receiver is the receiving endpoint. A simple channel can be created as follows:

   let (tx, rx) = channel();
   spawn(proc() {
       tx.send(10i);
   });
   assert_eq!(rx.recv(), 10i);

In this project, whenever a HTTPRequest or a HTTPResponse is received, we are going to send a new message using a 'Sender' object that could send variants of the type 'DevtoolsControlMsg'. This 'Sender<DevtoolsControlMsg>' objects sends the messages at appropriate times in the HTTPLoader function.

Design Patterns

Actor Model

The actor model<ref>http://en.wikipedia.org/wiki/Actor_model</ref> in computer science is a mathematical model of concurrent computation that treats "actors" as the universal primitives of concurrent computation: in response to a message that it receives, an actor can make local decisions, create more actors, send more messages, and determine how to respond to the next message received.

The Actor model adopts the philosophy that everything is an actor. This is similar to the everything is an object philosophy used by some object-oriented programming languages, but differs in that object-oriented software is typically executed sequentially, while the Actor model is inherently concurrent.

An actor is a computational entity that, in response to a message it receives, can concurrently:

• send a finite number of messages to other actors;
• create a finite number of new actors;
• designate the behavior to be used for the next message it receives.

UML Diagrams

Sequence Diagram

Proposed Test Cases

Testing the feature

The feature added will be tested using the following steps:

• Build Servo successfully with the changes (./mach build)
• Run Servo on a web page with the --devtools argument specifying a port number, say 6000 (./mach run [url] --devtools 6000)
• Open an instance of a recent version Firefox configured with Remote Debugging enabled.
• Connect to Servo running the web page on port 6000.
• Enable persistent logs in Settings. Select logging of Net messages from the Console.
• Navigate to another url from the webpage running Servo.
• The HTTP request and response messages should be logged on the Console.

Regression Testing

We want to ensure that everything is in order after we have completed our changes; more importantly, that we have not broken any existing functionality. Mozilla has an automated workflow in place for running tests on pull requests submitted to Servo. Servo has a test suite which runs on every reviewed pull request, via bors-servo and the buildbot.

After it sees an approval on a reviewed pull requests, bors-servo automatically merges the changes

It starts off running a suite of regression tests, and any failure is reported. It can be issued a retry command once the fixes are in place.

When all tests pass, the changes are successfully merged into master. The following is the set of tests that will be run after the pull request is reviewed:

Reference

<references/>