Expertiza_Wiki - User contributions [en]

CSC/ECE 517 Fall 2015 M1502 Improve HTTP monitoring devtool support

2015-11-10T04:37:07Z

Abhina: Created page with "Extending Developer Tools for Servo __TOC__ =='''Introduction'''== ===Rust=== Rust is a general purpose, [http://en.wikipedia.org/wiki/Programming_pa..."

Extending Developer Tools for Servo
__TOC__
=='''Introduction'''==
===Rust===
Rust is a general purpose, [http://en.wikipedia.org/wiki/Programming_paradigm#Multi-paradigm multi-paradigm], compiled programming language developed by [http://en.wikipedia.org/wiki/Mozilla Mozilla] Research. It is designed to be a "safe, concurrent, practical language", supporting [http://en.wikipedia.org/wiki/Purely_functional pure-functional], [http://en.wikipedia.org/wiki/Actor_model concurrent-actor], [http://en.wikipedia.org/wiki/Procedural_programming imperative-procedural], and object-oriented styles.<ref>http://en.wikipedia.org/wiki/Rust_%28programming_language%29</ref> Being a modern systems programming language focusing on safety and speed, it accomplishes these goals by being memory safe without using garbage collection.<ref>http://doc.rust-lang.org/nightly/intro.html</ref>

===Servo===
Servo is an experimental project to build a Web browser engine for a new generation of hardware: mobile devices, multi-core processors and high-performance GPUs. With Servo, we are rethinking the browser at every level of the technology stack — from input parsing to page layout to graphics rendering — to optimize for power efficiency and maximum parallelism.
Servo builds on top of Rust to provide a secure and reliable foundation. Memory safety at the core of the platform ensures a high degree of assurance in the browser’s trusted computing base. Rust’s lightweight task mechanism also promises to allow fine-grained isolation between browser components, such as tabs and extensions, without the need for expensive runtime protection schemes, like operating system process isolation.<ref>https://www.mozilla.org/en-US/research/projects/</ref>

=='''Background'''==
===Remote Developer Tools===

Firefox supports remote developer tools - ie. communicating with an arbitrary server that implements a protocol for exposing information about web content. You can use the [https://developer.mozilla.org/en-US/docs/Tools Firefox developer tools] on your desktop to debug Web sites and Web apps running in other browsers or runtimes. The other browser might be on the same device as the tools themselves or on a different device, such as a phone connected over USB.There are two types of runtime supported.a. Gecko-based runtimes- like Firefox Desktop, Firefox for Android, Firefox OS, and Thunderbird and b. Other runtimes- like Google Chrome Desktop, Chrome on Android, and Safari on iOS }}

==Build Process==

Following commands need to be entered for a normal build:-
**git clone https://github.com/servo/servo
**cd servo
**./mach build --dev
**./mach run tests/html/about-mozilla.html

==Run==

**./mach run [url] where [url] is the web address you want to run Servo for.

=='''Project Description'''==
Servo implements a very basic developer tools server that currently supports executing JS remotely and investigating the DOM tree in the document inspector. We want to expand these capabilities by completing previous work that enables remote logging from web content, and add new capabilities to log HTTP requests and responses to allow for easier debugging of network-related problems in Servo.

=='''Requirement Analysis'''==

Below are the requirements that will addressed in the final semester project.

**Implement the getRequestHeaders/getRequestPostData message handlers in components/devtools/actors/network_event.rs based on the values in theHttpReqest struct
**Add support for more flavours of the networkEventUpdate message than justresponseStart, by building the NetworkEventUpdateMsg` from a freeform JSON object instead of a serialized struct declaration
**Send real values for the fields of ResponseStartMsg (requires obtaining those values inhttp_loader.rs and sending them in send_response_to_devtools)
**Implement the getRequestCookies and getResponseCookies message handlers incomponents/devtools/actors/network_event.rs based on the Cookie/Set-Cookieheaders in the HttpRequest/HttpResponse structs
**Implement the getResponseContent/getResponseHeaders message handlers based on the data obtained in add_http_response
**Implement remaining flavours of networkEventUpdate message
**Address the TODO for event_actor in network_event.rs by providing the missing information as part of the original message transmitted by send_request_to_devtools

=='''Approach'''==

For task1 -we would use the ‘self’ variable (of type NetworkEvent) to extract the headers information.

For task2 – Modify handle_network_event such that NetworkEventUpdateMsg is created using a JSON object instead of a struct.(inside lib.rs)

For task 3 – Change response_start in network_event.rs file to include real values instead of the fake ones that is being sent till now.

For task 4 – Similar to task 1(located in the same file – network_event.rs), we will use the cookie information from the HttpRequest, HttpResponse structs.

For task 5 – Similar changes have to be made as in task 4 for this task.

For task 7 – This task will make use of the information added in the initial steps for our project.

=='''Design Patterns'''==
No design pattern required for this project.

=='''Reference'''==
<references/>
https://github.com/servo/servo/wiki/Expand-HTTP-request-response-monitoring
https://developer.mozilla.org/en-US/docs/Tools/Remote_Debugging
http://www-archive.mozilla.org/roadmap/app-architecture-diagram.png

CSC/ECE 517 Fall 2015/ExtendingDeveloperToolsforServo

2015-11-10T04:33:40Z

User:Abhina

2015-11-10T04:22:15Z

CSC/ECE 517 Fall 2015/oss/M1502/AAAASS

2015-11-07T03:58:44Z

Abhina: /* Adding pipeline Id */

=='''M1502: Improve HTTP monitoring devtool support'''==
Firefox supports remote developer tools - ie. communicating with an arbitrary server that implements a protocol for exposing information about web content. Servo implements a very basic developer tools server that currently supports executing JS remotely and investigating the DOM tree in the document inspector. We want to expand these capabilities by exposing further information about HTTP requests and responses that web content initiates.
<ref>https://github.com/servo/servo/wiki/Expand-HTTP-request-response-monitoring</ref>.

==''' Github Location''' ==
https://github.com/akumar21NCSU/servo

==''' Build Process''' ==
git clone https://github.com/servo/servo
cd servo
./mach build --dev
./mach run tests/html/about-mozilla.html

==''' Forked Branch''' ==
A branch was forked from https://github.com/servo/servo and placed at the following location: https://github.com/akumar21NCSU/servo

==''' Development''' ==
The primary goal of the development effort is to support the presence of a pipeline id.
The goal is to pass values to send_request_to_devtools derived from the actual request. The request containing the new pipeline id.
The development command is: ./mach build --dev

=='''Code Changes'''==

==='''Adding pipeline Id'''===

Pipeline id was added in http_loader.rs file for request and response messages.

<pre>
fn send_request_to_devtools(devtools_chan: Option<Sender<DevtoolsControlMsg>>,
request_id: String,
url: Url,
method: Method,
headers: Headers,
body: Option<Vec<u8>>,
pipeline_id: PipelineId) {

if let Some(ref chan) = devtools_chan {
let request = DevtoolsHttpRequest {
url: url, method: method, headers: headers, body: body, pipeline_id: pipeline_id };
let net_event = NetworkEvent::HttpRequest(request);

let msg = ChromeToDevtoolsControlMsg::NetworkEvent(request_id, net_event);
chan.send(DevtoolsControlMsg::FromChrome(msg)).unwrap();
}
}
</pre>
<pre>
fn send_response_to_devtools(devtools_chan: Option<Sender<DevtoolsControlMsg>>,
request_id: String,
headers: Option<Headers>,
status: Option<RawStatus>,
pipeline_id: PipelineId) {
if let Some(ref chan) = devtools_chan {
let response = DevtoolsHttpResponse { headers: headers, status: status, body: None, pipeline_id: pipeline_id };
let net_event_response = NetworkEvent::HttpResponse(response);

let msg = ChromeToDevtoolsControlMsg::NetworkEvent(request_id, net_event_response);
chan.send(DevtoolsControlMsg::FromChrome(msg)).unwrap();
}
}
</pre>
This required us to change HttpRequest and HttpResponse structs in lib.rs file inside devtool_traits.
<pre>
pub struct HttpRequest {
pub url: Url,
pub method: Method,
pub headers: Headers,
pub body: Option<Vec<u8>>,
pub pipeline_id: PipelineId,
}
</pre>

<pre>
pub struct HttpResponse {
pub headers: Option<Headers>,
pub status: Option<RawStatus>,
pub body: Option<Vec<u8>>,
pub pipeline_id: PipelineId,
}
</pre>

Changes were also made to the load_whole_resource function inside lib.rs for adding pipeline id

<pre>
pub fn load_whole_resource(resource_task: &ResourceTask, url: Url, pipeline_id: Option<PipelineId>)
-> Result<(Metadata, Vec<u8>), String> {
let (start_chan, start_port) = ipc::channel().unwrap();
resource_task.send(ControlMsg::Load(LoadData::new(url, pipeline_id),
LoadConsumer::Channel(start_chan))).unwrap();
let response = start_port.recv().unwrap();

let mut buf = vec!();
loop {
match response.progress_port.recv().unwrap() {
ProgressMsg::Payload(data) => buf.push_all(&data),
ProgressMsg::Done(Ok(())) => return Ok((response.metadata, buf)),
ProgressMsg::Done(Err(e)) => return Err(e)
}
}
}
</pre>

==='''Refactoring '''===

== '''Test '''==
A unit test is added to test the new functionality of the pipeline id.
The testing is run by performing the following command in the command line: ./mach test-wpt

==''' Implementation''' ==
Servo implements a very basic developer tools server that currently supports executing JS remotely and investigating the DOM tree in the document inspector. We want to expand these capabilities by exposing further information about HTTP requests and responses that web content initiates.
<ref>https://github.com/servo/servo/wiki/Expand-HTTP-request-response-monitoring</ref>

==''' Project Benefits & Challenges ''' ==
'''Benefits''':
1. Helps add additional debug to the HTTP monitoring in Servo
2. Provides developers capability of monitoring the data traffic
3. Mozilla messages are clearly analyzed using Servo
'''Challenges''':
Rust code development presents a challenge as the build environment takes time to build. Adding additional filters to the Servo causes failures initially as development tends to be a hit or miss. Communicating with both the TA and the Mozilla team members is critical.
It is generally expected that new changes should not introduce new crashes in the testsuite however this was not the case. The team did encounter crashes when unit tests were carried out.


== '''Design Pattern'''==
The requirements did not require us to implement any design pattern.

== '''External Links'''==
Also see:
[http://doc.servo.org/servo/index.html/ Development]
[https://github.com/servo/servo/wiki/Devtools/ Attaching Developer Tools]

== '''References''' ==
<references/>

CSC/ECE 517 Fall 2015/oss/M1502/AAAASS

2015-11-07T03:55:37Z

Abhina: /* Code Changes */

=='''M1502: Improve HTTP monitoring devtool support'''==
Firefox supports remote developer tools - ie. communicating with an arbitrary server that implements a protocol for exposing information about web content. Servo implements a very basic developer tools server that currently supports executing JS remotely and investigating the DOM tree in the document inspector. We want to expand these capabilities by exposing further information about HTTP requests and responses that web content initiates.
<ref>https://github.com/servo/servo/wiki/Expand-HTTP-request-response-monitoring</ref>.

==''' Github Location''' ==
https://github.com/akumar21NCSU/servo

==''' Build Process''' ==
git clone https://github.com/servo/servo
cd servo
./mach build --dev
./mach run tests/html/about-mozilla.html

==''' Forked Branch''' ==
A branch was forked from https://github.com/servo/servo and placed at the following location: https://github.com/akumar21NCSU/servo

==''' Development''' ==
The primary goal of the development effort is to support the presence of a pipeline id.
The goal is to pass values to send_request_to_devtools derived from the actual request. The request containing the new pipeline id.
The development command is: ./mach build --dev

=='''Code Changes'''==

==='''Adding pipeline Id'''===

Pipeline id was added in http_loader.rs file for request and response messages.

<pre>
fn send_request_to_devtools(devtools_chan: Option<Sender<DevtoolsControlMsg>>,
request_id: String,
url: Url,
method: Method,
headers: Headers,
body: Option<Vec<u8>>,
pipeline_id: PipelineId) {

if let Some(ref chan) = devtools_chan {
let request = DevtoolsHttpRequest {
url: url, method: method, headers: headers, body: body, pipeline_id: pipeline_id };
let net_event = NetworkEvent::HttpRequest(request);

let msg = ChromeToDevtoolsControlMsg::NetworkEvent(request_id, net_event);
chan.send(DevtoolsControlMsg::FromChrome(msg)).unwrap();
}
}
</pre>
<pre>
fn send_response_to_devtools(devtools_chan: Option<Sender<DevtoolsControlMsg>>,
request_id: String,
headers: Option<Headers>,
status: Option<RawStatus>,
pipeline_id: PipelineId) {
if let Some(ref chan) = devtools_chan {
let response = DevtoolsHttpResponse { headers: headers, status: status, body: None, pipeline_id: pipeline_id };
let net_event_response = NetworkEvent::HttpResponse(response);

let msg = ChromeToDevtoolsControlMsg::NetworkEvent(request_id, net_event_response);
chan.send(DevtoolsControlMsg::FromChrome(msg)).unwrap();
}
}
</pre>
This required us to change HttpRequest and HttpResponse structs in lib.rs file inside devtool_traits.
<pre>
pub struct HttpRequest {
pub url: Url,
pub method: Method,
pub headers: Headers,
pub body: Option<Vec<u8>>,
pub pipeline_id: PipelineId,
}
</pre>

<pre>
pub struct HttpResponse {
pub headers: Option<Headers>,
pub status: Option<RawStatus>,
pub body: Option<Vec<u8>>,
pub pipeline_id: PipelineId,
}
</pre>

==='''Refactoring '''===

== '''Test '''==
A unit test is added to test the new functionality of the pipeline id.
The testing is run by performing the following command in the command line: ./mach test-wpt

==''' Implementation''' ==
Servo implements a very basic developer tools server that currently supports executing JS remotely and investigating the DOM tree in the document inspector. We want to expand these capabilities by exposing further information about HTTP requests and responses that web content initiates.
<ref>https://github.com/servo/servo/wiki/Expand-HTTP-request-response-monitoring</ref>

==''' Project Benefits & Challenges ''' ==
'''Benefits''':
1. Helps add additional debug to the HTTP monitoring in Servo
2. Provides developers capability of monitoring the data traffic
3. Mozilla messages are clearly analyzed using Servo
'''Challenges''':
Rust code development presents a challenge as the build environment takes time to build. Adding additional filters to the Servo causes failures initially as development tends to be a hit or miss. Communicating with both the TA and the Mozilla team members is critical.
It is generally expected that new changes should not introduce new crashes in the testsuite however this was not the case. The team did encounter crashes when unit tests were carried out.


== '''Design Pattern'''==
The requirements did not require us to implement any design pattern.

== '''External Links'''==
Also see:
[http://doc.servo.org/servo/index.html/ Development]
[https://github.com/servo/servo/wiki/Devtools/ Attaching Developer Tools]

== '''References''' ==
<references/>

CSC/ECE 517 Fall 2015/oss/M1502/AAAASS

2015-11-07T03:49:07Z

Abhina:

=='''M1502: Improve HTTP monitoring devtool support'''==
Firefox supports remote developer tools - ie. communicating with an arbitrary server that implements a protocol for exposing information about web content. Servo implements a very basic developer tools server that currently supports executing JS remotely and investigating the DOM tree in the document inspector. We want to expand these capabilities by exposing further information about HTTP requests and responses that web content initiates.
<ref>https://github.com/servo/servo/wiki/Expand-HTTP-request-response-monitoring</ref>.

==''' Github Location''' ==
https://github.com/akumar21NCSU/servo

==''' Build Process''' ==
git clone https://github.com/servo/servo
cd servo
./mach build --dev
./mach run tests/html/about-mozilla.html

==''' Forked Branch''' ==
A branch was forked from https://github.com/servo/servo and placed at the following location: https://github.com/akumar21NCSU/servo

==''' Development''' ==
The primary goal of the development effort is to support the presence of a pipeline id.
The goal is to pass values to send_request_to_devtools derived from the actual request. The request containing the new pipeline id.
The development command is: ./mach build --dev

=='''Code Changes'''==

Pipeline id was added in http_loader.rs file for request and response messages.

<pre>
fn send_request_to_devtools(devtools_chan: Option<Sender<DevtoolsControlMsg>>,
request_id: String,
url: Url,
method: Method,
headers: Headers,
body: Option<Vec<u8>>,
pipeline_id: PipelineId) {

if let Some(ref chan) = devtools_chan {
let request = DevtoolsHttpRequest {
url: url, method: method, headers: headers, body: body, pipeline_id: pipeline_id };
let net_event = NetworkEvent::HttpRequest(request);

let msg = ChromeToDevtoolsControlMsg::NetworkEvent(request_id, net_event);
chan.send(DevtoolsControlMsg::FromChrome(msg)).unwrap();
}
}
</pre>
<pre>
fn send_response_to_devtools(devtools_chan: Option<Sender<DevtoolsControlMsg>>,
request_id: String,
headers: Option<Headers>,
status: Option<RawStatus>,
pipeline_id: PipelineId) {
if let Some(ref chan) = devtools_chan {
let response = DevtoolsHttpResponse { headers: headers, status: status, body: None, pipeline_id: pipeline_id };
let net_event_response = NetworkEvent::HttpResponse(response);

let msg = ChromeToDevtoolsControlMsg::NetworkEvent(request_id, net_event_response);
chan.send(DevtoolsControlMsg::FromChrome(msg)).unwrap();
}
}
</pre>
This required us to change HttpRequest and HttpResponse structs in lib.rs file inside devtool_traits.
<pre>
pub struct HttpRequest {
pub url: Url,
pub method: Method,
pub headers: Headers,
pub body: Option<Vec<u8>>,
pub pipeline_id: PipelineId,
}
</pre>

<pre>
pub struct HttpResponse {
pub headers: Option<Headers>,
pub status: Option<RawStatus>,
pub body: Option<Vec<u8>>,
pub pipeline_id: PipelineId,
}
</pre>

== '''Test '''==
A unit test is added to test the new functionality of the pipeline id.
The testing is run by performing the following command in the command line: ./mach test-wpt

==''' Implementation''' ==
Servo implements a very basic developer tools server that currently supports executing JS remotely and investigating the DOM tree in the document inspector. We want to expand these capabilities by exposing further information about HTTP requests and responses that web content initiates.
<ref>https://github.com/servo/servo/wiki/Expand-HTTP-request-response-monitoring</ref>

==''' Project Benefits & Challenges ''' ==
'''Benefits''':
1. Helps add additional debug to the HTTP monitoring in Servo
2. Provides developers capability of monitoring the data traffic
3. Mozilla messages are clearly analyzed using Servo
'''Challenges''':
Rust code development presents a challenge as the build environment takes time to build. Adding additional filters to the Servo causes failures initially as development tends to be a hit or miss. Communicating with both the TA and the Mozilla team members is critical.
It is generally expected that new changes should not introduce new crashes in the testsuite however this was not the case. The team did encounter crashes when unit tests were carried out.


== '''Design Pattern'''==
The requirements did not require us to implement any design pattern.

== '''External Links'''==
Also see:
[http://doc.servo.org/servo/index.html/ Development]
[https://github.com/servo/servo/wiki/Devtools/ Attaching Developer Tools]

== '''References''' ==
<references/>

CSC/ECE 517 Fall 2015/oss/M1502/AAAASS

2015-11-07T03:40:35Z

Abhina:

=='''M1502: Improve HTTP monitoring devtool support'''==
Firefox supports remote developer tools - ie. communicating with an arbitrary server that implements a protocol for exposing information about web content. Servo implements a very basic developer tools server that currently supports executing JS remotely and investigating the DOM tree in the document inspector. We want to expand these capabilities by exposing further information about HTTP requests and responses that web content initiates.
<ref>https://github.com/servo/servo/wiki/Expand-HTTP-request-response-monitoring</ref>.

==''' Github Location''' ==
https://github.com/akumar21NCSU/servo

==''' Build Process''' ==
git clone https://github.com/servo/servo
cd servo
./mach build --dev
./mach run tests/html/about-mozilla.html

==''' Forked Branch''' ==
A branch was forked from https://github.com/servo/servo and placed at the following location: https://github.com/akumar21NCSU/servo

==''' Development''' ==
The primary goal of the development effort is to support the presence of a pipeline id.
The goal is to pass values to send_request_to_devtools derived from the actual request. The request containing the new pipeline id.
The development command is: ./mach build --dev

=='''Code Changes'''==

Pipeline id was added in http_loader.rs file for request and response messages.

fn send_request_to_devtools(devtools_chan: Option<Sender<DevtoolsControlMsg>>,
request_id: String,
url: Url,
method: Method,
headers: Headers,
body: Option<Vec<u8>>,
pipeline_id: PipelineId) {

if let Some(ref chan) = devtools_chan {
let request = DevtoolsHttpRequest {
url: url, method: method, headers: headers, body: body, pipeline_id: pipeline_id };
let net_event = NetworkEvent::HttpRequest(request);

let msg = ChromeToDevtoolsControlMsg::NetworkEvent(request_id, net_event);
chan.send(DevtoolsControlMsg::FromChrome(msg)).unwrap();
}
}

fn send_response_to_devtools(devtools_chan: Option<Sender<DevtoolsControlMsg>>,
request_id: String,
headers: Option<Headers>,
status: Option<RawStatus>,
pipeline_id: PipelineId) {
if let Some(ref chan) = devtools_chan {
let response = DevtoolsHttpResponse { headers: headers, status: status, body: None, pipeline_id: pipeline_id };
let net_event_response = NetworkEvent::HttpResponse(response);

let msg = ChromeToDevtoolsControlMsg::NetworkEvent(request_id, net_event_response);
chan.send(DevtoolsControlMsg::FromChrome(msg)).unwrap();
}
}

This required us to change HttpRequest and HttpResponse structs in lib.rs file inside devtool_traits.

pub struct HttpRequest {
pub url: Url,
pub method: Method,
pub headers: Headers,
pub body: Option<Vec<u8>>,
pub pipeline_id: PipelineId,
}

#[derive(Debug, PartialEq)]
pub struct HttpResponse {
pub headers: Option<Headers>,
pub status: Option<RawStatus>,
pub body: Option<Vec<u8>>,
pub pipeline_id: PipelineId,
}

== '''Test '''==
A unit test is added to test the new functionality of the pipeline id.
The testing is run by performing the following command in the command line: ./mach test-wpt

==''' Implementation''' ==
Servo implements a very basic developer tools server that currently supports executing JS remotely and investigating the DOM tree in the document inspector. We want to expand these capabilities by exposing further information about HTTP requests and responses that web content initiates.
<ref>https://github.com/servo/servo/wiki/Expand-HTTP-request-response-monitoring</ref>

==''' Project Benefits & Challenges ''' ==
'''Benefits''':
1. Helps add additional debug to the HTTP monitoring in Servo
2. Provides developers capability of monitoring the data traffic
3. Mozilla messages are clearly analyzed using Servo
'''Challenges''':
Rust code development presents a challenge as the build environment takes time to build. Adding additional filters to the Servo causes failures initially as development tends to be a hit or miss. Communicating with both the TA and the Mozilla team members is critical.
It is generally expected that new changes should not introduce new crashes in the testsuite however this was not the case. The team did encounter crashes when unit tests were carried out.


== '''Design Pattern'''==
The requirements did not require us to implement any design pattern.

== '''External Links'''==
Also see:
[http://doc.servo.org/servo/index.html/ Development]
[https://github.com/servo/servo/wiki/Devtools/ Attaching Developer Tools]

== '''References''' ==
<references/>

CSC/ECE 517 Fall 2015/oss/M1502/AAAASS

2015-11-07T03:22:33Z

Abhina: