CSC/ECE 517 Fall 2015 M1504 Implement support for missing XMLHttpRequest APIs

Introduction

Servo

Servo <ref> https://github.com/servo/servo </ref> is a web browser layout engine written in Rust<ref>https://github.com/rust-lang/rust</ref> and is currently being developed by Mozilla Research. The aim of the project is not to create a full browser but is rather to create a highly parallel environment that allows for many components be handled by fine-grained, isolated tasks.<ref>https://en.wikipedia.org/wiki/Servo_(layout_engine)</ref>

Servo is built on top of Rust to provide a secure and reliable foundation and is focused on creating a reliable and fast browser engine.

Rust

Rust is a multi-paradigm, compiled programming language that is a good language for creating highly safe systems. Rust and Servo have a symbiotic relationship as the development of servo has influenced the design of the language.

Rust is a modern, fast, memory safe and multithreaded programming language that focuses on speed and safety for developing reliable and efficient systems. It eliminates all data races by having numerous compile-time safety checks that adds no runtime overhead.<ref> http://doc.rust-lang.org/nightly/book/README.html</ref>

Servo task Architecture

<ref group="image">https://github.com/servo/servo/wiki/Design</ref>

Description

• Each box corresponds to a Rust task.
• The primary tasks in the browser pipeline are represented by Blue boxes.
• Dashed lines indicate supervisor relationships.
• Gray boxes indicate tasks auxiliary to the browser pipeline.
• Communication channels are represented as solid lines.
• White boxes are represented as worker tasks. Each such box represents several tasks which varies with the workload.<ref>https://github.com/servo/servo/wiki/Design</ref>

Background

XMLHttpRequest

"XMLHttpRequest is a specification which defines APIs that provides scripted client functionality for transferring data between a client and a server."<ref>https://xhr.spec.whatwg.org/</ref> XMLHttpRequest provides a way for data to be retrieved from a URL without having to retrieve the entire page. It supports protocols other than HTTP and can be used to retrieve any type of data.<ref>https://developer.mozilla.org/en-US/docs/Web/API/XMLHttpRequest</ref>

Some of the APIs and properties present in XMLHttpRequest specification are overrideMimeType() , responseXML and withCredentials. The following javascript code snippets briefly explain about each of these APIs.

overrideMimeType<ref>https://github.com/servo/servo/blob/master/tests/wpt/web-platform-tests/XMLHttpRequest/overridemimetype-open-state-force-xml.htm</ref>

 
//override the Mime type to retrieve the response as an XML Object


 var client = new XMLHttpRequest();
 client.onreadystatechange = function() {
          ... //read the response as XML object
        };
        client.open("GET", "resources/status.py?type="
                    +encodeURIComponent('text/plain;charset=Shift-JIS')
                    +'&content='+ encodeURIComponent('<test><message>Hello World！</message></test>'));
        client.overrideMimeType('application/xml;charset=UTF-8');
        client.send();

responseXML property<ref>https://msdn.microsoft.com/en-us/library/ms534370(v=vs.85).aspx</ref>

//read the response from the server as an XML Object

var oReq = new XMLHttpRequest();
oReq.open("GET", "http://localhost/books.xml", false);
oReq.send();
console.log(oReq.responseXML.xml);

withCredentials property<ref>http://arunranga.com/examples/access-control/credentialedRequest.html</ref>

//create cross site requests using cookies as credentials
 
var invocation = new XMLHttpRequest();
var url = 'http://crossoriginurl.com/resources/access-control-with-credentials/';
invocation.open('GET', url, true);
invocation.withCredentials = "true";
invocation.onreadystatechange = handler;
invocation.send(); 

The above mentioned APIs and properties for XMLHttpRequest are currently unimplemented in servo.

Project Description

Our project aim is to implement few of the missing XMLHttpRequest APIs.<ref>https://github.com/servo/servo/wiki/Implement-support-for-missing-XMLHttpRequest-APIs</ref> The OSS project included implementing initial steps of the specifications. It basically incorporated implementing overrideMimeType method and adjusting related test expectations.

The implementation of the final project is described in a section below.

Requirement Analysis

Below is an overview of the various steps identified as a part of the requirement analysis and corresponding places in code in which these steps would be implemented:

XMLHttpRequest Interface

• We have to uncomment the responseXML API
• A webpage can make use of this API to read the document response

XMLHttpRequest DOM

1. Create helper methods final_mime_type and final_charset
   • Implement the two methods as per the behavior mentioned in the specification
2. Implement correct behavior for the text_response method
   • Make use of the final_charset and final_mime_type helper methods to determine the correct encoding for the response and decode the response using this encoding
3. Create and implement a new method "document_response" according to the xhr specification
4. Implement the correct behavior for the responseXML API to handle exceptions and return the document response using the method implemented above <ref>https://xhr.spec.whatwg.org/#dom-xmlhttprequest-responsexml</ref>
5. Implement the withCredentials APIs (setter and getter) as per the specification
6. Add support to the send() and open() API to test the value of the withCredentials attribute where necessary

net_traits

• Add a new member to the LoadData Structure in lib.rs file which will be used by the http_loader to conditionally exclude cookies from the XHR request based on the value of the member.

XHR Tests

• We need to pass most tests related to overrideMimeType, responseXML and withCredentials present in the XHR test-suite

Component Flow Diagram

The XMLHttpRequest API receives request from the web server, the implementation then checks the response type, if it is a text response then we use the charset from final_charset else we use the default charset that was received with the request. The same applies to document response.

Note :- UML diagram is not included because we do not have much class interaction in this project.

Design Patterns

We have used "State Pattern" for our final project implementation. Our implementation stores the new CHARSET and MIME type based on what user has requested. So once user has requested for override in this value, the design pattern ensures the system behaves as per this new value.

Implementation Details

The implementation for the final project involved following tasks:

• Implement the specified behavior for the override MIME type/final MIME type and override charset/final charset : We created two helper methods in xmlhttprequest.rs file: final_mime_type() and final_charset()
  • final_mime_type() method returns the final MIME type<ref>https://xhr.spec.whatwg.org/#final-mime-type</ref> depending on override MIME type and response MIME type
  • final_charset() method returns the final charset<ref>https://xhr.spec.whatwg.org/#final-charset</ref> depending on override charset and response charset
  • We have also determined encoding of text_response by calling the final_charset() helper method. But if final charset is not set (i.e. it is None) then UTF-8 is set as the default encoding. The step 4 mentioned in the text response documentation in order to add support for XML encoding guess stuff using XML spec was not to be implemented and hence FIXME was added in the code.

• Support the withCredentials API : We have implemented basic network level support to conditionally exclude cookies from the HTTP request by adding a boolean member: credentials_flag to LoadData structure in lib.rs file.
  • The default value of credentials_flag is set to true. credentials_flag is assigned value for cross-origin requests in Send() method of xmlhttprequest.rs file using the withCredentials attribute of XMLHttpRequest.
  • If the credentials_flag is false then cookies will not be set from the cookie jar for the HTTP request for which the code is added in http_loader.rs file. Also if the credentials_flag is false, then the cookies are not set in the cookie jar from HTTP response. As suggested by the Servo focal for this project, we have not implemented the first sub-task for withCredentials API.

• Support for document response in responseXML API : Added Document to the list of response type checks. New methods were created which contain handling for different mime types. A document object is created and its JS value is returned. The document object is stored in response_xml member of XHR, on subsequent calls to the method this object is returned if it has already been set previously.<ref>https://xhr.spec.whatwg.org/#response-document-object</ref>.For mime type text/html if any of the charset is NONE the requirement was to scan the first 1024 bytes of the header to identify the charset, this part has not been implemented in this project. The following new methods were added to file XMLHttprequest.rs as part of this task -
  • document_response - This method creates a document as per mime type returned by the final_mime_type() method created as part of OSS project. It internally calls other methods listed below.
  • document_text_html - This method creates a document flagged as an HTML document. This method is called by document_response in case the mime type is text/html. It calls the new_doc helper method.
  • handle_xml - This method creates a document flagged as a non HTML document. This method is called by document_response in case the mime type subtype is XML. It calls the new_doc helper method.
  • new_doc - This method creates a new document which is the final response based on the input received from document_text_response and handle_xml methods.

Test Cases

The tests have already been written for the XMLHttpRequest(XHR) APIs related to our project. So the test cases that pass after our changes to the code are as follows:

• If XHR is in unsent state, then overrideMimeType() is able to enforce Shift-JIS charset encoding (overridemimetype-unsent-state-force-shiftjis.htm)
• If XHR is in open state, then overrideMimeType() is able to enforce UTF-8 charset encoding (overridemimetype-open-state-force-utf-8.htm)
• If XHR with XML MIME type is in open state, then overrideMimeType() is able to enforce UTF-8 charset encoding (overridemimetype-open-state-force-xml.htm)
• If XHR is in HEADERS RECEIVED state, then overrideMimeType() is able to enforce Shift-JIS charset encoding (overridemimetype-headers-received-state-force-shiftjis.htm)
• The various responseXML document properties like title, contentType, doctype, cookie, etc. are correctly initialized (responsexml-document-properties.htm)
• In XHR, parsing of different responseXML MIME type is successful (responsexml-media-type.htm)
• In XHR, parsing of document received from responseXML is successful (send-redirect-no-location.htm, status-async.htm, status-basic.htm)
• In XHR, throw InvalidStateError if response type is not null or document (responsexml-non-document-types.htm)

Appendix

• Steps for installing rust
• Building Servo

References

<references/>

Images

<references group="image"/>