M1606: Implementing HTTP authorization UI and persistent sessions: Difference between revisions

From Expertiza_Wiki
Jump to navigation Jump to search
 
(24 intermediate revisions by 2 users not shown)
Line 3: Line 3:
[https://en.wikipedia.org/wiki/Web_page Web pages] make use of [https://en.wikipedia.org/wiki/Session_(computer_science) sessions] all the time. These are required to persist throughout the life of the user's interaction with the web site or web application.This is supported by Servo as of today. However it doesn't support the persistence of data upon the closing and reopening of the web browser.<ref> https://github.com/servo/servo/wiki/Persistent-sessions-student-project</ref>
[https://en.wikipedia.org/wiki/Web_page Web pages] make use of [https://en.wikipedia.org/wiki/Session_(computer_science) sessions] all the time. These are required to persist throughout the life of the user's interaction with the web site or web application.This is supported by Servo as of today. However it doesn't support the persistence of data upon the closing and reopening of the web browser.<ref> https://github.com/servo/servo/wiki/Persistent-sessions-student-project</ref>


With this project we aim to create the necessary infrastructure to support this.
With this project we have created the necessary infrastructure to support this.


== '''Introduction''' ==
== '''Introduction''' ==
Line 25: Line 25:
1) Added a new command line flag <code> --profile-dir [path]</code> that stores an optional directory path in the Opts struct in <code>opts.rs</code>, creates the directory if it does not exist.
1) Added a new command line flag <code> --profile-dir [path]</code> that stores an optional directory path in the Opts struct in <code>opts.rs</code>, creates the directory if it does not exist.


[[File:Step2Wiki2spring16.jpg]]


2) Created [http://doc.rust-lang.org/book/ffi.html  Rust FFI bindings] for the <code>[https://github.com/jdm/tinyfiledialogs tinyfiledialogs] </code>library to allow calling the C methods from Servo.
2) Created [http://doc.rust-lang.org/book/ffi.html  Rust FFI bindings] for the <code>[https://github.com/jdm/tinyfiledialogs tinyfiledialogs] </code>library to allow calling the C methods from Servo.


[[File:Step3-1Wiki2spring16.jpg]]
[[File:Step3-2Wiki2spring16.jpg]]


3) In <code>resource_thread.rs</code>, defined an HTTP authorization cache storage (username, password, URL) and instantiated it like the cookie_storage member (inside an <code>Arc<Rwlock<>></code> value, to enable sharing it between threads).
3) In <code>resource_thread.rs</code>, defined an HTTP authorization cache storage (username, password, URL) and instantiated it like the cookie_storage member (inside an <code>Arc<Rwlock<>></code> value, to enable sharing it between threads).


[[File:Step4Wiki2spring16.jpg]]


4) In <code>modify_request_headers</code> in <code>http_loader.rs</code>, implemented the remaining pieces of step 12 of the [https://fetch.spec.whatwg.org/#http-network-or-cache-fetch appropriate specification ] using the authorization cache.
4) In <code>modify_request_headers</code> in <code>http_loader.rs</code>, implemented the remaining pieces of step 12 of the [https://fetch.spec.whatwg.org/#http-network-or-cache-fetch appropriate specification ] using the authorization cache.


[[File:Step5Wiki2spring16.jpg]]
''' The Pull Requests for these steps can be found [http://wiki.expertiza.ncsu.edu/index.php/M1606:_Implementing_HTTP_authorization_UI_and_persistent_sessions#Initial_Steps_Pull_Requests here]'''


== '''Scope''' ==
== '''Scope''' ==
Line 58: Line 53:


== '''Implementation / Design Decisions''' ==
== '''Implementation / Design Decisions''' ==
Here are some of the design decisions that we have made for implementing the steps mentioned in [http://wiki.expertiza.ncsu.edu/index.php/M1606:_Implementing_HTTP_authorization_UI_and_persistent_sessions#Scope Scope] :


=== Step 1 ===
=== Step 1 ===
Line 65: Line 58:
Objective: Make an authorization UI appear when a 401 HTTP response is received  
Objective: Make an authorization UI appear when a 401 HTTP response is received  


Using the tinyfiledialogs RUST FFI bindings that the team created in the initial steps we will create a UI for the user to enter his username and password when a 401 request is received for a basic HTTP authentication.  We will first check if the status code of the http response is "unauthorized". If the code is unauthorized we will display the UI for the user to enter his username and password.  After the username and password are entered the request is restarted with the authentication header added.  If the request is now successful we create a new authentication entry struct and place it into the Authentication hash map for future use.
Using the tinyfiledialogs RUST FFI bindings that the team created in the initial steps we created a UI for the user to enter his username and password when a 401 request is received for a basic HTTP authentication.  We then first check if the status code of the http response is "unauthorized". If the code is unauthorized we then display the UI for the user to enter his username and password.  After the username and password are entered the request is restarted with the authentication header added.  If the request is successful we create a new authentication entry struct and place it into the Authentication hash map for future use.
 
The UML Diagram for this implementation can be found [http://wiki.expertiza.ncsu.edu/index.php/M1606:_Implementing_HTTP_authorization_UI_and_persistent_sessions#UML_for_step_1 here]


=== Step 2 ===
=== Step 2 ===
Line 71: Line 66:
Objective: On shutdown serialize the data contained in cookie_storage, hsts_list, and the new HTTP authorization cache, and write the serialized data in separate files inside the profile directory.
Objective: On shutdown serialize the data contained in cookie_storage, hsts_list, and the new HTTP authorization cache, and write the serialized data in separate files inside the profile directory.


When a Resource Thread exits we will check if the opts structure located in opts.rs contains an entry for the command line option "--profile dir".  If the option is present when the Resource Thread exits we will open or create a file for each data set we are saving.  After each file is opened or created we will serialize the data that corresponds to that file with the rust serialize library and save the serialized data to the file.
When a Resource Thread exits we check if the opts structure located in opts.rs contains an entry for the command line option "--profile dir".  If the option is present when the Resource Thread exits we open or create a file for each data set we are saving.  After each file is opened or created we serialize the data that corresponds to that file with the rust serialize library and save the serialized data to the file.
 
The UML Diagram for this implementation can be found [http://wiki.expertiza.ncsu.edu/index.php/M1606:_Implementing_HTTP_authorization_UI_and_persistent_sessions#UML_for_step_2 here]


=== Step 3 ===
=== Step 3 ===
Line 77: Line 74:
Objective: On shutdown serialize the local_data in storage_thread.r
Objective: On shutdown serialize the local_data in storage_thread.r


This is similar to step 2 and some code re-use will be used.  However, the difference between this step and step 2 is that the local data contained in storage_thread.rs will be the data that is being serialized and saved into the same directory.
This is similar to step 2 and some code re-use is used.  However, the difference between this step and step 2 is that the local data contained in storage_thread.rs is the data that is being serialized and saved into the same directory.


=== Step 4 ===
=== Step 4 ===
Line 83: Line 80:
Objective: If the command line option for profile-dir is present then load the serialized data present in the directory that is given.
Objective: If the command line option for profile-dir is present then load the serialized data present in the directory that is given.


When a resource thread is created we will check if the opts structure located in opts.rs contains a entry for the command line option "--profile dir".  If the option is present then the data from each file contained in the directory will be opened and deserialized using the serialize rust library. The data will then be placed in the appropriate fields. i.e (cookie_storage, hsts_lists, authorization_cache)
When a resource thread is created we check if the opts structure located in opts.rs contains a entry for the command line option "--profile dir".  If the option is present then the data from each file contained in the directory is opened and deserialized using the serialize rust library. The data is then be placed in the appropriate fields. i.e (cookie_storage, hsts_lists, authorization_cache)
 
The UML Diagram for this implementation can be found [http://wiki.expertiza.ncsu.edu/index.php/M1606:_Implementing_HTTP_authorization_UI_and_persistent_sessions#UML_for_step_4 here]
 
'''The Pull Requests for these steps can be found [http://wiki.expertiza.ncsu.edu/index.php/M1606:_Implementing_HTTP_authorization_UI_and_persistent_sessions#Subsequent_Steps_Pull_Requests here]'''


=='''UML Diagrams'''==
=='''UML Diagrams'''==
Line 99: Line 100:
==='''UML for step 4'''===
==='''UML for step 4'''===


[[File:517project.png ]]
[[File:STEP4.png ]]
 
== '''Pull Requests''' ==
All pull requests that fully implement the initial steps and the subsequent steps have been accepted by a servo maintainer and successfully merged to the servo master branch
 
==='''Initial Steps Pull Requests'''===
 
These are the pull requests that complete the initial steps of the project:
 
1)    <span class="plainlinks">[https://github.com/servo/servo/pull/10087 Added a new command line flag --profile-dir ]</span>
 
2) <span class="plainlinks">[https://github.com/jdm/tinyfiledialogs/pull/1 Added Rust FFI bindings to lib.rs ]</span>
 
3) <span class="plainlinks">[https://github.com/servo/servo/pull/10111 Added authorization cache to resources & set authorization header from it if url does not have credentials ]</span>
 
==='''Subsequent Steps Pull Requests'''===
 
These are the pull requests that complete the Final steps of the project:
 
1)    <span class="plainlinks">[https://github.com/servo/servo/pull/10328 401 authorization UI]</span>
 
2) <span class="plainlinks">[https://github.com/servo/servo/pull/10661 Writing the serialized data in separate files inside the profile directory]</span>
 
3) <span class="plainlinks">[https://github.com/servo/servo/pull/10800 Decoding the data and populating the appropriate fields]</span>


=='''Testing'''==
=='''Testing'''==


===Testing the new command line flag --profile-dir===
==='''Testing the new command line flag --profile-dir'''===
 
    git clone https://github.com/servo/servo.git
 
    cd servo
 
    ./mach build --release
 
    ./mach run tests/html/about-mozilla.html --profile-dir [directory name]
 
    //A new folder should be created if it does not exist with the directory name
 
=== '''Unit Test for set authorization header if url does not have credentials''' ===
The project maintainer instructed us to add a unit test to test the initial step functionality.  We have added a unit test to test that the authentication header is being set if credentials are contained in the cache
 
It can be run with the other unit tests with the command
 
    ./mach test-unit -p net
 
The unit test can be found in /servo/tests/unit/net/http_loader.rs
 
=== '''GUI test for the tinyfiledialogs lib'''===
In addition to creating the rust FFI bindings for the tinyfiledialog C library our team also created a example rust program to test the new rust library.
 
To run the example and see all of the tinyfiledialog windows do the following.
 
    Git clone https://github.com/jdm/tinyfiledialogs.git
 
    cd tinyfiledialogs/examples/tinytest
 
    cargo run
 
=== '''Unit Test for set authorization header if authorization credentials entered by user''' ===
The project maintainer instructed us to add a unit test that tests if a user entered the correct credentials on a 401 response that the correct basic authorization header is set.
 
It can be run with the other unit tests with the command
 
    ./mach test-unit -p net
 
==='''Gui testing for the entire project scope'''===
 
    $ git clone https://github.com/servo/servo.git
 
    $ cd servo
 
    $ ./mach build --release
 
    $ ./mach run --release http://browserspy.dk/password-ok.php --profile-dir profile_dir
 
    enter 'test' for username and enter 'test' for password
 
    //wait for servo to load successful response and then exit (make sure it is completely loaded before exiting or it may crash)
 
    //A new folder should be created if it does not exist with
    //the directory profile_dir that contains the credentials entered in json format
 
    //now run servo again with the same arguments to see that the auth cache was written to file and then loaded from file
 
    $ ./mach run --release http://browserspy.dk/password-ok.php --profile-dir profile_dir


Steps for Testing the new command line flag --profile-dir can be found [http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Spring_2016_OSS_M1606#Testing_the_new_command_line_flag_--profile-dir here]
    //The page should load with a success without needing any username or password this time


=== Unit Test for set authorization header if url does not have credentials ===
    //you can try other urls that contain the other data ( cookies, hsts_list, local_data ) and
    //close servo and look at the data written to the file (now all pages contain all data)


Steps for Unit Tests for set authorization header if url does not have credentials can be found [http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Spring_2016_OSS_M1606#Unit_Test_for_set_authorization_header_if_url_does_not_have_credentials here]
=='''Conclusion'''==


=== GUI test for the tinyfiledialogs lib ===
We have successfully completed all the steps mentioned in the [https://github.com/servo/servo/wiki/Persistent-sessions-student-project NCSU Student Projects Page]. All the Pull requests have been accepted and merged with the servo master branch.


Steps for GUI test for the tinyfiledialogs lib can be found  [http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Spring_2016_OSS_M1606#GUI_test_for_the_tinyfiledialogs_lib here ]
We have also created a [https://youtu.be/Lm-NZDOSK0Q Video] explaining our entire project. Select 720p quality in settings if you can.


=='''References'''==
=='''References'''==
<references/>
<references/>

Latest revision as of 21:38, 26 April 2016

M1606: Implementing HTTP authorization UI and persistent sessions

Web pages make use of sessions all the time. These are required to persist throughout the life of the user's interaction with the web site or web application.This is supported by Servo as of today. However it doesn't support the persistence of data upon the closing and reopening of the web browser.<ref> https://github.com/servo/servo/wiki/Persistent-sessions-student-project</ref>

With this project we have created the necessary infrastructure to support this.

Introduction

Servo

Servo is a web browser layout engine written in Rust and is currently being developed by Mozilla Research. The aim of the project is not to create a full browser but is rather to create a highly parallel environment that allows for many components be handled by fine-grained, isolated tasks. Servo is built on top of Rust to provide a secure and reliable foundation and is focused on creating a reliable and fast browser engine.<ref>https://en.wikipedia.org/wiki/Servo_(layout_engine)</ref>

Rust

Rust is a multi-paradigm, compiled programming language that is a good language for creating highly safe systems. Rust and Servo have a symbiotic relationship as the development of servo has influenced the design of the language. Rust is a modern, fast, memory safe and multithreaded programming language that focuses on speed and safety for developing reliable and efficient systems. It eliminates all data races by having numerous compile-time safety checks that adds no runtime overhead.<ref>https://en.wikipedia.org/wiki/Rust_(programming_language)</ref>

Previous Related Work

A Wiki Page regarding this work can be found here

1) Added a new command line flag --profile-dir [path] that stores an optional directory path in the Opts struct in opts.rs, creates the directory if it does not exist.


2) Created Rust FFI bindings for the tinyfiledialogs library to allow calling the C methods from Servo.


3) In resource_thread.rs, defined an HTTP authorization cache storage (username, password, URL) and instantiated it like the cookie_storage member (inside an Arc<Rwlock<>> value, to enable sharing it between threads).


4) In modify_request_headers in http_loader.rs, implemented the remaining pieces of step 12 of the appropriate specification using the authorization cache.

The Pull Requests for these steps can be found here

Scope

The scope of the project comprises of the 4 steps listed below:

1) Ensure the appearance of an authorization UI in case a 401 HTTP response is received. In load in the http_loader.rs, immediately before attempting to process an HTTP redirection, the new tinyfiledialogs is made use of to make two prompts to pop up (for username and password), subsequently the request is restarted and the new authorization value present applied. If an authorization value was indeed present and the response is successful, the credentials are added to the authorization cache.

2) If the command-line option of the profile directory is present when the ResourceThread is instructed to exit, the data contained in the cookie_storage, hsts_list, and the new HTTP authorization cache is serialized. This serialized data is then written in separate files inside the profile directory.

3) Perform similar serialization upon shutdown for local_data in storage_thread.rs, which represents the LocalStorage API.

4) Check if the command-line option is present in the profile directory in the ResourceThreadconstructor and look for files that will contain serialized versions of the previous steps. If such files do exist, populate the appropriate fields with deserialized versions of the file’s contents.

Design Patterns

The team was not given much freedom for design patterns. We followed and used the existing servo design strategy. You can read about the servo design here

We do use a facade pattern by creating a safe RUST wrapper around the RUST FFI for the tinyfiledialogs library that we will use for the UI. We could have just called the FFI functions from servo, but instead we created a safe and simple interface to call the library functions.

Implementation / Design Decisions

Step 1

Objective: Make an authorization UI appear when a 401 HTTP response is received

Using the tinyfiledialogs RUST FFI bindings that the team created in the initial steps we created a UI for the user to enter his username and password when a 401 request is received for a basic HTTP authentication. We then first check if the status code of the http response is "unauthorized". If the code is unauthorized we then display the UI for the user to enter his username and password. After the username and password are entered the request is restarted with the authentication header added. If the request is successful we create a new authentication entry struct and place it into the Authentication hash map for future use.

The UML Diagram for this implementation can be found here

Step 2

Objective: On shutdown serialize the data contained in cookie_storage, hsts_list, and the new HTTP authorization cache, and write the serialized data in separate files inside the profile directory.

When a Resource Thread exits we check if the opts structure located in opts.rs contains an entry for the command line option "--profile dir". If the option is present when the Resource Thread exits we open or create a file for each data set we are saving. After each file is opened or created we serialize the data that corresponds to that file with the rust serialize library and save the serialized data to the file.

The UML Diagram for this implementation can be found here

Step 3

Objective: On shutdown serialize the local_data in storage_thread.r

This is similar to step 2 and some code re-use is used. However, the difference between this step and step 2 is that the local data contained in storage_thread.rs is the data that is being serialized and saved into the same directory.

Step 4

Objective: If the command line option for profile-dir is present then load the serialized data present in the directory that is given.

When a resource thread is created we check if the opts structure located in opts.rs contains a entry for the command line option "--profile dir". If the option is present then the data from each file contained in the directory is opened and deserialized using the serialize rust library. The data is then be placed in the appropriate fields. i.e (cookie_storage, hsts_lists, authorization_cache)

The UML Diagram for this implementation can be found here

The Pull Requests for these steps can be found here

UML Diagrams

UML for step 1


UML for step 2


UML for step 4

Pull Requests

All pull requests that fully implement the initial steps and the subsequent steps have been accepted by a servo maintainer and successfully merged to the servo master branch

Initial Steps Pull Requests

These are the pull requests that complete the initial steps of the project:

1) Added a new command line flag --profile-dir

2) Added Rust FFI bindings to lib.rs

3) Added authorization cache to resources & set authorization header from it if url does not have credentials

Subsequent Steps Pull Requests

These are the pull requests that complete the Final steps of the project:

1) 401 authorization UI

2) Writing the serialized data in separate files inside the profile directory

3) Decoding the data and populating the appropriate fields

Testing

Testing the new command line flag --profile-dir

   git clone https://github.com/servo/servo.git
   cd servo
   ./mach build --release
   ./mach run tests/html/about-mozilla.html --profile-dir [directory name]
   //A new folder should be created if it does not exist with the directory name

Unit Test for set authorization header if url does not have credentials

The project maintainer instructed us to add a unit test to test the initial step functionality. We have added a unit test to test that the authentication header is being set if credentials are contained in the cache

It can be run with the other unit tests with the command

   ./mach test-unit -p net

The unit test can be found in /servo/tests/unit/net/http_loader.rs

GUI test for the tinyfiledialogs lib

In addition to creating the rust FFI bindings for the tinyfiledialog C library our team also created a example rust program to test the new rust library.

To run the example and see all of the tinyfiledialog windows do the following.

   Git clone https://github.com/jdm/tinyfiledialogs.git
   cd tinyfiledialogs/examples/tinytest
   cargo run

Unit Test for set authorization header if authorization credentials entered by user

The project maintainer instructed us to add a unit test that tests if a user entered the correct credentials on a 401 response that the correct basic authorization header is set.

It can be run with the other unit tests with the command

   ./mach test-unit -p net

Gui testing for the entire project scope

   $ git clone https://github.com/servo/servo.git
   $ cd servo
   $ ./mach build --release
   $ ./mach run --release http://browserspy.dk/password-ok.php --profile-dir profile_dir
   enter 'test' for username and enter 'test' for password
   //wait for servo to load successful response and then exit (make sure it is completely loaded before exiting or it may crash)
   //A new folder should be created if it does not exist with 
   //the directory profile_dir that contains the credentials entered in json format
   //now run servo again with the same arguments to see that the auth cache was written to file and then loaded from file
   $ ./mach run --release http://browserspy.dk/password-ok.php --profile-dir profile_dir
   //The page should load with a success without needing any username or password this time
   //you can try other urls that contain the other data ( cookies, hsts_list, local_data ) and 
   //close servo and look at the data written to the file (now all pages contain all data)

Conclusion

We have successfully completed all the steps mentioned in the NCSU Student Projects Page. All the Pull requests have been accepted and merged with the servo master branch.

We have also created a Video explaining our entire project. Select 720p quality in settings if you can.

References

<references/>