CSC/ECE 517 Fall 2015/M1506-Refactor-GLES2-Implementation: Difference between revisions

From Expertiza_Wiki
Jump to navigation Jump to search
No edit summary
No edit summary
 
(33 intermediate revisions by 2 users not shown)
Line 5: Line 5:
=='''Introduction'''==
=='''Introduction'''==
===Servo===
===Servo===
[https://en.wikipedia.org/wiki/Servo_(layout_engine) Servo]Servo is an open source prototype web browser layout engine being developed by Mozilla, and it is written in [https://www.rust-lang.org/ Rust] language. The main idea is to create a highly parallel environment, in which different components can be handled by fine grained, isolated tasks. The different components can be rendering, HTML parsing, etc.  
[https://en.wikipedia.org/wiki/Servo_(layout_engine) Servo]Servo is an open source prototype web browser layout engine being developed by Mozilla, and it is written in [https://www.rust-lang.org/ Rust] language. The main idea is to create a highly parallel environment, in which different components can be handled by fine grained, isolated tasks. The different components can be rendering, HTML parsing, etc.<ref>https://en.wikipedia.org/wiki/Servo_(layout_engine)</ref>


===Rust===
===Rust===
[http://doc.rust-lang.org/book/README.html Rust] is an open source systems programming language developed by Mozilla. Servo is written in Rust. The main purpose behind it's design is to be thread safe and concurrent. The emphasis is also on speed, safety and control of memory layout.
[http://doc.rust-lang.org/book/README.html Rust] is an open source systems programming language developed by Mozilla. Servo is written in Rust. The main purpose behind it's design is to be thread safe and concurrent. The emphasis is also on speed, safety and control of memory layout.<ref>https://doc.rust-lang.org/stable/book/</ref><ref>https://en.wikipedia.org/wiki/Rust_(programming_language)</ref>


=='''Project Description'''==
=='''Project Description'''==
* The project requirement initially stated that we build and Compile servo. Following are the steps for this:
===Setup of Development environment===
 
* Servo can be built and compiled in the following steps:
Servo is built with [https://mail.mozilla.org/pipermail/rust-dev/2014-March/009090.html Cargo], the Rust package manager. Mozilla's Mach tools are used to orchestrate the build and other tasks.
Servo is built with [https://mail.mozilla.org/pipermail/rust-dev/2014-March/009090.html Cargo], the Rust package manager. Mozilla's Mach tools are used to orchestrate the build and other tasks.


Line 19: Line 21:
     ./mach run tests/html/about-mozilla.html
     ./mach run tests/html/about-mozilla.html


* The next requirement was to build the Rust-layers independently of Servo. For this, we
* Next, we need a cargo override for servo to use our local copy of Rust-layers. For this, we created a <code>.cargo</code> folder in our home directory(same place that servo and Rust-layers reside), and added a <code>config</code> file to that folder. The content of that config file is a path to our local Rust-layers.
* Next, we had to over-ride cargo to use our local copy of Rust-layers, so we had to add a cargo override to it. For this, we created a <code>.cargo</code> folder in our home directory(same place that servo and Rust-layers reside), and added a <code>config</code> file to that folder. The content of that config file is a path to our local Rust-layers.
     paths = [path/to/rust_layers]
     paths = [path/to/rust_layers]
* Next, we had to add a new command line argument in Servo, which would allow selecting the graphics background (GL or ES2). For this, we made the following changes:
- We added a command line option which lets the user enter the option <code>-E</code> or <code>--es2</code> if the user wants to set ES2 as the graphic back end option. GL is set by default, hence if the user doesn't give any argument, Gl is selected as the graphic back end option. Example is as below
    ./mach run [url] -E
* Next, we had to add a flag in Rust-layers. This change was made in the file [https://github.com/prashantgupta24/rust-layers/blob/master/src/rendergl.rs Rust/src/rendergl.rs] in the <code>RenderContext::new</code> function. For this, we did the following change:
- We defined an enum <code>GraphicOption</code> which defined two options <code>GL</code> and <code>ES2</code> and then based on the boolean value from the servo <code>opts.rs</code>, we set a variable with the respective enum value <code>GL</code> or <code>ES2</code>.
* Then, we had to use the new command line option to pass the selected graphics back-end option to the Rust-layers context which we created in the previous step. For this we made changes to the [https://github.com/prashantgupta24/servo/blob/master/components/compositing/compositor.rs compositor.rs] in Compositing folder which is how Servo interacts with the Rust layers. The command line option is passed through the <code>initialize_compositing</code> function.
* Now we need to change the platform implementation in [https://github.com/servo/rust-layers/blob/master/src/platform/surface.rs surface.rs] and provide a generic implementation to dynamically select between GL & ES2.


<center>[[File:RefactorGLESMozilla.png]]</center>
===Overview===
* After building servo and rust-layers successfully, we added a new command line argument in Servo, which would allow user to select the graphics backend (GL or ES2).
 
Example is as below
    ./mach run [url] -G GL or ./mach run [url] --graphics GL (for GL)
    ./mach run [url] -G ES2 or ./mach run [url] --graphics ES2 (for ES2)
 
* This option selected is passed onto the rust-layers, which is modified to support ES2 for both linux and android versions. Hence, based on the option selected by the user, rust-layers renders the corresponding graphic backend on linux and android targets.To achieve this, the main changes are done in the platform to remove the EGL specific implementation from the Android platform to a more generic one, so that it can be used by both android and linux systems.


=='''Requirement Analysis'''==
=='''Requirement Analysis'''==


Earlier, only [https://en.wikipedia.org/wiki/OpenGL OpenGL] implementation was supported for Linux targets and we would be extending the [https://en.wikipedia.org/wiki/OpenGL_ES#2.0 ES2] implementation to support it for linux targets too. As a part of the final project, we propose to perform the following steps:
Earlier, only [https://en.wikipedia.org/wiki/OpenGL OpenGL] implementation was supported for Android targets and we would be extending the [https://en.wikipedia.org/wiki/OpenGL_ES#2.0 ES2] implementation to support it for Linux targets.
 
===Initial step===
 
Our [https://github.com/servo/servo/wiki/Refactor-GLES2-student-project initial step], implemented for the OSS project involved:
* compile Servo and ensure that it runs on tests/html/about-mozilla.html
* adding a new command-line option to allow selecting the graphics backend (GL or ES2) in <code>components/utils/opts.rs</code>
* clone rust-layers and build it independently of Servo
* adding a Cargo override for Servo to use the local copy of rust-layers
 
===Final Requirements===
 
* Enable selection of the render api working from the command line, so the user can pick whether to use GL or ES2
* Modify the necessary files so that the current display version of GL or ES2 are returned whenever either of them are selected
* Make the EGL libs available on Linux apart from android, for rust-layers to compile rust-egl on Linux build.
* Separate out the EGLImageNativeSurface implementation from Android and move it to EGL
* Enable use of EGL or GLX for Linux.
* In conclusion, implement a generic way of supporting EGL for both Linux and Android, such that ES2 is rendered as a display on linux targets as well.
 
=='''Implementation'''==
 
===Servo===
 
  Servo/components/utils/opts.rs
added the new command line option to enable user to select the option of GL or ES2 as the rendering backend. The options of GL or ES2 are stored as an enum called RenderApi , such that running the Servo with option --render-api=chosen_option sets the corresponding Api.
 
  Servo/ports/glutin/window.rs
<code>gl_version()</code> function is modified in a way that based on what is selected as the RenderApi option by the user, the corresponding version of the Api is set. To talk more in detail, if the render api is set to GL, it returns the current value (GL 2.1) & if set to ES2, it returns OpenGlEs 2.0.
 
<code>native_display()</code> for linux target is also modified to call the specific NativeDisplay constructor based on the RenderApi option chosen by the user. Hence for the GL case, the existing NativeDisplay::new constructor is called, but if ES2 is selected, NativeDisplay::from_es2() is called, which has been introduced by us in an attempt to support ES2 for the linux platform as well.
 
===Rust-Layers===
 
  rust-layers/cargo.toml
In order to make the egl libraries available to the Linux too, the cargo.toml is edited which defines the different dependencies that various platforms has. Before, the dependency on rust-egl was added only for android targets but now these dependencies have been added for Linux targets as well.
 
  rust-layers/src/platform/egl/surface.rs
In order to support EGL for both linux and android targets, we moved the implementation of <code>EGLImageNativeSurface</code> from the android specific <code>surface.rs</code> and introduced a new folder <code>egl</code> which contains its own <code>surface.rs</code> and implements a generic <code>EGLImageNativeSurface </code>


* Change the platform implementation to provide a more generic way to dynamically select between GL and ES2.  
  rust-layers/src/lib.rs
* Remove the Linux/Android specific #cfg flags.
Since egl needs to be supported for linux too now, the extern crate egl call needs to be linked on Linux as well now. Hence in the cfg flag, the target_os="linux" is added in addition to android.
* Place a general #cfg flag for both Android and Linux to provide a more generic implementation of dynamic selection between GL and ES2
* Make changes to EGL platform to make it work on desktop GL


=='''Files modified'''==
In the pub mod platform { }, introduced a new pub mod egl {} section to pull in the new <code>egl/surface.rs</code> implemented for both android and linux targets.


    Servo/components/utils/opts.rs
  rust-layers/src/platform/linux/surface.rs
added the new command line option to allow selecting the graphics backend.
In order to ensure that ES2 is supported for linux targets too, we need to enable the use of EGL & GLX in the linux specific <code>surface.rs</code>
rust-layers/src/rendergl.rs: changed the RenderContext new function to allow the dynamic selection of graphics backend.
A new enum is created for NativeDisplay which has two different values of <code>EGLDisplayInfo</code> & <code>GlxDisplayInfo</code> and respectively, two constructors are created . The existing NativeDisplay::new constructor is modified to return <code>NativeDisplay::Glx(info)</code> for the GL version , and the NativeDisplay::from_es2() constructor introduced earlier can return <code>NativeDisplay::::Egl(egl_info)</code> for the ES2 version.


=='''Files to be modified'''==
  rust-layers/src/platform/surface.rs
Earlier, the android target was using its own <code>surface.rs</code> and the linux target was using its own <code>surface.rs</code>. But after the implementation of <code>egl/surface.rs</code>, changes were made to platform <code>surface.rs</code> which uses the egl surface.rs for both android and linux now.


rust-layers/src/platform/surface.rs: providing generic GL implementation that can dynamically select between the 2 interfaces.
=='''Design Principles'''==


=='''Useful libraries'''==
We are adhering to the following design principles for our implementation:
 
'''Open-Closed Principle'''<ref>http://en.wikipedia.org/wiki/Open/closed_principle</ref>
 
Although our task is to modify the egl implementation to support it for linux targets too, we are not modifying the existing functionality of the android platform, instead we have extended the egl implementation to provide support for linux as well.
 
'''Interface Segregation Principle'''<ref>http://en.wikipedia.org/wiki/Interface_segregation_principle</ref>


waffle-gl/waffle: A C library for selecting an OpenGL API and window system at runtime.
The existing code currently has specific platform implementations for android and linux, and egl implementation is provided in android. We have extended the egl implementation to a more generic platform surface and divided the functionality for the native display into two small methods such that these methods can be individually called for whichever graphic backend option is chosen. This avoids the need to call the bigger functionality.


=='''Design Pattern'''==
=='''Design Pattern'''==
Line 60: Line 103:
We propose to use [https://en.wikipedia.org/wiki/Decorator_pattern Decorator] pattern. The decorator pattern gives the user the ability to choose between the implementations that are desired, and attach the corresponding implementation to the target dynamically at runtime.
We propose to use [https://en.wikipedia.org/wiki/Decorator_pattern Decorator] pattern. The decorator pattern gives the user the ability to choose between the implementations that are desired, and attach the corresponding implementation to the target dynamically at runtime.


=='''To Be Determined Information'''==
==Flowchart describing Project Implementation==
* Whether waffle-gl is to be imported, or simply used as example for base functionality
<center>[[File:RefactorGLESMozilla1506OODD.png]]</center>
* Where android surface.rs implements ES2 vs GL (ie, what is the dividing line between a GL graphics implementation, and an ES2 graphics implementation).
 
* Acquiring Android interface to test modified “general” GL_ES implementation.
==Class Diagram==
 
<center>[[File:Class_diagram_Mozilla_1506.png]]</center>
 
 
== Proposed Test Cases ==
 
Some of the test cases which will be used to validate our changes are as follows:
* Printing the command line option selected by the user in the <code>opts.rs</code>.This ensures if the command line option selected is correctly passed onto the servo.
  println!("selected {:?}", opts.render_api);
* Printing the command line option selected by the user in the <code>window.rs</code>.This ensures if the command line option selected is correctly propagated from the servo to the rust-layers.
* Test if the rust-egl compiles for the linux surface. A successful compilation ensures egl libraries are made available on Linux for rust-layers.
* Test if selecting an option ES2 though command line on a linux surface, renders a graphic display of ES2.
 
== Video ==
A walkthrough of our entire project is available on this link: [https://drive.google.com/file/d/0B9TVPg3YRoHqT2llQ0FfMWNvUEE/view Walkthrough Video]
 
=='''Useful libraries'''==
 
waffle-gl/waffle: A C library for selecting an OpenGL API and window system at runtime.
 
==Further Reading==
1.    [https://github.com/servo/servo Servo GitHub]<br>
2.    [https://github.com/servo/rust-layers Rust-layers GitHub]<br>
3.    [http://doc.crates.io/guide.html#overriding-dependencies Overriding Dependencies]<br>
4.    [http://rustbyexample.com/ Rust Language]<br>
5.    [http://javarevisited.blogspot.com/2014/11/strategy-design-pattern-in-java-using-Enum-Example.html Strategy Design Pattern]<br>


=='''References'''==
==References==
1.    https://doc.rust-lang.org/stable/book/<br>
<references/>
2.    https://en.wikipedia.org/wiki/Rust_(programming_language)<br>
3.    https://en.wikipedia.org/wiki/Servo_(layout_engine)<br>
4.    https://github.com/servo/servo/wiki/Refactor-GLES2-student-project<br>
5.    http://doc.crates.io/guide.html#overriding-dependencies<br>
6.    http://rustbyexample.com/<br>
7.    http://javarevisited.blogspot.com/2014/11/strategy-design-pattern-in-java-using-Enum-Example.html<br>

Latest revision as of 17:27, 4 December 2015

' Refactor GLES2 Student Project with SERVO & RUST '

Servo is a prototype web browser engine written in the RUST language.Servo uses a variety of back-end implementations for drawing graphics depending on the operating system.One of such back-end is only compatible with Android right now, and we want to extend and refactor that back-end to enable on all Linux systems..

Introduction

Servo

ServoServo is an open source prototype web browser layout engine being developed by Mozilla, and it is written in Rust language. The main idea is to create a highly parallel environment, in which different components can be handled by fine grained, isolated tasks. The different components can be rendering, HTML parsing, etc.<ref>https://en.wikipedia.org/wiki/Servo_(layout_engine)</ref>

Rust

Rust is an open source systems programming language developed by Mozilla. Servo is written in Rust. The main purpose behind it's design is to be thread safe and concurrent. The emphasis is also on speed, safety and control of memory layout.<ref>https://doc.rust-lang.org/stable/book/</ref><ref>https://en.wikipedia.org/wiki/Rust_(programming_language)</ref>

Project Description

Setup of Development environment

  • Servo can be built and compiled in the following steps:

Servo is built with Cargo, the Rust package manager. Mozilla's Mach tools are used to orchestrate the build and other tasks.

   git clone https://github.com/servo/servo
   cd servo
   ./mach build --dev
   ./mach run tests/html/about-mozilla.html
  • Next, we need a cargo override for servo to use our local copy of Rust-layers. For this, we created a .cargo folder in our home directory(same place that servo and Rust-layers reside), and added a config file to that folder. The content of that config file is a path to our local Rust-layers.
   paths = [path/to/rust_layers]

Overview

  • After building servo and rust-layers successfully, we added a new command line argument in Servo, which would allow user to select the graphics backend (GL or ES2).

Example is as below

   ./mach run [url] -G GL or ./mach run [url] --graphics GL (for GL)
   ./mach run [url] -G ES2 or ./mach run [url] --graphics ES2 (for ES2)
  • This option selected is passed onto the rust-layers, which is modified to support ES2 for both linux and android versions. Hence, based on the option selected by the user, rust-layers renders the corresponding graphic backend on linux and android targets.To achieve this, the main changes are done in the platform to remove the EGL specific implementation from the Android platform to a more generic one, so that it can be used by both android and linux systems.

Requirement Analysis

Earlier, only OpenGL implementation was supported for Android targets and we would be extending the ES2 implementation to support it for Linux targets.

Initial step

Our initial step, implemented for the OSS project involved:

  • compile Servo and ensure that it runs on tests/html/about-mozilla.html
  • adding a new command-line option to allow selecting the graphics backend (GL or ES2) in components/utils/opts.rs
  • clone rust-layers and build it independently of Servo
  • adding a Cargo override for Servo to use the local copy of rust-layers

Final Requirements

  • Enable selection of the render api working from the command line, so the user can pick whether to use GL or ES2
  • Modify the necessary files so that the current display version of GL or ES2 are returned whenever either of them are selected
  • Make the EGL libs available on Linux apart from android, for rust-layers to compile rust-egl on Linux build.
  • Separate out the EGLImageNativeSurface implementation from Android and move it to EGL
  • Enable use of EGL or GLX for Linux.
  • In conclusion, implement a generic way of supporting EGL for both Linux and Android, such that ES2 is rendered as a display on linux targets as well.

Implementation

Servo

 Servo/components/utils/opts.rs

added the new command line option to enable user to select the option of GL or ES2 as the rendering backend. The options of GL or ES2 are stored as an enum called RenderApi , such that running the Servo with option --render-api=chosen_option sets the corresponding Api.

 Servo/ports/glutin/window.rs

gl_version() function is modified in a way that based on what is selected as the RenderApi option by the user, the corresponding version of the Api is set. To talk more in detail, if the render api is set to GL, it returns the current value (GL 2.1) & if set to ES2, it returns OpenGlEs 2.0.

native_display() for linux target is also modified to call the specific NativeDisplay constructor based on the RenderApi option chosen by the user. Hence for the GL case, the existing NativeDisplay::new constructor is called, but if ES2 is selected, NativeDisplay::from_es2() is called, which has been introduced by us in an attempt to support ES2 for the linux platform as well.

Rust-Layers

 rust-layers/cargo.toml

In order to make the egl libraries available to the Linux too, the cargo.toml is edited which defines the different dependencies that various platforms has. Before, the dependency on rust-egl was added only for android targets but now these dependencies have been added for Linux targets as well.

 rust-layers/src/platform/egl/surface.rs

In order to support EGL for both linux and android targets, we moved the implementation of EGLImageNativeSurface from the android specific surface.rs and introduced a new folder egl which contains its own surface.rs and implements a generic EGLImageNativeSurface

 rust-layers/src/lib.rs

Since egl needs to be supported for linux too now, the extern crate egl call needs to be linked on Linux as well now. Hence in the cfg flag, the target_os="linux" is added in addition to android.

In the pub mod platform { }, introduced a new pub mod egl {} section to pull in the new egl/surface.rs implemented for both android and linux targets.

 rust-layers/src/platform/linux/surface.rs

In order to ensure that ES2 is supported for linux targets too, we need to enable the use of EGL & GLX in the linux specific surface.rs A new enum is created for NativeDisplay which has two different values of EGLDisplayInfo & GlxDisplayInfo and respectively, two constructors are created . The existing NativeDisplay::new constructor is modified to return NativeDisplay::Glx(info) for the GL version , and the NativeDisplay::from_es2() constructor introduced earlier can return NativeDisplay::::Egl(egl_info) for the ES2 version.

 rust-layers/src/platform/surface.rs

Earlier, the android target was using its own surface.rs and the linux target was using its own surface.rs. But after the implementation of egl/surface.rs, changes were made to platform surface.rs which uses the egl surface.rs for both android and linux now.

Design Principles

We are adhering to the following design principles for our implementation:

Open-Closed Principle<ref>http://en.wikipedia.org/wiki/Open/closed_principle</ref>

Although our task is to modify the egl implementation to support it for linux targets too, we are not modifying the existing functionality of the android platform, instead we have extended the egl implementation to provide support for linux as well.

Interface Segregation Principle<ref>http://en.wikipedia.org/wiki/Interface_segregation_principle</ref>

The existing code currently has specific platform implementations for android and linux, and egl implementation is provided in android. We have extended the egl implementation to a more generic platform surface and divided the functionality for the native display into two small methods such that these methods can be individually called for whichever graphic backend option is chosen. This avoids the need to call the bigger functionality.

Design Pattern

The aim is to use ES2 implementation of rust layers for both android and Linux. So we need to use a design pattern that will allow us to extend the current implementation to support Linux targets too and give the user the flexibility to choose between the two implementations for rendering the backend.

We propose to use Decorator pattern. The decorator pattern gives the user the ability to choose between the implementations that are desired, and attach the corresponding implementation to the target dynamically at runtime.

Flowchart describing Project Implementation

Class Diagram


Proposed Test Cases

Some of the test cases which will be used to validate our changes are as follows:

  • Printing the command line option selected by the user in the opts.rs.This ensures if the command line option selected is correctly passed onto the servo.
 println!("selected {:?}", opts.render_api);
  • Printing the command line option selected by the user in the window.rs.This ensures if the command line option selected is correctly propagated from the servo to the rust-layers.
  • Test if the rust-egl compiles for the linux surface. A successful compilation ensures egl libraries are made available on Linux for rust-layers.
  • Test if selecting an option ES2 though command line on a linux surface, renders a graphic display of ES2.

Video

A walkthrough of our entire project is available on this link: Walkthrough Video

Useful libraries

waffle-gl/waffle: A C library for selecting an OpenGL API and window system at runtime.

Further Reading

1. Servo GitHub
2. Rust-layers GitHub
3. Overriding Dependencies
4. Rust Language
5. Strategy Design Pattern

References

<references/>