Expertiza_Wiki - User contributions [en]

CSC/CSC 517 Spring 2020/Implement ImageBitMap WebAPI

2020-05-01T14:16:57Z

Nradhak2: /* Future Work */

== '''Background Information''' ==
This project aims to contribute to Mozilla's experimental browser engine called Servo, which is implemented in a language called RUST(useful for implementing features that need concurrency and memory safety).

Many of the components of Servo are still under development and one such feature is the [https://www.techopedia.com/definition/792/bitmap-bmp| ImageBitmap].

Major browsers support the ImageBitmap standard which can be used to create images that are ready to be drawn efficiently to [https://en.wikipedia.org/wiki/Canvas_element| HTML canvas elements]. Servo is a new, experimental browser that supports these canvas APIs.

The goal of and motivation behind this project is to implement support for image bitmaps and improve our canvas automated test coverage as a result.

== '''About ImageBitMap and Motivation behind the project''' ==
We usually decode images for a use with canvas to allow users to customize an avatar, crop an image, or just zoom in on a picture. The problem with decoding images is that it can be CPU intensive, and that can sometimes mean jank or checkerboarding.

But the createImageBitmap() method allows us to decode the image in the background and get access to a new ImageBitmap primitive, which you can draw into a canvas in the same way you would an <img> element, another canvas, or a video.

The aim of this project is to develop the ImageBitmap for the servo environment.

This can be done in the steps mentioned in the following section.

== '''Steps for implementation''' ==

'''Initial Phase'''
* '''Step 1: '''Add a ImageBitmap WebIDL interface to ''components/script/dom/webidl''s and Rust implementation in components/script/dom/imagebitmap.rs
* '''Step 2: ''' Add the ''createImageBitmap'' method that takes no extra x/y/w/h parameters in ''component/script/dom/webidls/WindowOrWorkerGlobalScope.webidl'' and implement the method in ''component/script/dom/window.rs'', handling the HTMLCanvasElement and OffscreenCanvas types from the possible image sources

'''Subsequent phase'''
* '''Step 1: '''Implement several remaining image source types (HTMLImageElement, ImageData, ImageBitmap)
* '''Step 2: '''Implement the ''createImageBitmap'' overload that accepts x/y/w/h parameters
* '''Step 3: '''Implement support for ImageBitmaps as canvas image sources in ''components/script/canvas_state.rs''
[[File:AllSteps.jpg]]

== '''Previous Implementation''' ==
The steps that have been implemented so far in this project by the previous batch are:

'''Step 1:''' Added the ImageBitmap interface to ''components/script/dom/webidls'' that represents a bitmap image which can be drawn to a <canvas> without undue latency. The interface contains height and weight as its attributes which are read only unsigned long integers.
<pre>
//[Exposed=(Window,Worker), Serializable, Transferable]
[Exposed=(Window,Worker)]
interface ImageBitmap {
readonly attribute unsigned long width;
readonly attribute unsigned long height;
//void close();
};

typedef (CanvasImageSource or
Blob or
ImageData) ImageBitmapSource;

enum ImageOrientation { "none", "flipY" };
enum PremultiplyAlpha { "none", "premultiply", "default" };
enum ColorSpaceConversion { "none", "default" };
enum ResizeQuality { "pixelated", "low", "medium", "high" };

dictionary ImageBitmapOptions {
ImageOrientation imageOrientation = "none";
PremultiplyAlpha premultiplyAlpha = "default";
ColorSpaceConversion colorSpaceConversion = "default";
[EnforceRange] unsigned long resizeWidth;
[EnforceRange] unsigned long resizeHeight;
ResizeQuality resizeQuality = "low";
};
</pre>

The ImageBitmap webidl also defines a dictionary for various ImageBitmap Options that can be used to modify the ImageBitmap object.

'''Step 2:''' Implemented the rust code for the webidl interface at ''components/script/dom/imagebitmap.rs''. For more details on the code [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Spring_2020_-_M2000._Implement_ImageBitMap_web_API#Implementation visit]
<pre>
use crate::dom::bindings::cell::DomRefCell;

use crate::dom::bindings::codegen::Bindings::ImageBitmapBinding::ImageBitmapMethods;
use crate::dom::bindings::root::DomRoot;
use crate::dom::globalscope::GlobalScope;

use crate::dom::bindings::error::Fallible;
use crate::dom::bindings::reflector::{reflect_dom_object, Reflector};
use dom_struct::dom_struct;

use std::vec::Vec;

#[dom_struct]
pub struct ImageBitmap {
reflector_: Reflector,
width: u32,
height: u32,
bitmap_data: DomRefCell<Vec<u8>>,
}

impl ImageBitmap {
fn new_inherited(width_arg: u32, height_arg: u32) -> ImageBitmap {
ImageBitmap {
reflector_: Reflector::new(),
width: width_arg,
height: height_arg,
bitmap_data: DomRefCell::new(vec![]),
}
}

#[allow(dead_code)]
pub fn new(global: &GlobalScope, width: u32, height: u32) -> Fallible<DomRoot<ImageBitmap>> {
//assigning to a variable the return object of new_inherited
let imagebitmap = Box::new(ImageBitmap::new_inherited(width, height));

Ok(reflect_dom_object(imagebitmap, global))
}
}
</pre>

The code also implements the getter methods for height and width attribute of an ImageBitmap object.
<pre>
impl ImageBitmapMethods for ImageBitmap {
// https://html.spec.whatwg.org/multipage/#dom-imagebitmap-height
fn Height(&self) -> u32 {
//to do: add a condition for checking detached internal slot
//and return 0 if set to true
self.height
}

// https://html.spec.whatwg.org/multipage/#dom-imagebitmap-width
fn Width(&self) -> u32 {
//to do: add a condition to check detached internal slot
////and return 0 if set to true
self.width
}
}
</pre>

== '''Proposed Implementation''' ==
Among the remaining steps in the initial and subsequent phases, the focus will be on step 2 of initial phase and once there is progress made on this step, implementation of the subsequent steps will take place.

;Implementing createImageBitmap web API

:*The createImageBitmap() method creates a bitmap from a given source, optionally cropped to contain only a portion of that source. The method exists on the global scope in both windows and workers. It accepts a variety of different image sources, and returns a Promise which resolves to an ImageBitmap.
:*This web interface is defined in the file ''components/script/dom/webidls/WindowOrWorkerGlobalScope.webidl'' and rust code is implemented at ''components/script/dom/globalscope.rs''. It is then called in ''components/script/dom/window.rs'' and ''components/script/dom/workerglobalscope.rs''
:*The syntax of the method looks like
:::const imageBitmapPromise = createImageBitmap(image[, options]);
:::const imageBitmapPromise = createImageBitmap(image, sx, sy, sw, sh[, options]);
::where the parameters indicate:
:::'''image:''' an image source, which can be an img element, a SVG image element, a video element, a canvas element, a blob object, an ImageData object or another ImageBitmap object.
:::'''sx, sy, sw, sh:''' if given, source image is cropped to the given pixels.
:::'''options (Optional):''' the ImageBitmap object's bitmap data is modified according to options. Available options are:
:::*imageOrientation
:::*premultiplyAlpha
:::*colorSpaceConversion
:::*resizeWidth
:::*resizeHeight
:::*resizeQuality
:::: You can read more about these options [https://developer.mozilla.org/en-US/docs/Web/API/WindowOrWorkerGlobalScope/createImageBitmap here]
::The return value is a Promise that is resolved when a new ImageBitmap is created.
:* As a first step, we will be implementing the method to handle canvas elements. Subsequently, we will be enhancing the method to handle other image sources and the x/y/w/h parameters.

;Implementing close() method from previous work

:* close() is a method in the ImageBitmap interface. It is implemented in ''components/script/dom/imagebitmap.rs'' file.
:* This method disposes of all graphical resources associated with an ImageBitmap.

== '''Current Progress''' ==

* Implemented createImageBitmap method for canvas image source(HTMLCanvasElement and OffscreenCanvas) using RUST programming language in the '''components/script/dom/globalscope.rs''' file. The comments in the following code will explain the functionality that it carries out.

<pre>
// https://html.spec.whatwg.org/multipage/#dom-createimagebitmap
pub fn create_image_bitmap(
&self,
image: ImageBitmapSource,
options: &ImageBitmapOptions,
) -> Rc<Promise> {
let in_realm_proof = AlreadyInRealm::assert(&self);
// Created a new promise object p
let p = Promise::new_in_current_realm(&self, InRealm::Already(&in_realm_proof));

// If resizewidth is present and is equal to 0, reject p with InvalidState error
if options.resizeWidth.map_or(false, |w| w == 0) {
p.reject_error(Error::InvalidState);
return p;
}

// If resizeheight is present and is equal to 0, reject p with InvalidState error
if options.resizeHeight.map_or(false, |w| w == 0) {
p.reject_error(Error::InvalidState);
return p;
}

// match the image to different types of image source and return the resolved p
let promise = match image {
// check if image source is HTMLCanvasElement
ImageBitmapSource::HTMLCanvasElement(ref canvas) => {
// https://html.spec.whatwg.org/multipage/#check-the-usability-of-the-image-argument
// check if the image source is valid
if !canvas.is_valid() {
p.reject_error(Error::InvalidState);
return p;
}

// the fetch_all_data() method returns the bitmap data and size of the image
if let Some((data, size)) = canvas.fetch_all_data() {
// convert the data to a <vec> type
let data = data
.map(|data| data.to_vec())
.unwrap_or_else(|| vec![0; size.area() as usize * 4]);
//create imageBitmap object using the height and width of the image
let image_bitmap = ImageBitmap::new(&self, size.width, size.height).unwrap();

// copy the data to bitmap_data of imageBitmap object
image_bitmap.set_bitmap_data(data);
// copy the origin clean flag of canvas image to origin_clean flag of imageBitmap object
image_bitmap.set_origin_clean(canvas.origin_is_clean());
// resolve p with imageBitmap object
p.resolve_native(&(image_bitmap));
}
p
},
// check if image source is OffscreenCanvas
ImageBitmapSource::OffscreenCanvas(ref canvas) => {
// https://html.spec.whatwg.org/multipage/#check-the-usability-of-the-image-argument
// check if the image source is valid
if !canvas.is_valid() {
p.reject_error(Error::InvalidState);
return p;
}

// the fetch_all_data() method returns the bitmap data and size of the image
if let Some((data, size)) = canvas.fetch_all_data() {
// convert the data to a <vec> type
let data = data
.map(|data| data.to_vec())
.unwrap_or_else(|| vec![0; size.area() as usize * 4]);
//create imageBitmap object using the height and width of the image
let image_bitmap = ImageBitmap::new(&self, size.width, size.height).unwrap();

// copy the data to bitmap_data of imageBitmap object
image_bitmap.set_bitmap_data(data);
// copy the origin clean flag of canvas image to origin_clean flag of imageBitmap object
image_bitmap.set_origin_clean(canvas.origin_is_clean());
// resolve p with imageBitmap object
p.resolve_native(&(image_bitmap));
}
p
},
_ => {
// if the image source is not HTMLCanvasElement or OffscreenCanvas, reject p with NotSupported Error
p.reject_error(Error::NotSupported);
return p;
},
};
promise
}
</pre>
* Tested the build using ‘mach’ commands

== '''Design of the createImageBitmap() method''' ==

On invoking the createImageBitmap(image, options) or the createImageBitmap(image sx, sy, sw, sh, options) the following in the sequence of actions

[[File:fig1.png|border]]

Switching on the "image" argument the following is the sequence of actions

=== If the image is an SVGImage===
[[File:fig2.png|border]]

=== If the image is a video===
[[File:ReFigure3.png|border]]

=== If the image is a canvas===
[[File:ReFigure4.png|border]]

=== If the image is an ImageBitmap===
[[File:ImageBitmap.jpg|border]]

=== If the image is an ImageData===
[[File:ImageData.jpg|border]]

=='''Test Plan'''==
The following are the respective steps in testing:
* Servo check should be successful.
The compiling time for servo build takes a lot of time. In order to make sure that our changes didn't break any of the existing features quickly, we run
<pre>
For Linux or macOS:
./mach check
</pre>
<pre>
For Windows:
mach.bat check
</pre>

[[File:machCheck.png]]

* Servo building test should be successful.
The compiling time is based on the CPU clock speed and the OS of the system where the file is being run. For a 64-bit Linux OS, it takes from about 1 hour to 2 hours for the entire build. It takes 2.5 hours on Windows OS. This is done to make sure that the end-to-end implementation can be successfully run in a system.
<pre>
For Linux or macOS:
./mach build --dev
</pre>
<pre>
For Windows:
mach.bat build --dev
</pre>
[[File:machBuild.png]]

* Check for the file tidiness (following standards of servo) using the command:
<pre>
For Linux or macOS:
./mach test-tidy
./mach fmt
</pre>
<pre>
For Windows:
mach.bat test-tidy
mach.bat fmt
</pre>

[[File:machTestTidy.png]]

*Servo automated Unit-testing should be successful.
Running the following commands will run all the unit-tests
<pre>
For Linux or macOS:
./mach test-unit
</pre>
<pre>
For Windows:
mach.bat test-unit
</pre>
Note: A log file can be maintained to make keep a check on the tests that are passing.
* Update the automated servo Web Platform tests(wpt):
For a DOM feature, extensive tests already exist under tests/wpt. Any change in DOM would require updating the expected results in the automated tests.

This first requires storing the log in raw format from a test run, for example by running
<pre>
./mach test-wpt --log-raw /path/for/storing/log/file/<filename>.log
</pre>

Once log is saved, then run to update test expectations
<pre>
./mach update-wpt /path/to/logfile/<filename>.log
</pre>

* Finally, to run the servo with a webpage (Google.com) to visually confirm that servo is working:
<pre>
For Linux or macOS:
./mach run https://www.google.com
</pre>
<pre>
For Windows:
mach.bat run https://www.google.com
</pre>

[[File:machRun.png]]

== '''Future Work ''' ==
The createImageBitmap() method can be extended to support the subsequent steps as shown in the steps for implementation.
1. The method can be extended to support other image sources such as SVGImage, video sources, ImageData source, and so on.
2. Overload the createImageBitmap() method to accept the s/y/w/h parameters
3. Also implement support for ImageBitmaps as canvas image sources in the canvas_state.rs file

=='''Code submission and Pull request'''==

'''Code: '''[https://github.com/ramyananth/servo] 
'''Pull Request: '''[https://github.com/servo/servo/pull/26296]

== '''Team Details''' ==

Jayalakshmi Vishwanathan (jviswan@ncsu.edu)

Nita Radhakrishnan (nradhak2@ncsu.edu)

Ramya Ananth (rananth2@ncsu.edu)

Sandeep Kundala (skundal@ncsu.edu)

== '''Mentor Details''' ==
Jay Modi

== '''References''' ==

* [https://en.wikipedia.org/wiki/Servo_(software)| What is servo?]
* [https://en.wikipedia.org/wiki/Rust_(programming_language)| what is RUST programming?]

CSC/CSC 517 Spring 2020/Implement ImageBitMap WebAPI

2020-05-01T14:15:45Z

Nradhak2:

== '''Background Information''' ==
This project aims to contribute to Mozilla's experimental browser engine called Servo, which is implemented in a language called RUST(useful for implementing features that need concurrency and memory safety).

Many of the components of Servo are still under development and one such feature is the [https://www.techopedia.com/definition/792/bitmap-bmp| ImageBitmap].

Major browsers support the ImageBitmap standard which can be used to create images that are ready to be drawn efficiently to [https://en.wikipedia.org/wiki/Canvas_element| HTML canvas elements]. Servo is a new, experimental browser that supports these canvas APIs.

The goal of and motivation behind this project is to implement support for image bitmaps and improve our canvas automated test coverage as a result.

== '''About ImageBitMap and Motivation behind the project''' ==
We usually decode images for a use with canvas to allow users to customize an avatar, crop an image, or just zoom in on a picture. The problem with decoding images is that it can be CPU intensive, and that can sometimes mean jank or checkerboarding.

But the createImageBitmap() method allows us to decode the image in the background and get access to a new ImageBitmap primitive, which you can draw into a canvas in the same way you would an <img> element, another canvas, or a video.

The aim of this project is to develop the ImageBitmap for the servo environment.

This can be done in the steps mentioned in the following section.

== '''Steps for implementation''' ==

'''Initial Phase'''
* '''Step 1: '''Add a ImageBitmap WebIDL interface to ''components/script/dom/webidl''s and Rust implementation in components/script/dom/imagebitmap.rs
* '''Step 2: ''' Add the ''createImageBitmap'' method that takes no extra x/y/w/h parameters in ''component/script/dom/webidls/WindowOrWorkerGlobalScope.webidl'' and implement the method in ''component/script/dom/window.rs'', handling the HTMLCanvasElement and OffscreenCanvas types from the possible image sources

'''Subsequent phase'''
* '''Step 1: '''Implement several remaining image source types (HTMLImageElement, ImageData, ImageBitmap)
* '''Step 2: '''Implement the ''createImageBitmap'' overload that accepts x/y/w/h parameters
* '''Step 3: '''Implement support for ImageBitmaps as canvas image sources in ''components/script/canvas_state.rs''
[[File:AllSteps.jpg]]

== '''Previous Implementation''' ==
The steps that have been implemented so far in this project by the previous batch are:

'''Step 1:''' Added the ImageBitmap interface to ''components/script/dom/webidls'' that represents a bitmap image which can be drawn to a <canvas> without undue latency. The interface contains height and weight as its attributes which are read only unsigned long integers.
<pre>
//[Exposed=(Window,Worker), Serializable, Transferable]
[Exposed=(Window,Worker)]
interface ImageBitmap {
readonly attribute unsigned long width;
readonly attribute unsigned long height;
//void close();
};

typedef (CanvasImageSource or
Blob or
ImageData) ImageBitmapSource;

enum ImageOrientation { "none", "flipY" };
enum PremultiplyAlpha { "none", "premultiply", "default" };
enum ColorSpaceConversion { "none", "default" };
enum ResizeQuality { "pixelated", "low", "medium", "high" };

dictionary ImageBitmapOptions {
ImageOrientation imageOrientation = "none";
PremultiplyAlpha premultiplyAlpha = "default";
ColorSpaceConversion colorSpaceConversion = "default";
[EnforceRange] unsigned long resizeWidth;
[EnforceRange] unsigned long resizeHeight;
ResizeQuality resizeQuality = "low";
};
</pre>

The ImageBitmap webidl also defines a dictionary for various ImageBitmap Options that can be used to modify the ImageBitmap object.

'''Step 2:''' Implemented the rust code for the webidl interface at ''components/script/dom/imagebitmap.rs''. For more details on the code [https://expertiza.csc.ncsu.edu/index.php/CSC/ECE_517_Spring_2020_-_M2000._Implement_ImageBitMap_web_API#Implementation visit]
<pre>
use crate::dom::bindings::cell::DomRefCell;

use crate::dom::bindings::codegen::Bindings::ImageBitmapBinding::ImageBitmapMethods;
use crate::dom::bindings::root::DomRoot;
use crate::dom::globalscope::GlobalScope;

use crate::dom::bindings::error::Fallible;
use crate::dom::bindings::reflector::{reflect_dom_object, Reflector};
use dom_struct::dom_struct;

use std::vec::Vec;

#[dom_struct]
pub struct ImageBitmap {
reflector_: Reflector,
width: u32,
height: u32,
bitmap_data: DomRefCell<Vec<u8>>,
}

impl ImageBitmap {
fn new_inherited(width_arg: u32, height_arg: u32) -> ImageBitmap {
ImageBitmap {
reflector_: Reflector::new(),
width: width_arg,
height: height_arg,
bitmap_data: DomRefCell::new(vec![]),
}
}

#[allow(dead_code)]
pub fn new(global: &GlobalScope, width: u32, height: u32) -> Fallible<DomRoot<ImageBitmap>> {
//assigning to a variable the return object of new_inherited
let imagebitmap = Box::new(ImageBitmap::new_inherited(width, height));

Ok(reflect_dom_object(imagebitmap, global))
}
}
</pre>

The code also implements the getter methods for height and width attribute of an ImageBitmap object.
<pre>
impl ImageBitmapMethods for ImageBitmap {
// https://html.spec.whatwg.org/multipage/#dom-imagebitmap-height
fn Height(&self) -> u32 {
//to do: add a condition for checking detached internal slot
//and return 0 if set to true
self.height
}

// https://html.spec.whatwg.org/multipage/#dom-imagebitmap-width
fn Width(&self) -> u32 {
//to do: add a condition to check detached internal slot
////and return 0 if set to true
self.width
}
}
</pre>

== '''Proposed Implementation''' ==
Among the remaining steps in the initial and subsequent phases, the focus will be on step 2 of initial phase and once there is progress made on this step, implementation of the subsequent steps will take place.

;Implementing createImageBitmap web API

:*The createImageBitmap() method creates a bitmap from a given source, optionally cropped to contain only a portion of that source. The method exists on the global scope in both windows and workers. It accepts a variety of different image sources, and returns a Promise which resolves to an ImageBitmap.
:*This web interface is defined in the file ''components/script/dom/webidls/WindowOrWorkerGlobalScope.webidl'' and rust code is implemented at ''components/script/dom/globalscope.rs''. It is then called in ''components/script/dom/window.rs'' and ''components/script/dom/workerglobalscope.rs''
:*The syntax of the method looks like
:::const imageBitmapPromise = createImageBitmap(image[, options]);
:::const imageBitmapPromise = createImageBitmap(image, sx, sy, sw, sh[, options]);
::where the parameters indicate:
:::'''image:''' an image source, which can be an img element, a SVG image element, a video element, a canvas element, a blob object, an ImageData object or another ImageBitmap object.
:::'''sx, sy, sw, sh:''' if given, source image is cropped to the given pixels.
:::'''options (Optional):''' the ImageBitmap object's bitmap data is modified according to options. Available options are:
:::*imageOrientation
:::*premultiplyAlpha
:::*colorSpaceConversion
:::*resizeWidth
:::*resizeHeight
:::*resizeQuality
:::: You can read more about these options [https://developer.mozilla.org/en-US/docs/Web/API/WindowOrWorkerGlobalScope/createImageBitmap here]
::The return value is a Promise that is resolved when a new ImageBitmap is created.
:* As a first step, we will be implementing the method to handle canvas elements. Subsequently, we will be enhancing the method to handle other image sources and the x/y/w/h parameters.

;Implementing close() method from previous work

:* close() is a method in the ImageBitmap interface. It is implemented in ''components/script/dom/imagebitmap.rs'' file.
:* This method disposes of all graphical resources associated with an ImageBitmap.

== '''Current Progress''' ==

* Implemented createImageBitmap method for canvas image source(HTMLCanvasElement and OffscreenCanvas) using RUST programming language in the '''components/script/dom/globalscope.rs''' file. The comments in the following code will explain the functionality that it carries out.

<pre>
// https://html.spec.whatwg.org/multipage/#dom-createimagebitmap
pub fn create_image_bitmap(
&self,
image: ImageBitmapSource,
options: &ImageBitmapOptions,
) -> Rc<Promise> {
let in_realm_proof = AlreadyInRealm::assert(&self);
// Created a new promise object p
let p = Promise::new_in_current_realm(&self, InRealm::Already(&in_realm_proof));

// If resizewidth is present and is equal to 0, reject p with InvalidState error
if options.resizeWidth.map_or(false, |w| w == 0) {
p.reject_error(Error::InvalidState);
return p;
}

// If resizeheight is present and is equal to 0, reject p with InvalidState error
if options.resizeHeight.map_or(false, |w| w == 0) {
p.reject_error(Error::InvalidState);
return p;
}

// match the image to different types of image source and return the resolved p
let promise = match image {
// check if image source is HTMLCanvasElement
ImageBitmapSource::HTMLCanvasElement(ref canvas) => {
// https://html.spec.whatwg.org/multipage/#check-the-usability-of-the-image-argument
// check if the image source is valid
if !canvas.is_valid() {
p.reject_error(Error::InvalidState);
return p;
}

// the fetch_all_data() method returns the bitmap data and size of the image
if let Some((data, size)) = canvas.fetch_all_data() {
// convert the data to a <vec> type
let data = data
.map(|data| data.to_vec())
.unwrap_or_else(|| vec![0; size.area() as usize * 4]);
//create imageBitmap object using the height and width of the image
let image_bitmap = ImageBitmap::new(&self, size.width, size.height).unwrap();

// copy the data to bitmap_data of imageBitmap object
image_bitmap.set_bitmap_data(data);
// copy the origin clean flag of canvas image to origin_clean flag of imageBitmap object
image_bitmap.set_origin_clean(canvas.origin_is_clean());
// resolve p with imageBitmap object
p.resolve_native(&(image_bitmap));
}
p
},
// check if image source is OffscreenCanvas
ImageBitmapSource::OffscreenCanvas(ref canvas) => {
// https://html.spec.whatwg.org/multipage/#check-the-usability-of-the-image-argument
// check if the image source is valid
if !canvas.is_valid() {
p.reject_error(Error::InvalidState);
return p;
}

// the fetch_all_data() method returns the bitmap data and size of the image
if let Some((data, size)) = canvas.fetch_all_data() {
// convert the data to a <vec> type
let data = data
.map(|data| data.to_vec())
.unwrap_or_else(|| vec![0; size.area() as usize * 4]);
//create imageBitmap object using the height and width of the image
let image_bitmap = ImageBitmap::new(&self, size.width, size.height).unwrap();

// copy the data to bitmap_data of imageBitmap object
image_bitmap.set_bitmap_data(data);
// copy the origin clean flag of canvas image to origin_clean flag of imageBitmap object
image_bitmap.set_origin_clean(canvas.origin_is_clean());
// resolve p with imageBitmap object
p.resolve_native(&(image_bitmap));
}
p
},
_ => {
// if the image source is not HTMLCanvasElement or OffscreenCanvas, reject p with NotSupported Error
p.reject_error(Error::NotSupported);
return p;
},
};
promise
}
</pre>
* Tested the build using ‘mach’ commands

== '''Design of the createImageBitmap() method''' ==

On invoking the createImageBitmap(image, options) or the createImageBitmap(image sx, sy, sw, sh, options) the following in the sequence of actions

[[File:fig1.png|border]]

Switching on the "image" argument the following is the sequence of actions

=== If the image is an SVGImage===
[[File:fig2.png|border]]

=== If the image is a video===
[[File:ReFigure3.png|border]]

=== If the image is a canvas===
[[File:ReFigure4.png|border]]

=== If the image is an ImageBitmap===
[[File:ImageBitmap.jpg|border]]

=== If the image is an ImageData===
[[File:ImageData.jpg|border]]

=='''Test Plan'''==
The following are the respective steps in testing:
* Servo check should be successful.
The compiling time for servo build takes a lot of time. In order to make sure that our changes didn't break any of the existing features quickly, we run
<pre>
For Linux or macOS:
./mach check
</pre>
<pre>
For Windows:
mach.bat check
</pre>

[[File:machCheck.png]]

* Servo building test should be successful.
The compiling time is based on the CPU clock speed and the OS of the system where the file is being run. For a 64-bit Linux OS, it takes from about 1 hour to 2 hours for the entire build. It takes 2.5 hours on Windows OS. This is done to make sure that the end-to-end implementation can be successfully run in a system.
<pre>
For Linux or macOS:
./mach build --dev
</pre>
<pre>
For Windows:
mach.bat build --dev
</pre>
[[File:machBuild.png]]

* Check for the file tidiness (following standards of servo) using the command:
<pre>
For Linux or macOS:
./mach test-tidy
./mach fmt
</pre>
<pre>
For Windows:
mach.bat test-tidy
mach.bat fmt
</pre>

[[File:machTestTidy.png]]

*Servo automated Unit-testing should be successful.
Running the following commands will run all the unit-tests
<pre>
For Linux or macOS:
./mach test-unit
</pre>
<pre>
For Windows:
mach.bat test-unit
</pre>
Note: A log file can be maintained to make keep a check on the tests that are passing.
* Update the automated servo Web Platform tests(wpt):
For a DOM feature, extensive tests already exist under tests/wpt. Any change in DOM would require updating the expected results in the automated tests.

This first requires storing the log in raw format from a test run, for example by running
<pre>
./mach test-wpt --log-raw /path/for/storing/log/file/<filename>.log
</pre>

Once log is saved, then run to update test expectations
<pre>
./mach update-wpt /path/to/logfile/<filename>.log
</pre>

* Finally, to run the servo with a webpage (Google.com) to visually confirm that servo is working:
<pre>
For Linux or macOS:
./mach run https://www.google.com
</pre>
<pre>
For Windows:
mach.bat run https://www.google.com
</pre>

[[File:machRun.png]]

== '''Future Work ''' ==
The createImageBitmap() method can be extended to support the subsequent steps as shown in the steps for implementation.
- The method can be extended to support other image sources such as SVGImage, video sources, ImageData source, and so on.
- Overload the createImageBitmap() method to accept the s/y/w/h parameters
- Also implement support for ImageBitmaps as canvas image sources in the canvas_state.rs file

=='''Code submission and Pull request'''==

'''Code: '''[https://github.com/ramyananth/servo] 
'''Pull Request: '''[https://github.com/servo/servo/pull/26296]

== '''Team Details''' ==

Jayalakshmi Vishwanathan (jviswan@ncsu.edu)

Nita Radhakrishnan (nradhak2@ncsu.edu)

Ramya Ananth (rananth2@ncsu.edu)

Sandeep Kundala (skundal@ncsu.edu)

== '''Mentor Details''' ==
Jay Modi

== '''References''' ==

* [https://en.wikipedia.org/wiki/Servo_(software)| What is servo?]
* [https://en.wikipedia.org/wiki/Rust_(programming_language)| what is RUST programming?]

CSC/CSC 517 Spring 2020/Implement ImageBitMap WebAPI

2020-04-13T21:19:56Z

Nradhak2: /* About ImageBitMap and Motivation behind the project */

CSC/CSC 517 Spring 2020/Implement ImageBitMap WebAPI

2020-04-13T21:19:27Z

Nradhak2: /* About ImageBitMap */

CSC/CSC 517 Spring 2020/Implement ImageBitMap WebAPI

2020-04-13T21:07:50Z

Nradhak2:

CSC/CSC 517 Spring 2020/Implement ImageBitMap WebAPI

2020-04-13T21:06:34Z

Nradhak2: /* Details about current work on implementation */

CSC/CSC 517 Spring 2020/Implement ImageBitMap WebAPI

2020-04-13T21:06:02Z

Nradhak2: /* Details about current work on implementation */

CSC/CSC 517 Spring 2020/Implement ImageBitMap WebAPI

2020-04-13T21:05:21Z

Nradhak2: /* Details about current work on implementation */

CSC/CSC 517 Spring 2020/Implement ImageBitMap WebAPI

2020-04-13T20:59:15Z

Nradhak2:

CSC/CSC 517 Spring 2020/Implement ImageBitMap WebAPI

2020-04-13T20:54:15Z

Nradhak2: /* Details about current work on implementation */

CSC/CSC 517 Spring 2020/Implement ImageBitMap WebAPI

2020-04-13T20:53:46Z

Nradhak2:

CSC/CSC 517 Spring 2020/Implement ImageBitMap WebAPI

2020-04-13T20:40:04Z

Nradhak2:

CSC/CSC 517 Spring 2020/Implement ImageBitMap WebAPI

2020-04-13T20:34:08Z

Nradhak2: /* Background Information */

CSC/CSC 517 Spring 2020/Implement ImageBitMap WebAPI

2020-04-08T00:45:20Z

Nradhak2:

CSC/CSC 517 Spring 2020/Implement ImageBitMap WebAPI

2020-04-08T00:11:41Z

Nradhak2: /* Steps for implementation */

CSC/CSC 517 Spring 2020/Implement ImageBitMap WebAPI

2020-04-08T00:11:25Z

Nradhak2:

CSC/CSC 517 Spring 2020/Implement ImageBitMap WebAPI

2020-04-08T00:10:44Z

Nradhak2:

CSC/CSC 517 Spring 2020/Implement ImageBitMap WebAPI

2020-04-08T00:02:46Z

Nradhak2:

CSC/CSC 517 Spring 2020/Implement ImageBitMap WebAPI

2020-04-07T23:58:26Z

Nradhak2:

CSC/CSC 517 Spring 2020/Implement ImageBitMap WebAPI

2020-04-07T23:57:33Z

Nradhak2:

CSC/CSC 517 Spring 2020/Implement ImageBitMap WebAPI

2020-04-07T23:49:35Z

Nradhak2: Created page with " == '''Background Information''' == This project aims to contribute to Mozilla's experimental browser engine called Servo, which is implemented in a language called RUST(usefu..."

File:Impersonate correct user.jpeg

2020-04-01T01:05:19Z

Nradhak2:

File:Step1.png

2020-04-01T00:30:15Z

Nradhak2: Nradhak2 uploaded a new version of File:Step1.png

Main Page/CSC/CSC 517 Spring 2020 Refactor impersonate controller

2020-03-31T23:57:10Z

Nradhak2: /* Test Plan */

This wiki page describes the work done under E2002 OSS Program for Spring 2020, in the CSC/ECE 517 course.

===About Expertiza===

[http://expertiza.ncsu.edu/ Expertiza] is an open-source project based on [http://rubyonrails.org/ Ruby on Rails] framework. Expertiza is a complete instructor-student usage website where the instructor can assign assignments, deadlines, grades, etc that is required for the course. Similarly, the students can use this website to perform the tasks required as part of the course like project or assignments submission, forming groups and collaborating with them, as well as reviewing projects and teammates.

This project focuses on a specific feature of expertiza which allows administrators, instructors or teaching assistants to impersonate another user (like a student) and access their account.
The demonstration for the feature is as shown below.

[[File:Impersonate guide.jpg|figure 1|frame|center]]

[[File:Non impersonate.jpg|figure 2|frame|center]]

[[File:Impersonated view.jpg|figure 3|frame|center]]

===Problem Statement===
The aim of the project is to refactor the impersonate controller. The pre-existing code had the following major issues.
*All functions related to impersonate controller were present in a single method ( from figure 4a and 4b)
*Presence of repetitive code ( from figure 4a and 4b)
*3 levels of block nesting (from figure 5)
* Too many return statements (from figure 6)
[[File:initial code1.jpg| figure 4a| frame|center]]
[[File:initial code2.jpg| figure 4b| frame|center]]
[[File:code climate3.jpg| figure 5| frame|center]]
[[File:code climate2.jpg| figure 6| frame|center]]

This project is focused on resolving the issues mentioned above.

===About Impersonate Controller===
Expertiza allows the administrators, instructors or teaching assistants to impersonate another user (like a student) and access their account. For example, an instructor impersonating a student’s account can view their assignments, stage deadlines, peer reviews and anything else that the student can view.

One thing to be noted is that most of these users can only impersonate users for whom they are a parent. For example, instructor6 is a parent of student3841 and not student3836; as a result, instructor6 can impersonate only 3841.

===Impersonate functionality navigation===
1. Instructor login: username -> instructor6, password -> password

when logged in as an instructor, under the manage option in the ribbon as in Figure 1, select impersonate user.
Upon redirected to impersonate page, enter the account which needs to be impersonated. It impersonates that user provided that user can be impersonated. Now a new button called revert appears on the ribbon as in figure 3, this can be used to revert the impersonation and return to the instructor profile.

===Problem Solution===
The above-mentioned issues have been tackled by refactoring the impersonate controller by splitting into many smaller methods which are later called by the main impersonate controller.

The following are the refactored new methods that help in tackling the issue1 apart from each being specifically for some issue rectification:
*check_if_user_impersonateable
*display_error_msg
*overwrite_session
*check_if_special_char
*do_main_operartion

=====check_if_user_impersonateable=====
This method plays the main role in tackling issue3 - 3 levels of block nesting apart from issue1.

'''Intial Code'''
<pre>
if user
unless original_user.can_impersonate? user
flash[:error] = "You cannot impersonate #{params[:user][:name]}."
redirect_back
return
end
</pre>
''' After recfactoring - Moved to separate method'''
<pre>
def check_if_user_impersonateable
if params[:impersonate].nil?
user = User.find_by(name: params[:user][:name])
if !@original_user.can_impersonate? user
@message = "You cannot impersonate '#{params[:user][:name]}'."
temp
AuthController.clear_user_info(session, nil)
else
overwrite_session
end
else
if !params[:impersonate][:name].empty?
user = User.find_by(name: params[:impersonate][:name])
overwrite_session
end
end
end
</pre>

=====display_error_msg=====
This method is used to tackle issues1, 2 and 4. All the error message related code is moved to this method.
<pre>
def display_error_msg
if params[:user]
@message = "No user exists with the name '#{params[:user][:name]}'."
elsif params[:impersonate]
@message = "No user exists with the name '#{params[:impersonate][:name]}'."
else
if params[:impersonate].nil?
@message = "You cannot impersonate '#{params[:user][:name]}'."
else
if !params[:impersonate][:name].empty?
@message = "You cannot impersonate '#{params[:impersonate][:name]}'."
else
@message = "No original account was found. Please close your browser and start a new session."
end
end
end
rescue Exception => e
flash[:error] = @message
redirect_to :back
end
</pre>
=====overwrite_session=====
This method reduces the number of return statements used in impersonate controller, apart from reducing the size of the controller.

====== Initial Code======
<pre>
if params[:impersonate].nil?
# check if special chars /\?<>|&$# are used to avoid html tags or system command
if warn_for_special_chars(params[:user][:name], "Username")
redirect_back
return
end
user = User.find_by(name: params[:user][:name])
if user
unless original_user.can_impersonate? user
flash[:error] = "You cannot impersonate #{params[:user][:name]}."
redirect_back
return
end
session[:super_user] = session[:user] if session[:super_user].nil?
AuthController.clear_user_info(session, nil)
session[:original_user] = original_user
session[:impersonate] = true
session[:user] = user
else
flash[:error] = message
redirect_back
return
end
else
# Impersonate a new account
if !params[:impersonate][:name].empty?
# check if special chars /\?<>|&$# are used to avoid html tags or system command
if warn_for_special_chars(params[:impersonate][:name], "Username")
redirect_back
return
end
user = User.find_by(name: params[:impersonate][:name])
if user
unless original_user.can_impersonate? user
flash[:error] = "You cannot impersonate #{params[:user][:name]}."
redirect_back
return
end
AuthController.clear_user_info(session, nil)
session[:user] = user
session[:impersonate] = true
session[:original_user] = original_user
else
flash[:error] = message
redirect_back
return
end
</pre>
====== After Refactoring - Moved to a separate method and accessed through the adapter method do_main_operation======
<pre>
def overwrite_session
#if not impersonatable, then original user's session remains
if params[:impersonate].nil?
user = User.find_by(name: params[:user][:name])
session[:super_user] = session[:user] if session[:super_user].nil?
AuthController.clear_user_info(session, nil)
session[:original_user] = @original_user
session[:impersonate] = true
session[:user] = user
else
#if some user is to be impersonated, their session details are overwritten onto the current to impersonate

if !params[:impersonate][:name].empty?
user = User.find_by(name: params[:impersonate][:name])
AuthController.clear_user_info(session, nil)
session[:user] = user
session[:impersonate] = true
session[:original_user] = @original_user
else
user = User.find_by(name: params[:user][:name])
AuthController.clear_user_info(session, nil)
session[:user] = session[:super_user]
user = session[:user]
session[:super_user] = nil
end
end
end
</pre>
=====check_if_special_char=====
This code is used to reduce one functionality performed under the impersonate controller. This method checks to see if the given username is acceptable.
<pre>
def check_if_special_char
if warn_for_special_chars(params[:user][:name], "Username")
redirect_back
return
end
end
</pre>
=====do_main_operation=====
This like an adapter method that is used to interface the impersonate method with display_error_msg and check_if_user_impersonatable. One main purpose to do this is to make the methods flexible for change apart from reducing the number of lines from the impersonate controller.
<pre>
def do_main_operation(user)
if user
check_if_user_impersonateable
else
display_error_msg
end
end
</pre>

=== Test Plan ===
The project can be tested from the UI as follows.

Checking if impersonating a user is working
1. Input: User that can be impersonated
- Login as the instructor (username -> instructor6 , password -> password
- Under "Manage" tab, select "impersonate user" option
- In the form, give the user ID as "student5890"
Now you will be able to see that user "student5890" has been impersonated.
You can now "revert" to the original user to return to the instructor's profile.

2. Input: User that cannot be impersonated
- Login as the instructor (username -> instructor6 , password -> password
- Under "Manage" tab, select "impersonate user" option
- In the form, give the user ID as "super_administrator2"
Now you will be able to see a message being displayed on the screen, that tells that the given user cannot be impersonated.

3. Input: User that does not exist
- Login as the instructor (username -> instructor6 , password -> password
- Under "Manage" tab, select "impersonate user" option
- In the form, give the user ID as "studentstudent"
Now you will be able to see a message being displayed on the screen, that tells that the given user does not exist.

Main Page/CSC/CSC 517 Spring 2020 Refactor impersonate controller

2020-03-31T23:07:02Z

Nradhak2:

Main Page/CSC/CSC 517 Spring 2020 Refactor impersonate controller

2020-03-21T15:55:09Z

Nradhak2: Created page with "This wiki page describes the work done under E2002 OSS Program for Spring 2020, in the CSC/ECE 517 course."

This wiki page describes the work done under E2002 OSS Program for Spring 2020, in the CSC/ECE 517 course.