Expertiza_Wiki - User contributions [en]

CSC/ECE 517 Spring 2018- Project M1802: 2D Canvas Rendering (Part 2)

2018-04-24T12:41:52Z

Baeastwo: /* Conclusion */

'''“M1802: Simplify the 2d Canvas Rendering”'''

As robust and fast Servo is, Servo’s implementation of its HTML 2D/3D “<canvas>” has several inefficiencies that make some websites perform slowly or run out of memory when performing complex canvas operations. The goal of this project was to make these websites perform better when loaded in Servo.

=Introduction=
This section briefly describes the main parts of this project: [https://en.wikipedia.org/wiki/Servo_(layout_engine) Servo], [https://www.rust-lang.org/en-US/ Rust], and [https://www.w3schools.com/graphics/canvas_intro.asp HTML Canvas].

==Servo==
Named after a robot from the show Mystery Science Theater 3000, Servo is an experimental web browser layout engine developed by Mozilla. It is currently being developed for 64-bit OS X, 64-bit Linux, 64-bit Windows, and Android. While working with Samsung, Mozilla seeks to create a highly parallel environment where the many things in a web browser like rendering, image decoding, and HTML parsing are handled by isolated tasks. [https://www.youtube.com/watch?v=Ry_RktGLKq4 Here] is a demo on how Servo performs against Firefox.

==Rust==
Since existing languages like C++ did not provide a direct way to achieve a parallel environment, Mozilla developed and uses a programming language called Rust to further develop Servo.

==HTML Canvas==
A <canvas> in HTML is a container for graphics that is used to draw graphics “on the fly” by using Javascript. However, the <canvas> element has no drawing abilities of its own since it only acts as a container for graphics. A script to actually draw the graphics. By using scripts, the HTML <canvas> can:
*Be animated
*Be interactive
*Draw text
*Be used in games
[[File:DrawImage.PNG|An example of several canvases from servo's tests.]]

=Project Description=
The goal of this project was to remove the inefficiencies of Servo’s implemetation of the 2D/3D canvas. As reported with issue [https://github.com/servo/servo/issues/10381 #10381], a gaming website called http://slither.io/ was performing extremely slowly as of Apr 3, 2016. As of Oct 21, 2016, it was reported with issue [https://github.com/servo/servo/issues/13879 #13879] that http://slither.io/ was, in fact, crashing when loaded onto Servo. The suggestion to these issues was to, first, create a test case that measured the amount of time that Servo spends under “drawImage” while doing operations related to a canvas. Another suggestion was to change Servo’s implementation from using one thread per canvas to one thread for all canvases. The [https://github.com/servo/servo/wiki/Canvas-rendering-project steps] laid out for this project by [https://www.mozilla.org/en-US/ Mozilla] (both the initial setup steps and optimizing the canvas implementation) are shown below.

==Initial Steps:==
*create a testcase that contains two canvases and uses the drawImage API to draw the contents of one canvas onto the other. Programmatically measure the time this operation takes.

*To prepare for the big switch from 1 threads per canvas to 1 thread for all canvases, add a struct CanvasId (u64) type to components/canvas_traits/canvas.rs and add a CanvasId member to each variant of the CanvasMsg enum.

*add a CanvasId member to Constellation in components/constellation/constellation.rs which is initialized to 0 and increased by 1 each time handle_create_canvas_paint_thread is called.

*make the response_sender argument of handle_create_canvas_paint_thread also include the new CanvasId value, and pass it as an argument to CanvasPaintThread::start. Store the id when it is received for use in all canvas messages.

*For each CanvasMsg that is processed by CanvasPaintThread, verify that the id received matches the id that was provided to CanvasPaintThread::start

==Subsequent Steps:==
*extract the innards of CanvasPaintThread into a CanvasData structure,

*make CanvasPaintThread store a hashtable of CanvasId->CanvasData

*as part of Constellation::start, create a canvas paint thread and store the channel to communicate with it as a member of Constellation. Remove the initial canvas id from the API of Constellation::start.

*when handle_create_canvas_paint_thread is invoked, communicate with the canvas thread and have it create a new entry in the hashtable.

*when the canvas thread receives a message, perform the operation on the appropriate canvas according to the provided id

*optimize the DrawImageInOther operation by drawing on the destination canvas directly, rather than relying on sending a message to another canvas thread. Remove the now-unnecessary IpcSender from the DrawImageInOther enum variant. Verify that the earlier test case demonstrates a performance improvement.

*report on how slither.io performs in Servo after all these changes

=Design Pattern=
Rust only has structs and traits, which are very different from classes and interfaces in terms of usage. Inheritance is effectively nonexistent in Rust, so GoF patterns are nearly always impossible to implement in Rust.

However, the GRASP pattern of Information Expert appears in the Subsequent Steps. CanvasPaintThread is no longer aware of the details of actual canvases, so it delegates the responsibility of actually painting on canvases to CanvasData.

Part of the goal of the Subsequent Steps was to increase the compliance to the Separation of Responsibilities principle. It took an object that previously both processed messages and painted canvases and converted it into an object that processes messages and another object that paints canvases.

=Implementation=
==General Information==

Our repo is [https://github.com/Brody-Eastwood/servo here].

===Building===
After setting up the environment required to develop for Servo, we built and compiled Servo as per the instructions on Servo’s Github [https://github.com/servo/servo repo]. We used Mozilla’s mach tools to build Servo with Cargo, which is the rust package manager.

<pre>
git clone https://github.com/servo/servo
cd servo
./mach build --dev
./mach run tests/html/about-mozilla.html
</pre>

===Testing===
After successfully building servo, we proceeded to test the amount of time spent under “drawImage” as shown below. The function "drawImage" copies the contents of one canvas in a view to another canvas in the same view. During the "Initial Steps," this test was created to test the initial performance of Servo with regards to canvas rendering.

<pre>
var t0 = performance.now();
drawImage(25, 25);
var t1 = performance.now();
document.getElementById('Test Result').innerHTML = "DrawImage took " + (t1 - t0) + " milliseconds.";
console.log("DrawImage took " + (t1 - t0) + " milliseconds.");
</pre>

[[File:TimedDrawImage.png]]

This was/will be used to measure the performance when drawing a single canvas.

===Running Servo===
After each successful build through our development process, we ran Servo as per the instructions on Servo’s Github repo.

<pre>
./servo [url] [arguments] # if you run with nightly build
./mach run [url] [arguments] # if you run with mach

# For example
./mach run https://www.google.com
</pre>

==Initial Steps==

The initial steps were the first OSS project. They did little to nothing to change the actual functionality of Servo, instead setting the Subsequent Steps up to have a smooth transition by finding all of the users of CanvasPaintThread and making them send CanvasIds, which can then be used to change CanvasPaintThread to the single-threaded design desired without having to touch the users again.

===Adding an Id to Canvas Communications===
As shown below, we then associated a CanvasId with each variant of a canvas message; these messages are passed around various parts of the Servo implementation such as the 2D canvas struct, view layout, and various scripts. This required us to make other changed to allow a CanvasId to be sent with each message, as shown below.

{| class="wikitable"
|-
! Before
! After
|-
| pub enum CanvasMsg {
Canvas2d(Canvas2dMsg),
FromLayout(FromLayoutMsg),
FromScript(FromScriptMsg),
Recreate(Size2D<i32>),
Close(),
}

| pub enum CanvasMsg {
Canvas2d(Canvas2dMsg, CanvasId),
FromLayout(FromLayoutMsg, CanvasId),
FromScript(FromScriptMsg, CanvasId),
Recreate(Size2D<i32>, CanvasId),
Close(CanvasId),
}
|}

Then, we made the necessary changed to the constellation and the canvas paint thread to be able to initialize, increment, and store the CanvasId. To finish up with the initial steps, we programmatically checked to make sure that each id that was processed by CanvasPaintThread was the same id that was provided when the thread was started (CanvasPaintThread::start).

<pre>
assert!(canvas_id == painter.canvas_id);
</pre>

===Performance===
Almost every time, http://slither.io/ ended up crashing. Other times, it performed very slowly. However, as of 4/2/2018, much more is left to be done (The Subsequent Steps).

==Subsequent Steps==

The Subsequent Steps were the final project. It builds off of what the Initial Steps lay down. Since the Initial Steps set up all of the users of CanvasPaintThread (even indirect users) to send CanvasIds with their messages, the conversion to use CanvasIds to identify individual canvases was seamless and invisible to the users.

===Overview===
Previously, a CanvasPaintThread was made for every Canvas, which was very slow when there are many Canvases. To fix this, a single CanvasPaintThread was made that manages all the canvases. The data previously stored in CanvasPaintThread unique to each Canvas was moved to a new data structure known as CanvasData.

Previously, communication channels were opened up between each CanvasPaintThread and what uses it and those channels send the CanvasIds established above alongside their messages. Now, there is a single channel that's repeatedly cloned which instead connects the single CanvasPaintThread and those same users. The users will be unable to tell the difference between the current implementation and the new implementation. The CanvasPaintThread will redirect those existing channels into painting using the appropriate CanvasData structure.

These changes involved moving nearly the entire contents of <code>servo/components/canvas/canvas_paint_thread.rs</code> into the new file <code>servo/components/canvas/canvas_data.rs</code>.

There were slight changes to <code>servo/components/constellation/constellation.rs</code> so that it doesn't try to create multiple CanvasPaintThreads, but those changes were minor and well described in the spec.

There were also changes to <code>servo/components/canvas_traits/canvas.rs</code> and <code>servo/components/script/dom/canvasrenderingcontext2d.rs</code> because of the removed parts of Canvas2dMsg::DrawImageInOther

===Initial Form of CanvasPaintThread===
From:
<code>servo/components/canvas/canvas_paint_thread.rs</code>

<code>
pub struct CanvasPaintThread<'a> {
drawtarget: DrawTarget,
/// TODO(pcwalton): Support multiple paths.
path_builder: PathBuilder,
state: CanvasPaintState<'a>,
saved_states: Vec<CanvasPaintState<'a>>,
webrender_api: webrender_api::RenderApi,
image_key: Option<webrender_api::ImageKey>,
/// An old webrender image key that can be deleted when the next epoch ends.
old_image_key: Option<webrender_api::ImageKey>,
/// An old webrender image key that can be deleted when the current epoch ends.
very_old_image_key: Option<webrender_api::ImageKey>,
canvas_id: CanvasId,
}
</code>

CanvasPaintThread initially stored all the information about a particular canvas. This includes both information used to represent the pixels actually on the canvas and information used to support drawing on the canvas.

There were many, many functions in the previous form of CanvasPaintThread because all the actual painting was done there.

CanvasIds were not usefully used. They were simply added in the initial steps so that the grunt work for the subsequent steps were simpler to implement and more focused.

===New Form of CanvasPaintThread===
====CanvasPaintThread====
From:
<code>servo/components/canvas/canvas_paint_thread.rs</code>

<code>
pub struct CanvasPaintThread <'a> {
canvases: HashMap<CanvasId, CanvasData<'a>>,
next_canvas_id: CanvasId,
}
</code>

The new CanvasPaintThread is much leaner. A single CanvasPaintThread exists with nothing but a HashMap of CanvasId->CanvasData structures and a CanvasId to represent what the next CanvasId will be.

The only functions it has are start (for initially starting it and also the thread itself, which passes CanvasMsg enums it receives around), create_canvas (for generating a new CanvasData structure), canvas (which is just a wrapper for the canvases hashmap to hide some ugliness in Rust's syntax), and process_canvas_2d_message (which exists solely because there are a lot of Canvas2dMsg enum variants and making the thread itself pass them around makes the code very hard to read).

Below is a snippet from process_canvas_2d_message, to give an idea as to what CanvasPaintThread does in general. There are far too many variants of the Canvas2dMsg enum to be reasonably put in this space, but the following snippet should give a rather good idea what it does in general.

<code>
match message {
Canvas2dMsg::FillText(text, x, y, max_width) => {
self.canvas(canvas_id).fill_text(text, x, y, max_width)
},
Canvas2dMsg::FillRect(ref rect) => {
self.canvas(canvas_id).fill_rect(rect)
},
Canvas2dMsg::StrokeRect(ref rect) => {
self.canvas(canvas_id).stroke_rect(rect)
},
Canvas2dMsg::ClearRect(ref rect) => {
self.canvas(canvas_id).clear_rect(rect)
},
...
}
</code>

In each Canvas2dMsg enum variant's "match" (think of it like a case statement for a switch but fancier), CanvasPaintThread simply takes the values out of the enum then calls the appropriate function on the appropriate CanvasData structure with those values. It does no checking of those values or otherwise processing them, simply passing them along.

There is one exception to this rule, but the message forces itself to act that way:

<code>
Canvas2dMsg::DrawImageInOther(
other_canvas_id,
image_size,
dest_rect,
source_rect,
smoothing
) => {
let mut image_data = self.canvas(canvas_id).read_pixels(
source_rect.to_i32(),
image_size);
// TODO: avoid double byte_swap.
byte_swap(&mut image_data);
self.canvas(other_canvas_id).draw_image(
image_data.into(),
source_rect.size,
dest_rect,
source_rect,
smoothing,
);
},
}
</code>

It pulls some data out of one CanvasData and then calls a method on the other CanvasData. For this message to be handled, a decision had to be made. Should CanvasData be aware of the existence of ''other'' CanvasData objects or should CanvasPaintThread do some processing to avoid that? There is the third option of making another object to handle it, but seeing as it's only a single message, that option was considered overkill at best, likely far worse. In the end, it was considered a smaller violation of proper design principles to let CanvasPaintThread move data from one CanvasData to another than to inform CanvasData structures about each other.

====CanvasData====
From:
<code>servo/components/canvas/canvas_data.rs</code>

<code>
pub struct CanvasData<'a> {
drawtarget: DrawTarget,
/// TODO(pcwalton): Support multiple paths.
path_builder: PathBuilder,
state: CanvasPaintState<'a>,
saved_states: Vec<CanvasPaintState<'a>>,
webrender_api: webrender_api::RenderApi,
image_key: Option<webrender_api::ImageKey>,
/// An old webrender image key that can be deleted when the next epoch ends.
old_image_key: Option<webrender_api::ImageKey>,
/// An old webrender image key that can be deleted when the current epoch ends.
very_old_image_key: Option<webrender_api::ImageKey>,
canvas_id: CanvasId,
}
</code>

You may notice the similarities between CanvasData and the old form of CanvasPaintThread! Now CanvasData does all the things CanvasPaintThread used to do for the purposes of painting on the canvases (for example, bezier_curve_to, arc, and set_line_cap among many others).

There's nothing particularly ''new'' in CanvasData. It's all just moved from CanvasPaintThread.

It is a very focused class now, though. All it does is paint on canvases. There's no processing of messages, only painting. Moreover, the draw_image_in_other function is nowhere to be found, instead replaced by a small amount of processing done by CanvasPaintThread, which just calls draw_image on the one that was originally considered "other" so there's no communication between the CanvasData structures ''at all''.

===New Form's Improvements===
====Design====
The new form keeps the responsibilities of converting CanvasMsg enums into their contents and using those contents to paint canvases separate, which improves the overall adherence to the Single Responsibility Principle of Servo. There was also a lot of odd indentation and otherwise messy code in the old form of CanvasPaintThread, which, in the process of preparing the pull request for these changes, were cleaned up, making the overall tidyness of Servo improve.

====Performance====
The new implementation is also much faster when multiple canvases are involved because there's no need for threading delay to send messages back and forth, waiting for those messages to be sent and received. There's less room for usage of multiple cores to speed up processing, but Servo is already so ''very'' parallel that there will be ''something'' to keep the other cores busy when CanvasPaintThread is stuck on one thread.

Slither.io doesn't run properly on my machine either before or after the changes, which makes reporting on the success of the changes with respect to Slither.io difficult at best. The framerate improved dramatically, even if it looks like... well... this still. I can't imagine that a website with over 3500 canvases could be made to run more slowly with this change, though.

[[File:Slither.png]]

Using our timed drawimage test, the performance improvement is... well... not really there. On the development build, the drawing time was cut nearly in half from around 19ms to around 9ms, but on the release build, the drawing time is effectively identical, varying between 1.3ms and 1.7ms for both versions. This isn't too surprising, seeing as only two canvases are made and the cost of communicating between two threads isn't very expensive compared to the thousands that the change was made to address.

=Conclusion=
So far we have associated each variant of a canvas message with a particular canvas id. Even though there is not much improvement is the performance of http://slither.io/, we believe that an improvement can be noticed after going through the Subsequent Steps. Here is a link to our [https://github.com/servo/servo/pull/20447 pull request]. On 4/2/2018, our contribution was merged into Servo's master branch!

The rendering of 2D canvases has been entirely moved to a single thread with each canvas identified by a CanvasId. The pull request for the final project is [https://github.com/servo/servo/pull/20680 here]. On 4/24/2018, our final project was merged into Servo's master branch!

=References=
#https://en.wikipedia.org/wiki/Servo_(layout_engine)
#https://www.rust-lang.org/en-US/
#https://github.com/servo/servo
#https://github.com/servo/servo/wiki/Canvas-rendering-project
#https://www.w3schools.com/graphics/canvas_intro.asp
#https://github.com/servo/servo/issues/10381
#https://github.com/servo/servo/issues/13879

CSC/ECE 517 Spring 2018- Project M1802: 2D Canvas Rendering (Part 2)

2018-04-24T03:07:47Z

Baeastwo: /* Design Pattern */

'''“M1802: Simplify the 2d Canvas Rendering”'''

As robust and fast Servo is, Servo’s implementation of its HTML 2D/3D “<canvas>” has several inefficiencies that make some websites perform slowly or run out of memory when performing complex canvas operations. The goal of this project was to make these websites perform better when loaded in Servo.

=Introduction=
This section briefly describes the main parts of this project: [https://en.wikipedia.org/wiki/Servo_(layout_engine) Servo], [https://www.rust-lang.org/en-US/ Rust], and [https://www.w3schools.com/graphics/canvas_intro.asp HTML Canvas].

==Servo==
Named after a robot from the show Mystery Science Theater 3000, Servo is an experimental web browser layout engine developed by Mozilla. It is currently being developed for 64-bit OS X, 64-bit Linux, 64-bit Windows, and Android. While working with Samsung, Mozilla seeks to create a highly parallel environment where the many things in a web browser like rendering, image decoding, and HTML parsing are handled by isolated tasks. [https://www.youtube.com/watch?v=Ry_RktGLKq4 Here] is a demo on how Servo performs against Firefox.

==Rust==
Since existing languages like C++ did not provide a direct way to achieve a parallel environment, Mozilla developed and uses a programming language called Rust to further develop Servo.

==HTML Canvas==
A <canvas> in HTML is a container for graphics that is used to draw graphics “on the fly” by using Javascript. However, the <canvas> element has no drawing abilities of its own since it only acts as a container for graphics. A script to actually draw the graphics. By using scripts, the HTML <canvas> can:
*Be animated
*Be interactive
*Draw text
*Be used in games
[[File:DrawImage.PNG|An example of several canvases from servo's tests.]]

=Project Description=
The goal of this project was to remove the inefficiencies of Servo’s implemetation of the 2D/3D canvas. As reported with issue [https://github.com/servo/servo/issues/10381 #10381], a gaming website called http://slither.io/ was performing extremely slowly as of Apr 3, 2016. As of Oct 21, 2016, it was reported with issue [https://github.com/servo/servo/issues/13879 #13879] that http://slither.io/ was, in fact, crashing when loaded onto Servo. The suggestion to these issues was to, first, create a test case that measured the amount of time that Servo spends under “drawImage” while doing operations related to a canvas. Another suggestion was to change Servo’s implementation from using one thread per canvas to one thread for all canvases. The [https://github.com/servo/servo/wiki/Canvas-rendering-project steps] laid out for this project by [https://www.mozilla.org/en-US/ Mozilla] (both the initial setup steps and optimizing the canvas implementation) are shown below.

==Initial Steps:==
*create a testcase that contains two canvases and uses the drawImage API to draw the contents of one canvas onto the other. Programmatically measure the time this operation takes.

*To prepare for the big switch from 1 threads per canvas to 1 thread for all canvases, add a struct CanvasId (u64) type to components/canvas_traits/canvas.rs and add a CanvasId member to each variant of the CanvasMsg enum.

*add a CanvasId member to Constellation in components/constellation/constellation.rs which is initialized to 0 and increased by 1 each time handle_create_canvas_paint_thread is called.

*make the response_sender argument of handle_create_canvas_paint_thread also include the new CanvasId value, and pass it as an argument to CanvasPaintThread::start. Store the id when it is received for use in all canvas messages.

*For each CanvasMsg that is processed by CanvasPaintThread, verify that the id received matches the id that was provided to CanvasPaintThread::start

==Subsequent Steps:==
*extract the innards of CanvasPaintThread into a CanvasData structure,

*make CanvasPaintThread store a hashtable of CanvasId->CanvasData

*as part of Constellation::start, create a canvas paint thread and store the channel to communicate with it as a member of Constellation. Remove the initial canvas id from the API of Constellation::start.

*when handle_create_canvas_paint_thread is invoked, communicate with the canvas thread and have it create a new entry in the hashtable.

*when the canvas thread receives a message, perform the operation on the appropriate canvas according to the provided id

*optimize the DrawImageInOther operation by drawing on the destination canvas directly, rather than relying on sending a message to another canvas thread. Remove the now-unnecessary IpcSender from the DrawImageInOther enum variant. Verify that the earlier test case demonstrates a performance improvement.

*report on how slither.io performs in Servo after all these changes

=Design Pattern=
Rust only has structs and traits, which are very different from classes and interfaces in terms of usage. Inheritance is effectively nonexistent in Rust, so GoF patterns are nearly always impossible to implement in Rust.

However, the GRASP pattern of Information Expert appears in the Subsequent Steps. CanvasPaintThread is no longer aware of the details of actual canvases, so it delegates the responsibility of actually painting on canvases to CanvasData.

Part of the goal of the Subsequent Steps was to increase the compliance to the Separation of Responsibilities principle. It took an object that previously both processed messages and painted canvases and converted it into an object that processes messages and another object that paints canvases.

=Implementation=
==General Information==

Our repo is [https://github.com/Brody-Eastwood/servo here].

===Building===
After setting up the environment required to develop for Servo, we built and compiled Servo as per the instructions on Servo’s Github [https://github.com/servo/servo repo]. We used Mozilla’s mach tools to build Servo with Cargo, which is the rust package manager.

<pre>
git clone https://github.com/servo/servo
cd servo
./mach build --dev
./mach run tests/html/about-mozilla.html
</pre>

===Testing===
After successfully building servo, we proceeded to test the amount of time spent under “drawImage” as shown below. The function "drawImage" copies the contents of one canvas in a view to another canvas in the same view. During the "Initial Steps," this test was created to test the initial performance of Servo with regards to canvas rendering.

<pre>
var t0 = performance.now();
drawImage(25, 25);
var t1 = performance.now();
document.getElementById('Test Result').innerHTML = "DrawImage took " + (t1 - t0) + " milliseconds.";
console.log("DrawImage took " + (t1 - t0) + " milliseconds.");
</pre>

[[File:TimedDrawImage.png]]

This was/will be used to measure the performance when drawing a single canvas.

===Running Servo===
After each successful build through our development process, we ran Servo as per the instructions on Servo’s Github repo.

<pre>
./servo [url] [arguments] # if you run with nightly build
./mach run [url] [arguments] # if you run with mach

# For example
./mach run https://www.google.com
</pre>

==Initial Steps==

The initial steps were the first OSS project. They did little to nothing to change the actual functionality of Servo, instead setting the Subsequent Steps up to have a smooth transition by finding all of the users of CanvasPaintThread and making them send CanvasIds, which can then be used to change CanvasPaintThread to the single-threaded design desired without having to touch the users again.

===Adding an Id to Canvas Communications===
As shown below, we then associated a CanvasId with each variant of a canvas message; these messages are passed around various parts of the Servo implementation such as the 2D canvas struct, view layout, and various scripts. This required us to make other changed to allow a CanvasId to be sent with each message, as shown below.

{| class="wikitable"
|-
! Before
! After
|-
| pub enum CanvasMsg {
Canvas2d(Canvas2dMsg),
FromLayout(FromLayoutMsg),
FromScript(FromScriptMsg),
Recreate(Size2D<i32>),
Close(),
}

| pub enum CanvasMsg {
Canvas2d(Canvas2dMsg, CanvasId),
FromLayout(FromLayoutMsg, CanvasId),
FromScript(FromScriptMsg, CanvasId),
Recreate(Size2D<i32>, CanvasId),
Close(CanvasId),
}
|}

Then, we made the necessary changed to the constellation and the canvas paint thread to be able to initialize, increment, and store the CanvasId. To finish up with the initial steps, we programmatically checked to make sure that each id that was processed by CanvasPaintThread was the same id that was provided when the thread was started (CanvasPaintThread::start).

<pre>
assert!(canvas_id == painter.canvas_id);
</pre>

===Performance===
Almost every time, http://slither.io/ ended up crashing. Other times, it performed very slowly. However, as of 4/2/2018, much more is left to be done (The Subsequent Steps).

==Subsequent Steps==

The Subsequent Steps were the final project. It builds off of what the Initial Steps lay down. Since the Initial Steps set up all of the users of CanvasPaintThread (even indirect users) to send CanvasIds with their messages, the conversion to use CanvasIds to identify individual canvases was seamless and invisible to the users.

===Overview===
Previously, a CanvasPaintThread was made for every Canvas, which was very slow when there are many Canvases. To fix this, a single CanvasPaintThread was made that manages all the canvases. The data previously stored in CanvasPaintThread unique to each Canvas was moved to a new data structure known as CanvasData.

Previously, communication channels were opened up between each CanvasPaintThread and what uses it and those channels send the CanvasIds established above alongside their messages. Now, there is a single channel that's repeatedly cloned which instead connects the single CanvasPaintThread and those same users. The users will be unable to tell the difference between the current implementation and the new implementation. The CanvasPaintThread will redirect those existing channels into painting using the appropriate CanvasData structure.

These changes involved moving nearly the entire contents of <code>servo/components/canvas/canvas_paint_thread.rs</code> into the new file <code>servo/components/canvas/canvas_data.rs</code>.

There were slight changes to <code>servo/components/constellation/constellation.rs</code> so that it doesn't try to create multiple CanvasPaintThreads, but those changes were minor and well described in the spec.

There were also changes to <code>servo/components/canvas_traits/canvas.rs</code> and <code>servo/components/script/dom/canvasrenderingcontext2d.rs</code> because of the removed parts of Canvas2dMsg::DrawImageInOther

===Initial Form of CanvasPaintThread===
From:
<code>servo/components/canvas/canvas_paint_thread.rs</code>

<code>
pub struct CanvasPaintThread<'a> {
drawtarget: DrawTarget,
/// TODO(pcwalton): Support multiple paths.
path_builder: PathBuilder,
state: CanvasPaintState<'a>,
saved_states: Vec<CanvasPaintState<'a>>,
webrender_api: webrender_api::RenderApi,
image_key: Option<webrender_api::ImageKey>,
/// An old webrender image key that can be deleted when the next epoch ends.
old_image_key: Option<webrender_api::ImageKey>,
/// An old webrender image key that can be deleted when the current epoch ends.
very_old_image_key: Option<webrender_api::ImageKey>,
canvas_id: CanvasId,
}
</code>

CanvasPaintThread initially stored all the information about a particular canvas. This includes both information used to represent the pixels actually on the canvas and information used to support drawing on the canvas.

There were many, many functions in the previous form of CanvasPaintThread because all the actual painting was done there.

CanvasIds were not usefully used. They were simply added in the initial steps so that the grunt work for the subsequent steps were simpler to implement and more focused.

===New Form of CanvasPaintThread===
====CanvasPaintThread====
From:
<code>servo/components/canvas/canvas_paint_thread.rs</code>

<code>
pub struct CanvasPaintThread <'a> {
canvases: HashMap<CanvasId, CanvasData<'a>>,
next_canvas_id: CanvasId,
}
</code>

The new CanvasPaintThread is much leaner. A single CanvasPaintThread exists with nothing but a HashMap of CanvasId->CanvasData structures and a CanvasId to represent what the next CanvasId will be.

The only functions it has are start (for initially starting it and also the thread itself, which passes CanvasMsg enums it receives around), create_canvas (for generating a new CanvasData structure), canvas (which is just a wrapper for the canvases hashmap to hide some ugliness in Rust's syntax), and process_canvas_2d_message (which exists solely because there are a lot of Canvas2dMsg enum variants and making the thread itself pass them around makes the code very hard to read).

Below is a snippet from process_canvas_2d_message, to give an idea as to what CanvasPaintThread does in general. There are far too many variants of the Canvas2dMsg enum to be reasonably put in this space, but the following snippet should give a rather good idea what it does in general.

<code>
match message {
Canvas2dMsg::FillText(text, x, y, max_width) => {
self.canvas(canvas_id).fill_text(text, x, y, max_width)
},
Canvas2dMsg::FillRect(ref rect) => {
self.canvas(canvas_id).fill_rect(rect)
},
Canvas2dMsg::StrokeRect(ref rect) => {
self.canvas(canvas_id).stroke_rect(rect)
},
Canvas2dMsg::ClearRect(ref rect) => {
self.canvas(canvas_id).clear_rect(rect)
},
...
}
</code>

In each Canvas2dMsg enum variant's "match" (think of it like a case statement for a switch but fancier), CanvasPaintThread simply takes the values out of the enum then calls the appropriate function on the appropriate CanvasData structure with those values. It does no checking of those values or otherwise processing them, simply passing them along.

There is one exception to this rule, but the message forces itself to act that way:

<code>
Canvas2dMsg::DrawImageInOther(
other_canvas_id,
image_size,
dest_rect,
source_rect,
smoothing
) => {
let mut image_data = self.canvas(canvas_id).read_pixels(
source_rect.to_i32(),
image_size);
// TODO: avoid double byte_swap.
byte_swap(&mut image_data);
self.canvas(other_canvas_id).draw_image(
image_data.into(),
source_rect.size,
dest_rect,
source_rect,
smoothing,
);
},
}
</code>

It pulls some data out of one CanvasData and then calls a method on the other CanvasData. For this message to be handled, a decision had to be made. Should CanvasData be aware of the existence of ''other'' CanvasData objects or should CanvasPaintThread do some processing to avoid that? There is the third option of making another object to handle it, but seeing as it's only a single message, that option was considered overkill at best, likely far worse. In the end, it was considered a smaller violation of proper design principles to let CanvasPaintThread move data from one CanvasData to another than to inform CanvasData structures about each other.

====CanvasData====
From:
<code>servo/components/canvas/canvas_data.rs</code>

<code>
pub struct CanvasData<'a> {
drawtarget: DrawTarget,
/// TODO(pcwalton): Support multiple paths.
path_builder: PathBuilder,
state: CanvasPaintState<'a>,
saved_states: Vec<CanvasPaintState<'a>>,
webrender_api: webrender_api::RenderApi,
image_key: Option<webrender_api::ImageKey>,
/// An old webrender image key that can be deleted when the next epoch ends.
old_image_key: Option<webrender_api::ImageKey>,
/// An old webrender image key that can be deleted when the current epoch ends.
very_old_image_key: Option<webrender_api::ImageKey>,
canvas_id: CanvasId,
}
</code>

You may notice the similarities between CanvasData and the old form of CanvasPaintThread! Now CanvasData does all the things CanvasPaintThread used to do for the purposes of painting on the canvases (for example, bezier_curve_to, arc, and set_line_cap among many others).

There's nothing particularly ''new'' in CanvasData. It's all just moved from CanvasPaintThread.

It is a very focused class now, though. All it does is paint on canvases. There's no processing of messages, only painting. Moreover, the draw_image_in_other function is nowhere to be found, instead replaced by a small amount of processing done by CanvasPaintThread, which just calls draw_image on the one that was originally considered "other" so there's no communication between the CanvasData structures ''at all''.

===New Form's Improvements===
====Design====
The new form keeps the responsibilities of converting CanvasMsg enums into their contents and using those contents to paint canvases separate, which improves the overall adherence to the Single Responsibility Principle of Servo. There was also a lot of odd indentation and otherwise messy code in the old form of CanvasPaintThread, which, in the process of preparing the pull request for these changes, were cleaned up, making the overall tidyness of Servo improve.

====Performance====
The new implementation is also much faster when multiple canvases are involved because there's no need for threading delay to send messages back and forth, waiting for those messages to be sent and received. There's less room for usage of multiple cores to speed up processing, but Servo is already so ''very'' parallel that there will be ''something'' to keep the other cores busy when CanvasPaintThread is stuck on one thread.

Slither.io doesn't run properly on my machine either before or after the changes, which makes reporting on the success of the changes with respect to Slither.io difficult at best. The framerate improved dramatically, even if it looks like... well... this still. I can't imagine that a website with over 3500 canvases could be made to run more slowly with this change, though.

[[File:Slither.png]]

Using our timed drawimage test, the performance improvement is... well... not really there. On the development build, the drawing time was cut nearly in half from around 19ms to around 9ms, but on the release build, the drawing time is effectively identical, varying between 1.3ms and 1.7ms for both versions. This isn't too surprising, seeing as only two canvases are made and the cost of communicating between two threads isn't very expensive compared to the thousands that the change was made to address.

=Conclusion=
So far we have associated each variant of a canvas message with a particular canvas id. Even though there is not much improvement is the performance of http://slither.io/, we believe that an improvement can be noticed after going through the Subsequent Steps. Here is a link to our [https://github.com/servo/servo/pull/20447 pull request]. On 4/2/2018, our contribution was merged into Servo's master branch!

The rendering of 2D canvases has been entirely moved to a single thread with each canvas identified by a CanvasId. The pull request for the final project is [https://github.com/servo/servo/pull/20680 here]. As of 4/23/2018, the tests are passing and only various "nit"s are being worked through. A merge is expected very soon.

=References=
#https://en.wikipedia.org/wiki/Servo_(layout_engine)
#https://www.rust-lang.org/en-US/
#https://github.com/servo/servo
#https://github.com/servo/servo/wiki/Canvas-rendering-project
#https://www.w3schools.com/graphics/canvas_intro.asp
#https://github.com/servo/servo/issues/10381
#https://github.com/servo/servo/issues/13879

CSC/ECE 517 Spring 2018- Project M1802: 2D Canvas Rendering (Part 2)

2018-04-24T03:05:48Z

Baeastwo: /* CanvasPaintThread */

'''“M1802: Simplify the 2d Canvas Rendering”'''

As robust and fast Servo is, Servo’s implementation of its HTML 2D/3D “<canvas>” has several inefficiencies that make some websites perform slowly or run out of memory when performing complex canvas operations. The goal of this project was to make these websites perform better when loaded in Servo.

=Introduction=
This section briefly describes the main parts of this project: [https://en.wikipedia.org/wiki/Servo_(layout_engine) Servo], [https://www.rust-lang.org/en-US/ Rust], and [https://www.w3schools.com/graphics/canvas_intro.asp HTML Canvas].

==Servo==
Named after a robot from the show Mystery Science Theater 3000, Servo is an experimental web browser layout engine developed by Mozilla. It is currently being developed for 64-bit OS X, 64-bit Linux, 64-bit Windows, and Android. While working with Samsung, Mozilla seeks to create a highly parallel environment where the many things in a web browser like rendering, image decoding, and HTML parsing are handled by isolated tasks. [https://www.youtube.com/watch?v=Ry_RktGLKq4 Here] is a demo on how Servo performs against Firefox.

==Rust==
Since existing languages like C++ did not provide a direct way to achieve a parallel environment, Mozilla developed and uses a programming language called Rust to further develop Servo.

==HTML Canvas==
A <canvas> in HTML is a container for graphics that is used to draw graphics “on the fly” by using Javascript. However, the <canvas> element has no drawing abilities of its own since it only acts as a container for graphics. A script to actually draw the graphics. By using scripts, the HTML <canvas> can:
*Be animated
*Be interactive
*Draw text
*Be used in games
[[File:DrawImage.PNG|An example of several canvases from servo's tests.]]

=Project Description=
The goal of this project was to remove the inefficiencies of Servo’s implemetation of the 2D/3D canvas. As reported with issue [https://github.com/servo/servo/issues/10381 #10381], a gaming website called http://slither.io/ was performing extremely slowly as of Apr 3, 2016. As of Oct 21, 2016, it was reported with issue [https://github.com/servo/servo/issues/13879 #13879] that http://slither.io/ was, in fact, crashing when loaded onto Servo. The suggestion to these issues was to, first, create a test case that measured the amount of time that Servo spends under “drawImage” while doing operations related to a canvas. Another suggestion was to change Servo’s implementation from using one thread per canvas to one thread for all canvases. The [https://github.com/servo/servo/wiki/Canvas-rendering-project steps] laid out for this project by [https://www.mozilla.org/en-US/ Mozilla] (both the initial setup steps and optimizing the canvas implementation) are shown below.

==Initial Steps:==
*create a testcase that contains two canvases and uses the drawImage API to draw the contents of one canvas onto the other. Programmatically measure the time this operation takes.

*To prepare for the big switch from 1 threads per canvas to 1 thread for all canvases, add a struct CanvasId (u64) type to components/canvas_traits/canvas.rs and add a CanvasId member to each variant of the CanvasMsg enum.

*add a CanvasId member to Constellation in components/constellation/constellation.rs which is initialized to 0 and increased by 1 each time handle_create_canvas_paint_thread is called.

*make the response_sender argument of handle_create_canvas_paint_thread also include the new CanvasId value, and pass it as an argument to CanvasPaintThread::start. Store the id when it is received for use in all canvas messages.

*For each CanvasMsg that is processed by CanvasPaintThread, verify that the id received matches the id that was provided to CanvasPaintThread::start

==Subsequent Steps:==
*extract the innards of CanvasPaintThread into a CanvasData structure,

*make CanvasPaintThread store a hashtable of CanvasId->CanvasData

*as part of Constellation::start, create a canvas paint thread and store the channel to communicate with it as a member of Constellation. Remove the initial canvas id from the API of Constellation::start.

*when handle_create_canvas_paint_thread is invoked, communicate with the canvas thread and have it create a new entry in the hashtable.

*when the canvas thread receives a message, perform the operation on the appropriate canvas according to the provided id

*optimize the DrawImageInOther operation by drawing on the destination canvas directly, rather than relying on sending a message to another canvas thread. Remove the now-unnecessary IpcSender from the DrawImageInOther enum variant. Verify that the earlier test case demonstrates a performance improvement.

*report on how slither.io performs in Servo after all these changes

=Design Pattern=
Design patterns were not applicable since our task was to add a few parts to code that already existed. Also, Rust only has structs and traits, which are very different from classes and interfaces in terms of usage. Inheritance is effectively nonexistent in Rust, so GoF patterns are nearly always impossible to implement in Rust.

Part of the goal of the Subsequent Steps was to increase the compliance to the Separation of Responsibilities principle, though. It took an object that previously both processed messages and painted canvases and converted it into an object that processes messages and another object that paints canvases.

=Implementation=
==General Information==

Our repo is [https://github.com/Brody-Eastwood/servo here].

===Building===
After setting up the environment required to develop for Servo, we built and compiled Servo as per the instructions on Servo’s Github [https://github.com/servo/servo repo]. We used Mozilla’s mach tools to build Servo with Cargo, which is the rust package manager.

<pre>
git clone https://github.com/servo/servo
cd servo
./mach build --dev
./mach run tests/html/about-mozilla.html
</pre>

===Testing===
After successfully building servo, we proceeded to test the amount of time spent under “drawImage” as shown below. The function "drawImage" copies the contents of one canvas in a view to another canvas in the same view. During the "Initial Steps," this test was created to test the initial performance of Servo with regards to canvas rendering.

<pre>
var t0 = performance.now();
drawImage(25, 25);
var t1 = performance.now();
document.getElementById('Test Result').innerHTML = "DrawImage took " + (t1 - t0) + " milliseconds.";
console.log("DrawImage took " + (t1 - t0) + " milliseconds.");
</pre>

[[File:TimedDrawImage.png]]

This was/will be used to measure the performance when drawing a single canvas.

===Running Servo===
After each successful build through our development process, we ran Servo as per the instructions on Servo’s Github repo.

<pre>
./servo [url] [arguments] # if you run with nightly build
./mach run [url] [arguments] # if you run with mach

# For example
./mach run https://www.google.com
</pre>

==Initial Steps==

The initial steps were the first OSS project. They did little to nothing to change the actual functionality of Servo, instead setting the Subsequent Steps up to have a smooth transition by finding all of the users of CanvasPaintThread and making them send CanvasIds, which can then be used to change CanvasPaintThread to the single-threaded design desired without having to touch the users again.

===Adding an Id to Canvas Communications===
As shown below, we then associated a CanvasId with each variant of a canvas message; these messages are passed around various parts of the Servo implementation such as the 2D canvas struct, view layout, and various scripts. This required us to make other changed to allow a CanvasId to be sent with each message, as shown below.

{| class="wikitable"
|-
! Before
! After
|-
| pub enum CanvasMsg {
Canvas2d(Canvas2dMsg),
FromLayout(FromLayoutMsg),
FromScript(FromScriptMsg),
Recreate(Size2D<i32>),
Close(),
}

| pub enum CanvasMsg {
Canvas2d(Canvas2dMsg, CanvasId),
FromLayout(FromLayoutMsg, CanvasId),
FromScript(FromScriptMsg, CanvasId),
Recreate(Size2D<i32>, CanvasId),
Close(CanvasId),
}
|}

Then, we made the necessary changed to the constellation and the canvas paint thread to be able to initialize, increment, and store the CanvasId. To finish up with the initial steps, we programmatically checked to make sure that each id that was processed by CanvasPaintThread was the same id that was provided when the thread was started (CanvasPaintThread::start).

<pre>
assert!(canvas_id == painter.canvas_id);
</pre>

===Performance===
Almost every time, http://slither.io/ ended up crashing. Other times, it performed very slowly. However, as of 4/2/2018, much more is left to be done (The Subsequent Steps).

==Subsequent Steps==

The Subsequent Steps were the final project. It builds off of what the Initial Steps lay down. Since the Initial Steps set up all of the users of CanvasPaintThread (even indirect users) to send CanvasIds with their messages, the conversion to use CanvasIds to identify individual canvases was seamless and invisible to the users.

===Overview===
Previously, a CanvasPaintThread was made for every Canvas, which was very slow when there are many Canvases. To fix this, a single CanvasPaintThread was made that manages all the canvases. The data previously stored in CanvasPaintThread unique to each Canvas was moved to a new data structure known as CanvasData.

Previously, communication channels were opened up between each CanvasPaintThread and what uses it and those channels send the CanvasIds established above alongside their messages. Now, there is a single channel that's repeatedly cloned which instead connects the single CanvasPaintThread and those same users. The users will be unable to tell the difference between the current implementation and the new implementation. The CanvasPaintThread will redirect those existing channels into painting using the appropriate CanvasData structure.

These changes involved moving nearly the entire contents of <code>servo/components/canvas/canvas_paint_thread.rs</code> into the new file <code>servo/components/canvas/canvas_data.rs</code>.

There were slight changes to <code>servo/components/constellation/constellation.rs</code> so that it doesn't try to create multiple CanvasPaintThreads, but those changes were minor and well described in the spec.

There were also changes to <code>servo/components/canvas_traits/canvas.rs</code> and <code>servo/components/script/dom/canvasrenderingcontext2d.rs</code> because of the removed parts of Canvas2dMsg::DrawImageInOther

===Initial Form of CanvasPaintThread===
From:
<code>servo/components/canvas/canvas_paint_thread.rs</code>

<code>
pub struct CanvasPaintThread<'a> {
drawtarget: DrawTarget,
/// TODO(pcwalton): Support multiple paths.
path_builder: PathBuilder,
state: CanvasPaintState<'a>,
saved_states: Vec<CanvasPaintState<'a>>,
webrender_api: webrender_api::RenderApi,
image_key: Option<webrender_api::ImageKey>,
/// An old webrender image key that can be deleted when the next epoch ends.
old_image_key: Option<webrender_api::ImageKey>,
/// An old webrender image key that can be deleted when the current epoch ends.
very_old_image_key: Option<webrender_api::ImageKey>,
canvas_id: CanvasId,
}
</code>

CanvasPaintThread initially stored all the information about a particular canvas. This includes both information used to represent the pixels actually on the canvas and information used to support drawing on the canvas.

There were many, many functions in the previous form of CanvasPaintThread because all the actual painting was done there.

CanvasIds were not usefully used. They were simply added in the initial steps so that the grunt work for the subsequent steps were simpler to implement and more focused.

===New Form of CanvasPaintThread===
====CanvasPaintThread====
From:
<code>servo/components/canvas/canvas_paint_thread.rs</code>

<code>
pub struct CanvasPaintThread <'a> {
canvases: HashMap<CanvasId, CanvasData<'a>>,
next_canvas_id: CanvasId,
}
</code>

The new CanvasPaintThread is much leaner. A single CanvasPaintThread exists with nothing but a HashMap of CanvasId->CanvasData structures and a CanvasId to represent what the next CanvasId will be.

The only functions it has are start (for initially starting it and also the thread itself, which passes CanvasMsg enums it receives around), create_canvas (for generating a new CanvasData structure), canvas (which is just a wrapper for the canvases hashmap to hide some ugliness in Rust's syntax), and process_canvas_2d_message (which exists solely because there are a lot of Canvas2dMsg enum variants and making the thread itself pass them around makes the code very hard to read).

Below is a snippet from process_canvas_2d_message, to give an idea as to what CanvasPaintThread does in general. There are far too many variants of the Canvas2dMsg enum to be reasonably put in this space, but the following snippet should give a rather good idea what it does in general.

<code>
match message {
Canvas2dMsg::FillText(text, x, y, max_width) => {
self.canvas(canvas_id).fill_text(text, x, y, max_width)
},
Canvas2dMsg::FillRect(ref rect) => {
self.canvas(canvas_id).fill_rect(rect)
},
Canvas2dMsg::StrokeRect(ref rect) => {
self.canvas(canvas_id).stroke_rect(rect)
},
Canvas2dMsg::ClearRect(ref rect) => {
self.canvas(canvas_id).clear_rect(rect)
},
...
}
</code>

In each Canvas2dMsg enum variant's "match" (think of it like a case statement for a switch but fancier), CanvasPaintThread simply takes the values out of the enum then calls the appropriate function on the appropriate CanvasData structure with those values. It does no checking of those values or otherwise processing them, simply passing them along.

There is one exception to this rule, but the message forces itself to act that way:

<code>
Canvas2dMsg::DrawImageInOther(
other_canvas_id,
image_size,
dest_rect,
source_rect,
smoothing
) => {
let mut image_data = self.canvas(canvas_id).read_pixels(
source_rect.to_i32(),
image_size);
// TODO: avoid double byte_swap.
byte_swap(&mut image_data);
self.canvas(other_canvas_id).draw_image(
image_data.into(),
source_rect.size,
dest_rect,
source_rect,
smoothing,
);
},
}
</code>

It pulls some data out of one CanvasData and then calls a method on the other CanvasData. For this message to be handled, a decision had to be made. Should CanvasData be aware of the existence of ''other'' CanvasData objects or should CanvasPaintThread do some processing to avoid that? There is the third option of making another object to handle it, but seeing as it's only a single message, that option was considered overkill at best, likely far worse. In the end, it was considered a smaller violation of proper design principles to let CanvasPaintThread move data from one CanvasData to another than to inform CanvasData structures about each other.

====CanvasData====
From:
<code>servo/components/canvas/canvas_data.rs</code>

<code>
pub struct CanvasData<'a> {
drawtarget: DrawTarget,
/// TODO(pcwalton): Support multiple paths.
path_builder: PathBuilder,
state: CanvasPaintState<'a>,
saved_states: Vec<CanvasPaintState<'a>>,
webrender_api: webrender_api::RenderApi,
image_key: Option<webrender_api::ImageKey>,
/// An old webrender image key that can be deleted when the next epoch ends.
old_image_key: Option<webrender_api::ImageKey>,
/// An old webrender image key that can be deleted when the current epoch ends.
very_old_image_key: Option<webrender_api::ImageKey>,
canvas_id: CanvasId,
}
</code>

You may notice the similarities between CanvasData and the old form of CanvasPaintThread! Now CanvasData does all the things CanvasPaintThread used to do for the purposes of painting on the canvases (for example, bezier_curve_to, arc, and set_line_cap among many others).

There's nothing particularly ''new'' in CanvasData. It's all just moved from CanvasPaintThread.

It is a very focused class now, though. All it does is paint on canvases. There's no processing of messages, only painting. Moreover, the draw_image_in_other function is nowhere to be found, instead replaced by a small amount of processing done by CanvasPaintThread, which just calls draw_image on the one that was originally considered "other" so there's no communication between the CanvasData structures ''at all''.

===New Form's Improvements===
====Design====
The new form keeps the responsibilities of converting CanvasMsg enums into their contents and using those contents to paint canvases separate, which improves the overall adherence to the Single Responsibility Principle of Servo. There was also a lot of odd indentation and otherwise messy code in the old form of CanvasPaintThread, which, in the process of preparing the pull request for these changes, were cleaned up, making the overall tidyness of Servo improve.

====Performance====
The new implementation is also much faster when multiple canvases are involved because there's no need for threading delay to send messages back and forth, waiting for those messages to be sent and received. There's less room for usage of multiple cores to speed up processing, but Servo is already so ''very'' parallel that there will be ''something'' to keep the other cores busy when CanvasPaintThread is stuck on one thread.

Slither.io doesn't run properly on my machine either before or after the changes, which makes reporting on the success of the changes with respect to Slither.io difficult at best. The framerate improved dramatically, even if it looks like... well... this still. I can't imagine that a website with over 3500 canvases could be made to run more slowly with this change, though.

[[File:Slither.png]]

Using our timed drawimage test, the performance improvement is... well... not really there. On the development build, the drawing time was cut nearly in half from around 19ms to around 9ms, but on the release build, the drawing time is effectively identical, varying between 1.3ms and 1.7ms for both versions. This isn't too surprising, seeing as only two canvases are made and the cost of communicating between two threads isn't very expensive compared to the thousands that the change was made to address.

=Conclusion=
So far we have associated each variant of a canvas message with a particular canvas id. Even though there is not much improvement is the performance of http://slither.io/, we believe that an improvement can be noticed after going through the Subsequent Steps. Here is a link to our [https://github.com/servo/servo/pull/20447 pull request]. On 4/2/2018, our contribution was merged into Servo's master branch!

The rendering of 2D canvases has been entirely moved to a single thread with each canvas identified by a CanvasId. The pull request for the final project is [https://github.com/servo/servo/pull/20680 here]. As of 4/23/2018, the tests are passing and only various "nit"s are being worked through. A merge is expected very soon.

=References=
#https://en.wikipedia.org/wiki/Servo_(layout_engine)
#https://www.rust-lang.org/en-US/
#https://github.com/servo/servo
#https://github.com/servo/servo/wiki/Canvas-rendering-project
#https://www.w3schools.com/graphics/canvas_intro.asp
#https://github.com/servo/servo/issues/10381
#https://github.com/servo/servo/issues/13879

CSC/ECE 517 Spring 2018- Project M1802: 2D Canvas Rendering (Part 2)

2018-04-24T03:05:35Z

Baeastwo: /* CanvasPaintThread */

'''“M1802: Simplify the 2d Canvas Rendering”'''

As robust and fast Servo is, Servo’s implementation of its HTML 2D/3D “<canvas>” has several inefficiencies that make some websites perform slowly or run out of memory when performing complex canvas operations. The goal of this project was to make these websites perform better when loaded in Servo.

=Introduction=
This section briefly describes the main parts of this project: [https://en.wikipedia.org/wiki/Servo_(layout_engine) Servo], [https://www.rust-lang.org/en-US/ Rust], and [https://www.w3schools.com/graphics/canvas_intro.asp HTML Canvas].

==Servo==
Named after a robot from the show Mystery Science Theater 3000, Servo is an experimental web browser layout engine developed by Mozilla. It is currently being developed for 64-bit OS X, 64-bit Linux, 64-bit Windows, and Android. While working with Samsung, Mozilla seeks to create a highly parallel environment where the many things in a web browser like rendering, image decoding, and HTML parsing are handled by isolated tasks. [https://www.youtube.com/watch?v=Ry_RktGLKq4 Here] is a demo on how Servo performs against Firefox.

==Rust==
Since existing languages like C++ did not provide a direct way to achieve a parallel environment, Mozilla developed and uses a programming language called Rust to further develop Servo.

==HTML Canvas==
A <canvas> in HTML is a container for graphics that is used to draw graphics “on the fly” by using Javascript. However, the <canvas> element has no drawing abilities of its own since it only acts as a container for graphics. A script to actually draw the graphics. By using scripts, the HTML <canvas> can:
*Be animated
*Be interactive
*Draw text
*Be used in games
[[File:DrawImage.PNG|An example of several canvases from servo's tests.]]

=Project Description=
The goal of this project was to remove the inefficiencies of Servo’s implemetation of the 2D/3D canvas. As reported with issue [https://github.com/servo/servo/issues/10381 #10381], a gaming website called http://slither.io/ was performing extremely slowly as of Apr 3, 2016. As of Oct 21, 2016, it was reported with issue [https://github.com/servo/servo/issues/13879 #13879] that http://slither.io/ was, in fact, crashing when loaded onto Servo. The suggestion to these issues was to, first, create a test case that measured the amount of time that Servo spends under “drawImage” while doing operations related to a canvas. Another suggestion was to change Servo’s implementation from using one thread per canvas to one thread for all canvases. The [https://github.com/servo/servo/wiki/Canvas-rendering-project steps] laid out for this project by [https://www.mozilla.org/en-US/ Mozilla] (both the initial setup steps and optimizing the canvas implementation) are shown below.

==Initial Steps:==
*create a testcase that contains two canvases and uses the drawImage API to draw the contents of one canvas onto the other. Programmatically measure the time this operation takes.

*To prepare for the big switch from 1 threads per canvas to 1 thread for all canvases, add a struct CanvasId (u64) type to components/canvas_traits/canvas.rs and add a CanvasId member to each variant of the CanvasMsg enum.

*add a CanvasId member to Constellation in components/constellation/constellation.rs which is initialized to 0 and increased by 1 each time handle_create_canvas_paint_thread is called.

*make the response_sender argument of handle_create_canvas_paint_thread also include the new CanvasId value, and pass it as an argument to CanvasPaintThread::start. Store the id when it is received for use in all canvas messages.

*For each CanvasMsg that is processed by CanvasPaintThread, verify that the id received matches the id that was provided to CanvasPaintThread::start

==Subsequent Steps:==
*extract the innards of CanvasPaintThread into a CanvasData structure,

*make CanvasPaintThread store a hashtable of CanvasId->CanvasData

*as part of Constellation::start, create a canvas paint thread and store the channel to communicate with it as a member of Constellation. Remove the initial canvas id from the API of Constellation::start.

*when handle_create_canvas_paint_thread is invoked, communicate with the canvas thread and have it create a new entry in the hashtable.

*when the canvas thread receives a message, perform the operation on the appropriate canvas according to the provided id

*optimize the DrawImageInOther operation by drawing on the destination canvas directly, rather than relying on sending a message to another canvas thread. Remove the now-unnecessary IpcSender from the DrawImageInOther enum variant. Verify that the earlier test case demonstrates a performance improvement.

*report on how slither.io performs in Servo after all these changes

=Design Pattern=
Design patterns were not applicable since our task was to add a few parts to code that already existed. Also, Rust only has structs and traits, which are very different from classes and interfaces in terms of usage. Inheritance is effectively nonexistent in Rust, so GoF patterns are nearly always impossible to implement in Rust.

Part of the goal of the Subsequent Steps was to increase the compliance to the Separation of Responsibilities principle, though. It took an object that previously both processed messages and painted canvases and converted it into an object that processes messages and another object that paints canvases.

=Implementation=
==General Information==

Our repo is [https://github.com/Brody-Eastwood/servo here].

===Building===
After setting up the environment required to develop for Servo, we built and compiled Servo as per the instructions on Servo’s Github [https://github.com/servo/servo repo]. We used Mozilla’s mach tools to build Servo with Cargo, which is the rust package manager.

<pre>
git clone https://github.com/servo/servo
cd servo
./mach build --dev
./mach run tests/html/about-mozilla.html
</pre>

===Testing===
After successfully building servo, we proceeded to test the amount of time spent under “drawImage” as shown below. The function "drawImage" copies the contents of one canvas in a view to another canvas in the same view. During the "Initial Steps," this test was created to test the initial performance of Servo with regards to canvas rendering.

<pre>
var t0 = performance.now();
drawImage(25, 25);
var t1 = performance.now();
document.getElementById('Test Result').innerHTML = "DrawImage took " + (t1 - t0) + " milliseconds.";
console.log("DrawImage took " + (t1 - t0) + " milliseconds.");
</pre>

[[File:TimedDrawImage.png]]

This was/will be used to measure the performance when drawing a single canvas.

===Running Servo===
After each successful build through our development process, we ran Servo as per the instructions on Servo’s Github repo.

<pre>
./servo [url] [arguments] # if you run with nightly build
./mach run [url] [arguments] # if you run with mach

# For example
./mach run https://www.google.com
</pre>

==Initial Steps==

The initial steps were the first OSS project. They did little to nothing to change the actual functionality of Servo, instead setting the Subsequent Steps up to have a smooth transition by finding all of the users of CanvasPaintThread and making them send CanvasIds, which can then be used to change CanvasPaintThread to the single-threaded design desired without having to touch the users again.

===Adding an Id to Canvas Communications===
As shown below, we then associated a CanvasId with each variant of a canvas message; these messages are passed around various parts of the Servo implementation such as the 2D canvas struct, view layout, and various scripts. This required us to make other changed to allow a CanvasId to be sent with each message, as shown below.

{| class="wikitable"
|-
! Before
! After
|-
| pub enum CanvasMsg {
Canvas2d(Canvas2dMsg),
FromLayout(FromLayoutMsg),
FromScript(FromScriptMsg),
Recreate(Size2D<i32>),
Close(),
}

| pub enum CanvasMsg {
Canvas2d(Canvas2dMsg, CanvasId),
FromLayout(FromLayoutMsg, CanvasId),
FromScript(FromScriptMsg, CanvasId),
Recreate(Size2D<i32>, CanvasId),
Close(CanvasId),
}
|}

Then, we made the necessary changed to the constellation and the canvas paint thread to be able to initialize, increment, and store the CanvasId. To finish up with the initial steps, we programmatically checked to make sure that each id that was processed by CanvasPaintThread was the same id that was provided when the thread was started (CanvasPaintThread::start).

<pre>
assert!(canvas_id == painter.canvas_id);
</pre>

===Performance===
Almost every time, http://slither.io/ ended up crashing. Other times, it performed very slowly. However, as of 4/2/2018, much more is left to be done (The Subsequent Steps).

==Subsequent Steps==

The Subsequent Steps were the final project. It builds off of what the Initial Steps lay down. Since the Initial Steps set up all of the users of CanvasPaintThread (even indirect users) to send CanvasIds with their messages, the conversion to use CanvasIds to identify individual canvases was seamless and invisible to the users.

===Overview===
Previously, a CanvasPaintThread was made for every Canvas, which was very slow when there are many Canvases. To fix this, a single CanvasPaintThread was made that manages all the canvases. The data previously stored in CanvasPaintThread unique to each Canvas was moved to a new data structure known as CanvasData.

Previously, communication channels were opened up between each CanvasPaintThread and what uses it and those channels send the CanvasIds established above alongside their messages. Now, there is a single channel that's repeatedly cloned which instead connects the single CanvasPaintThread and those same users. The users will be unable to tell the difference between the current implementation and the new implementation. The CanvasPaintThread will redirect those existing channels into painting using the appropriate CanvasData structure.

These changes involved moving nearly the entire contents of <code>servo/components/canvas/canvas_paint_thread.rs</code> into the new file <code>servo/components/canvas/canvas_data.rs</code>.

There were slight changes to <code>servo/components/constellation/constellation.rs</code> so that it doesn't try to create multiple CanvasPaintThreads, but those changes were minor and well described in the spec.

There were also changes to <code>servo/components/canvas_traits/canvas.rs</code> and <code>servo/components/script/dom/canvasrenderingcontext2d.rs</code> because of the removed parts of Canvas2dMsg::DrawImageInOther

===Initial Form of CanvasPaintThread===
From:
<code>servo/components/canvas/canvas_paint_thread.rs</code>

<code>
pub struct CanvasPaintThread<'a> {
drawtarget: DrawTarget,
/// TODO(pcwalton): Support multiple paths.
path_builder: PathBuilder,
state: CanvasPaintState<'a>,
saved_states: Vec<CanvasPaintState<'a>>,
webrender_api: webrender_api::RenderApi,
image_key: Option<webrender_api::ImageKey>,
/// An old webrender image key that can be deleted when the next epoch ends.
old_image_key: Option<webrender_api::ImageKey>,
/// An old webrender image key that can be deleted when the current epoch ends.
very_old_image_key: Option<webrender_api::ImageKey>,
canvas_id: CanvasId,
}
</code>

CanvasPaintThread initially stored all the information about a particular canvas. This includes both information used to represent the pixels actually on the canvas and information used to support drawing on the canvas.

There were many, many functions in the previous form of CanvasPaintThread because all the actual painting was done there.

CanvasIds were not usefully used. They were simply added in the initial steps so that the grunt work for the subsequent steps were simpler to implement and more focused.

===New Form of CanvasPaintThread===
====CanvasPaintThread====
From:
<code>servo/components/canvas/canvas_paint_thread.rs</code>

<code>
pub struct CanvasPaintThread <'a> {
canvases: HashMap<CanvasId, CanvasData<'a>>,
next_canvas_id: CanvasId,
}
</code>

The new CanvasPaintThread is much leaner. A single CanvasPaintThread exists with nothing but a HashMap of CanvasId->CanvasData structures and a CanvasId to represent what the next CanvasId will be.

The only functions it has are start (for initially starting it and also the thread itself, which passes CanvasMsg enums it receives around), create_canvas (for generating a new CanvasData structure), canvas (which is just a wrapper for the canvases hashmap to hide some ugliness in Rust's syntax), and process_canvas_2d_message (which exists solely because there are a lot of Canvas2dMsg enum variants and making the thread itself pass them around makes the code very hard to read).

Below is a snippet from process_canvas_2d_message, to give an idea as to what CanvasPaintThread does in general. There are far too many variants of the Canvas2dMsg enum to be reasonably put in this space, but the following snippet should give a rather good idea what it does in general.

<code>
match message {
Canvas2dMsg::FillText(text, x, y, max_width) => {
self.canvas(canvas_id).fill_text(text, x, y, max_width)
},
Canvas2dMsg::FillRect(ref rect) => {
self.canvas(canvas_id).fill_rect(rect)
},
Canvas2dMsg::StrokeRect(ref rect) => {
self.canvas(canvas_id).stroke_rect(rect)
},
Canvas2dMsg::ClearRect(ref rect) => {
self.canvas(canvas_id).clear_rect(rect)
},
...

}
</code>

In each Canvas2dMsg enum variant's "match" (think of it like a case statement for a switch but fancier), CanvasPaintThread simply takes the values out of the enum then calls the appropriate function on the appropriate CanvasData structure with those values. It does no checking of those values or otherwise processing them, simply passing them along.

There is one exception to this rule, but the message forces itself to act that way:

<code>
Canvas2dMsg::DrawImageInOther(
other_canvas_id,
image_size,
dest_rect,
source_rect,
smoothing
) => {
let mut image_data = self.canvas(canvas_id).read_pixels(
source_rect.to_i32(),
image_size);
// TODO: avoid double byte_swap.
byte_swap(&mut image_data);
self.canvas(other_canvas_id).draw_image(
image_data.into(),
source_rect.size,
dest_rect,
source_rect,
smoothing,
);
},
}
</code>

It pulls some data out of one CanvasData and then calls a method on the other CanvasData. For this message to be handled, a decision had to be made. Should CanvasData be aware of the existence of ''other'' CanvasData objects or should CanvasPaintThread do some processing to avoid that? There is the third option of making another object to handle it, but seeing as it's only a single message, that option was considered overkill at best, likely far worse. In the end, it was considered a smaller violation of proper design principles to let CanvasPaintThread move data from one CanvasData to another than to inform CanvasData structures about each other.

====CanvasData====
From:
<code>servo/components/canvas/canvas_data.rs</code>

<code>
pub struct CanvasData<'a> {
drawtarget: DrawTarget,
/// TODO(pcwalton): Support multiple paths.
path_builder: PathBuilder,
state: CanvasPaintState<'a>,
saved_states: Vec<CanvasPaintState<'a>>,
webrender_api: webrender_api::RenderApi,
image_key: Option<webrender_api::ImageKey>,
/// An old webrender image key that can be deleted when the next epoch ends.
old_image_key: Option<webrender_api::ImageKey>,
/// An old webrender image key that can be deleted when the current epoch ends.
very_old_image_key: Option<webrender_api::ImageKey>,
canvas_id: CanvasId,
}
</code>

You may notice the similarities between CanvasData and the old form of CanvasPaintThread! Now CanvasData does all the things CanvasPaintThread used to do for the purposes of painting on the canvases (for example, bezier_curve_to, arc, and set_line_cap among many others).

There's nothing particularly ''new'' in CanvasData. It's all just moved from CanvasPaintThread.

It is a very focused class now, though. All it does is paint on canvases. There's no processing of messages, only painting. Moreover, the draw_image_in_other function is nowhere to be found, instead replaced by a small amount of processing done by CanvasPaintThread, which just calls draw_image on the one that was originally considered "other" so there's no communication between the CanvasData structures ''at all''.

===New Form's Improvements===
====Design====
The new form keeps the responsibilities of converting CanvasMsg enums into their contents and using those contents to paint canvases separate, which improves the overall adherence to the Single Responsibility Principle of Servo. There was also a lot of odd indentation and otherwise messy code in the old form of CanvasPaintThread, which, in the process of preparing the pull request for these changes, were cleaned up, making the overall tidyness of Servo improve.

====Performance====
The new implementation is also much faster when multiple canvases are involved because there's no need for threading delay to send messages back and forth, waiting for those messages to be sent and received. There's less room for usage of multiple cores to speed up processing, but Servo is already so ''very'' parallel that there will be ''something'' to keep the other cores busy when CanvasPaintThread is stuck on one thread.

Slither.io doesn't run properly on my machine either before or after the changes, which makes reporting on the success of the changes with respect to Slither.io difficult at best. The framerate improved dramatically, even if it looks like... well... this still. I can't imagine that a website with over 3500 canvases could be made to run more slowly with this change, though.

[[File:Slither.png]]

Using our timed drawimage test, the performance improvement is... well... not really there. On the development build, the drawing time was cut nearly in half from around 19ms to around 9ms, but on the release build, the drawing time is effectively identical, varying between 1.3ms and 1.7ms for both versions. This isn't too surprising, seeing as only two canvases are made and the cost of communicating between two threads isn't very expensive compared to the thousands that the change was made to address.

=Conclusion=
So far we have associated each variant of a canvas message with a particular canvas id. Even though there is not much improvement is the performance of http://slither.io/, we believe that an improvement can be noticed after going through the Subsequent Steps. Here is a link to our [https://github.com/servo/servo/pull/20447 pull request]. On 4/2/2018, our contribution was merged into Servo's master branch!

The rendering of 2D canvases has been entirely moved to a single thread with each canvas identified by a CanvasId. The pull request for the final project is [https://github.com/servo/servo/pull/20680 here]. As of 4/23/2018, the tests are passing and only various "nit"s are being worked through. A merge is expected very soon.

=References=
#https://en.wikipedia.org/wiki/Servo_(layout_engine)
#https://www.rust-lang.org/en-US/
#https://github.com/servo/servo
#https://github.com/servo/servo/wiki/Canvas-rendering-project
#https://www.w3schools.com/graphics/canvas_intro.asp
#https://github.com/servo/servo/issues/10381
#https://github.com/servo/servo/issues/13879

CSC/ECE 517 Spring 2018- Project M1802: 2D Canvas Rendering (Part 2)

2018-04-24T02:50:04Z

Baeastwo: /* Initial Steps */

'''“M1802: Simplify the 2d Canvas Rendering”'''

As robust and fast Servo is, Servo’s implementation of its HTML 2D/3D “<canvas>” has several inefficiencies that make some websites perform slowly or run out of memory when performing complex canvas operations. The goal of this project was to make these websites perform better when loaded in Servo.

=Introduction=
This section briefly describes the main parts of this project: [https://en.wikipedia.org/wiki/Servo_(layout_engine) Servo], [https://www.rust-lang.org/en-US/ Rust], and [https://www.w3schools.com/graphics/canvas_intro.asp HTML Canvas].

==Servo==
Named after a robot from the show Mystery Science Theater 3000, Servo is an experimental web browser layout engine developed by Mozilla. It is currently being developed for 64-bit OS X, 64-bit Linux, 64-bit Windows, and Android. While working with Samsung, Mozilla seeks to create a highly parallel environment where the many things in a web browser like rendering, image decoding, and HTML parsing are handled by isolated tasks. [https://www.youtube.com/watch?v=Ry_RktGLKq4 Here] is a demo on how Servo performs against Firefox.

==Rust==
Since existing languages like C++ did not provide a direct way to achieve a parallel environment, Mozilla developed and uses a programming language called Rust to further develop Servo.

==HTML Canvas==
A <canvas> in HTML is a container for graphics that is used to draw graphics “on the fly” by using Javascript. However, the <canvas> element has no drawing abilities of its own since it only acts as a container for graphics. A script to actually draw the graphics. By using scripts, the HTML <canvas> can:
*Be animated
*Be interactive
*Draw text
*Be used in games
[[File:DrawImage.PNG|An example of several canvases from servo's tests.]]

=Project Description=
The goal of this project was to remove the inefficiencies of Servo’s implemetation of the 2D/3D canvas. As reported with issue [https://github.com/servo/servo/issues/10381 #10381], a gaming website called http://slither.io/ was performing extremely slowly as of Apr 3, 2016. As of Oct 21, 2016, it was reported with issue [https://github.com/servo/servo/issues/13879 #13879] that http://slither.io/ was, in fact, crashing when loaded onto Servo. The suggestion to these issues was to, first, create a test case that measured the amount of time that Servo spends under “drawImage” while doing operations related to a canvas. Another suggestion was to change Servo’s implementation from using one thread per canvas to one thread for all canvases. The [https://github.com/servo/servo/wiki/Canvas-rendering-project steps] laid out for this project by [https://www.mozilla.org/en-US/ Mozilla] (both the initial setup steps and optimizing the canvas implementation) are shown below.

==Initial Steps:==
*create a testcase that contains two canvases and uses the drawImage API to draw the contents of one canvas onto the other. Programmatically measure the time this operation takes.

*To prepare for the big switch from 1 threads per canvas to 1 thread for all canvases, add a struct CanvasId (u64) type to components/canvas_traits/canvas.rs and add a CanvasId member to each variant of the CanvasMsg enum.

*add a CanvasId member to Constellation in components/constellation/constellation.rs which is initialized to 0 and increased by 1 each time handle_create_canvas_paint_thread is called.

*make the response_sender argument of handle_create_canvas_paint_thread also include the new CanvasId value, and pass it as an argument to CanvasPaintThread::start. Store the id when it is received for use in all canvas messages.

*For each CanvasMsg that is processed by CanvasPaintThread, verify that the id received matches the id that was provided to CanvasPaintThread::start

==Subsequent Steps:==
*extract the innards of CanvasPaintThread into a CanvasData structure,

*make CanvasPaintThread store a hashtable of CanvasId->CanvasData

*as part of Constellation::start, create a canvas paint thread and store the channel to communicate with it as a member of Constellation. Remove the initial canvas id from the API of Constellation::start.

*when handle_create_canvas_paint_thread is invoked, communicate with the canvas thread and have it create a new entry in the hashtable.

*when the canvas thread receives a message, perform the operation on the appropriate canvas according to the provided id

*optimize the DrawImageInOther operation by drawing on the destination canvas directly, rather than relying on sending a message to another canvas thread. Remove the now-unnecessary IpcSender from the DrawImageInOther enum variant. Verify that the earlier test case demonstrates a performance improvement.

*report on how slither.io performs in Servo after all these changes

=Design Pattern=
Design patterns were not applicable since our task was to add a few parts to code that already existed. Also, Rust only has structs and traits, which are very different from classes and interfaces in terms of usage. Inheritance is effectively nonexistent in Rust, so GoF patterns are nearly always impossible to implement in Rust.

Part of the goal of the Subsequent Steps was to increase the compliance to the Separation of Responsibilities principle, though. It took an object that previously both processed messages and painted canvases and converted it into an object that processes messages and another object that paints canvases.

=Implementation=
==General Information==

Our repo is [https://github.com/Brody-Eastwood/servo here].

===Building===
After setting up the environment required to develop for Servo, we built and compiled Servo as per the instructions on Servo’s Github [https://github.com/servo/servo repo]. We used Mozilla’s mach tools to build Servo with Cargo, which is the rust package manager.

<pre>
git clone https://github.com/servo/servo
cd servo
./mach build --dev
./mach run tests/html/about-mozilla.html
</pre>

===Testing===
After successfully building servo, we proceeded to test the amount of time spent under “drawImage” as shown below. The function "drawImage" copies the contents of one canvas in a view to another canvas in the same view. During the "Initial Steps," this test was created to test the initial performance of Servo with regards to canvas rendering.

<pre>
var t0 = performance.now();
drawImage(25, 25);
var t1 = performance.now();
document.getElementById('Test Result').innerHTML = "DrawImage took " + (t1 - t0) + " milliseconds.";
console.log("DrawImage took " + (t1 - t0) + " milliseconds.");
</pre>

[[File:TimedDrawImage.png]]

This was/will be used to measure the performance when drawing a single canvas.

===Running Servo===
After each successful build through our development process, we ran Servo as per the instructions on Servo’s Github repo.

<pre>
./servo [url] [arguments] # if you run with nightly build
./mach run [url] [arguments] # if you run with mach

# For example
./mach run https://www.google.com
</pre>

==Initial Steps==

The initial steps were the first OSS project. They did little to nothing to change the actual functionality of Servo, instead setting the Subsequent Steps up to have a smooth transition by finding all of the users of CanvasPaintThread and making them send CanvasIds, which can then be used to change CanvasPaintThread to the single-threaded design desired without having to touch the users again.

===Adding an Id to Canvas Communications===
As shown below, we then associated a CanvasId with each variant of a canvas message; these messages are passed around various parts of the Servo implementation such as the 2D canvas struct, view layout, and various scripts. This required us to make other changed to allow a CanvasId to be sent with each message, as shown below.

{| class="wikitable"
|-
! Before
! After
|-
| pub enum CanvasMsg {
Canvas2d(Canvas2dMsg),
FromLayout(FromLayoutMsg),
FromScript(FromScriptMsg),
Recreate(Size2D<i32>),
Close(),
}

| pub enum CanvasMsg {
Canvas2d(Canvas2dMsg, CanvasId),
FromLayout(FromLayoutMsg, CanvasId),
FromScript(FromScriptMsg, CanvasId),
Recreate(Size2D<i32>, CanvasId),
Close(CanvasId),
}
|}

Then, we made the necessary changed to the constellation and the canvas paint thread to be able to initialize, increment, and store the CanvasId. To finish up with the initial steps, we programmatically checked to make sure that each id that was processed by CanvasPaintThread was the same id that was provided when the thread was started (CanvasPaintThread::start).

<pre>
assert!(canvas_id == painter.canvas_id);
</pre>

===Performance===
Almost every time, http://slither.io/ ended up crashing. Other times, it performed very slowly. However, as of 4/2/2018, much more is left to be done (The Subsequent Steps).

==Subsequent Steps==

The Subsequent Steps were the final project. It builds off of what the Initial Steps lay down. Since the Initial Steps set up all of the users of CanvasPaintThread (even indirect users) to send CanvasIds with their messages, the conversion to use CanvasIds to identify individual canvases was seamless and invisible to the users.

===Overview===
Previously, a CanvasPaintThread was made for every Canvas, which was very slow when there are many Canvases. To fix this, a single CanvasPaintThread was made that manages all the canvases. The data previously stored in CanvasPaintThread unique to each Canvas was moved to a new data structure known as CanvasData.

Previously, communication channels were opened up between each CanvasPaintThread and what uses it and those channels send the CanvasIds established above alongside their messages. Now, there is a single channel that's repeatedly cloned which instead connects the single CanvasPaintThread and those same users. The users will be unable to tell the difference between the current implementation and the new implementation. The CanvasPaintThread will redirect those existing channels into painting using the appropriate CanvasData structure.

These changes involved moving nearly the entire contents of <code>servo/components/canvas/canvas_paint_thread.rs</code> into the new file <code>servo/components/canvas/canvas_data.rs</code>.

There were slight changes to <code>servo/components/constellation/constellation.rs</code> so that it doesn't try to create multiple CanvasPaintThreads, but those changes were minor and well described in the spec.

There were also changes to <code>servo/components/canvas_traits/canvas.rs</code> and <code>servo/components/script/dom/canvasrenderingcontext2d.rs</code> because of the removed parts of Canvas2dMsg::DrawImageInOther

===Initial Form of CanvasPaintThread===
From:
<code>servo/components/canvas/canvas_paint_thread.rs</code>

<code>
pub struct CanvasPaintThread<'a> {
drawtarget: DrawTarget,
/// TODO(pcwalton): Support multiple paths.
path_builder: PathBuilder,
state: CanvasPaintState<'a>,
saved_states: Vec<CanvasPaintState<'a>>,
webrender_api: webrender_api::RenderApi,
image_key: Option<webrender_api::ImageKey>,
/// An old webrender image key that can be deleted when the next epoch ends.
old_image_key: Option<webrender_api::ImageKey>,
/// An old webrender image key that can be deleted when the current epoch ends.
very_old_image_key: Option<webrender_api::ImageKey>,
canvas_id: CanvasId,
}
</code>

CanvasPaintThread initially stored all the information about a particular canvas. This includes both information used to represent the pixels actually on the canvas and information used to support drawing on the canvas.

There were many, many functions in the previous form of CanvasPaintThread because all the actual painting was done there.

CanvasIds were not usefully used. They were simply added in the initial steps so that the grunt work for the subsequent steps were simpler to implement and more focused.

===New Form of CanvasPaintThread===
====CanvasPaintThread====
From:
<code>servo/components/canvas/canvas_paint_thread.rs</code>

<code>
pub struct CanvasPaintThread <'a> {
canvases: HashMap<CanvasId, CanvasData<'a>>,
next_canvas_id: CanvasId,
}
</code>

The new CanvasPaintThread is much leaner. A single CanvasPaintThread exists with nothing but a HashMap of CanvasId->CanvasData structures and a CanvasId to represent what the next CanvasId will be.

The only functions it has are start (for initially starting it and also the thread itself, which passes CanvasMsg enums it receives around), create_canvas (for generating a new CanvasData structure), and process_canvas_2d_message (which exists solely because there are a lot of Canvas2dMsg enum variants and making the thread itself pass them around makes the code very hard to read).

====CanvasData====
From:
<code>servo/components/canvas/canvas_data.rs</code>

<code>
pub struct CanvasData<'a> {
drawtarget: DrawTarget,
/// TODO(pcwalton): Support multiple paths.
path_builder: PathBuilder,
state: CanvasPaintState<'a>,
saved_states: Vec<CanvasPaintState<'a>>,
webrender_api: webrender_api::RenderApi,
image_key: Option<webrender_api::ImageKey>,
/// An old webrender image key that can be deleted when the next epoch ends.
old_image_key: Option<webrender_api::ImageKey>,
/// An old webrender image key that can be deleted when the current epoch ends.
very_old_image_key: Option<webrender_api::ImageKey>,
canvas_id: CanvasId,
}
</code>

You may notice the similarities between CanvasData and the old form of CanvasPaintThread! Now CanvasData does all the things CanvasPaintThread used to do for the purposes of painting on the canvases (for example, bezier_curve_to, arc, and set_line_cap among many others).

There's nothing particularly ''new'' in CanvasData. It's all just moved from CanvasPaintThread.

It is a very focused class now, though. All it does is paint on canvases. There's no processing of messages, only painting. Moreover, the draw_image_in_other function is nowhere to be found, instead replaced by a small amount of processing done by CanvasPaintThread, which just calls draw_image on the one that was originally considered "other" so there's no communication between the CanvasData structures ''at all''.

===New Form's Improvements===
====Design====
The new form keeps the responsibilities of converting CanvasMsg enums into their contents and using those contents to paint canvases separate, which improves the overall adherence to the Single Responsibility Principle of Servo. There was also a lot of odd indentation and otherwise messy code in the old form of CanvasPaintThread, which, in the process of preparing the pull request for these changes, were cleaned up, making the overall tidyness of Servo improve.

====Performance====
The new implementation is also much faster when multiple canvases are involved because there's no need for threading delay to send messages back and forth, waiting for those messages to be sent and received. There's less room for usage of multiple cores to speed up processing, but Servo is already so ''very'' parallel that there will be ''something'' to keep the other cores busy when CanvasPaintThread is stuck on one thread.

Slither.io doesn't run properly on my machine either before or after the changes, which makes reporting on the success of the changes with respect to Slither.io difficult at best. The framerate improved dramatically, even if it looks like... well... this still. I can't imagine that a website with over 3500 canvases could be made to run more slowly with this change, though.

[[File:Slither.png]]

Using our timed drawimage test, the performance improvement is... well... not really there. On the development build, the drawing time was cut nearly in half from around 19ms to around 9ms, but on the release build, the drawing time is effectively identical, varying between 1.3ms and 1.7ms for both versions. This isn't too surprising, seeing as only two canvases are made and the cost of communicating between two threads isn't very expensive compared to the thousands that the change was made to address.

=Conclusion=
So far we have associated each variant of a canvas message with a particular canvas id. Even though there is not much improvement is the performance of http://slither.io/, we believe that an improvement can be noticed after going through the Subsequent Steps. Here is a link to our [https://github.com/servo/servo/pull/20447 pull request]. On 4/2/2018, our contribution was merged into Servo's master branch!

The rendering of 2D canvases has been entirely moved to a single thread with each canvas identified by a CanvasId. The pull request for the final project is [https://github.com/servo/servo/pull/20680 here]. As of 4/23/2018, the tests are passing and only various "nit"s are being worked through. A merge is expected very soon.

=References=
#https://en.wikipedia.org/wiki/Servo_(layout_engine)
#https://www.rust-lang.org/en-US/
#https://github.com/servo/servo
#https://github.com/servo/servo/wiki/Canvas-rendering-project
#https://www.w3schools.com/graphics/canvas_intro.asp
#https://github.com/servo/servo/issues/10381
#https://github.com/servo/servo/issues/13879

CSC/ECE 517 Spring 2018- Project M1802: 2D Canvas Rendering (Part 2)

2018-04-24T02:47:22Z

Baeastwo: /* Design Pattern */

'''“M1802: Simplify the 2d Canvas Rendering”'''

As robust and fast Servo is, Servo’s implementation of its HTML 2D/3D “<canvas>” has several inefficiencies that make some websites perform slowly or run out of memory when performing complex canvas operations. The goal of this project was to make these websites perform better when loaded in Servo.

=Introduction=
This section briefly describes the main parts of this project: [https://en.wikipedia.org/wiki/Servo_(layout_engine) Servo], [https://www.rust-lang.org/en-US/ Rust], and [https://www.w3schools.com/graphics/canvas_intro.asp HTML Canvas].

==Servo==
Named after a robot from the show Mystery Science Theater 3000, Servo is an experimental web browser layout engine developed by Mozilla. It is currently being developed for 64-bit OS X, 64-bit Linux, 64-bit Windows, and Android. While working with Samsung, Mozilla seeks to create a highly parallel environment where the many things in a web browser like rendering, image decoding, and HTML parsing are handled by isolated tasks. [https://www.youtube.com/watch?v=Ry_RktGLKq4 Here] is a demo on how Servo performs against Firefox.

==Rust==
Since existing languages like C++ did not provide a direct way to achieve a parallel environment, Mozilla developed and uses a programming language called Rust to further develop Servo.

==HTML Canvas==
A <canvas> in HTML is a container for graphics that is used to draw graphics “on the fly” by using Javascript. However, the <canvas> element has no drawing abilities of its own since it only acts as a container for graphics. A script to actually draw the graphics. By using scripts, the HTML <canvas> can:
*Be animated
*Be interactive
*Draw text
*Be used in games
[[File:DrawImage.PNG|An example of several canvases from servo's tests.]]

=Project Description=
The goal of this project was to remove the inefficiencies of Servo’s implemetation of the 2D/3D canvas. As reported with issue [https://github.com/servo/servo/issues/10381 #10381], a gaming website called http://slither.io/ was performing extremely slowly as of Apr 3, 2016. As of Oct 21, 2016, it was reported with issue [https://github.com/servo/servo/issues/13879 #13879] that http://slither.io/ was, in fact, crashing when loaded onto Servo. The suggestion to these issues was to, first, create a test case that measured the amount of time that Servo spends under “drawImage” while doing operations related to a canvas. Another suggestion was to change Servo’s implementation from using one thread per canvas to one thread for all canvases. The [https://github.com/servo/servo/wiki/Canvas-rendering-project steps] laid out for this project by [https://www.mozilla.org/en-US/ Mozilla] (both the initial setup steps and optimizing the canvas implementation) are shown below.

==Initial Steps:==
*create a testcase that contains two canvases and uses the drawImage API to draw the contents of one canvas onto the other. Programmatically measure the time this operation takes.

*To prepare for the big switch from 1 threads per canvas to 1 thread for all canvases, add a struct CanvasId (u64) type to components/canvas_traits/canvas.rs and add a CanvasId member to each variant of the CanvasMsg enum.

*add a CanvasId member to Constellation in components/constellation/constellation.rs which is initialized to 0 and increased by 1 each time handle_create_canvas_paint_thread is called.

*make the response_sender argument of handle_create_canvas_paint_thread also include the new CanvasId value, and pass it as an argument to CanvasPaintThread::start. Store the id when it is received for use in all canvas messages.

*For each CanvasMsg that is processed by CanvasPaintThread, verify that the id received matches the id that was provided to CanvasPaintThread::start

==Subsequent Steps:==
*extract the innards of CanvasPaintThread into a CanvasData structure,

*make CanvasPaintThread store a hashtable of CanvasId->CanvasData

*as part of Constellation::start, create a canvas paint thread and store the channel to communicate with it as a member of Constellation. Remove the initial canvas id from the API of Constellation::start.

*when handle_create_canvas_paint_thread is invoked, communicate with the canvas thread and have it create a new entry in the hashtable.

*when the canvas thread receives a message, perform the operation on the appropriate canvas according to the provided id

*optimize the DrawImageInOther operation by drawing on the destination canvas directly, rather than relying on sending a message to another canvas thread. Remove the now-unnecessary IpcSender from the DrawImageInOther enum variant. Verify that the earlier test case demonstrates a performance improvement.

*report on how slither.io performs in Servo after all these changes

=Design Pattern=
Design patterns were not applicable since our task was to add a few parts to code that already existed. Also, Rust only has structs and traits, which are very different from classes and interfaces in terms of usage. Inheritance is effectively nonexistent in Rust, so GoF patterns are nearly always impossible to implement in Rust.

Part of the goal of the Subsequent Steps was to increase the compliance to the Separation of Responsibilities principle, though. It took an object that previously both processed messages and painted canvases and converted it into an object that processes messages and another object that paints canvases.

=Implementation=
==General Information==

Our repo is [https://github.com/Brody-Eastwood/servo here].

===Building===
After setting up the environment required to develop for Servo, we built and compiled Servo as per the instructions on Servo’s Github [https://github.com/servo/servo repo]. We used Mozilla’s mach tools to build Servo with Cargo, which is the rust package manager.

<pre>
git clone https://github.com/servo/servo
cd servo
./mach build --dev
./mach run tests/html/about-mozilla.html
</pre>

===Testing===
After successfully building servo, we proceeded to test the amount of time spent under “drawImage” as shown below. The function "drawImage" copies the contents of one canvas in a view to another canvas in the same view. During the "Initial Steps," this test was created to test the initial performance of Servo with regards to canvas rendering.

<pre>
var t0 = performance.now();
drawImage(25, 25);
var t1 = performance.now();
document.getElementById('Test Result').innerHTML = "DrawImage took " + (t1 - t0) + " milliseconds.";
console.log("DrawImage took " + (t1 - t0) + " milliseconds.");
</pre>

[[File:TimedDrawImage.png]]

This was/will be used to measure the performance when drawing a single canvas.

===Running Servo===
After each successful build through our development process, we ran Servo as per the instructions on Servo’s Github repo.

<pre>
./servo [url] [arguments] # if you run with nightly build
./mach run [url] [arguments] # if you run with mach

# For example
./mach run https://www.google.com
</pre>

==Initial Steps==

The initial steps were the first OSS project.

===Adding an Id to Canvas Communications===
As shown below, we then associated a CanvasId with each variant of a canvas message; these messages are passed around various parts of the Servo implementation such as the 2D canvas struct, view layout, and various scripts. This required us to make other changed to allow a CanvasId to be sent with each message, as shown below.

{| class="wikitable"
|-
! Before
! After
|-
| pub enum CanvasMsg {
Canvas2d(Canvas2dMsg),
FromLayout(FromLayoutMsg),
FromScript(FromScriptMsg),
Recreate(Size2D<i32>),
Close(),
}

| pub enum CanvasMsg {
Canvas2d(Canvas2dMsg, CanvasId),
FromLayout(FromLayoutMsg, CanvasId),
FromScript(FromScriptMsg, CanvasId),
Recreate(Size2D<i32>, CanvasId),
Close(CanvasId),
}
|}

Then, we made the necessary changed to the constellation and the canvas paint thread to be able to initialize, increment, and store the CanvasId. To finish up with the initial steps, we programmatically checked to make sure that each id that was processed by CanvasPaintThread was the same id that was provided when the thread was started (CanvasPaintThread::start).

<pre>
assert!(canvas_id == painter.canvas_id);
</pre>

===Performance===
Almost every time, http://slither.io/ ended up crashing. Other times, it performed very slowly. However, as of 4/2/2018, much more is left to be done (The Subsequent Steps).

==Subsequent Steps==

The Subsequent Steps were the final project. It builds off of what the Initial Steps lay down. Since the Initial Steps set up all of the users of CanvasPaintThread (even indirect users) to send CanvasIds with their messages, the conversion to use CanvasIds to identify individual canvases was seamless and invisible to the users.

===Overview===
Previously, a CanvasPaintThread was made for every Canvas, which was very slow when there are many Canvases. To fix this, a single CanvasPaintThread was made that manages all the canvases. The data previously stored in CanvasPaintThread unique to each Canvas was moved to a new data structure known as CanvasData.

Previously, communication channels were opened up between each CanvasPaintThread and what uses it and those channels send the CanvasIds established above alongside their messages. Now, there is a single channel that's repeatedly cloned which instead connects the single CanvasPaintThread and those same users. The users will be unable to tell the difference between the current implementation and the new implementation. The CanvasPaintThread will redirect those existing channels into painting using the appropriate CanvasData structure.

These changes involved moving nearly the entire contents of <code>servo/components/canvas/canvas_paint_thread.rs</code> into the new file <code>servo/components/canvas/canvas_data.rs</code>.

There were slight changes to <code>servo/components/constellation/constellation.rs</code> so that it doesn't try to create multiple CanvasPaintThreads, but those changes were minor and well described in the spec.

There were also changes to <code>servo/components/canvas_traits/canvas.rs</code> and <code>servo/components/script/dom/canvasrenderingcontext2d.rs</code> because of the removed parts of Canvas2dMsg::DrawImageInOther

===Initial Form of CanvasPaintThread===
From:
<code>servo/components/canvas/canvas_paint_thread.rs</code>

<code>
pub struct CanvasPaintThread<'a> {
drawtarget: DrawTarget,
/// TODO(pcwalton): Support multiple paths.
path_builder: PathBuilder,
state: CanvasPaintState<'a>,
saved_states: Vec<CanvasPaintState<'a>>,
webrender_api: webrender_api::RenderApi,
image_key: Option<webrender_api::ImageKey>,
/// An old webrender image key that can be deleted when the next epoch ends.
old_image_key: Option<webrender_api::ImageKey>,
/// An old webrender image key that can be deleted when the current epoch ends.
very_old_image_key: Option<webrender_api::ImageKey>,
canvas_id: CanvasId,
}
</code>

CanvasPaintThread initially stored all the information about a particular canvas. This includes both information used to represent the pixels actually on the canvas and information used to support drawing on the canvas.

There were many, many functions in the previous form of CanvasPaintThread because all the actual painting was done there.

CanvasIds were not usefully used. They were simply added in the initial steps so that the grunt work for the subsequent steps were simpler to implement and more focused.

===New Form of CanvasPaintThread===
====CanvasPaintThread====
From:
<code>servo/components/canvas/canvas_paint_thread.rs</code>

<code>
pub struct CanvasPaintThread <'a> {
canvases: HashMap<CanvasId, CanvasData<'a>>,
next_canvas_id: CanvasId,
}
</code>

The new CanvasPaintThread is much leaner. A single CanvasPaintThread exists with nothing but a HashMap of CanvasId->CanvasData structures and a CanvasId to represent what the next CanvasId will be.

The only functions it has are start (for initially starting it and also the thread itself, which passes CanvasMsg enums it receives around), create_canvas (for generating a new CanvasData structure), and process_canvas_2d_message (which exists solely because there are a lot of Canvas2dMsg enum variants and making the thread itself pass them around makes the code very hard to read).

====CanvasData====
From:
<code>servo/components/canvas/canvas_data.rs</code>

<code>
pub struct CanvasData<'a> {
drawtarget: DrawTarget,
/// TODO(pcwalton): Support multiple paths.
path_builder: PathBuilder,
state: CanvasPaintState<'a>,
saved_states: Vec<CanvasPaintState<'a>>,
webrender_api: webrender_api::RenderApi,
image_key: Option<webrender_api::ImageKey>,
/// An old webrender image key that can be deleted when the next epoch ends.
old_image_key: Option<webrender_api::ImageKey>,
/// An old webrender image key that can be deleted when the current epoch ends.
very_old_image_key: Option<webrender_api::ImageKey>,
canvas_id: CanvasId,
}
</code>

You may notice the similarities between CanvasData and the old form of CanvasPaintThread! Now CanvasData does all the things CanvasPaintThread used to do for the purposes of painting on the canvases (for example, bezier_curve_to, arc, and set_line_cap among many others).

There's nothing particularly ''new'' in CanvasData. It's all just moved from CanvasPaintThread.

It is a very focused class now, though. All it does is paint on canvases. There's no processing of messages, only painting. Moreover, the draw_image_in_other function is nowhere to be found, instead replaced by a small amount of processing done by CanvasPaintThread, which just calls draw_image on the one that was originally considered "other" so there's no communication between the CanvasData structures ''at all''.

===New Form's Improvements===
====Design====
The new form keeps the responsibilities of converting CanvasMsg enums into their contents and using those contents to paint canvases separate, which improves the overall adherence to the Single Responsibility Principle of Servo. There was also a lot of odd indentation and otherwise messy code in the old form of CanvasPaintThread, which, in the process of preparing the pull request for these changes, were cleaned up, making the overall tidyness of Servo improve.

====Performance====
The new implementation is also much faster when multiple canvases are involved because there's no need for threading delay to send messages back and forth, waiting for those messages to be sent and received. There's less room for usage of multiple cores to speed up processing, but Servo is already so ''very'' parallel that there will be ''something'' to keep the other cores busy when CanvasPaintThread is stuck on one thread.

Slither.io doesn't run properly on my machine either before or after the changes, which makes reporting on the success of the changes with respect to Slither.io difficult at best. The framerate improved dramatically, even if it looks like... well... this still. I can't imagine that a website with over 3500 canvases could be made to run more slowly with this change, though.

[[File:Slither.png]]

Using our timed drawimage test, the performance improvement is... well... not really there. On the development build, the drawing time was cut nearly in half from around 19ms to around 9ms, but on the release build, the drawing time is effectively identical, varying between 1.3ms and 1.7ms for both versions. This isn't too surprising, seeing as only two canvases are made and the cost of communicating between two threads isn't very expensive compared to the thousands that the change was made to address.

=Conclusion=
So far we have associated each variant of a canvas message with a particular canvas id. Even though there is not much improvement is the performance of http://slither.io/, we believe that an improvement can be noticed after going through the Subsequent Steps. Here is a link to our [https://github.com/servo/servo/pull/20447 pull request]. On 4/2/2018, our contribution was merged into Servo's master branch!

The rendering of 2D canvases has been entirely moved to a single thread with each canvas identified by a CanvasId. The pull request for the final project is [https://github.com/servo/servo/pull/20680 here]. As of 4/23/2018, the tests are passing and only various "nit"s are being worked through. A merge is expected very soon.

=References=
#https://en.wikipedia.org/wiki/Servo_(layout_engine)
#https://www.rust-lang.org/en-US/
#https://github.com/servo/servo
#https://github.com/servo/servo/wiki/Canvas-rendering-project
#https://www.w3schools.com/graphics/canvas_intro.asp
#https://github.com/servo/servo/issues/10381
#https://github.com/servo/servo/issues/13879

CSC/ECE 517 Spring 2018- Project M1802: 2D Canvas Rendering (Part 2)

2018-04-24T02:43:56Z

Baeastwo: /* Conclusion */

'''“M1802: Simplify the 2d Canvas Rendering”'''

As robust and fast Servo is, Servo’s implementation of its HTML 2D/3D “<canvas>” has several inefficiencies that make some websites perform slowly or run out of memory when performing complex canvas operations. The goal of this project was to make these websites perform better when loaded in Servo.

=Introduction=
This section briefly describes the main parts of this project: [https://en.wikipedia.org/wiki/Servo_(layout_engine) Servo], [https://www.rust-lang.org/en-US/ Rust], and [https://www.w3schools.com/graphics/canvas_intro.asp HTML Canvas].

==Servo==
Named after a robot from the show Mystery Science Theater 3000, Servo is an experimental web browser layout engine developed by Mozilla. It is currently being developed for 64-bit OS X, 64-bit Linux, 64-bit Windows, and Android. While working with Samsung, Mozilla seeks to create a highly parallel environment where the many things in a web browser like rendering, image decoding, and HTML parsing are handled by isolated tasks. [https://www.youtube.com/watch?v=Ry_RktGLKq4 Here] is a demo on how Servo performs against Firefox.

==Rust==
Since existing languages like C++ did not provide a direct way to achieve a parallel environment, Mozilla developed and uses a programming language called Rust to further develop Servo.

==HTML Canvas==
A <canvas> in HTML is a container for graphics that is used to draw graphics “on the fly” by using Javascript. However, the <canvas> element has no drawing abilities of its own since it only acts as a container for graphics. A script to actually draw the graphics. By using scripts, the HTML <canvas> can:
*Be animated
*Be interactive
*Draw text
*Be used in games
[[File:DrawImage.PNG|An example of several canvases from servo's tests.]]

=Project Description=
The goal of this project was to remove the inefficiencies of Servo’s implemetation of the 2D/3D canvas. As reported with issue [https://github.com/servo/servo/issues/10381 #10381], a gaming website called http://slither.io/ was performing extremely slowly as of Apr 3, 2016. As of Oct 21, 2016, it was reported with issue [https://github.com/servo/servo/issues/13879 #13879] that http://slither.io/ was, in fact, crashing when loaded onto Servo. The suggestion to these issues was to, first, create a test case that measured the amount of time that Servo spends under “drawImage” while doing operations related to a canvas. Another suggestion was to change Servo’s implementation from using one thread per canvas to one thread for all canvases. The [https://github.com/servo/servo/wiki/Canvas-rendering-project steps] laid out for this project by [https://www.mozilla.org/en-US/ Mozilla] (both the initial setup steps and optimizing the canvas implementation) are shown below.

==Initial Steps:==
*create a testcase that contains two canvases and uses the drawImage API to draw the contents of one canvas onto the other. Programmatically measure the time this operation takes.

*To prepare for the big switch from 1 threads per canvas to 1 thread for all canvases, add a struct CanvasId (u64) type to components/canvas_traits/canvas.rs and add a CanvasId member to each variant of the CanvasMsg enum.

*add a CanvasId member to Constellation in components/constellation/constellation.rs which is initialized to 0 and increased by 1 each time handle_create_canvas_paint_thread is called.

*make the response_sender argument of handle_create_canvas_paint_thread also include the new CanvasId value, and pass it as an argument to CanvasPaintThread::start. Store the id when it is received for use in all canvas messages.

*For each CanvasMsg that is processed by CanvasPaintThread, verify that the id received matches the id that was provided to CanvasPaintThread::start

==Subsequent Steps:==
*extract the innards of CanvasPaintThread into a CanvasData structure,

*make CanvasPaintThread store a hashtable of CanvasId->CanvasData

*as part of Constellation::start, create a canvas paint thread and store the channel to communicate with it as a member of Constellation. Remove the initial canvas id from the API of Constellation::start.

*when handle_create_canvas_paint_thread is invoked, communicate with the canvas thread and have it create a new entry in the hashtable.

*when the canvas thread receives a message, perform the operation on the appropriate canvas according to the provided id

*optimize the DrawImageInOther operation by drawing on the destination canvas directly, rather than relying on sending a message to another canvas thread. Remove the now-unnecessary IpcSender from the DrawImageInOther enum variant. Verify that the earlier test case demonstrates a performance improvement.

*report on how slither.io performs in Servo after all these changes

=Design Pattern=
Design patterns were not applicable since our task was to add a few parts to code that already existed.

=Implementation=
==General Information==

Our repo is [https://github.com/Brody-Eastwood/servo here].

===Building===
After setting up the environment required to develop for Servo, we built and compiled Servo as per the instructions on Servo’s Github [https://github.com/servo/servo repo]. We used Mozilla’s mach tools to build Servo with Cargo, which is the rust package manager.

<pre>
git clone https://github.com/servo/servo
cd servo
./mach build --dev
./mach run tests/html/about-mozilla.html
</pre>

===Testing===
After successfully building servo, we proceeded to test the amount of time spent under “drawImage” as shown below. The function "drawImage" copies the contents of one canvas in a view to another canvas in the same view. During the "Initial Steps," this test was created to test the initial performance of Servo with regards to canvas rendering.

<pre>
var t0 = performance.now();
drawImage(25, 25);
var t1 = performance.now();
document.getElementById('Test Result').innerHTML = "DrawImage took " + (t1 - t0) + " milliseconds.";
console.log("DrawImage took " + (t1 - t0) + " milliseconds.");
</pre>

[[File:TimedDrawImage.png]]

This was/will be used to measure the performance when drawing a single canvas.

===Running Servo===
After each successful build through our development process, we ran Servo as per the instructions on Servo’s Github repo.

<pre>
./servo [url] [arguments] # if you run with nightly build
./mach run [url] [arguments] # if you run with mach

# For example
./mach run https://www.google.com
</pre>

==Initial Steps==

The initial steps were the first OSS project.

===Adding an Id to Canvas Communications===
As shown below, we then associated a CanvasId with each variant of a canvas message; these messages are passed around various parts of the Servo implementation such as the 2D canvas struct, view layout, and various scripts. This required us to make other changed to allow a CanvasId to be sent with each message, as shown below.

{| class="wikitable"
|-
! Before
! After
|-
| pub enum CanvasMsg {
Canvas2d(Canvas2dMsg),
FromLayout(FromLayoutMsg),
FromScript(FromScriptMsg),
Recreate(Size2D<i32>),
Close(),
}

| pub enum CanvasMsg {
Canvas2d(Canvas2dMsg, CanvasId),
FromLayout(FromLayoutMsg, CanvasId),
FromScript(FromScriptMsg, CanvasId),
Recreate(Size2D<i32>, CanvasId),
Close(CanvasId),
}
|}

Then, we made the necessary changed to the constellation and the canvas paint thread to be able to initialize, increment, and store the CanvasId. To finish up with the initial steps, we programmatically checked to make sure that each id that was processed by CanvasPaintThread was the same id that was provided when the thread was started (CanvasPaintThread::start).

<pre>
assert!(canvas_id == painter.canvas_id);
</pre>

===Performance===
Almost every time, http://slither.io/ ended up crashing. Other times, it performed very slowly. However, as of 4/2/2018, much more is left to be done (The Subsequent Steps).

==Subsequent Steps==

The Subsequent Steps were the final project. It builds off of what the Initial Steps lay down. Since the Initial Steps set up all of the users of CanvasPaintThread (even indirect users) to send CanvasIds with their messages, the conversion to use CanvasIds to identify individual canvases was seamless and invisible to the users.

===Overview===
Previously, a CanvasPaintThread was made for every Canvas, which was very slow when there are many Canvases. To fix this, a single CanvasPaintThread was made that manages all the canvases. The data previously stored in CanvasPaintThread unique to each Canvas was moved to a new data structure known as CanvasData.

Previously, communication channels were opened up between each CanvasPaintThread and what uses it and those channels send the CanvasIds established above alongside their messages. Now, there is a single channel that's repeatedly cloned which instead connects the single CanvasPaintThread and those same users. The users will be unable to tell the difference between the current implementation and the new implementation. The CanvasPaintThread will redirect those existing channels into painting using the appropriate CanvasData structure.

These changes involved moving nearly the entire contents of <code>servo/components/canvas/canvas_paint_thread.rs</code> into the new file <code>servo/components/canvas/canvas_data.rs</code>.

There were slight changes to <code>servo/components/constellation/constellation.rs</code> so that it doesn't try to create multiple CanvasPaintThreads, but those changes were minor and well described in the spec.

There were also changes to <code>servo/components/canvas_traits/canvas.rs</code> and <code>servo/components/script/dom/canvasrenderingcontext2d.rs</code> because of the removed parts of Canvas2dMsg::DrawImageInOther

===Initial Form of CanvasPaintThread===
From:
<code>servo/components/canvas/canvas_paint_thread.rs</code>

<code>
pub struct CanvasPaintThread<'a> {
drawtarget: DrawTarget,
/// TODO(pcwalton): Support multiple paths.
path_builder: PathBuilder,
state: CanvasPaintState<'a>,
saved_states: Vec<CanvasPaintState<'a>>,
webrender_api: webrender_api::RenderApi,
image_key: Option<webrender_api::ImageKey>,
/// An old webrender image key that can be deleted when the next epoch ends.
old_image_key: Option<webrender_api::ImageKey>,
/// An old webrender image key that can be deleted when the current epoch ends.
very_old_image_key: Option<webrender_api::ImageKey>,
canvas_id: CanvasId,
}
</code>

CanvasPaintThread initially stored all the information about a particular canvas. This includes both information used to represent the pixels actually on the canvas and information used to support drawing on the canvas.

There were many, many functions in the previous form of CanvasPaintThread because all the actual painting was done there.

CanvasIds were not usefully used. They were simply added in the initial steps so that the grunt work for the subsequent steps were simpler to implement and more focused.

===New Form of CanvasPaintThread===
====CanvasPaintThread====
From:
<code>servo/components/canvas/canvas_paint_thread.rs</code>

<code>
pub struct CanvasPaintThread <'a> {
canvases: HashMap<CanvasId, CanvasData<'a>>,
next_canvas_id: CanvasId,
}
</code>

The new CanvasPaintThread is much leaner. A single CanvasPaintThread exists with nothing but a HashMap of CanvasId->CanvasData structures and a CanvasId to represent what the next CanvasId will be.

The only functions it has are start (for initially starting it and also the thread itself, which passes CanvasMsg enums it receives around), create_canvas (for generating a new CanvasData structure), and process_canvas_2d_message (which exists solely because there are a lot of Canvas2dMsg enum variants and making the thread itself pass them around makes the code very hard to read).

====CanvasData====
From:
<code>servo/components/canvas/canvas_data.rs</code>

<code>
pub struct CanvasData<'a> {
drawtarget: DrawTarget,
/// TODO(pcwalton): Support multiple paths.
path_builder: PathBuilder,
state: CanvasPaintState<'a>,
saved_states: Vec<CanvasPaintState<'a>>,
webrender_api: webrender_api::RenderApi,
image_key: Option<webrender_api::ImageKey>,
/// An old webrender image key that can be deleted when the next epoch ends.
old_image_key: Option<webrender_api::ImageKey>,
/// An old webrender image key that can be deleted when the current epoch ends.
very_old_image_key: Option<webrender_api::ImageKey>,
canvas_id: CanvasId,
}
</code>

You may notice the similarities between CanvasData and the old form of CanvasPaintThread! Now CanvasData does all the things CanvasPaintThread used to do for the purposes of painting on the canvases (for example, bezier_curve_to, arc, and set_line_cap among many others).

There's nothing particularly ''new'' in CanvasData. It's all just moved from CanvasPaintThread.

It is a very focused class now, though. All it does is paint on canvases. There's no processing of messages, only painting. Moreover, the draw_image_in_other function is nowhere to be found, instead replaced by a small amount of processing done by CanvasPaintThread, which just calls draw_image on the one that was originally considered "other" so there's no communication between the CanvasData structures ''at all''.

===New Form's Improvements===
====Design====
The new form keeps the responsibilities of converting CanvasMsg enums into their contents and using those contents to paint canvases separate, which improves the overall adherence to the Single Responsibility Principle of Servo. There was also a lot of odd indentation and otherwise messy code in the old form of CanvasPaintThread, which, in the process of preparing the pull request for these changes, were cleaned up, making the overall tidyness of Servo improve.

====Performance====
The new implementation is also much faster when multiple canvases are involved because there's no need for threading delay to send messages back and forth, waiting for those messages to be sent and received. There's less room for usage of multiple cores to speed up processing, but Servo is already so ''very'' parallel that there will be ''something'' to keep the other cores busy when CanvasPaintThread is stuck on one thread.

Slither.io doesn't run properly on my machine either before or after the changes, which makes reporting on the success of the changes with respect to Slither.io difficult at best. The framerate improved dramatically, even if it looks like... well... this still. I can't imagine that a website with over 3500 canvases could be made to run more slowly with this change, though.

[[File:Slither.png]]

Using our timed drawimage test, the performance improvement is... well... not really there. On the development build, the drawing time was cut nearly in half from around 19ms to around 9ms, but on the release build, the drawing time is effectively identical, varying between 1.3ms and 1.7ms for both versions. This isn't too surprising, seeing as only two canvases are made and the cost of communicating between two threads isn't very expensive compared to the thousands that the change was made to address.

=Conclusion=
So far we have associated each variant of a canvas message with a particular canvas id. Even though there is not much improvement is the performance of http://slither.io/, we believe that an improvement can be noticed after going through the Subsequent Steps. Here is a link to our [https://github.com/servo/servo/pull/20447 pull request]. On 4/2/2018, our contribution was merged into Servo's master branch!

The rendering of 2D canvases has been entirely moved to a single thread with each canvas identified by a CanvasId. The pull request for the final project is [https://github.com/servo/servo/pull/20680 here]. As of 4/23/2018, the tests are passing and only various "nit"s are being worked through. A merge is expected very soon.

=References=
#https://en.wikipedia.org/wiki/Servo_(layout_engine)
#https://www.rust-lang.org/en-US/
#https://github.com/servo/servo
#https://github.com/servo/servo/wiki/Canvas-rendering-project
#https://www.w3schools.com/graphics/canvas_intro.asp
#https://github.com/servo/servo/issues/10381
#https://github.com/servo/servo/issues/13879

File:Slither.png

2018-04-24T02:40:56Z

Baeastwo:

CSC/ECE 517 Spring 2018- Project M1802: 2D Canvas Rendering (Part 2)

2018-04-24T02:39:44Z

Baeastwo: /* Subsequent Steps */

'''“M1802: Simplify the 2d Canvas Rendering”'''

As robust and fast Servo is, Servo’s implementation of its HTML 2D/3D “<canvas>” has several inefficiencies that make some websites perform slowly or run out of memory when performing complex canvas operations. The goal of this project was to make these websites perform better when loaded in Servo.

=Introduction=
This section briefly describes the main parts of this project: [https://en.wikipedia.org/wiki/Servo_(layout_engine) Servo], [https://www.rust-lang.org/en-US/ Rust], and [https://www.w3schools.com/graphics/canvas_intro.asp HTML Canvas].

==Servo==
Named after a robot from the show Mystery Science Theater 3000, Servo is an experimental web browser layout engine developed by Mozilla. It is currently being developed for 64-bit OS X, 64-bit Linux, 64-bit Windows, and Android. While working with Samsung, Mozilla seeks to create a highly parallel environment where the many things in a web browser like rendering, image decoding, and HTML parsing are handled by isolated tasks. [https://www.youtube.com/watch?v=Ry_RktGLKq4 Here] is a demo on how Servo performs against Firefox.

==Rust==
Since existing languages like C++ did not provide a direct way to achieve a parallel environment, Mozilla developed and uses a programming language called Rust to further develop Servo.

==HTML Canvas==
A <canvas> in HTML is a container for graphics that is used to draw graphics “on the fly” by using Javascript. However, the <canvas> element has no drawing abilities of its own since it only acts as a container for graphics. A script to actually draw the graphics. By using scripts, the HTML <canvas> can:
*Be animated
*Be interactive
*Draw text
*Be used in games
[[File:DrawImage.PNG|An example of several canvases from servo's tests.]]

=Project Description=
The goal of this project was to remove the inefficiencies of Servo’s implemetation of the 2D/3D canvas. As reported with issue [https://github.com/servo/servo/issues/10381 #10381], a gaming website called http://slither.io/ was performing extremely slowly as of Apr 3, 2016. As of Oct 21, 2016, it was reported with issue [https://github.com/servo/servo/issues/13879 #13879] that http://slither.io/ was, in fact, crashing when loaded onto Servo. The suggestion to these issues was to, first, create a test case that measured the amount of time that Servo spends under “drawImage” while doing operations related to a canvas. Another suggestion was to change Servo’s implementation from using one thread per canvas to one thread for all canvases. The [https://github.com/servo/servo/wiki/Canvas-rendering-project steps] laid out for this project by [https://www.mozilla.org/en-US/ Mozilla] (both the initial setup steps and optimizing the canvas implementation) are shown below.

==Initial Steps:==
*create a testcase that contains two canvases and uses the drawImage API to draw the contents of one canvas onto the other. Programmatically measure the time this operation takes.

*To prepare for the big switch from 1 threads per canvas to 1 thread for all canvases, add a struct CanvasId (u64) type to components/canvas_traits/canvas.rs and add a CanvasId member to each variant of the CanvasMsg enum.

*add a CanvasId member to Constellation in components/constellation/constellation.rs which is initialized to 0 and increased by 1 each time handle_create_canvas_paint_thread is called.

*make the response_sender argument of handle_create_canvas_paint_thread also include the new CanvasId value, and pass it as an argument to CanvasPaintThread::start. Store the id when it is received for use in all canvas messages.

*For each CanvasMsg that is processed by CanvasPaintThread, verify that the id received matches the id that was provided to CanvasPaintThread::start

==Subsequent Steps:==
*extract the innards of CanvasPaintThread into a CanvasData structure,

*make CanvasPaintThread store a hashtable of CanvasId->CanvasData

*as part of Constellation::start, create a canvas paint thread and store the channel to communicate with it as a member of Constellation. Remove the initial canvas id from the API of Constellation::start.

*when handle_create_canvas_paint_thread is invoked, communicate with the canvas thread and have it create a new entry in the hashtable.

*when the canvas thread receives a message, perform the operation on the appropriate canvas according to the provided id

*optimize the DrawImageInOther operation by drawing on the destination canvas directly, rather than relying on sending a message to another canvas thread. Remove the now-unnecessary IpcSender from the DrawImageInOther enum variant. Verify that the earlier test case demonstrates a performance improvement.

*report on how slither.io performs in Servo after all these changes

=Design Pattern=
Design patterns were not applicable since our task was to add a few parts to code that already existed.

=Implementation=
==General Information==

Our repo is [https://github.com/Brody-Eastwood/servo here].

===Building===
After setting up the environment required to develop for Servo, we built and compiled Servo as per the instructions on Servo’s Github [https://github.com/servo/servo repo]. We used Mozilla’s mach tools to build Servo with Cargo, which is the rust package manager.

<pre>
git clone https://github.com/servo/servo
cd servo
./mach build --dev
./mach run tests/html/about-mozilla.html
</pre>

===Testing===
After successfully building servo, we proceeded to test the amount of time spent under “drawImage” as shown below. The function "drawImage" copies the contents of one canvas in a view to another canvas in the same view. During the "Initial Steps," this test was created to test the initial performance of Servo with regards to canvas rendering.

<pre>
var t0 = performance.now();
drawImage(25, 25);
var t1 = performance.now();
document.getElementById('Test Result').innerHTML = "DrawImage took " + (t1 - t0) + " milliseconds.";
console.log("DrawImage took " + (t1 - t0) + " milliseconds.");
</pre>

[[File:TimedDrawImage.png]]

This was/will be used to measure the performance when drawing a single canvas.

===Running Servo===
After each successful build through our development process, we ran Servo as per the instructions on Servo’s Github repo.

<pre>
./servo [url] [arguments] # if you run with nightly build
./mach run [url] [arguments] # if you run with mach

# For example
./mach run https://www.google.com
</pre>

==Initial Steps==

The initial steps were the first OSS project.

===Adding an Id to Canvas Communications===
As shown below, we then associated a CanvasId with each variant of a canvas message; these messages are passed around various parts of the Servo implementation such as the 2D canvas struct, view layout, and various scripts. This required us to make other changed to allow a CanvasId to be sent with each message, as shown below.

{| class="wikitable"
|-
! Before
! After
|-
| pub enum CanvasMsg {
Canvas2d(Canvas2dMsg),
FromLayout(FromLayoutMsg),
FromScript(FromScriptMsg),
Recreate(Size2D<i32>),
Close(),
}

| pub enum CanvasMsg {
Canvas2d(Canvas2dMsg, CanvasId),
FromLayout(FromLayoutMsg, CanvasId),
FromScript(FromScriptMsg, CanvasId),
Recreate(Size2D<i32>, CanvasId),
Close(CanvasId),
}
|}

Then, we made the necessary changed to the constellation and the canvas paint thread to be able to initialize, increment, and store the CanvasId. To finish up with the initial steps, we programmatically checked to make sure that each id that was processed by CanvasPaintThread was the same id that was provided when the thread was started (CanvasPaintThread::start).

<pre>
assert!(canvas_id == painter.canvas_id);
</pre>

===Performance===
Almost every time, http://slither.io/ ended up crashing. Other times, it performed very slowly. However, as of 4/2/2018, much more is left to be done (The Subsequent Steps).

==Subsequent Steps==

The Subsequent Steps were the final project. It builds off of what the Initial Steps lay down. Since the Initial Steps set up all of the users of CanvasPaintThread (even indirect users) to send CanvasIds with their messages, the conversion to use CanvasIds to identify individual canvases was seamless and invisible to the users.

===Overview===
Previously, a CanvasPaintThread was made for every Canvas, which was very slow when there are many Canvases. To fix this, a single CanvasPaintThread was made that manages all the canvases. The data previously stored in CanvasPaintThread unique to each Canvas was moved to a new data structure known as CanvasData.

Previously, communication channels were opened up between each CanvasPaintThread and what uses it and those channels send the CanvasIds established above alongside their messages. Now, there is a single channel that's repeatedly cloned which instead connects the single CanvasPaintThread and those same users. The users will be unable to tell the difference between the current implementation and the new implementation. The CanvasPaintThread will redirect those existing channels into painting using the appropriate CanvasData structure.

These changes involved moving nearly the entire contents of <code>servo/components/canvas/canvas_paint_thread.rs</code> into the new file <code>servo/components/canvas/canvas_data.rs</code>.

There were slight changes to <code>servo/components/constellation/constellation.rs</code> so that it doesn't try to create multiple CanvasPaintThreads, but those changes were minor and well described in the spec.

There were also changes to <code>servo/components/canvas_traits/canvas.rs</code> and <code>servo/components/script/dom/canvasrenderingcontext2d.rs</code> because of the removed parts of Canvas2dMsg::DrawImageInOther

===Initial Form of CanvasPaintThread===
From:
<code>servo/components/canvas/canvas_paint_thread.rs</code>

<code>
pub struct CanvasPaintThread<'a> {
drawtarget: DrawTarget,
/// TODO(pcwalton): Support multiple paths.
path_builder: PathBuilder,
state: CanvasPaintState<'a>,
saved_states: Vec<CanvasPaintState<'a>>,
webrender_api: webrender_api::RenderApi,
image_key: Option<webrender_api::ImageKey>,
/// An old webrender image key that can be deleted when the next epoch ends.
old_image_key: Option<webrender_api::ImageKey>,
/// An old webrender image key that can be deleted when the current epoch ends.
very_old_image_key: Option<webrender_api::ImageKey>,
canvas_id: CanvasId,
}
</code>

CanvasPaintThread initially stored all the information about a particular canvas. This includes both information used to represent the pixels actually on the canvas and information used to support drawing on the canvas.

There were many, many functions in the previous form of CanvasPaintThread because all the actual painting was done there.

CanvasIds were not usefully used. They were simply added in the initial steps so that the grunt work for the subsequent steps were simpler to implement and more focused.

===New Form of CanvasPaintThread===
====CanvasPaintThread====
From:
<code>servo/components/canvas/canvas_paint_thread.rs</code>

<code>
pub struct CanvasPaintThread <'a> {
canvases: HashMap<CanvasId, CanvasData<'a>>,
next_canvas_id: CanvasId,
}
</code>

The new CanvasPaintThread is much leaner. A single CanvasPaintThread exists with nothing but a HashMap of CanvasId->CanvasData structures and a CanvasId to represent what the next CanvasId will be.

The only functions it has are start (for initially starting it and also the thread itself, which passes CanvasMsg enums it receives around), create_canvas (for generating a new CanvasData structure), and process_canvas_2d_message (which exists solely because there are a lot of Canvas2dMsg enum variants and making the thread itself pass them around makes the code very hard to read).

====CanvasData====
From:
<code>servo/components/canvas/canvas_data.rs</code>

<code>
pub struct CanvasData<'a> {
drawtarget: DrawTarget,
/// TODO(pcwalton): Support multiple paths.
path_builder: PathBuilder,
state: CanvasPaintState<'a>,
saved_states: Vec<CanvasPaintState<'a>>,
webrender_api: webrender_api::RenderApi,
image_key: Option<webrender_api::ImageKey>,
/// An old webrender image key that can be deleted when the next epoch ends.
old_image_key: Option<webrender_api::ImageKey>,
/// An old webrender image key that can be deleted when the current epoch ends.
very_old_image_key: Option<webrender_api::ImageKey>,
canvas_id: CanvasId,
}
</code>

You may notice the similarities between CanvasData and the old form of CanvasPaintThread! Now CanvasData does all the things CanvasPaintThread used to do for the purposes of painting on the canvases (for example, bezier_curve_to, arc, and set_line_cap among many others).

There's nothing particularly ''new'' in CanvasData. It's all just moved from CanvasPaintThread.

It is a very focused class now, though. All it does is paint on canvases. There's no processing of messages, only painting. Moreover, the draw_image_in_other function is nowhere to be found, instead replaced by a small amount of processing done by CanvasPaintThread, which just calls draw_image on the one that was originally considered "other" so there's no communication between the CanvasData structures ''at all''.

===New Form's Improvements===
====Design====
The new form keeps the responsibilities of converting CanvasMsg enums into their contents and using those contents to paint canvases separate, which improves the overall adherence to the Single Responsibility Principle of Servo. There was also a lot of odd indentation and otherwise messy code in the old form of CanvasPaintThread, which, in the process of preparing the pull request for these changes, were cleaned up, making the overall tidyness of Servo improve.

====Performance====
The new implementation is also much faster when multiple canvases are involved because there's no need for threading delay to send messages back and forth, waiting for those messages to be sent and received. There's less room for usage of multiple cores to speed up processing, but Servo is already so ''very'' parallel that there will be ''something'' to keep the other cores busy when CanvasPaintThread is stuck on one thread.

Slither.io doesn't run properly on my machine either before or after the changes, which makes reporting on the success of the changes with respect to Slither.io difficult at best. The framerate improved dramatically, even if it looks like... well... this still. I can't imagine that a website with over 3500 canvases could be made to run more slowly with this change, though.

[[File:Slither.png]]

Using our timed drawimage test, the performance improvement is... well... not really there. On the development build, the drawing time was cut nearly in half from around 19ms to around 9ms, but on the release build, the drawing time is effectively identical, varying between 1.3ms and 1.7ms for both versions. This isn't too surprising, seeing as only two canvases are made and the cost of communicating between two threads isn't very expensive compared to the thousands that the change was made to address.

=Conclusion=
So far we have associated each variant of a canvas message with a particular canvas id. Even though there is not much improvement is the performance of http://slither.io/, we believe that an improvement can be noticed after going through the Subsequent Steps. Here is a link to our [https://github.com/servo/servo/pull/20447 pull request]. On 4/2/2018, our contribution was merged into Servo's master branch!

The Subsequent Steps should be able to be completed similarly. The instructions are similarly clear, and the team understands how to work on Servo much more clearly, so the difficulty of initially getting set up will not need to be done again.

=References=
#https://en.wikipedia.org/wiki/Servo_(layout_engine)
#https://www.rust-lang.org/en-US/
#https://github.com/servo/servo
#https://github.com/servo/servo/wiki/Canvas-rendering-project
#https://www.w3schools.com/graphics/canvas_intro.asp
#https://github.com/servo/servo/issues/10381
#https://github.com/servo/servo/issues/13879

CSC/ECE 517 Spring 2018- Project M1802: 2D Canvas Rendering (Part 2)

2018-04-24T01:15:28Z

Baeastwo: /* Subsequent Steps */

CSC/ECE 517 Spring 2018- Project M1802: 2D Canvas Rendering (Part 2)

2018-04-24T01:15:02Z

Baeastwo: /* Initial Steps */

MainPage

2018-04-14T01:09:55Z

Baeastwo: /* Expertiza */

==Expertiza==
* [[Expertiza documentation]]

* [[CSC/ECE 517 Summer 2008]]
* [[CSC/ECE 517 Fall 2010]]
* [[CSC/ECE 517 Fall 2011]]
* [[CSC/ECE 517 Fall 2012]]
* [[CSC/ECE 517 Fall 2013]]
* [[CSC/ECE 517 Fall 2014]]
* [[CSC/ECE 517 Fall 2015]]
* [[CSC/ECE 517 Fall 2016]]
* [[CSC/ECE 517 Spring 2014]]
* [[CSC/ECE 517 Spring 2015]]
* [[CSC/ECE 517 Spring 2016]]
* [[CSC/ECE 517 Spring 2017]]
* [[CSC/ECE 517 Fall 2017]]
* [[CSC/ECE 517 Spring 2018- Project Juniper:Bookmark Enhancements]]
* [[CSC/ECE 517 Spring 2018- Project E1803: Introducing a Student View for Instructors]]
* [[CSC/ECE 517 Spring 2018- Project E1804: OSS project Yellow: Topic management]]
* [[CSC/ECE_517_Spring_2018- Project E1805: Convolutional data extraction from Github]]
* [[CSC/ECE 517 Spring 2018- Project E1808: Refactor review_mapping_controller.rb]]
* [[CSC/ECE 517 Spring 2018- Project E1810: Show sample submissions and reviews]]
* [[CSC/ECE 517 Spring 2018/E1814 Write unit tests for collusion cycle.rb]]
* [[CSC/ECE 517 Spring 2018- Project M1802: 2D Canvas Rendering]]
* [[CSC/ECE 517 Spring 2018- Project M1802: 2D Canvas Rendering (Part 2)]]
* [[CSC/ECE 517 Spring 2018 - E1800: Add past-due assignments to task list]]
* [[CSC/ECE 517 Spring 2018- Project E1812: on the fly calc.rb]]
* [[CSC/ECE 517 Spring 2018- Project M1803: Implement a web page fuzzer to find rendering mismatches ]]
* [[CSC/ECE 517 Spring 2018- Project M1803: Implement a web page fuzzer to find rendering mismatches (Part 2)]]
* [[CSC/ECE 517 Spring 2018/E1813 Test Menu Items Model]]
* [[CSC/ECE 517 Spring 2018- Project E1816: Visualization for Instructors]]
* [[CSC/ECE 517 Spring 2018- Project E1817: Adding Student-generated Questions to Rubric]]
* [[CSC/ECE 517 Spring 2018- Project E1818: Role-based reviewing]]
* [[CSC/ECE 517 Spring 2018- Project E1822: Extend the functionality of badging]]
* [[CSC/ECE 517 Spring 2018- Project E1815: Improvements to review grader]]
* [[CSC/ECE 517 Spring 2018- Project E1824: Let course staff as well as students do reviews]]
* [[CSC/ECE 517 Spring 2018- Project E1819: Improve self review, link self and peer review to derive grades]]
* [[CSC 456 Spring 2011|CSC 456 Spring 2012]]
* [[ECE 633]]
* [[KCU]]
* [[Progress reports]]

==Application Behavior==
* [[Grading]]

==Metaprogramming==
* [[CSC/ECE_517_Spring_2013/ch1b_1k_hf|Lecture on Metaprogramming]]

==Development==

''Expertiza now has a Java dependency, so the machine you are using to develop Expertiza on should have the JVM installed.''

* [[Setting Up a Development Machine]]
* [[Creating a Linux Development Environment for Expertiza - Installation Guide]]
* [[Using git and github for projects]]
* [[Using heroku to deploy your projects]]
* [[How to Begin a Project from the Current Expertiza Repository]]
* [[Git]]
* [[How to Change a User's Password on a Development Machine]]
* [[Debugging Rails]]
* [http://rajanalwan.com/ui_guidelines/ Design Template]

==Production==
* [[Deploying to Production]]
* [[Downloading Production Data]]
* [[Accessing the Production Server]]

==Testing==
* [[Using Cucumber with Expertiza]]
* [[Rails Testing Overview]]
* [[Expertiza Continuous Integration]]

==References==
* [[Object-Oriented Design and Programming]]

CSC/ECE 517 Spring 2018- Project M1802: 2D Canvas Rendering (Part 2)

2018-04-14T01:09:09Z

Baeastwo: /* HTML Canvas */

CSC/ECE 517 Spring 2018- Project M1802: 2D Canvas Rendering (Part 2)

2018-04-14T01:08:40Z

Baeastwo: /* HTML Canvas */

CSC/ECE 517 Spring 2018- Project M1802: 2D Canvas Rendering (Part 2)

2018-04-14T01:06:42Z

Baeastwo: /* Testing */

File:DrawImage.PNG

2018-04-14T01:06:13Z

Baeastwo: DrawImage for Canvases

DrawImage for Canvases

File:TimedDrawImage.png

2018-04-14T01:05:48Z

Baeastwo: Timed DrawImage for Canvases

Timed DrawImage for Canvases

CSC/ECE 517 Spring 2018- Project M1802: 2D Canvas Rendering (Part 2)

2018-04-14T00:38:26Z

Baeastwo: /* Conclusion */

CSC/ECE 517 Spring 2018- Project M1802: 2D Canvas Rendering (Part 2)

2018-04-14T00:32:10Z

Baeastwo: /* Drawing Canvases with one Thread */

CSC/ECE 517 Spring 2018- Project M1802: 2D Canvas Rendering (Part 2)

2018-04-14T00:29:16Z

Baeastwo:

CSC/ECE 517 Spring 2018- Project M1802: 2D Canvas Rendering (Part 2)

2018-04-14T00:27:13Z

Baeastwo: /* Implementation */

CSC/ECE 517 Spring 2018- Project M1802: 2D Canvas Rendering (Part 2)

2018-04-10T01:39:03Z

Baeastwo: Created page with "'''“M1802: Simplify the 2d Canvas Rendering”''' As robust and fast Servo is, Servo’s implementation of its HTML 2D/3D “<canvas>” has several inefficiencies that make ..."