“M1802: Simplify the 2d Canvas Rendering”

As robust and fast Servo is, Servo’s implementation of its HTML 2D/3D “<canvas>” has several inefficiencies that make some websites perform slowly or run out of memory when performing complex canvas operations. The goal of this project was to make these websites perform better when loaded in Servo.

Introduction

This section briefly describes the main parts of this project: Servo, Rust, and HTML Canvas.

Servo

Named after a robot from the show Mystery Science Theater 3000, Servo is an experimental web browser layout engine developed by Mozilla. It is currently being developed for 64-bit OS X, 64-bit Linux, 64-bit Windows, and Android. While working with Samsung, Mozilla seeks to create a highly parallel environment where the many things in a web browser like rendering, image decoding, and HTML parsing are handled by isolated tasks. Here is a demo on how Servo performs against Firefox.

Rust

Since existing languages like C++ did not provide a direct way to achieve a parallel environment, Mozilla developed and uses a programming language called Rust to further develop Servo.

HTML Canvas

A <canvas> in HTML is a container for graphics that is used to draw graphics “on the fly” by using Javascript. However, the <canvas> element has no drawing abilities of its own since it only acts as a container for graphics. A script to actually draw the graphics. By using scripts, the HTML <canvas> can:

Be animated
Be interactive
Draw text
Be used in games

Project Description

The goal of this project was to remove the inefficiencies of Servo’s implemetation of the 2D/3D canvas. As reported with issue #10381, a gaming website called http://slither.io/ was performing extremely slowly as of Apr 3, 2016. As of Oct 21, 2016, it was reported with issue #13879 that http://slither.io/ was, in fact, crashing when loaded onto Servo. The suggestion to these issues was to, first, create a test case that measured the amount of time that Servo spends under “drawImage” while doing operations related to a canvas. Another suggestion was to change Servo’s implementation from using one thread per canvas to one thread for all canvases. The steps laid out for this project by Mozilla (both the initial setup steps and optimizing the canvas implementation) are shown below.

Initial Steps:

create a testcase that contains two canvases and uses the drawImage API to draw the contents of one canvas onto the other. Programmatically measure the time this operation takes.

To prepare for the big switch from 1 threads per canvas to 1 thread for all canvases, add a struct CanvasId (u64) type to components/canvas_traits/canvas.rs and add a CanvasId member to each variant of the CanvasMsg enum.

add a CanvasId member to Constellation in components/constellation/constellation.rs which is initialized to 0 and increased by 1 each time handle_create_canvas_paint_thread is called.

make the response_sender argument of handle_create_canvas_paint_thread also include the new CanvasId value, and pass it as an argument to CanvasPaintThread::start. Store the id when it is received for use in all canvas messages.

For each CanvasMsg that is processed by CanvasPaintThread, verify that the id received matches the id that was provided to CanvasPaintThread::start

Subsequent Steps:

extract the innards of CanvasPaintThread into a CanvasData structure,

make CanvasPaintThread store a hashtable of CanvasId->CanvasData

as part of Constellation::start, create a canvas paint thread and store the channel to communicate with it as a member of Constellation. Remove the initial canvas id from the API of Constellation::start.

when handle_create_canvas_paint_thread is invoked, communicate with the canvas thread and have it create a new entry in the hashtable.

when the canvas thread receives a message, perform the operation on the appropriate canvas according to the provided id

optimize the DrawImageInOther operation by drawing on the destination canvas directly, rather than relying on sending a message to another canvas thread. Remove the now-unnecessary IpcSender from the DrawImageInOther enum variant. Verify that the earlier test case demonstrates a performance improvement.

report on how slither.io performs in Servo after all these changes

Design Pattern

Rust only has structs and traits, which are very different from classes and interfaces in terms of usage. Inheritance is effectively nonexistent in Rust, so GoF patterns are nearly always impossible to implement in Rust.

However, the GRASP pattern of Information Expert appears in the Subsequent Steps. CanvasPaintThread is no longer aware of the details of actual canvases, so it delegates the responsibility of actually painting on canvases to CanvasData.

Part of the goal of the Subsequent Steps was to increase the compliance to the Separation of Responsibilities principle. It took an object that previously both processed messages and painted canvases and converted it into an object that processes messages and another object that paints canvases.

Implementation

General Information

Our repo is here.

Building

After setting up the environment required to develop for Servo, we built and compiled Servo as per the instructions on Servo’s Github repo. We used Mozilla’s mach tools to build Servo with Cargo, which is the rust package manager.

git clone https://github.com/servo/servo
cd servo
./mach build --dev
./mach run tests/html/about-mozilla.html

Testing

After successfully building servo, we proceeded to test the amount of time spent under “drawImage” as shown below. The function "drawImage" copies the contents of one canvas in a view to another canvas in the same view. During the "Initial Steps," this test was created to test the initial performance of Servo with regards to canvas rendering.

var t0 = performance.now();
drawImage(25, 25);
var t1 = performance.now();
document.getElementById('Test Result').innerHTML = "DrawImage took " + (t1 - t0) + " milliseconds.";
console.log("DrawImage took " + (t1 - t0) + " milliseconds.");

This was/will be used to measure the performance when drawing a single canvas.

Running Servo

After each successful build through our development process, we ran Servo as per the instructions on Servo’s Github repo.

./servo [url] [arguments] # if you run with nightly build
./mach run [url] [arguments] # if you run with mach

# For example
./mach run https://www.google.com

Initial Steps

The initial steps were the first OSS project. They did little to nothing to change the actual functionality of Servo, instead setting the Subsequent Steps up to have a smooth transition by finding all of the users of CanvasPaintThread and making them send CanvasIds, which can then be used to change CanvasPaintThread to the single-threaded design desired without having to touch the users again.

Adding an Id to Canvas Communications

As shown below, we then associated a CanvasId with each variant of a canvas message; these messages are passed around various parts of the Servo implementation such as the 2D canvas struct, view layout, and various scripts. This required us to make other changed to allow a CanvasId to be sent with each message, as shown below.

Before

After

pub enum CanvasMsg {

   Canvas2d(Canvas2dMsg),
   FromLayout(FromLayoutMsg),
   FromScript(FromScriptMsg),
   Recreate(Size2D<i32>),
   Close(),

}

pub enum CanvasMsg {

   Canvas2d(Canvas2dMsg, CanvasId),
   FromLayout(FromLayoutMsg, CanvasId),
   FromScript(FromScriptMsg, CanvasId),
   Recreate(Size2D<i32>, CanvasId),
   Close(CanvasId),

}

Then, we made the necessary changed to the constellation and the canvas paint thread to be able to initialize, increment, and store the CanvasId. To finish up with the initial steps, we programmatically checked to make sure that each id that was processed by CanvasPaintThread was the same id that was provided when the thread was started (CanvasPaintThread::start).

assert!(canvas_id == painter.canvas_id);

Performance

Almost every time, http://slither.io/ ended up crashing. Other times, it performed very slowly. However, as of 4/2/2018, much more is left to be done (The Subsequent Steps).

Subsequent Steps

The Subsequent Steps were the final project. It builds off of what the Initial Steps lay down. Since the Initial Steps set up all of the users of CanvasPaintThread (even indirect users) to send CanvasIds with their messages, the conversion to use CanvasIds to identify individual canvases was seamless and invisible to the users.

Overview

Previously, a CanvasPaintThread was made for every Canvas, which was very slow when there are many Canvases. To fix this, a single CanvasPaintThread was made that manages all the canvases. The data previously stored in CanvasPaintThread unique to each Canvas was moved to a new data structure known as CanvasData.

Previously, communication channels were opened up between each CanvasPaintThread and what uses it and those channels send the CanvasIds established above alongside their messages. Now, there is a single channel that's repeatedly cloned which instead connects the single CanvasPaintThread and those same users. The users will be unable to tell the difference between the current implementation and the new implementation. The CanvasPaintThread will redirect those existing channels into painting using the appropriate CanvasData structure.

These changes involved moving nearly the entire contents of servo/components/canvas/canvas_paint_thread.rs into the new file servo/components/canvas/canvas_data.rs.

There were slight changes to servo/components/constellation/constellation.rs so that it doesn't try to create multiple CanvasPaintThreads, but those changes were minor and well described in the spec.

There were also changes to servo/components/canvas_traits/canvas.rs and servo/components/script/dom/canvasrenderingcontext2d.rs because of the removed parts of Canvas2dMsg::DrawImageInOther

Initial Form of CanvasPaintThread

From: servo/components/canvas/canvas_paint_thread.rs

   pub struct CanvasPaintThread<'a> {
       drawtarget: DrawTarget,
       /// TODO(pcwalton): Support multiple paths.
       path_builder: PathBuilder,
       state: CanvasPaintState<'a>,
       saved_states: Vec<CanvasPaintState<'a>>,
       webrender_api: webrender_api::RenderApi,
       image_key: Option<webrender_api::ImageKey>,
       /// An old webrender image key that can be deleted when the next epoch ends.
       old_image_key: Option<webrender_api::ImageKey>,
       /// An old webrender image key that can be deleted when the current epoch ends.
       very_old_image_key: Option<webrender_api::ImageKey>,
       canvas_id: CanvasId,
   }

CanvasPaintThread initially stored all the information about a particular canvas. This includes both information used to represent the pixels actually on the canvas and information used to support drawing on the canvas.

There were many, many functions in the previous form of CanvasPaintThread because all the actual painting was done there.

CanvasIds were not usefully used. They were simply added in the initial steps so that the grunt work for the subsequent steps were simpler to implement and more focused.

New Form of CanvasPaintThread

CanvasPaintThread

From: servo/components/canvas/canvas_paint_thread.rs

   pub struct CanvasPaintThread <'a> {
       canvases: HashMap<CanvasId, CanvasData<'a>>,
       next_canvas_id: CanvasId,
   }

The new CanvasPaintThread is much leaner. A single CanvasPaintThread exists with nothing but a HashMap of CanvasId->CanvasData structures and a CanvasId to represent what the next CanvasId will be.

The only functions it has are start (for initially starting it and also the thread itself, which passes CanvasMsg enums it receives around), create_canvas (for generating a new CanvasData structure), canvas (which is just a wrapper for the canvases hashmap to hide some ugliness in Rust's syntax), and process_canvas_2d_message (which exists solely because there are a lot of Canvas2dMsg enum variants and making the thread itself pass them around makes the code very hard to read).

Below is a snippet from process_canvas_2d_message, to give an idea as to what CanvasPaintThread does in general. There are far too many variants of the Canvas2dMsg enum to be reasonably put in this space, but the following snippet should give a rather good idea what it does in general.

   match message {
       Canvas2dMsg::FillText(text, x, y, max_width) => {
           self.canvas(canvas_id).fill_text(text, x, y, max_width)
       },
       Canvas2dMsg::FillRect(ref rect) => {
           self.canvas(canvas_id).fill_rect(rect)
       },
       Canvas2dMsg::StrokeRect(ref rect) => {
           self.canvas(canvas_id).stroke_rect(rect)
       },
       Canvas2dMsg::ClearRect(ref rect) => {
           self.canvas(canvas_id).clear_rect(rect)
       },
   ...
   }

In each Canvas2dMsg enum variant's "match" (think of it like a case statement for a switch but fancier), CanvasPaintThread simply takes the values out of the enum then calls the appropriate function on the appropriate CanvasData structure with those values. It does no checking of those values or otherwise processing them, simply passing them along.

There is one exception to this rule, but the message forces itself to act that way:

   Canvas2dMsg::DrawImageInOther(
           other_canvas_id,
           image_size,
           dest_rect,
           source_rect,
           smoothing
       ) => {
           let mut image_data = self.canvas(canvas_id).read_pixels(
                       source_rect.to_i32(),
                       image_size);
           // TODO: avoid double byte_swap.
           byte_swap(&mut image_data);
           self.canvas(other_canvas_id).draw_image(
               image_data.into(),
               source_rect.size,
               dest_rect,
               source_rect,
               smoothing,
           );
       },
   }

It pulls some data out of one CanvasData and then calls a method on the other CanvasData. For this message to be handled, a decision had to be made. Should CanvasData be aware of the existence of other CanvasData objects or should CanvasPaintThread do some processing to avoid that? There is the third option of making another object to handle it, but seeing as it's only a single message, that option was considered overkill at best, likely far worse. In the end, it was considered a smaller violation of proper design principles to let CanvasPaintThread move data from one CanvasData to another than to inform CanvasData structures about each other.

CanvasData

From: servo/components/canvas/canvas_data.rs

   pub struct CanvasData<'a> {
       drawtarget: DrawTarget,
       /// TODO(pcwalton): Support multiple paths.
       path_builder: PathBuilder,
       state: CanvasPaintState<'a>,
       saved_states: Vec<CanvasPaintState<'a>>,
       webrender_api: webrender_api::RenderApi,
       image_key: Option<webrender_api::ImageKey>,
       /// An old webrender image key that can be deleted when the next epoch ends.
       old_image_key: Option<webrender_api::ImageKey>,
       /// An old webrender image key that can be deleted when the current epoch ends.
       very_old_image_key: Option<webrender_api::ImageKey>,
       canvas_id: CanvasId,
   }

You may notice the similarities between CanvasData and the old form of CanvasPaintThread! Now CanvasData does all the things CanvasPaintThread used to do for the purposes of painting on the canvases (for example, bezier_curve_to, arc, and set_line_cap among many others).

There's nothing particularly new in CanvasData. It's all just moved from CanvasPaintThread.

It is a very focused class now, though. All it does is paint on canvases. There's no processing of messages, only painting. Moreover, the draw_image_in_other function is nowhere to be found, instead replaced by a small amount of processing done by CanvasPaintThread, which just calls draw_image on the one that was originally considered "other" so there's no communication between the CanvasData structures at all.

New Form's Improvements

Design

The new form keeps the responsibilities of converting CanvasMsg enums into their contents and using those contents to paint canvases separate, which improves the overall adherence to the Single Responsibility Principle of Servo. There was also a lot of odd indentation and otherwise messy code in the old form of CanvasPaintThread, which, in the process of preparing the pull request for these changes, were cleaned up, making the overall tidyness of Servo improve.

Performance

The new implementation is also much faster when multiple canvases are involved because there's no need for threading delay to send messages back and forth, waiting for those messages to be sent and received. There's less room for usage of multiple cores to speed up processing, but Servo is already so very parallel that there will be something to keep the other cores busy when CanvasPaintThread is stuck on one thread.

Slither.io doesn't run properly on my machine either before or after the changes, which makes reporting on the success of the changes with respect to Slither.io difficult at best. The framerate improved dramatically, even if it looks like... well... this still. I can't imagine that a website with over 3500 canvases could be made to run more slowly with this change, though.

Using our timed drawimage test, the performance improvement is... well... not really there. On the development build, the drawing time was cut nearly in half from around 19ms to around 9ms, but on the release build, the drawing time is effectively identical, varying between 1.3ms and 1.7ms for both versions. This isn't too surprising, seeing as only two canvases are made and the cost of communicating between two threads isn't very expensive compared to the thousands that the change was made to address.

Conclusion

So far we have associated each variant of a canvas message with a particular canvas id. Even though there is not much improvement is the performance of http://slither.io/, we believe that an improvement can be noticed after going through the Subsequent Steps. Here is a link to our pull request. On 4/2/2018, our contribution was merged into Servo's master branch!

The rendering of 2D canvases has been entirely moved to a single thread with each canvas identified by a CanvasId. The pull request for the final project is here. On 4/24/2018, our final project was merged into Servo's master branch!

CSC/ECE 517 Spring 2018- Project M1802: 2D Canvas Rendering (Part 2)

Contents

Introduction

Servo

Rust

HTML Canvas

Project Description

Initial Steps:

Subsequent Steps:

Design Pattern

Implementation

General Information

Building

Testing

Running Servo

Initial Steps

Adding an Id to Canvas Communications

Performance

Subsequent Steps

Overview

Initial Form of CanvasPaintThread

New Form of CanvasPaintThread

CanvasPaintThread

CanvasData

New Form's Improvements

Design

Performance

Conclusion

References

Navigation menu

CSC/ECE 517 Spring 2018- Project M1802: 2D Canvas Rendering (Part 2)

Introduction

Servo

Rust

HTML Canvas

Project Description

Initial Steps:

Subsequent Steps:

Design Pattern

Implementation

General Information

Building

Testing

Running Servo

Initial Steps

Adding an Id to Canvas Communications

Performance

Subsequent Steps

Overview

Initial Form of CanvasPaintThread

New Form of CanvasPaintThread

CanvasPaintThread

CanvasData

New Form's Improvements

Design

Performance

Conclusion

References

Navigation menu

Search