CSC517 OffScreenCanvas Servo

About Servo:

Servo is a parallel web-engine project under Mozilla, which aims to develop a web engine to better utilize the potential of multiple processing units to parse and display web-pages faster than conventional browser engines. Servo is implemented using Rust programming language, which is similar to C, except the fact that it is specially tuned for better memory safety and concurrency features. Hence, servo seeks to create a highly parallel environment, in which many components are handled by fine-grained and isolated tasks.

Servo makes web-page rendering faster by using parallel layout, styling, web-renders and constellation. Basically, these help to sandbox processes, threads and tasks based on their privilege levels. Sandboxing not only makes way for more responsive web-pages, but it also makes then more secure and robust. Since, when a thread or javascript of a particular section of the webpage fails, it doesn't affects other tabs, browser or eve other elements of the same web-page. So, in a way, Rust lets the browser to "fail better" than other browsers.

CSS3 and HTML5 parser of servo engine have been successfully implemented using Rust, and it successfully passes the acid2 test. Although it is still in early phase of development, it renders web pages significantly more quickly and smoothly, compared to Mozilla's other rendering engines.

Why use rust ? Algorithms which deal with multiple threads running in parallel are inherently difficult, due to synchronization, data sharing and other typical issues which arise while designing parallel algorithms. C++ makes this process even harder as it does not have efficient mechanisms to deal with this. So, the idea behind rust and servo is to rewrite C++ (and create rust) and use that to rewrite a browser.

Off Screen Canvas:

The HTML specification defines a <canvas> element that can use a 2d or 3d rendering context. A new specification was recently developed that defines a canvas that can be used without being associated with an in-page canvas element, allowing for more efficient rendering. To use the OffscreenCanvas API, we have used a RenderingContext that has been obtained from an OffscreenCanvas object to generate new frames. Once a new frame has finished rendering in this context, the transferToImageBitmap() method can be called to save the most recent rendered image. This method returns an ImageBitmap object, which can be used in a variety of Web APIs and also in a second canvas without creating a transfer copy.

Web workers and Offscreen Canvas

Web Workers makes it possible to run a script operation in a background thread separate from the main execution thread of a web application. The advantage of this is that laborious processing can be performed in a separate thread, allowing the main (usually the UI) thread to run without being blocked/slowed down. So, JavaScript could run in the background without affecting the performance of the webpage, independent of other scripts. But, web-workers are treated as external files, which makes it impossible for them to access window object, document object and the parent object. Hence, web-workers can only communicate with the rendering-engine by passing messages. But, how is OffscreenCanvas rendered using web-workers?: Suppose the task is to render an image in the canvas. The rendering engine creates a web-worker, and sends it the image data that it needs to process. The web-worker does the required processing, and returns the resultant image back to the rendering-engine. In this way, an image is rendered using an entirely independent process from the webpage.

Problem Statement:

The following tasks were accomplished in the project:

• Creating the OffscreenCanvas and OffscreenCanvasRenderingContext2d interfaces with stub method implementations.

• Implementing the OffscreenCanvas Constructor [1] that creates a new canvas.

• The OffscreenCanvas Constructor has essentially 2 properties: OffscreenCanvas.width and OffscreenCanvas.height, which represents the width and height of the canvas respectively.

• The constructor also has the method OffscreenCanvas.getContext().

Web IDL

Web IDL is an interface definition language that can be used to describe interfaces that are intended to be implemented in web browsers. It is an IDL variant with:

• A number of features that allow one to more easily describe the behavior of common script objects in a web context.

• A mapping of how interfaces described with Web IDL correspond to language constructs within an ECMAScript execution environment.

Code Flow

• First, IDL files namely OffScreenCanvas.webidl and OffScreenCanvasRenderingContext2D.webidl were added at the locations components/script/dom/webidls/OffScreenCanvas.webidl and components/script/dom/webidls/OffScreenCanvasRenderingContext2D.webidl respectively.

• The corresponding .rs files were added which would implement the DOM interface described in the .webidl files.

Code Snippet

Here are the .webIDL files we added:

OffScreenCanvas.webidl

typedef (OffscreenCanvasRenderingContext2D or WebGLRenderingContext or WebGL2RenderingContext) OffscreenRenderingContext;

dictionary ImageEncodeOptions {
  DOMString type = "image/png";
  unrestricted double quality = 1.0;
};

enum OffscreenRenderingContextId { "2d", "webgl", "webgl2" };

[Constructor([EnforceRange] unsigned long long width, [EnforceRange] unsigned long long height), Exposed=(Window,Worker), Transferable]
interface OffscreenCanvas : EventTarget {
  attribute [EnforceRange] unsigned long long width;
  attribute [EnforceRange] unsigned long long height;

  OffscreenRenderingContext? getContext(OffscreenRenderingContextId contextId, optional any options = null);
  ImageBitmap transferToImageBitmap();
  Promise<Blob> convertToBlob(optional ImageEncodeOptions options);
};

OffScreenCanvasRenderingContext2D.webidl

/* This Source Code Form is subject to the terms of the Mozilla Public
 * License, v. 2.0. If a copy of the MPL was not distributed with this
 * file, You can obtain one at http://mozilla.org/MPL/2.0/. */

// https://html.spec.whatwg.org/multipage/#2dcontext

// typedef (HTMLImageElement or
//          SVGImageElement) HTMLOrSVGImageElement;
typedef HTMLImageElement HTMLOrSVGImageElement;

typedef (HTMLOrSVGImageElement or
         /*HTMLVideoElement or*/
         HTMLCanvasElement or
         /*ImageBitmap or*/
         /*OffscreenCanvas or*/
         /*CSSImageValue*/ CSSStyleValue) CanvasImageSource;

enum CanvasFillRule { "nonzero", "evenodd" };

[Exposed=Window]
interface CanvasRenderingContext2D {
  // back-reference to the canvas
  readonly attribute HTMLCanvasElement canvas;
};
CanvasRenderingContext2D implements CanvasState;
CanvasRenderingContext2D implements CanvasTransform;
CanvasRenderingContext2D implements CanvasCompositing;
CanvasRenderingContext2D implements CanvasImageSmoothing;
CanvasRenderingContext2D implements CanvasFillStrokeStyles;
CanvasRenderingContext2D implements CanvasShadowStyles;
CanvasRenderingContext2D implements CanvasFilters;
CanvasRenderingContext2D implements CanvasRect;
CanvasRenderingContext2D implements CanvasDrawPath;
CanvasRenderingContext2D implements CanvasUserInterface;
CanvasRenderingContext2D implements CanvasText;
CanvasRenderingContext2D implements CanvasDrawImage;
CanvasRenderingContext2D implements CanvasImageData;
CanvasRenderingContext2D implements CanvasPathDrawingStyles;
CanvasRenderingContext2D implements CanvasTextDrawingStyles;
CanvasRenderingContext2D implements CanvasPath;

[Exposed=(PaintWorklet, Window), NoInterfaceObject]
interface CanvasState {
  // state
  void save(); // push state on state stack
  void restore(); // pop state stack and restore state
};

[Exposed=(PaintWorklet, Window), NoInterfaceObject]
interface CanvasTransform {
  // transformations (default transform is the identity matrix)
  void scale(unrestricted double x, unrestricted double y);
  void rotate(unrestricted double angle);
  void translate(unrestricted double x, unrestricted double y);
  void transform(unrestricted double a,
                 unrestricted double b,
                 unrestricted double c,
                 unrestricted double d,
                 unrestricted double e,
                 unrestricted double f);

  // [NewObject] DOMMatrix getTransform();
  void setTransform(unrestricted double a,
                    unrestricted double b,
                    unrestricted double c,
                    unrestricted double d,
                    unrestricted double e,
                    unrestricted double f);
  // void setTransform(optional DOMMatrixInit matrix);
  void resetTransform();
};

[Exposed=(PaintWorklet, Window), NoInterfaceObject]
interface CanvasCompositing {
  // compositing
  attribute unrestricted double globalAlpha; // (default 1.0)
  attribute DOMString globalCompositeOperation; // (default source-over)
};

[Exposed=(PaintWorklet, Window), NoInterfaceObject]
interface CanvasImageSmoothing {
  // image smoothing
  attribute boolean imageSmoothingEnabled; // (default true)
  // attribute ImageSmoothingQuality imageSmoothingQuality; // (default low)
};

[Exposed=(PaintWorklet, Window), NoInterfaceObject]
interface CanvasFillStrokeStyles {
  // colours and styles (see also the CanvasDrawingStyles interface)
  attribute (DOMString or CanvasGradient or CanvasPattern) strokeStyle; // (default black)
  attribute (DOMString or CanvasGradient or CanvasPattern) fillStyle; // (default black)
  CanvasGradient createLinearGradient(double x0, double y0, double x1, double y1);
  [Throws]
  CanvasGradient createRadialGradient(double x0, double y0, double r0, double x1, double y1, double r1);
  [Throws]
  CanvasPattern createPattern(CanvasImageSource image, [TreatNullAs=EmptyString] DOMString repetition);
};

[Exposed=(PaintWorklet, Window), NoInterfaceObject]
interface CanvasShadowStyles {
  // shadows
  attribute unrestricted double shadowOffsetX; // (default 0)
  attribute unrestricted double shadowOffsetY; // (default 0)
  attribute unrestricted double shadowBlur; // (default 0)
  attribute DOMString shadowColor; // (default transparent black)
};

[Exposed=(PaintWorklet, Window), NoInterfaceObject]
interface CanvasFilters {
  // filters
  //attribute DOMString filter; // (default "none")
};

[Exposed=(PaintWorklet, Window), NoInterfaceObject]
interface CanvasRect {
  // rects
  void clearRect(unrestricted double x, unrestricted double y, unrestricted double w, unrestricted double h);
  void fillRect(unrestricted double x, unrestricted double y, unrestricted double w, unrestricted double h);
  void strokeRect(unrestricted double x, unrestricted double y, unrestricted double w, unrestricted double h);
};

[Exposed=(PaintWorklet, Window), NoInterfaceObject]
interface CanvasDrawPath {
  // path API (see also CanvasPath)
  void beginPath();
  void fill(optional CanvasFillRule fillRule = "nonzero");
  //void fill(Path2D path, optional CanvasFillRule fillRule = "nonzero");
  void stroke();
  //void stroke(Path2D path);
  void clip(optional CanvasFillRule fillRule = "nonzero");
  //void clip(Path2D path, optional CanvasFillRule fillRule = "nonzero");
  boolean isPointInPath(unrestricted double x, unrestricted double y,
                        optional CanvasFillRule fillRule = "nonzero");
  //boolean isPointInPath(Path2D path, unrestricted double x, unrestricted double y,
  //                      optional CanvasFillRule fillRule = "nonzero");
  //boolean isPointInStroke(unrestricted double x, unrestricted double y);
  //boolean isPointInStroke(Path2D path, unrestricted double x, unrestricted double y);
};

[Exposed=(PaintWorklet, Window), NoInterfaceObject]
interface CanvasUserInterface {
  //void drawFocusIfNeeded(Element element);
  //void drawFocusIfNeeded(Path2D path, Element element);
  //void scrollPathIntoView();
  //void scrollPathIntoView(Path2D path);
};

[Exposed=(PaintWorklet, Window), NoInterfaceObject]
interface CanvasText {
  // text (see also the CanvasPathDrawingStyles and CanvasTextDrawingStyles interfaces)
  [Pref="dom.canvas-text.enabled"]
  void fillText(DOMString text, unrestricted double x, unrestricted double y,
                optional unrestricted double maxWidth);
  //void strokeText(DOMString text, unrestricted double x, unrestricted double y,
  //                optional unrestricted double maxWidth);
  //TextMetrics measureText(DOMString text);
};

[Exposed=(PaintWorklet, Window), NoInterfaceObject]
interface CanvasDrawImage {
  // drawing images
  [Throws]
  void drawImage(CanvasImageSource image, unrestricted double dx, unrestricted double dy);
  [Throws]
  void drawImage(CanvasImageSource image, unrestricted double dx, unrestricted double dy,
                                          unrestricted double dw, unrestricted double dh);
  [Throws]
  void drawImage(CanvasImageSource image, unrestricted double sx, unrestricted double sy,
                                          unrestricted double sw, unrestricted double sh,
                                          unrestricted double dx, unrestricted double dy,
                                          unrestricted double dw, unrestricted double dh);
};

[Exposed=Window, NoInterfaceObject]
interface CanvasImageData {
  // pixel manipulation
  [Throws]
  ImageData createImageData(long sw, long sh);
  [Throws]
  ImageData createImageData(ImageData imagedata);
  [Throws]
  ImageData getImageData(long sx, long sy, long sw, long sh);
  void putImageData(ImageData imagedata, long dx, long dy);
  void putImageData(ImageData imagedata,
                    long dx, long dy,
                    long dirtyX, long dirtyY,
                    long dirtyWidth, long dirtyHeight);
};

enum CanvasLineCap { "butt", "round", "square" };
enum CanvasLineJoin { "round", "bevel", "miter"};
enum CanvasTextAlign { "start", "end", "left", "right", "center" };
enum CanvasTextBaseline { "top", "hanging", "middle", "alphabetic", "ideographic", "bottom" };
enum CanvasDirection { "ltr", "rtl", "inherit" };

[Exposed=(PaintWorklet, Window), NoInterfaceObject]
interface CanvasPathDrawingStyles {
  // line caps/joins
  attribute unrestricted double lineWidth; // (default 1)
  attribute CanvasLineCap lineCap; // (default "butt")
  attribute CanvasLineJoin lineJoin; // (default "miter")
  attribute unrestricted double miterLimit; // (default 10)

  // dashed lines
  //void setLineDash(sequence<unrestricted double> segments); // default empty
  //sequence<unrestricted double> getLineDash();
  //attribute unrestricted double lineDashOffset;
};

[Exposed=(PaintWorklet, Window), NoInterfaceObject]
interface CanvasTextDrawingStyles {
  // text
  //attribute DOMString font; // (default 10px sans-serif)
  //attribute CanvasTextAlign textAlign; // "start", "end", "left", "right", "center" (default: "start")
  //attribute CanvasTextBaseline textBaseline; // "top", "hanging", "middle", "alphabetic",
                                      // "ideographic", "bottom" (default: "alphabetic")
  //attribute CanvasDirection direction; // "ltr", "rtl", "inherit" (default: "inherit")
};

[Exposed=(PaintWorklet, Window), NoInterfaceObject]
interface CanvasPath {
  // shared path API methods
  void closePath();
  void moveTo(unrestricted double x, unrestricted double y);
  void lineTo(unrestricted double x, unrestricted double y);
  void quadraticCurveTo(unrestricted double cpx, unrestricted double cpy,
                        unrestricted double x, unrestricted double y);

  void bezierCurveTo(unrestricted double cp1x,
                     unrestricted double cp1y,
                     unrestricted double cp2x,
                     unrestricted double cp2y,
                     unrestricted double x,
                     unrestricted double y);

  [Throws]
  void arcTo(unrestricted double x1, unrestricted double y1,
             unrestricted double x2, unrestricted double y2,
             unrestricted double radius);

  void rect(unrestricted double x, unrestricted double y, unrestricted double w, unrestricted double h);

  [Throws]
  void arc(unrestricted double x, unrestricted double y, unrestricted double radius,
           unrestricted double startAngle, unrestricted double endAngle, optional boolean anticlockwise = false);

  [Throws]
  void ellipse(unrestricted double x, unrestricted double y, unrestricted double radius_x,
               unrestricted double radius_y, unrestricted double rotation, unrestricted double startAngle,
               unrestricted double endAngle, optional boolean anticlockwise = false);
};