CSC/ECE 517 Fall 2014/final M1455 yaaa: Difference between revisions

From Expertiza_Wiki
Jump to navigation Jump to search
 
(13 intermediate revisions by 3 users not shown)
Line 27: Line 27:
</pre>
</pre>


* Clone and build servo repository
* Clone servo repository
<pre>
<pre>
git clone https://www.github.com/servo/servo
git clone https://www.github.com/ankit3005/servo
cd servo
cd servo
</pre>
Ensure that modified dependencies point to right sources
<pre>
./mach update-cargo -p freetype-rs
./mach update-cargo -p image
./mach update-cargo -p freetype-rs --precise 753496406098651c83cdaf63db9f75dc42da17c7
./mach update-cargo -p image --precise 3a398831adb22f1200cb68682e446ede187d23b9
</pre>
Build and run basic tests to ensure build success.
<pre>
./mach build
./mach build
./mach run tests/html/about-mozilla.html
./mach run tests/html/about-mozilla.html
Line 118: Line 131:
4. Reported the timing differences for loading PNGs and non-PNGs on the same benchmarks.
4. Reported the timing differences for loading PNGs and non-PNGs on the same benchmarks.
Once the new image library was imported , passed in several test images to record the decoding timing for different images.
Once the new image library was imported , passed in several test images to record the decoding timing for different images.
The timing was compared with the decoding time for images by passing in the same images to the servo build containing the C image library.
The timing was compared with the decoding time for images by passing in the same images to the servo build containing the C image library. The results can be seen in the Test-Cases section below.


== Data and component design <ref>http://www.rust-ci.org/PistonDevelopers/piston/doc/image/index.html</ref>==
== Data and component design <ref>http://www.rust-ci.org/PistonDevelopers/piston/doc/image/index.html</ref>==
Line 313: Line 326:
<pre> ./mach run https://www.http://optipng.sourceforge.net/pngtech/img/lena.png</pre>
<pre> ./mach run https://www.http://optipng.sourceforge.net/pngtech/img/lena.png</pre>
Below you can see the image being rendered after importing the new rust image library: <br/>
Below you can see the image being rendered after importing the new rust image library: <br/>
<center>[[File:new_lena_jpg.PNG]]</center>
<center>[[File:lenaRenderedCorrectly.png]]</center>
 


As you can see the image is being rendered as intended.
Note: This was previously broken due to a bug in our code. We had used the wrong enum type while decoding images that resulted in distorted images. This was fixed by the following [https://github.com/ankit3005/servo/commit/dfa0b92b21144c01223db78d8570aa5ac890922c commit].


5. Our test cases include reporting the timing differences for loading PNGs and non-PNG images. The following report has been made with respect to the same:
5. Our test cases include reporting the timing differences for loading PNGs and non-PNG images. The following report has been made with respect to the same:
 
{| class="wikitable sortable mw-collapsible"
<center>[[File:Report001.PNG]]</center>
|-
!Image Source
!Resolution
!New library decode timing (in ms)
!Old library decode timing (in ms)
|-
|http://meesoft.logicnet.dk/Analyzer/help/Lenna.jpg
|256X256
|40.3814
|45.002006
|-
|http://peps.redprince.net/peps/tiger-Q300.png
|290X300
|78.0789
|29.636256
|-
|http://www.codeproject.com/KB/GDI-plus/ImageProcessing2/grayscale.jpg
|332X300
|106.7289
|64.298119
|-
|http://www.ampsoft.net/webdesign-l/img/JPEG-sample-landscape-standard.jpg
|300X200
|88.852
|42.395043
|-
|http://andreas.com/pixs/ping-sample.gif
|682X310
|106.9462
|124.54975
|-
|http://www.cs.cmu.edu/~chuck/lennapg/len_top.jpg
|400X225
|118.0558
|62.443986
|-
|http://www.roman10.net/wp-content/uploads/2011/08/sample_thumb.jpg
|484X316
|175.0287
|99.462362
|-
|http://www.kksou.com/php-gtk2/gif/sample6.png
|480X240
|223.9521
|44.864687
|-
|http://imagej.nih.gov/ij/images/lena.jpg
|512X512
|257.5638
|165.242363
|-
|http://people.sc.fsu.edu/~jburkardt/data/png/lena.png
|512X512
|397.7005
|88.834362
|-
|http://24.media.tumblr.com/tumblr_m97zes5x3k1rezf8ro1_500.jpg
|500X375
|312.1556
|129.003554
|-
|http://urlnextdoor.com/ai/design-layout/images/sample-png.png
|200X200
|152.053
|20.885724
|-
|https://cdn2.iconfinder.com/data/icons/free-3d-printer-icon-set/512/3d_objects.png
|512X512
|550.5805
|78.025999
|-
|http://www.samontab.com/web/wp-content/uploads/2012/06/lena3.png
|512X512
|811.8095
|97.916758
|-
|http://www.math.uga.edu/~mjlai/images/LenaRecovered.jpg
|1200X900
|909.2834
|662.842394
|-
|https://chadaphone.files.wordpress.com/2013/10/face_detection.png
|640X480
|942.4558
|113.816885
|-
|https://www.drupal.org/files/issues/sample_7.png
|800X600
|708.9273
|157.806566
|-
|http://optipng.sourceforge.net/pngtech/img/lena.png
|512X512
|924.4648
|95.02457
|-
|http://www.ultrasync.net/dee/warehouse_the_odd_stage/lena2_before_1024.png
|1024X1024
|1507.7304
|326.962991
|-
|http://menehune.opt.wfu.edu/CSC365_665/Proj1/Gif/lena.gif
|256X256
|2466.9467
|40.845023
|-
|http://www.openbsd.org/art/puffy/ppuf1000X907.gif
|1000x907
|56338.6327
|615.134272
|}


== Video ==
== Video ==

Latest revision as of 23:23, 4 December 2014

This wiki page contains design details for the project on Evaluate replacing C libraries with modern Rust equivalents for the Mozilla research project Servo.

Background Information <ref>http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Fall_2014/oss_M1455_asa</ref>

Rust

Rust is a new programming language for developing reliable and efficient systems. It is designed to support concurrency and parallelism in building platforms that take full advantage of modern hardware. Its static type system is safe and expressive and it provides strong guarantees about isolation, concurrency execution and memory safety. Rust combines powerful and flexible modern programming constructs with a clear performance model to make program efficiency predictable and manageable. One important way it achieves this is by allowing fine-grained control over memory allocation through contiguous records and stack allocation. This control is balanced with the absolute requirement of safety: Rust’s type system and runtime guarantee the absence of data races, buffer overflow, stack overflow or access to uninitialized or deallocated memory.

Servo

Servo is an experimental project to build a Web browser engine for a new generation of hardware: mobile devices, multi-core processors and high-performance GPUs. Servo builds on top of Rust to provide a secure and reliable foundation. Rust’s lightweight task mechanism promises to allow fine-grained isolation between browser components, such as tabs and extensions, without the need for expensive runtime protection schemes, like operating system process isolation. Servo is not designed to create a full browser but is rather focused on creating a reliable and fast browser engine.

Introduction

Servo currently depends on a lot of C libraries for image decoding. We wish to evaluate switching some of these to new Rust libraries. This project involves rewriting the code that uses these libraries as well as taking measurements before and after to determine the costs involved.

Setup of Development Environment

Servo is currently developed on 64bit OS X and 64bit Linux.

The steps needed to build on a Debian based 64 bit Linux machine are included below. The instructions for other platforms are available here.

  • Install the prerequisite dependencies
sudo apt-get install curl freeglut3-dev \
    libfreetype6-dev libgl1-mesa-dri libglib2.0-dev xorg-dev \
    msttcorefonts gperf g++ cmake python-virtualenv \
    libssl-dev libglfw-dev
  • Clone servo repository
git clone https://www.github.com/ankit3005/servo
cd servo


Ensure that modified dependencies point to right sources

./mach update-cargo -p freetype-rs
./mach update-cargo -p image
./mach update-cargo -p freetype-rs --precise 753496406098651c83cdaf63db9f75dc42da17c7
./mach update-cargo -p image --precise 3a398831adb22f1200cb68682e446ede187d23b9

Build and run basic tests to ensure build success.

./mach build
./mach run tests/html/about-mozilla.html

Architecture of system <ref>https://github.com/servo/servo/wiki/Design#task-supervision-diagram</ref>

Task Supervision Diagram

Task Communication Diagram

The above diagrams gives us an overview of the Servo's architecture.

  • Each box represents a Rust task.
  • Primary tasks are the ones which are represented by blue boxes.
  • Gray boxes are for auxiliary tasks.
  • White boxes are for worker tasks. Each such box represents several tasks, the precise number of which are decided by the workload.
  • Supervisor relationships are shown by dashed lines.
  • Communication channels are shown by solid lines.


The scope of our project is limited to changing the libraries used in the image decoding task shown above.

Requirement analysis

Servo currently depends on a lot of C libraries, because Rust equivalents for these libraries did not exist when the project started. Our project is to evaluate switching some of these to new Rust libraries that have since been created. It involves rewriting the code that uses these libraries as well as taking measurements before and after to determine the costs involved.<ref>https://github.com/servo/servo/wiki/Replace-C-libraries-student-project</ref>

Initial step

Our initial step, implemented for the OSS project involved:

  1. Building Servo.
  2. Adding timing code to the image decoding implementation in the net crate.
  3. Rebuilding Servo.
  4. Reporting numbers for different kinds of images (i.e. PNG, JPEG, GIF).

Final Requirements

Our final project requirements are to:

  • Build Servo and add code that reports (via the println! macro) the time required to decode images into displayable pixels.
  • Add image decoding timing to the profiler.
  • Import the freetype-rs library to replace rust-freetype in Servo, rewriting necessary code.<ref>https://github.com/servo/servo/issues/3369</ref> For this, we are to use Cargo, the dependency manager for Servo.
  • Import the rust-image library to replace the rust-stb-image and stb-image libraries and rewrite the load_from_memory function which uses these libraries.<ref>https://github.com/servo/servo/issues/3368</ref>
  • Report the timing differences for loading PNGs and non-PNGs on the same benchmarks. For this step, we are to consider profiling the result and optimizing the hotspots in rust-image.

Implementation of the Final Requirements

1. Add Image Decoding time to the Profiler To profile the time required by the load_from_memory to decode an image, we make a call to the profiler which outputs the time taken to decode on the console. To enable the profiler to do so, the following files were changed:

  • image_cache_task.rs
  • lib.rs
  • time.rs

To invoke the profiler, we added a category under which to group the timings and passed a TimeProfilerChan (ie. a channel for communicating with the profiler task) to the ImageCacheTask::new method. On successfully building and running servo, we get the following image timings by profiler on the console:

 ./mach run -p 1 https://optipng.sourceforge.net/pngtech/img/lena.html 


2. Imported the freetype-rs library to replace rust-freetype in Servo. We replaced the currently used rust-freetype library with the freetype-rs library. To enable proper functioning of servo after importing the new library, following files were modified in the repository PistonDevelopers/freetype-rs:

  • Cargo.toml
  • src/ffi.rs
  • src/lib.rs
  • src/lib.rs
  • src/library.rs
  • src/tt_os2.rs

These changes were committed and have been pulled by PistonDevelopers. Check the changes on pull request for more info.


3. Imported the rust-image library to replace the rust-stb-image and stb-image libraries and rewrite the load_from_memory function which uses these libraries. In order to import the new library we modified the following files files in the repository [1]:

  • Cargo.toml
  • components/net/lib.rs
  • components/net/image_cache_task.rs
  • components/net/image/base.rs
  • components/net/image/holder.rs
  • components/gfx/render_context.rs
  • components/gfx/lib.rs
  • components/gfx/Cargo.toml
  • components/gfx/display_list/mod.rs
  • components/layout/display_list_builder.rs
  • components/layout/lib.rs


4. Reported the timing differences for loading PNGs and non-PNGs on the same benchmarks. Once the new image library was imported , passed in several test images to record the decoding timing for different images. The timing was compared with the decoding time for images by passing in the same images to the servo build containing the C image library. The results can be seen in the Test-Cases section below.

Data and component design <ref>http://www.rust-ci.org/PistonDevelopers/piston/doc/image/index.html</ref>

Data Design

The system entities that the project deals with is present in the image crate.

The structs and enums used to represent images are as below :

Structures

ImageBuf An Image whose pixels are contained within a vector
Luma A type to hold a grayscale pixel
LumaA A type to hold a grayscale pixel with an alpha channel
MutPixels Mutable pixel iterator
Pixels Immutable pixel iterator
Rgb A type to hold an RGB pixel
Rgba A type to hold an RGB pixel with an alpha channel
SubImage A View into another image


Enums

ColorType An enumeration over supported color types and their bit depths
DynamicImage A Dynamic Image
FilterType Available Sampling Filters
ImageError An enumeration of Image Errors
ImageFormat An enumeration of supported image formats. Not all formats support both encoding and decoding.

Component Design

Rust image : It is an image processing library. This crate provides basic imaging processing functions and methods for converting to and from image formats. All image processing functions provided operate on types that implement the GenericImage trait and return an ImageBuf. The details of other modules and traits included are :


Modules<ref>http://doc.rust-lang.org/guide.html#crates-and-modules</ref>

gif Decoding of GIF Images
imageops Image Processing Functions
jpeg Decoding and Encoding of JPEG Images
png Decoding and Encoding of PNG Images
ppm Encoding of portable pixmap Images
webp Decoding of Webp Images


Traits<ref>http://rustbyexample.com/trait.html</ref>

GenericImage A trait for manipulating images.
ImageDecoder The trait that all decoders implement
MutableRefImage A trait for images that allow providing mutable references to pixels.
Pixel A trait that all pixels implement.


Functions

load Create a new image from a Reader
load_from_memory Create a new image from a byte slice
open Open the image located at the path specified. The image's format is determined from the path's file extension

Design Principles

We are adhering to the following design principles for our implementation:

Open-Closed Principle<ref>http://en.wikipedia.org/wiki/Open/closed_principle</ref>

Although our task is to evaluate replacing C libraries with Rust libraries, we won't actually be modifying any existing functionality.

Interface Segregation Principle<ref>http://en.wikipedia.org/wiki/Interface_segregation_principle</ref>

The existing code is currently using the rust-freetype library for rendering fonts on the browser. We will be replacing this with the freetype-rs library, which divides the functionality offered by the former into smaller libraries which can be individually implemented instead of calling the entire library.

Design Pattern

We have implemented the Adapter Pattern in our Project:

When we updated the image library, the function load_from_memory() in base.rs had to be updated since the return from the updated function accepts a different set of parameters. We implemented the Adapter pattern as a wrapper function over the new load_from_memory() function which performs the below functions: 1. Based on the file extension generate the ImageFormat Enum. 2. The return of the new function is a Result<DynamicImage>. This needs to be converted into a DynamicImage struct.

pub fn load_from_memory(buffer: &[u8],ext: &str) -> Option<DynamicImage> {
    if buffer.len() == 0 {
        return None;
    }
   else {
        //Retrieve the ImageFormat enum to be passed into the updated function
        let image_type: Option< servo_image::ImageFormat > = get_format(ext);
	if image_type == None
	{
	panic!("Image format not supported!");
	}
	else{
        let new_image_type: servo_image::ImageFormat = image_type.unwrap();
        
        //The updated function returns a Result<DynamicImage>
        //below code resolves it to be returned by the wrapper function
	let result = servo_image::load_from_memory(buffer,new_image_type);
	if result.is_ok() {
  	    let v = result.unwrap();
  	    return Some(v);
	}
	else  {	
	    return None;
	}		
   }
   }
   }

Flowchart describing Project Implementation


UML diagrams

Class Diagram

Test Cases

The project does not plan to add new functionality. The test-cases we propose to run are will ensure that tasks that could be performed with the older C libraries can be executed.

The following test cases are proposed.

1. The initial step test case includes printing the decoding time for various image formates and has been included in the wiki: initial step

2. The profiler is meant to capture the image decoding time take by the load_from_memory function. The profiler is run using the following command which gives the decoding time for the image passed as an argument:

./mach run -p 1 https://optipng.sourceforge.net/pngtech/img/lena.html

3. After replacing the rust-freetype library by the freetype-rs library, we have tested that it works as expected by passing a webpage url and checking that the text renders correctly.

./mach run http://simple.wikipedia.org/wiki/Main_Page

4. After replacing the rust-stb-image and stb-image libraries with rust-image library and rewriting the load_from_memory function, we tested the the imports worked as expected by passing in different formats of images as urls and rendering them.

 ./mach run https://www.http://optipng.sourceforge.net/pngtech/img/lena.png

Below you can see the image being rendered after importing the new rust image library:


As you can see the image is being rendered as intended. Note: This was previously broken due to a bug in our code. We had used the wrong enum type while decoding images that resulted in distorted images. This was fixed by the following commit.

5. Our test cases include reporting the timing differences for loading PNGs and non-PNG images. The following report has been made with respect to the same:

Image Source Resolution New library decode timing (in ms) Old library decode timing (in ms)
http://meesoft.logicnet.dk/Analyzer/help/Lenna.jpg 256X256 40.3814 45.002006
http://peps.redprince.net/peps/tiger-Q300.png 290X300 78.0789 29.636256
http://www.codeproject.com/KB/GDI-plus/ImageProcessing2/grayscale.jpg 332X300 106.7289 64.298119
http://www.ampsoft.net/webdesign-l/img/JPEG-sample-landscape-standard.jpg 300X200 88.852 42.395043
http://andreas.com/pixs/ping-sample.gif 682X310 106.9462 124.54975
http://www.cs.cmu.edu/~chuck/lennapg/len_top.jpg 400X225 118.0558 62.443986
http://www.roman10.net/wp-content/uploads/2011/08/sample_thumb.jpg 484X316 175.0287 99.462362
http://www.kksou.com/php-gtk2/gif/sample6.png 480X240 223.9521 44.864687
http://imagej.nih.gov/ij/images/lena.jpg 512X512 257.5638 165.242363
http://people.sc.fsu.edu/~jburkardt/data/png/lena.png 512X512 397.7005 88.834362
http://24.media.tumblr.com/tumblr_m97zes5x3k1rezf8ro1_500.jpg 500X375 312.1556 129.003554
http://urlnextdoor.com/ai/design-layout/images/sample-png.png 200X200 152.053 20.885724
https://cdn2.iconfinder.com/data/icons/free-3d-printer-icon-set/512/3d_objects.png 512X512 550.5805 78.025999
http://www.samontab.com/web/wp-content/uploads/2012/06/lena3.png 512X512 811.8095 97.916758
http://www.math.uga.edu/~mjlai/images/LenaRecovered.jpg 1200X900 909.2834 662.842394
https://chadaphone.files.wordpress.com/2013/10/face_detection.png 640X480 942.4558 113.816885
https://www.drupal.org/files/issues/sample_7.png 800X600 708.9273 157.806566
http://optipng.sourceforge.net/pngtech/img/lena.png 512X512 924.4648 95.02457
http://www.ultrasync.net/dee/warehouse_the_odd_stage/lena2_before_1024.png 1024X1024 1507.7304 326.962991
http://menehune.opt.wfu.edu/CSC365_665/Proj1/Gif/lena.gif 256X256 2466.9467 40.845023
http://www.openbsd.org/art/puffy/ppuf1000X907.gif 1000x907 56338.6327 615.134272

Video

A walkthrough of our entire project is available on this link: Final Project Walkthrough


References

<references/>

See Also

Initial Step Details