Expertiza_Wiki - User contributions [en]

CSC/ECE 517 Spring 2018- Project M1803: Implement a web page fuzzer to find rendering mismatches (Part 2)

2018-05-04T16:57:23Z

Adsimps3: /* Introduction */

By Alexander Simpson(adsimps3), Abhay Soni (asoni3), Dileep badveli (dbadvel) and Jake Batty(jbatty)

==Introduction==
This Mozilla project was broken in to 2 main parts: the previous work and the work to be done. The previous work was finished as a part of the OSS project. As a part of the OSS project (explained more below) we created a tool which generates random valid HTML files and automated servo. [https://en.wikipedia.org/wiki/Servo_(layout_engine) Servo] is experimental web browser developed by Mozilla to "to create a new layout engine using a modern programming language". By automating servo (the web browser) we were able to quickly see if servo could render the randomly generated pages.

The goal of this final project was to build off the original project while adding more features. The work we completed as a part of this project is split into a couple main parts. First, we extended the program created in the OSS project to also control Firefox. By also controlling Firefox, we then had 2 screenshots of the randomly generated content - Servo and Firefox . After getting both screenshots, we then compared them using Python and OpenCV. Finally, we were able to expand upon the page generation tool to allow the randomly generated web pages to have more properties/be more complex.

A demo video of our project can be found in 2 parts: [https://www.youtube.com/watch?v=7GP4v54rbQM&feature=youtu.be Part1] and [https://www.youtube.com/watch?v=PDo1G-e1ziE&feature=youtu.be Part2]

==Previous Work (Part of the OSS Project)==

As per the [https://github.com/servo/servo/wiki/Random-web-content-project project description], we were expected to complete the initial steps. The implementation is explained below for each of these steps.

:1) In a new repository, create a program that can generate a skeleton HTML file with a doctype, head element, and body element, print the result to stdout
::- Here is the [https://github.com/asoni3/Random-web-content-generator---CSC-517-OSS-Project link] to the repository which contains code_generation.py file which will be used to generate random valid HTML files.
:2) Add a module to the program that enable generating random content specific to the <head> element (such as inline CSS content inside of a <style> element) and add it to the generated output
::- The file [https://github.com/asoni3/Random-web-content-generator---CSC-517-OSS-Project/blob/master/code_generation.py code_generation.py], contains the code which generates random content specific to the head element and adds style on top of it. As seen in this code, after generating random content to the file, we will add CSS elements on top of this content. We have established a list of commonly used styles, weights, fonts, font_styles, and alignments which will be used at random. For practical purposes, we are limiting the number of options.
:3) Add a module to the program that enables generating random content specific to the <body> element (such as a <p> block that contains randomly generated text) and add it to the generated output
:4) Generate simple random CSS that affects randomly generated content (ie. if there is an element with an id foo, generate a CSS selector like #foo that applies a style like colorto it)
:5) Create a program under Servo's etc/ that launches Servo and causes it to [https://github.com/servo/servo/wiki/Control-Servo-using-WebDriver take a screenshot] of a particular URL - use this to take screenshots of pages randomly generated by the previous program
:Sample Screenshot:
:[[File:Servoscreenshot.png|border]]

==Lists of Tasks==
Below is a list of the tasks that were completed as a part of our final project. Below each task we have described how exactly it was implemented with code examples.

:'''1) Extend the program that controls Servo to also control Firefox using geckodriver'''
Task 1 is relatively simple. It just involves downloading geckodriver and running it. Geckodriver is an open source software engine that allows us to render marked content on a web browser. It should allow us to take screenshots of a particular URL, just like task 5 in the previous work section, but for Firefox instead.

:'''2) Compare the resulting screenshots and report the contents of the generated page if the screenshots differ'''
This task involves automating Firefox to use geckodriver and the current servo program. They both will create 2 different screenshots. If servo and Firefox render it differently, we will report that file and mark the differences.

:'''3) Extend the page generation tool with a bunch of additional strategies, such as:'''
:-Generating elements trees of arbitrary depth
:-Generating sibling elements
:-Extending the set of CSS properties that can be generated (display, background, float, padding, margin, border, etc.)
:-Extending the set of elements that can be generated (span, div, header elements, table (and associated table contents), etc.)
:-Randomly choose whether to generate a document in quirks mode or not

:'''UML Diagram of Tasks:'''
Below is a UML diagram of the tasks of the overall system. As you can see, it covers all 3 tasks. It starts with the code generation and then it splits and takes screenshots on both Firefox and Servo. It then compares then screenshots and reports the distance.
:[[File:UMLDiag.jpg|border|]]

==Implementation of Tasks==
====1) Extend the program that controls Servo to also control Firefox using geckodriver====
:To open up Firefox we would use geckodriver and selenium.After browser is opened,html files will be loaded one at a time. A screenshot would be taken of each html file and stored locally as a png file. The resolution of the file is the width and height variables which gets initialized by the arguments, as well as the num_of_files variable is how many different html pages of screenshots would be needed.
<pre>
browser = webdriver.Firefox() #opens up the firefox browser
for x in range(num_of_files):
file = os.path.abspath("file" + str(x) + ".html")
browser.get("file:///" + file) # go to html page with firefox
set_viewport(browser, width, height,x)
browser.save_screenshot("screen" + str(x) + ".png") # saves the current screen
browser.close()
</pre>

====2) Compare the resulting screenshots and report the contents of the generated page if the screenshots differ====
:To actually make the comparisons we use [https://opencv.org/ OpenCV]. OpenCV allows us to not only make comparisons but will also mark where the differences in. To actually do the comparisons we will first read in the images, convert the images to grayscale, and then call compare_ssim().
<pre>
imageA = cv2.imread(image1)
imageB = cv2.imread(image2)
grayA = cv2.cvtColor(imageA, cv2.COLOR_BGR2GRAY)
grayB = cv2.cvtColor(imageB, cv2.COLOR_BGR2GRAY)
(score, diff) = compare_ssim(grayA, grayB, full=True)
</pre>

This gets us a ''score'' and diff ''values''. The ''score'' variables represents how close the to images are to each other and the ''diff'' variables tells us where the differences are. If the ''score'' indicates that the images are different we then use the ''diff'' value to findContours() and then draw a rectangle around the differences.
<pre>
thresh = cv2.threshold(diff, 0, 255, cv2.THRESH_BINARY_INV | cv2.THRESH_OTSU)[1]
contours = cv2.findContours(thresh.copy(), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
contours = contours[0] if imutils.is_cv2() else contours[1]
</pre>

After getting contours, we get a rectangle for each contour by calling "(x, y, w, h) = cv2.boundingRect(c)". We then combine any rectangles that are inside each other or very close to each other. After combining the rectangles, we then add the rectangles to each image. Lastly we then save image A (with the rectangles to show the differences)
<pre>
for rrr in rectangles:
x = rrr[0]
y = rrr[1]
w = rrr[2]
h = rrr[3]
cv2.rectangle(imageA, (x, y), (x + w, y + h), (0, 0, 255), 2)
cv2.rectangle(imageB, (x, y), (x + w, y + h), (0, 0, 255), 2)
cv2.imwrite(filename,imageA)
</pre>

====3) Extend the page generation tool with a bunch of additional strategies, such as:====
:For task 3 there are several different parts, but the main goal was to increase the complexity of our randomly generated pages. The code_generation.py file was adapted to provide these functions. The implementation of each subtask is explained below.

'''Generating elements trees of arbitrary depth and Generating sibling elements
:We understood a tree of elements to mean divs inside of each other. By using this definition, it was quite simple to create a tree of elements. When RandomDiv is called we don't just return a div with a random section. We also recursively call RandomDiv a random amount of times. We also may call RandomDiv several time on the same level of the tree. This effectively creates sibling elements. Below is the code for RandomDiv.
<pre>
numDivs = random.randint(1,num_sibling_divs)
for i in range(0, numDivs):
yield '<div id="a'+str(random.randrange(0,count[0]))+'">\r\n'
yield RandomSection(count, max_depth, num_sibling_divs, tree_height, max_headers, min_headers)
if tree_height > 0:
treeHeight = random.randint(0, tree_height)
if treeHeight > 0:
yield RandomDiv(treeHeight - 1, count, max_depth, num_sibling_divs, max_headers, min_headers)
yield '</div>'
</pre>

'''Extending the set of CSS properties that can be generated (display, background, float, padding, margin, border, etc.)
:Implementing this was relatively simple. We just did the same thing we did before but with more CSS properties. Below is the code we used to do this. It does not include creating the arrays with different property values but shows us accessing the different arrays.
<pre>
string_css += 'display: ' + displayTypes[random.randint(0, 3)] + ';\r\n'
string_css += 'background-color: rgb(' + str(random.randint(0, 255)) + ',' + str(random.randint(0, 255)) + ',' + str(random.randint(0, 255)) +');\r\n'
string_css += 'float: ' + floatTypes[random.randint(0, 3)] + ';\r\n'
string_css += 'padding: ' + str(random.randint(0, 20)) + 'px ' + str(random.randint(0, 20)) + 'px ' + str(random.randint(0, 20)) + 'px ' + str(random.randint(0, 20)) + 'px;\r\n'
string_css += 'margin: ' + str(random.randint(0, 20)) + 'px ' + str(random.randint(0, 20)) + 'px ' + str(random.randint(0, 20)) + 'px ' + str(random.randint(0, 20)) + 'px;\r\n'
string_css += 'border-width: ' + str(random.randint(0, 20)) + 'px ' + str(random.randint(0, 20)) + 'px ' + str(random.randint(0, 20)) + 'px ' + str(random.randint(0, 20)) + 'px;\r\n'
string_css += 'border-style: ' + borderTypes[random.randint(0, 4)] + ';\r\n'
</pre>

'''Extending the set of elements that can be generated (span, div, header elements, table (and associated table contents), etc.)
:Separate functions were added to create a span, div, table, ordered list, unordered list, and table. Because a span element is in-line and used typically to label a small chunk of text, the RandomSpan function is called at random within the RandomSentence function. Previously, the RandomSentence function simply called RandomWord a random number of times to create a sentence. To add in a span a small percentage of the time, a variable was given a random integer value between 0-99. If the variable was less than 5, RandomSpan was called, else RandomWord was called.
<pre>
#generate a random sentence using random range function
def RandomSentence():
global count
num_of_words = random.randrange(5, 20)
yield RandomWord()
for _ in range(num_of_words-1):
#1 in 20 words will be within a span
z= random.randrange(0,99)
if z<5:
yield ' '
yield ''.join(RandomSpan())
else:
yield ' '
yield ''.join(RandomWord())
yield '. '
</pre>
The other newly added element types- divs, tables, lists- are called at random within the NestedElement function discussed above. These functions utilize the RandomWord and RandomSentence functions to create their elements. For example, see the RandomTable function below:
<pre>
def RandomTable():
column_count = random.randrange(1,10)
row_count = random.randrange(2,20)
yield '<table>\r\n'
#generate table head
yield '\t<tr>\r\n\t\t'
for _ in range(column_count):
yield '<th>'
yield RandomWord()
yield '</th>\r\n'
yield '\t</tr>'
#fill in rows
for _ in range(row_count-1):
yield '\t<tr>\r\n\t\t'
for _ in range(column_count):
yield '<td>'
yield str(random.randrange(0,1000))
yield '</td>'
yield '\r\n\t</tr>\r\n'
yield '</table>\r\n'
</pre>

'''Randomly choose whether to generate a document in quirks mode or not
:Quirks mode is enable whenever an HTML document does not have a DOCTYPE. To randomly enable Quirks mode we only added a DOCTYPE 50% of the time. Below is the code:
<pre>
quirksMode = random.randint(0,1)
if quirksMode == 1 or quirks_mode_possible == False:
yield '<!DOCTYPE html>'
</pre>

==Testing==
Because of the nature of this project, testing the newly added features will be very simple.
: * For task 1, once we are actually controlling Firefox using geckodriver we know its working (by getting a resulting screenshot).
: * Task 2 actually involves us comparing screenshots. By running the code with images that are the same and images that are different, we have proven that this comparison works.
: * Additionally, we have visually tested that task 3 is complete (seeing that the new page generation features work). While we could probably automate some of the testing for task 2 and 3 (to test all possible scenarios) it is out of the scope of this project.

==Conclusion==
The previously completed work allows us to generate simple html documents with a randomized structure, render the page in Servo, and take a screenshot of the page. This project furthered this work by rendering the pages in both Servo and Firefox, taking screenshots of the pages within both browsers, and reporting differences between the two. This project also expanded on the random content generator. This work will now allow users to evaluate Servo’s ability to load web pages. To make this testing even more informative.

CSC/ECE 517 Spring 2018- Project M1803: Implement a web page fuzzer to find rendering mismatches (Part 2)

2018-04-27T21:33:23Z

Adsimps3: /* 3) Extend the page generation tool with a bunch of additional strategies, such as: */

By Alexander Simpson(adsimps3), Abhay Soni (asoni3), Dileep badveli (dbadvel) and Jake Batty(jbatty)

==Introduction==
This Mozilla project was broken in to 2 main parts: the previous work and the work to be done. The previous work was finished as a part of the OSS project. As a part of the OSS project (explained more below) we created a tool which generates random valid HTML files and automated servo. [https://en.wikipedia.org/wiki/Servo_(layout_engine) Servo] is experimental web browser developed by Mozilla to "to create a new layout engine using a modern programming language". By automating servo (the web browser) we were able to quickly see if servo could render the randomly generated pages.

The goal of this final project was to build off the original project while adding more features. The work we completed as a part of this project is split into a couple main parts. First, we extended the program created in the OSS project to also control Firefox. By also controlling Firefox, we then had 2 screenshots of the randomly generated content - Servo and Firefox . After getting both screenshots, we then compared them using Python and OpenCV. Finally, we were able to expand upon the page generation tool to allow the randomly generated web pages to have more properties/be more complex.

==Previous Work (Part of the OSS Project)==

As per the [https://github.com/servo/servo/wiki/Random-web-content-project project description], we were expected to complete the initial steps. The implementation is explained below for each of these steps.

:1) In a new repository, create a program that can generate a skeleton HTML file with a doctype, head element, and body element, print the result to stdout
::- Here is the [https://github.com/asoni3/Random-web-content-generator---CSC-517-OSS-Project link] to the repository which contains code_generation.py file which will be used to generate random valid HTML files.
:2) Add a module to the program that enable generating random content specific to the <head> element (such as inline CSS content inside of a <style> element) and add it to the generated output
::- The file [https://github.com/asoni3/Random-web-content-generator---CSC-517-OSS-Project/blob/master/code_generation.py code_generation.py], contains the code which generates random content specific to the head element and adds style on top of it. As seen in this code, after generating random content to the file, we will add CSS elements on top of this content. We have established a list of commonly used styles, weights, fonts, font_styles, and alignments which will be used at random. For practical purposes, we are limiting the number of options.
:3) Add a module to the program that enables generating random content specific to the <body> element (such as a <p> block that contains randomly generated text) and add it to the generated output
:4) Generate simple random CSS that affects randomly generated content (ie. if there is an element with an id foo, generate a CSS selector like #foo that applies a style like colorto it)
:5) Create a program under Servo's etc/ that launches Servo and causes it to [https://github.com/servo/servo/wiki/Control-Servo-using-WebDriver take a screenshot] of a particular URL - use this to take screenshots of pages randomly generated by the previous program
:Sample Screenshot:
:[[File:Servoscreenshot.png|border]]

==Lists of Tasks==
Below is a list of the tasks that were completed as a part of our final project. Below each task we have described how exactly it was implemented with code examples.

:'''1) Extend the program that controls Servo to also control Firefox using geckodriver'''
Task 1 is relatively simple. It just involves downloading geckodriver and running it. Geckodriver is an open source software engine that allows us to render marked content on a web browser. It should allow us to take screenshots of a particular URL, just like task 5 in the previous work section, but for Firefox instead.

:'''2) Compare the resulting screenshots and report the contents of the generated page if the screenshots differ'''
This task involves automating Firefox to use geckodriver and the current servo program. They both will create 2 different screenshots. If servo and Firefox render it differently, we will report that file and mark the differences.

:'''3) Extend the page generation tool with a bunch of additional strategies, such as:'''
:-Generating elements trees of arbitrary depth
:-Generating sibling elements
:-Extending the set of CSS properties that can be generated (display, background, float, padding, margin, border, etc.)
:-Extending the set of elements that can be generated (span, div, header elements, table (and associated table contents), etc.)
:-Randomly choose whether to generate a document in quirks mode or not

:'''UML Diagram of Tasks:'''
Below is a UML diagram of the tasks of the overall system. As you can see, it covers all 3 tasks. It starts with the code generation and then it splits and takes screenshots on both Firefox and Servo. It then compares then screenshots and reports the distance.
:[[File:UMLDiag.jpg|border|]]

==Implementation of Tasks==
====1) Extend the program that controls Servo to also control Firefox using geckodriver====

====2) Compare the resulting screenshots and report the contents of the generated page if the screenshots differ====
:To actually make the comparisons we use [https://opencv.org/ OpenCV]. OpenCV allows us to not only make comparisons but will also mark where the differences in. To actually do the comparisons we will first read in the images, convert the images to grayscale, and then call compare_ssim().
<pre>
imageA = cv2.imread(image1)
imageB = cv2.imread(image2)
grayA = cv2.cvtColor(imageA, cv2.COLOR_BGR2GRAY)
grayB = cv2.cvtColor(imageB, cv2.COLOR_BGR2GRAY)
(score, diff) = compare_ssim(grayA, grayB, full=True)
</pre>

This gets us a ''score'' and diff ''values''. The ''score'' variables represents how close the to images are to each other and the ''diff'' variables tells us where the differences are. If the ''score'' indicates that the images are different we then use the ''diff'' value to findContours() and then draw a rectangle around the differences.
<pre>
thresh = cv2.threshold(diff, 0, 255, cv2.THRESH_BINARY_INV | cv2.THRESH_OTSU)[1]
contours = cv2.findContours(thresh.copy(), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
contours = contours[0] if imutils.is_cv2() else contours[1]
</pre>

After getting contours, we get a rectangle for each contour by calling "(x, y, w, h) = cv2.boundingRect(c)". We then combine any rectangles that are inside each other or very close to each other. After combining the rectangles, we then add the rectangles to each image. Lastly we then save image A (with the rectangles to show the differences)
<pre>
for rrr in rectangles:
x = rrr[0]
y = rrr[1]
w = rrr[2]
h = rrr[3]
cv2.rectangle(imageA, (x, y), (x + w, y + h), (0, 0, 255), 2)
cv2.rectangle(imageB, (x, y), (x + w, y + h), (0, 0, 255), 2)
cv2.imwrite(filename,imageA)
</pre>

====3) Extend the page generation tool with a bunch of additional strategies, such as:====
:For task 3 there are several different parts, but the main goal was to increase the complexity of our randomly generated pages. The code_generation.py file was adapted to provide these functions. The implementation of each subtask is explained below.

'''Generating elements trees of arbitrary depth and Generating sibling elements
:We understood a tree of elements to mean divs inside of each other. By using this definition, it was quite simple to create a tree of elements. When RandomDiv is called we don't just return a div with a random section. We also recursively call RandomDiv a random amount of times. We also may call RandomDiv several time on the same level of the tree. This effectively creates sibling elements. Below is the code for RandomDiv.
<pre>
numDivs = random.randint(1,num_sibling_divs)
for i in range(0, numDivs):
yield '<div id="a'+str(random.randrange(0,count[0]))+'">\r\n'
yield RandomSection(count, max_depth, num_sibling_divs, tree_height, max_headers, min_headers)
if tree_height > 0:
treeHeight = random.randint(0, tree_height)
if treeHeight > 0:
yield RandomDiv(treeHeight - 1, count, max_depth, num_sibling_divs, max_headers, min_headers)
yield '</div>'
</pre>

'''Extending the set of CSS properties that can be generated (display, background, float, padding, margin, border, etc.)
:Implementing this was relatively simple. We just did the same thing we did before but with more CSS properties. Below is the code we used to do this. It does not include creating the arrays with different property values but shows us accessing the different arrays.
<pre>
string_css += 'display: ' + displayTypes[random.randint(0, 3)] + ';\r\n'
string_css += 'background-color: rgb(' + str(random.randint(0, 255)) + ',' + str(random.randint(0, 255)) + ',' + str(random.randint(0, 255)) +');\r\n'
string_css += 'float: ' + floatTypes[random.randint(0, 3)] + ';\r\n'
string_css += 'padding: ' + str(random.randint(0, 20)) + 'px ' + str(random.randint(0, 20)) + 'px ' + str(random.randint(0, 20)) + 'px ' + str(random.randint(0, 20)) + 'px;\r\n'
string_css += 'margin: ' + str(random.randint(0, 20)) + 'px ' + str(random.randint(0, 20)) + 'px ' + str(random.randint(0, 20)) + 'px ' + str(random.randint(0, 20)) + 'px;\r\n'
string_css += 'border-width: ' + str(random.randint(0, 20)) + 'px ' + str(random.randint(0, 20)) + 'px ' + str(random.randint(0, 20)) + 'px ' + str(random.randint(0, 20)) + 'px;\r\n'
string_css += 'border-style: ' + borderTypes[random.randint(0, 4)] + ';\r\n'
</pre>

'''Extending the set of elements that can be generated (span, div, header elements, table (and associated table contents), etc.)
::::JAKE WILL YOU ADD THIS
'''Randomly choose whether to generate a document in quirks mode or not
:Quirks mode is enable whenever an HTML document does not have a DOCTYPE. To randomly enable Quirks mode we only added a DOCTYPE 50% of the time. Below is the code:
<pre>
quirksMode = random.randint(0,1)
if quirksMode == 1 or quirks_mode_possible == False:
yield '<!DOCTYPE html>'
</pre>

==Testing==
Because of the nature of this project, testing the newly added features will be very simple.
: * For task 1, once we are actually controlling Firefox using geckodriver we know its working (by getting a resulting screenshot).
: * Task 2 actually involves us comparing screenshots. By running the code with images that are the same and images that are different, we have proven that this comparison works.
: * Additionally, we have visually tested that task 3 is complete (seeing that the new page generation features work). While we could probably automate some of the testing for task 2 and 3 (to test all possible scenarios) it is out of the scope of this project.

==Conclusion==
The previously completed work allows us to generate simple html documents with a randomized structure, render the page in Servo, and take a screenshot of the page. This project furthered this work by rendering the pages in both Servo and Firefox, taking screenshots of the pages within both browsers, and reporting differences between the two. This project also expanded on the random content generator. This work will now allow users to evaluate Servo’s ability to load web pages. To make this testing even more informative.

CSC/ECE 517 Spring 2018- Project M1803: Implement a web page fuzzer to find rendering mismatches (Part 2)

2018-04-27T21:32:50Z

Adsimps3: /* 3) Extend the page generation tool with a bunch of additional strategies, such as: */

By Alexander Simpson(adsimps3), Abhay Soni (asoni3), Dileep badveli (dbadvel) and Jake Batty(jbatty)

==Introduction==
This Mozilla project was broken in to 2 main parts: the previous work and the work to be done. The previous work was finished as a part of the OSS project. As a part of the OSS project (explained more below) we created a tool which generates random valid HTML files and automated servo. [https://en.wikipedia.org/wiki/Servo_(layout_engine) Servo] is experimental web browser developed by Mozilla to "to create a new layout engine using a modern programming language". By automating servo (the web browser) we were able to quickly see if servo could render the randomly generated pages.

The goal of this final project was to build off the original project while adding more features. The work we completed as a part of this project is split into a couple main parts. First, we extended the program created in the OSS project to also control Firefox. By also controlling Firefox, we then had 2 screenshots of the randomly generated content - Servo and Firefox . After getting both screenshots, we then compared them using Python and OpenCV. Finally, we were able to expand upon the page generation tool to allow the randomly generated web pages to have more properties/be more complex.

==Previous Work (Part of the OSS Project)==

As per the [https://github.com/servo/servo/wiki/Random-web-content-project project description], we were expected to complete the initial steps. The implementation is explained below for each of these steps.

:1) In a new repository, create a program that can generate a skeleton HTML file with a doctype, head element, and body element, print the result to stdout
::- Here is the [https://github.com/asoni3/Random-web-content-generator---CSC-517-OSS-Project link] to the repository which contains code_generation.py file which will be used to generate random valid HTML files.
:2) Add a module to the program that enable generating random content specific to the <head> element (such as inline CSS content inside of a <style> element) and add it to the generated output
::- The file [https://github.com/asoni3/Random-web-content-generator---CSC-517-OSS-Project/blob/master/code_generation.py code_generation.py], contains the code which generates random content specific to the head element and adds style on top of it. As seen in this code, after generating random content to the file, we will add CSS elements on top of this content. We have established a list of commonly used styles, weights, fonts, font_styles, and alignments which will be used at random. For practical purposes, we are limiting the number of options.
:3) Add a module to the program that enables generating random content specific to the <body> element (such as a <p> block that contains randomly generated text) and add it to the generated output
:4) Generate simple random CSS that affects randomly generated content (ie. if there is an element with an id foo, generate a CSS selector like #foo that applies a style like colorto it)
:5) Create a program under Servo's etc/ that launches Servo and causes it to [https://github.com/servo/servo/wiki/Control-Servo-using-WebDriver take a screenshot] of a particular URL - use this to take screenshots of pages randomly generated by the previous program
:Sample Screenshot:
:[[File:Servoscreenshot.png|border]]

==Lists of Tasks==
Below is a list of the tasks that were completed as a part of our final project. Below each task we have described how exactly it was implemented with code examples.

:'''1) Extend the program that controls Servo to also control Firefox using geckodriver'''
Task 1 is relatively simple. It just involves downloading geckodriver and running it. Geckodriver is an open source software engine that allows us to render marked content on a web browser. It should allow us to take screenshots of a particular URL, just like task 5 in the previous work section, but for Firefox instead.

:'''2) Compare the resulting screenshots and report the contents of the generated page if the screenshots differ'''
This task involves automating Firefox to use geckodriver and the current servo program. They both will create 2 different screenshots. If servo and Firefox render it differently, we will report that file and mark the differences.

:'''3) Extend the page generation tool with a bunch of additional strategies, such as:'''
:-Generating elements trees of arbitrary depth
:-Generating sibling elements
:-Extending the set of CSS properties that can be generated (display, background, float, padding, margin, border, etc.)
:-Extending the set of elements that can be generated (span, div, header elements, table (and associated table contents), etc.)
:-Randomly choose whether to generate a document in quirks mode or not

:'''UML Diagram of Tasks:'''
Below is a UML diagram of the tasks of the overall system. As you can see, it covers all 3 tasks. It starts with the code generation and then it splits and takes screenshots on both Firefox and Servo. It then compares then screenshots and reports the distance.
:[[File:UMLDiag.jpg|border|]]

==Implementation of Tasks==
====1) Extend the program that controls Servo to also control Firefox using geckodriver====

====2) Compare the resulting screenshots and report the contents of the generated page if the screenshots differ====
:To actually make the comparisons we use [https://opencv.org/ OpenCV]. OpenCV allows us to not only make comparisons but will also mark where the differences in. To actually do the comparisons we will first read in the images, convert the images to grayscale, and then call compare_ssim().
<pre>
imageA = cv2.imread(image1)
imageB = cv2.imread(image2)
grayA = cv2.cvtColor(imageA, cv2.COLOR_BGR2GRAY)
grayB = cv2.cvtColor(imageB, cv2.COLOR_BGR2GRAY)
(score, diff) = compare_ssim(grayA, grayB, full=True)
</pre>

This gets us a ''score'' and diff ''values''. The ''score'' variables represents how close the to images are to each other and the ''diff'' variables tells us where the differences are. If the ''score'' indicates that the images are different we then use the ''diff'' value to findContours() and then draw a rectangle around the differences.
<pre>
thresh = cv2.threshold(diff, 0, 255, cv2.THRESH_BINARY_INV | cv2.THRESH_OTSU)[1]
contours = cv2.findContours(thresh.copy(), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
contours = contours[0] if imutils.is_cv2() else contours[1]
</pre>

After getting contours, we get a rectangle for each contour by calling "(x, y, w, h) = cv2.boundingRect(c)". We then combine any rectangles that are inside each other or very close to each other. After combining the rectangles, we then add the rectangles to each image. Lastly we then save image A (with the rectangles to show the differences)
<pre>
for rrr in rectangles:
x = rrr[0]
y = rrr[1]
w = rrr[2]
h = rrr[3]
cv2.rectangle(imageA, (x, y), (x + w, y + h), (0, 0, 255), 2)
cv2.rectangle(imageB, (x, y), (x + w, y + h), (0, 0, 255), 2)
cv2.imwrite(filename,imageA)
</pre>

====3) Extend the page generation tool with a bunch of additional strategies, such as:====
:For task 3 there are several different parts, but the main goal was to increase the complexity of our randomly generated pages. The code_generation.py file was adapted to provide these functions. The implementation of each subtask is explained below.

'''Generating elements trees of arbitrary depth and Generating sibling elements
:We understood a tree of elements to mean divs inside of each other. By using this definition, it was quite simple to create a tree of elements. When RandomDiv is called we don't just return a div with a random section. We also recursively call RandomDiv a random amount of times. We also may call RandomDiv several time on the same level of the tree. This effectively creates sibling elements. Below is the code for RandomDiv.
<pre>
numDivs = random.randint(1,num_sibling_divs)
for i in range(0, numDivs):
yield '<div id="a'+str(random.randrange(0,count[0]))+'">\r\n'
yield RandomSection(count, max_depth, num_sibling_divs, tree_height, max_headers, min_headers)
if tree_height > 0:
treeHeight = random.randint(0, tree_height)
if treeHeight > 0:
yield RandomDiv(treeHeight - 1, count, max_depth, num_sibling_divs, max_headers, min_headers)
yield '</div>'
</pre>

'''Extending the set of CSS properties that can be generated (display, background, float, padding, margin, border, etc.)
:Implementing this was relatively simple. We just did the same thing we did before but with more CSS properties. Below is the code we used to do this. It does not include creating the arrays with different property values but shows us accessing the different arrays.
<pre>
string_css += 'display: ' + displayTypes[random.randint(0, 3)] + ';\r\n'
string_css += 'background-color: rgb(' + str(random.randint(0, 255)) + ',' + str(random.randint(0, 255)) + ',' + str(random.randint(0, 255)) +');\r\n'
string_css += 'float: ' + floatTypes[random.randint(0, 3)] + ';\r\n'
string_css += 'padding: ' + str(random.randint(0, 20)) + 'px ' + str(random.randint(0, 20)) + 'px ' + str(random.randint(0, 20)) + 'px ' + str(random.randint(0, 20)) + 'px;\r\n'
string_css += 'margin: ' + str(random.randint(0, 20)) + 'px ' + str(random.randint(0, 20)) + 'px ' + str(random.randint(0, 20)) + 'px ' + str(random.randint(0, 20)) + 'px;\r\n'
string_css += 'border-width: ' + str(random.randint(0, 20)) + 'px ' + str(random.randint(0, 20)) + 'px ' + str(random.randint(0, 20)) + 'px ' + str(random.randint(0, 20)) + 'px;\r\n'
string_css += 'border-style: ' + borderTypes[random.randint(0, 4)] + ';\r\n'
</pre>

'''Extending the set of elements that can be generated (span, div, header elements, table (and associated table contents), etc.)
:
'''Randomly choose whether to generate a document in quirks mode or not
:Quirks mode is enable whenever an HTML document does not have a DOCTYPE. To randomly enable Quirks mode we only added a DOCTYPE 50% of the time. Below is the code:
<pre>
quirksMode = random.randint(0,1)
if quirksMode == 1 or quirks_mode_possible == False:
yield '<!DOCTYPE html>'
</pre>

==Testing==
Because of the nature of this project, testing the newly added features will be very simple.
: * For task 1, once we are actually controlling Firefox using geckodriver we know its working (by getting a resulting screenshot).
: * Task 2 actually involves us comparing screenshots. By running the code with images that are the same and images that are different, we have proven that this comparison works.
: * Additionally, we have visually tested that task 3 is complete (seeing that the new page generation features work). While we could probably automate some of the testing for task 2 and 3 (to test all possible scenarios) it is out of the scope of this project.

==Conclusion==
The previously completed work allows us to generate simple html documents with a randomized structure, render the page in Servo, and take a screenshot of the page. This project furthered this work by rendering the pages in both Servo and Firefox, taking screenshots of the pages within both browsers, and reporting differences between the two. This project also expanded on the random content generator. This work will now allow users to evaluate Servo’s ability to load web pages. To make this testing even more informative.

CSC/ECE 517 Spring 2018- Project M1803: Implement a web page fuzzer to find rendering mismatches (Part 2)

2018-04-27T21:31:56Z

Adsimps3: /* Implementation of Tasks */

By Alexander Simpson(adsimps3), Abhay Soni (asoni3), Dileep badveli (dbadvel) and Jake Batty(jbatty)

==Introduction==
This Mozilla project was broken in to 2 main parts: the previous work and the work to be done. The previous work was finished as a part of the OSS project. As a part of the OSS project (explained more below) we created a tool which generates random valid HTML files and automated servo. [https://en.wikipedia.org/wiki/Servo_(layout_engine) Servo] is experimental web browser developed by Mozilla to "to create a new layout engine using a modern programming language". By automating servo (the web browser) we were able to quickly see if servo could render the randomly generated pages.

The goal of this final project was to build off the original project while adding more features. The work we completed as a part of this project is split into a couple main parts. First, we extended the program created in the OSS project to also control Firefox. By also controlling Firefox, we then had 2 screenshots of the randomly generated content - Servo and Firefox . After getting both screenshots, we then compared them using Python and OpenCV. Finally, we were able to expand upon the page generation tool to allow the randomly generated web pages to have more properties/be more complex.

==Previous Work (Part of the OSS Project)==

As per the [https://github.com/servo/servo/wiki/Random-web-content-project project description], we were expected to complete the initial steps. The implementation is explained below for each of these steps.

:1) In a new repository, create a program that can generate a skeleton HTML file with a doctype, head element, and body element, print the result to stdout
::- Here is the [https://github.com/asoni3/Random-web-content-generator---CSC-517-OSS-Project link] to the repository which contains code_generation.py file which will be used to generate random valid HTML files.
:2) Add a module to the program that enable generating random content specific to the <head> element (such as inline CSS content inside of a <style> element) and add it to the generated output
::- The file [https://github.com/asoni3/Random-web-content-generator---CSC-517-OSS-Project/blob/master/code_generation.py code_generation.py], contains the code which generates random content specific to the head element and adds style on top of it. As seen in this code, after generating random content to the file, we will add CSS elements on top of this content. We have established a list of commonly used styles, weights, fonts, font_styles, and alignments which will be used at random. For practical purposes, we are limiting the number of options.
:3) Add a module to the program that enables generating random content specific to the <body> element (such as a <p> block that contains randomly generated text) and add it to the generated output
:4) Generate simple random CSS that affects randomly generated content (ie. if there is an element with an id foo, generate a CSS selector like #foo that applies a style like colorto it)
:5) Create a program under Servo's etc/ that launches Servo and causes it to [https://github.com/servo/servo/wiki/Control-Servo-using-WebDriver take a screenshot] of a particular URL - use this to take screenshots of pages randomly generated by the previous program
:Sample Screenshot:
:[[File:Servoscreenshot.png|border]]

==Lists of Tasks==
Below is a list of the tasks that were completed as a part of our final project. Below each task we have described how exactly it was implemented with code examples.

:'''1) Extend the program that controls Servo to also control Firefox using geckodriver'''
Task 1 is relatively simple. It just involves downloading geckodriver and running it. Geckodriver is an open source software engine that allows us to render marked content on a web browser. It should allow us to take screenshots of a particular URL, just like task 5 in the previous work section, but for Firefox instead.

:'''2) Compare the resulting screenshots and report the contents of the generated page if the screenshots differ'''
This task involves automating Firefox to use geckodriver and the current servo program. They both will create 2 different screenshots. If servo and Firefox render it differently, we will report that file and mark the differences.

:'''3) Extend the page generation tool with a bunch of additional strategies, such as:'''
:-Generating elements trees of arbitrary depth
:-Generating sibling elements
:-Extending the set of CSS properties that can be generated (display, background, float, padding, margin, border, etc.)
:-Extending the set of elements that can be generated (span, div, header elements, table (and associated table contents), etc.)
:-Randomly choose whether to generate a document in quirks mode or not

:'''UML Diagram of Tasks:'''
Below is a UML diagram of the tasks of the overall system. As you can see, it covers all 3 tasks. It starts with the code generation and then it splits and takes screenshots on both Firefox and Servo. It then compares then screenshots and reports the distance.
:[[File:UMLDiag.jpg|border|]]

==Implementation of Tasks==
====1) Extend the program that controls Servo to also control Firefox using geckodriver====

====2) Compare the resulting screenshots and report the contents of the generated page if the screenshots differ====
:To actually make the comparisons we use [https://opencv.org/ OpenCV]. OpenCV allows us to not only make comparisons but will also mark where the differences in. To actually do the comparisons we will first read in the images, convert the images to grayscale, and then call compare_ssim().
<pre>
imageA = cv2.imread(image1)
imageB = cv2.imread(image2)
grayA = cv2.cvtColor(imageA, cv2.COLOR_BGR2GRAY)
grayB = cv2.cvtColor(imageB, cv2.COLOR_BGR2GRAY)
(score, diff) = compare_ssim(grayA, grayB, full=True)
</pre>

This gets us a ''score'' and diff ''values''. The ''score'' variables represents how close the to images are to each other and the ''diff'' variables tells us where the differences are. If the ''score'' indicates that the images are different we then use the ''diff'' value to findContours() and then draw a rectangle around the differences.
<pre>
thresh = cv2.threshold(diff, 0, 255, cv2.THRESH_BINARY_INV | cv2.THRESH_OTSU)[1]
contours = cv2.findContours(thresh.copy(), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
contours = contours[0] if imutils.is_cv2() else contours[1]
</pre>

After getting contours, we get a rectangle for each contour by calling "(x, y, w, h) = cv2.boundingRect(c)". We then combine any rectangles that are inside each other or very close to each other. After combining the rectangles, we then add the rectangles to each image. Lastly we then save image A (with the rectangles to show the differences)
<pre>
for rrr in rectangles:
x = rrr[0]
y = rrr[1]
w = rrr[2]
h = rrr[3]
cv2.rectangle(imageA, (x, y), (x + w, y + h), (0, 0, 255), 2)
cv2.rectangle(imageB, (x, y), (x + w, y + h), (0, 0, 255), 2)
cv2.imwrite(filename,imageA)
</pre>

====3) Extend the page generation tool with a bunch of additional strategies, such as:====
:For task 3 there are several different parts, but the main goal was to increase the complexity of our randomly generated pages. The code_generation.py file was adapted to provide these functions. The implementation of each subtask is explained below.

'''Generating elements trees of arbitrary depth and Generating sibling elements
:We understood a tree of elements to mean divs inside of each other. By using this definition, it was quite simple to create a tree of elements. When RandomDiv is called we don't just return a div with a random section. We also recursively call RandomDiv a random amount of times. We also may call RandomDiv several time on the same level of the tree. This effectively creates sibling elements. Below is the code for RandomDiv.
<pre>
numDivs = random.randint(1,num_sibling_divs)
for i in range(0, numDivs):
yield '<div id="a'+str(random.randrange(0,count[0]))+'">\r\n'
yield RandomSection(count, max_depth, num_sibling_divs, tree_height, max_headers, min_headers)
if tree_height > 0:
treeHeight = random.randint(0, tree_height)
if treeHeight > 0:
yield RandomDiv(treeHeight - 1, count, max_depth, num_sibling_divs, max_headers, min_headers)
yield '</div>'
</pre>

'''Extending the set of CSS properties that can be generated (display, background, float, padding, margin, border, etc.)
:Implementing this was relatively simple. We just did the same thing we did before but with more CSS properties. Below is the code we used to do this. It does not include creating the arrays with different property values but shows us accessing the different arrays.
<pre>
string_css += 'display: ' + displayTypes[random.randint(0, 3)] + ';\r\n'
string_css += 'background-color: rgb(' + str(random.randint(0, 255)) + ',' + str(random.randint(0, 255)) + ',' + str(random.randint(0, 255)) +');\r\n'
string_css += 'float: ' + floatTypes[random.randint(0, 3)] + ';\r\n'
string_css += 'padding: ' + str(random.randint(0, 20)) + 'px ' + str(random.randint(0, 20)) + 'px ' + str(random.randint(0, 20)) + 'px ' + str(random.randint(0, 20)) + 'px;\r\n'
string_css += 'margin: ' + str(random.randint(0, 20)) + 'px ' + str(random.randint(0, 20)) + 'px ' + str(random.randint(0, 20)) + 'px ' + str(random.randint(0, 20)) + 'px;\r\n'
string_css += 'border-width: ' + str(random.randint(0, 20)) + 'px ' + str(random.randint(0, 20)) + 'px ' + str(random.randint(0, 20)) + 'px ' + str(random.randint(0, 20)) + 'px;\r\n'
string_css += 'border-style: ' + borderTypes[random.randint(0, 4)] + ';\r\n'
</pre>

'''Extending the set of elements that can be generated (span, div, header elements, table (and associated table contents), etc.)
:
'''Randomly choose whether to generate a document in quirks mode or not
:Quirks mode is enable whenever an HTML document does not have a DOCTYPE. To randomly enable Quirks mode we only added a DOCTYPE 50% of the time.

==Testing==
Because of the nature of this project, testing the newly added features will be very simple.
: * For task 1, once we are actually controlling Firefox using geckodriver we know its working (by getting a resulting screenshot).
: * Task 2 actually involves us comparing screenshots. By running the code with images that are the same and images that are different, we have proven that this comparison works.
: * Additionally, we have visually tested that task 3 is complete (seeing that the new page generation features work). While we could probably automate some of the testing for task 2 and 3 (to test all possible scenarios) it is out of the scope of this project.

==Conclusion==
The previously completed work allows us to generate simple html documents with a randomized structure, render the page in Servo, and take a screenshot of the page. This project furthered this work by rendering the pages in both Servo and Firefox, taking screenshots of the pages within both browsers, and reporting differences between the two. This project also expanded on the random content generator. This work will now allow users to evaluate Servo’s ability to load web pages. To make this testing even more informative.

CSC/ECE 517 Spring 2018- Project M1803: Implement a web page fuzzer to find rendering mismatches (Part 2)

2018-04-27T21:23:05Z

Adsimps3: /* Implementation of Tasks */

By Alexander Simpson(adsimps3), Abhay Soni (asoni3), Dileep badveli (dbadvel) and Jake Batty(jbatty)

==Introduction==
This Mozilla project was broken in to 2 main parts: the previous work and the work to be done. The previous work was finished as a part of the OSS project. As a part of the OSS project (explained more below) we created a tool which generates random valid HTML files and automated servo. [https://en.wikipedia.org/wiki/Servo_(layout_engine) Servo] is experimental web browser developed by Mozilla to "to create a new layout engine using a modern programming language". By automating servo (the web browser) we were able to quickly see if servo could render the randomly generated pages.

The goal of this final project was to build off the original project while adding more features. The work we completed as a part of this project is split into a couple main parts. First, we extended the program created in the OSS project to also control Firefox. By also controlling Firefox, we then had 2 screenshots of the randomly generated content - Servo and Firefox . After getting both screenshots, we then compared them using Python and OpenCV. Finally, we were able to expand upon the page generation tool to allow the randomly generated web pages to have more properties/be more complex.

==Previous Work (Part of the OSS Project)==

As per the [https://github.com/servo/servo/wiki/Random-web-content-project project description], we were expected to complete the initial steps. The implementation is explained below for each of these steps.

:1) In a new repository, create a program that can generate a skeleton HTML file with a doctype, head element, and body element, print the result to stdout
::- Here is the [https://github.com/asoni3/Random-web-content-generator---CSC-517-OSS-Project link] to the repository which contains code_generation.py file which will be used to generate random valid HTML files.
:2) Add a module to the program that enable generating random content specific to the <head> element (such as inline CSS content inside of a <style> element) and add it to the generated output
::- The file [https://github.com/asoni3/Random-web-content-generator---CSC-517-OSS-Project/blob/master/code_generation.py code_generation.py], contains the code which generates random content specific to the head element and adds style on top of it. As seen in this code, after generating random content to the file, we will add CSS elements on top of this content. We have established a list of commonly used styles, weights, fonts, font_styles, and alignments which will be used at random. For practical purposes, we are limiting the number of options.
:3) Add a module to the program that enables generating random content specific to the <body> element (such as a <p> block that contains randomly generated text) and add it to the generated output
:4) Generate simple random CSS that affects randomly generated content (ie. if there is an element with an id foo, generate a CSS selector like #foo that applies a style like colorto it)
:5) Create a program under Servo's etc/ that launches Servo and causes it to [https://github.com/servo/servo/wiki/Control-Servo-using-WebDriver take a screenshot] of a particular URL - use this to take screenshots of pages randomly generated by the previous program
:Sample Screenshot:
:[[File:Servoscreenshot.png|border]]

==Lists of Tasks==
Below is a list of the tasks that were completed as a part of our final project. Below each task we have described how exactly it was implemented with code examples.

:'''1) Extend the program that controls Servo to also control Firefox using geckodriver'''
Task 1 is relatively simple. It just involves downloading geckodriver and running it. Geckodriver is an open source software engine that allows us to render marked content on a web browser. It should allow us to take screenshots of a particular URL, just like task 5 in the previous work section, but for Firefox instead.

:'''2) Compare the resulting screenshots and report the contents of the generated page if the screenshots differ'''
This task involves automating Firefox to use geckodriver and the current servo program. They both will create 2 different screenshots. If servo and Firefox render it differently, we will report that file and mark the differences.

:'''3) Extend the page generation tool with a bunch of additional strategies, such as:'''
:-Generating elements trees of arbitrary depth
:-Generating sibling elements
:-Extending the set of CSS properties that can be generated (display, background, float, padding, margin, border, etc.)
:-Extending the set of elements that can be generated (span, div, header elements, table (and associated table contents), etc.)
:-Randomly choose whether to generate a document in quirks mode or not

:'''UML Diagram of Tasks:'''
Below is a UML diagram of the tasks of the overall system. As you can see, it covers all 3 tasks. It starts with the code generation and then it splits and takes screenshots on both Firefox and Servo. It then compares then screenshots and reports the distance.
:[[File:UMLDiag.jpg|border|]]

==Implementation of Tasks==
:'''1) Extend the program that controls Servo to also control Firefox using geckodriver'''

:'''2) Compare the resulting screenshots and report the contents of the generated page if the screenshots differ'''
:To actually make the comparisons we use [https://opencv.org/ OpenCV]. OpenCV allows us to not only make comparisons but will also mark where the differences in. To actually do the comparisons we will first read in the images, convert the images to grayscale, and then call compare_ssim().
<pre>
imageA = cv2.imread(image1)
imageB = cv2.imread(image2)
grayA = cv2.cvtColor(imageA, cv2.COLOR_BGR2GRAY)
grayB = cv2.cvtColor(imageB, cv2.COLOR_BGR2GRAY)
(score, diff) = compare_ssim(grayA, grayB, full=True)
</pre>

This gets us a ''score'' and diff ''values''. The ''score'' variables represents how close the to images are to each other and the ''diff'' variables tells us where the differences are. If the ''score'' indicates that the images are different we then use the ''diff'' value to findContours() and then draw a rectangle around the differences.
<pre>
thresh = cv2.threshold(diff, 0, 255, cv2.THRESH_BINARY_INV | cv2.THRESH_OTSU)[1]
contours = cv2.findContours(thresh.copy(), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
contours = contours[0] if imutils.is_cv2() else contours[1]
</pre>

After getting contours, we get a rectangle for each contour by calling "(x, y, w, h) = cv2.boundingRect(c)". We then combine any rectangles that are inside each other or very close to each other. After combining the rectangles, we then add the rectangles to each image. Lastly we then save image A (with the rectangles to show the differences)
<pre>
for rrr in rectangles:
x = rrr[0]
y = rrr[1]
w = rrr[2]
h = rrr[3]
cv2.rectangle(imageA, (x, y), (x + w, y + h), (0, 0, 255), 2)
cv2.rectangle(imageB, (x, y), (x + w, y + h), (0, 0, 255), 2)
cv2.imwrite(filename,imageA)
</pre>

:'''3) Extend the page generation tool with a bunch of additional strategies, such as:'''
:For task 3 there are several different parts, but the main goal was to increase the complexity of our randomly generated pages. The code_generation.py file was adapted to provide these functions. The implementation of each subtask is explained below.

'''Generating elements trees of arbitrary depth
:this

'''Generating sibling elements
:
'''Extending the set of CSS properties that can be generated (display, background, float, padding, margin, border, etc.)
:Implementing this was relatively simple. We just did the same thing we did before but with more CSS properties. Below is the code we used to do this. It does not include creating the arrays with different property values but shows us accessing the different arrays.
<pre>
string_css += 'display: ' + displayTypes[random.randint(0, 3)] + ';\r\n'
string_css += 'background-color: rgb(' + str(random.randint(0, 255)) + ',' + str(random.randint(0, 255)) + ',' + str(random.randint(0, 255)) +');\r\n'
string_css += 'float: ' + floatTypes[random.randint(0, 3)] + ';\r\n'
string_css += 'padding: ' + str(random.randint(0, 20)) + 'px ' + str(random.randint(0, 20)) + 'px ' + str(random.randint(0, 20)) + 'px ' + str(random.randint(0, 20)) + 'px;\r\n'
string_css += 'margin: ' + str(random.randint(0, 20)) + 'px ' + str(random.randint(0, 20)) + 'px ' + str(random.randint(0, 20)) + 'px ' + str(random.randint(0, 20)) + 'px;\r\n'
string_css += 'border-width: ' + str(random.randint(0, 20)) + 'px ' + str(random.randint(0, 20)) + 'px ' + str(random.randint(0, 20)) + 'px ' + str(random.randint(0, 20)) + 'px;\r\n'
string_css += 'border-style: ' + borderTypes[random.randint(0, 4)] + ';\r\n'
</pre>

'''Extending the set of elements that can be generated (span, div, header elements, table (and associated table contents), etc.)
:
'''Randomly choose whether to generate a document in quirks mode or not
:Quirks mode is enable whenever an HTML document does not have a DOCTYPE. To randomly enable Quirks mode we only added a DOCTYPE 50% of the time.

==Testing==
Because of the nature of this project, testing the newly added features will be very simple.
: * For task 1, once we are actually controlling Firefox using geckodriver we know its working (by getting a resulting screenshot).
: * Task 2 actually involves us comparing screenshots. By running the code with images that are the same and images that are different, we have proven that this comparison works.
: * Additionally, we have visually tested that task 3 is complete (seeing that the new page generation features work). While we could probably automate some of the testing for task 2 and 3 (to test all possible scenarios) it is out of the scope of this project.

==Conclusion==
The previously completed work allows us to generate simple html documents with a randomized structure, render the page in Servo, and take a screenshot of the page. This project furthered this work by rendering the pages in both Servo and Firefox, taking screenshots of the pages within both browsers, and reporting differences between the two. This project also expanded on the random content generator. This work will now allow users to evaluate Servo’s ability to load web pages. To make this testing even more informative.

CSC/ECE 517 Spring 2018- Project M1803: Implement a web page fuzzer to find rendering mismatches (Part 2)

2018-04-27T21:21:48Z

Adsimps3: /* Implementation of Tasks */

By Alexander Simpson(adsimps3), Abhay Soni (asoni3), Dileep badveli (dbadvel) and Jake Batty(jbatty)

==Introduction==
This Mozilla project was broken in to 2 main parts: the previous work and the work to be done. The previous work was finished as a part of the OSS project. As a part of the OSS project (explained more below) we created a tool which generates random valid HTML files and automated servo. [https://en.wikipedia.org/wiki/Servo_(layout_engine) Servo] is experimental web browser developed by Mozilla to "to create a new layout engine using a modern programming language". By automating servo (the web browser) we were able to quickly see if servo could render the randomly generated pages.

The goal of this final project was to build off the original project while adding more features. The work we completed as a part of this project is split into a couple main parts. First, we extended the program created in the OSS project to also control Firefox. By also controlling Firefox, we then had 2 screenshots of the randomly generated content - Servo and Firefox . After getting both screenshots, we then compared them using Python and OpenCV. Finally, we were able to expand upon the page generation tool to allow the randomly generated web pages to have more properties/be more complex.

==Previous Work (Part of the OSS Project)==

As per the [https://github.com/servo/servo/wiki/Random-web-content-project project description], we were expected to complete the initial steps. The implementation is explained below for each of these steps.

:1) In a new repository, create a program that can generate a skeleton HTML file with a doctype, head element, and body element, print the result to stdout
::- Here is the [https://github.com/asoni3/Random-web-content-generator---CSC-517-OSS-Project link] to the repository which contains code_generation.py file which will be used to generate random valid HTML files.
:2) Add a module to the program that enable generating random content specific to the <head> element (such as inline CSS content inside of a <style> element) and add it to the generated output
::- The file [https://github.com/asoni3/Random-web-content-generator---CSC-517-OSS-Project/blob/master/code_generation.py code_generation.py], contains the code which generates random content specific to the head element and adds style on top of it. As seen in this code, after generating random content to the file, we will add CSS elements on top of this content. We have established a list of commonly used styles, weights, fonts, font_styles, and alignments which will be used at random. For practical purposes, we are limiting the number of options.
:3) Add a module to the program that enables generating random content specific to the <body> element (such as a <p> block that contains randomly generated text) and add it to the generated output
:4) Generate simple random CSS that affects randomly generated content (ie. if there is an element with an id foo, generate a CSS selector like #foo that applies a style like colorto it)
:5) Create a program under Servo's etc/ that launches Servo and causes it to [https://github.com/servo/servo/wiki/Control-Servo-using-WebDriver take a screenshot] of a particular URL - use this to take screenshots of pages randomly generated by the previous program
:Sample Screenshot:
:[[File:Servoscreenshot.png|border]]

==Lists of Tasks==
Below is a list of the tasks that were completed as a part of our final project. Below each task we have described how exactly it was implemented with code examples.

===1) Extend the program that controls Servo to also control Firefox using geckodriver===
Task 1 is relatively simple. It just involves downloading geckodriver and running it. Geckodriver is an open source software engine that allows us to render marked content on a web browser. It should allow us to take screenshots of a particular URL, just like task 5 in the previous work section, but for Firefox instead.

===2) Compare the resulting screenshots and report the contents of the generated page if the screenshots differ===
This task involves automating Firefox to use geckodriver and the current servo program. They both will create 2 different screenshots. If servo and Firefox render it differently, we will report that file and mark the differences.

===3) Extend the page generation tool with a bunch of additional strategies, such as:===
:-Generating elements trees of arbitrary depth
:-Generating sibling elements
:-Extending the set of CSS properties that can be generated (display, background, float, padding, margin, border, etc.)
:-Extending the set of elements that can be generated (span, div, header elements, table (and associated table contents), etc.)
:-Randomly choose whether to generate a document in quirks mode or not

:'''UML Diagram of Tasks:'''
Below is a UML diagram of the tasks of the overall system. As you can see, it covers all 3 tasks. It starts with the code generation and then it splits and takes screenshots on both Firefox and Servo. It then compares then screenshots and reports the distance.
:[[File:UMLDiag.jpg|border|]]

==Implementation of Tasks==
:'''1) Extend the program that controls Servo to also control Firefox using geckodriver'''

:'''2) Compare the resulting screenshots and report the contents of the generated page if the screenshots differ'''
:To actually make the comparisons we use [https://opencv.org/ OpenCV]. OpenCV allows us to not only make comparisons but will also mark where the differences in. To actually do the comparisons we will first read in the images, convert the images to grayscale, and then call compare_ssim().
<pre>
imageA = cv2.imread(image1)
imageB = cv2.imread(image2)
grayA = cv2.cvtColor(imageA, cv2.COLOR_BGR2GRAY)
grayB = cv2.cvtColor(imageB, cv2.COLOR_BGR2GRAY)
(score, diff) = compare_ssim(grayA, grayB, full=True)
</pre>

This gets us a ''score'' and diff ''values''. The ''score'' variables represents how close the to images are to each other and the ''diff'' variables tells us where the differences are. If the ''score'' indicates that the images are different we then use the ''diff'' value to findContours() and then draw a rectangle around the differences.
<pre>
thresh = cv2.threshold(diff, 0, 255, cv2.THRESH_BINARY_INV | cv2.THRESH_OTSU)[1]
contours = cv2.findContours(thresh.copy(), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
contours = contours[0] if imutils.is_cv2() else contours[1]
</pre>

After getting contours, we get a rectangle for each contour by calling "(x, y, w, h) = cv2.boundingRect(c)". We then combine any rectangles that are inside each other or very close to each other. After combining the rectangles, we then add the rectangles to each image. Lastly we then save image A (with the rectangles to show the differences)
<pre>
for rrr in rectangles:
x = rrr[0]
y = rrr[1]
w = rrr[2]
h = rrr[3]
cv2.rectangle(imageA, (x, y), (x + w, y + h), (0, 0, 255), 2)
cv2.rectangle(imageB, (x, y), (x + w, y + h), (0, 0, 255), 2)
cv2.imwrite(filename,imageA)
</pre>

:'''3) Extend the page generation tool with a bunch of additional strategies, such as:'''
:For task 3 there are several different parts, but the main goal was to increase the complexity of our randomly generated pages. The code_generation.py file was adapted to provide these functions. The implementation of each subtask is explained below.

'''Generating elements trees of arbitrary depth
:this

'''Generating sibling elements
:
'''Extending the set of CSS properties that can be generated (display, background, float, padding, margin, border, etc.)
:Implementing this was relatively simple. We just did the same thing we did before but with more CSS properties. Below is the code we used to do this. It does not include creating the arrays with different property values but shows us accessing the different arrays.
<pre>
string_css += 'display: ' + displayTypes[random.randint(0, 3)] + ';\r\n'
string_css += 'background-color: rgb(' + str(random.randint(0, 255)) + ',' + str(random.randint(0, 255)) + ',' + str(random.randint(0, 255)) +');\r\n'
string_css += 'float: ' + floatTypes[random.randint(0, 3)] + ';\r\n'
string_css += 'padding: ' + str(random.randint(0, 20)) + 'px ' + str(random.randint(0, 20)) + 'px ' + str(random.randint(0, 20)) + 'px ' + str(random.randint(0, 20)) + 'px;\r\n'
string_css += 'margin: ' + str(random.randint(0, 20)) + 'px ' + str(random.randint(0, 20)) + 'px ' + str(random.randint(0, 20)) + 'px ' + str(random.randint(0, 20)) + 'px;\r\n'
string_css += 'border-width: ' + str(random.randint(0, 20)) + 'px ' + str(random.randint(0, 20)) + 'px ' + str(random.randint(0, 20)) + 'px ' + str(random.randint(0, 20)) + 'px;\r\n'
string_css += 'border-style: ' + borderTypes[random.randint(0, 4)] + ';\r\n'
</pre>

'''Extending the set of elements that can be generated (span, div, header elements, table (and associated table contents), etc.)
:
'''Randomly choose whether to generate a document in quirks mode or not
:

==Testing==
Because of the nature of this project, testing the newly added features will be very simple.
: * For task 1, once we are actually controlling Firefox using geckodriver we know its working (by getting a resulting screenshot).
: * Task 2 actually involves us comparing screenshots. By running the code with images that are the same and images that are different, we have proven that this comparison works.
: * Additionally, we have visually tested that task 3 is complete (seeing that the new page generation features work). While we could probably automate some of the testing for task 2 and 3 (to test all possible scenarios) it is out of the scope of this project.

==Conclusion==
The previously completed work allows us to generate simple html documents with a randomized structure, render the page in Servo, and take a screenshot of the page. This project furthered this work by rendering the pages in both Servo and Firefox, taking screenshots of the pages within both browsers, and reporting differences between the two. This project also expanded on the random content generator. This work will now allow users to evaluate Servo’s ability to load web pages. To make this testing even more informative.

CSC/ECE 517 Spring 2018- Project M1803: Implement a web page fuzzer to find rendering mismatches (Part 2)

2018-04-27T21:12:04Z

Adsimps3: /* 3) Extend the page generation tool with a bunch of additional strategies, such as: */

CSC/ECE 517 Spring 2018- Project M1803: Implement a web page fuzzer to find rendering mismatches (Part 2)

2018-04-27T21:11:47Z

Adsimps3: /* Lists of Tasks */

CSC/ECE 517 Spring 2018- Project M1803: Implement a web page fuzzer to find rendering mismatches (Part 2)

2018-04-27T21:07:18Z

Adsimps3: /* Implementation of Tasks */

CSC/ECE 517 Spring 2018- Project M1803: Implement a web page fuzzer to find rendering mismatches (Part 2)

2018-04-27T21:04:33Z

Adsimps3: /* Conclusion */

CSC/ECE 517 Spring 2018- Project M1803: Implement a web page fuzzer to find rendering mismatches (Part 2)

2018-04-27T21:02:51Z

Adsimps3: /* Testing */

CSC/ECE 517 Spring 2018- Project M1803: Implement a web page fuzzer to find rendering mismatches (Part 2)

2018-04-27T21:01:03Z

Adsimps3: /* Implementation of Tasks */

CSC/ECE 517 Spring 2018- Project M1803: Implement a web page fuzzer to find rendering mismatches (Part 2)

2018-04-27T20:58:42Z

Adsimps3: /* Implementation of Tasks */

CSC/ECE 517 Spring 2018- Project M1803: Implement a web page fuzzer to find rendering mismatches (Part 2)

2018-04-27T20:50:12Z

Adsimps3: /* Implementation of Tasks */

CSC/ECE 517 Spring 2018- Project M1803: Implement a web page fuzzer to find rendering mismatches (Part 2)

2018-04-27T20:47:53Z

Adsimps3: /* Lists of Tasks */

CSC/ECE 517 Spring 2018- Project M1803: Implement a web page fuzzer to find rendering mismatches (Part 2)

2018-04-27T20:45:42Z

Adsimps3:

CSC/ECE 517 Spring 2018- Project M1803: Implement a web page fuzzer to find rendering mismatches (Part 2)

2018-04-27T20:42:20Z

Adsimps3:

CSC/ECE 517 Spring 2018- Project M1803: Implement a web page fuzzer to find rendering mismatches (Part 2)

2018-04-27T20:41:49Z

Adsimps3: /* Work Completed */

CSC/ECE 517 Spring 2018- Project M1803: Implement a web page fuzzer to find rendering mismatches (Part 2)

2018-04-27T20:41:16Z

Adsimps3: /* Introduction */

CSC/ECE 517 Spring 2018- Project M1803: Implement a web page fuzzer to find rendering mismatches (Part 2)

2018-04-27T20:37:42Z

Adsimps3: /* Work to be done */

CSC/ECE 517 Spring 2018- Project M1803: Implement a web page fuzzer to find rendering mismatches (Part 2)

2018-04-15T20:50:35Z

Adsimps3: /* Work to be done */

CSC/ECE 517 Spring 2018- Project M1803: Implement a web page fuzzer to find rendering mismatches (Part 2)

2018-04-15T20:50:09Z

Adsimps3: /* Work to be done */

CSC/ECE 517 Spring 2018- Project M1803: Implement a web page fuzzer to find rendering mismatches (Part 2)

2018-04-15T20:48:39Z

Adsimps3: /* Introduction */

CSC/ECE 517 Spring 2018- Project M1803: Implement a web page fuzzer to find rendering mismatches (Part 2)

2018-04-15T20:48:02Z

Adsimps3: /* Introduction */

CSC/ECE 517 Spring 2018- Project M1803: Implement a web page fuzzer to find rendering mismatches (Part 2)

2018-04-15T20:42:57Z

Adsimps3:

CSC/ECE 517 Spring 2018- Project M1803: Implement a web page fuzzer to find rendering mismatches (Part 2)

2018-04-15T20:42:42Z

Adsimps3: /* Introduction */

CSC/ECE 517 Spring 2018- Project M1803: Implement a web page fuzzer to find rendering mismatches (Part 2)

2018-04-15T20:40:01Z

Adsimps3: /* Background */

By Alexander Simpson(adsimps3), Abhay Soni (asoni3), Dileep badveli (dbadvel) and Jake Batty(jbatty)

==Introduction==
This Mozilla project was broken in to 2 main parts: the initial and subsequent steps. The initial steps were finished as a part of the OSS project. The goal of this final project is to complete the subsequent steps. As a part of the OSS project (explained more below) we created a tool which generates random valid HTML files and automated servo. Now, as a part of this project we are supposed to extend the program to also control Firefox, compare the resulting screenshots, and expand upon the page generation tool.

==Background==
[https://en.wikipedia.org/wiki/Servo_(layout_engine) Servo] is research project developed by Mozilla to "to create a new layout engine using a modern programming language".

==Previous Work (Part of the OSS Project)==

As per the [https://github.com/servo/servo/wiki/Random-web-content-project project description], we were expected to complete the initial steps. The implementation is explained below for each of these steps.

:1) In a new repository, create a program that can generate a skeleton HTML file with a doctype, head element, and body element, print the result to stdout
::- Here is the [https://github.com/asoni3/Random-web-content-generator---CSC-517-OSS-Project link] to the repository which contains code_generation.py file which will be used to generate random valid HTML files.
:2) Add a module to the program that enable generating random content specific to the <head> element (such as inline CSS content inside of a <style> element) and add it to the generated output
::- The file [https://github.com/asoni3/Random-web-content-generator---CSC-517-OSS-Project/blob/master/code_generation.py code_generation.py], contains the code which generates random content specific to the head element and adds style on top of it. As seen in this code, after generating random content to the file, we will add CSS elements on top of this content. We have established a list of commonly used styles, weights, fonts, font_styles, and alignments which will be used at random. For practical purposes, we are limiting the number of options.
:3) Add a module to the program that enables generating random content specific to the <body> element (such as a <p> block that contains randomly generated text) and add it to the generated output
:4) Generate simple random CSS that affects randomly generated content (ie. if there is an element with an id foo, generate a CSS selector like #foo that applies a style like colorto it)
:5) Create a program under Servo's etc/ that launches Servo and causes it to [https://github.com/servo/servo/wiki/Control-Servo-using-WebDriver take a screenshot] of a particular URL - use this to take screenshots of pages randomly generated by the previous program
:Sample Screenshot:
:[[File:Servoscreenshot.png|border]]

==Work to be done==
Below is a list of the tasks to be done as a part of our final project. Below each task we have described what we think it will take to complete the respective task.

:'''1) Extend the program that controls Servo to also control Firefox using geckodriver'''
:Task 1 is relatively simple. It just involves downloading geckodriver and running it. Geckodriver is an open source software engine that allows us to render marked content on a web browser. It should allow us to take screenshots of a particular URL just like task 5 in the previous work section.

:'''2) Compare the resulting screenshots and report the contents of the generated page if the screenshots differ'''
:This task involves automating Firefox to use geckodriver and the current servo program. They both will create 2 different screenshots. If servo and Firefox render it differently, we will report that file and mark the differences.
:To actually make the comparisons we will use [https://opencv.org/ OpenCV]. OpenCV will allow us to not only make comparisons but will also mark where the differences in. To actually do the comparisons we will first convert the images to grayscale and then call compare_ssim(). This will get us a ''score'' and diff ''values''. The ''score'' variables represents how close the to images are to each other and the ''diff'' variables tells us where the differences are. If the ''score'' indicates that the images are different we will use the ''diff'' value to findContours() and then draw a rectangle around the differences.

:'''3) Extend the page generation tool with a bunch of additional strategies, such as:'''
::-Generating elements trees of arbitrary depth
::-Generating sibling elements
::-Extending the set of CSS properties that can be generated (display, background, float, padding, margin, border, etc.)
::-Extending the set of elements that can be generated (span, div, header elements, table (and associated table contents), etc.)
::-Randomly choose whether to generate a document in quirks mode or not
:For task 3 there are several different parts, but the main goal is to increase the complexity of our randomly generated pages. The code_generation.py file will be adapted to provide these functions. The function used to add random sections will have recursion added to generate elements within elements, resulting in a tree of html elements. The amounting of nesting will be random but limited. Within the sections, tables, lists, spans, and divs may potentially be created. This will involve writing new functions for creating each element, and allowing them to be chosen at random by the function that creates sections of the HTML document. Additionally, the function which provides randomized CSS styling will be extended to generate additional properties.

==Testing==
Because of the nature of this project, testing the newly added features will be very simple. For task 1, once we are actually controlling Firefox using geckodriver we will know it is working. Task 2 actually involves us comparing screenshots. By running the code with images that are the same and images that are different, we will be able to test to see if the comparisons work. Additionally, we will be able to visually test that task 3 is complete (seeing that the new page generation features work). While we could probably automate some of the testing for task 2 and 3 (to test all possible scenarios) it is out of the scope of this project.

==Conclusion==
The previously completed work allows us to generate simple html documents with a randomized structure, render the page in Servo, and take a screenshot of the page. We plan on furthering this work by rendering the pages in both Servo and Firefox, taking screenshots of the pages within both browsers, and reporting differences between the two. Doing so will allow users to evaluate Servo’s ability to load web pages. To make this testing even more informative, we plan to increase the complexity of the structure and styling of the randomly generated html documents.

CSC/ECE 517 Spring 2018- Project M1803: Implement a web page fuzzer to find rendering mismatches (Part 2)

2018-04-15T20:31:59Z

Adsimps3: /* Testing */

By Alexander Simpson(adsimps3), Abhay Soni (asoni3), Dileep badveli (dbadvel) and Jake Batty(jbatty)

==Introduction==
This Mozilla project was broken in to 2 main parts: the initial and subsequent steps. The initial steps were finished as a part of the OSS project. The goal of this final project is to complete the subsequent steps. As a part of the OSS project (explained more below) we created a tool which generates random valid HTML files and automated servo. Now, as a part of this project we are supposed to extend the program to also control Firefox, compare the resulting screenshots, and expand upon the page generation tool.

==Background==
TODO: explain servo

==Previous Work (Part of the OSS Project)==

As per the [https://github.com/servo/servo/wiki/Random-web-content-project project description], we were expected to complete the initial steps. The implementation is explained below for each of these steps.

:1) In a new repository, create a program that can generate a skeleton HTML file with a doctype, head element, and body element, print the result to stdout
::- Here is the [https://github.com/asoni3/Random-web-content-generator---CSC-517-OSS-Project link] to the repository which contains code_generation.py file which will be used to generate random valid HTML files.
:2) Add a module to the program that enable generating random content specific to the <head> element (such as inline CSS content inside of a <style> element) and add it to the generated output
::- The file [https://github.com/asoni3/Random-web-content-generator---CSC-517-OSS-Project/blob/master/code_generation.py code_generation.py], contains the code which generates random content specific to the head element and adds style on top of it. As seen in this code, after generating random content to the file, we will add CSS elements on top of this content. We have established a list of commonly used styles, weights, fonts, font_styles, and alignments which will be used at random. For practical purposes, we are limiting the number of options.
:3) Add a module to the program that enables generating random content specific to the <body> element (such as a <p> block that contains randomly generated text) and add it to the generated output
:4) Generate simple random CSS that affects randomly generated content (ie. if there is an element with an id foo, generate a CSS selector like #foo that applies a style like colorto it)
:5) Create a program under Servo's etc/ that launches Servo and causes it to [https://github.com/servo/servo/wiki/Control-Servo-using-WebDriver take a screenshot] of a particular URL - use this to take screenshots of pages randomly generated by the previous program
:Sample Screenshot:
:[[File:Servoscreenshot.png|border]]

==Work to be done==
Below is a list of the tasks to be done as a part of our final project. Below each task we have described what we think it will take to complete the respective task.

:'''1) Extend the program that controls Servo to also control Firefox using geckodriver'''
:Task 1 is relatively simple. It just involves downloading geckodriver and running it. Geckodriver is an open source software engine that allows us to render marked content on a web browser. It should allow us to take screenshots of a particular URL just like task 5 in the previous work section.

:'''2) Compare the resulting screenshots and report the contents of the generated page if the screenshots differ'''
:This task involves automating Firefox to use geckodriver and the current servo program. They both will create 2 different screenshots. If servo and Firefox render it differently, we will report that file and mark the differences.
:To actually make the comparisons we will use [https://opencv.org/ OpenCV]. OpenCV will allow us to not only make comparisons but will also mark where the differences in. To actually do the comparisons we will first convert the images to grayscale and then call compare_ssim(). This will get us a ''score'' and diff ''values''. The ''score'' variables represents how close the to images are to each other and the ''diff'' variables tells us where the differences are. If the ''score'' indicates that the images are different we will use the ''diff'' value to findContours() and then draw a rectangle around the differences.

:'''3) Extend the page generation tool with a bunch of additional strategies, such as:'''
::-Generating elements trees of arbitrary depth
::-Generating sibling elements
::-Extending the set of CSS properties that can be generated (display, background, float, padding, margin, border, etc.)
::-Extending the set of elements that can be generated (span, div, header elements, table (and associated table contents), etc.)
::-Randomly choose whether to generate a document in quirks mode or not
:For task 3 there are several different parts, but the main goal is to increase the complexity of our randomly generated pages. First we will increase the tree depth to an arbitrary depth. We will then generate sibling elements and increase the CSS styling options. Finally, we will increase the amount of HTML elements that can be generated and randomly choose whether to generate a document in quirks mode or not.
:The code_generation.py file will be adapted to provide these functions. The function used to add random sections will have recursion added to generate elements within elements, resulting in a tree of html elements. The amounting of nesting will be random but limited. Within the sections, tables, lists, spans, and divs may potentially be created. This will involve writing new functions for creating each element, and allowing them to be chosen at random by the function that creates sections of the HTML document.

==Testing==
Because of the nature of this project, testing the newly added features will be very simple. For task 1, once we are actually controlling Firefox using geckodriver we will know it is working. Task 2 actually involves us comparing screenshots. By running the code with images that are the same and images that are different, we will be able to test to see if the comparisons work. Additionally, we will be able to visually test that task 3 is complete (seeing that the new page generation features work). While we could probably automate some of the testing for task 2 and 3 (to test all possible scenarios) it is out of the scope of this project.

==Conclusion==
The previously completed work allows us to generate simple html documents with a randomized structure, render the page in Servo, and take a screenshot of the page. We plan on furthering this work by rendering the pages in both Servo and Firefox, taking screenshots of the pages within both browsers, and reporting differences between the two. Doing so will allow users to evaluate Servo’s ability to load web pages. To make this testing even more informative, we plan to increase the complexity of the structure and styling of the randomly generated html documents.

CSC/ECE 517 Spring 2018- Project M1803: Implement a web page fuzzer to find rendering mismatches (Part 2)

2018-04-15T20:30:57Z

Adsimps3: /* Testing */

By Alexander Simpson(adsimps3), Abhay Soni (asoni3), Dileep badveli (dbadvel) and Jake Batty(jbatty)

==Introduction==
This Mozilla project was broken in to 2 main parts: the initial and subsequent steps. The initial steps were finished as a part of the OSS project. The goal of this final project is to complete the subsequent steps. As a part of the OSS project (explained more below) we created a tool which generates random valid HTML files and automated servo. Now, as a part of this project we are supposed to extend the program to also control Firefox, compare the resulting screenshots, and expand upon the page generation tool.

==Background==
TODO: explain servo

==Previous Work (Part of the OSS Project)==

As per the [https://github.com/servo/servo/wiki/Random-web-content-project project description], we were expected to complete the initial steps. The implementation is explained below for each of these steps.

:1) In a new repository, create a program that can generate a skeleton HTML file with a doctype, head element, and body element, print the result to stdout
::- Here is the [https://github.com/asoni3/Random-web-content-generator---CSC-517-OSS-Project link] to the repository which contains code_generation.py file which will be used to generate random valid HTML files.
:2) Add a module to the program that enable generating random content specific to the <head> element (such as inline CSS content inside of a <style> element) and add it to the generated output
::- The file [https://github.com/asoni3/Random-web-content-generator---CSC-517-OSS-Project/blob/master/code_generation.py code_generation.py], contains the code which generates random content specific to the head element and adds style on top of it. As seen in this code, after generating random content to the file, we will add CSS elements on top of this content. We have established a list of commonly used styles, weights, fonts, font_styles, and alignments which will be used at random. For practical purposes, we are limiting the number of options.
:3) Add a module to the program that enables generating random content specific to the <body> element (such as a <p> block that contains randomly generated text) and add it to the generated output
:4) Generate simple random CSS that affects randomly generated content (ie. if there is an element with an id foo, generate a CSS selector like #foo that applies a style like colorto it)
:5) Create a program under Servo's etc/ that launches Servo and causes it to [https://github.com/servo/servo/wiki/Control-Servo-using-WebDriver take a screenshot] of a particular URL - use this to take screenshots of pages randomly generated by the previous program
:Sample Screenshot:
:[[File:Servoscreenshot.png|border]]

==Work to be done==
Below is a list of the tasks to be done as a part of our final project. Below each task we have described what we think it will take to complete the respective task.

:'''1) Extend the program that controls Servo to also control Firefox using geckodriver'''
:Task 1 is relatively simple. It just involves downloading geckodriver and running it. Geckodriver is an open source software engine that allows us to render marked content on a web browser. It should allow us to take screenshots of a particular URL just like task 5 in the previous work section.

:'''2) Compare the resulting screenshots and report the contents of the generated page if the screenshots differ'''
:This task involves automating Firefox to use geckodriver and the current servo program. They both will create 2 different screenshots. If servo and Firefox render it differently, we will report that file and mark the differences.
:To actually make the comparisons we will use [https://opencv.org/ OpenCV]. OpenCV will allow us to not only make comparisons but will also mark where the differences in. To actually do the comparisons we will first convert the images to grayscale and then call compare_ssim(). This will get us a ''score'' and diff ''values''. The ''score'' variables represents how close the to images are to each other and the ''diff'' variables tells us where the differences are. If the ''score'' indicates that the images are different we will use the ''diff'' value to findContours() and then draw a rectangle around the differences.

:'''3) Extend the page generation tool with a bunch of additional strategies, such as:'''
::-Generating elements trees of arbitrary depth
::-Generating sibling elements
::-Extending the set of CSS properties that can be generated (display, background, float, padding, margin, border, etc.)
::-Extending the set of elements that can be generated (span, div, header elements, table (and associated table contents), etc.)
::-Randomly choose whether to generate a document in quirks mode or not
:For task 3 there are several different parts, but the main goal is to increase the complexity of our randomly generated pages. First we will increase the tree depth to an arbitrary depth. We will then generate sibling elements and increase the CSS styling options. Finally, we will increase the amount of HTML elements that can be generated and randomly choose whether to generate a document in quirks mode or not.
:The code_generation.py file will be adapted to provide these functions. The function used to add random sections will have recursion added to generate elements within elements, resulting in a tree of html elements. The amounting of nesting will be random but limited. Within the sections, tables, lists, spans, and divs may potentially be created. This will involve writing new functions for creating each element, and allowing them to be chosen at random by the function that creates sections of the HTML document.

==Testing==
Because of the nature of this project, testing the newly added features will be very simple. For task 1, once we are actually controlling Firefox using geckodriver we will know it is working. For Task 2 actually involves us comparing screenshots. By running the code with images that are the same and images that are different, we will be able to test to see if the comparisons work. Additionally, we will be able to visually test that task 3 is complete (seeing that the new page generation features work). While we could probably automate some of the testing for task 2 and 3 (to test all possible scenarios) it is out of the scope of this project.

==Conclusion==
The previously completed work allows us to generate simple html documents with a randomized structure, render the page in Servo, and take a screenshot of the page. We plan on furthering this work by rendering the pages in both Servo and Firefox, taking screenshots of the pages within both browsers, and reporting differences between the two. Doing so will allow users to evaluate Servo’s ability to load web pages. To make this testing even more informative, we plan to increase the complexity of the structure and styling of the randomly generated html documents.

CSC/ECE 517 Spring 2018- Project M1803: Implement a web page fuzzer to find rendering mismatches (Part 2)

2018-04-15T20:30:19Z

Adsimps3: /* Testing */

By Alexander Simpson(adsimps3), Abhay Soni (asoni3), Dileep badveli (dbadvel) and Jake Batty(jbatty)

==Introduction==
This Mozilla project was broken in to 2 main parts: the initial and subsequent steps. The initial steps were finished as a part of the OSS project. The goal of this final project is to complete the subsequent steps. As a part of the OSS project (explained more below) we created a tool which generates random valid HTML files and automated servo. Now, as a part of this project we are supposed to extend the program to also control Firefox, compare the resulting screenshots, and expand upon the page generation tool.

==Background==
TODO: explain servo

==Previous Work (Part of the OSS Project)==

As per the [https://github.com/servo/servo/wiki/Random-web-content-project project description], we were expected to complete the initial steps. The implementation is explained below for each of these steps.

:1) In a new repository, create a program that can generate a skeleton HTML file with a doctype, head element, and body element, print the result to stdout
::- Here is the [https://github.com/asoni3/Random-web-content-generator---CSC-517-OSS-Project link] to the repository which contains code_generation.py file which will be used to generate random valid HTML files.
:2) Add a module to the program that enable generating random content specific to the <head> element (such as inline CSS content inside of a <style> element) and add it to the generated output
::- The file [https://github.com/asoni3/Random-web-content-generator---CSC-517-OSS-Project/blob/master/code_generation.py code_generation.py], contains the code which generates random content specific to the head element and adds style on top of it. As seen in this code, after generating random content to the file, we will add CSS elements on top of this content. We have established a list of commonly used styles, weights, fonts, font_styles, and alignments which will be used at random. For practical purposes, we are limiting the number of options.
:3) Add a module to the program that enables generating random content specific to the <body> element (such as a <p> block that contains randomly generated text) and add it to the generated output
:4) Generate simple random CSS that affects randomly generated content (ie. if there is an element with an id foo, generate a CSS selector like #foo that applies a style like colorto it)
:5) Create a program under Servo's etc/ that launches Servo and causes it to [https://github.com/servo/servo/wiki/Control-Servo-using-WebDriver take a screenshot] of a particular URL - use this to take screenshots of pages randomly generated by the previous program
:Sample Screenshot:
:[[File:Servoscreenshot.png|border]]

==Work to be done==
Below is a list of the tasks to be done as a part of our final project. Below each task we have described what we think it will take to complete the respective task.

:'''1) Extend the program that controls Servo to also control Firefox using geckodriver'''
:Task 1 is relatively simple. It just involves downloading geckodriver and running it. Geckodriver is an open source software engine that allows us to render marked content on a web browser. It should allow us to take screenshots of a particular URL just like task 5 in the previous work section.

:'''2) Compare the resulting screenshots and report the contents of the generated page if the screenshots differ'''
:This task involves automating Firefox to use geckodriver and the current servo program. They both will create 2 different screenshots. If servo and Firefox render it differently, we will report that file and mark the differences.
:To actually make the comparisons we will use [https://opencv.org/ OpenCV]. OpenCV will allow us to not only make comparisons but will also mark where the differences in. To actually do the comparisons we will first convert the images to grayscale and then call compare_ssim(). This will get us a ''score'' and diff ''values''. The ''score'' variables represents how close the to images are to each other and the ''diff'' variables tells us where the differences are. If the ''score'' indicates that the images are different we will use the ''diff'' value to findContours() and then draw a rectangle around the differences.

:'''3) Extend the page generation tool with a bunch of additional strategies, such as:'''
::-Generating elements trees of arbitrary depth
::-Generating sibling elements
::-Extending the set of CSS properties that can be generated (display, background, float, padding, margin, border, etc.)
::-Extending the set of elements that can be generated (span, div, header elements, table (and associated table contents), etc.)
::-Randomly choose whether to generate a document in quirks mode or not
:For task 3 there are several different parts, but the main goal is to increase the complexity of our randomly generated pages. First we will increase the tree depth to an arbitrary depth. We will then generate sibling elements and increase the CSS styling options. Finally, we will increase the amount of HTML elements that can be generated and randomly choose whether to generate a document in quirks mode or not.
:The code_generation.py file will be adapted to provide these functions. The function used to add random sections will have recursion added to generate elements within elements, resulting in a tree of html elements. The amounting of nesting will be random but limited. Within the sections, tables, lists, spans, and divs may potentially be created. This will involve writing new functions for creating each element, and allowing them to be chosen at random by the function that creates sections of the HTML document.

==Testing==
Because of the nature of this project, testing the newly added features will be very simple. For task 1, once we are actually controlling Task 2 actually involves us comparing screenshots. By running the code with images that are the same and images that are different, we will be able to test to see if the comparisons work. Additionally, we will be able to visually test that task 3 is complete (seeing that the new page generation features work). While we could probably automate some of the testing for task 2 and 3 (to test all possible scenarios) it is out of the scope of this project.

==Conclusion==
The previously completed work allows us to generate simple html documents with a randomized structure, render the page in Servo, and take a screenshot of the page. We plan on furthering this work by rendering the pages in both Servo and Firefox, taking screenshots of the pages within both browsers, and reporting differences between the two. Doing so will allow users to evaluate Servo’s ability to load web pages. To make this testing even more informative, we plan to increase the complexity of the structure and styling of the randomly generated html documents.

CSC/ECE 517 Spring 2018- Project M1803: Implement a web page fuzzer to find rendering mismatches (Part 2)

2018-04-15T20:29:54Z

Adsimps3: /* Testing */

By Alexander Simpson(adsimps3), Abhay Soni (asoni3), Dileep badveli (dbadvel) and Jake Batty(jbatty)

==Introduction==
This Mozilla project was broken in to 2 main parts: the initial and subsequent steps. The initial steps were finished as a part of the OSS project. The goal of this final project is to complete the subsequent steps. As a part of the OSS project (explained more below) we created a tool which generates random valid HTML files and automated servo. Now, as a part of this project we are supposed to extend the program to also control Firefox, compare the resulting screenshots, and expand upon the page generation tool.

==Background==
TODO: explain servo

==Previous Work (Part of the OSS Project)==

As per the [https://github.com/servo/servo/wiki/Random-web-content-project project description], we were expected to complete the initial steps. The implementation is explained below for each of these steps.

:1) In a new repository, create a program that can generate a skeleton HTML file with a doctype, head element, and body element, print the result to stdout
::- Here is the [https://github.com/asoni3/Random-web-content-generator---CSC-517-OSS-Project link] to the repository which contains code_generation.py file which will be used to generate random valid HTML files.
:2) Add a module to the program that enable generating random content specific to the <head> element (such as inline CSS content inside of a <style> element) and add it to the generated output
::- The file [https://github.com/asoni3/Random-web-content-generator---CSC-517-OSS-Project/blob/master/code_generation.py code_generation.py], contains the code which generates random content specific to the head element and adds style on top of it. As seen in this code, after generating random content to the file, we will add CSS elements on top of this content. We have established a list of commonly used styles, weights, fonts, font_styles, and alignments which will be used at random. For practical purposes, we are limiting the number of options.
:3) Add a module to the program that enables generating random content specific to the <body> element (such as a <p> block that contains randomly generated text) and add it to the generated output
:4) Generate simple random CSS that affects randomly generated content (ie. if there is an element with an id foo, generate a CSS selector like #foo that applies a style like colorto it)
:5) Create a program under Servo's etc/ that launches Servo and causes it to [https://github.com/servo/servo/wiki/Control-Servo-using-WebDriver take a screenshot] of a particular URL - use this to take screenshots of pages randomly generated by the previous program
:Sample Screenshot:
:[[File:Servoscreenshot.png|border]]

==Work to be done==
Below is a list of the tasks to be done as a part of our final project. Below each task we have described what we think it will take to complete the respective task.

:'''1) Extend the program that controls Servo to also control Firefox using geckodriver'''
:Task 1 is relatively simple. It just involves downloading geckodriver and running it. Geckodriver is an open source software engine that allows us to render marked content on a web browser. It should allow us to take screenshots of a particular URL just like task 5 in the previous work section.

:'''2) Compare the resulting screenshots and report the contents of the generated page if the screenshots differ'''
:This task involves automating Firefox to use geckodriver and the current servo program. They both will create 2 different screenshots. If servo and Firefox render it differently, we will report that file and mark the differences.
:To actually make the comparisons we will use [https://opencv.org/ OpenCV]. OpenCV will allow us to not only make comparisons but will also mark where the differences in. To actually do the comparisons we will first convert the images to grayscale and then call compare_ssim(). This will get us a ''score'' and diff ''values''. The ''score'' variables represents how close the to images are to each other and the ''diff'' variables tells us where the differences are. If the ''score'' indicates that the images are different we will use the ''diff'' value to findContours() and then draw a rectangle around the differences.

:'''3) Extend the page generation tool with a bunch of additional strategies, such as:'''
::-Generating elements trees of arbitrary depth
::-Generating sibling elements
::-Extending the set of CSS properties that can be generated (display, background, float, padding, margin, border, etc.)
::-Extending the set of elements that can be generated (span, div, header elements, table (and associated table contents), etc.)
::-Randomly choose whether to generate a document in quirks mode or not
:For task 3 there are several different parts, but the main goal is to increase the complexity of our randomly generated pages. First we will increase the tree depth to an arbitrary depth. We will then generate sibling elements and increase the CSS styling options. Finally, we will increase the amount of HTML elements that can be generated and randomly choose whether to generate a document in quirks mode or not.
:The code_generation.py file will be adapted to provide these functions. The function used to add random sections will have recursion added to generate elements within elements, resulting in a tree of html elements. The amounting of nesting will be random but limited. Within the sections, tables, lists, spans, and divs may potentially be created. This will involve writing new functions for creating each element, and allowing them to be chosen at random by the function that creates sections of the HTML document.

==Testing==
Because of the nature of this project, testing the newly added features will be very simple. For task 1, Task 2 actually involves us comparing screenshots. By running the code with images that are the same and images that are different, we will be able to test to see if the comparisons work. Additionally, we will be able to visually test that task 3 is complete (seeing that the new page generation features work). While we could probably automate some of the testing for task 2 and 3 (to test all possible scenarios) it is out of the scope of this project.

==Conclusion==
The previously completed work allows us to generate simple html documents with a randomized structure, render the page in Servo, and take a screenshot of the page. We plan on furthering this work by rendering the pages in both Servo and Firefox, taking screenshots of the pages within both browsers, and reporting differences between the two. Doing so will allow users to evaluate Servo’s ability to load web pages. To make this testing even more informative, we plan to increase the complexity of the structure and styling of the randomly generated html documents.

CSC/ECE 517 Spring 2018- Project M1803: Implement a web page fuzzer to find rendering mismatches (Part 2)

2018-04-15T20:28:06Z

Adsimps3: /* Work to be done */

By Alexander Simpson(adsimps3), Abhay Soni (asoni3), Dileep badveli (dbadvel) and Jake Batty(jbatty)

==Introduction==
This Mozilla project was broken in to 2 main parts: the initial and subsequent steps. The initial steps were finished as a part of the OSS project. The goal of this final project is to complete the subsequent steps. As a part of the OSS project (explained more below) we created a tool which generates random valid HTML files and automated servo. Now, as a part of this project we are supposed to extend the program to also control Firefox, compare the resulting screenshots, and expand upon the page generation tool.

==Background==
TODO: explain servo

==Previous Work (Part of the OSS Project)==

As per the [https://github.com/servo/servo/wiki/Random-web-content-project project description], we were expected to complete the initial steps. The implementation is explained below for each of these steps.

:1) In a new repository, create a program that can generate a skeleton HTML file with a doctype, head element, and body element, print the result to stdout
::- Here is the [https://github.com/asoni3/Random-web-content-generator---CSC-517-OSS-Project link] to the repository which contains code_generation.py file which will be used to generate random valid HTML files.
:2) Add a module to the program that enable generating random content specific to the <head> element (such as inline CSS content inside of a <style> element) and add it to the generated output
::- The file [https://github.com/asoni3/Random-web-content-generator---CSC-517-OSS-Project/blob/master/code_generation.py code_generation.py], contains the code which generates random content specific to the head element and adds style on top of it. As seen in this code, after generating random content to the file, we will add CSS elements on top of this content. We have established a list of commonly used styles, weights, fonts, font_styles, and alignments which will be used at random. For practical purposes, we are limiting the number of options.
:3) Add a module to the program that enables generating random content specific to the <body> element (such as a <p> block that contains randomly generated text) and add it to the generated output
:4) Generate simple random CSS that affects randomly generated content (ie. if there is an element with an id foo, generate a CSS selector like #foo that applies a style like colorto it)
:5) Create a program under Servo's etc/ that launches Servo and causes it to [https://github.com/servo/servo/wiki/Control-Servo-using-WebDriver take a screenshot] of a particular URL - use this to take screenshots of pages randomly generated by the previous program
:Sample Screenshot:
:[[File:Servoscreenshot.png|border]]

==Work to be done==
Below is a list of the tasks to be done as a part of our final project. Below each task we have described what we think it will take to complete the respective task.

:'''1) Extend the program that controls Servo to also control Firefox using geckodriver'''
:Task 1 is relatively simple. It just involves downloading geckodriver and running it. Geckodriver is an open source software engine that allows us to render marked content on a web browser. It should allow us to take screenshots of a particular URL just like task 5 in the previous work section.

:'''2) Compare the resulting screenshots and report the contents of the generated page if the screenshots differ'''
:This task involves automating Firefox to use geckodriver and the current servo program. They both will create 2 different screenshots. If servo and Firefox render it differently, we will report that file and mark the differences.
:To actually make the comparisons we will use [https://opencv.org/ OpenCV]. OpenCV will allow us to not only make comparisons but will also mark where the differences in. To actually do the comparisons we will first convert the images to grayscale and then call compare_ssim(). This will get us a ''score'' and diff ''values''. The ''score'' variables represents how close the to images are to each other and the ''diff'' variables tells us where the differences are. If the ''score'' indicates that the images are different we will use the ''diff'' value to findContours() and then draw a rectangle around the differences.

:'''3) Extend the page generation tool with a bunch of additional strategies, such as:'''
::-Generating elements trees of arbitrary depth
::-Generating sibling elements
::-Extending the set of CSS properties that can be generated (display, background, float, padding, margin, border, etc.)
::-Extending the set of elements that can be generated (span, div, header elements, table (and associated table contents), etc.)
::-Randomly choose whether to generate a document in quirks mode or not
:For task 3 there are several different parts, but the main goal is to increase the complexity of our randomly generated pages. First we will increase the tree depth to an arbitrary depth. We will then generate sibling elements and increase the CSS styling options. Finally, we will increase the amount of HTML elements that can be generated and randomly choose whether to generate a document in quirks mode or not.
:The code_generation.py file will be adapted to provide these functions. The function used to add random sections will have recursion added to generate elements within elements, resulting in a tree of html elements. The amounting of nesting will be random but limited. Within the sections, tables, lists, spans, and divs may potentially be created. This will involve writing new functions for creating each element, and allowing them to be chosen at random by the function that creates sections of the HTML document.

==Testing==
Because of the nature of this project, testing the newly added features will be very simple. Task 2 actually involves us comparing screenshots. By comparing the actual screenshots we will test that things work. Additionally, we will be able to visually test that task 3 is complete (seeing that the new page generation features work). While we could probably automate some of the testing for task 3 (to test all possible scenarios) it is out of the scope of this project.

==Conclusion==
The previously completed work allows us to generate simple html documents with a randomized structure, render the page in Servo, and take a screenshot of the page. We plan on furthering this work by rendering the pages in both Servo and Firefox, taking screenshots of the pages within both browsers, and reporting differences between the two. Doing so will allow users to evaluate Servo’s ability to load web pages. To make this testing even more informative, we plan to increase the complexity of the structure and styling of the randomly generated html documents.

CSC/ECE 517 Spring 2018- Project M1803: Implement a web page fuzzer to find rendering mismatches (Part 2)

2018-04-15T20:27:49Z

Adsimps3: /* Work to be done */

By Alexander Simpson(adsimps3), Abhay Soni (asoni3), Dileep badveli (dbadvel) and Jake Batty(jbatty)

==Introduction==
This Mozilla project was broken in to 2 main parts: the initial and subsequent steps. The initial steps were finished as a part of the OSS project. The goal of this final project is to complete the subsequent steps. As a part of the OSS project (explained more below) we created a tool which generates random valid HTML files and automated servo. Now, as a part of this project we are supposed to extend the program to also control Firefox, compare the resulting screenshots, and expand upon the page generation tool.

==Background==
TODO: explain servo

==Previous Work (Part of the OSS Project)==

As per the [https://github.com/servo/servo/wiki/Random-web-content-project project description], we were expected to complete the initial steps. The implementation is explained below for each of these steps.

:1) In a new repository, create a program that can generate a skeleton HTML file with a doctype, head element, and body element, print the result to stdout
::- Here is the [https://github.com/asoni3/Random-web-content-generator---CSC-517-OSS-Project link] to the repository which contains code_generation.py file which will be used to generate random valid HTML files.
:2) Add a module to the program that enable generating random content specific to the <head> element (such as inline CSS content inside of a <style> element) and add it to the generated output
::- The file [https://github.com/asoni3/Random-web-content-generator---CSC-517-OSS-Project/blob/master/code_generation.py code_generation.py], contains the code which generates random content specific to the head element and adds style on top of it. As seen in this code, after generating random content to the file, we will add CSS elements on top of this content. We have established a list of commonly used styles, weights, fonts, font_styles, and alignments which will be used at random. For practical purposes, we are limiting the number of options.
:3) Add a module to the program that enables generating random content specific to the <body> element (such as a <p> block that contains randomly generated text) and add it to the generated output
:4) Generate simple random CSS that affects randomly generated content (ie. if there is an element with an id foo, generate a CSS selector like #foo that applies a style like colorto it)
:5) Create a program under Servo's etc/ that launches Servo and causes it to [https://github.com/servo/servo/wiki/Control-Servo-using-WebDriver take a screenshot] of a particular URL - use this to take screenshots of pages randomly generated by the previous program
:Sample Screenshot:
:[[File:Servoscreenshot.png|border]]

==Work to be done==
Below is a list of the tasks to be done as a part of our final project. Below each task we have described what we think it will take to complete the respective task.

:'''1) Extend the program that controls Servo to also control Firefox using geckodriver'''
:Task 1 is relatively simple. It just involves downloading geckodriver and running it. Geckodriver is an open source software engine that allows us to render marked content on a web browser. It should allow us to take screenshots of a particular URL just like task 5 in the previous work section.

:'''2) Compare the resulting screenshots and report the contents of the generated page if the screenshots differ'''
:This task involves automating Firefox to use geckodriver and the current servo program. They both will create 2 different screenshots. If servo and Firefox render it differently, we will report that file and mark the differences.
:To actually make the comparisons we will use [https://opencv.org/ OpenCV]. OpenCV will allow us to not only make comparisons but will also mark where the differences in. To actually do the comparisons we will first convert the images to grayscale and then call compare_ssim(). This will get us a ''score'' and diff ''values''. The ''score'' variables represents how close the to images are to each other and the ''diff'' variables tells us where the differences are. If the ''score'' indicates that the images are different we will use the ''diff'' value to findContours() and then draw a rectangle around the differences.

:'''3) Extend the page generation tool with a bunch of additional strategies, such as:'''
::-Generating elements trees of arbitrary depth
::-Generating sibling elements
::-Extending the set of CSS properties that can be generated (display, background, float, padding, margin, border, etc.)
::-Extending the set of elements that can be generated (span, div, header elements, table (and associated table contents), etc.)
::-Randomly choose whether to generate a document in quirks mode or not
:For task 3 there are several different parts, but the main goal is to increase the complexity of our randomly generated pages. First we will increase the tree depth to an arbitrary depth. We will then generate sibling elements and increase the CSS styling options. Finally, we will increase the amount of HTML elements that can be generated and randomly choose whether to generate a document in quirks mode or not.
:The code_generation.py file will be adapted to provide these functions. The function used to add random sections will have recursion added to generate elements within elements, resulting in a tree of html elements. The amounting of nesting will be random but limited. Within the sections, tables, lists, spans, and divs may potentially be created. This will involve writing new functions for creating each element, and allowing them to be chosen at random by the function that creates sections of the HTML document.

==Testing==
Because of the nature of this project, testing the newly added features will be very simple. Task 2 actually involves us comparing screenshots. By comparing the actual screenshots we will test that things work. Additionally, we will be able to visually test that task 3 is complete (seeing that the new page generation features work). While we could probably automate some of the testing for task 3 (to test all possible scenarios) it is out of the scope of this project.

==Conclusion==
The previously completed work allows us to generate simple html documents with a randomized structure, render the page in Servo, and take a screenshot of the page. We plan on furthering this work by rendering the pages in both Servo and Firefox, taking screenshots of the pages within both browsers, and reporting differences between the two. Doing so will allow users to evaluate Servo’s ability to load web pages. To make this testing even more informative, we plan to increase the complexity of the structure and styling of the randomly generated html documents.

CSC/ECE 517 Spring 2018- Project M1803: Implement a web page fuzzer to find rendering mismatches (Part 2)

2018-04-15T20:26:53Z

Adsimps3: /* Work to be done */

By Alexander Simpson(adsimps3), Abhay Soni (asoni3), Dileep badveli (dbadvel) and Jake Batty(jbatty)

==Introduction==
This Mozilla project was broken in to 2 main parts: the initial and subsequent steps. The initial steps were finished as a part of the OSS project. The goal of this final project is to complete the subsequent steps. As a part of the OSS project (explained more below) we created a tool which generates random valid HTML files and automated servo. Now, as a part of this project we are supposed to extend the program to also control Firefox, compare the resulting screenshots, and expand upon the page generation tool.

==Background==
TODO: explain servo

==Previous Work (Part of the OSS Project)==

As per the [https://github.com/servo/servo/wiki/Random-web-content-project project description], we were expected to complete the initial steps. The implementation is explained below for each of these steps.

:1) In a new repository, create a program that can generate a skeleton HTML file with a doctype, head element, and body element, print the result to stdout
::- Here is the [https://github.com/asoni3/Random-web-content-generator---CSC-517-OSS-Project link] to the repository which contains code_generation.py file which will be used to generate random valid HTML files.
:2) Add a module to the program that enable generating random content specific to the <head> element (such as inline CSS content inside of a <style> element) and add it to the generated output
::- The file [https://github.com/asoni3/Random-web-content-generator---CSC-517-OSS-Project/blob/master/code_generation.py code_generation.py], contains the code which generates random content specific to the head element and adds style on top of it. As seen in this code, after generating random content to the file, we will add CSS elements on top of this content. We have established a list of commonly used styles, weights, fonts, font_styles, and alignments which will be used at random. For practical purposes, we are limiting the number of options.
:3) Add a module to the program that enables generating random content specific to the <body> element (such as a <p> block that contains randomly generated text) and add it to the generated output
:4) Generate simple random CSS that affects randomly generated content (ie. if there is an element with an id foo, generate a CSS selector like #foo that applies a style like colorto it)
:5) Create a program under Servo's etc/ that launches Servo and causes it to [https://github.com/servo/servo/wiki/Control-Servo-using-WebDriver take a screenshot] of a particular URL - use this to take screenshots of pages randomly generated by the previous program
:Sample Screenshot:
:[[File:Servoscreenshot.png|border]]

==Work to be done==
Below is a list of the tasks to be done as a part of our final project. Below each task we have described what we think it will take to complete the respective task.

:'''1) Extend the program that controls Servo to also control Firefox using geckodriver'''
:Task 1 is relatively simple. It just involves downloading geckodriver and running it. Geckodriver is an open source software engine that allows us to render marked content on a web browser. It should allow us to take screenshots of a particular URL just like task 5 in the previous work section.

:'''2) Compare the resulting screenshots and report the contents of the generated page if the screenshots differ'''
:This task involves automating Firefox to use geckodriver and the current servo program. They both will create 2 different screenshots. If servo and Firefox render it differently, we will report that file and mark the differences.
:To actually make the comparisons we will use [https://opencv.org/ OpenCV]. OpenCV will allow us to not only make comparisons but will also mark where the differences in. To actually do the comparisons we will first convert the images to grayscale and then call compare_ssim(). This will get us a score and diff values. The score represents how close the to images are to each other and the diff tells us where the differences are. If the score indicates that the images are different we will use the diff value to findContours() and then draw a rectangle around the differences.

:'''3) Extend the page generation tool with a bunch of additional strategies, such as:'''
::-Generating elements trees of arbitrary depth
::-Generating sibling elements
::-Extending the set of CSS properties that can be generated (display, background, float, padding, margin, border, etc.)
::-Extending the set of elements that can be generated (span, div, header elements, table (and associated table contents), etc.)
::-Randomly choose whether to generate a document in quirks mode or not
:For task 3 there are several different parts, but the main goal is to increase the complexity of our randomly generated pages. First we will increase the tree depth to an arbitrary depth. We will then generate sibling elements and increase the CSS styling options. Finally, we will increase the amount of HTML elements that can be generated and randomly choose whether to generate a document in quirks mode or not.
:The code_generation.py file will be adapted to provide these functions. The function used to add random sections will have recursion added to generate elements within elements, resulting in a tree of html elements. The amounting of nesting will be random but limited. Within the sections, tables, lists, spans, and divs may potentially be created. This will involve writing new functions for creating each element, and allowing them to be chosen at random by the function that creates sections of the HTML document.

==Testing==
Because of the nature of this project, testing the newly added features will be very simple. Task 2 actually involves us comparing screenshots. By comparing the actual screenshots we will test that things work. Additionally, we will be able to visually test that task 3 is complete (seeing that the new page generation features work). While we could probably automate some of the testing for task 3 (to test all possible scenarios) it is out of the scope of this project.

==Conclusion==
The previously completed work allows us to generate simple html documents with a randomized structure, render the page in Servo, and take a screenshot of the page. We plan on furthering this work by rendering the pages in both Servo and Firefox, taking screenshots of the pages within both browsers, and reporting differences between the two. Doing so will allow users to evaluate Servo’s ability to load web pages. To make this testing even more informative, we plan to increase the complexity of the structure and styling of the randomly generated html documents.

CSC/ECE 517 Spring 2018- Project M1803: Implement a web page fuzzer to find rendering mismatches (Part 2)

2018-04-15T20:07:30Z

Adsimps3: /* Work to be done */

By Alexander Simpson(adsimps3), Abhay Soni (asoni3), Dileep badveli (dbadvel) and Jake Batty(jbatty)

==Introduction==
This Mozilla project was broken in to 2 main parts: the initial and subsequent steps. The initial steps were finished as a part of the OSS project. The goal of this final project is to complete the subsequent steps. As a part of the OSS project (explained more below) we created a tool which generates random valid HTML files and automated servo. Now, as a part of this project we are supposed to extend the program to also control Firefox, compare the resulting screenshots, and expand upon the page generation tool.

==Background==
TODO: explain servo

==Previous Work (Part of the OSS Project)==

As per the [https://github.com/servo/servo/wiki/Random-web-content-project project description], we were expected to complete the initial steps. The implementation is explained below for each of these steps.

:1) In a new repository, create a program that can generate a skeleton HTML file with a doctype, head element, and body element, print the result to stdout
::- Here is the [https://github.com/asoni3/Random-web-content-generator---CSC-517-OSS-Project link] to the repository which contains code_generation.py file which will be used to generate random valid HTML files.
:2) Add a module to the program that enable generating random content specific to the <head> element (such as inline CSS content inside of a <style> element) and add it to the generated output
::- The file [https://github.com/asoni3/Random-web-content-generator---CSC-517-OSS-Project/blob/master/code_generation.py code_generation.py], contains the code which generates random content specific to the head element and adds style on top of it. As seen in this code, after generating random content to the file, we will add CSS elements on top of this content. We have established a list of commonly used styles, weights, fonts, font_styles, and alignments which will be used at random. For practical purposes, we are limiting the number of options.
:3) Add a module to the program that enables generating random content specific to the <body> element (such as a <p> block that contains randomly generated text) and add it to the generated output
:4) Generate simple random CSS that affects randomly generated content (ie. if there is an element with an id foo, generate a CSS selector like #foo that applies a style like colorto it)
:5) Create a program under Servo's etc/ that launches Servo and causes it to [https://github.com/servo/servo/wiki/Control-Servo-using-WebDriver take a screenshot] of a particular URL - use this to take screenshots of pages randomly generated by the previous program
:Sample Screenshot:
:[[File:Servoscreenshot.png|border]]

==Work to be done==
Below is a list of the tasks to be done as a part of our final project. Below each task we have described what we think it will take to complete the respective task.

:'''1) Extend the program that controls Servo to also control Firefox using geckodriver'''
:Task 1 is relatively simple. It just involves downloading geckodriver and running it. Geckodriver is an open source software engine that allows us to render marked content on a web browser. It should allow us to take screenshots of a particular URL just like task 5 in the previous work section.

:'''2) Compare the resulting screenshots and report the contents of the generated page if the screenshots differ'''
:This task involves automating Firefox to use geckodriver and the current servo program. They both will create 2 different screenshots. If servo and Firefox render it differently, we will report that file and mark the differences.

:'''3) Extend the page generation tool with a bunch of additional strategies, such as:'''
::-Generating elements trees of arbitrary depth
::-Generating sibling elements
::-Extending the set of CSS properties that can be generated (display, background, float, padding, margin, border, etc.)
::-Extending the set of elements that can be generated (span, div, header elements, table (and associated table contents), etc.)
::-Randomly choose whether to generate a document in quirks mode or not
:For task 3 there are several different parts, but the main goal is to increase the complexity of our randomly generated pages. First we will increase the tree depth to an arbitrary depth. We will then generate sibling elements and increase the CSS styling options. Finally, we will increase the amount of HTML elements that can be generated and randomly choose whether to generate a document in quirks mode or not.
:To actually make the changes to the page generation tool we will add code to the [https://github.com/asoni3/Random-web-content-generator---CSC-517-OSS-Project/blob/master/code_generation.py code_generation.py] file.

==Testing==
Because of the nature of this project, testing the newly added features will be very simple. Task 2 actually involves us comparing screenshots. By comparing the actual screenshots we will test that things work. Additionally, we will be able to visually test that task 3 is complete (seeing that the new page generation features work). While we could probably automate some of the testing for task 3 (to test all possible scenarios) it is out of the scope of this project.

==Conclusion==
The previously completed work allows us to generate simple html documents with a randomized structure, render the page in Servo, and take a screenshot of the page. We plan on furthering this work by rendering the pages in both Servo and Firefox, taking screenshots of the pages within both browsers, and reporting differences between the two. Doing so will allow users to evaluate Servo’s ability to load web pages. To make this testing even more informative, we plan to increase the complexity of the structure and styling of the randomly generated html documents.

CSC/ECE 517 Spring 2018- Project M1803: Implement a web page fuzzer to find rendering mismatches (Part 2)

2018-04-15T20:03:25Z

Adsimps3: /* Work to be done */

By Alexander Simpson(adsimps3), Abhay Soni (asoni3), Dileep badveli (dbadvel) and Jake Batty(jbatty)

==Introduction==
This Mozilla project was broken in to 2 main parts: the initial and subsequent steps. The initial steps were finished as a part of the OSS project. The goal of this final project is to complete the subsequent steps. As a part of the OSS project (explained more below) we created a tool which generates random valid HTML files and automated servo. Now, as a part of this project we are supposed to extend the program to also control Firefox, compare the resulting screenshots, and expand upon the page generation tool.

==Background==
TODO: explain servo

==Previous Work (Part of the OSS Project)==

As per the [https://github.com/servo/servo/wiki/Random-web-content-project project description], we were expected to complete the initial steps. The implementation is explained below for each of these steps.

:1) In a new repository, create a program that can generate a skeleton HTML file with a doctype, head element, and body element, print the result to stdout
::- Here is the [https://github.com/asoni3/Random-web-content-generator---CSC-517-OSS-Project link] to the repository which contains code_generation.py file which will be used to generate random valid HTML files.
:2) Add a module to the program that enable generating random content specific to the <head> element (such as inline CSS content inside of a <style> element) and add it to the generated output
::- The file [https://github.com/asoni3/Random-web-content-generator---CSC-517-OSS-Project/blob/master/code_generation.py code_generation.py], contains the code which generates random content specific to the head element and adds style on top of it. As seen in this code, after generating random content to the file, we will add CSS elements on top of this content. We have established a list of commonly used styles, weights, fonts, font_styles, and alignments which will be used at random. For practical purposes, we are limiting the number of options.
:3) Add a module to the program that enables generating random content specific to the <body> element (such as a <p> block that contains randomly generated text) and add it to the generated output
:4) Generate simple random CSS that affects randomly generated content (ie. if there is an element with an id foo, generate a CSS selector like #foo that applies a style like colorto it)
:5) Create a program under Servo's etc/ that launches Servo and causes it to [https://github.com/servo/servo/wiki/Control-Servo-using-WebDriver take a screenshot] of a particular URL - use this to take screenshots of pages randomly generated by the previous program
:Sample Screenshot:
:[[File:Servoscreenshot.png|border]]

==Work to be done==
Below is a list of the tasks to be done as a part of our final project. Below each task we have described what we think it will take to complete the respective task.

:'''1) Extend the program that controls Servo to also control Firefox using geckodriver'''
:Task 1 is relatively simple. It just involves downloading geckodriver and running it. Geckodriver is an open source software engine that allows us to render marked content on a web browser. It should allow us to take screenshots of a particular URL just like task 5 in the previous work section.

:'''2) Compare the resulting screenshots and report the contents of the generated page if the screenshots differ'''
:This task involves automating Firefox to use geckodriver and the current servo program. They both will create 2 different screenshots. If servo and Firefox render it differently, we will report that file and mark the differences.

:'''3) Extend the page generation tool with a bunch of additional strategies, such as:'''
::-Generating elements trees of arbitrary depth
::-Generating sibling elements
::-Extending the set of CSS properties that can be generated (display, background, float, padding, margin, border, etc.)
::-Extending the set of elements that can be generated (span, div, header elements, table (and associated table contents), etc.)
::-Randomly choose whether to generate a document in quirks mode or not
:For task 3 there are several different parts, but the main goal is to increase the complexity of our randomly generated pages. First we will increase the tree depth to an arbitrary depth. We will then generate sibling elements and increase the CSS styling options. Finally, we will increase the amount of HTML elements that can be generated and randomly choose whether to generate a document in quirks mode or not.
:To actually make the changes to the page generation tool we will add code to the [https://github.com/asoni3/Random-web-content-generator---CSC-517-OSS-Project/blob/master/code_generation.py code_generation.py] file. We will add functions for most of the new randomly generated properties we are adding. For example,

==Testing==
Because of the nature of this project, testing the newly added features will be very simple. Task 2 actually involves us comparing screenshots. By comparing the actual screenshots we will test that things work. Additionally, we will be able to visually test that task 3 is complete (seeing that the new page generation features work). While we could probably automate some of the testing for task 3 (to test all possible scenarios) it is out of the scope of this project.

==Conclusion==
The previously completed work allows us to generate simple html documents with a randomized structure, render the page in Servo, and take a screenshot of the page. We plan on furthering this work by rendering the pages in both Servo and Firefox, taking screenshots of the pages within both browsers, and reporting differences between the two. Doing so will allow users to evaluate Servo’s ability to load web pages. To make this testing even more informative, we plan to increase the complexity of the structure and styling of the randomly generated html documents.

CSC/ECE 517 Spring 2018- Project M1803: Implement a web page fuzzer to find rendering mismatches (Part 2)

2018-04-15T19:57:56Z

Adsimps3: /* Work to be done */

By Alexander Simpson(adsimps3), Abhay Soni (asoni3), Dileep badveli (dbadvel) and Jake Batty(jbatty)

==Introduction==
This Mozilla project was broken in to 2 main parts: the initial and subsequent steps. The initial steps were finished as a part of the OSS project. The goal of this final project is to complete the subsequent steps. As a part of the OSS project (explained more below) we created a tool which generates random valid HTML files and automated servo. Now, as a part of this project we are supposed to extend the program to also control Firefox, compare the resulting screenshots, and expand upon the page generation tool.

==Background==
TODO: explain servo

==Previous Work (Part of the OSS Project)==

As per the [https://github.com/servo/servo/wiki/Random-web-content-project project description], we were expected to complete the initial steps. The implementation is explained below for each of these steps.

:1) In a new repository, create a program that can generate a skeleton HTML file with a doctype, head element, and body element, print the result to stdout
::- Here is the [https://github.com/asoni3/Random-web-content-generator---CSC-517-OSS-Project link] to the repository which contains code_generation.py file which will be used to generate random valid HTML files.
:2) Add a module to the program that enable generating random content specific to the <head> element (such as inline CSS content inside of a <style> element) and add it to the generated output
::- The file [https://github.com/asoni3/Random-web-content-generator---CSC-517-OSS-Project/blob/master/code_generation.py code_generation.py], contains the code which generates random content specific to the head element and adds style on top of it. As seen in this code, after generating random content to the file, we will add CSS elements on top of this content. We have established a list of commonly used styles, weights, fonts, font_styles, and alignments which will be used at random. For practical purposes, we are limiting the number of options.
:3) Add a module to the program that enables generating random content specific to the <body> element (such as a <p> block that contains randomly generated text) and add it to the generated output
:4) Generate simple random CSS that affects randomly generated content (ie. if there is an element with an id foo, generate a CSS selector like #foo that applies a style like colorto it)
:5) Create a program under Servo's etc/ that launches Servo and causes it to [https://github.com/servo/servo/wiki/Control-Servo-using-WebDriver take a screenshot] of a particular URL - use this to take screenshots of pages randomly generated by the previous program
:Sample Screenshot:
:[[File:Servoscreenshot.png|border]]

==Work to be done==
Below is a list of the tasks to be done as a part of our final project. Below each task we have described what we think it will take to complete the respective task.

:'''1) Extend the program that controls Servo to also control Firefox using geckodriver'''
:Task 1 is relatively simple. It just involves downloading geckodriver and running it. Geckodriver is an open source software engine that allows us to render marked content on a web browser. It should allow us to take screenshots of a particular URL just like task 5 in the previous work section.

:'''2) Compare the resulting screenshots and report the contents of the generated page if the screenshots differ'''
:This task involves automating Firefox to use geckodriver and the current servo program. They both will create 2 different screenshots. If servo and Firefox render it differently, we will report that file and mark the differences.

:'''3) Extend the page generation tool with a bunch of additional strategies, such as:'''
::-Generating elements trees of arbitrary depth
::-Generating sibling elements
::-Extending the set of CSS properties that can be generated (display, background, float, padding, margin, border, etc.)
::-Extending the set of elements that can be generated (span, div, header elements, table (and associated table contents), etc.)
::-Randomly choose whether to generate a document in quirks mode or not
:For task 3 there are several different parts, but the main goal is to increase the complexity of our randomly generated pages. First we will increase the tree depth to an arbitrary depth. We will then generate sibling elements and increase the CSS styling options. Finally, we will increase the amount of HTML elements that can be generated and randomly choose whether to generate a document in quirks mode or not.
:To actually make the changes to the page generation tool we will add code to the [https://github.com/asoni3/Random-web-content-generator---CSC-517-OSS-Project/blob/master/code_generation.py code_generation.py] file.

==Testing==
Because of the nature of this project, testing the newly added features will be very simple. Task 2 actually involves us comparing screenshots. By comparing the actual screenshots we will test that things work. Additionally, we will be able to visually test that task 3 is complete (seeing that the new page generation features work). While we could probably automate some of the testing for task 3 (to test all possible scenarios) it is out of the scope of this project.

==Conclusion==
The previously completed work allows us to generate simple html documents with a randomized structure, render the page in Servo, and take a screenshot of the page. We plan on furthering this work by rendering the pages in both Servo and Firefox, taking screenshots of the pages within both browsers, and reporting differences between the two. Doing so will allow users to evaluate Servo’s ability to load web pages. To make this testing even more informative, we plan to increase the complexity of the structure and styling of the randomly generated html documents.

CSC/ECE 517 Spring 2018- Project M1803: Implement a web page fuzzer to find rendering mismatches (Part 2)

2018-04-15T19:56:23Z

Adsimps3: /* Work to be done */

By Alexander Simpson(adsimps3), Abhay Soni (asoni3), Dileep badveli (dbadvel) and Jake Batty(jbatty)

==Introduction==
This Mozilla project was broken in to 2 main parts: the initial and subsequent steps. The initial steps were finished as a part of the OSS project. The goal of this final project is to complete the subsequent steps. As a part of the OSS project (explained more below) we created a tool which generates random valid HTML files and automated servo. Now, as a part of this project we are supposed to extend the program to also control Firefox, compare the resulting screenshots, and expand upon the page generation tool.

==Background==
TODO: explain servo

==Previous Work (Part of the OSS Project)==

As per the [https://github.com/servo/servo/wiki/Random-web-content-project project description], we were expected to complete the initial steps. The implementation is explained below for each of these steps.

:1) In a new repository, create a program that can generate a skeleton HTML file with a doctype, head element, and body element, print the result to stdout
::- Here is the [https://github.com/asoni3/Random-web-content-generator---CSC-517-OSS-Project link] to the repository which contains code_generation.py file which will be used to generate random valid HTML files.
:2) Add a module to the program that enable generating random content specific to the <head> element (such as inline CSS content inside of a <style> element) and add it to the generated output
::- The file [https://github.com/asoni3/Random-web-content-generator---CSC-517-OSS-Project/blob/master/code_generation.py code_generation.py], contains the code which generates random content specific to the head element and adds style on top of it. As seen in this code, after generating random content to the file, we will add CSS elements on top of this content. We have established a list of commonly used styles, weights, fonts, font_styles, and alignments which will be used at random. For practical purposes, we are limiting the number of options.
:3) Add a module to the program that enables generating random content specific to the <body> element (such as a <p> block that contains randomly generated text) and add it to the generated output
:4) Generate simple random CSS that affects randomly generated content (ie. if there is an element with an id foo, generate a CSS selector like #foo that applies a style like colorto it)
:5) Create a program under Servo's etc/ that launches Servo and causes it to [https://github.com/servo/servo/wiki/Control-Servo-using-WebDriver take a screenshot] of a particular URL - use this to take screenshots of pages randomly generated by the previous program
:Sample Screenshot:
:[[File:Servoscreenshot.png|border]]

==Work to be done==
Below is a list of the tasks to be done as a part of our final project. Below each task we have described what we think it will take to complete the respective task.

:'''1) Extend the program that controls Servo to also control Firefox using geckodriver'''
:Task 1 is relatively simple. It just involves downloading geckodriver and running it. Geckodriver is an open source software engine that allows us to render marked content on a web browser. It should allow us to take screenshots of a particular URL just like task 5 in the previous work section.

:'''2) Compare the resulting screenshots and report the contents of the generated page if the screenshots differ'''
:This task involves automating Firefox to use geckodriver and the current servo program. They both will create 2 different screenshots. If servo and Firefox render it differently, we will report that file and mark the differences.

:'''3) Extend the page generation tool with a bunch of additional strategies, such as:'''
::-Generating elements trees of arbitrary depth
::-Generating sibling elements
::-Extending the set of CSS properties that can be generated (display, background, float, padding, margin, border, etc.)
::-Extending the set of elements that can be generated (span, div, header elements, table (and associated table contents), etc.)
::-Randomly choose whether to generate a document in quirks mode or not
:For task 3 there are several different parts, but the main goal is to increase the complexity of our randomly generated pages. First we will increase the tree depth to an arbitrary depth. We will then generate sibling elements and increase the CSS styling options. Finally, we will increase the amount of HTML elements that can be generated and randomly choose whether to generate a document in quirks mode or not.

To actually make the changes to the page generation tool we will add code to the [https://github.com/asoni3/Random-web-content-generator---CSC-517-OSS-Project/blob/master/code_generation.py]

==Testing==
Because of the nature of this project, testing the newly added features will be very simple. Task 2 actually involves us comparing screenshots. By comparing the actual screenshots we will test that things work. Additionally, we will be able to visually test that task 3 is complete (seeing that the new page generation features work). While we could probably automate some of the testing for task 3 (to test all possible scenarios) it is out of the scope of this project.

==Conclusion==
The previously completed work allows us to generate simple html documents with a randomized structure, render the page in Servo, and take a screenshot of the page. We plan on furthering this work by rendering the pages in both Servo and Firefox, taking screenshots of the pages within both browsers, and reporting differences between the two. Doing so will allow users to evaluate Servo’s ability to load web pages. To make this testing even more informative, we plan to increase the complexity of the structure and styling of the randomly generated html documents.

CSC/ECE 517 Spring 2018- Project M1803: Implement a web page fuzzer to find rendering mismatches (Part 2)

2018-04-15T19:53:16Z

Adsimps3:

By Alexander Simpson(adsimps3), Abhay Soni (asoni3), Dileep badveli (dbadvel) and Jake Batty(jbatty)

==Introduction==
This Mozilla project was broken in to 2 main parts: the initial and subsequent steps. The initial steps were finished as a part of the OSS project. The goal of this final project is to complete the subsequent steps. As a part of the OSS project (explained more below) we created a tool which generates random valid HTML files and automated servo. Now, as a part of this project we are supposed to extend the program to also control Firefox, compare the resulting screenshots, and expand upon the page generation tool.

==Background==
TODO: explain servo

==Previous Work (Part of the OSS Project)==

As per the [https://github.com/servo/servo/wiki/Random-web-content-project project description], we were expected to complete the initial steps. The implementation is explained below for each of these steps.

:1) In a new repository, create a program that can generate a skeleton HTML file with a doctype, head element, and body element, print the result to stdout
::- Here is the [https://github.com/asoni3/Random-web-content-generator---CSC-517-OSS-Project link] to the repository which contains code_generation.py file which will be used to generate random valid HTML files.
:2) Add a module to the program that enable generating random content specific to the <head> element (such as inline CSS content inside of a <style> element) and add it to the generated output
::- The file [https://github.com/asoni3/Random-web-content-generator---CSC-517-OSS-Project/blob/master/code_generation.py code_generation.py], contains the code which generates random content specific to the head element and adds style on top of it. As seen in this code, after generating random content to the file, we will add CSS elements on top of this content. We have established a list of commonly used styles, weights, fonts, font_styles, and alignments which will be used at random. For practical purposes, we are limiting the number of options.
:3) Add a module to the program that enables generating random content specific to the <body> element (such as a <p> block that contains randomly generated text) and add it to the generated output
:4) Generate simple random CSS that affects randomly generated content (ie. if there is an element with an id foo, generate a CSS selector like #foo that applies a style like colorto it)
:5) Create a program under Servo's etc/ that launches Servo and causes it to [https://github.com/servo/servo/wiki/Control-Servo-using-WebDriver take a screenshot] of a particular URL - use this to take screenshots of pages randomly generated by the previous program
:Sample Screenshot:
:[[File:Servoscreenshot.png|border]]

==Work to be done==
Below is a list of the tasks to be done as a part of our final project. Below each task we have described what we think it will take to complete the respective task.

:'''1) Extend the program that controls Servo to also control Firefox using geckodriver'''
:Task 1 is relatively simple. It just involves downloading geckodriver and running it. Geckodriver is an open source software engine that allows us to render marked content on a web browser. It should allow us to take screenshots of a particular URL just like task 5 in the previous work section.

:'''2) Compare the resulting screenshots and report the contents of the generated page if the screenshots differ'''
:This task involves automating Firefox to use geckodriver and the current servo program. They both will create 2 different screenshots. If servo and Firefox render it differently, we will report that file and mark the differences.

:'''3) Extend the page generation tool with a bunch of additional strategies, such as:'''
::-Generating elements trees of arbitrary depth
::-Generating sibling elements
::-Extending the set of CSS properties that can be generated (display, background, float, padding, margin, border, etc.)
::-Extending the set of elements that can be generated (span, div, header elements, table (and associated table contents), etc.)
::-Randomly choose whether to generate a document in quirks mode or not
:For task 3 there are several different parts, but the main goal is to increase the complexity of our randomly generated pages. First we will increase the tree depth to an arbitrary depth. We will then generate sibling elements and increase the CSS styling options. Finally, we will increase the amount of HTML elements that can be generated and randomly choose whether to generate a document in quirks mode or not.

==Testing==
Because of the nature of this project, testing the newly added features will be very simple. Task 2 actually involves us comparing screenshots. By comparing the actual screenshots we will test that things work. Additionally, we will be able to visually test that task 3 is complete (seeing that the new page generation features work). While we could probably automate some of the testing for task 3 (to test all possible scenarios) it is out of the scope of this project.

==Conclusion==
The previously completed work allows us to generate simple html documents with a randomized structure, render the page in Servo, and take a screenshot of the page. We plan on furthering this work by rendering the pages in both Servo and Firefox, taking screenshots of the pages within both browsers, and reporting differences between the two. Doing so will allow users to evaluate Servo’s ability to load web pages. To make this testing even more informative, we plan to increase the complexity of the structure and styling of the randomly generated html documents.

CSC/ECE 517 Spring 2018- Project M1803: Implement a web page fuzzer to find rendering mismatches (Part 2)

2018-04-15T19:46:51Z

Adsimps3: /* Introduction */

By Alexander Simpson(adsimps3), Abhay Soni (asoni3), Dileep badveli (dbadvel) and Jake Batty(jbatty)

==Introduction==
This Mozilla project was broken in to 2 main parts: the initial and subsequent steps. The initial steps were finished as a part of the OSS project. The goal of this final project is to complete the subsequent steps. As a part of the OSS project (explained more below) we created a tool which generates random valid HTML files and automated servo. Now, as a part of this project we are supposed to extend the program to also control Firefox, compare the resulting screenshots, and expand upon the page generation tool.

==Background==
TODO: explain servo

==Previous Work (Part of the OSS Project)==

As per the [https://github.com/servo/servo/wiki/Random-web-content-project project description], we were expected to complete the initial steps. The implementation is explained below for each of these steps.

:1) In a new repository, create a program that can generate a skeleton HTML file with a doctype, head element, and body element, print the result to stdout
::- Here is the [https://github.com/asoni3/Random-web-content-generator---CSC-517-OSS-Project link] to the repository which contains code_generation.py file which will be used to generate random valid HTML files.
:2) Add a module to the program that enable generating random content specific to the <head> element (such as inline CSS content inside of a <style> element) and add it to the generated output
::- The file [https://github.com/asoni3/Random-web-content-generator---CSC-517-OSS-Project/blob/master/code_generation.py code_generation.py], contains the code which generates random content specific to the head element and adds style on top of it. As seen in this code, after generating random content to the file, we will add CSS elements on top of this content. We have established a list of commonly used styles, weights, fonts, font_styles, and alignments which will be used at random. For practical purposes, we are limiting the number of options.
:3) Add a module to the program that enables generating random content specific to the <body> element (such as a <p> block that contains randomly generated text) and add it to the generated output
:4) Generate simple random CSS that affects randomly generated content (ie. if there is an element with an id foo, generate a CSS selector like #foo that applies a style like colorto it)
:5) Create a program under Servo's etc/ that launches Servo and causes it to [https://github.com/servo/servo/wiki/Control-Servo-using-WebDriver take a screenshot] of a particular URL - use this to take screenshots of pages randomly generated by the previous program
:Sample Screenshot:
:[[File:Servoscreenshot.png|border]]

==Work to be done==
Below is a list of the tasks to be done as a part of our final project. Below each task we have described what we think it will take to complete the respective task.

:'''1) Extend the program that controls Servo to also control Firefox using geckodriver'''
:Task 1 is relatively simple. It just involves downloading geckodriver and running it. Geckodriver is an open source software engine that allows us to render marked content on a web browser. It should allow us to take screenshots of a particular URL just like task 5 in the previous work section.

:'''2) Compare the resulting screenshots and report the contents of the generated page if the screenshots differ'''
:This task involves automating Firefox to use geckodriver and the current servo program. They both will create 2 different screenshots. If servo and Firefox render it differently, we will report that file and mark the differences.

:'''3) Extend the page generation tool with a bunch of additional strategies, such as:'''
::-Generating elements trees of arbitrary depth
::-Generating sibling elements
::-Extending the set of CSS properties that can be generated (display, background, float, padding, margin, border, etc.)
::-Extending the set of elements that can be generated (span, div, header elements, table (and associated table contents), etc.)
::-Randomly choose whether to generate a document in quirks mode or not
:For task 3 there are several different parts, but the main goal is to increase the complexity of our randomly generated pages. First we will increase the tree depth to an arbitrary depth. We will then generate sibling elements and increase the CSS styling options. Finally, we will increase the amount of HTML elements that can be generated and randomly choose whether to generate a document in quirks mode or not.

==Conclusion==
The previously completed work allows us to generate simple html documents with a randomized structure, render the page in Servo, and take a screenshot of the page. We plan on furthering this work by rendering the pages in both Servo and Firefox, taking screenshots of the pages within both browsers, and reporting differences between the two. Doing so will allow users to evaluate Servo’s ability to load web pages. To make this testing even more informative, we plan to increase the complexity of the structure and styling of the randomly generated html documents.

CSC/ECE 517 Spring 2018- Project M1803: Implement a web page fuzzer to find rendering mismatches (Part 2)

2018-04-15T19:46:22Z

Adsimps3:

By Alexander Simpson(adsimps3), Abhay Soni (asoni3), Dileep badveli (dbadvel) and Jake Batty(jbatty)

==Introduction==
This Mozilla project was broken in to 2 main parts: the initial and subsequent steps. The initial steps were finished as a part of the OSS project. So the goal of this final project is to complete the subsequent steps. As a part of the OSS project (explained more below) we created a tool which generates random valid HTML files and automated servo. Now, as a part of this project we are supposed to extend the program to also control Firefox, compare the resulting screenshots, and expand upon the page generation tool.

==Background==
TODO: explain servo

==Previous Work (Part of the OSS Project)==

As per the [https://github.com/servo/servo/wiki/Random-web-content-project project description], we were expected to complete the initial steps. The implementation is explained below for each of these steps.

:1) In a new repository, create a program that can generate a skeleton HTML file with a doctype, head element, and body element, print the result to stdout
::- Here is the [https://github.com/asoni3/Random-web-content-generator---CSC-517-OSS-Project link] to the repository which contains code_generation.py file which will be used to generate random valid HTML files.
:2) Add a module to the program that enable generating random content specific to the <head> element (such as inline CSS content inside of a <style> element) and add it to the generated output
::- The file [https://github.com/asoni3/Random-web-content-generator---CSC-517-OSS-Project/blob/master/code_generation.py code_generation.py], contains the code which generates random content specific to the head element and adds style on top of it. As seen in this code, after generating random content to the file, we will add CSS elements on top of this content. We have established a list of commonly used styles, weights, fonts, font_styles, and alignments which will be used at random. For practical purposes, we are limiting the number of options.
:3) Add a module to the program that enables generating random content specific to the <body> element (such as a <p> block that contains randomly generated text) and add it to the generated output
:4) Generate simple random CSS that affects randomly generated content (ie. if there is an element with an id foo, generate a CSS selector like #foo that applies a style like colorto it)
:5) Create a program under Servo's etc/ that launches Servo and causes it to [https://github.com/servo/servo/wiki/Control-Servo-using-WebDriver take a screenshot] of a particular URL - use this to take screenshots of pages randomly generated by the previous program
:Sample Screenshot:
:[[File:Servoscreenshot.png|border]]

==Work to be done==
Below is a list of the tasks to be done as a part of our final project. Below each task we have described what we think it will take to complete the respective task.

:'''1) Extend the program that controls Servo to also control Firefox using geckodriver'''
:Task 1 is relatively simple. It just involves downloading geckodriver and running it. Geckodriver is an open source software engine that allows us to render marked content on a web browser. It should allow us to take screenshots of a particular URL just like task 5 in the previous work section.

:'''2) Compare the resulting screenshots and report the contents of the generated page if the screenshots differ'''
:This task involves automating Firefox to use geckodriver and the current servo program. They both will create 2 different screenshots. If servo and Firefox render it differently, we will report that file and mark the differences.

:'''3) Extend the page generation tool with a bunch of additional strategies, such as:'''
::-Generating elements trees of arbitrary depth
::-Generating sibling elements
::-Extending the set of CSS properties that can be generated (display, background, float, padding, margin, border, etc.)
::-Extending the set of elements that can be generated (span, div, header elements, table (and associated table contents), etc.)
::-Randomly choose whether to generate a document in quirks mode or not
:For task 3 there are several different parts, but the main goal is to increase the complexity of our randomly generated pages. First we will increase the tree depth to an arbitrary depth. We will then generate sibling elements and increase the CSS styling options. Finally, we will increase the amount of HTML elements that can be generated and randomly choose whether to generate a document in quirks mode or not.

==Conclusion==
The previously completed work allows us to generate simple html documents with a randomized structure, render the page in Servo, and take a screenshot of the page. We plan on furthering this work by rendering the pages in both Servo and Firefox, taking screenshots of the pages within both browsers, and reporting differences between the two. Doing so will allow users to evaluate Servo’s ability to load web pages. To make this testing even more informative, we plan to increase the complexity of the structure and styling of the randomly generated html documents.

CSC/ECE 517 Spring 2018- Project M1803: Implement a web page fuzzer to find rendering mismatches (Part 2)

2018-04-15T19:45:50Z

Adsimps3: /* Introduction */

By Alexander Simpson(adsimps3), Abhay Soni (asoni3), Dileep badveli (dbadvel) and Jake Batty(jbatty)

==Introduction==
This Mozilla project was broken in to 2 main parts: the initial and subsequent steps. The initial steps were finished as a part of the OSS project. So the goal of this final project is to complete the subsequent steps. As a part of the OSS project (explained more below) we created a tool which generates random valid HTML files and automated servo. Now, as a part of this project we are supposed to extend the program to also control Firefox, compare the resulting screenshots, and expand upon the page generation tool.

==Previous Work (Part of the OSS Project)==

As per the [https://github.com/servo/servo/wiki/Random-web-content-project project description], we were expected to complete the initial steps. The implementation is explained below for each of these steps.

:1) In a new repository, create a program that can generate a skeleton HTML file with a doctype, head element, and body element, print the result to stdout
::- Here is the [https://github.com/asoni3/Random-web-content-generator---CSC-517-OSS-Project link] to the repository which contains code_generation.py file which will be used to generate random valid HTML files.
:2) Add a module to the program that enable generating random content specific to the <head> element (such as inline CSS content inside of a <style> element) and add it to the generated output
::- The file [https://github.com/asoni3/Random-web-content-generator---CSC-517-OSS-Project/blob/master/code_generation.py code_generation.py], contains the code which generates random content specific to the head element and adds style on top of it. As seen in this code, after generating random content to the file, we will add CSS elements on top of this content. We have established a list of commonly used styles, weights, fonts, font_styles, and alignments which will be used at random. For practical purposes, we are limiting the number of options.
:3) Add a module to the program that enables generating random content specific to the <body> element (such as a <p> block that contains randomly generated text) and add it to the generated output
:4) Generate simple random CSS that affects randomly generated content (ie. if there is an element with an id foo, generate a CSS selector like #foo that applies a style like colorto it)
:5) Create a program under Servo's etc/ that launches Servo and causes it to [https://github.com/servo/servo/wiki/Control-Servo-using-WebDriver take a screenshot] of a particular URL - use this to take screenshots of pages randomly generated by the previous program
:Sample Screenshot:
:[[File:Servoscreenshot.png|border]]

==Work to be done==
Below is a list of the tasks to be done as a part of our final project. Below each task we have described what we think it will take to complete the respective task.

:'''1) Extend the program that controls Servo to also control Firefox using geckodriver'''
:Task 1 is relatively simple. It just involves downloading geckodriver and running it. Geckodriver is an open source software engine that allows us to render marked content on a web browser. It should allow us to take screenshots of a particular URL just like task 5 in the previous work section.

:'''2) Compare the resulting screenshots and report the contents of the generated page if the screenshots differ'''
:This task involves automating Firefox to use geckodriver and the current servo program. They both will create 2 different screenshots. If servo and Firefox render it differently, we will report that file and mark the differences.

:'''3) Extend the page generation tool with a bunch of additional strategies, such as:'''
::-Generating elements trees of arbitrary depth
::-Generating sibling elements
::-Extending the set of CSS properties that can be generated (display, background, float, padding, margin, border, etc.)
::-Extending the set of elements that can be generated (span, div, header elements, table (and associated table contents), etc.)
::-Randomly choose whether to generate a document in quirks mode or not
:For task 3 there are several different parts, but the main goal is to increase the complexity of our randomly generated pages. First we will increase the tree depth to an arbitrary depth. We will then generate sibling elements and increase the CSS styling options. Finally, we will increase the amount of HTML elements that can be generated and randomly choose whether to generate a document in quirks mode or not.

==Conclusion==
The previously completed work allows us to generate simple html documents with a randomized structure, render the page in Servo, and take a screenshot of the page. We plan on furthering this work by rendering the pages in both Servo and Firefox, taking screenshots of the pages within both browsers, and reporting differences between the two. Doing so will allow users to evaluate Servo’s ability to load web pages. To make this testing even more informative, we plan to increase the complexity of the structure and styling of the randomly generated html documents.

CSC/ECE 517 Spring 2018- Project E1812: on the fly calc.rb

2018-04-03T00:44:31Z

Adsimps3: /* NOTE to reviewers */

==E1812: Unit tests for on_the_fly_calc.rb==

This wiki page gives a description of changes made under Expertiza OSS project E1812 assignment for Spring 2018, CSC/ECE 517.

===NOTE to reviewers===
Because this project is just a unit test it cannot be tested via UI. Per Dr. Gehringer's instructions we have posted a video to prove this works. In the video I run the unit tests from the terminal and then check the coverage. Our goal of our project was to have at least 90% coverage which we did. A link to the video is below.

[https://www.youtube.com/watch?v=hmTCzboAyPE Video]

__TOC__

==Expertiza Background==

[http://expertiza.ncsu.edu/ Expertiza] is an open source web application developed with [http://rubyonrails.org/ Ruby on Rails] framework. This application allows students to access assignments posted by an instructor, submit their work (articles, codes, websites), review others work submissions. Expertiza gives a chance to improve students’ work based on their peer-reviews, and it can also help in grading students’ submissions based on that feedback. Students can form their teams for an assignment, otherwise, teams can be assigned automatically.

----

== Problem Statement ==
The OnTheFlyCalc model does not have any test cases corresponding to it. Thus, the task of this assignment is to write unit tests for on_the_fly_calc.rb using RSpec. The test cases should be fast, effective, and achieve the maximum code coverage, i.e. > 90%.

----
=== OnTheFlyCalc ===

On_the_fly_calc is a module that is included in assignment.rb. This module calculates the score for both students and instructors. E.g., when a student clicks “view my score”, Expertiza calculates the review scores for each review. This is called “on the fly” because expertiza stores the review scores based on each question (in answers table), but does not store the total score that a reviewer gives to a reviewee. It directly calculates the total review score for each review, using the score values in the answer table during run-time.

=== Created/Modified Files ===
As a part of the project the file listed below was created and saved under spec/models folder:
*on_the_fly_calc_spec.rb

The file listed below was modified due to some code issues in it
*on_the_fly_calc.rb (path: app/models)
1. scores(question) function returned scores which was being considered as recursive calls and thus we changed the return variable to score which is being modified in the functions.
2. score_team[:scores] = Answer.compute_scores(assessments, questions[:review]) line in the scores was changed to score_team[:scores] = Answer.compute_scores(assessments, questions)
this was done because the compute scores function in Answers was throwing an error while trying to deference the index of questions using a symbol.
3. grades_by_rounds[round_sym] = Answer.compute_scores(assessments, questions[round_sym]) to rades_by_rounds[round_sym] = Answer.compute_scores(assessments, questions)
same reason as above.

=== Thoughts ===
What we need to do is to write tests for four public methods listed in the module OnTheFLyCalc. However, the on_the_fly_calc itself has some code issues in #scores, and IDE reports some errors when we run the test case for it. Thus, we need to modify the module itself a little.

----

== Tools ==

===RSpec ===
For this project we use [https://en.wikipedia.org/wiki/RSpec RSpec] which is a testing tool for Ruby, created for [https://en.wikipedia.org/wiki/Behavior-driven_development behavior-driven development] (BDD).

----

== Plan of Work ==
In order to accomplish this task, that is to build tests for the OnTheFlyCalc model, we need to implement the following plan:

1. Set up the Expertiza environment;

2. Understand the functionality of the model;

3. Understand the relative methods and objects involved with each function of the model;

4. Create test objects;

5. Write ‘it block’ test scenarios with appropriate contexts for all public functions of the model, and check if all [https://relishapp.com/rspec/rspec-expectations/docs expectations ] pass.

----

'''Expertiza Environment'''

We installed VM VirtualBox, and imported Ubuntu-Expertiza image into VirtualBox. In Git, Expertiza master brunch was forked, in the VM terminal we cloned it and then run all required commands to finish setting up the environment.

----

== Testing ==

Listed below are the functions in on_the_fly_calc.rb and the RSpec unit tests corresponding to the function names along with a list of scenarios tested.

===Function: compute_total_score===
The function compute total score totals the scores of each questionnaire in an assignment.

<pre>
def compute_total_score(scores)
total = 0
self.questionnaires.each {|questionnaire| total += questionnaire.get_weighted_score(self, scores) }
total
end
</pre>

''To test this function''

we mocked the calls made in the Questionnaire class.
for example to mock the call

<pre>
round = AssignmentQuestionnaire.find_by(assignment_id: assignment.id, questionnaire_id: self.id).used_in_round
</pre>

we used the rspec line which looks like this

<pre>
allow(AssignmentQuestionnaire).to receive(:find_by).with(assignment_id: 1, questionnaire_id: nil).and_return(double('AssignmentQuestionnaire', used_in_round: 1))
</pre>

similarly, other calls made to

<pre>
self.assignment_questionnaires.find_by(assignment_id: assignment.id)
</pre>

was mocked using calls that look like :-

<pre>
allow(ReviewQuestionnaire).to receive_message_chain(:assignment_questionnaires,:find_by).with(no_args).with(assignment_id: 1).and_return(double('AssignmentQuestionnaire', id: 1))
</pre>

'''cases for consideration :'''

1. when avg score equals nil

2. when avg score is not equal to nil

----

===Function: compute_reviews_hash===
Below is most of the code used in the test case. It is relatively simple. The first test case is for when assignments vary by rubrics. We check that the function returns {}. This is correct because we set reviewer to nil so that reviewer will be set to {} in the "reviewer = {} if reviewer.nil?" statement. In the second test case we check that it returns {1=>{1=>50}, 2=>{1=>30}}. This is correct because we create 2 response maps and set their value to 50 and 30.

context 'when current assignment varys rubrics by round' do
it 'scores varying rubrics and returns review scores' do
allow(assignment).to receive(:varying_rubrics_by_round?).and_return(TRUE)
allow(assignment).to receive(:rounds_of_reviews).and_return(1)
temp = assignment.compute_reviews_hash()
expect(temp).to eql({})
end
end

context 'when current assignment does not vary rubrics by round' do
it 'scores varying rubrics and returns review scores' do
temp = assignment.compute_reviews_hash()
expect(temp).to eql({1=>{1=>50}, 2=>{1=>30}})
end
end

===Function: compute_avg_and_ranges_hash===
This function calculates the average score and score range for each reviewee(team).

<pre>
def compute_avg_and_ranges_hash
scores = {}
contributors = self.contributors # assignment_teams
if self.varying_rubrics_by_round?
rounds = self.rounds_of_reviews
(1..rounds).each do |round|
review_questionnaire_id = review_questionnaire_id(round)
questions = Question.where('questionnaire_id = ?', review_questionnaire_id)
contributors.each do |contributor|
assessments = ReviewResponseMap.get_assessments_for(contributor)
assessments = assessments.select {|assessment| assessment.round == round }
scores[contributor.id] = {} if round == 1
scores[contributor.id][round] = {}
scores[contributor.id][round] = Answer.compute_scores(assessments, questions)
end
end
else
review_questionnaire_id = review_questionnaire_id()
questions = Question.where('questionnaire_id = ?', review_questionnaire_id)
contributors.each do |contributor|
assessments = ReviewResponseMap.get_assessments_for(contributor)
scores[contributor.id] = {}
scores[contributor.id] = Answer.compute_scores(assessments, questions)
end
end
scores
end
</pre>

===Function: scores===
<pre>
def scores(questions)
index = 0
score_assignment
self.teams.each do |team|
score_team = {}
score_team[:team] = team
if self.varying_rubrics_by_round?
calculate_rounds
calculate_score
calculate_assessment
else
assessments = ReviewResponseMap.get_assessments_for(team)
score_team[:scores] = Answer.compute_scores(assessments, questions)
end
index += 1
end
score
end
end
</pre>

This method calls a lot of private methods to calculate a score hash. To test this method we had to mock calls made in these private methods.

When ''varying_rubrics_by_round?'' is true this method calls
1. calculate_rounds

2. calculate_score

3. calculate_assessment

we mocked the call to 1. so it is not covered in our coverage as it just initializes our hashes.

to mock the call to 2. and 3. calls such as the ones given below can be seen in our test cases.

<pre>
allow(on_the_fly_calc).to receive(:index).and_return(0)
allow(on_the_fly_calc).to receive(:num_review_rounds).and_return([1,2])
allow(on_the_fly_calc).to receive(:team).and_return(double('AssignmentTeam'))
allow(ReviewResponseMap).to receive(:get_responses_for_team_round).with(any_args).and_return([])
allow(on_the_fly_calc).to receive(:questions).and_return(questions)
allow(Answer).to receive(:compute_scores).with([],[question1]).and_return({})
allow(on_the_fly_calc).to receive(:round_sym).and_return(:review1)
allow(on_the_fly_calc).to receive(:grades_by_rounds).and_return(0)
allow(on_the_fly_calc).to receive(:score_assignment).and_return('')
</pre>

we are basically mocking calls to run through all lines of the methods and only calls that is returning something or are assigning value to a variable need to be mocked.

for example the call
<pre>
allow(Answer).to receive(:compute_scores).with([],[question1]).and_return({})
</pre>
is to mock the call made to ''compute_score'' function in the ''Answer'' class we made it to return an empty hash.

'''cases for consideration :'''

1. when current assignment varys rubrics by round and number of assessments is 0.in this case
<pre>
allow(on_the_fly_calc).to receive(:total_num_of_assessments).and_return(0)
</pre>
this call is made to mask the value of total_num_of_assessments value, similarly for others.

2. when current assignment varys rubrics by round and number of assessments is non-zero.

3. when current assignment does not vary rubrics by round

----

===Running RSpec===
*To run RSpec for a particular file (on_the_fly_calc_spec.rb):
<pre>
$ rspec spec/models/on_the_fly_calc_spec.rb
</pre>

----

===Coverage Results===

We have a 92.74% coverage.
Below is a picture of the results from the coverage.

[[File:on_the_fly_test_results.png]]

==References==
1. [https://en.wikipedia.org/wiki/RSpec RSpec Wiki] <br>
2. [http://rspec.info/ RSpec.info] <br>
3. [https://www.youtube.com/watch?v=dzkVfaKChSU&feature=youtu.be&t=35s youtube.com TDD: RSpec Test Structure] <br>
4. [https://github.com/expertiza/expertiza/blob/3ce553a2d0258ea05bced910abae5d209a7f55d6/spec/models/response_spec.rb github/expertiza response_spec example] <br>
5. [https://relishapp.com/rspec RSpec Documentation] <br>

CSC/ECE 517 Spring 2018- Project E1812: on the fly calc.rb

2018-04-03T00:43:46Z

Adsimps3: /* NOTE to reviewers */

==E1812: Unit tests for on_the_fly_calc.rb==

This wiki page gives a description of changes made under Expertiza OSS project E1812 assignment for Spring 2018, CSC/ECE 517.

===NOTE to reviewers===
Because this project is just a unit test it cannot be tested via UI. Per Dr. Gehringer's instructions we have posted a video to prove this works. In the video I run the unit tests from the terminal and then check the coverage. Our goal of our project was to have at least 90% coverage which we did. A link to the video is below.

[https://www.youtube.com/watch?v=hmTCzboAyPE]

__TOC__

==Expertiza Background==

[http://expertiza.ncsu.edu/ Expertiza] is an open source web application developed with [http://rubyonrails.org/ Ruby on Rails] framework. This application allows students to access assignments posted by an instructor, submit their work (articles, codes, websites), review others work submissions. Expertiza gives a chance to improve students’ work based on their peer-reviews, and it can also help in grading students’ submissions based on that feedback. Students can form their teams for an assignment, otherwise, teams can be assigned automatically.

----

== Problem Statement ==
The OnTheFlyCalc model does not have any test cases corresponding to it. Thus, the task of this assignment is to write unit tests for on_the_fly_calc.rb using RSpec. The test cases should be fast, effective, and achieve the maximum code coverage, i.e. > 90%.

----
=== OnTheFlyCalc ===

On_the_fly_calc is a module that is included in assignment.rb. This module calculates the score for both students and instructors. E.g., when a student clicks “view my score”, Expertiza calculates the review scores for each review. This is called “on the fly” because expertiza stores the review scores based on each question (in answers table), but does not store the total score that a reviewer gives to a reviewee. It directly calculates the total review score for each review, using the score values in the answer table during run-time.

=== Created/Modified Files ===
As a part of the project the file listed below was created and saved under spec/models folder:
*on_the_fly_calc_spec.rb

The file listed below was modified due to some code issues in it
*on_the_fly_calc.rb (path: app/models)
1. scores(question) function returned scores which was being considered as recursive calls and thus we changed the return variable to score which is being modified in the functions.
2. score_team[:scores] = Answer.compute_scores(assessments, questions[:review]) line in the scores was changed to score_team[:scores] = Answer.compute_scores(assessments, questions)
this was done because the compute scores function in Answers was throwing an error while trying to deference the index of questions using a symbol.
3. grades_by_rounds[round_sym] = Answer.compute_scores(assessments, questions[round_sym]) to rades_by_rounds[round_sym] = Answer.compute_scores(assessments, questions)
same reason as above.

=== Thoughts ===
What we need to do is to write tests for four public methods listed in the module OnTheFLyCalc. However, the on_the_fly_calc itself has some code issues in #scores, and IDE reports some errors when we run the test case for it. Thus, we need to modify the module itself a little.

----

== Tools ==

===RSpec ===
For this project we use [https://en.wikipedia.org/wiki/RSpec RSpec] which is a testing tool for Ruby, created for [https://en.wikipedia.org/wiki/Behavior-driven_development behavior-driven development] (BDD).

----

== Plan of Work ==
In order to accomplish this task, that is to build tests for the OnTheFlyCalc model, we need to implement the following plan:

1. Set up the Expertiza environment;

2. Understand the functionality of the model;

3. Understand the relative methods and objects involved with each function of the model;

4. Create test objects;

5. Write ‘it block’ test scenarios with appropriate contexts for all public functions of the model, and check if all [https://relishapp.com/rspec/rspec-expectations/docs expectations ] pass.

----

'''Expertiza Environment'''

We installed VM VirtualBox, and imported Ubuntu-Expertiza image into VirtualBox. In Git, Expertiza master brunch was forked, in the VM terminal we cloned it and then run all required commands to finish setting up the environment.

----

== Testing ==

Listed below are the functions in on_the_fly_calc.rb and the RSpec unit tests corresponding to the function names along with a list of scenarios tested.

===Function: compute_total_score===
The function compute total score totals the scores of each questionnaire in an assignment.

<pre>
def compute_total_score(scores)
total = 0
self.questionnaires.each {|questionnaire| total += questionnaire.get_weighted_score(self, scores) }
total
end
</pre>

''To test this function''

we mocked the calls made in the Questionnaire class.
for example to mock the call

<pre>
round = AssignmentQuestionnaire.find_by(assignment_id: assignment.id, questionnaire_id: self.id).used_in_round
</pre>

we used the rspec line which looks like this

<pre>
allow(AssignmentQuestionnaire).to receive(:find_by).with(assignment_id: 1, questionnaire_id: nil).and_return(double('AssignmentQuestionnaire', used_in_round: 1))
</pre>

similarly, other calls made to

<pre>
self.assignment_questionnaires.find_by(assignment_id: assignment.id)
</pre>

was mocked using calls that look like :-

<pre>
allow(ReviewQuestionnaire).to receive_message_chain(:assignment_questionnaires,:find_by).with(no_args).with(assignment_id: 1).and_return(double('AssignmentQuestionnaire', id: 1))
</pre>

'''cases for consideration :'''

1. when avg score equals nil

2. when avg score is not equal to nil

----

===Function: compute_reviews_hash===
Below is most of the code used in the test case. It is relatively simple. The first test case is for when assignments vary by rubrics. We check that the function returns {}. This is correct because we set reviewer to nil so that reviewer will be set to {} in the "reviewer = {} if reviewer.nil?" statement. In the second test case we check that it returns {1=>{1=>50}, 2=>{1=>30}}. This is correct because we create 2 response maps and set their value to 50 and 30.

context 'when current assignment varys rubrics by round' do
it 'scores varying rubrics and returns review scores' do
allow(assignment).to receive(:varying_rubrics_by_round?).and_return(TRUE)
allow(assignment).to receive(:rounds_of_reviews).and_return(1)
temp = assignment.compute_reviews_hash()
expect(temp).to eql({})
end
end

context 'when current assignment does not vary rubrics by round' do
it 'scores varying rubrics and returns review scores' do
temp = assignment.compute_reviews_hash()
expect(temp).to eql({1=>{1=>50}, 2=>{1=>30}})
end
end

===Function: compute_avg_and_ranges_hash===
This function calculates the average score and score range for each reviewee(team).

<pre>
def compute_avg_and_ranges_hash
scores = {}
contributors = self.contributors # assignment_teams
if self.varying_rubrics_by_round?
rounds = self.rounds_of_reviews
(1..rounds).each do |round|
review_questionnaire_id = review_questionnaire_id(round)
questions = Question.where('questionnaire_id = ?', review_questionnaire_id)
contributors.each do |contributor|
assessments = ReviewResponseMap.get_assessments_for(contributor)
assessments = assessments.select {|assessment| assessment.round == round }
scores[contributor.id] = {} if round == 1
scores[contributor.id][round] = {}
scores[contributor.id][round] = Answer.compute_scores(assessments, questions)
end
end
else
review_questionnaire_id = review_questionnaire_id()
questions = Question.where('questionnaire_id = ?', review_questionnaire_id)
contributors.each do |contributor|
assessments = ReviewResponseMap.get_assessments_for(contributor)
scores[contributor.id] = {}
scores[contributor.id] = Answer.compute_scores(assessments, questions)
end
end
scores
end
</pre>

===Function: scores===
<pre>
def scores(questions)
index = 0
score_assignment
self.teams.each do |team|
score_team = {}
score_team[:team] = team
if self.varying_rubrics_by_round?
calculate_rounds
calculate_score
calculate_assessment
else
assessments = ReviewResponseMap.get_assessments_for(team)
score_team[:scores] = Answer.compute_scores(assessments, questions)
end
index += 1
end
score
end
end
</pre>

This method calls a lot of private methods to calculate a score hash. To test this method we had to mock calls made in these private methods.

When ''varying_rubrics_by_round?'' is true this method calls
1. calculate_rounds

2. calculate_score

3. calculate_assessment

we mocked the call to 1. so it is not covered in our coverage as it just initializes our hashes.

to mock the call to 2. and 3. calls such as the ones given below can be seen in our test cases.

<pre>
allow(on_the_fly_calc).to receive(:index).and_return(0)
allow(on_the_fly_calc).to receive(:num_review_rounds).and_return([1,2])
allow(on_the_fly_calc).to receive(:team).and_return(double('AssignmentTeam'))
allow(ReviewResponseMap).to receive(:get_responses_for_team_round).with(any_args).and_return([])
allow(on_the_fly_calc).to receive(:questions).and_return(questions)
allow(Answer).to receive(:compute_scores).with([],[question1]).and_return({})
allow(on_the_fly_calc).to receive(:round_sym).and_return(:review1)
allow(on_the_fly_calc).to receive(:grades_by_rounds).and_return(0)
allow(on_the_fly_calc).to receive(:score_assignment).and_return('')
</pre>

we are basically mocking calls to run through all lines of the methods and only calls that is returning something or are assigning value to a variable need to be mocked.

for example the call
<pre>
allow(Answer).to receive(:compute_scores).with([],[question1]).and_return({})
</pre>
is to mock the call made to ''compute_score'' function in the ''Answer'' class we made it to return an empty hash.

'''cases for consideration :'''

1. when current assignment varys rubrics by round and number of assessments is 0.in this case
<pre>
allow(on_the_fly_calc).to receive(:total_num_of_assessments).and_return(0)
</pre>
this call is made to mask the value of total_num_of_assessments value, similarly for others.

2. when current assignment varys rubrics by round and number of assessments is non-zero.

3. when current assignment does not vary rubrics by round

----

===Running RSpec===
*To run RSpec for a particular file (on_the_fly_calc_spec.rb):
<pre>
$ rspec spec/models/on_the_fly_calc_spec.rb
</pre>

----

===Coverage Results===

We have a 92.74% coverage.
Below is a picture of the results from the coverage.

[[File:on_the_fly_test_results.png]]

==References==
1. [https://en.wikipedia.org/wiki/RSpec RSpec Wiki] <br>
2. [http://rspec.info/ RSpec.info] <br>
3. [https://www.youtube.com/watch?v=dzkVfaKChSU&feature=youtu.be&t=35s youtube.com TDD: RSpec Test Structure] <br>
4. [https://github.com/expertiza/expertiza/blob/3ce553a2d0258ea05bced910abae5d209a7f55d6/spec/models/response_spec.rb github/expertiza response_spec example] <br>
5. [https://relishapp.com/rspec RSpec Documentation] <br>

CSC/ECE 517 Spring 2018- Project E1812: on the fly calc.rb

2018-04-03T00:22:14Z

Adsimps3: /* Coverage Results */

==E1812: Unit tests for on_the_fly_calc.rb==

This wiki page gives a description of changes made under Expertiza OSS project E1812 assignment for Spring 2018, CSC/ECE 517.

===NOTE to reviewers===

''' This documentation is still under implementation'''

Mocks and stubs are used for the unit tests so they can't be tested by UI.

__TOC__

==Expertiza Background==

[http://expertiza.ncsu.edu/ Expertiza] is an open source web application developed with [http://rubyonrails.org/ Ruby on Rails] framework. This application allows students to access assignments posted by an instructor, submit their work (articles, codes, websites), review others work submissions. Expertiza gives a chance to improve students’ work based on their peer-reviews, and it can also help in grading students’ submissions based on that feedback. Students can form their teams for an assignment, otherwise, teams can be assigned automatically.

----

== Problem Statement ==
The OnTheFlyCalc model does not have any test cases corresponding to it. Thus, the task of this assignment is to write unit tests for on_the_fly_calc.rb using RSpec. The test cases should be fast, effective, and achieve the maximum code coverage, i.e. > 90%.

----
=== OnTheFlyCalc ===

On_the_fly_calc is a module that is included in assignment.rb. This module calculates the score for both students and instructors. E.g., when a student clicks “view my score”, Expertiza calculates the review scores for each review. This is called “on the fly” because expertiza stores the review scores based on each question (in answers table), but does not store the total score that a reviewer gives to a reviewee. It directly calculates the total review score for each review, using the score values in the answer table during run-time.

=== Created/Modified Files ===
As a part of the project the file listed below was created and saved under spec/models folder:
*on_the_fly_calc_spec.rb

The file listed below was modified due to some code issues in it:
*on_the_fly_calc.rb (path: app/models)

=== Thoughts ===
What we need to do is to write tests for four public methods listed in the module OnTheFLyCalc. However, the on_the_fly_calc itself has some code issues in #scores, and IDE reports some errors when we run the test case for it. Thus, we need to modify the module itself a little.

----

== Tools ==

===RSpec ===
For this project we use [https://en.wikipedia.org/wiki/RSpec RSpec] which is a testing tool for Ruby, created for [https://en.wikipedia.org/wiki/Behavior-driven_development behavior-driven development] (BDD).

----

== Plan of Work ==
In order to accomplish this task, that is to build tests for the OnTheFlyCalc model, we need to implement the following plan:

1. Set up the Expertiza environment;

2. Understand the functionality of the model;

3. Understand the relative methods and objects involved with each function of the model;

4. Create test objects;

5. Write ‘it block’ test scenarios with appropriate contexts for all public functions of the model, and check if all [https://relishapp.com/rspec/rspec-expectations/docs expectations ] pass.

----

'''Expertiza Environment'''

We installed VM VirtualBox, and imported Ubuntu-Expertiza image into VirtualBox. In Git, Expertiza master brunch was forked, in the VM terminal we cloned it and then run all required commands to finish setting up the environment.

----

== Testing ==

Listed below are the functions in on_the_fly_calc.rb and the RSpec unit tests corresponding to the function names along with a list of scenarios tested.

===Function: compute_total_score===
The function compute total score totals the scores of each questionnaire in an assignment.

<pre>
def compute_total_score(scores)
total = 0
self.questionnaires.each {|questionnaire| total += questionnaire.get_weighted_score(self, scores) }
total
end
</pre>

''To test this function''

we mocked the calls made in the Questionnaire class.
for example to mock the call

<pre>
round = AssignmentQuestionnaire.find_by(assignment_id: assignment.id, questionnaire_id: self.id).used_in_round
</pre>

we used the rspec line which looks like this

<pre>
allow(AssignmentQuestionnaire).to receive(:find_by).with(assignment_id: 1, questionnaire_id: nil).and_return(double('AssignmentQuestionnaire', used_in_round: 1))
</pre>

similarly, other calls made to

<pre>
self.assignment_questionnaires.find_by(assignment_id: assignment.id)
</pre>

was mocked using calls that look like :-

<pre>
allow(ReviewQuestionnaire).to receive_message_chain(:assignment_questionnaires,:find_by).with(no_args).with(assignment_id: 1).and_return(double('AssignmentQuestionnaire', id: 1))
</pre>

'''cases for consideration :'''

1. when avg score equals nil

2. when avg score is not equal to nil

----

===Function: compute_reviews_hash===
Below is most of the code used in the test case. It is relatively simple. The first test case is for when assignments vary by rubrics. We check that the function returns {}. This is correct because we set reviewer to nil so that reviewer will be set to {} in the "reviewer = {} if reviewer.nil?" statement. In the second test case we check that it returns {1=>{1=>50}, 2=>{1=>30}}. This is correct because we create 2 response maps and set their value to 50 and 30.

context 'when current assignment varys rubrics by round' do
it 'scores varying rubrics and returns review scores' do
allow(assignment).to receive(:varying_rubrics_by_round?).and_return(TRUE)
allow(assignment).to receive(:rounds_of_reviews).and_return(1)
temp = assignment.compute_reviews_hash()
expect(temp).to eql({})
end
end

context 'when current assignment does not vary rubrics by round' do
it 'scores varying rubrics and returns review scores' do
temp = assignment.compute_reviews_hash()
expect(temp).to eql({1=>{1=>50}, 2=>{1=>30}})
end
end

===Function: compute_avg_and_ranges_hash===

===Function: scores===
<pre>
def scores(questions)
index = 0
score_assignment
self.teams.each do |team|
score_team = {}
score_team[:team] = team
if self.varying_rubrics_by_round?
calculate_rounds
calculate_score
calculate_assessment
else
assessments = ReviewResponseMap.get_assessments_for(team)
score_team[:scores] = Answer.compute_scores(assessments, questions)
end
index += 1
end
score
end
end
</pre>

This method calls a lot of private methods to calculate a score hash. To test this method we had to mock calls made in these private methods.

When ''varying_rubrics_by_round?'' is true this method calls
1. calculate_rounds

2. calculate_score

3. calculate_assessment

we mocked the call to 1. so it is not covered in our coverage as it just initializes our hashes.

to mock the call to 2. and 3. calls such as the ones given below can be seen in our test cases.

<pre>
allow(on_the_fly_calc).to receive(:index).and_return(0)
allow(on_the_fly_calc).to receive(:num_review_rounds).and_return([1,2])
allow(on_the_fly_calc).to receive(:team).and_return(double('AssignmentTeam'))
allow(ReviewResponseMap).to receive(:get_responses_for_team_round).with(any_args).and_return([])
allow(on_the_fly_calc).to receive(:questions).and_return(questions)
allow(Answer).to receive(:compute_scores).with([],[question1]).and_return({})
allow(on_the_fly_calc).to receive(:round_sym).and_return(:review1)
allow(on_the_fly_calc).to receive(:grades_by_rounds).and_return(0)
allow(on_the_fly_calc).to receive(:score_assignment).and_return('')
</pre>

we are basically mocking calls to run through all lines of the methods and only calls that is returning something or are assigning value to a variable need to be mocked.

for example the call
<pre>
allow(Answer).to receive(:compute_scores).with([],[question1]).and_return({})
</pre>
is to mock the call made to ''compute_score'' function in the ''Answer'' class we made it to return an empty hash.

'''cases for consideration :'''

1. when current assignment varys rubrics by round and number of assessments is 0.in this case
<pre>
allow(on_the_fly_calc).to receive(:total_num_of_assessments).and_return(0)
</pre>
this call is made to mask the value of total_num_of_assessments value, similarly for others.

2. when current assignment varys rubrics by round and number of assessments is non-zero.

3. when current assignment does not vary rubrics by round

----

===Running RSpec===
*To run RSpec for a particular file (on_the_fly_calc_spec.rb):
<pre>
$ rspec spec/models/on_the_fly_calc_spec.rb
</pre>

----

===Coverage Results===

We have a 92.74% coverage.
Below is a picture of the results from the coverage.

[[File:on_the_fly_test_results.png]]

==References==
1. [https://en.wikipedia.org/wiki/RSpec RSpec Wiki] <br>
2. [http://rspec.info/ RSpec.info] <br>
3. [https://www.youtube.com/watch?v=dzkVfaKChSU&feature=youtu.be&t=35s youtube.com TDD: RSpec Test Structure] <br>
4. [https://github.com/expertiza/expertiza/blob/3ce553a2d0258ea05bced910abae5d209a7f55d6/spec/models/response_spec.rb github/expertiza response_spec example] <br>
5. [https://relishapp.com/rspec RSpec Documentation] <br>

File:On the fly test results.png

2018-04-03T00:21:50Z

Adsimps3:

CSC/ECE 517 Spring 2018- Project E1812: on the fly calc.rb

2018-04-03T00:21:05Z

Adsimps3: /* Coverage Results */

==E1812: Unit tests for on_the_fly_calc.rb==

This wiki page gives a description of changes made under Expertiza OSS project E1812 assignment for Spring 2018, CSC/ECE 517.

===NOTE to reviewers===

''' This documentation is still under implementation'''

Mocks and stubs are used for the unit tests so they can't be tested by UI.

__TOC__

==Expertiza Background==

[http://expertiza.ncsu.edu/ Expertiza] is an open source web application developed with [http://rubyonrails.org/ Ruby on Rails] framework. This application allows students to access assignments posted by an instructor, submit their work (articles, codes, websites), review others work submissions. Expertiza gives a chance to improve students’ work based on their peer-reviews, and it can also help in grading students’ submissions based on that feedback. Students can form their teams for an assignment, otherwise, teams can be assigned automatically.

----

== Problem Statement ==
The OnTheFlyCalc model does not have any test cases corresponding to it. Thus, the task of this assignment is to write unit tests for on_the_fly_calc.rb using RSpec. The test cases should be fast, effective, and achieve the maximum code coverage, i.e. > 90%.

----
=== OnTheFlyCalc ===

On_the_fly_calc is a module that is included in assignment.rb. This module calculates the score for both students and instructors. E.g., when a student clicks “view my score”, Expertiza calculates the review scores for each review. This is called “on the fly” because expertiza stores the review scores based on each question (in answers table), but does not store the total score that a reviewer gives to a reviewee. It directly calculates the total review score for each review, using the score values in the answer table during run-time.

=== Created/Modified Files ===
As a part of the project the file listed below was created and saved under spec/models folder:
*on_the_fly_calc_spec.rb

The file listed below was modified due to some code issues in it:
*on_the_fly_calc.rb (path: app/models)

=== Thoughts ===
What we need to do is to write tests for four public methods listed in the module OnTheFLyCalc. However, the on_the_fly_calc itself has some code issues in #scores, and IDE reports some errors when we run the test case for it. Thus, we need to modify the module itself a little.

----

== Tools ==

===RSpec ===
For this project we use [https://en.wikipedia.org/wiki/RSpec RSpec] which is a testing tool for Ruby, created for [https://en.wikipedia.org/wiki/Behavior-driven_development behavior-driven development] (BDD).

----

== Plan of Work ==
In order to accomplish this task, that is to build tests for the OnTheFlyCalc model, we need to implement the following plan:

1. Set up the Expertiza environment;

2. Understand the functionality of the model;

3. Understand the relative methods and objects involved with each function of the model;

4. Create test objects;

5. Write ‘it block’ test scenarios with appropriate contexts for all public functions of the model, and check if all [https://relishapp.com/rspec/rspec-expectations/docs expectations ] pass.

----

'''Expertiza Environment'''

We installed VM VirtualBox, and imported Ubuntu-Expertiza image into VirtualBox. In Git, Expertiza master brunch was forked, in the VM terminal we cloned it and then run all required commands to finish setting up the environment.

----

== Testing ==

Listed below are the functions in on_the_fly_calc.rb and the RSpec unit tests corresponding to the function names along with a list of scenarios tested.

===Function: compute_total_score===
The function compute total score totals the scores of each questionnaire in an assignment.

<pre>
def compute_total_score(scores)
total = 0
self.questionnaires.each {|questionnaire| total += questionnaire.get_weighted_score(self, scores) }
total
end
</pre>

''To test this function''

we mocked the calls made in the Questionnaire class.
for example to mock the call

<pre>
round = AssignmentQuestionnaire.find_by(assignment_id: assignment.id, questionnaire_id: self.id).used_in_round
</pre>

we used the rspec line which looks like this

<pre>
allow(AssignmentQuestionnaire).to receive(:find_by).with(assignment_id: 1, questionnaire_id: nil).and_return(double('AssignmentQuestionnaire', used_in_round: 1))
</pre>

similarly, other calls made to

<pre>
self.assignment_questionnaires.find_by(assignment_id: assignment.id)
</pre>

was mocked using calls that look like :-

<pre>
allow(ReviewQuestionnaire).to receive_message_chain(:assignment_questionnaires,:find_by).with(no_args).with(assignment_id: 1).and_return(double('AssignmentQuestionnaire', id: 1))
</pre>

'''cases for consideration :'''

1. when avg score equals nil

2. when avg score is not equal to nil

----

===Function: compute_reviews_hash===
Below is most of the code used in the test case. It is relatively simple. The first test case is for when assignments vary by rubrics. We check that the function returns {}. This is correct because we set reviewer to nil so that reviewer will be set to {} in the "reviewer = {} if reviewer.nil?" statement. In the second test case we check that it returns {1=>{1=>50}, 2=>{1=>30}}. This is correct because we create 2 response maps and set their value to 50 and 30.

context 'when current assignment varys rubrics by round' do
it 'scores varying rubrics and returns review scores' do
allow(assignment).to receive(:varying_rubrics_by_round?).and_return(TRUE)
allow(assignment).to receive(:rounds_of_reviews).and_return(1)
temp = assignment.compute_reviews_hash()
expect(temp).to eql({})
end
end

context 'when current assignment does not vary rubrics by round' do
it 'scores varying rubrics and returns review scores' do
temp = assignment.compute_reviews_hash()
expect(temp).to eql({1=>{1=>50}, 2=>{1=>30}})
end
end

===Function: compute_avg_and_ranges_hash===

===Function: scores===
<pre>
def scores(questions)
index = 0
score_assignment
self.teams.each do |team|
score_team = {}
score_team[:team] = team
if self.varying_rubrics_by_round?
calculate_rounds
calculate_score
calculate_assessment
else
assessments = ReviewResponseMap.get_assessments_for(team)
score_team[:scores] = Answer.compute_scores(assessments, questions)
end
index += 1
end
score
end
end
</pre>

This method calls a lot of private methods to calculate a score hash. To test this method we had to mock calls made in these private methods.

When ''varying_rubrics_by_round?'' is true this method calls
1. calculate_rounds

2. calculate_score

3. calculate_assessment

we mocked the call to 1. so it is not covered in our coverage as it just initializes our hashes.

to mock the call to 2. and 3. calls such as the ones given below can be seen in our test cases.

<pre>
allow(on_the_fly_calc).to receive(:index).and_return(0)
allow(on_the_fly_calc).to receive(:num_review_rounds).and_return([1,2])
allow(on_the_fly_calc).to receive(:team).and_return(double('AssignmentTeam'))
allow(ReviewResponseMap).to receive(:get_responses_for_team_round).with(any_args).and_return([])
allow(on_the_fly_calc).to receive(:questions).and_return(questions)
allow(Answer).to receive(:compute_scores).with([],[question1]).and_return({})
allow(on_the_fly_calc).to receive(:round_sym).and_return(:review1)
allow(on_the_fly_calc).to receive(:grades_by_rounds).and_return(0)
allow(on_the_fly_calc).to receive(:score_assignment).and_return('')
</pre>

we are basically mocking calls to run through all lines of the methods and only calls that is returning something or are assigning value to a variable need to be mocked.

for example the call
<pre>
allow(Answer).to receive(:compute_scores).with([],[question1]).and_return({})
</pre>
is to mock the call made to ''compute_score'' function in the ''Answer'' class we made it to return an empty hash.

'''cases for consideration :'''

1. when current assignment varys rubrics by round and number of assessments is 0.in this case
<pre>
allow(on_the_fly_calc).to receive(:total_num_of_assessments).and_return(0)
</pre>
this call is made to mask the value of total_num_of_assessments value, similarly for others.

2. when current assignment varys rubrics by round and number of assessments is non-zero.

3. when current assignment does not vary rubrics by round

----

===Running RSpec===
*To run RSpec for a particular file (on_the_fly_calc_spec.rb):
<pre>
$ rspec spec/models/on_the_fly_calc_spec.rb
</pre>

----

===Coverage Results===

We have a 92.74% coverage.
Below is a picture of the results from the coverage.

[[File:ONTHEFLYTESTRESULTS.png]]

==References==
1. [https://en.wikipedia.org/wiki/RSpec RSpec Wiki] <br>
2. [http://rspec.info/ RSpec.info] <br>
3. [https://www.youtube.com/watch?v=dzkVfaKChSU&feature=youtu.be&t=35s youtube.com TDD: RSpec Test Structure] <br>
4. [https://github.com/expertiza/expertiza/blob/3ce553a2d0258ea05bced910abae5d209a7f55d6/spec/models/response_spec.rb github/expertiza response_spec example] <br>
5. [https://relishapp.com/rspec RSpec Documentation] <br>

CSC/ECE 517 Spring 2018- Project E1812: on the fly calc.rb

2018-04-03T00:20:28Z

Adsimps3: /* Coverage Results */

==E1812: Unit tests for on_the_fly_calc.rb==

This wiki page gives a description of changes made under Expertiza OSS project E1812 assignment for Spring 2018, CSC/ECE 517.

===NOTE to reviewers===

''' This documentation is still under implementation'''

Mocks and stubs are used for the unit tests so they can't be tested by UI.

__TOC__

==Expertiza Background==

[http://expertiza.ncsu.edu/ Expertiza] is an open source web application developed with [http://rubyonrails.org/ Ruby on Rails] framework. This application allows students to access assignments posted by an instructor, submit their work (articles, codes, websites), review others work submissions. Expertiza gives a chance to improve students’ work based on their peer-reviews, and it can also help in grading students’ submissions based on that feedback. Students can form their teams for an assignment, otherwise, teams can be assigned automatically.

----

== Problem Statement ==
The OnTheFlyCalc model does not have any test cases corresponding to it. Thus, the task of this assignment is to write unit tests for on_the_fly_calc.rb using RSpec. The test cases should be fast, effective, and achieve the maximum code coverage, i.e. > 90%.

----
=== OnTheFlyCalc ===

On_the_fly_calc is a module that is included in assignment.rb. This module calculates the score for both students and instructors. E.g., when a student clicks “view my score”, Expertiza calculates the review scores for each review. This is called “on the fly” because expertiza stores the review scores based on each question (in answers table), but does not store the total score that a reviewer gives to a reviewee. It directly calculates the total review score for each review, using the score values in the answer table during run-time.

=== Created/Modified Files ===
As a part of the project the file listed below was created and saved under spec/models folder:
*on_the_fly_calc_spec.rb

The file listed below was modified due to some code issues in it:
*on_the_fly_calc.rb (path: app/models)

=== Thoughts ===
What we need to do is to write tests for four public methods listed in the module OnTheFLyCalc. However, the on_the_fly_calc itself has some code issues in #scores, and IDE reports some errors when we run the test case for it. Thus, we need to modify the module itself a little.

----

== Tools ==

===RSpec ===
For this project we use [https://en.wikipedia.org/wiki/RSpec RSpec] which is a testing tool for Ruby, created for [https://en.wikipedia.org/wiki/Behavior-driven_development behavior-driven development] (BDD).

----

== Plan of Work ==
In order to accomplish this task, that is to build tests for the OnTheFlyCalc model, we need to implement the following plan:

1. Set up the Expertiza environment;

2. Understand the functionality of the model;

3. Understand the relative methods and objects involved with each function of the model;

4. Create test objects;

5. Write ‘it block’ test scenarios with appropriate contexts for all public functions of the model, and check if all [https://relishapp.com/rspec/rspec-expectations/docs expectations ] pass.

----

'''Expertiza Environment'''

We installed VM VirtualBox, and imported Ubuntu-Expertiza image into VirtualBox. In Git, Expertiza master brunch was forked, in the VM terminal we cloned it and then run all required commands to finish setting up the environment.

----

== Testing ==

Listed below are the functions in on_the_fly_calc.rb and the RSpec unit tests corresponding to the function names along with a list of scenarios tested.

===Function: compute_total_score===
The function compute total score totals the scores of each questionnaire in an assignment.

<pre>
def compute_total_score(scores)
total = 0
self.questionnaires.each {|questionnaire| total += questionnaire.get_weighted_score(self, scores) }
total
end
</pre>

''To test this function''

we mocked the calls made in the Questionnaire class.
for example to mock the call

<pre>
round = AssignmentQuestionnaire.find_by(assignment_id: assignment.id, questionnaire_id: self.id).used_in_round
</pre>

we used the rspec line which looks like this

<pre>
allow(AssignmentQuestionnaire).to receive(:find_by).with(assignment_id: 1, questionnaire_id: nil).and_return(double('AssignmentQuestionnaire', used_in_round: 1))
</pre>

similarly, other calls made to

<pre>
self.assignment_questionnaires.find_by(assignment_id: assignment.id)
</pre>

was mocked using calls that look like :-

<pre>
allow(ReviewQuestionnaire).to receive_message_chain(:assignment_questionnaires,:find_by).with(no_args).with(assignment_id: 1).and_return(double('AssignmentQuestionnaire', id: 1))
</pre>

'''cases for consideration :'''

1. when avg score equals nil

2. when avg score is not equal to nil

----

===Function: compute_reviews_hash===
Below is most of the code used in the test case. It is relatively simple. The first test case is for when assignments vary by rubrics. We check that the function returns {}. This is correct because we set reviewer to nil so that reviewer will be set to {} in the "reviewer = {} if reviewer.nil?" statement. In the second test case we check that it returns {1=>{1=>50}, 2=>{1=>30}}. This is correct because we create 2 response maps and set their value to 50 and 30.

context 'when current assignment varys rubrics by round' do
it 'scores varying rubrics and returns review scores' do
allow(assignment).to receive(:varying_rubrics_by_round?).and_return(TRUE)
allow(assignment).to receive(:rounds_of_reviews).and_return(1)
temp = assignment.compute_reviews_hash()
expect(temp).to eql({})
end
end

context 'when current assignment does not vary rubrics by round' do
it 'scores varying rubrics and returns review scores' do
temp = assignment.compute_reviews_hash()
expect(temp).to eql({1=>{1=>50}, 2=>{1=>30}})
end
end

===Function: compute_avg_and_ranges_hash===

===Function: scores===
<pre>
def scores(questions)
index = 0
score_assignment
self.teams.each do |team|
score_team = {}
score_team[:team] = team
if self.varying_rubrics_by_round?
calculate_rounds
calculate_score
calculate_assessment
else
assessments = ReviewResponseMap.get_assessments_for(team)
score_team[:scores] = Answer.compute_scores(assessments, questions)
end
index += 1
end
score
end
end
</pre>

This method calls a lot of private methods to calculate a score hash. To test this method we had to mock calls made in these private methods.

When ''varying_rubrics_by_round?'' is true this method calls
1. calculate_rounds

2. calculate_score

3. calculate_assessment

we mocked the call to 1. so it is not covered in our coverage as it just initializes our hashes.

to mock the call to 2. and 3. calls such as the ones given below can be seen in our test cases.

<pre>
allow(on_the_fly_calc).to receive(:index).and_return(0)
allow(on_the_fly_calc).to receive(:num_review_rounds).and_return([1,2])
allow(on_the_fly_calc).to receive(:team).and_return(double('AssignmentTeam'))
allow(ReviewResponseMap).to receive(:get_responses_for_team_round).with(any_args).and_return([])
allow(on_the_fly_calc).to receive(:questions).and_return(questions)
allow(Answer).to receive(:compute_scores).with([],[question1]).and_return({})
allow(on_the_fly_calc).to receive(:round_sym).and_return(:review1)
allow(on_the_fly_calc).to receive(:grades_by_rounds).and_return(0)
allow(on_the_fly_calc).to receive(:score_assignment).and_return('')
</pre>

we are basically mocking calls to run through all lines of the methods and only calls that is returning something or are assigning value to a variable need to be mocked.

for example the call
<pre>
allow(Answer).to receive(:compute_scores).with([],[question1]).and_return({})
</pre>
is to mock the call made to ''compute_score'' function in the ''Answer'' class we made it to return an empty hash.

'''cases for consideration :'''

1. when current assignment varys rubrics by round and number of assessments is 0.in this case
<pre>
allow(on_the_fly_calc).to receive(:total_num_of_assessments).and_return(0)
</pre>
this call is made to mask the value of total_num_of_assessments value, similarly for others.

2. when current assignment varys rubrics by round and number of assessments is non-zero.

3. when current assignment does not vary rubrics by round

----

===Running RSpec===
*To run RSpec for a particular file (on_the_fly_calc_spec.rb):
<pre>
$ rspec spec/models/on_the_fly_calc_spec.rb
</pre>

----

===Coverage Results===

We have a 92.74% coverage.
Below is a picture of the results from the coverage.

[[File:ONTHEFLYTESTRESULTS]]

==References==
1. [https://en.wikipedia.org/wiki/RSpec RSpec Wiki] <br>
2. [http://rspec.info/ RSpec.info] <br>
3. [https://www.youtube.com/watch?v=dzkVfaKChSU&feature=youtu.be&t=35s youtube.com TDD: RSpec Test Structure] <br>
4. [https://github.com/expertiza/expertiza/blob/3ce553a2d0258ea05bced910abae5d209a7f55d6/spec/models/response_spec.rb github/expertiza response_spec example] <br>
5. [https://relishapp.com/rspec RSpec Documentation] <br>

CSC/ECE 517 Spring 2018- Project E1812: on the fly calc.rb

2018-04-03T00:18:30Z

Adsimps3: /* Coverage Results */

==E1812: Unit tests for on_the_fly_calc.rb==

This wiki page gives a description of changes made under Expertiza OSS project E1812 assignment for Spring 2018, CSC/ECE 517.

===NOTE to reviewers===

''' This documentation is still under implementation'''

Mocks and stubs are used for the unit tests so they can't be tested by UI.

__TOC__

==Expertiza Background==

[http://expertiza.ncsu.edu/ Expertiza] is an open source web application developed with [http://rubyonrails.org/ Ruby on Rails] framework. This application allows students to access assignments posted by an instructor, submit their work (articles, codes, websites), review others work submissions. Expertiza gives a chance to improve students’ work based on their peer-reviews, and it can also help in grading students’ submissions based on that feedback. Students can form their teams for an assignment, otherwise, teams can be assigned automatically.

----

== Problem Statement ==
The OnTheFlyCalc model does not have any test cases corresponding to it. Thus, the task of this assignment is to write unit tests for on_the_fly_calc.rb using RSpec. The test cases should be fast, effective, and achieve the maximum code coverage, i.e. > 90%.

----
=== OnTheFlyCalc ===

On_the_fly_calc is a module that is included in assignment.rb. This module calculates the score for both students and instructors. E.g., when a student clicks “view my score”, Expertiza calculates the review scores for each review. This is called “on the fly” because expertiza stores the review scores based on each question (in answers table), but does not store the total score that a reviewer gives to a reviewee. It directly calculates the total review score for each review, using the score values in the answer table during run-time.

=== Created/Modified Files ===
As a part of the project the file listed below was created and saved under spec/models folder:
*on_the_fly_calc_spec.rb

The file listed below was modified due to some code issues in it:
*on_the_fly_calc.rb (path: app/models)

=== Thoughts ===
What we need to do is to write tests for four public methods listed in the module OnTheFLyCalc. However, the on_the_fly_calc itself has some code issues in #scores, and IDE reports some errors when we run the test case for it. Thus, we need to modify the module itself a little.

----

== Tools ==

===RSpec ===
For this project we use [https://en.wikipedia.org/wiki/RSpec RSpec] which is a testing tool for Ruby, created for [https://en.wikipedia.org/wiki/Behavior-driven_development behavior-driven development] (BDD).

----

== Plan of Work ==
In order to accomplish this task, that is to build tests for the OnTheFlyCalc model, we need to implement the following plan:

1. Set up the Expertiza environment;

2. Understand the functionality of the model;

3. Understand the relative methods and objects involved with each function of the model;

4. Create test objects;

5. Write ‘it block’ test scenarios with appropriate contexts for all public functions of the model, and check if all [https://relishapp.com/rspec/rspec-expectations/docs expectations ] pass.

----

'''Expertiza Environment'''

We installed VM VirtualBox, and imported Ubuntu-Expertiza image into VirtualBox. In Git, Expertiza master brunch was forked, in the VM terminal we cloned it and then run all required commands to finish setting up the environment.

----

== Testing ==

Listed below are the functions in on_the_fly_calc.rb and the RSpec unit tests corresponding to the function names along with a list of scenarios tested.

===Function: compute_total_score===
The function compute total score totals the scores of each questionnaire in an assignment.

<pre>
def compute_total_score(scores)
total = 0
self.questionnaires.each {|questionnaire| total += questionnaire.get_weighted_score(self, scores) }
total
end
</pre>

''To test this function''

we mocked the calls made in the Questionnaire class.
for example to mock the call

<pre>
round = AssignmentQuestionnaire.find_by(assignment_id: assignment.id, questionnaire_id: self.id).used_in_round
</pre>

we used the rspec line which looks like this

<pre>
allow(AssignmentQuestionnaire).to receive(:find_by).with(assignment_id: 1, questionnaire_id: nil).and_return(double('AssignmentQuestionnaire', used_in_round: 1))
</pre>

similarly, other calls made to

<pre>
self.assignment_questionnaires.find_by(assignment_id: assignment.id)
</pre>

was mocked using calls that look like :-

<pre>
allow(ReviewQuestionnaire).to receive_message_chain(:assignment_questionnaires,:find_by).with(no_args).with(assignment_id: 1).and_return(double('AssignmentQuestionnaire', id: 1))
</pre>

'''cases for consideration :'''

1. when avg score equals nil

2. when avg score is not equal to nil

----

===Function: compute_reviews_hash===
Below is most of the code used in the test case. It is relatively simple. The first test case is for when assignments vary by rubrics. We check that the function returns {}. This is correct because we set reviewer to nil so that reviewer will be set to {} in the "reviewer = {} if reviewer.nil?" statement. In the second test case we check that it returns {1=>{1=>50}, 2=>{1=>30}}. This is correct because we create 2 response maps and set their value to 50 and 30.

context 'when current assignment varys rubrics by round' do
it 'scores varying rubrics and returns review scores' do
allow(assignment).to receive(:varying_rubrics_by_round?).and_return(TRUE)
allow(assignment).to receive(:rounds_of_reviews).and_return(1)
temp = assignment.compute_reviews_hash()
expect(temp).to eql({})
end
end

context 'when current assignment does not vary rubrics by round' do
it 'scores varying rubrics and returns review scores' do
temp = assignment.compute_reviews_hash()
expect(temp).to eql({1=>{1=>50}, 2=>{1=>30}})
end
end

===Function: compute_avg_and_ranges_hash===

===Function: scores===
<pre>
def scores(questions)
index = 0
score_assignment
self.teams.each do |team|
score_team = {}
score_team[:team] = team
if self.varying_rubrics_by_round?
calculate_rounds
calculate_score
calculate_assessment
else
assessments = ReviewResponseMap.get_assessments_for(team)
score_team[:scores] = Answer.compute_scores(assessments, questions)
end
index += 1
end
score
end
end
</pre>

This method calls a lot of private methods to calculate a score hash. To test this method we had to mock calls made in these private methods.

When ''varying_rubrics_by_round?'' is true this method calls
1. calculate_rounds

2. calculate_score

3. calculate_assessment

we mocked the call to 1. so it is not covered in our coverage as it just initializes our hashes.

to mock the call to 2. and 3. calls such as the ones given below can be seen in our test cases.

<pre>
allow(on_the_fly_calc).to receive(:index).and_return(0)
allow(on_the_fly_calc).to receive(:num_review_rounds).and_return([1,2])
allow(on_the_fly_calc).to receive(:team).and_return(double('AssignmentTeam'))
allow(ReviewResponseMap).to receive(:get_responses_for_team_round).with(any_args).and_return([])
allow(on_the_fly_calc).to receive(:questions).and_return(questions)
allow(Answer).to receive(:compute_scores).with([],[question1]).and_return({})
allow(on_the_fly_calc).to receive(:round_sym).and_return(:review1)
allow(on_the_fly_calc).to receive(:grades_by_rounds).and_return(0)
allow(on_the_fly_calc).to receive(:score_assignment).and_return('')
</pre>

we are basically mocking calls to run through all lines of the methods and only calls that is returning something or are assigning value to a variable need to be mocked.

for example the call
<pre>
allow(Answer).to receive(:compute_scores).with([],[question1]).and_return({})
</pre>
is to mock the call made to ''compute_score'' function in the ''Answer'' class we made it to return an empty hash.

'''cases for consideration :'''

1. when current assignment varys rubrics by round and number of assessments is 0.in this case
<pre>
allow(on_the_fly_calc).to receive(:total_num_of_assessments).and_return(0)
</pre>
this call is made to mask the value of total_num_of_assessments value, similarly for others.

2. when current assignment varys rubrics by round and number of assessments is non-zero.

3. when current assignment does not vary rubrics by round

----

===Running RSpec===
*To run RSpec for a particular file (on_the_fly_calc_spec.rb):
<pre>
$ rspec spec/models/on_the_fly_calc_spec.rb
</pre>

----

===Coverage Results===

We have a 92.74% coverage.
Below is a picture of the results from the coverage.

[[File:ontheflytest.png]]

==References==
1. [https://en.wikipedia.org/wiki/RSpec RSpec Wiki] <br>
2. [http://rspec.info/ RSpec.info] <br>
3. [https://www.youtube.com/watch?v=dzkVfaKChSU&feature=youtu.be&t=35s youtube.com TDD: RSpec Test Structure] <br>
4. [https://github.com/expertiza/expertiza/blob/3ce553a2d0258ea05bced910abae5d209a7f55d6/spec/models/response_spec.rb github/expertiza response_spec example] <br>
5. [https://relishapp.com/rspec RSpec Documentation] <br>

File:ONTHEFLYTESTRESULTS.png

2018-04-03T00:16:20Z

Adsimps3: uploaded a new version of "File:ONTHEFLYTESTRESULTS.png"