CSC/ECE 517 Fall 2014/ch1b 27 js: Difference between revisions

From Expertiza_Wiki
Jump to navigation Jump to search
 
(10 intermediate revisions by 2 users not shown)
Line 15: Line 15:
With the majority of software applications today being implemented as [http://en.wikipedia.org/wiki/Web_application web applications] to be used over the internet, testing is becoming an indispensable part of the software development lifecycle ([http://en.wikipedia.org/wiki/Software_development_process SDLC]) of these applications. [http://en.wikipedia.org/wiki/Test_automation Test automation] is used where manual testing needs to be carried out repeatedly making it a cumbersome task. Test automation uses a software tool for this purpose on the application to be tested. It has proven to be highly efficient and critical in larger software development companies. Additionally, automation can be a valuable way to make Test Driven Development ([http://en.wikipedia.org/wiki/Test-driven_development TDD]) tasks an important part of the software development workflow.
With the majority of software applications today being implemented as [http://en.wikipedia.org/wiki/Web_application web applications] to be used over the internet, testing is becoming an indispensable part of the software development lifecycle ([http://en.wikipedia.org/wiki/Software_development_process SDLC]) of these applications. [http://en.wikipedia.org/wiki/Test_automation Test automation] is used where manual testing needs to be carried out repeatedly making it a cumbersome task. Test automation uses a software tool for this purpose on the application to be tested. It has proven to be highly efficient and critical in larger software development companies. Additionally, automation can be a valuable way to make Test Driven Development ([http://en.wikipedia.org/wiki/Test-driven_development TDD]) tasks an important part of the software development workflow.
   
   
Test Automation has a number of advantages mostly dealing with repetitiveness of the tests and the time taken,
Test Automation has a number of advantages mostly dealing with repetitiveness of the tests and the time taken.
* '''Covers cases that manual testing probably cannot'''
* '''Covers cases that manual testing probably cannot'''
# Finds defects missed out my manual testing
# Finds defects missed out my manual testing
Line 25: Line 25:
# Automated tests perform the exact same steps and records all details precisely
# Automated tests perform the exact same steps and records all details precisely
# They also increase the depth and scope of tests to improve the quality of the software
# They also increase the depth and scope of tests to improve the quality of the software


=='''Brief History'''==
=='''Brief History'''==
Line 36: Line 34:
The year 2006 marked the beginning of the development of the 'WebDriver project' which was what they believed ‘a tool of the future’. The WebDriver was initiated to solve the problems that were faced due to the fact that Selenium was a based on JavaScript and a lot of security limitations browsers apply to Javascript.
The year 2006 marked the beginning of the development of the 'WebDriver project' which was what they believed ‘a tool of the future’. The WebDriver was initiated to solve the problems that were faced due to the fact that Selenium was a based on JavaScript and a lot of security limitations browsers apply to Javascript.
In 2008, both Selenium and WebDriver merged to provide a common set of features to all users which is called Selenium 2.
In 2008, both Selenium and WebDriver merged to provide a common set of features to all users which is called Selenium 2.
Simon Stewart, the creator of the WebDriver Project, in a joint email to the WebDriver and Selenium community on August 6, 2009 said the following,
Simon Stewart, the creator of the WebDriver Project, in a joint email to the WebDriver and Selenium community on August 6, 2009 said the following.


“Why are the projects merging? Partly because webdriver addresses some shortcomings in selenium (by being able to bypass the JS sandbox, for example. And we’ve got a gorgeous API), partly because selenium addresses some shortcomings in webdriver (such as supporting a broader range of browsers) and partly because the main selenium contributors and I felt that it was the best way to offer users the best possible framework.”<ref name ="Introduction to Selenium">[http://docs.seleniumhq.org/docs/01_introducing_selenium.jsp#selenium-history Introducing Selenium]. ''SeleniumHQ: Browser Automation''. Selenium Project, 4 Oct. 2014. Web. 06 Oct. 2014.</ref>
:“Why are the projects merging? Partly because webdriver addresses some shortcomings in selenium (by being able to bypass the JS sandbox, for example. And we’ve got a gorgeous API), partly because selenium addresses some shortcomings in webdriver (such as supporting a broader range of browsers) and partly because the main selenium contributors and I felt that it was the best way to offer users the best possible framework.”<ref name ="Introduction to Selenium"> [http://docs.seleniumhq.org/docs/01_introducing_selenium.jsp#selenium-history Introducing Selenium]. ''SeleniumHQ: Browser Automation''. Selenium Project, 4 Oct. 2014. Web. 06 Oct. 2014.</ref>


====Watir====
====Watir====


''Watir'' started as a library to teach testers to write ruby code and grew to be an IE implementation which now supports IE, Firefox, Safari, Google Chrome and Opera.
''Watir'' started as a library to teach testers to write ruby code and grew to be an IE implementation which now supports IE, Firefox, Safari, Google Chrome and Opera.
Watir was primarily developed by Bret Pettichord and Paul Rogers. Watir has since then has a couple of stable releases for Watir classic as well as Watir WebDriver. The most recent releases being<ref>[http://watir.com/category/releases/ Watir Releases]. Watir: Web Application Testing in Ruby. 28 April, 2013</ref>,
Watir was primarily developed by Bret Pettichord and Paul Rogers. Watir has since then has a couple of stable releases for Watir classic as well as Watir WebDriver. The most recent releases being<ref>[http://watir.com/category/releases/ Watir Releases]. Watir: Web Application Testing in Ruby. 28 April, 2013</ref>.


[[File: Watir.gif|right|Watir logo|link=http://upload.wikimedia.org/wikipedia/commons/7/7e/Watir.gif]]
[[File: Watir.gif|right|Watir logo|link=http://upload.wikimedia.org/wikipedia/commons/7/7e/Watir.gif]]
Line 55: Line 53:
Watir has a google group ([https://groups.google.com/forum/#!forum/watir-general Watir General]) for its support and contribution while Selenium also has a [http://seleniumforum.forumotion.net/ Selenium forum] apart from its google group called [https://groups.google.com/forum/#!forum/selenium-users Selenium Users].
Watir has a google group ([https://groups.google.com/forum/#!forum/watir-general Watir General]) for its support and contribution while Selenium also has a [http://seleniumforum.forumotion.net/ Selenium forum] apart from its google group called [https://groups.google.com/forum/#!forum/selenium-users Selenium Users].


=='''Pros/Cons'''==
===Selenium===
*Pros
# Open Source, free Software
# Easy Installation
# Scripting Techniques
##Easy recording and playback of scripts
##Scripts are modular, can be reused
# Selenium was first written in Java but it also supports .Net, Ruby, Perl, PHP and Python. This is a big plus when you want to build your framework in a language that has the highest adoption in the organization it is being built within. Unlike tools like [http://en.wikipedia.org/wiki/HP_QuickTest_Professional QTP] which force you to use [http://en.wikipedia.org/wiki/VBScript VBScript]
# Integration with third party tools is possible
# You can instantiate several concurrent tests with Selenium since no dedicated machine is required
*Cons
#  Selenium has concept of locators that support common attributes like id, names etc as well as [http://en.wikipedia.org/wiki/XPath XPATH], [http://en.wikipedia.org/wiki/JavaScript javascript] DOM and others. However, this is still not as smart as say the [http://www.tutorialspoint.com/qtp/qtp_object_repository.htm Object Repository] (OR) in QTP. That being said, QTP’s OR comes with it´s baggage as well.<ref>[http://www.qualitytesting.info/forum/topics/selenium-automation-tool Selenium Automation Tool]</ref>
# It does not generate detailed results like the result viewer
# No option to verify images
===Watir===
*Pros
# Open Source
# Opens browser as a normal way how the user opens it unlike Selenium which needs to create an object to work with
# Changing registry details(Proxy-http,socks) is very easy <ref>Raveendran, 28 Oct., 2009 [http://raveendran.wordpress.com/2009/10/28/watir-vs-selenium/ Selenium Vs Watir ]</ref>


*Cons
# Knowledge in the Ruby programming language is required
# Does not support certain HTML elements
# Limited reporting capabilities
# There is no recorder for Watir unlike Selenium for playback


=='''Comparison of Features, Functionality, and More'''==
=='''Comparison of Features, Functionality, and More'''==
Line 108: Line 133:
*Example
*Example
This is how a typical Selenium code to find the term say, “Automation” on Google and display the results on the page would look like in Java using the [https://code.google.com/p/selenium/wiki/HtmlUnitDriver HtmlUnitDriver] would look like.
This is how a typical Selenium code to find the term say, “Automation” on Google and display the results on the page would look like in Java using the [https://code.google.com/p/selenium/wiki/HtmlUnitDriver HtmlUnitDriver] would look like.
The HtmlUnitDriver is a pure Java driver that runs entirely in-memory. Because of this, you won't see a new browser window open <ref>[https://code.google.com/p/selenium/wiki/GettingStarted Getting started with Selenium]. Selenium: Browser Automation Framework. Google Code. 8 Feb, 2014.</ref>,
The HtmlUnitDriver is a pure Java driver that runs entirely in-memory. Because of this, you won't see a new browser window open <ref>[https://code.google.com/p/selenium/wiki/GettingStarted Getting started with Selenium] Selenium: Browser Automation Framework. Google Code. 8 Feb, 2014.</ref>.


       package org.selenium.example;<br /><br />
       package org.selenium.example;<br /><br />
Line 157: Line 182:
*Example:
*Example:
This is how a typical Watir code to find the term say, “Automation” on Google and display the results on the page would look like using Ruby<ref>Chernenko, Iryna. [http://blog.qatestlab.com/2011/02/23/automated-testing-with-watir-tutorial-for-begginers-part-1/ Watir Example]. QATestLab Blog: Automated Testing with Watir, 23 Feb 2011.
This is how a typical Watir code to find the term say, “Automation” on Google and display the results on the page would look like using Ruby<ref>Chernenko, Iryna. [http://blog.qatestlab.com/2011/02/23/automated-testing-with-watir-tutorial-for-begginers-part-1/ Watir Example]. QATestLab Blog: Automated Testing with Watir, 23 Feb 2011.
</ref>
</ref>.


         # Required in ruby 1.8.7
         # Required in ruby 1.8.7
Line 185: Line 210:
         ie.close
         ie.close


On compiling and running this and passing the test, you will see something like,
On compiling and running this and passing the test, you will see something like


         ''Output:''
         ''Output:''

Latest revision as of 02:34, 14 October 2014

Selenium vs. Watir

Developing web applications is proving to be a challenging task with each passing day given the complex requirements and the demanding nature of the these applications. For the web applications to be robust we need to put in place some good testing strategies.

Selenium is one of the most popularly used open source automation tools for web based applications. It is a JavaScript framework that runs in your web browser to interact with various elements in your browser. All in all, Selenium is a set of different software tools each with a different approach to supporting test automation. Selenium provides a rich set of testing functions that can be customized to suit the testing requirements.

Watir, is another lightweight automation testing tool that is widely used for web app testing. It is an open-source (BSD) family of Ruby libraries for automating web browsers. It allows you to write tests that are easy to read and maintain<ref>Web Application Testing in Ruby</ref>. Despite the fact that it is a set of Ruby libraries, it works perfectly, independent of the language that the web application is developed in. Watir as a tool is extremely simple and flexible.



Motivation

Need for test automation

With the majority of software applications today being implemented as web applications to be used over the internet, testing is becoming an indispensable part of the software development lifecycle (SDLC) of these applications. Test automation is used where manual testing needs to be carried out repeatedly making it a cumbersome task. Test automation uses a software tool for this purpose on the application to be tested. It has proven to be highly efficient and critical in larger software development companies. Additionally, automation can be a valuable way to make Test Driven Development (TDD) tasks an important part of the software development workflow.

Test Automation has a number of advantages mostly dealing with repetitiveness of the tests and the time taken.

  • Covers cases that manual testing probably cannot
  1. Finds defects missed out my manual testing
  2. Automated testing can simulate numerous virtual users interacting with the web software/application
  • Helps save time and money
  1. Once created, automated tests can be executed repeatedly at no additional costs and at a much faster pace. Saving time directly translates into saving cost.
  2. Supports virtually unlimited iterations of test case execution
  • Increase test coverage and accuracy
  1. Automated tests perform the exact same steps and records all details precisely
  2. They also increase the depth and scope of tests to improve the quality of the software

Brief History

Selenium

Selenium Logo
Selenium Logo

Selenium was developed in the year 2004 by Jason Huggins and has been continuously evolving ever since. Jason Huggins developed this as an internal tool for ThoughtWorks as a javascript library which came to be known as the Selenium core. Selenium core is now the basis of the Selenium Remote Control (the server) and Selenium IDE. The year 2006 marked the beginning of the development of the 'WebDriver project' which was what they believed ‘a tool of the future’. The WebDriver was initiated to solve the problems that were faced due to the fact that Selenium was a based on JavaScript and a lot of security limitations browsers apply to Javascript. In 2008, both Selenium and WebDriver merged to provide a common set of features to all users which is called Selenium 2. Simon Stewart, the creator of the WebDriver Project, in a joint email to the WebDriver and Selenium community on August 6, 2009 said the following.

“Why are the projects merging? Partly because webdriver addresses some shortcomings in selenium (by being able to bypass the JS sandbox, for example. And we’ve got a gorgeous API), partly because selenium addresses some shortcomings in webdriver (such as supporting a broader range of browsers) and partly because the main selenium contributors and I felt that it was the best way to offer users the best possible framework.”<ref name ="Introduction to Selenium"> Introducing Selenium. SeleniumHQ: Browser Automation. Selenium Project, 4 Oct. 2014. Web. 06 Oct. 2014.</ref>

Watir

Watir started as a library to teach testers to write ruby code and grew to be an IE implementation which now supports IE, Firefox, Safari, Google Chrome and Opera. Watir was primarily developed by Bret Pettichord and Paul Rogers. Watir has since then has a couple of stable releases for Watir classic as well as Watir WebDriver. The most recent releases being<ref>Watir Releases. Watir: Web Application Testing in Ruby. 28 April, 2013</ref>.

Watir logo
Watir logo
  • Watir-Classic
  1. 0.6.3 April 10, 2013 (Release Notes)
  • Watir-WebDriver
  1. 3.6.0 March 16, 2013 (Release Notes)
  2. 3.5.0 March 10, 2013 (Release Notes)

Watir has a google group (Watir General) for its support and contribution while Selenium also has a Selenium forum apart from its google group called Selenium Users.

Pros/Cons

Selenium

  • Pros
  1. Open Source, free Software
  2. Easy Installation
  3. Scripting Techniques
    1. Easy recording and playback of scripts
    2. Scripts are modular, can be reused
  4. Selenium was first written in Java but it also supports .Net, Ruby, Perl, PHP and Python. This is a big plus when you want to build your framework in a language that has the highest adoption in the organization it is being built within. Unlike tools like QTP which force you to use VBScript
  5. Integration with third party tools is possible
  6. You can instantiate several concurrent tests with Selenium since no dedicated machine is required
  • Cons
  1. Selenium has concept of locators that support common attributes like id, names etc as well as XPATH, javascript DOM and others. However, this is still not as smart as say the Object Repository (OR) in QTP. That being said, QTP’s OR comes with it´s baggage as well.<ref>Selenium Automation Tool</ref>
  2. It does not generate detailed results like the result viewer
  3. No option to verify images

Watir

  • Pros
  1. Open Source
  2. Opens browser as a normal way how the user opens it unlike Selenium which needs to create an object to work with
  3. Changing registry details(Proxy-http,socks) is very easy <ref>Raveendran, 28 Oct., 2009 Selenium Vs Watir </ref>
  • Cons
  1. Knowledge in the Ruby programming language is required
  2. Does not support certain HTML elements
  3. Limited reporting capabilities
  4. There is no recorder for Watir unlike Selenium for playback

Comparison of Features, Functionality, and More

Some features or functionality may be repeated here. However, we will go into a bit more detail than previously mentioned.

Open Source

Both Selenium and Watir are open source! This allows for greater extensibility and a deeper understanding of the inner workings of both of the frameworks, if the developer/tester chooses to delve into such details. Additionally, the open source community allows for a more transparent, user-driven development and support cycle. Users (the tester/developer) can choose to submit bug fixes or enhancements as patches, or they can choose to just use it “straight out of the box.”

Application Language Agnostic

Both Selenium and Watir are application language agnostic. Because both Selenium and Watir test UI functionality in the browser, they can be used to test any web application.

Language-specific Test Cases

Both Selenium and Watir require test cases to be written in some programming language. Traditionally, both frameworks had limited choices in terms of language bindings. However, with the advent of WebDriver, both became much more flexible. WebDriver provided the bindings needed for many languages. Today, the following are supported:

Selenium*

*According to SeleniumHQ’s download page.

Watir

Unfortunately (or fortunately), Ruby is the only supported language for Watir. After all, Watir does stand for “Web Application Testing in Ruby.” However, there are other frameworks which were inspired by Watir (but entirely separate) available in Java and C# - Watij and Watin, respectively <ref>Djorgjievski, Darko. Watir-WebDriver: Control the Browser. SitePoint. SitePoint Pty Ltd, 18 Apr. 2014. Web. 05 Oct. 2014.</ref>. There may be other similarly inspired frameworks available in other languages, but these are the most notable.

Cross-Browser and Headless Browser Support

Watir Meets WebDriver <ref>Watir Meets WebDriver Image</ref>


Traditionally, both Selenium and Watir were limited in the number of browsers they supported. Watir originally supported Internet Explorer and headless browsers exclusively. Headless browsers were supported via the use of Celerity <ref>Scott, Alister. Watir, Selenium & WebDriver. Watirmelon. Watirmelon, 10 Apr. 2010. Web. 05 Oct. 2014.</ref>. Selenium started out as a Javascript library, so naturally it supported more browsers such as IE, Firefox, Safari, etc <ref name = "Introduction to Selenium" />. However, it did not support headless browsers. Today, browser support looks much better for both Selenium and Watir, largely due to the introduction of WebDriver. With WebDriver in existence, support for most browsers was added to Watir via the gem "watir-webdriver," and Celerity became obsolete because headless browser support was included in WebDriver. Similarly for Selenium, support for headless browsers was added, as well as making the API much simpler and more concise <ref>Selenium WebDriver. SeleniumHQ: Browser Automation. Selenium Project, n.d. Web. 06 Oct. 2014.</ref>. Both Selenium <ref>Semerau, Luke. WebDriverForMobileBrowsers. Selenium. Google Project Hosting, 14 Jan. 2013. Web. 06 Oct. 2014 </ref> and Watir <ref>Scott, Alister. Mobile Devices. Watir WebDriver. n.d. Web. 06 Oct. 2014. </ref> have also gained support of Mobile browsers in recent years through WebDriver. However, there is the obvious limitation that one can only test a browser which is supported on the running platform. In other words, Safari and Internet explorer can only be tested on OS X and Windows environments, respectively**.

** Some work-arounds or programs may exist to allow these browsers to be tested on other platforms. Here, we are assuming no special additional configuration - just the base operating system and browsers. With virtualization technologies such as KVM, VirtualBox, and others, this isn't a problem anyway. Virtual machines can be created with any operating system, as needed.

Cross-Platform Support

Both Watir and Selenium run on most popular distributions of Windows, Mac, and Linux. Selenium doesn't have a list of officially supported operating system versions. Instead they generally support any version that is currently popular. Only a few versions are listed which they run test builds on. These are versions which are officially known to work, but many more are supported as well.

Because Watir is a collection of Ruby Gems, it runs on any platform that Ruby runs on. Like Selenium, this includes nearly all popular platforms.



Getting started

Selenium WebDriver


Install/Configure (Java)

Download binaries from here to configure Selenium. Once you have all your binaries in a directory, you can open any IDE of your choice and add this directory with all the .jar files to the CLASSPATH

  • Example

This is how a typical Selenium code to find the term say, “Automation” on Google and display the results on the page would look like in Java using the HtmlUnitDriver would look like. The HtmlUnitDriver is a pure Java driver that runs entirely in-memory. Because of this, you won't see a new browser window open <ref>Getting started with Selenium Selenium: Browser Automation Framework. Google Code. 8 Feb, 2014.</ref>.

      package org.selenium.example;

import org.openqa.selenium.By; import org.openqa.selenium.WebDriver; import org.openqa.selenium.WebElement; import org.openqa.selenium.htmlunit.HtmlUnitDriver;
public class Example { public static void main(String[] args) { // Create a new instance of the html unit driver // Notice that the remainder of the code relies on the interface, // not the implementation. WebDriver driver = new HtmlUnitDriver();
System.out.println("Beginning test for Google search using Selenium"); // And now use this to visit Google System.out.println("Visit www.google.com" ); driver.get("http://www.google.com");
// Find the text input element by its name WebElement element = driver.findElement(By.name("q"));
// Enter something to search for System.out.println("Type Automation in the search bar" ); element.sendKeys("Automation!");
// Now submit the form. WebDriver will find the form for us from the element element.submit();
// Check the title of the page System.out.println("Expected output" ); System.out.println("Page title is: " + driver.getTitle()); driver.quit(); } }

If you compile and run it, you should see a line with the title of the Google search results as output.

      Output:
       Beginning test for Google search using Selenium </ br>
       Visit www.google.com </ br>
       Expected output </ br>
       Page title is: Google

Install/Configure for other languages

Configuring Selenium for languages other than Java and for other development environments can be looked up here. viz(C#, Ruby, Python, PHP, Perl, Javascript)

Watir


Install/Configure (Ruby)

Drivers for Watir are installed as gems which are nothing but Ruby libraries which can be accessed over the internet. All the instructions for installing Watir on various operating systems (viz. Windows, Mac OS, Linux) can be found here.

  • Example:

This is how a typical Watir code to find the term say, “Automation” on Google and display the results on the page would look like using Ruby<ref>Chernenko, Iryna. Watir Example. QATestLab Blog: Automated Testing with Watir, 23 Feb 2011. </ref>.

       # Required in ruby 1.8.7
       require 'rubygems'
       # Watir IE driver
       require 'watir'
ie = Watir::IE.new
# text to show on console puts "Beginning of the test for Google search engine" puts "Step 1: Go to the Google homepage" ie.goto "http://www.google.com"
#set a variable search_text = "Automation" puts "Step 2: enter "+ search_text +" in the search text field." ie.text_field(:name, "q").set search_text # "q" is the name of the search field
puts " Step 3: click the 'Google Search' button." ie.button(:name, "btnG").click # "btnG" is the name of the Search button
puts " Expected Result:" puts " A Google page with results should be shown. 'Watir, pronounced water' should be high on the list." puts " Actual Result:" if ie.text.include? "Watir, pronounced water" puts " Test Passed. Found the text string: 'Watir, pronounced water'. Actual Results match Expected Results." else puts " Test Failed! Could not find text: 'Watir, pronounced water'." end puts "End of test: Google search." ie.close

On compiling and running this and passing the test, you will see something like

       Output:
       Beginning of the test for Google search engine 
       Step 1:  Go to the Google homepage 
       Step 2: enter Automation in the search text field. 
       Step 3: click the 'Google Search' button. 
       Expected Result: 
       A Google page with results should be shown. 'Watir, pronounced water' should be high on the list. 	
       Test Passed. Found the text string: 'Watir, pronounced water'. Actual Results match Expected Results. 
       End of test: Google search. 

Similar Tools

  • Sahi, uses the Sahi Scripting Language
  • SOATest, for testing and validating API driven applications
  • TestComplete, for Desktops, Web and Mobile


See Also

References

<references />


External Links