CSC/ECE 517 Fall 2014/ch1b 27 js

Selenium vs. Watir

Developing web applications is proving to be a challenging task with each passing day given the complex requirements and the demanding nature of the these applications. For the web applications to be robust we need to put in place some good testing strategies.

Selenium is one of the most popularly used open source automation tools for web based applications. It is a JavaScript framework that runs in your web browser to interact with various elements in your browser. All in all, Selenium is a set of different software tools each with a different approach to supporting test automation. Selenium provides a rich set of testing functions that can be customized to suit the testing requirements.

Watir, is another lightweight automation testing tool that is widely used for web app testing. It is an open-source (BSD) family of Ruby libraries for automating web browsers. It allows you to write tests that are easy to read and maintain<ref>Web Application Testing in Ruby</ref>. Despite the fact that it is a set of Ruby libraries, it works perfectly, independent of the language that the web application is developed in. Watir as a tool is extremely simple and flexible.

Motivation

Need for test automation

With the majority of software applications today being implemented as web applications to be used over the internet, testing is becoming an indispensable part of the software development lifecycle (SDLC) of these applications. Test automation is used where manual testing needs to be carried out repeatedly making it a cumbersome task. Test automation uses a software tool for this purpose on the application to be tested. It has proven to be highly efficient and critical in larger software development companies. Additionally, automation can be a valuable way to make Test Driven Development (TDD) tasks an important part of the software development workflow.

Test Automation has a number of advantages mostly dealing with repetitiveness of the tests and the time taken.

• Covers cases that manual testing probably cannot

1. Finds defects missed out my manual testing
2. Automated testing can simulate numerous virtual users interacting with the web software/application

• Helps save time and money

1. Once created, automated tests can be executed repeatedly at no additional costs and at a much faster pace. Saving time directly translates into saving cost.
2. Supports virtually unlimited iterations of test case execution

• Increase test coverage and accuracy

1. Automated tests perform the exact same steps and records all details precisely
2. They also increase the depth and scope of tests to improve the quality of the software

Brief History

Selenium

Selenium Logo

Selenium was developed in the year 2004 by Jason Huggins and has been continuously evolving ever since. Jason Huggins developed this as an internal tool for ThoughtWorks as a javascript library which came to be known as the Selenium core. Selenium core is now the basis of the Selenium Remote Control (the server) and Selenium IDE. The year 2006 marked the beginning of the development of the 'WebDriver project' which was what they believed ‘a tool of the future’. The WebDriver was initiated to solve the problems that were faced due to the fact that Selenium was a based on JavaScript and a lot of security limitations browsers apply to Javascript. In 2008, both Selenium and WebDriver merged to provide a common set of features to all users which is called Selenium 2. Simon Stewart, the creator of the WebDriver Project, in a joint email to the WebDriver and Selenium community on August 6, 2009 said the following.

“Why are the projects merging? Partly because webdriver addresses some shortcomings in selenium (by being able to bypass the JS sandbox, for example. And we’ve got a gorgeous API), partly because selenium addresses some shortcomings in webdriver (such as supporting a broader range of browsers) and partly because the main selenium contributors and I felt that it was the best way to offer users the best possible framework.”<ref name ="Introduction to Selenium"> Introducing Selenium. SeleniumHQ: Browser Automation. Selenium Project, 4 Oct. 2014. Web. 06 Oct. 2014.</ref>

Watir

Watir started as a library to teach testers to write ruby code and grew to be an IE implementation which now supports IE, Firefox, Safari, Google Chrome and Opera. Watir was primarily developed by Bret Pettichord and Paul Rogers. Watir has since then has a couple of stable releases for Watir classic as well as Watir WebDriver. The most recent releases being<ref>Watir Releases. Watir: Web Application Testing in Ruby. 28 April, 2013</ref>.

Watir logo

• Watir-Classic

1. 0.6.3 April 10, 2013 (Release Notes)
   

• Watir-WebDriver

1. 3.6.0 March 16, 2013 (Release Notes)
   
2. 3.5.0 March 10, 2013 (Release Notes)
   

Watir has a google group (Watir General) for its support and contribution while Selenium also has a Selenium forum apart from its google group called Selenium Users.

Pros/Cons

Selenium

• Pros

1. Open Source, free Software
2. Easy Installation
3. Scripting Techniques
   1. Easy recording and playback of scripts
   2. Scripts are modular, can be reused
4. Selenium was first written in Java but it also supports .Net, Ruby, Perl, PHP and Python. This is a big plus when you want to build your framework in a language that has the highest adoption in the organization it is being built within. Unlike tools like QTP which force you to use VBScript
5. Integration with third party tools is possible
6. You can instantiate several concurrent tests with Selenium since no dedicated machine is required

• Cons

1. Selenium has concept of locators that support common attributes like id, names etc as well as XPATH, javascript DOM and others. However, this is still not as smart as say the Object Repository (OR) in QTP. That being said, QTP’s OR comes with it´s baggage as well.<ref>Selenium Automation Tool</ref>
2. It does not generate detailed results like the result viewer
3. No option to verify images

Watir

• Pros

1. Open Source
2. Opens browser as a normal way how the user opens it unlike Selenium which needs to create an object to work with
3. Changing registry details(Proxy-http,socks) is very easy <ref>Raveendran, 28 Oct., 2009 Selenium Vs Watir </ref>

• Cons

1. Knowledge in the Ruby programming language is required
2. Does not support certain HTML elements
3. Limited reporting capabilities
4. There is no recorder for Watir unlike Selenium for playback

Comparison of Features, Functionality, and More

Some features or functionality may be repeated here. However, we will go into a bit more detail than previously mentioned.

Open Source

Both Selenium and Watir are open source! This allows for greater extensibility and a deeper understanding of the inner workings of both of the frameworks, if the developer/tester chooses to delve into such details. Additionally, the open source community allows for a more transparent, user-driven development and support cycle. Users (the tester/developer) can choose to submit bug fixes or enhancements as patches, or they can choose to just use it “straight out of the box.”

Application Language Agnostic

Both Selenium and Watir are application language agnostic. Because both Selenium and Watir test UI functionality in the browser, they can be used to test any web application.

Language-specific Test Cases

Both Selenium and Watir require test cases to be written in some programming language. Traditionally, both frameworks had limited choices in terms of language bindings. However, with the advent of WebDriver, both became much more flexible. WebDriver provided the bindings needed for many languages. Today, the following are supported:

Selenium*

• Officially-supported: Java, C#, Ruby, Python, Javascript/Node.js

• Unofficial/community-supported: Perl, PHP, Haskell, Objective-C, R, Dart, Tcl

*According to SeleniumHQ’s download page.

Watir

Unfortunately (or fortunately), Ruby is the only supported language for Watir. After all, Watir does stand for “Web Application Testing in Ruby.” However, there are other frameworks which were inspired by Watir (but entirely separate) available in Java and C# - Watij and Watin, respectively <ref>Djorgjievski, Darko. Watir-WebDriver: Control the Browser. SitePoint. SitePoint Pty Ltd, 18 Apr. 2014. Web. 05 Oct. 2014.</ref>. There may be other similarly inspired frameworks available in other languages, but these are the most notable.

Cross-Browser and Headless Browser Support

Watir Meets WebDriver <ref>Watir Meets WebDriver Image</ref>

Traditionally, both Selenium and Watir were limited in the number of browsers they supported. Watir originally supported Internet Explorer and headless browsers exclusively. Headless browsers were supported via the use of Celerity <ref>Scott, Alister. Watir, Selenium & WebDriver. Watirmelon. Watirmelon, 10 Apr. 2010. Web. 05 Oct. 2014.</ref>. Selenium started out as a Javascript library, so naturally it supported more browsers such as IE, Firefox, Safari, etc <ref name = "Introduction to Selenium" />. However, it did not support headless browsers. Today, browser support looks much better for both Selenium and Watir, largely due to the introduction of WebDriver. With WebDriver in existence, support for most browsers was added to Watir via the gem "watir-webdriver," and Celerity became obsolete because headless browser support was included in WebDriver. Similarly for Selenium, support for headless browsers was added, as well as making the API much simpler and more concise <ref>Selenium WebDriver. SeleniumHQ: Browser Automation. Selenium Project, n.d. Web. 06 Oct. 2014.</ref>. Both Selenium <ref>Semerau, Luke. WebDriverForMobileBrowsers. Selenium. Google Project Hosting, 14 Jan. 2013. Web. 06 Oct. 2014 </ref> and Watir <ref>Scott, Alister. Mobile Devices. Watir WebDriver. n.d. Web. 06 Oct. 2014. </ref> have also gained support of Mobile browsers in recent years through WebDriver. However, there is the obvious limitation that one can only test a browser which is supported on the running platform. In other words, Safari and Internet explorer can only be tested on OS X and Windows environments, respectively**.

** Some work-arounds or programs may exist to allow these browsers to be tested on other platforms. Here, we are assuming no special additional configuration - just the base operating system and browsers. With virtualization technologies such as KVM, VirtualBox, and others, this isn't a problem anyway. Virtual machines can be created with any operating system, as needed.

Cross-Platform Support

Both Watir and Selenium run on most popular distributions of Windows, Mac, and Linux. Selenium doesn't have a list of officially supported operating system versions. Instead they generally support any version that is currently popular. Only a few versions are listed which they run test builds on. These are versions which are officially known to work, but many more are supported as well.

Because Watir is a collection of Ruby Gems, it runs on any platform that Ruby runs on. Like Selenium, this includes nearly all popular platforms.

Getting started

Selenium WebDriver

---

Install/Configure (Java)

Download binaries from here to configure Selenium. Once you have all your binaries in a directory, you can open any IDE of your choice and add this directory with all the .jar files to the CLASSPATH

• Example

This is how a typical Selenium code to find the term say, “Automation” on Google and display the results on the page would look like in Java using the HtmlUnitDriver would look like. The HtmlUnitDriver is a pure Java driver that runs entirely in-memory. Because of this, you won't see a new browser window open <ref>Getting started with Selenium Selenium: Browser Automation Framework. Google Code. 8 Feb, 2014.</ref>.

      package org.selenium.example;


      import org.openqa.selenium.By;
      import org.openqa.selenium.WebDriver;
      import org.openqa.selenium.WebElement;
      import org.openqa.selenium.htmlunit.HtmlUnitDriver;

      public class Example  {
      public static void main(String[] args) {
       // Create a new instance of the html unit driver
       // Notice that the remainder of the code relies on the interface,
       // not the implementation.
       WebDriver driver = new HtmlUnitDriver();

       System.out.println("Beginning test for Google search using Selenium");
       // And now use this to visit Google
       System.out.println("Visit www.google.com" );
       driver.get("http://www.google.com");

       // Find the text input element by its name
       WebElement element = driver.findElement(By.name("q"));

       // Enter something to search for
       System.out.println("Type Automation in the search bar" );
       element.sendKeys("Automation!");

       // Now submit the form. WebDriver will find the form for us from the element
       element.submit();

       // Check the title of the page
       System.out.println("Expected output" );
       System.out.println("Page title is: " + driver.getTitle());
       driver.quit();
       }
   }

If you compile and run it, you should see a line with the title of the Google search results as output.

      Output:
       Beginning test for Google search using Selenium </ br>
       Visit www.google.com </ br>
       Expected output </ br>
       Page title is: Google

Install/Configure for other languages

Configuring Selenium for languages other than Java and for other development environments can be looked up here. viz(C#, Ruby, Python, PHP, Perl, Javascript)

Watir

---

Install/Configure (Ruby)

Drivers for Watir are installed as gems which are nothing but Ruby libraries which can be accessed over the internet. All the instructions for installing Watir on various operating systems (viz. Windows, Mac OS, Linux) can be found here.

• Example:

This is how a typical Watir code to find the term say, “Automation” on Google and display the results on the page would look like using Ruby<ref>Chernenko, Iryna. Watir Example. QATestLab Blog: Automated Testing with Watir, 23 Feb 2011. </ref>.

       # Required in ruby 1.8.7
       require 'rubygems'
       # Watir IE driver
       require 'watir'
 
       ie = Watir::IE.new
 
       # text to show on console
       puts "Beginning of the test for Google search engine"
       puts "Step 1:  Go to the Google homepage"
       ie.goto "http://www.google.com"

       #set a variable
       search_text = "Automation"
       puts "Step 2: enter "+ search_text +" in the search text field."
       ie.text_field(:name, "q").set search_text   # "q" is the name of the search field

       puts " Step 3: click the 'Google Search' button."
       ie.button(:name, "btnG").click # "btnG" is the name of the Search button

       puts " Expected Result:"
       puts "  A Google page with results should be shown. 'Watir, pronounced water' should be high on the list."
       puts " Actual Result:"
       if ie.text.include? "Watir, pronounced water"	
            puts "  Test Passed. Found the text string: 'Watir, pronounced water'. Actual Results match Expected Results."
       else	
	     puts "  Test Failed! Could not find text: 'Watir, pronounced water'."
       end
       puts "End of test: Google search."
       ie.close

On compiling and running this and passing the test, you will see something like

       Output:
       Beginning of the test for Google search engine 
       Step 1:  Go to the Google homepage 
       Step 2: enter Automation in the search text field. 
       Step 3: click the 'Google Search' button. 
       Expected Result: 
       A Google page with results should be shown. 'Watir, pronounced water' should be high on the list. 	
       Test Passed. Found the text string: 'Watir, pronounced water'. Actual Results match Expected Results. 
       End of test: Google search. 

Similar Tools

• Sahi, uses the Sahi Scripting Language
  
• SOATest, for testing and validating API driven applications
  
• TestComplete, for Desktops, Web and Mobile
  

See Also

• Migrating unit tests from Selenium to Watir Web Driver
  
• Learning Selenium IDE
  
• List of web testing tools
  
• OWASP List of Acceptance Testing tools
• Study of Browser Based Automated Test Tools WATIR and Selenium Nisha Gogna, August 2014, International Journal of Information and Education Technology, Vol. 4, No. 4

References

<references />

External Links

• Selenium Wiki Page
  
• Selenium Source Code
  
• Watir Wiki Page
  
• Watir Source Code
  
• SauceLabs HomePage