CSC/ECE 517 Fall 2009/wiki1a 1 JunitTest

Writing Effective JUnit Test Cases

---

JUnit is a testing framework aimed at making unit testing easy by simplifying the process of writing test cases and by providing automated running of tests and test suites.

The objective of this document is to provide a repository of rules that can be used in-order to write an effective Junit test. This doc also discusses various sites and suggest which are the best sites to read for an overview of the topic and to understand the different philosophies of test-case design.

Introduction

Unit Testing Overview

The primary goal of unit testing is to take the smallest piece of testable software in the application, isolate it from the remainder of the code, and determine whether it behaves exactly as you expect.

Test Case
It describes a scenario that can be exercised through the application.
Each test case is divided into two parts: input and expected output.
The input part lists all the test case statements that create variables or assign values to variables.
The expected output part indicates the expected results; it shows either assertions or the message 'no exception' (when no assertions exist).

The main issue in constructing a unit test is deciding the scope of test cases.
If the scope is too narrow, then the tests will be trivial and the objects might pass the tests, but there will be no design of their interactions and interactions of objects are important for object oriented design.
On the other hand if the scope is too broad, then there is a high chance that not every component of the new code will get tested.
The programmer is then reduced to testing-by-poking-around, which is not an effective test strategy.

JUnit Overview

Non-automated testing is usually like this:

• An expression is used in a debugger, you can change debug expressions without recompiling, and you can wait to decide what to write until you have seen the running objects.
  
• Expressions are tested as statements which print to the standard output stream.
  

But its not preferred because:

• They require human judgment to analyze their results.
  
• They are cumbersome, non repeatable - you can only execute one debug expression at a time and a program with too many print statements causes the dreaded "Scroll Blindness".
  

On the upper hand, JUnit tests, which uses the automated framework does not require human judgment to interpret, and it is easy to run many of them at the same time.
When you need to test something, here is what you do:

• Annotate a method with @org.junit.Test
  
• When you want to check a value, import org.junit.Assert.* statically, call assertTrue() and pass a boolean that is true if the test succeeds
  

So what exactly is JUnit ??
It is a open-source unit testing framework for Java and provides a way to organize test data and perform repeatable test execution.
In JUnit, one has to write Java code, called a test class, that describes test data, invokes the methods to be tested, and determines test results.
JUnit was originally written by Erich Gamma and Kent Beck.
The official JUnit home page is http://junit.org.
JUnit is Open Source Software, released under IBM's Common Public License Version 0.5 and hosted on SourceForge.
It is an instance of the xUnit architecture for unit testing frameworks. JUnit features include:

• Assertions for testing expected results
  
• Test fixtures for sharing common test data
  
• Test runners for running tests
  

JUnit mailing lists and forums
JUnit user list
JUnit announcements
JUnit developer list
JUnit official documentation
JUnit Test Infected: Programmers Love Writing Tests
JUnit Cookbook
JUnit - A Cook's Tour
JUnit FAQ
Although JUnit testing costs some time and money, but unit testing provides some crucial advantages, viz:
Automation of the testing process
Reduces difficulties of discovering errors contained in more complex pieces of the application
Test coverage is often enhanced because attention is given to each unit.
For example,
if you have two units and decide it would be more cost effective to glue them together and initially test them as an integrated unit, an error could occur in a variety of places, now:

• Is the error due to a defect in unit 1?
  
• Is the error due to a defect in unit 2?
  
• Is the error due to defects in both units?
  
• Is the error due to a defect in the interface between the units?
  
• Is the error due to a defect in the test?
  

Finding the error (or errors) in the integrated module is much more complicated than first isolating the units, testing each, then integrating them and testing the whole.

Websites to lookout

These are some of the best webpages which will help you in getting more description as well as details about Junit.
Also it will help you understand the test case design philosophy.

For Overview

• Junit testing basics:
  

http://articles.techrepublic.com.com/5100-10878_11-1027676.html

• Basics of Junit:
  

http://junit.sourceforge.net/doc/cookbook/cookbook.htm http://junit.sourceforge.net/doc/faq/faq.htm#overview

For Ideas on test case design

• For successful test automation:
  

http://www.io.com/~wazmo/papers/seven_steps.html

• Unit testing principles:
  

http://www.acm.org/ubiquity/views/t_burns_1.html

• For test suite effectiveness:
  

http://java.sys-con.com/node/299945

How to get started

Step 1: Installing JUnit

• Download latest version of JUnit from "http://sourceforge.net/projects/junit/files/".
  
• Then install JUnit on your platform of choice:
  

Windows
To install JUnit on Windows, follow these steps:

• Unzip the junit.zip distribution file to a directory referred to as %JUNIT_HOME%.
  
• Add JUnit to the classpath:
  

set CLASSPATH=%JUNIT_HOME%\junit.jar

Unix (bash)
To install JUnit on Unix, follow these steps:

• Unzip the junit.zip distribution file to a directory referred to as $JUNIT_HOME.
  
• Add JUnit to the classpath:
  

export CLASSPATH=$JUNIT_HOME/junit.jar

Step 2: Writing a Test Case

To write a test case, follow these steps:

• Implement a class derived from TestCase, a base class provided by JUnit.
  

   public class BooksCartTest extends TestCase

• Be sure the test class belongs to the same package as that of the unit you wish to test
  
• Override the setUp() method to initialize object(s) under test.
  
• Optionally override the tearDown() method to release object(s) under test.
  
• Define one or more public testXXX() methods that exercise the object(s) under test and assert expected results.
  

The following is an example test case:

import junit.framework.TestCase;

public class BooksCartTest extends TestCase {

    private BooksCart cart1;
    private BooksCart cart2;

    /**
     * Sets up the test fixture.
     *
     * Called before every test case method.
     */
    protected void setUp() {

        cart1 = new BooksCart(6,8);
        cart2 = new BooksCart(7,10);
    }

    /**
     * Tears down the test fixture.
     *
     * Called after every test case method.
     */
    protected void tearDown() {
        // release objects under test here, if necessary
    }

    /**
     * Tests emptying the cart.
     */
    public void testEmpty() {

        cart1.empty();
        cart2.empty();
        assertEquals(cart1, new BooksCart(3, 9));
        assertTrue(!cart2.equals(cart1));
    }
   }

Methods used in JUnit:

• An assert method is a JUnit method that performs a test, and throws an AssertionFailedError if the test fails
  
• static void assertTrue(boolean test)
  

  static void assertTrue(String message, boolean test)

- Throws an AssertionFailedError if the test fails
- The optional message is included in the Error

• static void assertFalse(boolean test)
  

  static void assertFalse(String message, boolean test)

- Throws an AssertionFailedError if the test fails

Step 3: Writing a Test Suite

Next, we'll write a test suite that includes several test cases. The test suite will allow us to run all of its test cases in one fell swoop.
To write a test suite, follow these steps:

• Write a Java class that defines a static suite() factory method that creates a TestSuite containing all the tests.
  
• Optionally define a main() method that runs the TestSuite in batch mode.
  

The following is an example test suite:

import junit.framework.Test;
import junit.framework.TestSuite;

public class BooksCartTestSuite {
  
    public static Test suite() {

        TestSuite suite = new TestSuite();
  
        //
        // The BooksCartTest we created above.
        //
        suite.addTestSuite(BooksCartTest.class);

        //
        // Another example test suite of tests.
        // 
        suite.addTest(CreditCardTestSuite.suite());

        //
        // Add more tests here
        //

        return suite;
    }

    /**
     * Runs the test suite using the textual runner.
     */
    public static void main(String[] args) {
        junit.textui.TestRunner.run(suite());
    }
}

Step 4: Running the Test

Now that we've written a test suite containing a collection of test cases and other test suites, we can run either the test suite or any of its test cases individually. Running a TestSuite will automatically run all of its subordinate TestCase instances and TestSuite instances. Running a TestCase will automatically invoke all of its public testXXX() methods. JUnit provides both a textual and a graphical user interface. Both user interfaces indicate how many tests were run, any errors or failures, and a simple completion status. The simplicity of the user interfaces is the key to running tests quickly. You should be able to run your tests and know the test status with a glance, much like you do with a compiler. To run our test case using the textual user interface, use:

java junit.textui.TestRunner BooksCartTest 

The textual user interface displays "OK" if all the tests passed and failure messages if any of the tests failed.
To run the test case using the graphical user interface, use:

java junit.swingui.TestRunner BooksCartTest 

The graphical user interface displays a Swing window with a green progress bar if all the tests passed or a red progress bar if any of the tests failed.
The BooksCartTest Suite can be run similarly:

java junit.swingui.TestRunner BooksCartTestSuite

Rules for designing effective JUnit Test cases

• Identify and assemble a list of all the actions that your program should be able to perform. use cases,nonfunctional requirements specifications, test case specifications, user interface design documents, mockups, user profiles, and various additional artifacts.
• Identify the code's entry points - central pieces of code that exercise the functionality that the code as a whole is designed to undertake.
• Pair entry points with the use cases that they implement.
• Create test cases by applying the initialize/work/check procedure.
  • Initialize: Create the environment that the test expects to run in. The initialization code can either be in the beginning of the test or in the setUp() method.
  • Work: Call the code that is being tested, capturing any interesting output and recording any interesting statistics.
  • Check: Use assert statements to ensure that the code worked as expected.
• Develop runtime event diagrams and use them to facilitate testing.
• Follow KISS rule:It's a bad idea to keep the input source name and expression in the test case. Some items (server names, usernames and passwords, etc.) should not live in the test files, which should be configurable for the specific deployment. Rather, design the test cases to facilitate separation of test drivers and test data, test driver reuse over a larger set of data, and test data reuse over a larger set of test drivers. On the other hand, don't over-architect a simple test case implementation. Typically, the test case specification already defines most of the system states and allows parameter descriptions for a scenario, so there's no point in making everything parameterizable in the test implementation.

Rules for coding effective JUnit Tests

Using Test Fixture for a common set of objects

Using a test fixture avoids duplicating the code necessary to initialize (and cleanup) the common objects. Tests can use the objects (variables) in a test fixture, with each test invoking different methods on objects in the fixture and asserting different expected results. Each test runs in its own test fixture to isolate tests from the changes made by other tests. That is, tests don't share the state of objects in the test fixture. Because the tests are isolated, they can be run in any order. To create a test fixture, declare instance variables for the common objects. Initialize these objects in a public void method annotated with @Before. The JUnit framework automatically invokes any @Before methods before each test is run. The following example shows a test fixture with a common Collection object.

package junitfaq;

import org.junit.*;
import static org.junit.Assert.*;
import java.util.*;

public class SimpleTest {

    private Collection<Object> collection;

    @Before
    public void setUp() {
    collection = new ArrayList<Object>();
    }

    @Test
    public void testEmptyCollection() {
    assertTrue(collection.isEmpty());
    }

    @Test
    public void testOneItemCollection() {
    collection.add("itemA");
    assertEquals(1, collection.size());
    }
}

Given this test, the methods might execute in the following order:
setUp()
testEmptyCollection()
setUp()
testOneItemCollection()
The ordering of test-method invocations is not guaranteed, so testOneItemCollection() might be executed before testEmptyCollection(). But it doesn't matter, because each method gets its own instance of the collection. Although JUnit provides a new instance of the fixture objects for each test method, if you allocate any external resources in a @Before method, you should release them after the test runs by annotating a method with @After. The JUnit framework automatically invokes any @After methods after each test is run. For example:

package junitfaq;

import org.junit.*;
import static org.junit.Assert.*;
import java.io.*;

public class OutputTest {

    private File output;

    @Before
    public void createOutputFile() {
    output = new File(...);
    }

    @After
    public void deleteOutputFile() {
    output.delete();
    }

    @Test
    public void testSomethingWithFile() {
    ...
    }
}

With this test, the methods will execute in the following order:

• createOutputFile()
• testSomethingWithFile()
• deleteOutputFile()

To Test a method that doesnot return anything

If a method doesn't return a value, it might have some side effect. Actually, if it doesn't return a value AND doesn't have a side effect, it isn't doing anything. There may be a way to verify that the side effect actually occurred as expected. For example, consider the add() method in the Collection classes. There are ways of verifying that the side effect happened (i.e. the object was added). You can check the size and assert that it is what is expected:

    public void testCollectionAdd() {
        Collection collection = new ArrayList();
        assertEquals(0, collection.size());
        collection.add("itemA");
        assertEquals(1, collection.size());
        collection.add("itemB");
        assertEquals(2, collection.size());
    }

To test an expected/unexpected exception

• Case of expected exception

Add the optional expected attribute to the @Test annotation. The following is an example test that passes when the expected IndexOutOfBoundsException is raised:

@Test(expected=IndexOutOfBoundsException.class)
public void testIndexOutOfBoundsException() {
    ArrayList emptyList = new ArrayList();
    Object o = emptyList.get(0);
}

• Case of unexpected exception

Declare the exception in the throws clause of the test method and don't catch the exception within the test method. Uncaught exceptions will cause the test to fail with an error. The following is an example test that fails when the IndexOutOfBoundsException is raised:

@Test
public void testIndexOutOfBoundsExceptionNotRaised() 
throws IndexOutOfBoundsException {

    ArrayList emptyList = new ArrayList();
    Object o = emptyList.get(0);
}

To Test protected/private methods

• Case for protected methods

Place your tests in the same package as the classes under test.

• Case for private methods

Testing private methods may be an indication that those methods should be moved into another class to promote reusability.

Conditions to test get() and set() methods

Unit tests are intended to alleviate fear that something might break. If you think a get() or set() method could reasonably break, or has in fact contributed to a defect, then by all means write a test. In short, test until you're confident. What you choose to test is subjective, based on your experiences and confidence level. Remember to be practical and maximize your testing investment.

To run setUp() and tearDown() code once for all tests

You can add a @BeforeClass annotation to a method to be run before all the tests in a class, and a @AfterClass annotation to a method to be run after all the tests in a class. Here's an example:

    package junitfaq;

    import org.junit.*;
    import static org.junit.Assert.*;
    import java.util.*;
    
    public class SimpleTest {
    
        private Collection collection;
	
        @BeforeClass
        public static void oneTimeSetUp() {
            // one-time initialization code        
        }

        @AfterClass
        public static void oneTimeTearDown() {
            // one-time cleanup code
        }

        @Before
        public void setUp() {
            collection = new ArrayList();
        }
	
        @After
        public void tearDown() {
            collection.clear();
        }

        @Test
        public void testEmptyCollection() {
            assertTrue(collection.isEmpty());
        }
	
        @Test
        public void testOneItemCollection() {
            collection.add("itemA");
            assertEquals(1, collection.size());
        }
    }

Given this test, the methods will execute in the following order:

• oneTimeSetUp()
• setUp()
• testEmptyCollection()
• tearDown()
• setUp()
• testOneItemCollection()
• tearDown()
• oneTimeTearDown()

Conclusion

• By following the guidelines given in this articles, one can write a good unit test using junit, which is as important as writing good quality code.
  
• A good quality unit test will take care of all classes, boundary conditions, domain data and functionality.
  
• A good test can be creative to explore all possibilities, thinking out of box.
  
• Finally automating the test using tool like junit which is in the native java language itself helps in saving manual intervention and time/money.
  
  

References

• http://articles.techrepublic.com.com/5100-10878_11-1027676.html
• http://msdn.microsoft.com/en-us/library/aa292197%28VS.71%29.asp
• http://junit.sourceforge.net/doc/cookbook/cookbook.htm
• http://junit.sourceforge.net/doc/faq/faq.htm#overview
• http://clarkware.com/articles/JUnitPrimer.html#usage
• http://users.csc.calpoly.edu/~jdalbey/205/Resources/junitQuickStart.html
• http://www.cis.upenn.edu/~matuszek/cit591-2004/Lectures/
• http://en.wikipedia.org/wiki/JUnit
• http://www.laliluna.de/eclipse-junit-testing-tutorial.html
• http://junit.sourceforge.net/doc/faq/faq.htm

---