CSC/ECE 517 Fall 2009/wiki1a 2 sn

Comparison of Mock Frameworks

When performing tests, it's convenient to have the objects be in a particular configuration, so that boundary cases can be tested. Setting up the objects can sometimes be complicated. There are different Mock Frameworks out in the development community that can be used to develop these tests. This page attempts to define and compare these different types of frameworks.

Definition

In object-oriented programming, mock objects are simulated objects that mimic the behavior of real objects in controlled ways. A computer programmer typically creates a mock object to test the behavior of some other object, in much the same way that a car designer uses a crash test dummy to simulate the dynamic behavior of a human in vehicle impacts. 3

Mock frameworks are the environments that are available to create these mock objects and mock tests. There are many different types of frameworks for different platforms. They are a set of programmable APIs that allow creation of mock and stub objects in relative easy fashion. Mock frameworks save the developer from the need to write repetitive code to test or simulate object interactions.

Concepts

Why Mock?

• The concept behind mock objects is that there is a need to create an object that will take the place of the real object. This mock object will expect a certain method to be called with certain parameters and when that happens, it will return an expected result. When writing units tests for a class that would normally use the real object, we can instead supply it with a mock object. This allows us a new level of flexibility in testing.1

• Ease of modifying the mock object during separate tests to get it to get it to return a range of different data. Test can pass in valid, invalid, and extreme ranges to test how the code calling it handles such situations.1

• Simulation of failures, such as the inability to connect to a database, to test the failure mode of classes.

• Encourages better structured tests and, more importantly, improved domain code by preserving encapsulation, reducing dependencies and clarifying the interactions between classes. A running Object-Oriented program is a web of objects that collaborate by calling methods on each other. There sometimes can be many dependencies that have to be met in order to call certain objects. To make the complex dependencies be easily testable, mock objects are created in order to “mock” the stages that the object needs to be in, in order to call upon another object.

• Often times in programming, only the changes in state of an object are tested. This would be acceptable only if there was one object. However, many real world applications contain hundreds upon hundreds of different objects. Mock objects have changed the focus of test from thinking about the changes in state of an object to thinking about its interactions with other objects.2

Different Types

Proxy

A proxy object is an object that is used to take the place of a real object. This is the original concept for all mock frameworks. In the case of mock objects, a proxy object is used to imitate the real object your code is dependent on. The proxy object is created with the mocking framework, and then set it on the object using either a setter or constructor. One has to be able to set the dependency up through an external means. This is one of the reasons Dependency Injection frameworks like Java Spring have increase in popularity because they allow the ability to inject the proxy objects without modifying code. Proxy

Class Remapping

The second form of mocking is to remap the class file in the class loader. The concept is relatively new and is provided by the new java.lang.Instrument class. The basic idea is that the framework tells the class loader to remap the reference to another class file it will load which will be the mocked class. This allows one to be able to mock objects that are created by using the new operator much like other Object Oriented languages. Although this approach provides more functionality than the proxy object approach, it is much more complex and harder to write and fully understand. Class Mapping

Examples of Mocking

Lets say you have a an interface that looks like this:

 public interface FileInterface 
 {
   public void openFile(String filename) ;
   public void closeFile() ;
   public String readFile() ;    
 }

Then you have a class that looks like this:

 public class FileManipulator implements FileInterface 
 {
   private BufferedReader in;
   private boolean error = false; 
   public void openFile(String filename) 
   {
     try 
     {
       in = new BufferedReader(new FileReader(filename));
     } catch (FileNotFoundException e) {}
   }
   public void closeFile() 
   {
     try 
     {
       in.close();
     } catch (IOException e) {}
   }
   public String readFile() 
   {
     String s;
     try 
     {
       s = in.readLine();
     } catch (IOException e) {}
   
    return s;
   }
 }

The Mock File would look something like this:

 public class MockFileManipulator implements FileInterface 
 {
   private boolean initialized = false; 
   private boolean error = false; 
   private String filename;
   public void openFile(String filename) 
   {
     if (filename == null || filename.equals("")) 
     {
       Assert.fail("Name of file is either null or blank");
     }  
     this.filename = filename;
     initialized = true; 
   }
   public void closeFile() 
   {
     if (!initialized) 
     { 
       Assert.fail("Trying to close before open");
     }  
   }
   public String readFile() 
   {
     if (!initialized) 
     { 
       Assert.fail("Trying to read before open");
       return null;
     } 
     if (filename.equals("readerror")) 
     { 
       error = true;
       return "";
     } 
     return null; 
   }
 }

Then when you have a test class or the class that you use the FileManipulator class, instead of:

FileManipulator fileChanger = new FileManipulator ();
fileChanger.openFile("MyTestFile.txt")
fileChanger.readFile();
fileChanger.closeFile();

Use the MockFileManipulator constructor to define the data object.

MockFileManipulator fileChanger = new MockFileManipulator ();
fileChanger.openFile("MyTestFile.txt")
fileChanger.readFile();
fileChanger.closeFile();

Comparison of Different Systems

EasyMock

• EasyMock has been the first dynamic Mock Object generator, relieving users of hand-writing Mock Objects, or generating code for them.
• Dynamic creation of Mock Objects
• Supports re-factoring-safe Mock Objects
• Ability to return values and exceptions.
• Method order checking

http://easymock.org/

MOQ

• Moq is the only mocking library for .NET developed from scratch to take full advantage of .NET 3.5 and C# 3.0 features.
• Supports mocking interfaces as well as classes.
• Supports the overriding of expectations where the test can set default expectations in a fixture setup, and override as needed on certain tests
• Intercept and raise events on mocks

http://code.google.com/p/moq/.

NMock

• NMock is a dynamic mock object library for .NET.
• Dynamic creation of Mock Objects
• Allows expectations to be defined and fails the test if any expectations are violated
• Expectations are specified beforehand and verified on the fly as the code under test is being executed, rather than afterward using assertions
• Implementations of interfaces are generate on the fly at runtime

http://www.nmock.org/index.html

Rhino Mocks

• Only supports mocks of interfaces, delegates and classes, including those with parametrized constructors.
• Expectations on the called methods by using strongly typed mocks instead of strings.
• Recursive mocking

http://ayende.com/projects/rhino-mocks.aspx

jMock

• Forces test to be explicit about the argument values that will be passed to the expected methods
• Ability to write custom stubs, constraints, and Invocation Matchers
• Works well with the auto completion and refactoring features of your IDE

http://www.jmock.org/index.html

Mocha

• Similar to jMock only for Ruby
• Supports testing frameworks: Test::Unit, RSpec, test/spec, expectations, Dust, MiniTest and JtestR.

http://mocha.rubyforge.org/

Test::MockObject

• Similar to jMock only for Perl
• Simple testing techniques

http://search.cpan.org/dist/Test-MockObject/lib/Test/MockObject.pm

JMockit

• Simpler and more succinct APIs for writing tests with behavior verification or state verification
• Provides other tools for supporting the creation of large test suites
• Uses class remapping instead of the original proxy dependencies

https://jmockit.dev.java.net/

Conclusion

A mock object framework can make sure that the method under test, when executed, will in fact call certain functions on the mock object and that the method under test will react in an appropriate way to whatever the mock objects do. Most parts of a software system do not work in isolation, but collaborate with other parts to get their job done. Writing tests provides a framework to think about functionality, Mock Objects provides a framework for making assertions about those relationships and for simulating responses. Mock Objects also allows programmers to make their tests only as precise as they need to be.

The different frameworks provide a myriad of options for software developers to produce the correct mock tests for their software. The more complex the system, the more important it is when selecting the mock framework. Support for the framework is essential as well.

Links

1. Java Mock Framework Comparisons

2. Using Mocks And Tests To Design Role-Based Objects

3. Wikipedia - Mock object

4. Java Mock Objects