CSC/ECE 517 Fall 2007/wiki3 1 vb

Problem Statement

Compare the support for assertions in various o-o programming languages. How well is it integrated with the language (instead of being supplied by libraries)? How many kinds of assertions are supported? How are assertions used in the various flavors of XUnit testing frameworks?

Introduction

Sometimes debugging and testing of the code becomes a very tedious task. It becomes difficult to track the problem or able to point out the location of bug in the solution. This is where assertions come into picture. Assertions are Boolean expressions that checks for proper state of the program, method calls for invocation, change in state of data in the class, error handling and the method code for computation.

The syntax of assertion is <assertion_type>(<condition>, <message>); where <assertion_type> indicates what is the assertion made, <condition> indicates a boolean expression that helps to determine whether the assertion is violated or not; <message> will display an error information when the assertion is violated.

Types of Assertions

1) Preconditions: This defines the condition to be satisfied in order to call a method. These conditions are checked at the entry point of method.

2) Postconditions: This defines what the method does. This condition is evaluated at the exit point of the method.

3) Invariants: This defines state-space consistency for a class. It is evaluated at the entry and exit point of a class.

4) Data assertions: This defines conditions that must be met at a particular line of code. These conditions are evaluated at that particular line of code.

Example of Assertion

As stated above Assertion returns a boolean value, if its false, indicates a bug in the code. It prevents the flow of control to fall into an invalid state which would otherwise can cause serious failures.

Following is a simple example of assertion:

/* Defining a object of type TreeNode */
TreeNode root;

/* Asserting that the root is not null */
Assert root != null;

This asserts that root is not null. If it is then it throws a AssertionError. In a way, if any line after the assert statement is executed then it can assume that root is not null.

In the remaining sections, we compare different OO languages and analyze the support of assertion in the languages.

Integration of Assertion

Assertions support can be put inside the language at many levels. Different languages support assertions at different levels, hence it becomes important to have the knowledge of these levels before delving into the specifics of each language. Following lists and describes these levels briefly:

1. Assertions in design by contract: Assertions can be a part of the documentation. They describe the state of the code before executing a particular statement and the state achieved after executing that statement. They can also specify certain invariants of a class.

2. Assertions for Runtime Checking: Assertions can be used in the actual code implementation to find out if the state of the code is consistent and holds valid values for all the variables taking part in the computations at that point. The main advantage of this approach is that the bug is discovered as soon as it is induced in the system, instead of finding it out later with the help of side effects.

3. Assertions during the development cycle: Normally assertions are enabled during the development cycle by the programmer to find the bugs as quickly as possible. This helps in faster development and less number of bugs later on.

4. Static assertions: Static assertions are performed at compile time and report a more evident kind of bugs which are very easy to spot and fix.

Analysis of Support for Assertion in different Programming Languages

The analysis basically focuses on the following few factors

1. Level of integration of Assertions in the language

2. How wide is the functionality provided by assertions.

Assertions support in Java

It is important to place the assertion statements in the appropriate location for better readability of the program. In Java, assertions are placed in the class definition section of the program. Placing the preconditions and postconditions at the start of the class makes them easy to identify. Since precondition is evaluated at the beginning of method call, placing it in the beginning of the class is appropriate but postconditions are treated in a different way. Since they are evaluated at the exit point of method, it must be placed at the top of the method (or before the return statement of the method) and have an assertion tool evaluate the postcondition at all the appropriate conditions. Hence this will improve the readability of the program. Similarly, invariant is placed in the class definition and evaluated for every entry and exit point of every public method declared in the class.

The Java programming language has very less support for assertions but there are different approaches which help to support assertions in java language. They are as follows:

a) Built in: Here, support for assertions is directly included in the Java programming language. Java contains language constructs to formulate the assertions. The syntax of assertion command is checked by the compiler and there is a runtime environment in order to check the assertion at the time of execution. This approach helps to integrate assertions into the programming language and the compiler messages are consistent. Also debugging tools can consider assertions (display stack traces and accurate line numbers).

b) Preprocessing: This is the most common approach for the support of assertions. Here the assertions are not placed in the program but are incorporated into the program as comments. There is preprocessor which can interpret the comments and convert the comments into programming code. The advantage of using this approach is to separate the assertion from programming logic. This approach is useful for languages that do not support assertions. The shortcoming of this approach is that original code may be altered and hence the line numbers of compilers do not match with the line numbers of the program. This approach is implemented in Java programming language since it has very little support for assertions.

The iContract Tool for Java programming uses this approach. Programmers add assertion code in the JavaDoc in the form of comments. Then iContract tool translates these comments into assertion code. Another example is Jass (Java with Assertions). The developers write the assertions in Jass and use a preprocessor to translate them into java code.

Integration of Assertion in Java

The assertion is supported in J2SE 1.4. This is supported by adding the keyword “assert” to the language, and AssertionError class. As mentioned above, assertion is a Boolean expression which must be true during the runtime. This facility can be enabled and disabled at runtime.

Declaration of Assertion

Assertion is declared in two forms

assert Expression;
assert Expression1  :  Expression2;

In both of the above forms, the condition must be true else it will throw an AssertionError at runtime.

Some of the examples are as follows:

Assert Salary > 3000;
Assert isStatusEnabled ();

Code using assertion: //AssertionDemo.java

Class AssertionDemo{
  
  public double withdrawMoney(double balance , double amount){
    assert balance >= amount;
    return balance – amount;
  }
  Public static void main(String args[]){
    System.out.println( withdrawMoney(1000,500) );
    System.out.println( withdrawMoney(1000,2000) );
  }
}

In the above example, the method withdrawMoney is called and the paramters double and amount. The method contains an assertion statement that checks if the balance is greater or equal to the amount. If the condition if false, then AssertionError is thrown.

Assertion support in C #

In C#, one can use Assert Method by using either Debug or Trace Class (belongs to System.Diagnostics namespace). C# follows the Design by Contract Approach for implementing assertions. In order to provide assertions in C#, Contract Sharp has been developed which is syntax directed, code development tool with a GUI that’s helps C# programmers in using assertions in method and class.

Supported Types of Assertions

C# supports imperative (namespace is System.Diagnostics.Debug) and declarative assertions

Imperative Assertion: It is supported using the namespace System.Diagnostic.Debug. If there is an aassertion violation, it thrown an AssertionError dialog. This functionality is completely customizable. Example is shown below:

Declarative Assertions: The assertion is declared in compilation attribute and hence is a part of meta data. It can contain preconditions and postconditons.

– RequiresAttribute for pre conditions – EnsuresAttribute for post conditions

Declarative assertions are applied for asbtract and interface members and are inherited. They need to be enabled at the assembly level.

RequiresAttribute: The condition statement is passed to the compilation attribute and then the C# calls the method Debug.Assert(). This can be enabled and disabled at the assembly level.

EnsuresAttribute: Similarly the condition to asset is passed to compilation attribute but it is checked for every return statement of the method. This function call is useful for method with many return statements. The local variable “result” indicates the returned expression

Some of the built-in Assertion attributes:

• NotNullAttribute

– Expands into “parameter != null” for parameters
– Expands into “result != null” for returned values

• PositiveAttribute

• StrictPositiveAttribute

• ValidIndexAttribute

– Expands into “entity >= 0 && entity < Count” or “entity >= 0 && entity < Length” or

Disadvantages of using Debug.Assert

While using Debug.Assert, one must ensure that code inside the Assert does not affect the results of the program if assert is removed else it might introduce an error when the program is compiled. For example :

// unsafe code
Debug.Assert (meas(i) != 0 );

If we build the program in release mode, the call to meas is removed and hence counter will not be updated (since meas function increments the counter)

Assert Arguments

Trace, Assert and Debug. Assert takes up to three arguments. The first argument is mandatory while others are optional. First argument is the condition to be met. Example is below

Debug.Assert (stacksize > 0 );
Trace.Assert (stacksize > 0 ); 

If the assert takes in two or more arguments, one argument is the condition while others are information or messages to be displayed. Example is below:

Debug.Assert ( stacksize > 0, "Out of stack space" );
Trace.Assert ( stacksize > 0, "Out of stack space" );

Example code for assertion in C#

using System.Diagnostics;
class ConfigFile {
   bool isFileOpen;
   public void Open(string strFile) {
       // Pre-conditions
       Debug.Assert(!isFileOpen, "Config file already open.",
           "You can only call Open() once.");
       Debug.Assert(strFile.Length > 0); // check if the length of the    file is more than 0
       isFileOpen = true;
       // ...
   }

   public static void Main() {
       ConfigFile file = new ConfigFile();
       file.Open("Joe.xml");
       file.Open("Joe.xml"); // Causes an assertion!
   }
}

The main function calls the open method which contains the assertion statement. If the condition of the assertion is satisfied (checks if the file contains contents), the file is opened else it will thrown an AssertionError.

Assertion Support in Eiffel

Eiffel has an inbuilt support for Design by Contract Assertions. There is a language and environment support assertions in Eiffel. And unlike other languages, in Eiffel if an Assertion fails, there is a way to handle that situation. In other words the assertions has the capability to handle exceptions which is not present in most of the OO languages.

Usage of Assertions in Eiffel

The following example displays a simple use of Assertion in Eiffel :

square_root (x: REAL): REAL is
/*Returns the positive square root of x. require*/
  positive_argument: x >= 0.0;

do
/* Code to calculate square root */
ensure
  correct_root: Result * Result = x;
  positive_root: Result >= 0;
end;

The above code defines a function which returns a positive square root of the argument. The code also states that the method takes a real argument which is greater than or equal to zero. Result is a special variable that is used to return values from the methods. The ensure keyword makes sure that the result is correctly calculated and also it is positive.

Eiffel also supports Invariant clauses. As mentioned at the top of the article, these apply to entire classes , and are used to state rules that always hole. For example, a stack class with an empty flag and a count of items held might have the following invariant clause:

invariant
count >= 0;
empty = (count = 0);

The above code states that the count value, i.e. the number of items on the stack can never be less than zero. The second line states that the empty flag can be true only if the count is zero.

Types of Assertion in Eiffel

1). The assertion in Eiffel is closely related to the exception mechanism, which can be independently found in other OO languages like Java. An exception is raised whenever an assertion fails in Eiffel. The exception can caught by a method or can be passed back to the caller. If the exception is not handled anywhere in the hierarchy then the program fails. On the other hand, debuggers can also be used to catch these exceptions.

2). In Eiffel a contract can be defined between clients and servers. Therefore assertions help in avoiding misunderstandings between coders and speed up integration.

3). When new classes are derived from existing ones by inheritance there are design rules which specify the relationship between the assertions in the parent and child classes. These rules ensure that the child can correctly be substituted for the parent in all cases.

4).Eiffel also supports assertions at runtime and thus provides faster debugging capabilities.

Assertion Support in ruby

Ruby alone does not support Assertions , but if combined with Test::Unit Ruby can become quite strong as far as Assertions are concerned. Unit Testing with the help assertions makes it very easy to fix bugs while keeping regressing to a minimum.

How to Use Assertions in Ruby

Require ‘test/unit’ and set your test class to inherit from Test::Unit::TestCase 1). Write methods prefixed with test_ 2). Insert assert statements according to the code logic 3). Run the tests

Following is an example of how assertions can be used in ruby:

my_first_test.rb

require 'test/unit'

class MyFirstTest < Test::Unit::TestCase
  def test_for_truth
    assert true
  end
end

Another example of assertions in Ruby can be found below. Suppose we have defined a MyMath Class which performs basic mathematical functionalities. Now how do we write a Unit Test case which could test all the functionalities?

my_math_test.rb
require 'my_math'
require 'test/unit'

class MyMathTest < Test::Unit::TestCcase
  def test_addition
    assert_equal 4, MyMath.run("2+2")
    assert_equal 4, MyMath.run("1+3")
    assert_equal 5, MyMath.run("5+0")
    assert_equal 0, MyMath.run("-5 + 5")
 end

  def test_subtraction
    assert_equal 0, MyMath.run("2-2")
    assert_equal 1, MyMath.run("2-1")
    assert_equal -1, MyMath.run("2-3")
  end
end

You can find a comprehensive list of assert functions supported by the Test::Unit framework at this page.

Assertion Methods used in Xunit Testing Framework

Assertion Method is a way to obtain an outcome that is executable by computer and useful to user. The outcome of the test is expressed as a series of assertion statements that specifies the condition that would state the test as passed. These assertions are implemented by calling Assertion Methods which are provided by Test Automation Framework or by the test automater as Custom Assertions.

Assertion reduces the complexity of the conditional test logics by moving the complex code into Test Utility Methods. Hence these methods can be used again and again. All members of Xunity Testing Framework provide Assertion Methods.

Implementation of Assertion

The important features for implemenation are:

1) How to call the Assertion Methods.
2) Which method to call among all the Assertion Methods.
3) What information to be displayed in the Assertion Message.

Assertion Methods are named according to how they are accessed. Some of the common variations for accessing Assertion Methods are:

a) Assertion Methods are inherited from Testcase Superclass provided by the framework. Hence they are invoked as though that is provided locally on TestCase Class.
b) These methods can be provided using globally accessible class or module. They are invoked using class or module name. For eg : Assert.assertTrue(x)
c) Assertion methods are provided as mixins or macros. For eg Ruby TestUnit.

Assertion Messages

This is an optional argument in Assertion Methods that provides text message to user when the assertion fails. In Xunit, it is last argument in the list while in Junit, it is the first argument.

Some of the basic Assertion Methods in Xunit Family is below:

1). Single Outcome Assertions: For example Fail. It does not take any arguments since it behaves the same all the time. For example:

fail( "Expected an exception" );
unfinishedTest();

2). Stated Outcome Assertions: Such as assertNotNull(anObjectReference) and assertTrue(aBooleanExpression). These evaluate a single argument. For example:

assertNotNull( a );
assertTrue( b > c );
assertNonZero( b );

3). Expected Exception Assertions such as assert_raises( expectedError) { codeToExecute }. These evaluate a block of code and a single expected exception argument. For example:

assert_raises( RuntimeError, "Should have raised error")
{
  flight.setMileage(-1122) 
}

4). Equality Assertions such as assertEqual(expected, actual); these compare two objects or values for equality. For example:

Assert.AreEqual( x, y );

5). Fuzzy Equality Assertion such as assertEqual(expected, actual, tolerance); these determine whether two values are "close enough" to each other by using a "tolerance" or "comparison mask". For example:

assertEquals( expectedXml, actualXml, elementsToCompare );

Example of Assertion Method:

/*
 * Asserts that two objects are equal. If they are not
 * an AssertionFailedError is thrown with the given message.*/

 static public void assertEquals(String message, Object expected, Object actual) {
     if (expected == null && actual == null)
        return;
     if (expected != null && expected.equals(actual))
        return;
     failNotEquals(message, expected, actual);
  }

References

• http://en.wikipedia.org/wiki/Assertion_%28computing%29

• http://glu.ttono.us/articles/2005/10/30/why-and-how-ruby-and-rails-unit-testing

• http://www.ntecs.de/old-hp/s-direktnet/Data/eiffel/EiffelCpp.txt

• http://xunitpatterns.com/Assertion%20Method.html

• http://java.sun.com/j2se/1.4.2/docs/guide/lang/assert.html

• http://www.codeproject.com/cs/design/DbCwithXCSharp.asp

• http://www.jot.fm/issues/issue_2005_09/article2.pdf

Further Reading

• http://www.jot.fm/issues/issue_2002_08/article1/

• http://www2.sys-con.com/itsg/virtualcd/java/archives/0402/mahmoud/index.html

• http://www.ruby-doc.org/stdlib/libdoc/test/unit/rdoc/classes/Test/Unit/Assertions.html

• http://www.eng.cse.dmu.ac.uk/~hgs/ruby/ruby-unit.html

• http://java.sun.com/j2se/1.5.0/docs/guide/language/assert.html