CSC/ECE 517 Fall 2009/wiki1b 12 testFrmwk

The intent of this document is to list the popular testing frameworks for popular Object Oriented languages.

What is a testing framework?

Wikipedia defines a framework as "A basic conceptual structure used to solve a complex issue. Conceptually we can consider testing frameworks as a guidance, infrastructure, tool to facilitate testing. Below is a comprehensive list of popular testing frameworks for popular Object Oriented languages.

Testing frameworks for object-oriented languages

C++

C++ is a very popular object oriented language. It is statically typed, free form and multi-paradigm programming language. The development of C++ began as an enhancement for C.

Aeryn - Aeryn is a C++ testing framework. It is primarily intended for unit testing. It is adaptable enough to handle integration testing and can be used for other forms of C++ testing.

ATF : It was developed as a part of Google Summer of code 2007. It is a collection of libraries and utilities designed to ease unattended application testing in the hands of developers and end users of a specific piece of software.

Cantata++ : It provides ability to modify calls from software under test. It is a commercial tool.

Cfix : It is a unit testing framework for both C and C++. It is specialized for unmanaged Windows development. It supports the development of both user and kernel mode unit tests.

CppUnit : It is unit testing framework. It started as a port of JUnit to C++.

CUTE : It is an eclipse plug-in for C++ unit testing.

FCTest : It is a lightweight, cross platform, framework for quickly generating x-unit tests in C/C++. The entire framework is available as a single header file.

TUT : It is a small and portable unit test framework for C++. The inputs to it are based on templaets

UnitTest++: It is a light weight unit testing framework for C++. It makes minimal use of advanced library and language features.

Visual Assert: It is a visual studio plug-in. It is based on cfix testing framework.

C#

C# is a object oriented language that has a syntax that is similar to C and C++. C# is a part of the .NET framework and it is compiled to Microsoft Intermediate Language(MSIL) which is similar to java byte code. MSIL allows C# to be platform independent.

NUnit – NUnit is an opensource testing framework. It is ported from JUnit. NUnit brings XUnit to all .NET languages.

PEX – It is an automated white box testing tool for .NET.

csUnit – It is an open source testing tool for the .NET framework.

MSTest – MSTest is a unit testing framework that is developed by Microsoft which is integrated Microsoft Visual Studio. It has a design that is similar to NUnit.

MbUnit– provides advanced unit testing support with advanced fixtures. It is an extensible unit testing framework for .NET Framework that taken in and goes beyond xUnit pattern testing.

Groovy

Groovy is a dynamic language built to run on Java Virtual machine. This language has additional features that are inspired by languages like python, Ruby and smalltalk. Groovy simplifies testing by providing unit test (using JUnit) and mock frameworks as part of the built in features.

Test'N'Groove - 'Test and Groove' framework is the integration of Groovy and TestNG frameworks.

Groovy Mocks - This framework is built in with Groovy and provides excellent support for mocking alternatives while testing Groovy code

Gmock - This is a mocking framework for Groovy. Simple syntax of this framework makes it popular as it makes the tests more readable and concise

easyb - This is a behavior driven development framework that can be used to test Groovy

spock - Spock is a testing and specification framework for Groovy applications also. This is popular as it has highly expressive specification language and it combines the best features of other tools like JUnit, jMock, and RSPec

Java

Java is an object oriented language that derives most of its syntax from C and C++. It has a much simpler object oriented model when compared to C and C++. Java applications are compiled into byte codes and the byte code can be run on any JVM regardless of the underlying architecture.

JUnit – It is the most oldest and most popular java testing frameworks available, JUnit has become the de facto standard for unit testing.

JTiger - It is a JDK 1.5 annotation based testing framework that leverages many of the new features that JDK 1.5 introduced.

TestNG – It is a combination of JUnit and NUnit but it also includes several new features that make it easier to use.

CACTUS – It is a simple test framework for unit testing server side java cod

LISP

LISP is one of oldest and a multi-paradigm, reflective programming language. The name is said to be derived from "List Processing" as the language features linked list as a major data structure. The language also features dynamic typing and functional programming support.

CLUnit is a unit test tool for defining and running unit tests for common LISP language

LIFT is another framework for testing LISP based applications. LIFT supports interactive testing and is inspired by SUnit and JUnit.

LISPUnit is a popular framework designed for beginners of LISP programming and has common LISP library that supports unit testing

FiveAM is a unit testing framework similar to LISP and does hierarchical, composable test suites

Stefil features a more interactive approach to testing as it provides report on test failures and the conditions that caused the failure

Objective C

Objective C is designed to provide C language , the full object oriented programming capabilities based on Smalltalk style messaging.

OCUnit is a XUnit style testing framework for testing Objective C language based applications.

ObjcUnit is another testing framework based on JUnit, adapted for Objective C

WOTest is an unit testing framework for Objective C which integrates fully with XCode

PHP

PHP is a general purpose scripting language which is widely used for web development

PHPUnit is a unit testing framework for PHP programming language based on JUnit framework

SimpleTest is an unit testing framework for PHP programming language and it supports mock objects which is used to regress the web-applications. This framework is based on JUnit/PHPUnit

SnapTest is a unit testing framework compliant with JUnit type of test harness and also provides test output that is complaint with TAP

Python

Python is a general purpose high level programming language. It supports multiple programming paradigms. It is often used as a scripting language.

PyUnit – It is the standard unit testing framework module for python. It is the pythons version of JUnit. It is a part of the pythons standard library.

Doctest - It is a module included in Pthons programming language standard library that allows the easy generation of tests based on output from the standard Python interpreter shell.

Nose - It is a unit test based testing framework for python that makes writing and running tests easier. It provides an alternate test discovery and running process for unit test. Py.test – py.test is a command line tool to collect and run automated tests. It is a distributed testing tool.

PERL

Perl is a high-level, general-purpose, interpreted, dynamic programming language well known for its text processing capabilities. Perl is inspired by AWK, Smalltalk, LISP, C, C++, Unix Shell etc.

TAP - TAP stands for Test Anything Protocol. This is used to allow communication between unit tests and any automated testing framework

Test::Harness - This is a standard test runner for PERL which leverages TAP parser for running various tests

Test::More - Another testing framework that is famous for the testing utilities

Test::Class - This framework provides a simple way to create classes and objects for testing in the xUnit/JUnit Style

Test::Builder - A module for making more testing modules

Test::Unit (a.k.a. PerlUnit) - This framework supports unit testing in an object oriented development paradigm. This is derived from JUnit

Test::DBUnit - Another xUnit based testing framework for testing database modules

Test::Able - This is an xUnit style testing framework inspired by Test::Class with

Moose . This framework was created to overcome the limitations of Test::Class and leverage the functions of Moose.

LectroTest - This is an automatic, specification-based testing tool for Perl. It is modeled after the QuickCheck module for the Haskell programming language.

PerlActor - This is a simple automated executable acceptance test framework for Perl

http://langworth.com/pub/perl_test_refcard.pdf - List of testing modules for PERL

Ruby

Ruby is a dynamic, reflective objected oriented language. It has the syntax that was inspired by Perl and Small talk. It supports multiple programming paradigm. It is a single pass interpreted language.

Test::Unit - It is a ruby unit testing framework.

RSpec – It is a behavior driven development framework for Ruby Programming language. It is inspired by JBehave

Cucumber - It is a tool that can execute plain-text functional descriptions as automated tests.

Scala

Scala stands for "Scalable Language". Scala is a programming language that is designed to integrate the features of Object-orient programming and functional programming. Scala runs on JVM and is compatible with existing java programs.

ScalaTest - is a traditional xUnit style testing framework

ScalaCheck, a library similar to Haskell's QuickCheck and is used for automated unit testing of Scala based application

Specs is a Behavior driven development library for Scala

Rehersal is another unit testing framework for Scala intended as an easier-to-use replacement for SUnit in the standard library

ScUnit is an unit testing framework based on several Java testing and mocking frameworks like JUnit, TestNG,JBehave, EasyMock

Smalltalk

Smalltalk is an object oriented, dynamically typed language and is often considered as a pioneer for many other object oriented languages like Java, Python, Ruby etc.

SUnit - Unit Testing framework for Smalltalk. It is the original source of the xUnit desig

References

Below are the references for the terms that are used in this document.

http://en.wikipedia.org/wiki/Multi-paradigm_programming_language#Multi-paradigm_programming_language

http://en.wikipedia.org/wiki/Functional_programming

http://en.wikipedia.org/wiki/Procedural_programming

http://en.wikipedia.org/wiki/Reflection_(computer_science)

http://en.wikipedia.org/wiki/Metaprogramming

http://en.wikipedia.org/wiki/Object-oriented_programming

Resources

Below are the resources that would help the reader to jump start on the object oriented languages that are listed in this document

http://www.cplusplus.com/doc/tutorial

http://msdn.microsoft.com/en-us/vcsharp/aa336809.aspx

http://groovy.codehaus.org/Documentation

http://java.sun.com/new2java

http://developer.apple.com/mac/library/documentation/Cocoa/Conceptual/ObjectiveC/Introduction/introObjectiveC.html

http://us2.php.net/tut.php

http://www.python.org/doc

http://www.perl.org/docs.html

http://www.ruby-lang.org/en/documentation

http://www.scala-lang.org/node/197