CSC/ECE 517 Fall 2009/wiki1b 12 testFrmwk
The intent of this document is to list the popular testing frameworks for popular Object Oriented languages.
What is a testing framework?
Wikipedia defines a framework as "A basic conceptual structure used to solve a complex issue. Conceptually we can consider testing frameworks as a guidance, infrastructure, tool to facilitate testing. Below is a comprehensive list of popular testing frameworks for popular Object Oriented languages.
Testing frameworks for object-oriented languages
C++
C++ is a very popular object oriented language. It is statically typed, free form and multi-paradigm programming language. The development of C++ began as an enhancement for C.
Aeryn - Aeryn is a C++ testing framework. It is primarily intended for unit testing. It is adaptable enough to handle integration testing and can be used for other forms of C++ testing.
ATF : It was developed as a part of Google Summer of code 2007. It is a collection of libraries and utilities designed to ease unattended application testing in the hands of developers and end users of a specific piece of software.
Cantata++ : It provides ability to modify calls from software under test. It is a commercial tool.
Cfix : It is a unit testing framework for both C and C++. It is specialized for unmanaged Windows development. It supports the development of both user and kernel mode unit tests.
CppUnit : It is unit testing framework. It started as a port of JUnit to C++.
CUTE : It is an eclipse plug-in for C++ unit testing.
FCTest : It is a lightweight, cross platform, framework for quickly generating x-unit tests in C/C++. The entire framework is available as a single header file.
TUT : It is a small and portable unit test framework for C++. The inputs to it are based on templaets
UnitTest++: It is a light weight unit testing framework for C++. It makes minimal use of advanced library and language features.
Visual Assert: It is a visual studio plug-in. It is based on cfix testing framework.
C#
C# is a object oriented language that has a syntax that is similar to C and C++. C# is a part of the .NET framework and it is compiled to Microsoft Intermediate Language(MSIL) which is similar to java byte code. MSIL allows C# to be platform independent.
NUnit – NUnit is an opensource testing framework. It is ported from JUnit. NUnit brings XUnit to all .NET languages.
PEX – It is an automated white box testing tool for .NET.
csUnit – It is an open source testing tool for the .NET framework.
MSTest – MSTest is a unit testing framework that is developed by Microsoft which is integrated Microsoft Visual Studio. It has a design that is similar to NUnit.
MbUnit– provides advanced unit testing support with advanced fixtures. It is an extensible unit testing framework for .NET Framework that taken in and goes beyond xUnit pattern testing.
Groovy
Groovy is a dynamic language built to run on Java Virtual machine. This language has additional features that are inspired by languages like python, Ruby and smalltalk. Groovy simplifies testing by providing unit test (using JUnit) and mock frameworks as part of the built in features.
Test'N'Groove - 'Test and Groove' framework is the integration of Groovy and TestNG frameworks.
Groovy Mocks - This framework is built in with Groovy and provides excellent support for mocking alternatives while testing Groovy code
Gmock - This is a mocking framework for Groovy. Simple syntax of this framework makes it popular as it makes the tests more readable and concise
easyb - This is a behavior driven development framework that can be used to test Groovy
spock - Spock is a testing and specification framework for Groovy applications also. This is popular as it has highly expressive specification language and it combines the best features of other tools like JUnit, jMock, and RSPec
Java
Java is an object oriented language that derives most of its syntax from C and C++. It has a much simpler object oriented model when compared to C and C++. Java applications are compiled into byte codes and the byte code can be run on any JVM regardless of the underlying architecture.
JUnit – It is the most oldest and most popular java testing frameworks available, JUnit has become the de facto standard for unit testing.
JTiger - It is a JDK 1.5 annotation based testing framework that leverages many of the new features that JDK 1.5 introduced.
TestNG – It is a combination of JUnit and NUnit but it also includes several new features that make it easier to use.
CACTUS – It is a simple test framework for unit testing server side java cod
LISP
LISP is one of oldest and a multi-paradigm, reflective programming language. The name is said to be derived from "List Processing" as the language features linked list as a major data structure. The language also features dynamic typing and functional programming support.
CLUnit is a unit test tool for defining and running unit tests for common LISP language
LIFT is another framework for testing LISP based applications. LIFT supports interactive testing and is inspired by SUnit and JUnit.
LISPUnit is a popular framework designed for beginners of LISP programming and has common LISP library that supports unit testing
FiveAM is a unit testing framework similar to LISP and does hierarchical, composable test suites
Stefil features a more interactive approach to testing as it provides report on test failures and the conditions that caused the failure
Objective C
Objective C is designed to provide C language , the full object oriented programming capabilities based on Smalltalk style messaging.
OCUnit is a XUnit style testing framework for testing Objective C language based applications.
ObjcUnit is another testing framework based on JUnit, adapted for Objective C
WOTest is an unit testing framework for Objective C which integrates fully with XCode
PHP
PHP is a general purpose scripting language which is widely used for web development
PHPUnit is a unit testing framework for PHP programming language based on JUnit framework
SimpleTest is an unit testing framework for PHP programming language and it supports mock objects which is used to regress the web-applications. This framework is based on JUnit/PHPUnit
SnapTest is a unit testing framework compliant with JUnit type of test harness and also provides test output that is complaint with TAP
Python
Python is a general purpose high level programming language. It supports multiple programming paradigms. It is often used as a scripting language.
PyUnit – It is the standard unit testing framework module for python. It is the pythons version of JUnit. It is a part of the pythons standard library.
Doctest - It is a module included in Pthons programming language standard library that allows the easy generation of tests based on output from the standard Python interpreter shell.
Nose - It is a unit test based testing framework for python that makes writing and running tests easier. It provides an alternate test discovery and running process for unit test. Py.test – py.test is a command line tool to collect and run automated tests. It is a distributed testing tool.
PERL
Perl is a high-level, general-purpose, interpreted, dynamic programming language well known for its text processing capabilities. Perl is inspired by AWK, Smalltalk, LISP, C, C++, Unix Shell etc.
TAP - TAP stands for Test Anything Protocol. This is used to allow communication between unit tests and any automated testing framework
Test::Harness - This is a standard test runner for PERL which leverages TAP parser for running various tests
Test::More - Another testing framework that is famous for the testing utilities
Test::Class - This framework provides a simple way to create classes and objects for testing in the xUnit/JUnit Style
Test::Builder - A module for making more testing modules
Test::Unit (a.k.a. PerlUnit) - This framework supports unit testing in an object oriented development paradigm. This is derived from JUnit
Test::DBUnit - Another xUnit based testing framework for testing database modules
Test::Able - This is an xUnit style testing framework inspired by Test::Class with
Moose . This framework was created to overcome the limitations of Test::Class and leverage the functions of Moose.
LectroTest - This is an automatic, specification-based testing tool for Perl. It is modeled after the QuickCheck module for the Haskell programming language.
PerlActor - This is a simple automated executable acceptance test framework for Perl
http://langworth.com/pub/perl_test_refcard.pdf - List of testing modules for PERL
Ruby
Ruby is a dynamic, reflective objected oriented language. It has the syntax that was inspired by Perl and Small talk. It supports multiple programming paradigm. It is a single pass interpreted language.
Test::Unit - It is a ruby unit testing framework.
RSpec – It is a behavior driven development framework for Ruby Programming language. It is inspired by JBehave
Cucumber - It is a tool that can execute plain-text functional descriptions as automated tests.
Scala
Scala stands for "Scalable Language". Scala is a programming language that is designed to integrate the features of Object-orient programming and functional programming. Scala runs on JVM and is compatible with existing java programs.
ScalaTest - is a traditional xUnit style testing framework
ScalaCheck, a library similar to Haskell's QuickCheck and is used for automated unit testing of Scala based application
Specs is a Behavior driven development library for Scala
Rehersal is another unit testing framework for Scala intended as an easier-to-use replacement for SUnit in the standard library
ScUnit is an unit testing framework based on several Java testing and mocking frameworks like JUnit, TestNG,JBehave, EasyMock
Smalltalk
Smalltalk is an object oriented, dynamically typed language and is often considered as a pioneer for many other object oriented languages like Java, Python, Ruby etc.
SUnit - Unit Testing framework for Smalltalk. It is the original source of the xUnit desig
References
Below are the references for the terms that are used in this document.
http://en.wikipedia.org/wiki/Multi-paradigm_programming_language#Multi-paradigm_programming_language http://en.wikipedia.org/wiki/Functional_programming http://en.wikipedia.org/wiki/Procedural_programming http://en.wikipedia.org/wiki/Reflection_(computer_science) http://en.wikipedia.org/wiki/Metaprogramming http://en.wikipedia.org/wiki/Object-oriented_programming
Resources
Below are the resources that would help the reader to jump start on the object oriented languages that are listed in this document
http://www.cplusplus.com/doc/tutorial http://msdn.microsoft.com/en-us/vcsharp/aa336809.aspx http://groovy.codehaus.org/Documentation http://java.sun.com/new2java/ http://developer.apple.com/mac/library/documentation/Cocoa/Conceptual/ObjectiveC/Introduction/introObjectiveC.html http://us2.php.net/tut.php http://www.python.org/doc/ http://www.perl.org/docs.html http://www.ruby-lang.org/en/documentation/ http://www.scala-lang.org/node/197