Unit Test for Buddi: Difference between revisions
No edit summary |
No edit summary |
||
(One intermediate revision by the same user not shown) | |||
Line 117: | Line 117: | ||
Of course, from a human factors standpoint, you'd need to modify the code so that it's flexible enough to allow people to eat their ice cream first, if so desired. In which case, you'd need to add a test to prove that your four-year old nephew's ice cream comes with everyone else's salads, but Grandma's ice cream comes at the end with your cappuccino. | Of course, from a human factors standpoint, you'd need to modify the code so that it's flexible enough to allow people to eat their ice cream first, if so desired. In which case, you'd need to add a test to prove that your four-year old nephew's ice cream comes with everyone else's salads, but Grandma's ice cream comes at the end with your cappuccino. | ||
=== Range === | |||
Range is a convenient catch-all word for the situation where a variable's type allows it to take on a wider range of values than you need—or want. For instance, a person's age is typically represented as an integer, but no one has ever lived to be 200,000 years old, even though that's a perfectly valid integer value. Similarly, there are only 360 degrees in a circle, even though degrees are commonly stored in an integer. | |||
In good object oriented design, you do not use a raw native type (e.g., an int or Integer) to store a bounded-integer value such as an age, or a compass heading. | |||
Bearing.java | |||
/** | |||
* Compass bearing | |||
*/ | |||
public class Bearing { | |||
protected int bearing; // 0..359 | |||
/** | |||
* Initialize a bearing to a value from 0..359 | |||
*/ | |||
public Bearing(int num_degrees) { | |||
if (num_degrees < 0 || num_degrees > 359) { | |||
throw new RuntimeException("Bad bearing"); | |||
} | |||
bearing = num_degrees; | |||
} | |||
/** | |||
* Return the angle between our bearing and another. | |||
* May be negative. | |||
*/ | |||
public int angleBetween(Bearing anOther) { | |||
return bearing - anOther.bearing; | |||
} | |||
} | |||
Notice that the angle returned is just an int, as we are not placing any range restrictions on the result (it may be negative, etc.) | |||
By encapsulating the concept of a bearing within a class, you've now got one place in the system that can filter out bad data. You cannot create a Bearing object with out of range values. Thus, the rest of the system can use Bearing objects and be assured that they contain only reasonable values. | |||
Other ranges may not be as straightforward. For instance, suppose you have a class that maintains two sets of x, y coordinates. These are just integers, with arbitrary values, but the constraint on the range is such that the two points must describe a rectangle with no side greater than 100 units. That is, the allowed range of values for both x, y pairs is interdependent. You'll want a range test for any method that can affect a coordinate to ensure that the resulting range of the x, y pairs remains legitimate. For more information on this topic, see "invariants" in the Design Issues chapter on page 99. | |||
Since you will likely call this from a number of different tests, it probably makes sense to make a new assert method: | |||
public static final int MAX_DIST = 100; | |||
public void assertPairRange(String message, | |||
Point one, Point two) { | |||
assertTrue(message, | |||
Math.abs(one.x - two.x) <= MAX_DIST); | |||
assertTrue(message, | |||
Math.abs(one.y - two.y) <= MAX_DIST); | |||
} | |||
But the most common ranges you'll want to test probably depend on physical data structure issues, not application domain constraints. Take a simple example like a stack class that implements a stack of Strings using an array: | |||
MyStack.java | |||
public class MyStack { | |||
public MyStack() { | |||
stack = new String[100]; | |||
next_index = 0; | |||
} | |||
public String pop() { | |||
return stack[--next_index]; | |||
} | |||
// Delete n items from the stack en-masse | |||
public void delete(int n) { | |||
next_index -= n; | |||
} | |||
public void push(String aString) { | |||
stack[next_index++] = aString; | |||
} | |||
public String top() { | |||
return stack[next_index-1]; | |||
} | |||
private int next_index; | |||
private String[] stack; | |||
} | |||
There are some potential bugs lurking here, as there are no checks at all for either an empty stack or a stack overflow. However we manipulate the index variable next index,one thing is supposed to be always true: (next_index >= 0&& next index < stack.length). We'd like to check to make sure this expression is true. | |||
Both next_index and stack are private variables; you don't want to have to expose those just for the sake of testing. There are several ways around this problem; for now we'll just make a special method in MyStack named checkInvariant(): | |||
MyStack.java | |||
Public void checkInvariant() | |||
throws InvariantException { | |||
// JDK 1.4 can use assert() instead | |||
if ( !(next_index >= 0 && | |||
next_index < stack.length)) { | |||
throw new InvariantException( | |||
"next_index out of range: " + | |||
next_index + | |||
" for stack length " + | |||
stack.length); | |||
} | |||
} | |||
Now a test method can call checkInvariant() to ensure that nothing has gone awry inside the guts of the stack class, without having direct access to those same guts. | |||
TestMyStock.java | |||
import junit.framework.*; | |||
public class TestMyStack extends TestCase { | |||
public void testEmpty() throws InvariantException { | |||
MyStack stack = new MyStack(); | |||
stack.checkInvariant(); | |||
stack.push("sample"); | |||
stack.checkInvariant(); | |||
// Popping ;last element ok | |||
assertEquals("sample", stack.pop()); | |||
stack.checkInvariant(); | |||
// Delete from empty stack | |||
stack.delete(1); | |||
stack.checkInvariant(); | |||
} | |||
} | |||
When you run this test, you'll quickly see that we need to add some range checking! | |||
There was 1 error: | |||
1) testEmpty(TestMyStack)InvariantException: | |||
next_index out of range: -1 for stack length of 100 | |||
at MyStack.checkInvariant(MyStack.java:31) | |||
at TestMyStack.testEmpty(TestMyStack.java:16) | |||
It's much easier to find and fix this sort of error here in a simple testing environment instead of buried in a real application. | |||
Almost any indexing concept (whether it's a genuine integer index or not) should be extensively tested. Here are a few ideas to get you started: | |||
* | |||
Start and End index have the same value | |||
* | |||
First is greater than Last | |||
* | |||
Index is negative | |||
* | |||
Index is greater than allowed | |||
* | |||
Count doesn't match actual number of items | |||
* | |||
… | |||
=== Reference === | |||
What things does your method reference that are outside the scope of the method itself? Any external dependencies? What state does the class have to have? What other conditions must exist in order for the method to work? | |||
For example, a method in a web application to display a customer's account history might require that the customer is first logged on. The method pop() for a stack requires a non- empty stack. Shifting the transmission in your car to Park from Drive requires that the car is stopped. | |||
If you have to make assumptions about the state of the class and the state of other objects or the global application, then you need to test your code to make sure that it is well-behaved if those conditions are not met. For example, the code for the microprocessor-controlled transmission might have unit tests that check for that particular condition: the state of the transmission (whether it can shift into Park or not) depends on the state of the car (is it in motion or stopped). | |||
public void testJamItIntoPark() { | |||
transmission.shift(DRIVE); | |||
car.accelerateTo(35); | |||
assertEquals(DRIVE, transmission.getGear()); | |||
// should silently ignore | |||
transmission.shift(PARK); | |||
assertEquals(DRIVE, transmission.getGear()); | |||
car.accelerateTo(0); // i.e., stop | |||
car.brakeToStop(); | |||
// should work now | |||
transmission.shift(PARK); | |||
assertEquals(PARK, transmission.getGear()); | |||
} | |||
The preconditions for a given method specify what state the world must be in for this method to run. In this case, the precondition for putting the transmission in park is that the car's engine (a separate component elsewhere in the application's world) must be at a stop. That's a documented requirement for the method, so we want to make sure that the method will behave gracefully (in this particular case, just ignore the request silently) in case the precondition is not met. | |||
At the end of the method, postconditions are those things that you guarantee your method will make happen. Direct results returned by the method are one obvious thing to check, but if the method has any side-effects then you need to check those as well. In this case, applying the brakes has the side effect of stopping the car. | |||
Some languages even have built-in support for preconditions and postconditions; interested readers might want to read about Eiffel in Object-Oriented Software Construction [Mey97]. | |||
=== Existence === | |||
A large number of potential bugs can be discovered by asking the key question "does some given thing exist?". | |||
For any value you are passed in or maintain, ask yourself what would happen to the method if the value didn't exist—if it were null, or blank, or zero. | |||
Many Java library methods will throw an exception of some sort when faced with non-existent data. The problem is that it's hard to debug a generic runtime exception buried deep in some library. But an exception that reports "Age isn't set" makes tracking down the problem much easier. | |||
Most methods will blow up if expected data is not available, and that's probably not what you want them to do. So you test for the condition—see what happens if you get a null instead of a CustomerRecord because some search failed. See what happens if the file doesn't exist, or if the network is unavailable. | |||
Ah, yes: things in the environment can wink out of existence as well—networks, files' URLs, license keys, users, printers—you name it. All of these things may not exist when you expect them to, so be sure to test with plenty of nulls, zeros, empty strings and other nihilist trappings. | |||
Make sure your method can stand up to nothing. | |||
=== Cardinality === | |||
Cardinality has nothing to do with either highly-placed religious figures or small red birds, but instead with counting. | |||
Computer programmers (your humble authors included) are really bad at counting, especially past 10 when the fingers can no longer assist us. For instance, answer the following question quickly, off the top of your head, without benefit of fingers, paper, or UML: | |||
If you've got 12 feet of lawn that you want to fence, and each section of fencing is 3 feet wide, how many fence posts do you need? | |||
If you're like most of us, you probably answered "4" without thinking too hard about it. Pity is, that's wrong—you need five fence posts as shown in Figure 5.1 on page 57. This model, and the subsequent common errors, come up so often that they are graced with the name "fence post errors." | |||
It's one of many ways you can end up being "off by one;" an occasionally fatal condition that afflicts all programmers from time to time. So you need to think about ways to test how well your method counts, and check to see just how many of a thing you may have. | |||
Image from book | |||
Figure 5.1: A Set of Fence posts | |||
It's a related problem to Existence, but now you want to make sure you have exactly as many as you need, or that you've made exactly as many as needed. In most cases, the count of some set of values is only interesting in these three cases: | |||
1. | |||
Zero | |||
2. | |||
One | |||
3. | |||
More than one | |||
It's called the "0–1-n Rule," and it's based on the premise that if you can handle more than one of something, you can probably handle 10, 20, or 1,000 just as easily. Most of the time that's true, so many of our tests for cardinality are concerned with whether we have 2 or more of something. Of course there are situations where an exact count makes a difference—10 might be important to you, or 250. | |||
Suppose you are maintaining a list of the Top-Ten food items ordered in a pancake house. Every time an order is taken, you have to adjust the top-ten list. You also provide the current top-ten list as a real-time data feed to the pancake boss's PDA. What sort of things might you want to test for? | |||
* | |||
Can you produce a report when there aren't yet ten items in the list? | |||
* | |||
Can you produce a report when there are no items on the list? | |||
* | |||
Can you produce a report when there is only one item on the list? | |||
* | |||
Can you add an item when there aren't yet ten items in the list? | |||
* | |||
Can you add an item when there are no items on the list? | |||
* | |||
Can you add an item when there is only one item on the list? | |||
* | |||
What if there aren't ten items on the menu? | |||
* | |||
What if there are no items on the menu? | |||
Having gone through all that, the boss now changes his mind and wants a top-twenty list instead. What do you have to change? | |||
The correct answer is "one line," something like the following: | |||
public int getMaxEntries() { | |||
return 20; | |||
} | |||
Now, when the boss gets overwhelmed and pleads with you to change this to be a top-five report (his PDA is pretty small, after all), you can go back and change this one number. The test should automatically follow suit, because it uses the same accessor. | |||
So in the end, the tests concentrate on boundary conditions of 0, 1, and n, where n can—and will—change as the business demands | |||
=== Time === | |||
The last boundary condition in the CORRECT acronym is Time. There are several aspects to time you need to keep in mind: | |||
* | |||
Relative time (ordering in time) | |||
* | |||
Absolute time (elapsed and wall clock) | |||
* | |||
Concurrency issues | |||
Some interfaces are inherently stateful; you expect that login() will be called before logout(),that prepareStatement() is called before executeStatement(), connect() before read() which is before close(),and so on. | |||
What happens if those methods are called out of order? Maybe you should try calling methods out of the expected order. Try skipping the first, last and middle of a sequence. Just as order of data may have mattered to you in the earlier examples (as we described in "Ordering" on page 48), now it's the order of the calling sequence of methods. | |||
Relative time might also include issues of timeouts in the code: how long your method is willing to wait for some ephemeral resource to become available. As we'll discuss shortly, you'll want to exercise possible error conditions in your code, including things such as timeouts. Maybe you've got conditions that aren't guarded by timeouts—can you think of a situation where the code might get "stuck" waiting forever for something that might not happen? | |||
This leads us to issues of elapsed time. What if something you are waiting for takes "too much" time? What if your method takes too much time to return to the caller? | |||
Then there's the actual wall clock time to consider. Most of the time, this makes no difference whatsoever to code. But every now and then, time of day will matter, perhaps in subtle ways. Here's a quick question, true or false: every day of the year is 24 hours long? | |||
The answer is "it depends." In UTC (Universal Coordinated Time, the modern version of Greenwich Mean Time, or GMT), the answer is YES. In areas of the world that do not observe Daylight Savings Time (DST), the answer is YES. In most of the U.S. (which does observe DST), the answer is NO. In April, you'll have a day with 23 hours (spring forward) and in October you'll have a day with 25 (fall back). This means that arithmetic won't always work as you expect; 1:45AM plus 30 minutes might equal 1:15, for instance. | |||
But you've tested any time-sensitive code on those boundary days, right? For locations that honor DST and for those that do not? | |||
Oh, and don't assume that any underlying library handles these issues correctly on your behalf. Unfortunately, when it comes to time, there's a lot of broken code out there. | |||
Finally, one of the most insidious problems brought about by time occurs in the context of concurrency and synchronized access issues. It would take an entire book to cover designing, implementing, and debugging multi-threaded, concurrent programs, so we won't take the time now to go into details, except to point out that most code you write in most languages today will be run in a multi-threaded environment. | |||
So ask yourself, what will happen if multiple threads use this same object at the same time? Are there global or instance-level data or methods that need to be synchronized? How about external access to files or hardware? Be sure to add the synchronized keyword to any data element or method that needs it, and try firing off multiple threads as part of your test.[2] | |||
[2]JUnit as shipped has some issues with multi-threaded test cases, but there are various fixes available on the net. | |||
=== Try it Yourself === | |||
Now that we've covered the Right-BICEP and CORRECT way to come up with tests, it's your turn to try. | |||
For each of the following examples and scenarios, write down as many possible unit tests as you can think of. | |||
Exercises | |||
1. | |||
A simple stack class. Push String objects onto the stack, and pop them off according to normal stack semantics. This class provides the following methods: | |||
StackExercise.java | |||
public interface StackExercise { | |||
/** | |||
* Return and remove the most recent item from | |||
* the top of the stack. | |||
* Throws StackEmptyException | |||
* if the stack is empty | |||
*/ | |||
public String pop() throws StackEmptyException; | |||
/** | |||
* Add an item to the top of the stack. | |||
*/ | |||
public void push(String item); | |||
/** | |||
* Return but do not remove the most recent | |||
* item from the top of the stack. | |||
* Throws StackEmptyException | |||
* if the stack is empty | |||
*/ | |||
public String top() throws StackEmptyException; | |||
/** | |||
* Returns true if the stack is empty. | |||
*/ | |||
public boolean isEmpty(); | |||
} | |||
Here are some hints to get you started: what is likely to break? How should the stack behave when it is first initialized? After it's been used for a while? Does it really do what it claims to do? | |||
Image from book | |||
2. | |||
A shopping cart. This class lets you add, delete, and count the items in a shopping cart. | |||
What sort of boundary conditions might come up? Are there any implicit restrictions on what you can delete? Are there any interesting issues if the cart is empty? | |||
public interface ShoppingCart { | |||
/** | |||
* Add this many of this item to the | |||
* shopping cart. | |||
*/ | |||
public void addItems(Item anItem, int quantity) | |||
throws NegativeCountException; | |||
/** | |||
* Delete this many of this item from the | |||
* shopping cart | |||
*/ | |||
public void deleteItems(Item anItem, | |||
int quantity) | |||
throws NegativeCountException, | |||
NoSuchItemException; | |||
/** | |||
* Count of all items in the cart | |||
* (that is, all items x qty each) | |||
*/ | |||
public int itemCount(); | |||
/** | |||
* Return Iterator of all items | |||
* (see Java Collection's doc) | |||
*/ | |||
public Iterator iterator(); | |||
} | |||
Image from book | |||
3. | |||
A fax scheduler. This code will send faxes from a specified file name to a U.S. phone number. There is a validation requirement; a U.S. phone number with area code must be of the form xnn-nnn-nnnn,where x must be a digit in the range [2..9] and n can be [0..9]. The following blocks are reserved and are not currently valid area codes: x11, x9n, 37n, 96n. | |||
The method's signature is: | |||
/** | |||
* Send the named file as a fax to the | |||
* given phone number. | |||
*/ | |||
public boolean sendFax(String phone, | |||
String filename) | |||
throws MissingOrBadFileException, | |||
PhoneFormatException, | |||
PhoneAreaCodeException; | |||
Given these requirements, what tests for boundary conditions can you think of? | |||
Image from book | |||
4. | |||
An automatic sewing machine that does embroidery. The class that controls it takes a few basic commands. The coordinates (0,0) represent the lower-left corner of the machine. x and y increase as you move toward the upper-right corner, whose coordinates are x = getTableSize().width 1 and y = getTableSize().height - 1. | |||
Coordinates are specified in fractions of centimeters. | |||
public void moveTo(float x, float y); | |||
public void sewTo(float x, float y); | |||
public void setWorkpieceSize(float width, | |||
float height); | |||
public Size getWorkpieceSize(); | |||
public Size getTableSize(); | |||
There are some real-world constraints that might be interesting: you can't sew thin air, of course, and you can't sew a workpiece bigger than the machine. | |||
Given these requirements, what boundary conditions can you think of? | |||
Image from book | |||
5. | |||
Audio/Video Editing Transport. A class that provides methods to control a VCR or tape deck. There's the notion of a "current position" that lies somewhere between the beginning of tape (BOT) and the end of tape (EOT). | |||
You can ask for the current position and move from there to another given position. Fast-forward moves from current position toward EOT by some amount. Rewind moves from current position toward BOT by some amount. | |||
When tapes are first loaded, they are positioned at BOT automatically. | |||
AVTransport.java | |||
public interface AVTransport { | |||
/** | |||
* Move the current position ahead by this many | |||
* seconds. Fast-forwarding past end-of-tape | |||
* leaves the position at end-of-tape | |||
*/ | |||
public void fastForward(float seconds); | |||
/** | |||
* Move the current position backwards by this | |||
* many seconds. Rewinding past zero leaves | |||
* the position at zero | |||
*/ | |||
public void rewind(float seconds); | |||
/** | |||
* Return current time position in seconds | |||
*/ | |||
public float currentTimePosition(); | |||
/** | |||
* Mark the current time position with this label | |||
*/ | |||
public void markTimePosition(String name); | |||
/** | |||
* Change the current position to the one | |||
* associated with the marked name | |||
*/ | |||
public void gotoMark(String name); | |||
} | |||
Image from book | |||
6. | |||
Audio/Video Editing Transport, Release 2.0. As above, but now you can position in seconds, minutes, or frames (there are exactly 30 frames per second in this example), and you can move relative to the beginning or the end. | |||
Image from book | |||
Answers | |||
1. | |||
* | |||
For a brand-new stack, isEmpty() should be true, top() and pop() should throw exceptions. | |||
* | |||
Starting with an empty stack, call push() to push a test string onto the stack. Verify that top() returns that string several times in a row, and that isEmpty() returns false. | |||
* | |||
Call pop() to remove the test string, and verify that it is the same string. [3] isEmpty() should now be true. Call pop() again verify an exception is thrown. | |||
* | |||
Now do the same test again, but this time add multiple items to the stack. Make sure you get the rights ones back, in the right order (the most recent item added should be the one returned). | |||
* | |||
Push a null onto the stack and pop it; confirm you get a null back. | |||
* | |||
Ensure you can use the stack after it has thrown exceptions. | |||
2. | |||
* | |||
Call addItems with quantity of 0 and itemCount should remain the same. | |||
* | |||
Call deleteItem with quantity of 0 and itemCount should remain the same. | |||
* | |||
Call addItems with a negative quantity and it should raise an exception. | |||
* | |||
Call deleteItem with a negative quantity and it should raise an exception. | |||
* | |||
Call addItems and the item count should increase, whether the item exists already or not. | |||
* | |||
Call deleteItem where the item doesn't exist and it should raise an exception. | |||
* | |||
Call deleteItem when there are no items in the cart and itemCount should remain at 0. | |||
* | |||
Call deleteItem where the quantity is larger than the number of those items in the cart and it should raise an exception. | |||
* | |||
Call iterator when there are no items in the cart and it should return an empty iterator (i.e., it's a real iterator object (not null) that contains no items). | |||
* | |||
Call addItem several times for a couple of items and verify that contents of the cart match what was added (as reported via iterator() and itemCount()). | |||
Hint: you can combine several of these asserts into a single test. For instance, you might start with an empty cart, add 3 of an item, then delete one of them at a time. | |||
3. | |||
* | |||
Phone numbers with an area code of 111, 211, up to 911, 290, 291, etc, 999, 370-379, or 960-969 should throw a PhoneAreaCodeException. | |||
* | |||
A phone number with too many digits (in one of each set of number, area code, prefix, number) should throw a PhoneFormatException. | |||
* | |||
A phone number with not enough digits (in one of each set) should throw a PhoneFormatException. | |||
* | |||
A phone number with illegal characters (spaces, letters, etc.) should throw a PhoneFormatException. | |||
* | |||
A phone number that's missing dashes should throw a PhoneFormatException. | |||
* | |||
A phone number with multiple dashes should throw a Phone-FormatException. | |||
* | |||
A null phone number should throw a PhoneFormatException. | |||
* | |||
A file that doesn't exist should throw a MissingOrBadFile-Exception. | |||
* | |||
A null filename should also throw that exception. | |||
* | |||
An empty file should throw a MissingOrBadFileException. | |||
* | |||
A file that's not in the correct format should throw a Missing-OrBadFileException. | |||
4. | |||
* | |||
Huge value for one or both coordinates | |||
* | |||
Huge value for workpiece size | |||
* | |||
Zero or negative value for one or both coordinates | |||
* | |||
Zero or negative value for workpiece size | |||
* | |||
Coordinates that move off the workpiece | |||
* | |||
Workpiece bigger than the table | |||
5. | |||
* | |||
Verify that the initial position is BOT. | |||
* | |||
Fast forward by some allowed amount (not past end of tape), then rewind by same amount. Should be at initial location. | |||
* | |||
Rewind by some allowed amount amount (not past beginning of tape), then fast forward by same amount. Should be at initial location. | |||
* | |||
Fast forward past end of tape, then rewind by same amount. Should be before the initial location by an appropriate amount to reflect the fact that you can't advance the location past the end of tape. | |||
* | |||
Try the same thing in the other direction (rewind past beginning of tape). | |||
* | |||
Mark various positions and return to them after moving the current position around. | |||
* | |||
Mark a position and return to it without moving in between. | |||
6. | |||
Cross-check results using different units: move in one unit and verify your position using another unit; move forward in one unit and back in another, and so on. | |||
[3]In this case, assertEquals() isn't good enough; you need assert-Same() to ensure it's the same object. |
Latest revision as of 01:24, 20 October 2009
Unit Test for Buddi
CORRECT Boundary Conditions
Many bugs in code occur around "boundary conditions," that is, under conditions where the code's behavior may be different from the normal, day-to-day routine.
For instance, suppose you have a function that takes two integers:
public int calculate(int a, int b) { return a/(a+b); }
Most of the time, this code will return a number just as you expect. But if the sum of a and b happens to equal zero, you will get an ArithmeticException instead of a return value. That is a boundary condition—a place where things might suddenly go wrong, or at least behave differently from your expectations.
To help you think of tests for boundary conditions, we'll use the acronym CORRECT:
- Conformance — Does the value conform to an expected format?
- Ordering — Is the set of values ordered or unordered as appropriate?
- Range — Is the value within reasonable minimum and maximum values?
- Reference — Does the code reference anything external that isn't under direct control of the code itself?
- Existence — Does the value exist (e.g., is non-null, non-zero, present in a set, etc.)?
- Cardinality — Are there exactly enough values?
- Time (absolute and relative) — Is everything happening in order? At the right time? In time?
Let's look at each one of these in turn. Remember that for each of these areas, you want to consider data that is passed in as arguments to your method as well as internal data that you maintain inside your method and class.
The underlying question that you want to answer fully is:
What else can go wrong?
Once you think of something that could go wrong, write a test for it. Once that test passes, again ask yourself, "what else can go wrong?" and write another test, and so on.
Conformance
Many times you expect or produce data that must conform to some specific format. An e-mail address, for instance, isn't just a simple string. You expect that it must be of the form:
*
name@somewhere.com
With the possibility of extra dotted parts:
*
firstname.lastname@subdomain.somewhere.com
And even oddballs like this one:
*
firstname.lastname%somewhere@subdomain.somewhere.com
Suppose you are writing a method that will extract the user's name from their e-mail address. You'll expect that the user's name is the portion before the "@" sign. What will your code do if there is no "@" sign? Will it work? Throw a runtime exception? Is this a boundary condition you need to consider?[1]
Validating formatted string data such as e-mail addresses, phone numbers, account numbers, or file names is usually straightforward. But what about more complex structured data? Suppose you are reading some sort of report data that contains a header record linked to some number of data records, and finally to a trailer record. How many conditions might we have to test?
*
What if there's no header, just data and a trailer? *
What if there's no data, just a header and trailer? *
What if there's no trailer, just a header and data? *
What if there's just a trailer? *
What if there's just a header? *
What if there's just data?
Just as with the simpler e-mail address example, you have to consider what will happen if the data does not conform to the structure you think it should.
And of course, if you are creating something like an e-mail address (possibly building it up from different sources) or the structured data above, you want to test your result to make sure it conforms.
[1]E-mail addresses are actually very complicated. A close reading of RFC822 may surprise you.
Ordering
Another area to consider is the order of data, or the position of one piece of data within a larger collection. For instance, in the largest() example in the previous chapter, one bug manifested itself depending on whether the largest number you were searching for was at the beginning or end of the list.
That's one aspect of ordering. Any kind of search routine should be tested for conditions where the search target is first or last, as many common bugs can be found that way.
For another aspect of ordering, suppose you are writing a method that is passed a collection containing a restaurant order. You would probably expect that the appetizers will appear first in the order, followed by the salad (and that all-important dressing choice), then the entree and finally a decadent dessert involving lots of chocolate.
What happens to your code if the dessert is first, and the entree is last?
If there's a chance that sort of thing can happen, and if it's the responsibility of your method to deal with it if it does, then you need to test for this condition and address the problem. Now, it may be that this is not something your method needs to worry about. Perhaps this needs to be addressed at the user input level (see "Testing Invalid Parameters" later on).
If you're writing a sort routine, what might happen if the set of data is already ordered? Or worse yet, sorted in precisely reverse order? Ask yourself if that could cause trouble—if these are conditions that might be worth testing, too.
If you are supposed to maintain something in order, check that it is. For example, if your method is part of the GUI that is sending the dinner order back to the kitchen, you should have a test that verifies that the items are in the correct serving order:
public void testKitchenOrder() {
Order order = new Order(); FoodItem dessert = new Dessert("Chocolate Souffle"); FoodItem entree = new Entree("Beef Oscar"); FoodItem salad = new Salad("Tossed", "Parmesan Peppercorn"); // Add out of order order.addFoodItem(dessert); order.addFoodItem(entree); order.addFoodItem(salad); // But should come out in serving order Iterator itr = order.iterator(); assertEquals(salad, itr.next()); assertEquals(entree, itr.next()); assertEquals(dessert, itr.next()); // No more left assertFalse(itr.hasNext());
}
Of course, from a human factors standpoint, you'd need to modify the code so that it's flexible enough to allow people to eat their ice cream first, if so desired. In which case, you'd need to add a test to prove that your four-year old nephew's ice cream comes with everyone else's salads, but Grandma's ice cream comes at the end with your cappuccino.
Range
Range is a convenient catch-all word for the situation where a variable's type allows it to take on a wider range of values than you need—or want. For instance, a person's age is typically represented as an integer, but no one has ever lived to be 200,000 years old, even though that's a perfectly valid integer value. Similarly, there are only 360 degrees in a circle, even though degrees are commonly stored in an integer.
In good object oriented design, you do not use a raw native type (e.g., an int or Integer) to store a bounded-integer value such as an age, or a compass heading.
Bearing.java
/**
* Compass bearing */
public class Bearing {
protected int bearing; // 0..359 /** * Initialize a bearing to a value from 0..359 */ public Bearing(int num_degrees) { if (num_degrees < 0 || num_degrees > 359) { throw new RuntimeException("Bad bearing"); } bearing = num_degrees; } /** * Return the angle between our bearing and another. * May be negative. */ public int angleBetween(Bearing anOther) { return bearing - anOther.bearing; }
}
Notice that the angle returned is just an int, as we are not placing any range restrictions on the result (it may be negative, etc.)
By encapsulating the concept of a bearing within a class, you've now got one place in the system that can filter out bad data. You cannot create a Bearing object with out of range values. Thus, the rest of the system can use Bearing objects and be assured that they contain only reasonable values.
Other ranges may not be as straightforward. For instance, suppose you have a class that maintains two sets of x, y coordinates. These are just integers, with arbitrary values, but the constraint on the range is such that the two points must describe a rectangle with no side greater than 100 units. That is, the allowed range of values for both x, y pairs is interdependent. You'll want a range test for any method that can affect a coordinate to ensure that the resulting range of the x, y pairs remains legitimate. For more information on this topic, see "invariants" in the Design Issues chapter on page 99.
Since you will likely call this from a number of different tests, it probably makes sense to make a new assert method:
public static final int MAX_DIST = 100;
public void assertPairRange(String message,
Point one, Point two) { assertTrue(message, Math.abs(one.x - two.x) <= MAX_DIST); assertTrue(message, Math.abs(one.y - two.y) <= MAX_DIST);
}
But the most common ranges you'll want to test probably depend on physical data structure issues, not application domain constraints. Take a simple example like a stack class that implements a stack of Strings using an array:
MyStack.java
public class MyStack {
public MyStack() { stack = new String[100]; next_index = 0; } public String pop() { return stack[--next_index]; } // Delete n items from the stack en-masse public void delete(int n) { next_index -= n; } public void push(String aString) { stack[next_index++] = aString; } public String top() { return stack[next_index-1]; }
private int next_index; private String[] stack;
}
There are some potential bugs lurking here, as there are no checks at all for either an empty stack or a stack overflow. However we manipulate the index variable next index,one thing is supposed to be always true: (next_index >= 0&& next index < stack.length). We'd like to check to make sure this expression is true.
Both next_index and stack are private variables; you don't want to have to expose those just for the sake of testing. There are several ways around this problem; for now we'll just make a special method in MyStack named checkInvariant():
MyStack.java
Public void checkInvariant()
throws InvariantException { // JDK 1.4 can use assert() instead if ( !(next_index >= 0 && next_index < stack.length)) { throw new InvariantException( "next_index out of range: " + next_index + " for stack length " + stack.length); }
}
Now a test method can call checkInvariant() to ensure that nothing has gone awry inside the guts of the stack class, without having direct access to those same guts.
TestMyStock.java
import junit.framework.*;
public class TestMyStack extends TestCase {
public void testEmpty() throws InvariantException { MyStack stack = new MyStack();
stack.checkInvariant(); stack.push("sample"); stack.checkInvariant(); // Popping ;last element ok assertEquals("sample", stack.pop()); stack.checkInvariant(); // Delete from empty stack stack.delete(1); stack.checkInvariant(); }
}
When you run this test, you'll quickly see that we need to add some range checking!
There was 1 error: 1) testEmpty(TestMyStack)InvariantException:
next_index out of range: -1 for stack length of 100 at MyStack.checkInvariant(MyStack.java:31) at TestMyStack.testEmpty(TestMyStack.java:16)
It's much easier to find and fix this sort of error here in a simple testing environment instead of buried in a real application.
Almost any indexing concept (whether it's a genuine integer index or not) should be extensively tested. Here are a few ideas to get you started:
*
Start and End index have the same value *
First is greater than Last *
Index is negative *
Index is greater than allowed *
Count doesn't match actual number of items *
…
Reference
What things does your method reference that are outside the scope of the method itself? Any external dependencies? What state does the class have to have? What other conditions must exist in order for the method to work?
For example, a method in a web application to display a customer's account history might require that the customer is first logged on. The method pop() for a stack requires a non- empty stack. Shifting the transmission in your car to Park from Drive requires that the car is stopped.
If you have to make assumptions about the state of the class and the state of other objects or the global application, then you need to test your code to make sure that it is well-behaved if those conditions are not met. For example, the code for the microprocessor-controlled transmission might have unit tests that check for that particular condition: the state of the transmission (whether it can shift into Park or not) depends on the state of the car (is it in motion or stopped).
public void testJamItIntoPark() {
transmission.shift(DRIVE); car.accelerateTo(35); assertEquals(DRIVE, transmission.getGear()); // should silently ignore transmission.shift(PARK); assertEquals(DRIVE, transmission.getGear()); car.accelerateTo(0); // i.e., stop car.brakeToStop(); // should work now transmission.shift(PARK); assertEquals(PARK, transmission.getGear());
}
The preconditions for a given method specify what state the world must be in for this method to run. In this case, the precondition for putting the transmission in park is that the car's engine (a separate component elsewhere in the application's world) must be at a stop. That's a documented requirement for the method, so we want to make sure that the method will behave gracefully (in this particular case, just ignore the request silently) in case the precondition is not met.
At the end of the method, postconditions are those things that you guarantee your method will make happen. Direct results returned by the method are one obvious thing to check, but if the method has any side-effects then you need to check those as well. In this case, applying the brakes has the side effect of stopping the car.
Some languages even have built-in support for preconditions and postconditions; interested readers might want to read about Eiffel in Object-Oriented Software Construction [Mey97].
Existence
A large number of potential bugs can be discovered by asking the key question "does some given thing exist?".
For any value you are passed in or maintain, ask yourself what would happen to the method if the value didn't exist—if it were null, or blank, or zero.
Many Java library methods will throw an exception of some sort when faced with non-existent data. The problem is that it's hard to debug a generic runtime exception buried deep in some library. But an exception that reports "Age isn't set" makes tracking down the problem much easier.
Most methods will blow up if expected data is not available, and that's probably not what you want them to do. So you test for the condition—see what happens if you get a null instead of a CustomerRecord because some search failed. See what happens if the file doesn't exist, or if the network is unavailable.
Ah, yes: things in the environment can wink out of existence as well—networks, files' URLs, license keys, users, printers—you name it. All of these things may not exist when you expect them to, so be sure to test with plenty of nulls, zeros, empty strings and other nihilist trappings.
Make sure your method can stand up to nothing.
Cardinality
Cardinality has nothing to do with either highly-placed religious figures or small red birds, but instead with counting.
Computer programmers (your humble authors included) are really bad at counting, especially past 10 when the fingers can no longer assist us. For instance, answer the following question quickly, off the top of your head, without benefit of fingers, paper, or UML:
If you've got 12 feet of lawn that you want to fence, and each section of fencing is 3 feet wide, how many fence posts do you need?
If you're like most of us, you probably answered "4" without thinking too hard about it. Pity is, that's wrong—you need five fence posts as shown in Figure 5.1 on page 57. This model, and the subsequent common errors, come up so often that they are graced with the name "fence post errors."
It's one of many ways you can end up being "off by one;" an occasionally fatal condition that afflicts all programmers from time to time. So you need to think about ways to test how well your method counts, and check to see just how many of a thing you may have. Image from book Figure 5.1: A Set of Fence posts
It's a related problem to Existence, but now you want to make sure you have exactly as many as you need, or that you've made exactly as many as needed. In most cases, the count of some set of values is only interesting in these three cases:
1.
Zero 2.
One 3.
More than one
It's called the "0–1-n Rule," and it's based on the premise that if you can handle more than one of something, you can probably handle 10, 20, or 1,000 just as easily. Most of the time that's true, so many of our tests for cardinality are concerned with whether we have 2 or more of something. Of course there are situations where an exact count makes a difference—10 might be important to you, or 250.
Suppose you are maintaining a list of the Top-Ten food items ordered in a pancake house. Every time an order is taken, you have to adjust the top-ten list. You also provide the current top-ten list as a real-time data feed to the pancake boss's PDA. What sort of things might you want to test for?
*
Can you produce a report when there aren't yet ten items in the list? *
Can you produce a report when there are no items on the list? *
Can you produce a report when there is only one item on the list? *
Can you add an item when there aren't yet ten items in the list? *
Can you add an item when there are no items on the list? *
Can you add an item when there is only one item on the list? *
What if there aren't ten items on the menu? *
What if there are no items on the menu?
Having gone through all that, the boss now changes his mind and wants a top-twenty list instead. What do you have to change?
The correct answer is "one line," something like the following:
public int getMaxEntries() {
return 20;
}
Now, when the boss gets overwhelmed and pleads with you to change this to be a top-five report (his PDA is pretty small, after all), you can go back and change this one number. The test should automatically follow suit, because it uses the same accessor.
So in the end, the tests concentrate on boundary conditions of 0, 1, and n, where n can—and will—change as the business demands
Time
The last boundary condition in the CORRECT acronym is Time. There are several aspects to time you need to keep in mind:
*
Relative time (ordering in time) *
Absolute time (elapsed and wall clock) *
Concurrency issues
Some interfaces are inherently stateful; you expect that login() will be called before logout(),that prepareStatement() is called before executeStatement(), connect() before read() which is before close(),and so on.
What happens if those methods are called out of order? Maybe you should try calling methods out of the expected order. Try skipping the first, last and middle of a sequence. Just as order of data may have mattered to you in the earlier examples (as we described in "Ordering" on page 48), now it's the order of the calling sequence of methods.
Relative time might also include issues of timeouts in the code: how long your method is willing to wait for some ephemeral resource to become available. As we'll discuss shortly, you'll want to exercise possible error conditions in your code, including things such as timeouts. Maybe you've got conditions that aren't guarded by timeouts—can you think of a situation where the code might get "stuck" waiting forever for something that might not happen?
This leads us to issues of elapsed time. What if something you are waiting for takes "too much" time? What if your method takes too much time to return to the caller?
Then there's the actual wall clock time to consider. Most of the time, this makes no difference whatsoever to code. But every now and then, time of day will matter, perhaps in subtle ways. Here's a quick question, true or false: every day of the year is 24 hours long?
The answer is "it depends." In UTC (Universal Coordinated Time, the modern version of Greenwich Mean Time, or GMT), the answer is YES. In areas of the world that do not observe Daylight Savings Time (DST), the answer is YES. In most of the U.S. (which does observe DST), the answer is NO. In April, you'll have a day with 23 hours (spring forward) and in October you'll have a day with 25 (fall back). This means that arithmetic won't always work as you expect; 1:45AM plus 30 minutes might equal 1:15, for instance.
But you've tested any time-sensitive code on those boundary days, right? For locations that honor DST and for those that do not?
Oh, and don't assume that any underlying library handles these issues correctly on your behalf. Unfortunately, when it comes to time, there's a lot of broken code out there.
Finally, one of the most insidious problems brought about by time occurs in the context of concurrency and synchronized access issues. It would take an entire book to cover designing, implementing, and debugging multi-threaded, concurrent programs, so we won't take the time now to go into details, except to point out that most code you write in most languages today will be run in a multi-threaded environment.
So ask yourself, what will happen if multiple threads use this same object at the same time? Are there global or instance-level data or methods that need to be synchronized? How about external access to files or hardware? Be sure to add the synchronized keyword to any data element or method that needs it, and try firing off multiple threads as part of your test.[2]
[2]JUnit as shipped has some issues with multi-threaded test cases, but there are various fixes available on the net.
Try it Yourself
Now that we've covered the Right-BICEP and CORRECT way to come up with tests, it's your turn to try.
For each of the following examples and scenarios, write down as many possible unit tests as you can think of. Exercises
1.
A simple stack class. Push String objects onto the stack, and pop them off according to normal stack semantics. This class provides the following methods:
StackExercise.java
public interface StackExercise {
/** * Return and remove the most recent item from * the top of the stack. * Throws StackEmptyException * if the stack is empty */ public String pop() throws StackEmptyException;
/** * Add an item to the top of the stack. */ public void push(String item); /** * Return but do not remove the most recent * item from the top of the stack. * Throws StackEmptyException * if the stack is empty */ public String top() throws StackEmptyException; /** * Returns true if the stack is empty. */ public boolean isEmpty();
}
Here are some hints to get you started: what is likely to break? How should the stack behave when it is first initialized? After it's been used for a while? Does it really do what it claims to do?
Image from book
2.
A shopping cart. This class lets you add, delete, and count the items in a shopping cart.
What sort of boundary conditions might come up? Are there any implicit restrictions on what you can delete? Are there any interesting issues if the cart is empty?
public interface ShoppingCart {
/** * Add this many of this item to the * shopping cart. */ public void addItems(Item anItem, int quantity) throws NegativeCountException; /** * Delete this many of this item from the * shopping cart */ public void deleteItems(Item anItem, int quantity) throws NegativeCountException, NoSuchItemException; /** * Count of all items in the cart * (that is, all items x qty each) */ public int itemCount(); /** * Return Iterator of all items * (see Java Collection's doc) */ public Iterator iterator();
}
Image from book
3.
A fax scheduler. This code will send faxes from a specified file name to a U.S. phone number. There is a validation requirement; a U.S. phone number with area code must be of the form xnn-nnn-nnnn,where x must be a digit in the range [2..9] and n can be [0..9]. The following blocks are reserved and are not currently valid area codes: x11, x9n, 37n, 96n.
The method's signature is:
/**
* Send the named file as a fax to the * given phone number. */
public boolean sendFax(String phone,
String filename) throws MissingOrBadFileException, PhoneFormatException, PhoneAreaCodeException;
Given these requirements, what tests for boundary conditions can you think of?
Image from book
4.
An automatic sewing machine that does embroidery. The class that controls it takes a few basic commands. The coordinates (0,0) represent the lower-left corner of the machine. x and y increase as you move toward the upper-right corner, whose coordinates are x = getTableSize().width 1 and y = getTableSize().height - 1.
Coordinates are specified in fractions of centimeters.
public void moveTo(float x, float y); public void sewTo(float x, float y); public void setWorkpieceSize(float width,
float height);
public Size getWorkpieceSize(); public Size getTableSize();
There are some real-world constraints that might be interesting: you can't sew thin air, of course, and you can't sew a workpiece bigger than the machine.
Given these requirements, what boundary conditions can you think of?
Image from book
5.
Audio/Video Editing Transport. A class that provides methods to control a VCR or tape deck. There's the notion of a "current position" that lies somewhere between the beginning of tape (BOT) and the end of tape (EOT).
You can ask for the current position and move from there to another given position. Fast-forward moves from current position toward EOT by some amount. Rewind moves from current position toward BOT by some amount.
When tapes are first loaded, they are positioned at BOT automatically.
AVTransport.java
public interface AVTransport {
/** * Move the current position ahead by this many * seconds. Fast-forwarding past end-of-tape * leaves the position at end-of-tape */ public void fastForward(float seconds); /** * Move the current position backwards by this * many seconds. Rewinding past zero leaves * the position at zero */ public void rewind(float seconds); /** * Return current time position in seconds */ public float currentTimePosition(); /** * Mark the current time position with this label */ public void markTimePosition(String name); /** * Change the current position to the one * associated with the marked name */ public void gotoMark(String name);
}
Image from book
6.
Audio/Video Editing Transport, Release 2.0. As above, but now you can position in seconds, minutes, or frames (there are exactly 30 frames per second in this example), and you can move relative to the beginning or the end.
Image from book
Answers
1.
*
For a brand-new stack, isEmpty() should be true, top() and pop() should throw exceptions. *
Starting with an empty stack, call push() to push a test string onto the stack. Verify that top() returns that string several times in a row, and that isEmpty() returns false. *
Call pop() to remove the test string, and verify that it is the same string. [3] isEmpty() should now be true. Call pop() again verify an exception is thrown. *
Now do the same test again, but this time add multiple items to the stack. Make sure you get the rights ones back, in the right order (the most recent item added should be the one returned). *
Push a null onto the stack and pop it; confirm you get a null back. *
Ensure you can use the stack after it has thrown exceptions.
2.
*
Call addItems with quantity of 0 and itemCount should remain the same. *
Call deleteItem with quantity of 0 and itemCount should remain the same. *
Call addItems with a negative quantity and it should raise an exception. *
Call deleteItem with a negative quantity and it should raise an exception. *
Call addItems and the item count should increase, whether the item exists already or not. *
Call deleteItem where the item doesn't exist and it should raise an exception. *
Call deleteItem when there are no items in the cart and itemCount should remain at 0. *
Call deleteItem where the quantity is larger than the number of those items in the cart and it should raise an exception. *
Call iterator when there are no items in the cart and it should return an empty iterator (i.e., it's a real iterator object (not null) that contains no items). *
Call addItem several times for a couple of items and verify that contents of the cart match what was added (as reported via iterator() and itemCount()).
Hint: you can combine several of these asserts into a single test. For instance, you might start with an empty cart, add 3 of an item, then delete one of them at a time.
3.
*
Phone numbers with an area code of 111, 211, up to 911, 290, 291, etc, 999, 370-379, or 960-969 should throw a PhoneAreaCodeException. *
A phone number with too many digits (in one of each set of number, area code, prefix, number) should throw a PhoneFormatException. *
A phone number with not enough digits (in one of each set) should throw a PhoneFormatException. *
A phone number with illegal characters (spaces, letters, etc.) should throw a PhoneFormatException. *
A phone number that's missing dashes should throw a PhoneFormatException. *
A phone number with multiple dashes should throw a Phone-FormatException. *
A null phone number should throw a PhoneFormatException. *
A file that doesn't exist should throw a MissingOrBadFile-Exception. *
A null filename should also throw that exception. *
An empty file should throw a MissingOrBadFileException. *
A file that's not in the correct format should throw a Missing-OrBadFileException.
4.
*
Huge value for one or both coordinates *
Huge value for workpiece size *
Zero or negative value for one or both coordinates *
Zero or negative value for workpiece size *
Coordinates that move off the workpiece *
Workpiece bigger than the table
5.
*
Verify that the initial position is BOT. *
Fast forward by some allowed amount (not past end of tape), then rewind by same amount. Should be at initial location. *
Rewind by some allowed amount amount (not past beginning of tape), then fast forward by same amount. Should be at initial location. *
Fast forward past end of tape, then rewind by same amount. Should be before the initial location by an appropriate amount to reflect the fact that you can't advance the location past the end of tape. *
Try the same thing in the other direction (rewind past beginning of tape). *
Mark various positions and return to them after moving the current position around. *
Mark a position and return to it without moving in between.
6.
Cross-check results using different units: move in one unit and verify your position using another unit; move forward in one unit and back in another, and so on.
[3]In this case, assertEquals() isn't good enough; you need assert-Same() to ensure it's the same object.