Test Purpose and Planning

Type: Rules
Sources: Doug Hubbard (How to Measure Anything, Chapter 4)


SW testing is a kind of measurement: it's purpose is to reduce uncertainty about the quality of the product.
Note: For improved clarity i added examples in the respective notes. Relate testing to the simple task of replacing a broken board on a fence.


We need to find answers to the following questions before we test, i.e measure:

R1: What is the decision the test is about to support?

Sometimes teams test because a standard says so, and not because they understand what will be decided - probably by others - as a result of the test.
The Fence: We measure the height of the old board to decide how long the new one needs to be.

R2: What is the observable definition of the product or its quality, in terms of results?

Sometimes teams test without a clear requirements specification. So what do they test against? In many cases the test cases (if any) are the first written statement of the requirements, really. You need to find out what quality means to the many stakeholders of the product, before you start to measure.
The Fence: the product we need to cut out of wood has certain dimensions. So the product is defined by these dimensions (and many more attributes).

R3: How, exactly, does the measurement matter to the decision being asked?

Sometimes teams test without a clear understanding of how the test results inform the decision, or the decision maker. If the test matters at all, there must be some observable effect on decisions or behavior. If the decision is already made, or if there's no potential behavior change as a result of the test, then don't test.
The Fence: if the new board is already cut then try it, don't measure again.

R4: How much do you already know about the quality of the product?

Sometimes teams test just because the project plan says so, although they already know the product is of inferior quality. Maybe a code inspection has already produced that information and thereby has significantly reduced the uncertainty. Valuable time and money is wasted if we conduct our preplanned test.
The Fence: if the fence conforms to an industry standard you can probably buy another board as a ready-made component. Why measure?

R5: What is the real value of the information being produced by the test?

Sometimes teams test without knowing what testing is for. It is for reducing the uncertainty about the product's quality. It is not for finding bugs.
The Fence: Consider dismounting the rest of the fence. Have you heard about that new tax on fences? (If you think a fence tax is ridiculous, read about window tax).


Measure twice, cut once. But check if the product test really matters.

Related Pages

Where to go from here? Link other pages in bullet list.

rating: 0+x

If you want to know more about a topic, simply tag the article with a 'morePlease' or 'examplesPlease' tag!


Unless otherwise stated, the content of this page is licensed under Creative Commons Attribution-ShareAlike 3.0 License