Quality is Adherence to Specifications

April 18th, 2017

Some of us take the testing of software products seriously; perhaps too seriously.

It is often the case that a software product may respond in an adverse manner when presented an input it would never receive in 'real life'. Should we test for this?

While an enumeration of all possible input values within a given range would certainly comprise a complete set of boundary tests would it not be more efficient to only test a representative set. For example, given 'latitude' as an input we would just test (for valid inputs) -180.00, +180.00, -179.00, +179.00, 0.00, -1.00, +1.00, etc. rather than a complete enumeration of values ranging from -180.00 to +180.00.

So one might argue the efficient testing of a software product should simply attempt to verify the product conforms to its specifications.

We can even simplify this process further by providing a set of inputs and expected output for each testable function (or testable unit) of the product.

So we might say a valid login test is defined as posting http://someurl with an email address and password and verifying the returned value contains the phrase 'Welcome Joe'. This roughly translates to the User Story 'I would like to be able to log in a user'.

This same test could be used to verify both the positive and negative functionality of the application being posted, if for example we tried to login in and it failed before we created the joe account this would be expected and then if we tried again after we created the joe account, in which case it should pass.

This 'test driven' approach allows the customer to drive the requirements at each stage and to a degree the development schedule. It also reduces the amount of over planning sometimes associated with longer term projects.

But perhaps more importantly if the requirements are created in this manner where they simply provide a set of inputs and expected outputs for each testable unit then the tests are automatically created as a byproduct of this process so no additional resource is required to test the product and even the tests themselves may be automatically generated.