Dipping back into ‘Lessons Learned in Software Testing’ after bit of a break from it, I rediscovered the topics on Applied Epistemology, namely that this is what testing is. It jumped out at me, because I’ve just encountered an example of it in my own testing.
As the book states, one topic within epistemology is ‘meaning and ambiguity in natural language’. Think about that for a minute or two, and I’m sure you’ll easily come up with examples that have affected your testing. My example is below:
The product I was testing holds address data, and as with any product that holds this kind of information, the data can easily become invalid or out of date, and need cleansing or maintenance. Part of this will include marking addresses as ‘Goneaways’. There are a number of ways of doing this, you could either compare your data to a screening file (in the UK we have a GAS, or Goneaway suppression file, for example) or you could rely on the fact that any post to a goneaway address should be returned. Since there’s no point sending out communications to these addresses, but given that legally we need to retain all information that we have for a set period, then it follows that we need a way to mark these addresses without deleting them. They’re not valid addresses for us to send to, and you might even go so far as to label them as ‘Bad’.
Unfortunately, that’s exactly what happened. Somewhere between gathering requirements, and producing code, ‘Goneaway’ got converted to ‘Bad’. Now, a label is just a marker, and if everyone had the same understanding, then this wouldn’t have been a huge problem. It became a problem, because ‘Bad’ was then interpreted to mean structurally incorrect and in need of correction. Of course, a Goneaway’s address is usually structurally sound, and is a valid address in the sense that the postal service can happily locate it, so to mark it as ‘Bad’ – as in incorrect – was not the correct solution to our problem.
‘So what?’ you might be asking at this point – well, the key is in the possible scenarios that would make use of this marker. Imagine that someone else moves into this ‘Goneaway’ address, but that this someone else is also on our database. If we associate this Bad Address to them, then we won’t send out any communications to them. If we unmark the address as ‘Bad’, then we’ll end up sending communications to both the new resident and the previous resident, at the same address.
Marking an address as a Goneaway subtly alters the requirement, as there is nothing wrong with the address itself – we actually want to flag the link between the address and the resident. That way, if the address is reused, then a new link can be created to a new resident, and this new link can remain unmarked. The use of the word ‘Bad’ led to ambiguity, and this ambiguity was successfully picked up by walking through test scenarios based on the original requirements.