OK, so in my last blog I posed a question, and the answers that came back got it right – which I was hoping for, but wasn’t expecting. This leads me to believe that those brave enough to comment (as WordPress tells me that there were a lot more views than comments!) are more logically-thinking than average – or have possibly just heard of the problem before!

In doing a bit of research before writing this half of the post, I realised that this ‘hypothetical’ scenario I’d been taught was actually a real one, from a real quiz show aired in America. The host of that quiz show was named Monty Hall, and in his case, the prizes were goats and a car, rather than money and beans. The underlying puzzle goes even further back – full details can be found on Wikipedia.

The basic answer of course, is that you should take the opportunity to move, every time. You started out with a one in three chance, and Monty gives you the option to switch to a two in three chance.

According to Wikipedia, when this was first discussed thousands of people, many with PHDs, disagreed and insisted that it didn’t really matter whether you switched as the chances were now 50:50. I found the same when discussing this with colleagues. Some others insist that you’re better off staying where you are, although I’ve never really got a rational explanation as to why they thought this.

The reason why it isn’t 50:50 is because the door Monty chose wasn’t chosen randomly. He couldn’t choose the door you were already on, and he couldn’t choose the one with the prize behind it – as otherwise he can’t ask you if you want to switch. Because of that, Monty is limited to choosing the losing door, which fundamentally alters the statistics of the choice you’re then given.

The fact that many people think the choice boils down to 50:50 is another example of cognitive bias – it’s the opposite of the gambler’s fallacy where someone predicting the result of an 11th coin toss where the first ten have all been heads, will generally vote tails as they feel it’s overdue. In reality, the coin is either biased or the chances remain 50:50 – unlike our example above, the previous result has no bearing on future results.

For testers, awareness of these cognitive biases is important. As far as possible, we need to appreciate the actual expected results of our tests, without bias.

One place to go to get this awareness is John Stevenson’s blog, which often looks at psychology and its impact on testing.