Coincidence and Its Impact on Testing

Apart from being the last day of 2011, yesterday was a regular day. I chose to watch Schindler’s List. For me, a typical experience of watching a good meaningful movie includes a follow-up web surfing on the subject, its director(s), actors, related movies etc. Yesterday was no different. I opened the movie’s Wikipedia page which opened the way for a long web surfing session. I went to various websites/wikipedia pages to know more about Schindler and other people who did similar courageous attempts at saving lives. Throughout this session, one common word appeared on all pages – “Nazi” – as all of these had to do something with the Nazi concentration camps.

Here comes the coincidence! During this session of surfing, I got a comment posted on my website titled “Grammar Nazi Correction” about a typo error in a free Python book which I have co-authored with Chetan. (I came to know about this via the notification email). Another thing that came to my mind was that in the past 2-3 days I have got this suggestion from atleast 5 different individuals although the book was released about an year back. This in turn made me recall a SPAM comment which had happened a couple of years back when someone copy pasted one of the genuine comments (by Ajay Balamurugadas ) on my website – an attempt at manipulating the default wordpress setting that once an email is trusted, it can be used to post anything without moderation.

What was my reaction? The precise use of the word “Nazi” in my current web browser session ruled out blind spamming. I became worried. This can happen only if  either some malware local to my computer is doing so or someone sniffing the traffic is playing a prank.

I didn’t think at all beyond this yesterday. Today when I woke up, out of interest, I picked up this thread again for investigation. I opened up a new browser tab to validate the address mentioned in the signature ( we must never click the links directly in the email, if suspicious ). It was good. How does it matter?; a malware/prankster could still use a genuine address. Till this point, I was still not focusing on the title itself. I was so biased about not looking at it! Almost unwillingly, I did a google search for the phrase “Grammar Nazi Correction” and I could have kicked myself, if only I were able to manage to do so. Following is one of the definitions I found about “grammar nazis”:

Grammar Nazi: an elitist who gets off on correcting others’ grammar and spelling because it makes him feel superior.

The person who sent the comment was being polite while sending a suggestion for grammatical correction and I, instead of appreciating that, labeled him as malware!

I was amazed to see how coincidence can create a strange bias about investigating what we observe. Club it with some past experience and you get a recipe for failure.

In testing, we are taught to observe the software behavior while we are interacting with it. One lesson which I learnt from this experience is:

Treat every observation as a coincidence with respect to your actions. Then rule out the coincidence with careful investigation.

In simple words,

Do not assume that all software behavior that you sense is a result of your actions.

My first natural reaction to an event in the object I am interacting with is to associate that event with my own actions. Almost every time, I spend more energy in proving the same to myself rather than an unbiased investigation where the starting point is dis-associating the actions from the observations and then relating them if found so. It is also important to learn why that event got triggered if not my actions.

Many problems do not get reported because the tester is not able to see the problem again as s/he is trying to reproduce it based on his/her actions. On the other side, some problems get reported but are never reproduced based on the actions provided. Going by the above argument on coincidence, there is a big possibility that the problem being reported was not an outcome of the actions mentioned in “steps to reproduce”. There were other reasons which could be totally unrelated to the actions. It could have something to be do with an event in operating system, other softwares installed, a batch process  in software under test, a trigger in the database because some other event, which could have happened irrespective of what your actions were .

That’s what I had to discuss about coincidence as of now.

As a side note, and a lighter one, it’s important to understand whether the recipient knows about the phrases that we use. One of my friends ate less, thinking he has to pay for food and beverages when the waitress of airline lounge answered ‘no’ to his question – ‘Is everything on the house?’. Of course he had certain assumptions about waitress’s English and he paid for it (well, not literally:-)). Another example. Indians are known to translate the phrases, sayings ( Hindi: kahavat/ muhavare) to English ( or other language of communication ). E.g. “haath khade kar dena”, literally: to lift one’s hands, meaning – “to refuse to do something when one was being expected to do it, esp at a critical point. I’ve seen not just once or one person, using the phrase “s/he lifted his/her hands” while communicating in English even to non-Indian counter-parts ( who have probably become so used to it, that they just nod their head in response rather than breaking their head to understand the meaning ). When we use words to say more than what they literally mean, we must consider the cultural and language aspects of the receiver(s).

Thanks to Jen Heilemann, who posted that comment which led me to writing the first blog post of 2012.

Happy New Year!