The GOLD Heuristic – Go One Layer Down

While testing, one of the useful ways to get more ideas for testing as well as breaking free of constraints that prevent you from exercising some test ideas is the GOLD heuristic – Go One Layer Down.

The concept is simple, although from one implementation to another the ground work required might vary. There are two variants of this. In one variant, you have access to the underlying layer to see the exact implementation. In the second variant, you gather information about it by asking questions about the same from the rest of the team, studying similar applications, looking through the various possibilities in which the actual implementation could exist and ruling out the same with careful observations.

It can be used by black box testers as it does not require code level access. Thinking in this manner helps testers in coming up with questions that lead to powerful test ideas.

Let’s discuss a few examples:

• While testing a web application, applying the GOLD heuristic could mean understanding the underlying protocol (mostly HTTP) and how the functionality of the application could be making use of the same. As an example, the moment you know that HTTP is a stateless protocol and the application under test requires a unique business flow be established for an end user, you know that there would some sort of state management in play and one or more of the headers/cookies/hidden variables could be used to depict the same. In such a case you can do a good amount of testing by skipping the web interface altogether and talking at the HTTP layer to the server. This can be done as tool assisted exploratory testing using web proxies like Fiddler/WebScarab/BurpSuite. It can also be automated using tools or custom-built modules that talk at HTTP level. This is a great supplement to the GUI-click/mouse emulation tools commonly used in test automation.

• In contrast to the above example, sometimes you may not have the option of directly talking to the application logic at the underlying layer. The GOLD heuristic stills comes in handy. Let’s take the example of a command line interface. Some common tests which a tester conducts are duplicate switch testing, non-existent switch testing etc. If a tester is able to think how internally the different command lines switches are going to represented, s/he can come up with a precise list of tests. E.g. if the internal data structure that is used to represent the switches is a dictionary, the test for duplicate switches is not as important ( as the dictionary would disallow duplicate keys). Duplicate boolean switch testing would also be purposeless. The duplicate parameter switch testing becomes important as one would want to know whether the second entry is discarded or is used to update the parameter switch option value, in which case one should see the second value overriding the first one.

• Another example of the GOLD heuristic is thinking about queues and servers in the underlying logic. One need not know the exact implementation. Just acknowledging that there are various points in the execution where queuing takes place, a tester can come up with questions like – how can the queue get filled up and what happens then? Is the queue size enough for anticipated usage scenarios? etc.

• The GOLD heuristic also helps a tester in having a high level logical system view of the application under test. When a tester can see a system as a combination of various smaller components, it gives clarity of thought and while communicating with rest of the team, s/he can communicate with precise information or atleast educated guesses about which part of the system might be causing the problem.

I have found this very useful when I need to break the constraints which the top layer puts on exercising test ideas. The web interface, for example, puts such restrictions via the client side scripting that prevent experimenting with various types of inputs; the browser puts a restriction on the URL size; the flow of an application via the browser puts a restriction on playing with the sequence of operations; a local GUI/CLI interface puts a restriction on the sequence of API calls and so on.

You can as well use this heuristic recursively – apply it at the top layer, then the lower one and so on.

Try it out and you might find it useful. Your experience could be different, it’s a heuristic afterall.