Attention to details, a curse for testers!

Attention to details, a curse for testers!

Any common user of an application can spot a bug. It does not matter how technically proficient and educated they are, it does not matter what background they are coming from, once they are bugged by system’s behavior, they will mark it as a bug, and stop using the product.  This small change in the cycle of universe can affect your business, profits, and you will end up empty handed in the end.

Testers do the same experiment, only difference is that they do it in a controlled environment. They test and explore your applications as customers, as high end technicians. They also encounter the same bugs, which can be discovered by the customers. They do it earlier, so the damage is minimal and can be fixed. But they are also humans. They can also make mistakes, and they do. It’s a curse which they deal with day in and day out.

Samuel the tester:

Samuel is working on a mobile application as a black box tester. It is a smart phone application which can control an air conditioner’s temperature, fan speed and modes. It is called “Smart AC Remote”. The application has a simple list of requirements as:

1. Once launched the Smart AC Remote runs in the background, and can be activated from the notifications menu.
2. Once activated, it will display the current temperature, fan speed, and modes on one single screen, and user can adjust this as per their own needs.
3. A user can save the settings of the application, and once the application is relaunched, the settings will be applied to the AC unit as per the preferences.
4. The application, however allow users to adjust the temperature to new settings.
5. Initial values for AC temperature and Fan speed can be set.
6. The temperature maximum value can be set to 33 Degree Celsius, and minimum to 16 Degree Celsius.
7. The Fan speed is set with maximum and minimum limits of 5 degrees either way, 1 being slowest and 5 being the fastest.
8. The setting for Mode includes, Fan Only, Cool, Dry, and Eco.
9. These all settings can be set to initial values and once the system is offline the settings are restored to default values.
10. On reconnection, the system will check for initial values and then resets itself to new values.

At the Boundaries:

With maximum and minimum values in his hand, and the preference inputs to play around, Samuel, started to think on Boundary Test the system. To do this he needs to input values which can touch the boundary. He starts off with the Temperature.

He started from the scratch by downloading the application, and while doing that he noticed, that the progress bar showed the download, is not visible, however, the system does showed “completed” message once it was downloaded. Observation recorded.

One Step at a time:

On the second lag, he taps on the application, and accessed the initial value settings. Here he entered all the values as Zero and checked if the system is generating any warning or not, and it was. So he moved forward and tried to break the upper boundary. He entered highest number in the temperature field, which allowed 2 digit entry only, it did not give him the warning, and also showed the converted temperature in Fahrenheit as well. He noted the anomaly, but this was actually the smoke, so he need to test it further, next he tapped save. The system generated the warning message of “temperature entered is too high” and did not exit, the sanity worked, everyone is saved.

He moved forward.

Since the initial values are now set, Samuel can play with the input values in temperature and fan speed, and to find bugs he will work around the boundaries. This is an irony of software development that bugs are found at the boundaries.

The hidden bug!

The maximum value of temperature is 33, and minimum is 16. What Samuel now needed to do was to break this boundary and see if system stops him from doing so. He increased the temperature 1 digit at a time, and reached 33 degrees. Using his tester instinct to break perception, Samuel tried once more to increase the value, the number did not change, he tapped the temperature increase button 5 times, but no change occurred.

Okay, so the boundary has restricted Samuel from breaking the threshold, let’s see if it does the same when the minimum values pose the same behavior. But, when Samuel tapped decrease temperature button, to his surprise, the temperature jumped to 39 degrees Celsius. That’s a bug definitely. So he note that down, but what caused it? How does temperature jumped to 6 degrees out of the boundary? It’s something which can be encountered by any normal user of the AC remote control, but for Samuel who is a software tester the context has different angles.

Go Deep

Samuel pulled up the calculation worksheet on the computer. He accessed the log entries of the application, which were in a text file, and was provided by the developers. He checked he could see the last calculated entry, 39 degrees, and he could also see the last unchanged entry as 33 degrees. There was no other entry, except, his empty taps on the increase temperature.

So where was the bug hidden? Samuel now have to switch his thinking pattern. He has to move to the known spectrum to unknown spectrum of context.

What’s known to Samuel is the actual boundary of the values and that the boundary can be breached. But what’s unknown to Samuel is why that error has occurred. He looked at the log again. The error is hidden within the activity where the value reached the threshold of 33 degrees, and the value jumped to 39 degrees. In between these two, there were few “no activity” entries of temperature increase command, when Samuel tried to jump that boundary.

The last entry, where the value jumped the boundary was when Samuel decreased the temperature.

That’s It! He thought of himself. This decrease command caused some internal logic glitch, which made this temperature boundary breached. Now Samuel has to think as a developer.

Think like Developers:

So what variables are involved here?

1. Temperature Initial Value
2. Temperature exiting value
3. Command Value
4. If / Else condition
5. Print command

Considering the behavior, the application works fine till the boundary is reached, it even stopped the user to go any further, but something happens when the decreased function is invoked.

Internally, the logic is invisible for Samuel. So he need to use the calculations and pseudo scripting to get to the bottom of this.

Apparently, what’s happening is that the developer is not initializing the values of the temperature increment counter, when it reached 33 degree limit. On each of Samuel’s attempt to increase the temperature, the developer confidently did not displayed that on screen, but on backend, the counter kept adding up the values. When Samuel, decreased the temperature, the counter with its current value i.e. 33 + 6, put the temperature to 39 Degrees. Hence BUG!

Testers are not common users:

For any common user, it would be easier to detect, and can also cause damage to the AC unit, as the temperature can go beyond extreme limit, if the Smart Remote is used excessively. The common user story ends here, but a tester’s story starts from there.

The way a detailed tester put up their findings, affects the chain of solution delivery from the very bottom of logic design to the very top of a satisfied client. It involves deep analysis, technical know-how, and understanding of logic. It also required domain knowledge and expertise, and finally a descriptive, to the point, and accurate report for developers.

So next time when you ask what testers do, be prepared of what you get in return!