Avoiding the sunken cost fallacy in testing and knowing when to switch.
Facing up to the inefficiencies of your test automation framework can be difficult. Embracing automation will mean abandoning lots of manual work, but sticking with outdated processes is a case of throwing good money after bad. So, when is the right time to adopt change?
When I was asked to write a blog post on the subject of how and when to change your test automation framework, it got me thinking. Surely, the more important question is this: how do I know if I need to change my test automation framework?
There are lots of blog posts out there covering topics like how to select the right framework in the first place and how to avoid common mistakes in doing so. But I have found very little on how to work out if your current solution is fit for purpose, and what you need to do if it’s not.
That’s where this blog steps in…
Let’s face it, it's never easy to admit that you may have adopted the wrong framework, or that what was the right approach previously simply isn’t best practice anymore.
Maybe you are part of a team that has someone who thinks they know best because of their past experience. For the purpose of this example, we’ll call this particular employee Bob.
Bob is a self-proclaimed framework ninja who has spent many hours – and vast amounts of the test budget – on a framework that is slow and difficult to work with. You may be in a situation where these practices just about work for you, but you know there is a better way of doing things. If this sounds familiar, it’s time to embrace change. More time and money spent on Bob’s framework is delaying the inevitable while wasting resources.
Test automation is changing, and our two friends Artificial Intelligence (AI) and Machine Learning (ML) are playing an increasingly important role. Sooner or later, we are going to have to admit that even with the very best frameworks, it's going to be hard, if not impossible, to match the speed and coverage that autonomous automation can deliver.
So, we’ve decided to move away from our framework, but how do we bring Bob on board? How do we convince him to leave his pride and joy behind? What can we tempt him with?
Well, let’s take the exciting emergence of spidering AI, which uses ML to autonomously write tests for an application. Although this is still very much in the early stages of development, there are already a number of tools that are able to cover large areas of an application in a matter of minutes. These record and capture everything concerning the application, such as network traffic, downloading the HTML of every page, and taking screenshots. All of this information can then be compared to a known state with changes identified – all without having to write a single test. Even Bob’s going to love that!
I am part of an exciting team that has already made huge steps into this world with Virtuoso. Our POCs are returning some really exciting results, in some cases reducing authoring time by at least 80%. We are even able to write executable automation tests before a single line of code has been written by using Natural Language Programming.
This has opened the door to an effective shift-left approach, allowing automated tests to be written at the same time as the acceptance criteria. We are also working on a number of other exciting features that are truly changing the way we approach test automation. This isn’t just changing how we test, it’s changing what we can achieve with testing.
Now is the time to reassess and look at the truly amazing advances that are being made by AI in testing. We have exhausted what can be achieved in automation frameworks, while AI-powered features continue to offer us more ways of reducing manual effort in everything from authoring to maintenance. The earlier you can transition to an intelligent quality assistance platform, the more time and money you will save in the long run.
It’s important to note these advances are about more than just another “bolt-on” tool to your existing framework. This is about ripping up the rule book and truly changing your approach to automation.
How do you know if you need to change your test automation framework? If you are forward-looking and want to adopt the latest technology then it is probably time to take a look at Virtuoso. You won’t regret it!
