How a professional service provider was able to execute automated end-to-end tests in under 10 minutes that previously needed 3 days of manual testing.
Advances in test automation aren’t just helping teams in their regular day-to-day work. They’re offering rapid solutions to immediate problems that simply weren’t possible with old fashioned Quality Assurance.
Over the last two decades we’ve taken big strides forward in the way we develop software. Continuous Delivery and DevOps have given us new tools and fresh ideas to develop pipelines that deliver quality software much more frequently than before. But over the same period testing has remained stubbornly resistant to change. It’s still the major bottleneck in most software delivery life cycles due to the necessity and reliance on manual processes. At least, it was.
Automation is driving a revolution in testing, so much so that we’re now able to talk about a new paradigm in QA - Quality Assistance.
Intelligent Quality Assistance is able to tackle problems that look impossible to the manual tester. The speed at which we’re now able to automate exploration, mapping, test creation and execution opens up a world of new possibilities for QA teams.
The automation and intelligence in Virtuoso means that, unlike manual frameworks, it’s capable of providing rapid solutions to mission critical problems. Let’s consider an example....
One of our customers, found themselves in the following situation. As you read through, try to imagine how you would have responded, what tools you might have used, etc. Would you have made the launch date?
They were 8 days away from launching an MVP to selected customers. At this late stage, the testing agency hired to get the application ready for the launch called to say they couldn’t complete the work in time.
What’s more, they weren’t able to hand over anything meaningful for others to build on. There was no specific testing strategy, test cases, or scenarios. A large number of end-to-end workflows in the application weren’t working and there are a lot of issues with UI/UX. There was zero automation.
What do you do? Not so long ago you would have had no choice but to postpone the release. Not any more.
So, how are we going to be ready for launch in 8 days? The application needs an intelligent, highly automated solution to have any chance of meeting the deadline. Old fashioned Quality Assurance and manual frameworks just don’t have enough time to get the work done. This is where the automation and intelligence in Virtuoso can offer a solution.
With zero setup the first thing Virtuoso can do in a situation like this is to give you a head start by authoring automated test cases before we even see a line of code in the application. These were dry run once we had access to the application and saved a huge amount of time.
Virtuoso’s NLP engine and test data management features also meant that all of the scenarios could be automated very quickly using simple English and no code. And the tests were executed across browsers without any configuration. This gave the customer the ability to rapidly execute continuous cycles of testing enabling them to build, test, fix, retest and release within a very short window.
How did Virtuoso perform compared with a manual framework? We measured it to find out...
Virtuoso automated 4 modules of the application in 4 days. The expectation was that we would only be able to automate 1 module.
Virtuoso was able to execute 21 end-to-end scenarios and 26 negative scenarios in under 7 minutes. The same job would have taken a manual tester A full 24 hours, or 3 working days.
All in-scope journeys were automated in 4 days with a total execution time of 6 minutes and 30 seconds. The same exercise would have taken at least 2 weeks with a Selenium based framework, which would’ve meant no MVP launch for our customer.
The intelligent automation of testing has arrived. Intelligent Quality Assistance is not only changing how we test, it’s also changing what we can achieve with testing. Find out how Virtuoso can transform your pipeline - maybe even your whole business - with a free demo from one of our experts.
