Case Study
Building Test Automation for True Shift-Left
Virtuoso QA
Published on
See Virtuoso in action
Book a walkthrough on your applications and your workflows. Bring a requirement, a user journey, or a brittle Selenium script, and watch the loop run on something you recognise.
JATO Dynamics
JATO Dynamics was founded in 1984 and provides the world's most timely, accurate, and up-to-date information on vehicle specifications, pricing, sales, and registrations. As part of a modernization program, JATO Dynamics decided to redevelop its core vehicle intelligence platform as a web application. JATO had its own team developing the back end while a third-party software house developed the front end.
The Challenges
Developing and testing the new application presented several challenges. It was trying to build the platform from the ground up with no reusable or pre-existing user journeys. The back-end and front-end were being developed separately, creating inevitable issues around handovers and feedback loops.
The frequency and velocity of changes to the back end were affecting the UI and the mobile requirements were completely undefined. Furthermore, time was limited and there weren’t many automation skills or any prospect of increasing the size of the team.
The Solutions
By adopting Virtuoso and taking advantage of its easy-to-use and intuitive interface due to its natural language model, JATO was able to start realizing the benefits right away.
User flows and wireframes allowed them to begin building automation immediately so that it could run in tandem with development. Testing was able to keep pace with the frequency of changes thanks to Virtuoso QA’s rapid test authoring.
There weren’t any defined requirements for testing across mobile, but Virtuoso QA was still able to execute all authored journeys across mobile devices, providing insights for future development. The shortage of automation skills wasn’t a problem either. Virtuoso is so easy to use that with only minimal training JATO was soon able to take ownership of its own automation.
Even with programming skills, many other tools, especially open-source ones, programmers may not be able to start immediately as they lack a programming language specific to the tool they are using and they seek help from online forums. Virtuoso QA allows users to set it up, schedule work, start using it immediately, track progress, and recap work done previously on the project. The JATO test team was able to quickly build their own in-sprint automation with no requirements for frameworks, infrastructure, or up-front costs. UI regressions were easily identified using Virtuoso’s snapshot feature.
From the beginning of the project, JATO was able to achieve true shift-left as 100% of the user journeys were automated before the application was built. Having all of the test cases authored by Virtuoso’s NLP engine means that JATO has dynamic documentation and full version control of the application. They are also able to shift right by executing future journeys in production.
The Results
How did JATO’s use of Virtuoso translate into numbers? Let’s have a closer look at the key metrics.
0 hours frameworks setup or configuration
1 sprint to go from zero automation to full in-sprint automation
2 days to onboard the team and set up for success.
42 hours total authoring time, this includes any post-deployment tidy-up work
100% of in-scope user journey flows automated before the code was written
80% of the user flows executed successfully to the point of application failure needing only minor adjustment
21 application snapshots delivering iterative automated behavioral insight and visual testing
400+ journey executions - end-to-end user flows to test functionality
5 Minutes - Execution time for all 25 E-2-E user flows
In Summary
JATO successfully adopted Virtuoso QA on day one to shift testing left by automating full feature testing before any of the applications had been built enabling them to release weeks faster.
It went from zero automation to in-sprint automation in just two weeks and with only a couple of days of onboarding and training, with no need to spend time on frameworks of infrastructure.
With Virtuoso QA, JATO has improved testing efficiency and coverage by utilizing a fully collaborative quality assurance platform that has made testing more accessible, faster, and easier, while also giving them a blueprint for dynamic web testing success.
