Implementing an "automation first" testing strategy

iClicker is a subsidiary of Macmillan learning, providing digital products and services for higher and lifelong education around the globe.

Key to iClicker making a successful transition to automated testing was avoiding baggage that often comes with it. In iClicker's piloting of automation, they experienced real challenges due to complexity, usability, and technical debt.

iClicker needed a test automation solution which fully integrates with agile ways of working and tooling.‍

Challenges

• An over-reliance on technical engineers to automate tests
• Lead time to automation would be too long
• Dynamic nature of iClicker development would create the growing problem of test maintenance

‍Solution

iClicker and Virtuoso ran a pilot taking iClicker's 62 regression scenarios as the use case. The success criteria centred around speed, ease of use, agility, and scalability. iClicker's team of manual QAs were given access to Virtuoso following a two-hour platform orientation.

‍Results

• Time needed to create automation was reduced by 88% from 340 hours to just 40 hours
• Time needed to execute automation was reduced by 82% from 2.75 hours to less than 30 mins
• iClicker regression has now reduced from 128 hours to 30 mins
• Cost per manual execution was reduced by 99.2%, from $1,280 to $10
• Cost per automated test execution was reduced by 63.6%, from $27.50 to $10

‍

In Summary

Above and beyond just the time and cost savings Virtuoso provided, iClicker loved how quick and easy it was to get started and to use. The speed and simplicity of getting up and running coupled with the multiple browsers/OS support enabled them to achieve a level of coverage they simply wouldn't have achieved in the short amount of time that they were using Virtuoso.

Want to find out more? Then watch Virtuoso and MacMillan leanings joint webinar, Getting Fit for the Future of Test Automation.