“This is awesome, saves us so much time when performing acceptance testing and ensures we identify issues before our customers do.” - Xhenis Alimadhi - Technical Support Engineer - Harbr

Customer: Digital data software and services provider 

Pain Point: Unable to deliver necessary test coverage

Harbr is a data exchange company providing enterprise customers software that allows them to securely create, visualise, and collaborate on data products. The engineering team was comprised of six developers and three DevOps engineers running an agile development cycle. The engineering team were responsible for producing unit tests with the customer success team running risk-based exploratory tests manually as a sanity check before triggering a release into the customer environment.

The Goal

Harbr identified that a lack of standardisation and governance created a significant risk that delivery and customer success could be jeopardised. They decided to opt for goals over two distinct phases. To begin they wanted to quickly adopt test automation and focus on authoring tests to ensure that risks to delivery and customer success were mitigated. Following this, the strategy would be broadened with the ultimate aim of having a large test coverage, with in-sprint testing, using current resources, and without impacting development.

Challenges

The team understood that there were significant risks and challenges. Their primary drivers for needing to adopt test automation was their inability to: 

• Judge release readiness and produce quality (pre-and post-release)
• Gain visibility on regressions 
• Meaningfully understand when to stop a deployment and/or stop development
• See how the platform performed under load

All of which contributed to significant bottlenecks and breaking points.

Solution: A customer-centric approach

Harbr implemented a pragmatic test and governance approach as part of customer delivery and success. By leveraging the Virtuoso platform they were able to: 

• Create meaningful and repeatable tests from day one 
• Author tests without the need for engineering resources 
• Define and automate a base regression pack of tests that tests the high priority customer use cases

Two weeks to a regression pack

The base regression pack was created within two weeks of using the tool. Therefore mitigating the risks of deploying faulty software. Harbr continued to build from their baseline and have expanded test coverage without ever having to call upon a developer. They have now started to:

• Drastically reduce the time it takes to create and maintain test by reusing journey across multiple variants 
• Ensured the quality of their software by implementing data-driven testing
  

Harbr put the reasons for the success, in part, down to the fact that:

“Virtuoso is very easy to use.”