How will Continuous Testing minimize risk to your business? Well you've come to the right place
Business is all about risk. It’s a gamble. You place your bets and hope that you picked the right horse. Of course, it is a lot more sophisticated than what you would find at your local bookmakers. People and organizations de-risk their gamble by making well-informed decisions on their bets. Software testing is one way of doing this. It makes sure you have high enough quality software to meet your users' demands. But it is not that simple; traditional approaches to testing are too slow. This means you have a new risk to consider: time to market. It is all well and good to have quality software, but it may be to little avail if your competitors have stolen a march on you and gotten there first. Continuous Testing is a methodology that simultaneously ensures quality releases and speeds up time to market. It’s a card you should keep, not throw away.
Continuous Testing is the process where automated tests are automatically triggered during a software delivery pipeline. This results in immediate feedback being given as to the business risks associated with releasing the software. It is an approach that is linked to Continuous Integration/Continuous Delivery (CI/CD), where you are always ready to release and, therefore, always testing.
There are several connected drivers for the establishment and adoption of Continuous Testing, but we will start by discussing two of them: increasing complexity of applications and increasing speed at which software is developed. Businesses are left with a dilemma. They are creating ever-more complex software at an evergrowing velocity. Testing has always struggled to keep pace with the development pipeline, so how can testing keep pace with these two pressures? The answer is by associating risk with testing through the use of Continuous Testing.
Having a large level of automated testing may be faster than manual testing, but it will still not keep pace with the modern software delivery lifecycle (SDLC). You may also have a lot of test coverage, but that does not mean you are de-risking your releases. Let’s dig into that with an example, and we will keep it simple with an example application under test (AUT). Before a release, we run three test cases that validate the following three assertions:
Two of them pass and one fails, or, 66% of test cases have passed. This may meet the threshold for release (let’s just say it does for the example), but we do not know how critical the test that fails is (we can guess with this example, but we are keeping it hypothetical). If the "make purchase" case has failed, there is far more risk than if the email invoice to the customer test case has failed. Of course, the customer service department is not going to be very happy, but you get the picture.
For Continuous Testing, you need to attach a level of risk to each test case. Let’s see how we would do with the above example. Risk is always calculated on a 100% basis. The test cases this time would have a level of risk attached to them:
Here, if one test fails, you could have anywhere between 10% and 90% risk coverage. And we know testing everything when being squeezed on time and resources is not going to happen. So you can choose to only run tests that give you 80% risk coverage which will be much less testing than 80% test coverage.
The above is obviously an oversimplification but illustrates how the first step to Continuous Testing is to access which test cases constitute the largest risk. This will ensure that your test suite is not filled with redundancy or bloat and give you real insights into how ready you are to release (which technically should be always).
As the old adage goes, “failure to prepare is to prepare to fail.” And the same is true with Continuous Testing with the planning stage being one of the most important stages. Requirements should be written and ranked with critical business risk at the forefront. This will ensure that:
There is always going to be risk involved in business; it is pretty much the name of the game. There is no such thing as a “dead cert”. But a Continuous Testing approach allows you to calculate that risk while keeping up frequent releases. Or, to put it another way, minimizing risk that your competitors get to market before you. And there is even better news: automated testing tools have gotten better. QA teams are able to author tests before the application is created, author and execute at never-before-seen speeds, and get self-healing tests. But adopting test automation does not happen overnight, and a risk-led approach through Continuous Testing will give you the biggest gains fastest. And though we can never guarantee 100% risk-free software, the speed to test that is now available means you can get pretty close if you follow the right strategy from the outset.
