Making the Business Case for Test Automation

For starters, it's unfortunate we even have to do this. Test-driven development has been a best practice in modern software development for two decades now. Asking developers to build software without automated tests is like asking a carpenter to wait until the end of the project to use a tape measure. Seems silly, right?

That said, putting in place test automation where there is none can get expensive, and there's a growing list of tools and techniques that have been over-hyped and led to disappointment (ever hear of record-and-playback?). So it makes sense that companies want to understand the impact to the business before getting started.

Good news. There has never been an easier time to make the case for doing test automation than there is today. Modern testing frameworks have reduced creation and maintenance costs. Cloud computing has fundamentally changed the economics of running tests in parallel. And DevOps initiatives are leaving teams stuck needing to run more tests in more environments more often.

Here is a guide to help you make your case.

Finding the Benefits of Doing Test Automation

Determine what percentage of your development team's time is spent fixing defects instead of working on new functionality.

This is a big one. Teams with solid test automation practices may find themselves spending as little as 1 - 5% or their time on defects while other teams without those practices may be spending as much as 20-30% of their time on defects (or even more!) How much do you spend on development each year and what would an extra 25% be worth to you?

To be fair, you probably won't regain that full 25% as developers typically end up spending more time helping with test automation. But you can think of that as time spent fixing defects before they impact customers instead of afterwards.

The bottom line is that working on new functionality drives incremental revenue, while working on defects eats away at your net profit margin.

Determine how much time is spent on support and workarounds.

It has been commonly stated in the industry and seems true from experience that the cost of fixing a defect increases the further in the software development process it goes before being fixed. This is especially true for defects that make it into production.

How much time does your support team spend fielding questions from customers? Merging issues that have been reported multiple times? Working with your product team to prioritize fixes? Creating and supporting workarounds?

In larger organizations, there have even been cases where an entire team of temp workers have been called in to manually process records to keep mission critical activities running until software can be fixed. Ouch!

Determine how much time is spent off-hours fixing issues and hot fixing production.

Whether you're in Support, Technical Operations, Development or any other part of the organization, having an alert that interrupts your weekend, important life events, time with kids, or sleep just plain stinks.

One of the greatest benefits for the developer who tests, is they are in more control of keeping their work at work. Developers who work on high quality systems tend to be happier and stay longer, so don't forget about your recruiting expenses and employee retention.

Determine the impact of quality on customer retention and satisfaction.

Quality doesn't typically bring new customers, but it is one of the largest factors in keeping them around. See if your organization is asking customers whether or not they're satisfied and why they're leaving. Ask if you can see what questions they're being asked. Odds are, a lot of them tie back to quality.

Does your organization have any objectives around improving these numbers? Do they have data that ties customer satisfaction back to probability of cancellation? If so, this data could be useful in making your case.

Finding the Costs of Doing Test Automation

Decide who will be writing tests, reviewing tests, and fixing test failures.

More and more teams are opting to have developers be responsible for test automation. There have been many attempts at eliminating the need to write code in order to automate tests, and the results just haven't been there yet.

While approaches like record-and-playback can lead to very quickly having "non-technical" people create an automated suite, these suites become incredibly difficult to maintain as changed are rolled out.

Decide how many tests and how often you plan on running the tests.

Will you be running tests on every commit? Every pull request? Every release? Every deployment? And how many tests will you run? Are you just testing new functionality? Or are you regression testing to make sure that previously delivered features continue to work every single deployment.

Find the break even point for manual versus automated testing.

Manual tests generally have a lower initial implementation cost and a higher operating cost. So as the number of features being tested and the number of times the tests are being run increase, your tests will inevitably hit a point where automation is less expensive than manual testing.

Some manual testing teams cope by cutting back on the scope of testing. This is a dangerous path which inevitably leads to quality issues.

As DevOps initiatives become more pervasive, the number of times the tests are being run is also increasing at an alarming rate. Teams want to run the same tests against more versions of the code and more environments (Feature Environments, Dev, QA, RC, Beta, Staging, Prod, Customer Sites, etc.)

Determine your testing infrastructure and support costs.

Where will the tests run? If your answer is on the developer's workstation, then that answer will only be good for a few weeks. Once you have a lot of tests, you need to have infrastructure specifically for running tests.

Many teams have turned to CI/CD for running their tests. While this is a great way to get the tests off the developer's workstation and to trigger test runs, CI/CD systems were designed to build software and are incredibly lacking when it comes to running large numbers of tests at scale and providing analytics and accountability for the test results.

Setting up CI/CD to run tests at high levels of concurrency, analyze the results, configure testing environments correctly, and provide workflow around reviewing the test results is a whole lot of work. That's why more-and-more teams are turning to Test Automation Platforms like Testery Cloud, which eliminate most of the work required to do all of this.

Getting Help if You Need It

Don't let all of this dissuade you from getting started with test automation. There has never been a better time than now to get started.

And even if you only put a handful of important tests in place that run on every build, that's far better than not having them at all!

If you would like additional help getting started with test automation, we'd be happy to help you put together a business case for test automation at no cost to you.