There are several aspects that a Test Manager has to consider before a testing tool can be selected for purchase and deployment. A Test Manager /QA Manager is responsible for ensuring that all testing tools meet these criteria:

• Enhance value of work done by testing team
• Give positive Return On Investment (ROI)
• Have real and prolonged benefit

To this end, the Test Manager must perform cost-benefit analysis before a tool is sourced or built. The ROI analysis should include recurring as well as one-time expenses, which may be measured in terms of money, time, resources or risks covered.

Table of contents

1. One time costs of using tools
2. Recurring costs of tools
3. Advantages of using tools
4. Risks associated with tools

Return On Investment (ROI) on a software testing tool is the difference between the total cost incurred over the lifetime of the tool (including cost of hardware infrastructure, software licenses, one time or recurring costs listed below, training, people cost etc) versus the business value derived from the usage of the tool (like those listed under tool advantages like reduction in test implementation and management effort, cost savings from decline in human error, savings from test automation etc).

It is difficult to derive the exact ROI that can be achieved since some of the benefits could be intangible and the calculation of savings achieved in future, are estimated values.

In the long run, a tool should ideally help reduce the cost of quality. This may not be true for all tools since there could be cases where a tool is required regulatory or compliance purposes.

One time costs of using tools

These are some of the one time costs:

• Specifying requirements which the tool needs to fulfill in order to achieve the teams goal
• Multiple tool evaluations and vendor selection
• Acquiring the tool by purchasing, customizing, or developing
• Conducting on-boarding training
• Integrating with additional tools
• Acquiring required hardware and software for supporting the tool

Recurring costs of tools

Some of the recurring costs are:

• Tool ownership – support and licensing fees, maintenance, continuous training, maintaining tool generated artifacts
• Using in different environments by porting it
• Customizing for future requirements
• Optimizing quality and process improvement to use the tool efficiently

Every tool has an opportunity cost associated with it, which includes time required in activities like obtaining, managing, training, using, etc. This opportunity cost must be considered by the Test Manager to allocate the required resources well in time.

Advantages of using tools

It is necessary for the Test Manager to consider the advantages of using a tool. Some of the benefits could be:

• Decrease in work which is repetitive
• Decline in cycle time for testing (Example – Run regression test using automation)
• Decline in costs incurred for test execution
• Rise in some testing types like automatic regression testing
• Decline in human errors, like more accurate comparisons, less invalid data generation, correct data entry etc.
• Reduced effort to get testing information from reports
• Use of testing assets like test cases, scripts and data again
• More testing that could not be done without tools, like load testing, performance testing etc
• Enhanced testers’ status and that of the testing team, as they demonstrate better understanding, knowledge and use of advanced tools

Risks associated with tools

It should be noted that all tools have risks associated with them and all of them may not be able to provide benefits that offset the risks. The risks associated with a testing tool have been covered in the syllabus for Foundation Level.

Test Managers must think about these risks when deciding ROI:

• Is the organization mature enough to put the tool to optimum use
• Maintenance of artifacts generated/used by the tool when the application or software being tested is changed. For example: Automation script may be auto generated or written manually in order to use an automation tool to test an application. If that application functionality is modified, the script may have to be updated accordingly.
• Business value associated with testing may be diminished as Test Analysts’ contribution decreases. Example – If only automated testing is performed, efficiency of defect detection may decrease.

Testing teams generally use a combination of test tools. So, return on investment (ROI) is also a combination of every tools being used. As the tools must interact with each other and share information, the organization must have a long term strategy for test tools.

Now that we have understood the costs and risks associated with tools, we will explore some of the parameters that influence selection of testing tool in the next topic.

Other popular articles:

• How to select a testing tool? Open Source, Vendor Tools & Custom Development
• ISTQB Advanced Level Test Manager Study Material
• What are the risks or disadvantages of using the testing tools?
• How to manage software testing tool lifecycle and tool metrics?
• What is cost of quality in software testing?