Several techniques are available for Risk Based Testing (RBT). Many of these are very informal. For example, in some informal methods, tester investigates risks to quality while doing exploratory tests. This may drive the whole testing process but also focus more on probability of a defect occurring rather than its impact.

Also, it may not take into consideration the input from cross-functional stakeholders. These tests also run the risk of being subjective and relying on experience and expertise of the tester. So, they are unable to take full advantage of risk based testing techniques.

Table of contents

1. Requirements and risk
2. Risk identification and analysis techniques
3. Stakeholders role in risk management
4. Closure activities in risk based testing

In order to get full advantage of RBT techniques while minimizing costs, testers and test managers use lightweight approach to risk based testing.

Such approaches combine flexibility offered by informal techniques with consent building capacity found in approaches which are formal.

Some risk based testing techniques are given below:

• Product Risk Management (PRisMa)
• Pragmatic Risk Analysis and Management (PRAM)
• Systematic Software Testing (SST)

Some of the additional characteristics they possess are:

• They have matured over a period of time inside organizations where efficiency was the most important parameter, due to testers experience with risk based testing techniques.
• They depend heavily upon inclusion of wide cross-section of people from all stakeholder groups, including business as well as technical aspects, in early stages of risk detection and analysis. They are most effective if included during the initial phase of the project.
• At this stage they are able to reduce the quality risks significantly and work products of risk analysis may impact design specification as well as implementation.
• The output generated is usually in a matrix or tabular form, which can easily act as the foundation for planning the test and creating test cases.
• This also impacts the later test monitoring and analysis.
• It also makes reporting about residual risks easier.

Requirements and risk

It must be noted that some techniques like Systematic Software Testing can be used only after requirement specifications are available to act as input. This ensures that all the requirements have been covered in risk based testing.

However, requirement specifications may not include all potential risks, especially the non-functional ones. It is the responsibility of Test Managers that these risks do not remain neglected.

• If good requirements and requirements that are prioritized are part of the requirements specification, a strong relationship between requirement priority and risk levels is observed.
• PRAM and PRisMa use a judicious combination of risk based and requirements based strategies.
• They primarily use requirements specifications as input but can use the stakeholders inputs as well.
• Risk detection and analysis process can also be used to create an agreement between the stakeholders concerning the right test approach to be adopted.
• But the stakeholders need to allocate time for participating in the group discussions or one-one-one sessions.
• If enough numbers and groups of stakeholders do not participate, this may cause gaps in risk analysis.
• Also, stakeholders may not have the same opinion all the time regarding the risk levels.

So the group leader needs to guide the discussion in such a way that a consensus is reached amicably. The leader must possess all the qualities of an expert moderator to do this successfully.

Risk identification and analysis techniques

Lightweight techniques of risk analysis are quite like the formal techniques. They highlight the technical or business risks by weighing upon the probability of a risk occurring and factors affecting this probability.

However, the only two factors used by these techniques are – likelihood of a risk and its impact. Also, they use simplistic qualitative judgements and scales to come to a decision.

This lightweight approach makes the techniques flexible, applicable to a wide range of industries, accessible to teams with varied experience and expertise level.

In case of more formal and heavyweight approach, test manager can use any of the following risk identification and analysis techniques:

• Cost of exposure – here, three pieces of information are gathered about each risk to quality. First, probability of risk, in percentage, of total number of risks. Second, price to be paid for each loss, if it happens during production. Third, price to be paid for testing each failure.
• Hazard analysis – it tries to detect the hazards associated with each risk
• Quality Function Deployment (QFD) – It tests the consequences of risks that occur because the requirements gathering team failed to correctly understand the requirements of the users or customers.
• Fault Tree Analysis (FTA) – Here the actual failures that were observed during the complete development lifecycle are put through rigorous root cause analysis till all the causes / reasons are known.
• Failure Mode and Effect Analysis (FEMA) – FEMA has many variants too. Here the potential risks to quality, their probable causes and effects are ascertained. Then, a rating for detection, priority and impact is also given.

The technique which is to be utilized and the level of formality for performing risk based testing is decided by considering many factors like project specifications, processes to be adopted and product to be delivered.

For instance, an informal technique like exploratory method is suited to quick fix or patch testing. However, in case of Agile approach, analysis of quality risks must be completely integrated into each iteration at its onset. The risks must also be documented and recorded as part of customer story.

For projects where safety and outcome is critical, risk based testing must be formality as well as documentation intensive.

The technique chosen also decides the inputs required, processes to be followed and outputs obtained.

Some examples of inputs used in risk based testing include:

• Project specifications
• Stakeholders experience
• Data from previous projects

Some examples of processes in risk based testing are:

• Identifying risks
• Assessing risks
• Controlling risks

Some examples of outputs in risk based testing are:

• List of risks to quality
• List of risks to project
• List of risks to testing
• Risk levels for each risk
• Optimum distribution of resources for testing
• Shortcomings in input documents
• Points to be considered for each risk

Stakeholders role in risk management

Having the best group of stakeholders during risk detection and evaluation is essential for accomplishing risk based testing successfully. This is because each stakeholder has his or her own perception of a quality product, their priority items, areas of importance, depending on their category.

There are two categories of stakeholders – business and technical.

End users, customers, help desk personnel, technical support executives, operations staff, etc. are the business stakeholders. They have knowledge about the users and customers and hence understand their concerns and expectations. So they can assist in detecting business risks and their impact.

Product developers, database administrators, network administrators, system architects, etc. are the technical stakeholders. They know about the undocumented techniques (or simply a specific sequence of steps) that can be used to cause failure in a software. So they can identify technical risks and probability of their occurring.

Some of the stakeholders like Subject Matter Experts (SME) can possess both technical and business view. This happens when they have knowledge about both business and technical aspects because of their work profile.

During risk identification, the list of risks can get really long as each stakeholder shares his own perception of a risk item.

It is unnecessary to argue over whether a risk should be included in the list because if even one person thinks of an item as potential risk, it is definitely a risk. What is important is to reach an agreement over level of that risk.

If we take the example of lightweight test techniques, giving a ranking to each risk listed is essential piece of the whole procedure. Once the ranking is finalized, all stakeholders must use the same scale. The ranking is assigned after the group agrees over it.

Engaging the stakeholders in the analysis of quality risks is advantageous to the Test Manager. In case of projects with incomplete requirement specifications, stakeholders can provide guidance, thereby improving the capability of risk detection teams.

This is because both requirement specifications and stakeholder inputs are being used in the analysis process, making it comprehensive.

Closure activities in risk based testing

After the risk based testing is complete, teams must evaluate their success level at the time of test closure.

To do this, they must answer the following questions:

• Was the percentage of risks that are important, more than the less important ones?
• Were the important weaknesses detected in the initial stages of test execution?
• Was the team capable of explaining the risks detected to the stakeholders?
• If the team left out any tests, did they have lower risk level as compared to the risks that were tested?

These questions must be answered by the whole team using predefined metrics.

If the testing was successful, the answer to all these questions must be yes.

Test Managers must strive to establish processes that improve upon the metrics used to answer these four questions and improve the risk analysis process efficiency.

The manager also needs to consider the correlation among the predefined metrics, critical goals accomplished by the testing team and management behavior with respect to test outcomes.

In the next topic we will discuss Test selection techniques

Other popular articles:

• What is Product risk in software testing?
• What is Risk based testing?
• What is test status report? and How to report test status?
• What is Experience- based testing technique?
• What is Risk analysis?