7.7 Do Not Forget to Define Performance, Success and Failure Criteria
99
The performance criteria is the set of criteria for which the system is
designed/created and does not necessarily correspond to the success/failure criteria necessary for the risk assessment. When performing a risk assessment it is indeed
paramount to understand the metric (the “viewing angle”, e.g., corporate, investor,
regulators, public) of the success/failure criteria. For instance, production, maintenance, energy use, business interruption, health and safety, environmental, legal and
social impacts, share value, financials, etc., can all be valid viewing angles (failure “dimensions”), but for a specific risk assessment addressing the concerns of
a particular stakeholder the success/failure criteria may differ from the corporate
performance criteria. This is why it is important that the hazard and risk register
allows drilling from different angles to evaluate risks.
7.8 Do Not Use Common Practice Matrix Approaches,
PIGs, FMEAs
The prior chapters have already shown the limitations of these common practice
approaches and their bad use, and the specs have clearly shown that they simply do
not make the cut anymore.
A recent paper entitled “The Risk of Using Risk Matrices” (Thomas et al. 2014)
shows that oil and gas are also victims of risk matrix. The paper analyses the
widespread use of Probability Impact Graphs (PIGs, risk matrix) in oil and gas.
It comes to very similar conclusions to those reached by other academics and
practitioners and ourselves.
However, we are very aware that most companies, dam owners and operators may
already have PIGs, risk matrices (indexed or qualitative), etc., at hand.
7.8.1 “Classic” Risk Matrices Deficiencies
Risk-Acceptance Inconsistency
The regions depicted in common practice PIGs feature arbitrary stepped borders
which generally bear no relation with actual corporate or societal risk tolerance.
Thus those risk matrices lead to misleading decision-making support.
Range Compression
Risk Matrix and their probabilities, Consequences (Losses), risk indexes using scores
to mimic expected-loss calculations do not reflect actual “distances between risks”,
i.e., specifically, the difference in their expected loss. Below you can see Consequences (Losses) split into five classes. Million dollars are used as a metric for
Losses. In the usual log-scale (Fig. 7.4a) the intervals seem equally spaced. However, when displaying the same values in decimal scale (Fig. 7.4b) we can see a
significant “range compression”.
99
The performance criteria is the set of criteria for which the system is
designed/created and does not necessarily correspond to the success/failure criteria necessary for the risk assessment. When performing a risk assessment it is indeed
paramount to understand the metric (the “viewing angle”, e.g., corporate, investor,
regulators, public) of the success/failure criteria. For instance, production, maintenance, energy use, business interruption, health and safety, environmental, legal and
social impacts, share value, financials, etc., can all be valid viewing angles (failure “dimensions”), but for a specific risk assessment addressing the concerns of
a particular stakeholder the success/failure criteria may differ from the corporate
performance criteria. This is why it is important that the hazard and risk register
allows drilling from different angles to evaluate risks.
7.8 Do Not Use Common Practice Matrix Approaches,
PIGs, FMEAs
The prior chapters have already shown the limitations of these common practice
approaches and their bad use, and the specs have clearly shown that they simply do
not make the cut anymore.
A recent paper entitled “The Risk of Using Risk Matrices” (Thomas et al. 2014)
shows that oil and gas are also victims of risk matrix. The paper analyses the
widespread use of Probability Impact Graphs (PIGs, risk matrix) in oil and gas.
It comes to very similar conclusions to those reached by other academics and
practitioners and ourselves.
However, we are very aware that most companies, dam owners and operators may
already have PIGs, risk matrices (indexed or qualitative), etc., at hand.
7.8.1 “Classic” Risk Matrices Deficiencies
Risk-Acceptance Inconsistency
The regions depicted in common practice PIGs feature arbitrary stepped borders
which generally bear no relation with actual corporate or societal risk tolerance.
Thus those risk matrices lead to misleading decision-making support.
Range Compression
Risk Matrix and their probabilities, Consequences (Losses), risk indexes using scores
to mimic expected-loss calculations do not reflect actual “distances between risks”,
i.e., specifically, the difference in their expected loss. Below you can see Consequences (Losses) split into five classes. Million dollars are used as a metric for
Losses. In the usual log-scale (Fig. 7.4a) the intervals seem equally spaced. However, when displaying the same values in decimal scale (Fig. 7.4b) we can see a
significant “range compression”.