11 Applicability Domain: Towards a More Formal …
217
Fig. 11.1 In the context of virtual screening (a) a global accuracy of 83% provides an acceptable
level of confidence across a large number of structures to select from. On the other hand, in the
context of risk assessment (b), the same 83% corresponds to a one in six chances of observing a
potentially lethal outcome. The confidence is therefore perceived as weaker in this latter case
in silico prediction systems, and the Organization of Economic Co-operation and
Development (OECD) has included AD as part of the requirements for (Q)SAR
models [1], the OECD defines AD as follows:
Applicability Domain is the response and chemical structure space in which the model makes
predictions with a given reliability.
Thanks to this formalisation effort, most modern in silico prediction systems
feature a way to identify if a compound is part of their AD or not, and thus provide
a confidence estimate at the individual prediction level. Unfortunately, we observe
a strong heterogeneity in the wide spectrum of methods developed for this purpose.
Existing methods are based on variable definitions of the AD and often rely on
different ways to consider the problem at the source. In their well-structured review
[2], M. Mathea et al. compare the main methods available and their contribution in
formalising AD. Despite the important and valuable effort to tackle the AD problem
[3–15], the lack of a standard definition and the variety of existing implementations,
dramatically reduces the ability to assess a prediction or compare predictions from
different systems.
Although the OECD definition is an important step forward to express the intended
aim of the AD and formalise its scope, it remains vague about practical aspects and
does not provide implementation guidelines. The OECD definition is based on several
difficult concepts and formalising each of these concepts is a challenging task. For
instance:
• When is it valid to use a model to make a prediction?
217
Fig. 11.1 In the context of virtual screening (a) a global accuracy of 83% provides an acceptable
level of confidence across a large number of structures to select from. On the other hand, in the
context of risk assessment (b), the same 83% corresponds to a one in six chances of observing a
potentially lethal outcome. The confidence is therefore perceived as weaker in this latter case
in silico prediction systems, and the Organization of Economic Co-operation and
Development (OECD) has included AD as part of the requirements for (Q)SAR
models [1], the OECD defines AD as follows:
Applicability Domain is the response and chemical structure space in which the model makes
predictions with a given reliability.
Thanks to this formalisation effort, most modern in silico prediction systems
feature a way to identify if a compound is part of their AD or not, and thus provide
a confidence estimate at the individual prediction level. Unfortunately, we observe
a strong heterogeneity in the wide spectrum of methods developed for this purpose.
Existing methods are based on variable definitions of the AD and often rely on
different ways to consider the problem at the source. In their well-structured review
[2], M. Mathea et al. compare the main methods available and their contribution in
formalising AD. Despite the important and valuable effort to tackle the AD problem
[3–15], the lack of a standard definition and the variety of existing implementations,
dramatically reduces the ability to assess a prediction or compare predictions from
different systems.
Although the OECD definition is an important step forward to express the intended
aim of the AD and formalise its scope, it remains vague about practical aspects and
does not provide implementation guidelines. The OECD definition is based on several
difficult concepts and formalising each of these concepts is a challenging task. For
instance:
• When is it valid to use a model to make a prediction?