reproducible. Ways to quantify boundaries to identify outliers by statisticians are
discussed elsewhere [89]. In QSSR studies, Lipnick recommended that outliers lie
3 or more standard deviations away from the mean of residual [90]. Sigman has
calculated the degree of deviation to quantify the error of extrapolation data. A
tolerance of 10–20% error was accepted for inliers [91].
5.3.8 Applicability Domain (AD)
Like all models, regressions capture parts of reality. Therefore, all models are
inherently incorrect but are likely to be useful within a certain limit. Models are
highly dependent upon the inputs. They will break down if we try to predict unseen
data that drastically differ from the input data. This is a common problem around
extrapolation (prediction that falls outside the domain covered by the training data).
‘Domain of applicability’ or ‘applicability domain’ (AD) is a non-rigid limit
between extrapolation and interpolation (prediction that falls within the domain
covered by the training data). In practice, the line between performance of chemicals
within AD and outside is not a sharp one. The change in predictability of a model is a
gradual rather than a sudden drop across AD. AD therefore acts as a guide which
needs to be substantiated by expert judgement. AD helps define a quantitation limit
(where predictions are not quantitative anymore) or define the domain of linearity of
the model (outside this, it is not necessarily linear anymore). Defining an AD with
QSSR should help the end user of the model balance the validity of a predicted
value. Some argue that defining an AD is in fact inappropriate because the utilization
of the model outside the AD isn’t outright invalid, only less reliable. Furthermore,
predictions of compounds within a model AD should still be treated with caution in
certain cases [92]. AD therefore helps the end user to be aware of the approximate
limits of a model, between extrapolation and interpolation, and to make choices
accordingly.
One definition of AD is ‘The applicability domain of a (Q)SAR model is the
response and chemical structure space in which the model makes predictions with a
given reliability’ [57]. One way to define an AD is to give it a descriptive definition
based on the structures and the physiochemical properties of the training dataset and
the observed properties (dependent variables). This serves the purpose above without being too rigid about the exact mathematical formula of the domain. More
quantitative assessments of AD are described elsewhere [57].
5.3.9 When a Model Disappoints. . .
Even when compounds appear to be within the AD, there is no guarantee that the
prediction of such compounds will be reliable. In QSAR, an activity landscape might
contain ‘cliffs’ where a sudden change in activity is observed with subtle structural
changes. In the presence of these activity cliffs, even interpolation can fail. An
equivalence of the notorious activity cliffs seen in QSAR and QSSR is perhaps the
178
R. Ardkhean et al.
discussed elsewhere [89]. In QSSR studies, Lipnick recommended that outliers lie
3 or more standard deviations away from the mean of residual [90]. Sigman has
calculated the degree of deviation to quantify the error of extrapolation data. A
tolerance of 10–20% error was accepted for inliers [91].
5.3.8 Applicability Domain (AD)
Like all models, regressions capture parts of reality. Therefore, all models are
inherently incorrect but are likely to be useful within a certain limit. Models are
highly dependent upon the inputs. They will break down if we try to predict unseen
data that drastically differ from the input data. This is a common problem around
extrapolation (prediction that falls outside the domain covered by the training data).
‘Domain of applicability’ or ‘applicability domain’ (AD) is a non-rigid limit
between extrapolation and interpolation (prediction that falls within the domain
covered by the training data). In practice, the line between performance of chemicals
within AD and outside is not a sharp one. The change in predictability of a model is a
gradual rather than a sudden drop across AD. AD therefore acts as a guide which
needs to be substantiated by expert judgement. AD helps define a quantitation limit
(where predictions are not quantitative anymore) or define the domain of linearity of
the model (outside this, it is not necessarily linear anymore). Defining an AD with
QSSR should help the end user of the model balance the validity of a predicted
value. Some argue that defining an AD is in fact inappropriate because the utilization
of the model outside the AD isn’t outright invalid, only less reliable. Furthermore,
predictions of compounds within a model AD should still be treated with caution in
certain cases [92]. AD therefore helps the end user to be aware of the approximate
limits of a model, between extrapolation and interpolation, and to make choices
accordingly.
One definition of AD is ‘The applicability domain of a (Q)SAR model is the
response and chemical structure space in which the model makes predictions with a
given reliability’ [57]. One way to define an AD is to give it a descriptive definition
based on the structures and the physiochemical properties of the training dataset and
the observed properties (dependent variables). This serves the purpose above without being too rigid about the exact mathematical formula of the domain. More
quantitative assessments of AD are described elsewhere [57].
5.3.9 When a Model Disappoints. . .
Even when compounds appear to be within the AD, there is no guarantee that the
prediction of such compounds will be reliable. In QSAR, an activity landscape might
contain ‘cliffs’ where a sudden change in activity is observed with subtle structural
changes. In the presence of these activity cliffs, even interpolation can fail. An
equivalence of the notorious activity cliffs seen in QSAR and QSSR is perhaps the
178
R. Ardkhean et al.