474
N. Ya. Golovenko et al.
Cross validation is the statistical practice of partitioning a sample of data into
subsets, such that the analysis is initially performed on a single subset, while the
other subset(s) are retained for subsequent use in confirming and validating the initial analysis. The initial subset of data is known as the training set, while the other
subset(s) are known as validation sets. Two types of cross validation can be used in
QSAR analysis: leave-one-out and leave-group-out cross validation. The latter is a
more severe method for model robustness estimation [50].
Determination coefficient Q
2
calculated in cross-validation terms is the main
characteristic of model robustness. Q
2
is calculated by the following formula:
(14.4)
where Y pred is a predicted value of activity, Y actual is an actual or experimental value
of activity and Y mean is the mean activity value. However, goodness-of-fit or robustness should not be confused with the ability of a model to make predictions [51].
Usage of external validation (test) set is the only way for the estimation of model
predictivity. Thus, a certain fraction of the dataset molecules is removed into a test
set before the modeling process begins (remaining compounds form the training
set). Once a model has been developed, predictions can be made for the test set.
However, even in the case of a beneficial effect, one should be aware that this
may only represent the model’s ability to predict a certain test set. It is important,
therefore, that both training and test sets cover the structural space of the complete
dataset as large as possible.
The most similar or dissimilar compounds as well as those randomly chosen,
taking into account activity variation, are selected for the external set in our studies
[43]. Structural dissimilarity/similarity is obtained for all initial training set molecules on the basis of relevant structural descriptors. In our opinion, use of the whole
set of descriptors generated in the very beginning is not completely correct, because
during QSAR research we are interested not only in structural similarity by itself,
but from the point of view of the investigated properties. Thus, mentioned descriptors selection will help in avoiding some distortions caused by the insignificance of
structural parameters from the initial set for this concrete task.
The targets of the first level are activity prediction or virtual screening. Any
descriptors could be used here, even those hardly interpretable or noninterpretable,
such as different topological indices, informational-topological indices or eigenvalues of various structural matrices. The aims of the second level must include
the interpretability of QSAR models obtained. Only descriptors that have clear
physico–chemical sense (e.g., reflecting parameters of the molecule such as dipole
moment, lipophilicity, polarizability and van der Waals volume) can be used at this
level. Analysis of QSAR models corresponding to this level allows one to reveal
structural factors promoting or interfering the property investigated.
2
2
2
(
)
1
(
)
pred
actual
actual
mean
Y
Y
Q
Y
Y
γ
γ
-
= -
-
∑
∑
N. Ya. Golovenko et al.
Cross validation is the statistical practice of partitioning a sample of data into
subsets, such that the analysis is initially performed on a single subset, while the
other subset(s) are retained for subsequent use in confirming and validating the initial analysis. The initial subset of data is known as the training set, while the other
subset(s) are known as validation sets. Two types of cross validation can be used in
QSAR analysis: leave-one-out and leave-group-out cross validation. The latter is a
more severe method for model robustness estimation [50].
Determination coefficient Q
2
calculated in cross-validation terms is the main
characteristic of model robustness. Q
2
is calculated by the following formula:
(14.4)
where Y pred is a predicted value of activity, Y actual is an actual or experimental value
of activity and Y mean is the mean activity value. However, goodness-of-fit or robustness should not be confused with the ability of a model to make predictions [51].
Usage of external validation (test) set is the only way for the estimation of model
predictivity. Thus, a certain fraction of the dataset molecules is removed into a test
set before the modeling process begins (remaining compounds form the training
set). Once a model has been developed, predictions can be made for the test set.
However, even in the case of a beneficial effect, one should be aware that this
may only represent the model’s ability to predict a certain test set. It is important,
therefore, that both training and test sets cover the structural space of the complete
dataset as large as possible.
The most similar or dissimilar compounds as well as those randomly chosen,
taking into account activity variation, are selected for the external set in our studies
[43]. Structural dissimilarity/similarity is obtained for all initial training set molecules on the basis of relevant structural descriptors. In our opinion, use of the whole
set of descriptors generated in the very beginning is not completely correct, because
during QSAR research we are interested not only in structural similarity by itself,
but from the point of view of the investigated properties. Thus, mentioned descriptors selection will help in avoiding some distortions caused by the insignificance of
structural parameters from the initial set for this concrete task.
The targets of the first level are activity prediction or virtual screening. Any
descriptors could be used here, even those hardly interpretable or noninterpretable,
such as different topological indices, informational-topological indices or eigenvalues of various structural matrices. The aims of the second level must include
the interpretability of QSAR models obtained. Only descriptors that have clear
physico–chemical sense (e.g., reflecting parameters of the molecule such as dipole
moment, lipophilicity, polarizability and van der Waals volume) can be used at this
level. Analysis of QSAR models corresponding to this level allows one to reveal
structural factors promoting or interfering the property investigated.
2
2
2
(
)
1
(
)
pred
actual
actual
mean
Y
Y
Q
Y
Y
γ
γ
-
= -
-
∑
∑