5.3.5 Internal Validation
The next step is to validate the models. This can be divided into internal and external
validation. Internal validation is considered necessary, but not sufficient, for the
demonstration of a suitable model [82]. External validation must also be carried out
prior to application of the model. A variety of internal validation methods are
available [81]. Leave-one-out (LOOCV) or leave-many-out cross-validation
(LMOCV) are among the most used in asymmetric catalysis QSSR. In LMOCV or
LOOCV, a number (M) or one (O) data point is left out as a test set and the rest of the
data is used as a training set. The measure of fit of the resultant models, the predictive
squared correlation coefficient, q
2 , is calculated (Eq. 6). Because there are many
possible ways to split the data when M > 1, fixed iterations of LMOCV are
commonly used within an affordable computational time. As a consequence, not
all possible combinations of data splitting are evaluated. This means the q
2 from
non-exhaustive LMOCV may not be the same for different observers depending on
how the data is split. This is not a problem for LOOCV. Tropsha’s recommended
value for LOOCV q
2 is >0.6 [81]. There are many other methods for QSSR
validation such as bootstrapping [86] and y-randomization [87].
q
2
¼ 1:0 À
P n
i¼1
y i À b y i
ð
Þ
2
P n
i¼1
y i À y tr
ð
Þ
2
ð6Þ
where y i , b y i , y tr ¼ measured, predicted and averaged dependent variable and n ¼ number of data points in the training set.
5.3.6 External Validation
A set of data that has not been used to train the model is called an external testing set. It is
highly recommended that the external testing set must be representative of the whole
range of both dependent and independent parameters. The definition of q
2
ext is similar to
Eq. 6. The term y from the testing data or, alternatively, the training set has been used;
however, the latter has been shown to give too optimistic values in some case studies
[88]. Tropsha published multiple recommended criteria for predictive power of a model
based on external testing data such as R
2
ext > 0:6 and q
2
ext > 0.5 [81].
5.3.7 Outliers
Outliers may reflect experimental error or model failure. In the latter scenario, this
may indicate the involvement of a different process, such as competing side reactions or a change in mechanism. This is why it is crucial to make sure data is
Ligand Design for Asymmetric Catalysis: Combining Mechanistic and. . .
177
The next step is to validate the models. This can be divided into internal and external
validation. Internal validation is considered necessary, but not sufficient, for the
demonstration of a suitable model [82]. External validation must also be carried out
prior to application of the model. A variety of internal validation methods are
available [81]. Leave-one-out (LOOCV) or leave-many-out cross-validation
(LMOCV) are among the most used in asymmetric catalysis QSSR. In LMOCV or
LOOCV, a number (M) or one (O) data point is left out as a test set and the rest of the
data is used as a training set. The measure of fit of the resultant models, the predictive
squared correlation coefficient, q
2 , is calculated (Eq. 6). Because there are many
possible ways to split the data when M > 1, fixed iterations of LMOCV are
commonly used within an affordable computational time. As a consequence, not
all possible combinations of data splitting are evaluated. This means the q
2 from
non-exhaustive LMOCV may not be the same for different observers depending on
how the data is split. This is not a problem for LOOCV. Tropsha’s recommended
value for LOOCV q
2 is >0.6 [81]. There are many other methods for QSSR
validation such as bootstrapping [86] and y-randomization [87].
q
2
¼ 1:0 À
P n
i¼1
y i À b y i
ð
Þ
2
P n
i¼1
y i À y tr
ð
Þ
2
ð6Þ
where y i , b y i , y tr ¼ measured, predicted and averaged dependent variable and n ¼ number of data points in the training set.
5.3.6 External Validation
A set of data that has not been used to train the model is called an external testing set. It is
highly recommended that the external testing set must be representative of the whole
range of both dependent and independent parameters. The definition of q
2
ext is similar to
Eq. 6. The term y from the testing data or, alternatively, the training set has been used;
however, the latter has been shown to give too optimistic values in some case studies
[88]. Tropsha published multiple recommended criteria for predictive power of a model
based on external testing data such as R
2
ext > 0:6 and q
2
ext > 0.5 [81].
5.3.7 Outliers
Outliers may reflect experimental error or model failure. In the latter scenario, this
may indicate the involvement of a different process, such as competing side reactions or a change in mechanism. This is why it is crucial to make sure data is
Ligand Design for Asymmetric Catalysis: Combining Mechanistic and. . .
177