527
15 (How to) Profit from Molecular Dynamics-based Ensemble Docking
X-ray structure is found [95]. Korb et al. formulated the ensemble pose prediction
index—a metric to evaluate the ranking of poses for an individual ligand for a set of
ensemble members [121]. A value over 0.5 indicates that the ranking according to
the scoring function identifies the pose with the lowest RMSD to the reference on
the top position of the ensemble results. For the recovery of a near-native pose from
an ensemble without the corresponding protein structure, a higher RMSD tolerance
up to 2.5 Å can sometimes be allowed. This is due to the necessary superposition
of the protein complexes, which might not give a unique solution regarding alternative protein conformations [38]. Another aspect in docking considering protein
flexibility is the recovery of a native complex geometry. Not only could the position
of the ligand but also the conformation of the protein be evaluated in respect to the
experimental structures of alternative conformations. This recovery of the proper
protein geometry of the complex is sometimes performed visually (e.g., see Fig. 4 in
[70], Fig. 10 in [69]). However this part was not yet part of systematic evaluations.
Second, the ranking of different ligands according to their activity is another criterion. Direct correlation of docking scores with experimental data, not being part of
the training, is currently out of reach for docking, as discussed before. Paulsen et al.
classified a set of active ligands into bins of activity classes and evaluated if the
compounds are scored within the bins or neighboring categories. The ranking accuracy by binning is then expressed as success fraction [86]. Categorization of activity
data is a way to deal with the inaccuracies of the experimental data. The lack of consistent activity data might be a reason why this kind of metric is not that frequently
applied. Moreover, the ranking of a compound series is a question of lead optimization where other methods than docking play a major role (see Sect. 15.2.2.5).
In contrast, molecular docking is a classical high-throughput technique with its
major application in virtual screening. Therefore, the discrimination between active
and inactive compounds is a suitable criterion. The enrichment factor is the rate of
recovered active compounds for a given fraction of a database. Its absolute value
depends on the data set composition and this limits the comparison of this measure
for different applications [122]. Displaying the true positive rate (selectivity) versus
the false positive rate (1 - specificity) according to the ranking by docking corresponds to a receiver-operator characteristic curve (ROC) of a docking run [123].
The area under this curve (AUC) is an established measure in the field of virtual
screening which gives 1 for a perfect model and 0.5 for a model with random performance [122]. This metric is sometimes also called the “predictive power” [43].
To focus on early enrichment the logarithmic scaling of the fraction of inactive
compounds or decoys can be chosen [115] or some weighting can be applied, e.g.,
the robust initial enrichment (RIE) and the Boltzmann enhanced discrimination of
ROC (BedROC) [122]. If an ensemble of protein conformations is evaluated, each
conformation has its own receiver operator characteristic and the distribution of the
AUC values seems to be normally distributed [43]. Individual conformations outperform the X-ray structures whereas the majority results in AUC values below the
corresponding X-ray structure. Such a comparison can be useful for the selection of
conformations representing the ensemble for a subsequent docking application (see
Sect. 15.2.1.4) and the validation of an MD-derived ensemble over experimental
15 (How to) Profit from Molecular Dynamics-based Ensemble Docking
X-ray structure is found [95]. Korb et al. formulated the ensemble pose prediction
index—a metric to evaluate the ranking of poses for an individual ligand for a set of
ensemble members [121]. A value over 0.5 indicates that the ranking according to
the scoring function identifies the pose with the lowest RMSD to the reference on
the top position of the ensemble results. For the recovery of a near-native pose from
an ensemble without the corresponding protein structure, a higher RMSD tolerance
up to 2.5 Å can sometimes be allowed. This is due to the necessary superposition
of the protein complexes, which might not give a unique solution regarding alternative protein conformations [38]. Another aspect in docking considering protein
flexibility is the recovery of a native complex geometry. Not only could the position
of the ligand but also the conformation of the protein be evaluated in respect to the
experimental structures of alternative conformations. This recovery of the proper
protein geometry of the complex is sometimes performed visually (e.g., see Fig. 4 in
[70], Fig. 10 in [69]). However this part was not yet part of systematic evaluations.
Second, the ranking of different ligands according to their activity is another criterion. Direct correlation of docking scores with experimental data, not being part of
the training, is currently out of reach for docking, as discussed before. Paulsen et al.
classified a set of active ligands into bins of activity classes and evaluated if the
compounds are scored within the bins or neighboring categories. The ranking accuracy by binning is then expressed as success fraction [86]. Categorization of activity
data is a way to deal with the inaccuracies of the experimental data. The lack of consistent activity data might be a reason why this kind of metric is not that frequently
applied. Moreover, the ranking of a compound series is a question of lead optimization where other methods than docking play a major role (see Sect. 15.2.2.5).
In contrast, molecular docking is a classical high-throughput technique with its
major application in virtual screening. Therefore, the discrimination between active
and inactive compounds is a suitable criterion. The enrichment factor is the rate of
recovered active compounds for a given fraction of a database. Its absolute value
depends on the data set composition and this limits the comparison of this measure
for different applications [122]. Displaying the true positive rate (selectivity) versus
the false positive rate (1 - specificity) according to the ranking by docking corresponds to a receiver-operator characteristic curve (ROC) of a docking run [123].
The area under this curve (AUC) is an established measure in the field of virtual
screening which gives 1 for a perfect model and 0.5 for a model with random performance [122]. This metric is sometimes also called the “predictive power” [43].
To focus on early enrichment the logarithmic scaling of the fraction of inactive
compounds or decoys can be chosen [115] or some weighting can be applied, e.g.,
the robust initial enrichment (RIE) and the Boltzmann enhanced discrimination of
ROC (BedROC) [122]. If an ensemble of protein conformations is evaluated, each
conformation has its own receiver operator characteristic and the distribution of the
AUC values seems to be normally distributed [43]. Individual conformations outperform the X-ray structures whereas the majority results in AUC values below the
corresponding X-ray structure. Such a comparison can be useful for the selection of
conformations representing the ensemble for a subsequent docking application (see
Sect. 15.2.1.4) and the validation of an MD-derived ensemble over experimental