14 Predictive Modeling of Tox21 Data
289
Fig. 14.1 Performance
distribution of in vivo
toxicity prediction models
built with different datasets
measured by AUC-ROC
built for the human endpoints performed significantly better than the models of the
animal (mouse/rat/rabbit) toxicity endpoints. The compound structure-based models
showed overall better performance than the activity-based models with an average
AUC-ROC of 0.78 (Fig. 14.1), underlying the ongoing need to further expand the
battery of in vitro assays. However, the performance of the structure-based models did not show any species difference. Compared to the activity-based models,
the structure-based models performed significantly better for the mouse/rat/rabbit
toxicity endpoints, but not as significantly for the human toxicity endpoints. Combining the compound activity and structure data resulted in significantly better models
(average AUC-ROC of 0.84), than the models built with activity or structure alone
(Fig. 14.1). Similar to the structure-based models, the species difference between the
model performances disappeared.
14.3.2 Expanding Biological Space Coverage Improves
Human Toxicity Prediction
A subset of the Tox21 10K library is composed of approved drugs [20], and human
adverse effect data are publicly available for some of these drugs. To address the
issue of species differences and re-evaluate the utility of the Tox21 in vitro human
cell-based assay data, we collected adverse drug effect (ADE) data, a common manifest of human toxicity, and rebuilt models to predict this type of toxicity [19]. For
comparison purposes, we also conducted the first meta-analysis to evaluate the performance of animal in vivo toxicity data in predicting human adverse outcomes in
parallel with in vitro assay data [19]. Animal toxicity datasets do not seem to have
a clear advantage over human cell-based data in predicting human in vivo effects
based on these modeling results (Fig. 14.2). Models built with in vivo animal toxi-
289
Fig. 14.1 Performance
distribution of in vivo
toxicity prediction models
built with different datasets
measured by AUC-ROC
built for the human endpoints performed significantly better than the models of the
animal (mouse/rat/rabbit) toxicity endpoints. The compound structure-based models
showed overall better performance than the activity-based models with an average
AUC-ROC of 0.78 (Fig. 14.1), underlying the ongoing need to further expand the
battery of in vitro assays. However, the performance of the structure-based models did not show any species difference. Compared to the activity-based models,
the structure-based models performed significantly better for the mouse/rat/rabbit
toxicity endpoints, but not as significantly for the human toxicity endpoints. Combining the compound activity and structure data resulted in significantly better models
(average AUC-ROC of 0.84), than the models built with activity or structure alone
(Fig. 14.1). Similar to the structure-based models, the species difference between the
model performances disappeared.
14.3.2 Expanding Biological Space Coverage Improves
Human Toxicity Prediction
A subset of the Tox21 10K library is composed of approved drugs [20], and human
adverse effect data are publicly available for some of these drugs. To address the
issue of species differences and re-evaluate the utility of the Tox21 in vitro human
cell-based assay data, we collected adverse drug effect (ADE) data, a common manifest of human toxicity, and rebuilt models to predict this type of toxicity [19]. For
comparison purposes, we also conducted the first meta-analysis to evaluate the performance of animal in vivo toxicity data in predicting human adverse outcomes in
parallel with in vitro assay data [19]. Animal toxicity datasets do not seem to have
a clear advantage over human cell-based data in predicting human in vivo effects
based on these modeling results (Fig. 14.2). Models built with in vivo animal toxi-