14 Predictive Modeling of Tox21 Data
293
14.4 Discussion/Notes
The Tox21 10K collection has been screened against over 50 assays yielding highquality datasets. When applied to predicting in vivo toxicity, models built with the
current set of Tox21 assay activity profiles showed reasonable but less than ideal
performance for most in vivo toxicity endpoints and appeared to be less predictive
than the models built with chemical structures [17]. On the other hand, the assay
data-based models performed markedly better in predicting human toxicity, though
just a few endpoints, than animal toxicity, whereas the structure-based models did
not show this species selectivity. As all of the Tox21 assays screened so far are cellbased assays constructed from human cells or cell lines, species difference may be a
significant contributing factor to the less than ideal performance of the activity-based
models on predicting animal toxicity endpoints. New drugs that passed animal testing
are often known to fail in human clinical trials because of lack of effect or unexpected
toxicity [43]. Other studies also showed that animal data only predicted human
outcomes approximately half of the time [44]. Our analysis revealed that animal
toxicity-based models did not perform significantly better than assay activity-based
models in predicting human adverse drug effects, providing further evidence for this
species-related issue [19]. This exemplifies the need to have in vivo human toxicity
data, i.e., clinical toxicity data presently not readily available to the public, in order
to better assess the predictive value of the human in vitro assay data. Nevertheless,
combining activity data with structure information significantly improved the model
performance for most of the in vivo endpoints, which served as a validation of
the value of the in vitro assay data when applied to in vivo toxicity prediction. The
model performance in addition highlighted the importance of data quality. The Tox21
in vitro qHTS data showed good reproducibility (Table 14.1) [17]. The in vivo data
used in modeling were also evaluated for reproducibility, and a significant positive
correlation was found between the reproducibility of the in vivo endpoint and the
performance of the model built for that endpoint, suggesting that the performance of
in vivo toxicity prediction models could be further improved if better quality in vivo
data were available.
As the current Tox21 assays focused primarily on nuclear receptor signaling and
stress response pathways, all aspects of biology involved in toxic response are not
covered sufficiently, indicating the need to expand the coverage of the biological
space by including assays that target additional pathways relevant for toxicity. As
surrogates of assay data, adding drug target information significantly improved the
performance of Tox21 in vitro assay data-based models in predicting human ADEs
[19]. These DTAs have good coverage of the drug target space known in the literature
and can be considered validated experimental or assay data, and thus produced good
predictive models even with a small selected subset that provides sufficient expansion
of the biological space. In addition to limited target space coverage, the current
assay data used for modeling is primary HTS data without further validation and
thus undoubtedly confounded with noise and assay artifacts. These results again
highlight the importance of data quality and selecting the right assays. Validated DTA
293
14.4 Discussion/Notes
The Tox21 10K collection has been screened against over 50 assays yielding highquality datasets. When applied to predicting in vivo toxicity, models built with the
current set of Tox21 assay activity profiles showed reasonable but less than ideal
performance for most in vivo toxicity endpoints and appeared to be less predictive
than the models built with chemical structures [17]. On the other hand, the assay
data-based models performed markedly better in predicting human toxicity, though
just a few endpoints, than animal toxicity, whereas the structure-based models did
not show this species selectivity. As all of the Tox21 assays screened so far are cellbased assays constructed from human cells or cell lines, species difference may be a
significant contributing factor to the less than ideal performance of the activity-based
models on predicting animal toxicity endpoints. New drugs that passed animal testing
are often known to fail in human clinical trials because of lack of effect or unexpected
toxicity [43]. Other studies also showed that animal data only predicted human
outcomes approximately half of the time [44]. Our analysis revealed that animal
toxicity-based models did not perform significantly better than assay activity-based
models in predicting human adverse drug effects, providing further evidence for this
species-related issue [19]. This exemplifies the need to have in vivo human toxicity
data, i.e., clinical toxicity data presently not readily available to the public, in order
to better assess the predictive value of the human in vitro assay data. Nevertheless,
combining activity data with structure information significantly improved the model
performance for most of the in vivo endpoints, which served as a validation of
the value of the in vitro assay data when applied to in vivo toxicity prediction. The
model performance in addition highlighted the importance of data quality. The Tox21
in vitro qHTS data showed good reproducibility (Table 14.1) [17]. The in vivo data
used in modeling were also evaluated for reproducibility, and a significant positive
correlation was found between the reproducibility of the in vivo endpoint and the
performance of the model built for that endpoint, suggesting that the performance of
in vivo toxicity prediction models could be further improved if better quality in vivo
data were available.
As the current Tox21 assays focused primarily on nuclear receptor signaling and
stress response pathways, all aspects of biology involved in toxic response are not
covered sufficiently, indicating the need to expand the coverage of the biological
space by including assays that target additional pathways relevant for toxicity. As
surrogates of assay data, adding drug target information significantly improved the
performance of Tox21 in vitro assay data-based models in predicting human ADEs
[19]. These DTAs have good coverage of the drug target space known in the literature
and can be considered validated experimental or assay data, and thus produced good
predictive models even with a small selected subset that provides sufficient expansion
of the biological space. In addition to limited target space coverage, the current
assay data used for modeling is primary HTS data without further validation and
thus undoubtedly confounded with noise and assay artifacts. These results again
highlight the importance of data quality and selecting the right assays. Validated DTA