13 Predicting the Risks of Drug-Induced Liver Injury …
271
the model. The hold-out process was repeated 1000 times to generate training/test
sets pairs. Cross-validation was applied inside the training set to evaluate model
performance. Label permutation testing with the DILI severity annotations randomly
shuffled was applied to check whether the model could generate results better than
random.
The optimized MOA-DILI model employed four assays, i.e., ARE-bla (antioxidant response element), ER-luc-bg1-4e2-antagonist (ERalpha, BG1 cell line), gh3tre-antagonist (thyroid receptor), and PPARG-bla-agonist (peroxisome proliferatoractivated receptor gamma). Furthermore, a prediction accuracy of 0.757 in 5-fold
cross-validation and 0.695 in hold-out testing was observed for the optimized MOADILI model, which is significantly higher than the results obtained from the permutation test (Fig. 13.2b). Moreover, this optimized model has a significantly higher predictive performance than the conventional QSAR model only (Table 13.3), demonstrating the improved predictive power for hepatotoxicity by integrating MOA data
of drugs.
Another modified QSAR model was also developed, namely DILI prediction
systems [72] which aims to translate the post-marketing surveillance information
back to the preclinical stage for improving DILI prediction performance. In DILI
prediction systems model, it is hypothesized that there exists a set of hepato-related
side effects with discriminative power to distinguish between drugs with or without
the risk for DILI. Then, in silico models could be developed for those hepato-related
side effects based on drug’s chemical structure with machine learning algorithms.
Based on SIDER datasets [73], 13 different hepato-related side effects were identified
and corresponding models were developed by using naïve Bayesian classifier in a
single cohesive prediction system. The DILI prediction systems yielded 60–70%
accuracies when evaluated using drugs from different DILI annotations. Furthermore,
it was found that when a drug was predicted as positive by at least three side effects,
the positive predictive value could be boosted to 91%.
13.4 Conclusion
Reliably predicting the risk for DILI in humans is still an unmet need in the research
community [34]. Accurate annotation of DILI risk is vital for the development of
Table 13.3 Overall
Performance of AOPs-DILI
model in training and test set
Model types
5-fold
cross-validation
Hold-out test
MOA-DILI model
0.757 (0.022)
0.695 (0.043)
Conventional
QSAR model
0.658 (0.031)
0.663 (0.04)
Label permutated
model
0.582 (0.042)
0.500 (0.063)
271
the model. The hold-out process was repeated 1000 times to generate training/test
sets pairs. Cross-validation was applied inside the training set to evaluate model
performance. Label permutation testing with the DILI severity annotations randomly
shuffled was applied to check whether the model could generate results better than
random.
The optimized MOA-DILI model employed four assays, i.e., ARE-bla (antioxidant response element), ER-luc-bg1-4e2-antagonist (ERalpha, BG1 cell line), gh3tre-antagonist (thyroid receptor), and PPARG-bla-agonist (peroxisome proliferatoractivated receptor gamma). Furthermore, a prediction accuracy of 0.757 in 5-fold
cross-validation and 0.695 in hold-out testing was observed for the optimized MOADILI model, which is significantly higher than the results obtained from the permutation test (Fig. 13.2b). Moreover, this optimized model has a significantly higher predictive performance than the conventional QSAR model only (Table 13.3), demonstrating the improved predictive power for hepatotoxicity by integrating MOA data
of drugs.
Another modified QSAR model was also developed, namely DILI prediction
systems [72] which aims to translate the post-marketing surveillance information
back to the preclinical stage for improving DILI prediction performance. In DILI
prediction systems model, it is hypothesized that there exists a set of hepato-related
side effects with discriminative power to distinguish between drugs with or without
the risk for DILI. Then, in silico models could be developed for those hepato-related
side effects based on drug’s chemical structure with machine learning algorithms.
Based on SIDER datasets [73], 13 different hepato-related side effects were identified
and corresponding models were developed by using naïve Bayesian classifier in a
single cohesive prediction system. The DILI prediction systems yielded 60–70%
accuracies when evaluated using drugs from different DILI annotations. Furthermore,
it was found that when a drug was predicted as positive by at least three side effects,
the positive predictive value could be boosted to 91%.
13.4 Conclusion
Reliably predicting the risk for DILI in humans is still an unmet need in the research
community [34]. Accurate annotation of DILI risk is vital for the development of
Table 13.3 Overall
Performance of AOPs-DILI
model in training and test set
Model types
5-fold
cross-validation
Hold-out test
MOA-DILI model
0.757 (0.022)
0.695 (0.043)
Conventional
QSAR model
0.658 (0.031)
0.663 (0.04)
Label permutated
model
0.582 (0.042)
0.500 (0.063)