400
B. Hardy et al.
to creative commons licenses, enabling maximum use and update of developments.
Both QSAR and ITS applications will be made available providing cases and practice
examples for review and discussion both online and during the OpenTox conferences
and workshops planned for Europe, USA, and Asian regions. The proposed practices
will include:
• Principle of reproducibility—review propositional principle;
• Practices and specifications for application interoperability layers for data—providing APIs to important public databases in predictive toxicology and supporting
the principle of reproducibility;
• Reproducible Q(SAR) demonstration and evaluation—reproducible workflow for
Q(SAR) model building and applied to published reference predictive models;
• Reproducible integrated testing strategy (ITS) demonstration—reproducible
workflow for ITS and risk assessment of skin sensitization.
Based on the results of the discussions and exercises at the OpenTox conferences
and workshop, we will draft an OpenTox guidance to provide draft best practice
guidance for implementation of the Reproducibility Principles.
The need for reproducibility is increasing dramatically as data analyses becomes
more complex, involving larger datasets and more sophisticated computations. By
building case studies for in silico reproducibility, we aim to encourage scientists
to follow demonstrated examples and publishers to adopt the developed guidelines
as recommendations for future publications. Within OpenTox and related initiatives
such as OpenRiskNet and NanoCommons, we propose to establish propositional
“practices for reproducible in silico computational science” and specific recommendations on applying them to computational toxicology. We will also establish a guideline for data collection (by performing systematic literature review), data curation
(both experimental and structural data and involving outlier detection techniques),
data management (including long-term storage and publication), tracking (workflow
documentation and version control), and licensing of computational tools (giving
preference for open and/or free tools). The two areas initially addressed will be: (a)
the field of (Quantitative) structure activity relationships, aiming to establish reproducibility, complementing existing OECD guidelines on best practices for QSAR
modeling; (b) integrating testing strategies (ITSs) involving a Bayesian approach to
weight of evidence from a combination of in silico models and in vitro testing results.
In both cases (QSAR and ITS), we are implementing demonstration implementations as workflows which are presented online to the OpenTox community for evaluation within a community challenge proposed to enhance and enrich best practices
while simultaneously reproducing published studies. We will identify key studies
published in the last decade; our selection criteria includes: the publications’ impact,
number of citations and its influence in shaping the field of new alternative methods
in safety assessment (voted by a panel of domain experts) as well as the availability
and support of the original authors to collaborate, if possible. We will evaluate and
select case studies for the community challenge, where we will work with community members and the study authors to rehabilitate their published outcomes for
reproducible in silico computations, which will be discussed at an international sci-
B. Hardy et al.
to creative commons licenses, enabling maximum use and update of developments.
Both QSAR and ITS applications will be made available providing cases and practice
examples for review and discussion both online and during the OpenTox conferences
and workshops planned for Europe, USA, and Asian regions. The proposed practices
will include:
• Principle of reproducibility—review propositional principle;
• Practices and specifications for application interoperability layers for data—providing APIs to important public databases in predictive toxicology and supporting
the principle of reproducibility;
• Reproducible Q(SAR) demonstration and evaluation—reproducible workflow for
Q(SAR) model building and applied to published reference predictive models;
• Reproducible integrated testing strategy (ITS) demonstration—reproducible
workflow for ITS and risk assessment of skin sensitization.
Based on the results of the discussions and exercises at the OpenTox conferences
and workshop, we will draft an OpenTox guidance to provide draft best practice
guidance for implementation of the Reproducibility Principles.
The need for reproducibility is increasing dramatically as data analyses becomes
more complex, involving larger datasets and more sophisticated computations. By
building case studies for in silico reproducibility, we aim to encourage scientists
to follow demonstrated examples and publishers to adopt the developed guidelines
as recommendations for future publications. Within OpenTox and related initiatives
such as OpenRiskNet and NanoCommons, we propose to establish propositional
“practices for reproducible in silico computational science” and specific recommendations on applying them to computational toxicology. We will also establish a guideline for data collection (by performing systematic literature review), data curation
(both experimental and structural data and involving outlier detection techniques),
data management (including long-term storage and publication), tracking (workflow
documentation and version control), and licensing of computational tools (giving
preference for open and/or free tools). The two areas initially addressed will be: (a)
the field of (Quantitative) structure activity relationships, aiming to establish reproducibility, complementing existing OECD guidelines on best practices for QSAR
modeling; (b) integrating testing strategies (ITSs) involving a Bayesian approach to
weight of evidence from a combination of in silico models and in vitro testing results.
In both cases (QSAR and ITS), we are implementing demonstration implementations as workflows which are presented online to the OpenTox community for evaluation within a community challenge proposed to enhance and enrich best practices
while simultaneously reproducing published studies. We will identify key studies
published in the last decade; our selection criteria includes: the publications’ impact,
number of citations and its influence in shaping the field of new alternative methods
in safety assessment (voted by a panel of domain experts) as well as the availability
and support of the original authors to collaborate, if possible. We will evaluate and
select case studies for the community challenge, where we will work with community members and the study authors to rehabilitate their published outcomes for
reproducible in silico computations, which will be discussed at an international sci-