3.3.4 Database Creation
The power of collecting data and aggregating it into databases and deriving
knowledgebases has been clearly outlined above. The community is at a point
where technological advances in crystallographic data generation mean there is
potential for crystal structures to be generated at a much faster rate, which would
in turn drive exploitation of databases. However, the process of growing a database
involves collecting, cleaning, validating and adding value to data – the latter
involving adding some chemical meaning to a crystal structure represented in CIF.
Until recently this was a very time-consuming and manual process. DeCIFer [156]
was therefore a necessary piece of software, developed to assign chemistry to
crystallographic data, and has been used as part of the process of building the
CSD since 2013. The process includes the detection of bonds, selection of a polymer
unit where necessary, resolution of disorder and assignment of bond types and
formal charges. Each assigned structure is accompanied by an estimate of the
reliability that the chemistry has been correctly assigned and, in cases where the
reliability is low, pointers towards the likely source(s) of errors. As well as information provided in an individual CIF, the collection of data in the CSD is used in
various ways. Element-element pair distances from the CSD are used to aid in the
detection and assignment of bonds. A Bayesian, probabilistic approach is used to
improve the reliability and correctness of structure assignment and in some cases,
existing chemical diagrams in the CSD are used in the generation of diagrams for
incoming structures.
Adding value to data at the point of ingestion is only one aspect of this process –
cleaning and validation are also key. In preparation for inclusion into the CSD, a
piece of software, PreQuest [157], has been in use both at the CCDC and also by
some crystallographers as the method of curating structures into a database. A recent
successor to PreQuest, CSD Editor, enables Scientific Editors at the CCDC more
easily to curate entries for the CSD and forms part of the database management
infrastructure at the CCDC known as CSD-Xpedite. As a ‘self-service’ solution that
enables crystallographers to better manage depositions, a new service ‘My Structures’ [158] was developed. My Structures partly addressed the data bottleneck and
reduces burdens on both the part of the depositor and the database provider, and it
enables users to view their previously deposited data, visualise structures, check the
status of depositions and download and retrieve their datasets. My Structures and
CSD Editor also provide the basis for users to create and maintain databases of their
own structures. This means that in-house or proprietary crystal structures can be
archived and searched either independently or in conjunction with the CSD.
3.3.5 Biological Integration
Small molecule crystal structures and organic ligands can be used to support research
in areas of biology. Since the early 1990s, data derived from the CSD has been used
Leading Edge Chemical Crystallography Service Provision and Its Impact on. . .
117
The power of collecting data and aggregating it into databases and deriving
knowledgebases has been clearly outlined above. The community is at a point
where technological advances in crystallographic data generation mean there is
potential for crystal structures to be generated at a much faster rate, which would
in turn drive exploitation of databases. However, the process of growing a database
involves collecting, cleaning, validating and adding value to data – the latter
involving adding some chemical meaning to a crystal structure represented in CIF.
Until recently this was a very time-consuming and manual process. DeCIFer [156]
was therefore a necessary piece of software, developed to assign chemistry to
crystallographic data, and has been used as part of the process of building the
CSD since 2013. The process includes the detection of bonds, selection of a polymer
unit where necessary, resolution of disorder and assignment of bond types and
formal charges. Each assigned structure is accompanied by an estimate of the
reliability that the chemistry has been correctly assigned and, in cases where the
reliability is low, pointers towards the likely source(s) of errors. As well as information provided in an individual CIF, the collection of data in the CSD is used in
various ways. Element-element pair distances from the CSD are used to aid in the
detection and assignment of bonds. A Bayesian, probabilistic approach is used to
improve the reliability and correctness of structure assignment and in some cases,
existing chemical diagrams in the CSD are used in the generation of diagrams for
incoming structures.
Adding value to data at the point of ingestion is only one aspect of this process –
cleaning and validation are also key. In preparation for inclusion into the CSD, a
piece of software, PreQuest [157], has been in use both at the CCDC and also by
some crystallographers as the method of curating structures into a database. A recent
successor to PreQuest, CSD Editor, enables Scientific Editors at the CCDC more
easily to curate entries for the CSD and forms part of the database management
infrastructure at the CCDC known as CSD-Xpedite. As a ‘self-service’ solution that
enables crystallographers to better manage depositions, a new service ‘My Structures’ [158] was developed. My Structures partly addressed the data bottleneck and
reduces burdens on both the part of the depositor and the database provider, and it
enables users to view their previously deposited data, visualise structures, check the
status of depositions and download and retrieve their datasets. My Structures and
CSD Editor also provide the basis for users to create and maintain databases of their
own structures. This means that in-house or proprietary crystal structures can be
archived and searched either independently or in conjunction with the CSD.
3.3.5 Biological Integration
Small molecule crystal structures and organic ligands can be used to support research
in areas of biology. Since the early 1990s, data derived from the CSD has been used
Leading Edge Chemical Crystallography Service Provision and Its Impact on. . .
117