2.3 The Effect on Crystallographic Practice . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
2.4 Future Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104
3 The Database Revolution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
3.1 Nature of the CSD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
3.2 The Evolution of the Structural Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
3.3 The Transition to Informatics: Methods and Tools to Leverage Databases . . . . . . . . . 114
3.4 Areas Where Data-Driven Methods Are Making an Impact . . . . . . . . . . . . . . . . . . . . . . . . . 119
4 Closing the Loop and Future Prospects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
4.1 How Is Data Now Driving the Scientific Process and What Is the Future? . . . . . . . . . 121
4.2 How Far Can Single-Crystal Diffraction Structure Analysis Be Developed? . . . . . . . 125
4.3 Conclusions and Challenges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131
Abstract National facilities provide state-of-the-art crystallographic instrumentation and processes and tend to act as an indicator for the direction of a community in
the medium term. There has been a significant step up in terms of instrumentation
and approach in the last 10 years which has driven data generation. This has had a
significant impact on databases – in turn we observe a substantial change in the use
of the Cambridge Structural Database (CSD) from relatively basic search/retrieve to
gaining deep understanding about factors that govern the solid state. Databases are
now able to drive new science in areas such as crystal engineering. Looking forward,
we will see more automated pipelining of the data generation process, and this will
require better integration with databases. Databases will provide more predictive
power – and this will inform the science/crystallography that should be done.
Keywords Cambridge Structural Database (CSD) · Central facilities · Crystal
structure data · Crystallographic instrumentation · Crystallography · Data science ·
Single-crystal X-ray diffraction · Structural informatics · Synchrotron
1 Introduction to the Modern Crystallographic
Environment
1.1 The Development of the Crystallographic ‘Facility’
As the crystallographic technique matured during the twentieth century, the instrumentation and nature of the laboratory evolved. In the first half of the century, the
laboratory tended to consist of bespoke equipment, often constructed in-house and
only used by a small number of highly trained scientists whose research area was
centred around solving key crystal structures. However, the real power of the
technique was demonstrated around the middle of the century as much larger protein
structures began to be determined [1]. This scientific development, along with the
70
S. J. Coles et al.
2.4 Future Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104
3 The Database Revolution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
3.1 Nature of the CSD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
3.2 The Evolution of the Structural Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
3.3 The Transition to Informatics: Methods and Tools to Leverage Databases . . . . . . . . . 114
3.4 Areas Where Data-Driven Methods Are Making an Impact . . . . . . . . . . . . . . . . . . . . . . . . . 119
4 Closing the Loop and Future Prospects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
4.1 How Is Data Now Driving the Scientific Process and What Is the Future? . . . . . . . . . 121
4.2 How Far Can Single-Crystal Diffraction Structure Analysis Be Developed? . . . . . . . 125
4.3 Conclusions and Challenges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131
Abstract National facilities provide state-of-the-art crystallographic instrumentation and processes and tend to act as an indicator for the direction of a community in
the medium term. There has been a significant step up in terms of instrumentation
and approach in the last 10 years which has driven data generation. This has had a
significant impact on databases – in turn we observe a substantial change in the use
of the Cambridge Structural Database (CSD) from relatively basic search/retrieve to
gaining deep understanding about factors that govern the solid state. Databases are
now able to drive new science in areas such as crystal engineering. Looking forward,
we will see more automated pipelining of the data generation process, and this will
require better integration with databases. Databases will provide more predictive
power – and this will inform the science/crystallography that should be done.
Keywords Cambridge Structural Database (CSD) · Central facilities · Crystal
structure data · Crystallographic instrumentation · Crystallography · Data science ·
Single-crystal X-ray diffraction · Structural informatics · Synchrotron
1 Introduction to the Modern Crystallographic
Environment
1.1 The Development of the Crystallographic ‘Facility’
As the crystallographic technique matured during the twentieth century, the instrumentation and nature of the laboratory evolved. In the first half of the century, the
laboratory tended to consist of bespoke equipment, often constructed in-house and
only used by a small number of highly trained scientists whose research area was
centred around solving key crystal structures. However, the real power of the
technique was demonstrated around the middle of the century as much larger protein
structures began to be determined [1]. This scientific development, along with the
70
S. J. Coles et al.