very rapidly. It is no longer viable for a facility to operate in a primarily ‘manual’
mode in terms of where data is stored, how it is moved around, how it is identified,
how it is accessed, how it is monitored and how it is curated. Diffractometer control
software has made some advances in these respects, but invariably the small
molecule crystallography facility will have to develop mechanisms to do this. A
range of solutions exist, for example, some basic scripting can ensure that data is
sensibly and safely moved and there are numerous off-the-shelf automated mechanisms for storage, back-up and archival – often provided by the institution/organisation in which the facility is located.
However, the issues of identifying, and thereby being able to search and retrieve,
data are often overlooked, and as the volume of data scales, this becomes a
significant problem. A further aspect that becomes more intractable as the volume
of data blooms is that of curation – how does one make sure data is migrated to the
most recent formats and storage media; how does one keep track of what data
‘belongs’ to whom and what you are ‘allowed’ to do with it; how do you know
what has been published and what has not? These matters are not easily solved by a
piece of technology and largely still have to be addressed by knowledgeable staff.
These factors also lead into the second main impact – that of the publication
bottleneck. For some time now, the pace at which structures are generated has been
greater than that at which they can be published. Despite the increase in digitalisation
of much of the publication process, there has barely been increase in speed of
publication, and the process has essentially remained unchanged for decades –
particularly in respect of the ‘publication’ of data. In chemistry and small molecule
crystallography, there persists a culture where a crystal structure is tied to the
publication of the associated chemistry, and often the latter takes a considerably
longer time to perform and understand. Accordingly, data generation far outstrips
data publication, and facilities are building up vast archives of redundant data. There
have been some steps towards addressing this situation, but there is still a long way
to go in terms of cultural change. With the advent of the World Wide Web,
researchers have experimented with self-publishing, from putting some results on
a personal website through to depositing significant volumes in online repositories.
The rise of general repository systems such as Figshare [73] has had quite an impact
in some fields, although in general these generic systems tend to cater better for
academic institutions, publishers and similar large organisations. However, the
advantage of using a domain-specific repository, where available, is that data can
be validated, checked and interpreted appropriately, so while in small molecule
crystallography there have been some local- and project-level attempts to develop
repositories, e.g. eCrystals and related projects [74], they have largely failed to gain
widespread traction. However, CCDC is now beginning to address this, with a
marked uptake in ‘CSD Communications’ [75]. For years there had been a ‘private
communication’ route to deposit structures in the CSD that did not have an associated literature publication, but this was largely not used. With the cultural shift we
have seen recently in terms of availability of digital resources, CSD Communications, due in part to the provision of an enhanced deposition process, has begun to
thrive and now numbers around 30,000 entries. The ability to independently make a
Leading Edge Chemical Crystallography Service Provision and Its Impact on. . .
99
mode in terms of where data is stored, how it is moved around, how it is identified,
how it is accessed, how it is monitored and how it is curated. Diffractometer control
software has made some advances in these respects, but invariably the small
molecule crystallography facility will have to develop mechanisms to do this. A
range of solutions exist, for example, some basic scripting can ensure that data is
sensibly and safely moved and there are numerous off-the-shelf automated mechanisms for storage, back-up and archival – often provided by the institution/organisation in which the facility is located.
However, the issues of identifying, and thereby being able to search and retrieve,
data are often overlooked, and as the volume of data scales, this becomes a
significant problem. A further aspect that becomes more intractable as the volume
of data blooms is that of curation – how does one make sure data is migrated to the
most recent formats and storage media; how does one keep track of what data
‘belongs’ to whom and what you are ‘allowed’ to do with it; how do you know
what has been published and what has not? These matters are not easily solved by a
piece of technology and largely still have to be addressed by knowledgeable staff.
These factors also lead into the second main impact – that of the publication
bottleneck. For some time now, the pace at which structures are generated has been
greater than that at which they can be published. Despite the increase in digitalisation
of much of the publication process, there has barely been increase in speed of
publication, and the process has essentially remained unchanged for decades –
particularly in respect of the ‘publication’ of data. In chemistry and small molecule
crystallography, there persists a culture where a crystal structure is tied to the
publication of the associated chemistry, and often the latter takes a considerably
longer time to perform and understand. Accordingly, data generation far outstrips
data publication, and facilities are building up vast archives of redundant data. There
have been some steps towards addressing this situation, but there is still a long way
to go in terms of cultural change. With the advent of the World Wide Web,
researchers have experimented with self-publishing, from putting some results on
a personal website through to depositing significant volumes in online repositories.
The rise of general repository systems such as Figshare [73] has had quite an impact
in some fields, although in general these generic systems tend to cater better for
academic institutions, publishers and similar large organisations. However, the
advantage of using a domain-specific repository, where available, is that data can
be validated, checked and interpreted appropriately, so while in small molecule
crystallography there have been some local- and project-level attempts to develop
repositories, e.g. eCrystals and related projects [74], they have largely failed to gain
widespread traction. However, CCDC is now beginning to address this, with a
marked uptake in ‘CSD Communications’ [75]. For years there had been a ‘private
communication’ route to deposit structures in the CSD that did not have an associated literature publication, but this was largely not used. With the cultural shift we
have seen recently in terms of availability of digital resources, CSD Communications, due in part to the provision of an enhanced deposition process, has begun to
thrive and now numbers around 30,000 entries. The ability to independently make a
Leading Edge Chemical Crystallography Service Provision and Its Impact on. . .
99