Introduction

The 21st Century has seen a massive growth in the quantity and complexity of data produced by scientific software. At present, such data is typically confined to the researcher's computer for analysis, and eventually condensed down into a set of key results and figures for publication in a journal. The raw data itself is rarely made available to the wider community. Furthermore, the software (which can itself be a research product) is often not shared because of the current lack of intrinsic motivators for the researcher who developed it, thereby preventing the data from being recomputed by others. In academia the impact factor of an individual is commonly measured by the number of publications and their associated citation counts; making the software available can be seen as an additional effort that will not reap significant rewards for the researcher since they will gain little recognition from something which is not viewed in the same regard as a journal article. Even in the case where the software is made available, it is often poorly referenced; usually either the user manual or the software's website are cited, but neither resource is satisfactory since they do not provide information about the specific revision of the source code that was used to produce the data. Without the means of replicating and verifying the correctness of a result, which requires full and open access to both the source code and raw data [1,2], its credibility could be considered questionable [3]. Reproducibility and recomputability are therefore becoming increasingly recognised by scientific communities as measures of research quality.

With the release of new tools such as Fidgit [4], the Mozilla Science Lab's “Code as a Research Object” digital object identifier (DOI) generator for GitHub repositories [5], and the dvn [6] and rfigshare [7] packages which interface with the Dataverse Network (http://www.thedata.org) and Figshare (http://www.figshare.com), publishing software and data online is becoming increasingly straight-forward. However, care must be taken to ensure that (a) the version of the software that is published is exactly the same as the version that generated the data, (b) that any provenance metadata (e.g. the DOIs of the software and input data) is appended to the output data before being published, and (c) that all necessary files are selected for publication (i.e. any collected/measured input data such as bathymetry for an ocean simulation, and any output data such as field solutions and plots). Doing so by hand can be a complicated, time-consuming and error-prone task.

In order to further encourage the publication of software and data online, we have developed an open-source Python-based library called PyRDM which provides the capability to integrate research data management into the workflows of scientific software packages, so that the software and data can be curated at the push of a button. The functionality of PyRDM is encapsulated in several Python modules which, in short, facilitate the automated publication of scientific software and data via online, citable repositories. Specifically, the Figshare online repository service is used to store the software and data, and provides DOIs for both resources so they can be appended to any provenance metadata. Not only does this allow a specific version of the software or data to be properly cited in a journal publication, it can also reduce data duplication by enabling the sharing of data with colleagues around the world. From the point-of-view of an individual researcher, being able to cite the code and data using a DOI brings reputation-related benefits since these outputs are often included in online publishing records (e.g. in Google Scholar profiles and h-indexes). Furthermore, the design philosophy of PyRDM is such that it is something that 'just works' from a user perspective, requiring minimal effort past the initial integration step.

One application of PyRDM has involved its integration into the workflow of Fluidity [8,9], an open-source computational fluid dynamics code. For a given simulation, users only need to enable a 'publish' option in the simulation's setup file (see Figure 1) and provide a minimal amount of information concerning which files they wish to publish (e.g. "*.vtu" will publish all VTK-based data files). All other issues such as determining which software version was used to run the simulation; checking whether that version has been published already (so its associated DOI can be reused); affiliating all authors of the software with the Figshare repository; appending DOIs to the provenance metadata; and using MD5 checksums to determine which data files do not need to be re-uploaded to the Figshare servers each time a new version of the data is published (using the same DOI), are all handled automatically with the help of PyRDM. This is discussed in greater detail in the next section.

Implementation and architecture

PyRDM has been implemented using the Python programming language, and its design is centred around two classes, Publisher and Figshare. The Figshare class handles all the communication with the Figshare servers via the Figshare API to create code repositories and datasets, and upload any files. The Publisher class handles any tasks that are specific to the publication of software and data, such as determining which files are to be uploaded and automatically associating all authors with Figshare author IDs to the software repository. This is illustrated by the UML diagram in Figure 2. Note that the use of ORCID researcher IDs will likely be the preferred approach for affiliating authors in the future.

The publication of software (with a known version) is performed using the publish_software method of the Publisher class. The method first searches Figshare to determine whether the same version of the software has already been published. This version is based on a unique SHA-1 hash corresponding to a specific revision of the software in its repository; currently, only public Git repositories hosted by GitHub are supported. If the specific version of the software has already been published, then the method returns details of the existing publication (including the DOI). Otherwise, it creates a new code repository on Figshare, downloads the source code (with the specified SHA-1 hash) from the software's Git repository as a .zip file, and then uploads it to Figshare. The SHA-1 hash is then added as a 'tag' to the Figshare publication for future search and identification purposes.

To ensure that all authors are properly attributed to the software's publication, the publish_software method also looks for an AUTHORS file and, if present, parses it for Figshare author IDs and automatically adds any additional authors. Note that only the software's name, SHA-1 hash, location of the local Git repository containing the software, and the category of the software (e.g. “Computational Physics”) are passed as arguments to publish_software; all other parameters, such as the Figshare authentication keys and location of the GitHub repository online, are obtained from the PyRDM configuration file which is populated just once by the user and by interrogating the local Git repository.

The publication of input and output data is performed using the publish_data method of the Publisher class. When the method is called, it creates a new fileset on Figshare and uploads the files whose paths are specified (along with a title and description of the data) in a dictionary of parameters which is passed in by the calling program (i.e. a software-specific publishing tool, explained below). It then returns details of the fileset, including its ID and DOI. Alternatively, PyRDM can be provided with a known fileset ID to update any existing data which creates a new version of that fileset on Figshare, but keeps the same DOI.

In the event of having to update a published fileset or create a new version altogether, it would be inconvenient to re-upload any files that have not been modified since the first time they were published. This is especially true if the files are large; examples include computational meshes, forcing data, and any field values used as initial conditions. Therefore, PyRDM only performs selective updates of files. The first time a file is uploaded to Figshare, a corresponding MD5 checksum file is created and stored locally. The next time the user attempts to upload that file, PyRDM determines whether the MD5 checksum of the file (in its current state) is the same as the MD5 checksum stored in the corresponding checksum file. If it is the same, the file is not re-uploaded.

The PyRDM modules can be imported and used in a separate Python program which acts as the automated publishing tool for a specific piece of scientific software. An example which considers the computational fluid dynamics code Fluidity can be found in the 'bin' directory of PyRDM. In this example, publication is viewed as a staged process; users may publish the software and input data first, run the simulation and then publish the output data. Alternatively, users can perform all of the publishing steps once the simulation has finished. When executing the Fluidity-specific publishing tool, the user only needs to provide the location of the XML-based configuration file of the simulation being considered (for details concerning the location of the input/output files) and what they would like published (software, input data or output data). At each stage, the Figshare code/fileset ID and DOI information are stored in the simulation's configuration file. Users can also provide PyRDM with an existing ID and/or DOI through the configuration file. When publishing output data, the DOIs of the software and input data used to produce that output data are appended to the simulation output's metadata to ensure data provenance. An example is given below:

<constant name="FluidityVersion" type="string" value="1baf80aac1e7e735b1cf182bc20761a0c6df7767"/>
<constant name="SoftwareDOI" type="string" value="http://dx.doi.org/10.6084/m9.figshare.1035081"/>
<constant name="InputDataDOI" type="string" value="http://dx.doi.org/10.6084/m9.figshare.1035083"/>
<constant name="CompileTime" type="string" value="May 23 2014 15:22:23"/>
<constant name="StartTime" type="string" value="20140523 154857.775+0100"/>

Quality control

Unit testing is carried out on the separate modules within PyRDM to verify their correctness. In particular, tests within the Figshare module create a temporary and private dummy dataset on the Figshare servers (without minting a new DOI), perform various operations on that dataset via the Figshare module's methods (e.g. adding a file, searching for that dataset, and adding a tag), then checks via assertion statements that the dataset has been modified in the correct way. The dummy dataset is then deleted after the unit tests have taken place. The tests within the Publisher module verify that the MD5 checksum functionality does indeed correctly determine which files need to be re-uploaded, and that the PyRDM AUTHORS file is correctly parsed for the Figshare author IDs.

Before publishing to Figshare, a trial run of the Fluidity-specific publishing tool which uses PyRDM was performed using data from a simulation of a 'top hat' advection problem (see [9] for more information) in order to test its usability and uncover any bugs that were not detected by the unit tests. The automated publication of the software, input data and output data was successfully achieved, yielding the following DOIs:

• Software: http://dx.doi.org/10.6084/m9.figshare.1035081
• Input data: http://dx.doi.org/10.6084/m9.figshare.1035083
• Output data: http://dx.doi.org/10.6084/m9.figshare.1035093

Operating system

Linux (tested on Ubuntu Precise (12.04) 32-bit and 64-bit)

Programming language

Python 2.7.3

Additional system requirements

The computer will need enough free disk space to accommodate a complete download of the scientific software from GitHub, in order to then upload it to Figshare. A connection to the Internet will also be required.

Dependencies

• requests-oauthlib >= 0.4.0 (https://github.com/requests/requests-oauthlib)
• GitPython >= 0.3.2.RC1 (https://pypi.python.org/pypi/GitPython/)
• pdfLaTeX >= 1.40.10-2.2 (http://www.tug.org/applications/pdftex/), to build the user manual from the LaTeX source file.

List of contributors

1. Jacobs, Christian T. (Lead developer)
2. Avdis, Alexandros (PI/Supervisor, Developer)
3. Gorman, Gerard J. (PI/Supervisor)
4. Piggott, Matthew D. (Supervisor)

Archive

Code repository

Language

English