Authors are invited to recommend or ask for the exclusion of specific individuals from the peer review process. The journal does not guarantee to use these suggestions. All reviewers must be independent from the submission and will be asked to declare all competing interests.

Software Metapapers

Two general principles guide the reviewing process at the Journal of Open Research Software:

1. We are reviewing the accuracy and quality of the metadata rather than the software, however there will be a minimum level of quality of software required so that it is possible to review
2. We expect all metapapers to be able to pass after revisions, unless the software is not openly available and/or extremely difficult to reuse.

After submission, a software metapaper is sent to multiple independent reviewers.

Each reviewer attempts to download the software based on the information in the submitted paper and:

• checks that the software behaves as described
• checks that the information in the metapaper is correct, in particular the contributors, license, and limitations
• writes up a review

Reviews are combined to provide a decision (accept, accept after minor revisions, re-review after major revisions, reject) and a checklist of revisions and suggestions are sent to the author, along with the reviews. Reviewers are also encouraged, but not required, to sign their reviews.

All JORS software papers are peer reviewed according to the following standardised review form:

The paper contents Is the title of the paper descriptive and objective? Does the Abstract give an indication of the software's functionality, and where it would be used? Do the keywords enable a reader to search for the software? Does the Introduction give enough background information to understand the context of the software's development and use? Does the Implementation and Architecture section give enough information to get an idea of how the software is designed, and any constraints that may be placed on its use? Does the Quality Control section adequately explain how the software results can be trusted? Does the Reuse section provide concrete and useful suggestions for reuse of the software, for instance: other potential applications, ways of extending or modifying the software, integration with other software? Are figures and diagrams used to enhance the description? Are they clear and meaningful? Do you believe that another researcher could take the software and use it, or take the software and build on it?

The deposited software Is the software in a suitable repository? (see http://openresearchsoftware.metajnl.com/about/editorialPolicies#custom-0 for more information) Does the software have a suitable open licence? (see our FAQs for more information)* If the Archive section is filled out, is the link in the form of a persistent identifier, e.g. a DOI? Can you download the software from this link? If the Code Repository section is filled out, does the identifier link to the appropriate place to download the source code? Can you download the source code from this link? Is the software license included in the software in the repository? Is it included in the source code? Is sample input and output data provided with the software? Is the code adequately documented? Can a reader understand how to build/deploy/install/run the software, and identify whether the software is operating as expected? Does the software run on the systems specified? (if you do not have access to a system with the prerequisite requirements, let us know). Is it obvious what the support mechanisms for the software are?

 

Issues in Research Software

PLEASE NOTE this article type is currently closed to new submissions due to a lack of an editor. 

The following review criteria relate to our long-form articles submitted under to the Issues in Research Software section. Articles should not exceed 3-4000 words with a constrained list of references. The article can be structured with subheadings as the author sees fit, though an abstract is required. Please see below for more detailed criteria.

General Peer-review Criteria

• Does the paper provide information that is useful for the community?
• Is the paper clear to read and defines unfamiliar concepts?
• Does the paper understand and recognise other efforts in the area in order to frame the discussion?
• Does the paper appear credible and trustworthy?
• Does the paper summarise the experiences of the authors, including reflection on the importance of the arguments and conclusions?
• Is the paper in scope?
• Is the paper written for a general research audience?

Criteria for position papers:

• Is the viewpoint expressed clearly and in a well-organised way?
• Does the paper provide a foundation for others to resolve challenges in the area?
• Does the paper present a unique, though biased, solution or a unique approach to solving a problem?
• Does the paper demonstrate a command of the issues and research behind them?
• Does the paper provide useful ideas which are backed by personal experience?
• Does the paper show that supporting evidence for both sides of an argument has been considered?

Criteria for survey papers:

• Does the paper present a concise but broad summary of an area that is accessible to a general reader?
• Does the paper cover a sufficiently large proportion of the area being surveyed?
• Does the author present a deep knowledge of the field, including main directions and controversies?
• Does the author provide a summary of the main challenges that are being addressed by the papers / methods being surveyed?
• Does the paper provide a critical analysis of each paper or method included?
• Does the paper convey not just a list of papers/method/results but increases the understanding of the structure and direction of the area?

Criteria for experience reports:

• Does the paper clearly state the research problem to which their efforts relating to software were applied?
• Does the paper summarise the challenges faced and the decision process for choosing solutions to these challenges?
• Does the paper distill the authors experiences in a way that they can be understood by readers in the same or related fields?
• Does the paper distill the authors experiences in a way that can be understood by readers undertaking similar types of work?
• Does the paper present clear recommendations based on the authors experiences that can be applied by other readers?

The journal is happy to accept submissions of papers that have been loaded onto preprint servers or personal websites, have been presented at conferences, or other informal communication channels. These formats will not be deemed prior publication. Authors must retain copyright to such postings. Authors are encouraged to link any prior posting of their paper to the final published version within the journal, if it is editorially accepted.

Members of the editorial team/board are permitted to submit their own papers to the journal. In cases where an author is associated with the journal, they will be removed from all editorial tasks for that paper and another member of the team will be assigned responsibility for overseeing peer review. A competing interest must also be declared within the submission and any resulting publication.

Reviewer Guidelines

Reviewers are asked to provide comment on the below topics and guidelines:

• Content: Does the article fit within the scope of the journal? Is the submission original, relevant and rigorous? Is the author’s depth of understanding of the issues researched adequate? Are the sources and references adequate? Has the existing knowledge base been explored and built upon? Are the chosen methodologies appropriate and have they and the evidential base been appropriately used? Does the conclusion reflect the argument in the main body text and bring something new to the debate?
• Structure and argument: Does the abstract summarise the arguments in a succinct and accurate way? Is the manuscript logically structured and do the arguments flow coherently? Is there enough reference to methodology in the introduction and are the arguments fully evidenced and substantiated? Does the introduction signpost the arguments in the logical way and does the conclusion adequately summarise them?
• Figures/tables: Does the author’s use of tables, charts, figures or maps illustrate the arguments and support the evidential base? Is the quality of the formatting and presentation adequate?
• Formatting: Does the submitted file adhere to the general author guidelines listed for the journal? Are the citations and references formatted to house-style?
• Language: Is the text well written and jargon free? Please comment on the quality of English and need for grammatical improvement.

COPE

The journal is a registered member of the Committee on Publication Ethics (COPE) and is committed to following COPE guidelines.

Preprint Policy

The journal allows authors to deposit draft versions of their paper into a suitable preprint server, on condition that the author agrees to the below:

• The author retains copyright to the preprint and developed works from it, and is permitted to submit to the journal.
• The author declares that a preprint is available within the cover letter presented during submission. This must include a link to the location of the preprint.
• The author acknowledges that having a preprint publicly available means that the journal cannot guarantee the anonymity of the author during the review process, even if they anonymise the submitted files (see review policy).
• Should the submission be published, the authors are expected to update the information associated with the preprint version to show that a final version has been published in the journal, including the DOI linking directly to the publication.

ORCID

The journal strongly recommends that all authors submitting a paper register an account with Open Researcher and Contributor Identifier (ORCID). Registration provides a unique and persistent digital identifier for the account that enables accurate attribution and improves the discoverability of published papers, ensuring that the correct author receives the correct credit for their work. As the ORCID remains the same throughout the lifetime of the account, changes of name, affiliation, or research area do not effect the discoverability of an author's past work and aid correspondence with colleagues.

The journal encourages all corresponding authors to include an ORCID within their submitting author data whilst co-authors are recommended to include one. ORCID numbers should be added to the author data upon submission and will be published alongside the submitted paper, should it be accepted.

Data Policy

The journal strongly encourages authors to make all data associated with their submission openly available, according to the FAIR principles (Findable, Accessible, Interoperable, Reusable). This should be linked to from a Data Accessibility Statement within the submitted paper, which will be made public upon publication. If data is not being made available with the journal publication, a statement from the author should be provided to explain why. Data obtained from other sources must be appropriately credited. When depositing data for a submission, the below should be considered:

• The repository the data is deposited in must be suitable for this subject and have a sustainability model.
• The data must be deposited under an open license that permits unrestricted access (e.g. CC0, CC-BY). More restrictive licenses should only be used if a valid reason (e.g. legal) is present.
• The deposited data must include a version that is in an open, non-proprietary format.
• The deposited data must have been labelled in such a way that a 3rd party can make sense of it (e.g. sensible column headers, descriptions in a readme text file). 
• Research involving human subjects, human material, or human data, must have been performed in accordance with the Declaration of Helsinki. Where applicable, the studies must have been approved by an appropriate ethics committee. The identity of the research subject should be anonymised whenever possible. For research involving human subjects, informed consent to participate in the study must be obtained from participants (or their legal guardian). 
• A ‘Data Accessibility Statement’ should be added to the submission, prior to the reference list, providing the details of the data accessibility, including the DOI linking to it. If the data is restricted in any way, the reasoning should be given. 

A list of data repositories is available at http://oad.simmons.edu/oadwiki/Data_repositories.

Authorship

All listed authors must qualify as such, as defined in our authorship guidelines, which have been developed from the ICMJE definitions. All authors must have given permission to be listed on the submitted paper.

Competing Interests, Funding and Ethics 

To ensure transparency, all authors, reviewers and editors are required to declare any interests that could compromise, conflict or influence the validity of the publication. Competing interests guidelines can be viewed here.

In addition, authors are required to specify funding sources and detail requirements for ethical research in the submitted manuscript (see Author Guidelines).

Plagiarism

The journal views plagiarism and the use of other people’s work without proper attribution and permission as a serious offence. All submissions to the journal are checked via Crossref's Similarity Check to assess if any content closely matches content already available elsewhere. If a high similarity report is produced, or an allegation is made against a submission or publication, the journal will carry out an independent investigation in accordance with COPE guidelines and remedial action will be taken as per the journal's Appeals, Complaints & Misconduct policy if deemed appropriate.

Corrections and Retractions

In accordance with guidelines from the Committee on Publication Ethics (where applicable), the Press handles different kinds of error. All articles have their proofs checked prior to publication by the author/editor, which should ensure that content errors are not present. Please contact the journal if you believe an article needs correcting.

Post-publication changes to the publication are not permitted unless in exceptional circumstances. If an error is discovered in a published article then the publisher will assess whether a Correction paper or Retraction is required. Visit our Correction Policy page for more information.

Appeals, Complaints & Misconduct

Appeals, complaints, or allegations of misconduct will be taken with utmost seriousness, regardless of whether those involved are internal or external to the journal, or whether the submission in question is pre- or post-publication. If an allegation is made to the journal, it must also be passed on to the publisher, who will follow guidelines from the Committee on Publication Ethics (COPE) on how to address the nature of the problem.

Should an individual wish to submit an appeal, complaint or raise an issue of potential misconduct regarding the journal or its content, they should first read the full Appeals, Complaints, and Misconducts policy and then contact that editor in chief and/or the publisher to explain their concerns.

Expected behaviour

The journal does not tolerate abusive behaviour or correspondence towards its staff, academic editors, authors, or reviewers. Any person engaged with the journal who resorts to abusive behaviour or correspondence will have their contribution immediately withdrawn and future engagement with the journal will be at the discretion of the editor and/or publisher.