ReScience C is a platinum open-access peer-reviewed journal that targets computational research and encourages the explicit replication of already published research, promoting new and open-source implementations in order to ensure that the original research is reproducible. You can read about the ideas behind ReScience C in the article Sustainable computational science: the ReScience initiative

To achieve this goal, the whole publishing chain is radically different from other traditional scientific journals. ReScience C lives on GitHub where each new implementation of a computational study is made available together with comments, explanations and tests. Each submission takes the form of an issue that is publicly reviewed and tested in order to guarantee that any researcher can re-use it. If you ever replicated computational results (or failed at) from the literature in your research, ReScience C is the perfect place to publish your new implementation.

ReScience C is collaborative and open by design. Everything can be forked and modified. Don’t hesitate to write a submission, join us and to become a reviewer.

Overview of the submission process

The ReScience C editorial board unites scientists who are committed to the open source community. Each editorial board member is specialised in a specific domain of science and is proficient in several programming languages and/or environments. Our aim is to provide all authors with an efficient, constructive and public editorial process.

Submitted entries are first considered by a member of the editorial board, who may decide to reject the submission (mainly because it has already been replicated and is publicly available), or assign it to two reviewers for further review and tests. The reviewers evaluate the code and the accompanying material in continuous interaction with the authors through the PR discussion section. If both reviewers managed to run the code and obtain the same results as the ones advertised in the accompanying material, and if they consider that these results are a replication of the original work, the submission is accepted. If any of the two reviewers cannot reproduce the results before the deadline, the submission is rejected and authors are encouraged to resubmit an improved version later.

Criteria for Publication

To be considered for publication in ReScience C, any given submission must satisfy the following criteria:

• Rigorous methodology
• Original source code
• Substantial evidence for replication of the original results or explanations why original results cannot be replicated

Furthermore, you cannot submit the replication of your own research, nor the research of your close collaborators. We believe such restrictions will favor the cross-fertilization of research and the spread of knowledge.

ReScience C accepts negative results as well, meaning here a failure to replicate the original results. The authors of the failed replication are expected to document their claim of failure, and to have attempted to contact the authors of the original article.

Article Scope

The accompanying article should help the reader ascertain if the replication was successful and as such, the main focus should be on the replicated results and on any obstacles encountered during the replication.

In many cases it can be useful to include high-level summaries of the original model or of your source code. Your goal should be to make the article sufficiently self-contained, but not to repeat all aspects of the original article or your documentation. Which amount of detail is sufficient may vary from case to case and is ultimately up to the reviewers to judge. If a reader has to be familiar with the original article in order to understand your explanations, consider that some readers may be unable to access it if it is behind a paywall.

Open Access

ReScience C applies the Creative Commons Attribution (CC BY) license to all works it publishes. Under the CC BY license, authors retain ownership of the copyright for their article, but authors allow anyone to download, reuse, reprint, modify, distribute, and/or copy articles in ReScience C journal, so long as the original authors and source are cited. No permission is required from the authors or the publishers.

Editorial Board

Topic Editors

Frequently Asked Questions

If your question is not answered below, please create an issue on GitHub and ask it there.

What’s the difference between replication and reproduction?

There is no consensus yet on what exactly these two terms mean, so here is how we understand and use them.

Reproduction of a computational study means running the same computation on the same input data, and then checking if the results are the same, or at least “close enough” when it comes to numerical approximations. Reproduction can be considered as software testing at the level of a complete study.

Replication of a scientific study (computational or other) means repeating a published protocol, respecting its spirit and intentions but varying the technical details. For computational work, this would mean using different software, running a simulation from different initial conditions, etc. The idea is to change something that everyone believes shouldn’t matter, and see if the scientific conclusions are affected or not.

Reproduction verifies that a computation was recorded with enough detail that it can be analyzed later or by someone else. Replication explores which details matter for reaching a specific scientific conclusion. A replication attempt is most useful if reproducibility has already been verified. Otherwise, if replication fails, leads to different conclusions, you cannot trace back the differences in the results to the underlying code and data.

In the ideal world, the reproducibility of computational studies would be checked automatically before publication. In the real world, most published studies do not even claim to be reproducible, because the code or the input data are not made public. Given this situation, a ReScience C publication provides two important services: by replicating the original work, it provides an independent implementation of the original computational protocol, and by making this new implementation public, it is reproducible and thus a safer basis for future research to build on.

What do the title prefixes Re, Rp, ¬Re, and ¬Rp mean?

Re means replication. Rp means reproduction. Either one can be negated in case of failure to give: ¬Re and ¬Rp.

Are there any publication fees?

No. ReScience C promotes open access and relies on the volunteer work of editors and reviewers. And the free hosting by GitHub.

ReScience C is a platinum open-access journal, which means that authors do not pay to publish and readers do not pay to read.

Can I submit a paper using proprietary tools such as MATLAB?

No. If we aim at replicability, we have to make sure the proposed implementation allows anyone to re-run and understand the code without any barriers. Proprietary software constitutes such a barrier because of their price and their closed nature. An implementation specifically targeting a free alternative such as Octave will be accepted though.

Is it mandatory to have an open-source license on the code?

Yes. The code accompanying ReScience C articles will be archived at Zenodo along with the article and data, entirely under open licenses. This is part of the open access principle underlying ReScience C. For the choice of a license for the code, we refer to the Debian Free Software Guidelines.

Can I submit the replication of my own research?

No. Mistakes in the implementation of research questions and methods are often due to biases authors invariably have, consciously or not. One’s biases will inevitably carry over to how one approaches a replication. Perhaps even more importantly, we aim at the cross-fertilization of research and trying to replicate the work of one’s peers might pave the way for a future collaboration, or may give rise to new ideas as a result of the replication effort.

Can I suggest a replication?

Yes. If you want to suggest an article for a replication, just open a new issue in the call for replication repository and give the reference of the original article and possibly the reason you would like to see this article replicated (please refrain from suggesting your own work). Note that you’re also encouraged to register as a reviewer such that you can review the replication you’ve been proposing if someone actually takes up the challenge.

If you’re looking for some challenge, you can also look at the current list of suggestions.

If my submission is rejected, can I resubmit it?

This depends on the reason your submission was rejected. If the original research has already been replicated previously (in ReScience C or elsewhere), it is unlikely we’ll consider another replication, even if you think your implementation is better. However, if your submission has been rejected because reviewers cannot make it run, you are encouraged to re-submit once the identified problems have been solved.

What if I fail to replicate a research result?

Some research may not be replicable. Before declaring a research result non-replicable, we require extra caution to be taken. In addition to scrutiny of your submission by reviewers and editors, we will contact the authors of the original research, and issue a challenge to the ReScience C community to spot and report (using the issue tracker) errors in your implementation. If no errors are found, your work will be accepted and the original research will be declared non-replicable.

Is there a preferred implementation language?

We favor a Python implementation because this language offers a scientific programming stack that can handle research in many domains of science (see for example the Scipy and EuroScipy conferences). Python is also often reported by scientists to easier to learn and use than other languages. Furthermore, using Python might allow us to use the IPython notebook and the upcoming Jupyter environments.

However, other languages or tools are acceptable if they are mainstream in the respective domain of research.

I’m a student, can I submit?

Yes! Students are strongly encouraged to submit their work. Although the ReScience C publishing model is a bit different from other academic journals, it can give students a first experience at peer-reviewed scholarly publishing, including meeting standards of scientific rigor and addressing reviewers’ comments. Publishing in ReScience C is also a way to actively contribute to open science while adding to one’s publication record.

How do I know if my replication is successful?

This depends mainly on the original paper introducing the research results to be replicated. Results might be presented in a qualitative or quantitative way and you’ll have to clearly state why you think your implementation of a method or protocol is a faithful replication of the original results.

What kind of research can I replicate?

Any computational research in any domain of science provided an editor from the Board has the expertise to edit your submission. Our editorial board is growing to increase the scientific domains being covered. If no editor is able to edit your submission, you may propose a guest editor (who must be willing to work with our GitHub-based editorial processes).

Does ReScience C issue DOIs (Digital Object Identifiers)?

Yes, every article published in ReScience C receives a DOI.