Q: Why DoubleBlind?
There are many reasons for this decision, not least the considerable number of requests from the community. For those who are interested in motivations for double blind reviewing, a very well argued, referenced and evidenced article in favour of double blind review processes for Software Engineering conferences can be found in the blog post by Claire Le Goues: https://www.cs.cmu.edu/~clegoues/double-blind.html. And here is a list of double-blind resources from Robert Feldt: http://www.robertfeldt.net/advice/double_blind_reviewing/. And a more formal study of the subject by Moritz Beller and Alberto Bacchelli: https://peerj.com/preprints/1757/
Q: How to prepare your paper for double-blind reviewing?
In order to comply, you do not have to make your identity undiscoverable; the double-blind aspect of the review process is not an adversarial identity discovery process. Essentially, the guiding principle should be to maximize the number of people who could plausibly be authors, subject to the constraint that no change is made to any technical details of the work. Therefore, you should ensure that the reviewers are able to read and review your paper without having to know who any of the authors are. Specifically, this involves at least the following three points:
Omitting all authors’ names from the title page.
Referring to your own work in the third person. You should not change the names of your own tools, approaches or systems, since this would clearly compromise the review process. It breaks the constraint that “no change is made to any technical details of the work”. However, you should always refer to the authorship/provenance of tools, approaches or systems in the third person, so that it is credible that another author could have written your paper.
Not relying on supplementary material (your web site, github repository, youTube channel) in the paper or on the rebuttal submitted during the clarification period. Supplementary information might result in revealing author identities.
Here is some excellent advice on anonymization from ACM: https://icer.hosting.acm.org/icer-2017/additional-advice-on-anonymization/
Q: What about additional information to support repeatability or verifiability of the reported results?
ICSE’18 puts a strong emphasis on creation of quality artifacts and repeatability and verifiability of experiences reported in the papers. An artifact evaluation committee is put in place to review artifacts accompanying all accepted papers, without the need to conceal identity of the authors.
Q: I previously published an earlier version of this work in a venue that doesn’t have double-blind. What should I do about acknowledging that previous work?
A: Double-blind does not and cannot mean that it’s impossible for the referees to discover The identity of the author. However, we require authors to help make it easy for author identity to not play a role in the reviewing process. Therefore, we ask that in the materials you submit to us to be reviewed author identity is not revealed.
In the particular, if the work you are submitting for review has previously been published in a non-peer-reviewed venue (e.g., arXiv departmental tech report), there is no need to cite it, because unrefereed work is not truly part of the scientific literature. If the previous work is published in a peer-reviewed venue, then it should be cited, but in the third person so that it is not clear whether or not this work was done by the author of the submitted paper or some other set of authors unknown.
Q: We have an early version of our work under review (e.g. short new ideas paper). Can we still submit a development of the work to ICSE
A: No. If the new ideas paper has already been accepted and was thus available to reviewers, then they could judge the degree of novelty and there would be no problem submitting to ICSE. Of course, the reviewers wouldn’t necessarily know that the new ideas paper was by the same authors, but that’s irrelevant to the assessment of the degree of novelty (and of course you would cite it in the third person). But if the new ideas paper is still under the review, you cannot submit its development to ICSE (without it being counted as a double-submission).
Q: Our submission makes use of work from a PhD thesis which is been published. Citing the PhD thesis might compromise anonymity. What should we do?
A: It’s perfectly OK to publish work from a PhD, and there’s no need to cite it in the version submitted for review because prior PhD publication does not compromise novelty. In the final (post-review, camera ready) version of the paper, please do cite the PhD thesis to acknowledge its contribution, but in the refereed version of the paper that you submit, please refrain from citing the PhD thesis, to increase K-anonymity.
However, you need not worry whether or not the PhD thesis has appeared, since your job is only to help the committee review your work without awareness of author identity, but not to make it impossible for them to discover the identity of authors. The referees will be trying hard not to discover the authors’ identity, so they will likely not be searching the web to check whether there is a PhD thesis related to this work.
Q: What if we want to cite some unpublished work of our own (as motivation for example)
A: This is OK, so long as the unpublished work is made public and that it does not fall under the scope of the question “We have an early version of our work under review” above. If the unpublished paper is an earlier version of the paper you want to submit to ICSE, then you have to wait until your earlier version is through the review process before you can build on it with further submissions. Otherwise, if the unpublished work is not an earlier version of the proposed ICSE submission, then you should simply make it available (on a website for example) and cite it (in the third person, as normal, to preserve anonymity).
Q: When does artifact evaluation occur and should my artifact be blinded for submission?
A: There are two issues here. You are welcome to submit a zip file with your code / data / case studies as part of your technical paper review. This happens at the paper submission time. All of the information should be blinded, and the committee MAY (but does not have to) look at this information if it finds it relevant for evaluating your technical paper. After your paper is accepted to ICSE’18 Technical Track, you will receive an invitation to submit the artifacts you developed as part of your work (including those you submitted for paper evaluation) to a separate committee called Artifact Evaluation committee. Such data should be submitted UNBLINDED. The committee will install your tools and run your scripts and compare the results with what is reported in the paper. It will then award you a badge. This committee cannot reject your technical paper; it can only assess the quality of your artifacts.
Q: How do I format the paper for submission to ICSE’18?
A: First of all, all tracks expect submissions in the same format. For MS Word users: Use MS Word docx ICSE 2018 template, or use the ACM_SigConf.docx template following the instructions from NEW ACM formatting guidelines. For LaTex users: Use NEW ACM formatting guidelines. Upon unzipping the zip file, run ‘make all’. This will produce the acmart.cls file. You should use the conference format described in sample-sigconf.tex to format your paper.
Second, check your track for the limit on the number of pages allowed. ICSE’18 Technical track allows 10 pages + 2 pages of references. Third, your submission should be in PDF.
Last modified: August 10, 2017