Tuesday, 28 September 2010

Science journal editors: a taxonomy

After many years of publishing papers, I have come to recognise wide diversity among journal editors. This variation has major consequences for authors, and it is important that they recognise the creature they are dealing with, if they want their work to be published in as timely and painless a way as possible. I have therefore developed a tripartite system of classification to guide authors.



Taxonomy of the Genus Editoris



Class 1
This species of editor should be avoided at all costs.

The Returning Officer
This humble creature has a very limited brain and is unable to make decisions. It can, however, count, and it therefore uses a strategy of accumulating reviewer reports until a consensus is reached. Typically, it is risk averse, and a single negative report will lead to rejection of a paper, even if other reports are glowing. If you aren’t rejected, an initial communication from a Returning Officer will say “Please address all the comments of the reviewers in your revision”, giving no guidance about how to deal with contradictory recommendations. When you submit your revision, the Returning Officer will send it back to all the reviewers, even if only minor changes were made, leading to unnecessary delay in publication and more toil for overworked reviewers. Since the Returning Officer cannot make a decision unless there is convergence of reviewer opinions, most papers are doomed to a long process with an ultimately negative outcome.

The Automaton

This is a subspecies of Returning Officer which has no human characteristics at all.  It evolved relatively recently with the advent of web-based journal submission systems. It generates letters written in computerese and does not read communications from authors. My most recent experience of an Automaton was with Journal of Neuroscience. The letter from the editor gave a rather ambiguous message, stating that the paper was potentially acceptable, but that major revision was required, and it would need to go back to reviewers. It also included the statement: 
   Violations: -The gender of the species should be mentioned in the methods
The dictionary definition of ‘violation’ includes such phases as “the act of violating, treating with violence, or injuring; the state of being violated. Ravishment; rape; outrage.” I decided it might be unwise to point this out to the editor, but I did explain that the “gender of species” was actually given in a table in the Methods section.  A further round of review took place, and the reviews (which were very useful) were accompanied by another letter from the Automaton. It was identical to the previous letter, gave no indication that the editor had read the paper or my response to reviewers, and simply upped the ante on the violation front, as it now stated:

   Violations: -The species is not mentioned in the abstract;   
   -The gender of the species should be mentioned in the methods
In what, thankfully, proved to be the final round of revision, I put the word “gender” in the text of the Methods. I explained, though, that I was reluctant to put “human children” in the Abstract, as this would be a tautology.

The Vacillator

This is a slightly more evolved form of Returning Officer, which is capable of decision-making, but prone to fits of paralysis when confronted by conflicting information.  The hallmark of a Vacillator is that, rather than waiting for consensus between reviewers, it responds to conflicting opinions by seeking yet more opinions, so that a paper may accumulate as many as four or five reviewers.

A variant known as Vacillator statistica sometimes inhabits the environment of medical journals, where the assumption is made that neither the editor nor the researchers understand statistics, so you are asked at submission whether a statistician was consulted. My experience suggests that if you say no, then after an initial round of review, the paper goes to a statistician if it looks promising.  It would be fine if the journal employed statisticians who could give a rapid response, but in my case, a brief paper sent to Archives of Disease in Childhood sat for months with a statistical reviewer, who eventually concluded that we did indeed know how to compute an odds ratio.

The Sloth
The Sloth has powers of judgement but finds journal editing tedious, so engages with the process only intermittently.  The motivation of the Sloth is often mysterious; it may have become an editor to embellish its curriculum vitae, and is then bewildered when it realises that work is involved. It is important to distinguish the true Sloth, who just can’t summon up the energy to edit a paper, from Crypto-sloths, who may have genuine reasons for tardiness; editors, after all, are beset by life events and health problems just like the rest of us.  Vacillators may also be mistaken for Sloths, because of the slowness of their responding, but their level of activity in soliciting reviews is a key distinguishing feature. Even Paragons (see below) may get unfairly categorised as Sloths, as they are dependent on reviewers, who can delay the editorial process significantly. A true Paragon, however, will be pro-active in informing an author if there are unusual reasons for delays, whereas the distinguishing feature of a Sloth is that it is unresponsive to communications and blithely unconcerned about the impact of delays on authors.

Class 2
Species in class 2 pose less of a threat to an author’s career, but can nevertheless be dangerous to mental health.

The Talent Scout
This species is found in the rarefied habitats of the top high-impact journals, although it starting to spread and may now be found in medium-impact journals who have introduced a triage process. The Talent Scout’s principal concern is whether a research finding has star quality. The species is distinguished from other species by including individuals who are not active researchers:  many are individuals with a doctorate in science who have moved into science journalism. Although it can be depressing to have one’s work judged as too unsexy for publication by someone with no expertise in  your field, the decision-making process is usually mercifully quick, making it possible to regroup and resubmit elsewhere. Although this means that the impact on the author is less severe than for Class 1 editors, it does have potentially worrying implications for science as a whole, because it introduces bias. For instance, it is all too easy to see why Science published a study of a computer-based intervention for language problems in children: the study was headed by a top neuroscientist, the method was innovative, and it demonstrated potential to help children with a common neurodevelopmental disorder. A study like this presses all the buttons for the Talent Scout. However, the methodology was weak and subsequent randomised controlled trials (RCTs) have been disappointing (see review). I don’t know if authors of those RCTs would have tried to publish them in Science, but my guess is that if they did, their papers would have been rejected because it is simply much less interesting to show that something doesn’t work, than to provide evidence that it does (see blog).

The Deity

The Deity is the opposite of the Vacillator: the Deity makes decisions which may strike authors as unfair or subjective, but which are absolute and irreversible. Deities do not engage in correspondence with authors, but delegate this to office staff, as I found on the one occasion when I tried to engage in debate with a Deity from PNAS. I was incensed by a reviewer report that maintained a postdoc and I had been ‘cherrypicking’ results because we’d used an automated artefact removal procedure to remove noisy trials from a study using event-related potentials (ERPs). The reviewer clearly had no expertise in ERP methods and so did not realise that we were following standard practice, and that the idea of cherrypicking was just silly – it would be quicker to re-run the experiment than to go through the thousands of individual trials removing data we didn’t like the look of. In my letter to the editor, I explained that I did not want the paper reconsidered, but I did want an acknowledgement of the fact that I had not been fudging the data. What ensued was a tedious correspondence with a member of editorial staff, whose response was to send the paper back to the reviewer as part of an ‘appeal’ process, and to then inform me that the reviewer still didn’t like the paper.  Nowhere in this process did the Deity descend from the heights to offer any comment. Indeed, I still wonder whether this Deity was really an Automaton. It showed no signs of having any sense of morality.

Another encounter with a Deity was when I sent a paper to New England Journal of Medicine. Since I thought this should have at least warranted review (novel study with important clinical and theoretical implications), I wrote to ask what the reason was for rejecting it without review. The response from editorial staff was classic Deity: they could not give me any feedback as the paper had not been sent out for review.


Class 3
Species in class 3 are typified by their attitude to the job of editor, which is neither as bean-counter, nor as gatekeeper, but as facilitating the communication of high quality research.

The Paragon

The Paragon reads manuscripts and treats reviewer reports as advisory rather than as votes. He or she aims to make decisions fairly, promptly and transparently. Confronted with conflicting reviewer reports, a Paragon makes an honest attempt to adjudicate between them, and explains clearly to the author what needs to be done – or why a paper has been rejected. The Paragon will listen to author complaints, but not be swayed by personal friendship or flattery.  I’ve often heard authors complain about a Paragon who writes such a long decision letter that it is equivalent to a further reviewer report: I don’t see that as cause for complaint. I would sometimes do that myself when I was a journal editor (needless to say, I tried hard to be a Paragon), and I saw it as part of my job to pick up on important points that were missed by reviewers. Paragons write personal letters to authors, and to thank particularly helpful reviewers, rather than relying on computer-generated bureaucratese.

The Obsessive
The Obsessive is a Paragon that has gone over the top. Obsessives are not dangerous like class 1 and 2 editors: they typically damage themselves rather than the authors, to whom they are just irritating.  They essentially take upon themselves the job of copy editor, requiring authors to make minor changes to formatting and punctuation, rather than restricting themselves to matters of content and substance. Journal publishers have got wise to the fact that they can save a lot of money by sacking all the copy editors and requiring the academic editor to do the work instead, and they realise they have hit gold if they can find a natural Obsessive to do this. Academics should be aware of this trap: their training equips them to judge the science, and they should not spend hours looking for extraneous full stops and missing italicisation.  They should remember that they already do work, typically for no reward, for publishers who make a lot of money from journals, and they should demand that the publisher offers appropriate support to them and their authors.  (They should also employ people to assist authors with graphics – see blog).

Summing up
The main problem for authors is that you often don’t know what species of editor you are dealing with until after the event of submitting a paper. In my main field of psychology, I am impressed at how many journals do have Paragons. The APA journals are usually good, in my experience, though Obsessives do make an appearance, and I know of one case where a junior colleague’s career was seriously blighted by a mega-Sloth.  It’s harder to generalise about small moderate-impact journals: many of them are overseen by a dedicated Paragon, but my impression is that you can only be a Paragon for 10 years at most. Editors who have served a longer term than this are liable to transmute into Sloths.  I also publish in the fields of neuroscience and genetics, and here I’ve struck more variability, with Returning Officers, Automatons and Deities being fairly prevalent.  If you want to publish in the really top journals, you have to grapple with Talent Scouts:  my attempts to make my work exciting enough for them have been singularly unsuccessful, and I've given up on them, but it may encourage younger readers to know that I’ve had a happy and successful career all the same.

Note: The author was co-editor of Journal of Child Psychology and Psychiatry from 1990-1993 and Chief Editor from 1994-1997. This year she signed up as an Academic Editor for PLOS One, in support of their Open Access publishing policy.

4 comments:

  1. I had a Vacillator statistica who ended up holding up a paper for nearly 2 years. The statistician turned out to be perpetually on holiday or away; when after nearly a year, he eventually did read the paper, he misunderstood it and objected to it based on something he thought we were saying that we weren't.

    I think it can work if the statistician is used to reviewing the kind of papers that the journal deals with, and who is actually being paid for it so he does it quickly and professionally.

    ReplyDelete
  2. This is one of the interesting peace of work about it . I really like it.


    Flowers to Pakistan

    ReplyDelete
  3. haha! My editor was a paragon in previous reviews but in the latest process seemed to become a vascillator and then a sloth. Do these species cross-breed? How else to explain it?

    ReplyDelete