Sunday, 15 November 2015

Who's afraid of Open Data

Cartoon by John R. McKiernan, downloaded from:

I was at a small conference last year, catching up on gossip over drinks, and somehow the topic moved on to journals, and the pros and cons of publishing in different outlets. I was doing my best to advocate for open access, and to challenge the obsession with journal impact factors. I was getting the usual stuff about how early-career scientists couldn't hope to have a career unless they had papers in Nature and Science, but then the conversation took an interesting turn.

"Anyhow," said eminent Professor X. "One of my postdocs had a really bad experience with a PLOS journal."

Everyone was agog. Nothing better at conference drinks than a new twist on the story of evil reviewer 3.  We waited for him to continue. But the problem was not with the reviewers.

"Yup. She published this paper in PLOS Biology, and of course she signed all their forms. She then gave a talk about the study, and there was this man in the audience, someone from a poky little university that nobody had ever heard of, who started challenging her conclusions. She debated with him, but then, when she gets back she has an email from him asking for her data."

We wait with bated breath for the next revelation.

"Well, she refused of course, but then this despicable person wrote to the journal, and they told her that she had to give it to him! It was in the papers she had signed."

Murmurs of sympathy from those gathered round. Except, of course, me. I just waited for the denouement. What had happened next, I asked.

"She had to give him the data. It was really terrible. I mean, she's just a young researcher starting out."

I was still waiting for the denouement. Except that there was no more. That was it! Being made to give your data to someone was a terrible thing. So, being me, I asked, why was that a problem? Several people looked at me as if I was crazy.

"Well, how would you like it if you had spent years of your life gathering data, data which you might want to analyse further, and some person you have never heard comes out of nowhere demanding to have it?"

"Well, they won't stop you analysing it," I said.

"But they may scoop you and find something interesting in it before you have a chance to publish it!"

I was reminded of all of this at a small meeting that we had in Oxford last week, following up on the publication of a report of a symposium I'd chaired on Reproducibility and Reliability of Biomedical Research. Thanks to funding by the St John's College Research Centre, a small group of us were able to get together to consider ways in which we could take forward some of the ideas in the report for enhancing reproducibility. We covered a number of topics, but the one I want to focus on here is data-sharing.

A move toward making data and analyses open is being promoted in a top-down fashion by several journals, and universities and publishers have been developing platforms to make this possible. But many scientists are resisting this process, and putting forward all kinds of argument against it. I think we have to take such concerns seriously: it is all too easy to mandate new actions for scientists to follow that have unintended consequences and just lead to time-wasting, bureaucracy or perverse incentives. But in this case I don't think the objections withstand scrutiny. Here are the main ones we identified at our meeting:

1.  Lack of time to curate data;  Data are only useful if they are understandable, and documenting a dataset adequately is a non-trivial task;

2.  Personal investment - sense of not wanting to give away data that had taken time and trouble to collect to other researchers who are perceived as freeloaders;

3. Concerns about being scooped before the analysis is complete;

4.  Fear of errors being found in the data;

5.  Ethical concerns about confidentiality of personal data, especially in the context of clinical research;

6.  Possibility that others with a different agenda may misuse the data, e.g. perform selective analysis that misrepresented the findings;

These have partial overlap with points raised by Giorgio Ascoli (2015) when describing NeuroMorpho.Org, an online data-sharing repository for digital reconstructions of neuronal morphology. Despite the great success of the repository, it is still the case that many people fail to respond to requests to share their data, and points 1 and 2 seemed the most common reasons.

As Ascoli noted, however, there are huge benefits to data-sharing, which outweigh the time costs. Shared data can be used for studies that go beyond the scope of the original work, with particular benefits arising when there is pooling of datasets. Some illustrative examples from the field of brain imaging were provided by Thomas Nichols at our meeting (slides here), where a range of initiatives is being developed to facilitate open data. Data-sharing is also beneficial for reproducibility: researchers will check data more carefully when it is to be shared, and even if nobody consults the data, the fact it is available gives confidence in the findings. Shared data can also be invaluable for hands-on training. A nice example comes from Nicole Janz, who teaches a replication workshop in social sciences in Cambridge, where students pick a recently published article in their field and try to obtain the data so they can replicate the analysis and results.

These are mostly benefits to the scientific community, but what about the 'freeloader' argument? Why should others benefit when you have done all the hard work? In fact, when we consider that scientists are usually receiving public money to make scientific discoveries, this line of argument does not appear morally defensible. But in any case, it is not true that the scientists who do the sharing have no benefits. For a start, they will see an increase in citations, as others use their data. And another point, often overlooked, is that uncurated data often become unusable by the original researcher, let alone other scientists, if it is not documented properly and stored on a safe digital site. Like many others, I've had the irritating experience of going back to some old data only to find I can't remember what some of the variable names refer to, or whether I should be focusing on the version called final, finalfinal, or ultimate. I've also had the experience of data being stored on a kind of floppy disk, or coded by a software package that had a brief flowering of life for around 5 years before disappearing completely.

Concerns about being scooped are frequently cited, but are seldom justified. Indeed, if we move to a situation where a dataset is a publication with its own identifier, then the original researcher will get credit every time someone else uses the dataset. And in general, having more than one person doing an analysis is an important safeguard, ensuring that results are truly replicable and not just a consequence of a particular analytic decision (see this article for an illustration of how re-analysis can change conclusions).

The 'fear of errors' argument is, of understandable but not defensible. The way to respond is to say of course there will be errors – there always are. We have to change our culture so that we do not regard it as a source of shame to publish data in which there are errors, but rather as an inevitability that is best dealt with by making the data public so the errors can be tracked down.

Ethical concerns about confidentiality of personal data are a different matter. In some cases, participants in a study have been given explicit reassurances that their data will not be shared: this was standard practice for many years before it was recognised that such blanket restrictions were unhelpful and typically went way beyond what most participants wanted – which was that their identifiable data would not be shared.  With training in sophisticated anonymization procedures, it is usually possible to create a dataset that can be shared safely without any risk to the privacy of personal information; researchers should be anticipating such usage and ensuring that participants are given the option to sign up to it.

Fears about misuse of data can be well-justified when researchers are working on controversial areas where they are subject to concerted attacks by groups with vested interests or ideological objections to their work. There are some instructive examples here and here. Nevertheless, my view is that such threats are best dealt with by making the data totally open. If this is done, any attempt to cherrypick or distort the results will be evident to any reputable scientist who scrutinises the data. This can take time and energy, but ultimately an unscientific attempt to discredit a scientist by alternative analysis will rebound on those who make it.  In that regard, science really is self-correcting. If the data are available, then different analyses may give different results, but a consensus of the competent should emerge in the long run, leaving the valid conclusions stronger than before.

I'd welcome comments from those who have started to use open data and to hear your experiences, good or bad.

P.S. As I was finalising this post, I came across some great tweets from the OpenCon meeting taking place right now in Brussels. Anyone seeking inspiration and guidance for moving to an open science model should follow the #opencon hashtag, which links to materials such as these: Slides from keynote by Erin McKiernan, and resources at

Friday, 11 September 2015

The rationalist spa: an unmet need

I’ve had a few spa experiences, though I’m hardly a connoisseur. I've found  a spa is a good place to chill out and gossip with friends, and on the rare occasions when I stay at a fancy hotel, I’ve come to enjoy the process of whiling away the period between lunch and cocktails in a hot bubbly tub.

At the same time, I’ve become fascinated by the language used to advertise the available activities. I suspect most people who go to a spa don’t have any specific ailments, but they want to come away feeling more relaxed and vibrant, and the talk of 'therapies' and 'treatments' manages to create the comforting illusion that health can be shifted from suboptimal to optimal by various potions and practices. Particularly intriguing is the focus on foodstuffs. In a spa, you don’t ingest food: you rub it on the skin, sniff it, or slather yourself with it. I’ve always suspected there is an element of sublimation in this: most women who wish to remain slender have to suppress the desire to eat delicious things, and the spa provides an opportunity to interact with food without getting fat.
In the hotel I’ve been staying in you could be massaged with essential oils of grapefruit, have mango butter rubbed on your feet or head, be scrubbed with papaya and mandarin, or have your muscles relaxed by a concoction of rice and milk. Rather disappointingly, they didn't offer the ultimate decadence: a ‘chocolate wrap’, where you start with a warm milk soak, then get scrubbed with vanilla and bran, before being enveloped in a warm cocoa butter ‘masque’ (a word that always reminds me of Edgar Allan Poe's scary story).
It’s clear that there’s a lucrative market for this kind of thing. Many of us feel stressed by modern life, and if being smothered in chocolate or fruit makes us feel relaxed, why not?
Alas, though, for me the sense of relaxation is counteracted by irritation with the garbage that you have to endure while undergoing something as basic as a pedicure. Some of it is just overblown advertising guff, e.g. “Drawing on the elemental wisdom of nature, our treatments both invoke and restore the body’s natural equilibrium”. What does this even mean? The images are of blockage and decay being removed: “Clearing stagnant energy is the focus of the Spring Clean Scrub”. But the worst examples are those with medical overtones, with talk of healing, detoxification, and wellness. We are told that: “This wrap is highly effective in purging toxins and boosting the blood and lymphatic circulation” or "The polyphenols in cocoa delay the ageing process, causing you to look younger". Or even “The body is generously encased within a cooling serum”. Cripes! If you look up serum in a dictionary, this is a seriously scary idea.
I have, of course, always just gone with the flow. Attempting to debate the scientific basis of aromatherapy or ayurveda with a practitioner who makes their living administering these treatments is unlikely to make either of us more relaxed or vibrant. But I do wish someone would open a spa for rationalists, where one could go and get a good massage or get encased in mud just for the fun of it, without a lot of guff about energy blockages and deep-seated toxins.

Sunday, 30 August 2015

Opportunity cost: A new red flag for evaluating interventions for neurodevelopmental disorders

Back in 2012, I wrote a blogpost offering advice to parents who were trying to navigate their way through the jungle of alternative interventions for children with dyslexia. I suggested a set of questions that should be asked of any new intervention, and identified a set of 'red flags', i.e., things that should make people think twice before embracing a new treatment.

The need for an update came to mind as I reflected on the Arrowsmith program, an educational approach that has been around in Canada since the 1980s, but has recently taken Australia and New Zealand by storm. Despite credulous press coverage in the UK, Arrowsmith has not, as far as I know, taken off here. Australia, however, is a different story, with Arrowsmith being taken up by the Catholic Education Office in Sydney after they found 'dramatic results' in a pilot evaluation.

For those who remember the Dore programme, this seems like an action replay. Dore was big in both the UK and Australia in the period around 2007-2008. Like Dore, it used the language of neuroscience, claiming that its approach treated the underlying brain problem, rather than the symptoms of conditions such as dyslexia and ADHD. Parents were clamouring for it, it was widely promoted in the media, and many people signed up for long-term payment plans to cover a course of treatment. People like me, who worked in the area of neurodevelopmental disorders, were unimpressed by the small amount of published data on the program, and found the theoretical account of brain changes unconvincing (see this critique). However, we were largely ignored until a Four Corners documentary was made by Australian ABC, featuring critics as well as advocates of Dore. Soon after, the company collapsed, leaving both employees of Dore and many families who had signed up to long-term financial deals, high and dry. It was a thoroughly dismal episode in the history of intervention for children with neurodevelopmental problems.

With Arrowsmith, we seem to be at the start of a similar cycle in Australia. Parents, hearing about the wondrous results of the program, are lobbying for it to be made more widely available. There are even stories of parents moving to Canada so that their child can reap the benefits of Arrowsmith. Yet Arrowsmith ticks many of the 'red flags' that I blogged about, lacks any scientific evidence for efficacy, and has attracted criticism from mainstream experts in children's learning difficulties. As with Dore, the Arrowsmith people seem to have learned that if you add some sciency-sounding neuroscience terms to justify what you do, people will be impressed. It is easy to give the impression that you are doing something much more remarkable than just training skills through repetition.

They also miss the point that, as Rabbitt (2015, p 235) noted regarding brain-training in general: "Many researchers have been frustrated to find that ability on any particular skill is surprisingly specific and often does not generalise even to other quite similar situations." There's little point in training children to type numbers into a computer rapidly if all that happens is that they get better at typing numbers into a computer. For this to be a viable educational strategy, you'd need to show that this skill had knock-on effects on other learning. That hasn't been done, and all the evidence from mainstream psychology suggests it would be unusual to see such transfer of training effects.

Having failed to get a reply to a request for more information from the Catholic Education Office in Sydney, I decided to look at the evidence for the program that was cited by Arrowsmith's proponents. An ongoing study by Dr Lara Boyd of the University of British Columbia features prominently on their website, but, alas, Dr Boyd was unresponsive to an email request for more information. It would seem that in the thirty-five years Arrowsmith has been around, there have been no properly conducted trials of its effectiveness, but there are a few reports of uncontrolled studies looking at children's cognitive scores and attainments before and after the intervention. One of the most comprehensive reviews is in the D.Phil. thesis of Debra Kemp-Koo from the University of Saskatchewan in 2013. In her introduction, Dr Kemp-Koo included an account of a study of children attending the private Arrowsmith school in Toronto:
All of the students in the study completed at least one year in the Arrowsmith program with most of them completing two years and some of them completing three years. At the end of the study many students had completed their Arrowsmith studies and left for other educational pursuits. The other students had not completed their Arrowsmith studies and continued at the Arrowsmith School. Most of the students who participated in the study were taking 6 forty minute modules of Arrowsmith programming a day with 1 forty minute period a day each of English and math at the Arrowsmith School. Some of the students took only Arrowsmith programming or took four modules of Arrowsmith programming with the other half of their day spent at the Arrowsmith school or another school in academic instruction (p. 34-35; my emphasis).
Two of my original red flags concerned financial costs, but I now realise it is important to consider opportunity costs: i.e., if you enlist your child in this intervention, what opportunities are they going to miss out as a consequence? For many of the interventions I've looked at, the time investment is not negligible, but Arrowsmith seems in a league of its own. The cost of spending one to three years working on unevidenced, repetitive exercises is to miss out on substantial parts of a regular academic curriculum. As Kemp-Koo (2013) remarked:
The Arrowsmith program itself does not focus on academic instruction, although some of these students did receive some academic instruction apart from their Arrowsmith programming. The length of time away from academic instruction could increase the amount of time needed to catch up with the academic instruction these students have missed. (p. 35; my emphasis).

Kemp-Koo, D. (2013). A case study of the Learning Disabilities Association of Saskatchewan (LDAS) Arrowsmith Program. Doctor of Philosophy thesis, University of Saskatchewan, Saskatoon.  

Rabbitt, P. M. A. (2015). The aging mind. London and New York: Routledge.

Saturday, 11 July 2015

Publishing replication failures: some lessons from history

I recently travelled to Lismore, Ireland, to speak at the annual Robert Boyle summer school. I had been intrigued by the invitation, as it was clear this was not the usual kind of scientific meeting. The theme of Robert Boyle, who was born in Lismore Castle, was approached from very different angles, and those attending included historians of science, scientists, journalists, as well as interested members of the public. We were treated to reconstructions of some of Boyle's livelier experiments, heard wonderful Irish music, and we celebrated the installation of a plaque at Lismore Castle to honour Katherine Jones, Boyle's remarkable sister, who was also a scientist.

My talk was on the future of scientific scholarly publication, a topic that the Royal Society had explored in a series of meetings to celebrate the 350th Anniversary of the publication of Philosophical Transactions. I'm particularly interested in the extent to which current publishing culture discourages good science, and I concluded by proposing the kind of model that I recently blogged about, where the traditional science journal is no longer relevant to communicating science.

What I hadn't anticipated was the relevance of some of Boyle's writing to such contemporary themes.

Boyle, of course, didn't have to grapple with issues such as the Journal Impact Factor or Open Access payments. But some of the topics he covered are remarkably contemporary. He would have been interested in the views of Jason Mitchell, John L. Loeb Associate Professor of the Social Sciences at Harvard, who created a stir last year by writing a piece entitled "On the emptiness of failed replications". I see that the essay has now been removed from the Harvard website, but the main points can be found here*. It was initially thought to be a parody, but it seems to have been a sincere attempt at defending the thesis that "unsuccessful experiments have no meaningful scientific value." Furthermore, according to Mitchell, "Whether they mean to or not, authors and editors of failed replications are publicly impugning the scientific integrity of their colleagues." I have taken issue with this standpoint in an earlier blogpost; my view is that we should not assume that a failure to replicate a result is due to fraud or malpractice, but rather should encourage replication attempts as a means of establishing which results are reproducible.

I am most grateful to Eoin Gill of Calmast for pointing me to Robert Boyle's writings on this topic, and for sending me transcripts of the most relevant bits. Boyle has two essays on "the Unsuccessfulness of Experiments" in a collection of papers entitled “Certain Physiological Essays and other Tracts”. In these he discusses (at inordinate length!) the problems that arise when an experimental result fails to replicate. He starts by noting that such unsuccessful experiments are not uncommon:
… in the serious and effectual prosecution of Experimental Philosophy, I must add one discouragement more, which will perhaps as much surprize you as dishearten you; and it is, That besides that you will find …… many of the Experiments publish'd by Authors, or related to you by the persons you converse with, false or unsuccessful, … you will meet with several Observations and Experiments, which though communicated for true by Candid Authors or undistrusted Eye-witnesses, or perhaps recommended to you by your own experience, may upon further tryal disappoint your expectation, either not at all succeeding constantly, or at least varying much from what you expected. (opening passage)
He is interested in exploring the reasons for such failure; his first explanation seems equivalent to one that those using statistical analyses are all too familiar with – a chance false positive result.
And that if you should have the luck to make an Experiment once, without being able to perform the same thing again, you might be apt to look upon such disappointments as the effects of an unfriendliness in Nature or Fortune to your particular attempts, as proceed but from a secret contingency incident to some experiments, by whomsoever they be tryed. (p. 44)
And he urges the reader not to be discouraged – replication failures happen to everyone!
…. though some of your Experiments should not always prove constant, you have divers Partners in that infelicity, who have not been discouraged by it. (p. 44)
He identifies various possible systematic reasons for such failure: a problem with skill of the experimenter, with purity of ingredients, or variation in the specific context in which the experiment is conducted. He even, implicitly, addresses statistical power, noting how one needs many observations to distinguish what is general from individual variation.
…the great variety in the number, magnitude, position, figure, &c. of the parts taken notice of by Anatomical Writers in their dissections of that one Subject the humane body, about which many errors would have been delivered by Anatomists, if the frequency of dissections had not enabled them to discern betwixt those things that are generally and uniformly found in dissected bodies, and those which are but rarely, and (if I may so speak) through some wantonness or other deviation of Nature, to be met with. (p. 94)
Because of such uncertainties, Boyle emphasises the need for replication, and the dangers of building complex theory on the basis of a single experiment:
….try those Experiments very carefully, and more than once, upon which you mean to build considerable Superstructures either theorical or practical, and to think it unsafe to rely too much upon single Experiments, especially when you have to deal in Minerals: for many to their ruine have found, that what they at first look'd upon as a happy Mineral Experiment has prov'd in the issue the most unfortunate they ever made. (p. 106)
I'm sure there are some modern scientists who must be thinking their lives may have been made much easier if they had heeded this advice. But perhaps the most relevant to the modern world, where there is such concern about the consequences of failure to replicate, are Boyle's comments on the reputational impact of publishing irreproducible results:
…if an Author that is wont to deliver things upon his own knowledge, and shews himself careful not to be deceived, and unwilling to deceive his Readers, shall deliver any thing as having try'd or seen it, which yet agrees not with our tryals of it; I think it but a piece of Equity, becoming both a Christian and a Philosopher, to think (unless we have some manifest reason to the contrary) that he set down his Experiment or Observation as he made it, though for some latent reason it does not constantly hold; and that therefore though his Experiment be not to be rely'd upon, yet his sincerity is not to be rejected. Nay, if the Author be such an one as has intentionally and really deserved well of Mankind, for my part I can be so grateful to him, as not only to forbear to distrust his Veracity, as if he had not done or seen what he says he did or saw, but to forbear to reject his Experiments, till I have tryed whether or no by some change of Circumstances they may not be brought to succeed. (p. 107)
The importance of fostering a 'no blame' culture was one theme that emerged in a recent meeting on Reproducibility and Reliability of Biomedical Research at the Academy of Medical Sciences. It seems that in this, as in so many other aspects of science, Boyle's views are well-suited to the 21st century.

For more on Robert Boyle, see here

12th July 2015: Thanks to DaniĆ«l Lakens who pointed me to the Wayback machine, where earlier versions of the article can be found:*/

Friday, 3 July 2015

Bishopblog catalogue (updated 3rd July 2015)


Those of you who follow this blog may have noticed a lack of thematic coherence. I write about whatever is exercising my mind at the time, which can range from technical aspects of statistics to the design of bathroom taps. I decided it might be helpful to introduce a bit of order into this chaotic melange, so here is a catalogue of posts by topic.

Language impairment, dyslexia and related disorders
The common childhood disorders that have been left out in the cold (1 Dec 2010) What's in a name? (18 Dec 2010) Neuroprognosis in dyslexia (22 Dec 2010) Where commercial and clinical interests collide: Auditory processing disorder (6 Mar 2011) Auditory processing disorder (30 Mar 2011) Special educational needs: will they be met by the Green paper proposals? (9 Apr 2011) Is poor parenting really to blame for children's school problems? (3 Jun 2011) Early intervention: what's not to like? (1 Sep 2011) Lies, damned lies and spin (15 Oct 2011) A message to the world (31 Oct 2011) Vitamins, genes and language (13 Nov 2011) Neuroscientific interventions for dyslexia: red flags (24 Feb 2012) Phonics screening: sense and sensibility (3 Apr 2012) What Chomsky doesn't get about child language (3 Sept 2012) Data from the phonics screen (1 Oct 2012) Auditory processing disorder: schisms and skirmishes (27 Oct 2012) High-impact journals (Action video games and dyslexia: critique) (10 Mar 2013) Overhyped genetic findings: the case of dyslexia (16 Jun 2013) The arcuate fasciculus and word learning (11 Aug 2013) Changing children's brains (17 Aug 2013) Raising awareness of language learning impairments (26 Sep 2013) Good and bad news on the phonics screen (5 Oct 2013) What is educational neuroscience? (25 Jan 2014) Parent talk and child language (17 Feb 2014) My thoughts on the dyslexia debate (20 Mar 2014) Labels for unexplained language difficulties in children (23 Aug 2014) International reading comparisons: Is England really do so poorly? (14 Sep 2014) Our early assessments of schoolchildren are misleading and damaging (4 May 2015)

Autism diagnosis in cultural context (16 May 2011) Are our ‘gold standard’ autism diagnostic instruments fit for purpose? (30 May 2011) How common is autism? (7 Jun 2011) Autism and hypersystematising parents (21 Jun 2011) An open letter to Baroness Susan Greenfield (4 Aug 2011) Susan Greenfield and autistic spectrum disorder: was she misrepresented? (12 Aug 2011) Psychoanalytic treatment for autism: Interviews with French analysts (23 Jan 2012) The ‘autism epidemic’ and diagnostic substitution (4 Jun 2012) How wishful thinking is damaging Peta's cause (9 June 2014)

Developmental disorders/paediatrics
The hidden cost of neglected tropical diseases (25 Nov 2010) The National Children's Study: a view from across the pond (25 Jun 2011) The kids are all right in daycare (14 Sep 2011) Moderate drinking in pregnancy: toxic or benign? (21 Nov 2012) Changing the landscape of psychiatric research (11 May 2014)

Where does the myth of a gene for things like intelligence come from? (9 Sep 2010) Genes for optimism, dyslexia and obesity and other mythical beasts (10 Sep 2010) The X and Y of sex differences (11 May 2011) Review of How Genes Influence Behaviour (5 Jun 2011) Getting genetic effect sizes in perspective (20 Apr 2012) Moderate drinking in pregnancy: toxic or benign? (21 Nov 2012) Genes, brains and lateralisation (22 Dec 2012) Genetic variation and neuroimaging (11 Jan 2013) Have we become slower and dumber? (15 May 2013) Overhyped genetic findings: the case of dyslexia (16 Jun 2013)

Neuroprognosis in dyslexia (22 Dec 2010) Brain scans show that… (11 Jun 2011)  Time for neuroimaging (and PNAS) to clean up its act (5 Mar 2012) Neuronal migration in language learning impairments (2 May 2012) Sharing of MRI datasets (6 May 2012) Genetic variation and neuroimaging (1 Jan 2013) The arcuate fasciculus and word learning (11 Aug 2013) Changing children's brains (17 Aug 2013) What is educational neuroscience? ( 25 Jan 2014) Changing the landscape of psychiatric research (11 May 2014)

Book review: biography of Richard Doll (5 Jun 2010) Book review: the Invisible Gorilla (30 Jun 2010) The difference between p < .05 and a screening test (23 Jul 2010) Three ways to improve cognitive test scores without intervention (14 Aug 2010) A short nerdy post about the use of percentiles (13 Apr 2011) The joys of inventing data (5 Oct 2011) Getting genetic effect sizes in perspective (20 Apr 2012) Causal models of developmental disorders: the perils of correlational data (24 Jun 2012) Data from the phonics screen (1 Oct 2012)Moderate drinking in pregnancy: toxic or benign? (1 Nov 2012) Flaky chocolate and the New England Journal of Medicine (13 Nov 2012) Interpreting unexpected significant results (7 June 2013) Data analysis: Ten tips I wish I'd known earlier (18 Apr 2014) Data sharing: exciting but scary (26 May 2014) Percentages, quasi-statistics and bad arguments (21 July 2014)

Journalism/science communication
Orwellian prize for scientific misrepresentation (1 Jun 2010) Journalists and the 'scientific breakthrough' (13 Jun 2010) Science journal editors: a taxonomy (28 Sep 2010) Orwellian prize for journalistic misrepresentation: an update (29 Jan 2011) Academic publishing: why isn't psychology like physics? (26 Feb 2011) Scientific communication: the Comment option (25 May 2011) Accentuate the negative (26 Oct 2011) Publishers, psychological tests and greed (30 Dec 2011) Time for academics to withdraw free labour (7 Jan 2012) Novelty, interest and replicability (19 Jan 2012) 2011 Orwellian Prize for Journalistic Misrepresentation (29 Jan 2012) Time for neuroimaging (and PNAS) to clean up its act (5 Mar 2012) Communicating science in the age of the internet (13 Jul 2012) How to bury your academic writing (26 Aug 2012) High-impact journals: where newsworthiness trumps methodology (10 Mar 2013) Blogging as post-publication peer review (21 Mar 2013) A short rant about numbered journal references (5 Apr 2013) Schizophrenia and child abuse in the media (26 May 2013) Why we need pre-registration (6 Jul 2013) On the need for responsible reporting of research (10 Oct 2013) A New Year's letter to academic publishers (4 Jan 2014) Will Elsevier say sorry? (21 Mar 2015) How long does a scientific paper need to be? (20 Apr 2015) Will traditional science journals disappear? (17 May 2015) My collapse of confidence in Frontiers journals (7 Jun 2015)

Social Media
A gentle introduction to Twitter for the apprehensive academic (14 Jun 2011) Your Twitter Profile: The Importance of Not Being Earnest (19 Nov 2011) Will I still be tweeting in 2013? (2 Jan 2012) Blogging in the service of science (10 Mar 2012) Blogging as post-publication peer review (21 Mar 2013) The impact of blogging on reputation ( 27 Dec 2013) WeSpeechies: A meeting point on Twitter (12 Apr 2014)

Academic life
An exciting day in the life of a scientist (24 Jun 2010) How our current reward structures have distorted and damaged science (6 Aug 2010) The challenge for science: speech by Colin Blakemore (14 Oct 2010) When ethics regulations have unethical consequences (14 Dec 2010) A day working from home (23 Dec 2010) Should we ration research grant applications? (8 Jan 2011) The one hour lecture (11 Mar 2011) The expansion of research regulators (20 Mar 2011) Should we ever fight lies with lies? (19 Jun 2011) How to survive in psychological research (13 Jul 2011) So you want to be a research assistant? (25 Aug 2011) NHS research ethics procedures: a modern-day Circumlocution Office (18 Dec 2011) The REF: a monster that sucks time and money from academic institutions (20 Mar 2012) The ultimate email auto-response (12 Apr 2012) Well, this should be easy…. (21 May 2012) Journal impact factors and REF2014 (19 Jan 2013)  An alternative to REF2014 (26 Jan 2013) Postgraduate education: time for a rethink (9 Feb 2013) High-impact journals: where newsworthiness trumps methodology (10 Mar 2013) Ten things that can sink a grant proposal (19 Mar 2013)Blogging as post-publication peer review (21 Mar 2013) The academic backlog (9 May 2013) Research fraud: More scrutiny by administrators is not the answer (17 Jun 2013) Discussion meeting vs conference: in praise of slower science (21 Jun 2013) Why we need pre-registration (6 Jul 2013) Evaluate, evaluate, evaluate (12 Sep 2013) High time to revise the PhD thesis format (9 Oct 2013) The Matthew effect and REF2014 (15 Oct 2013) Pressures against cumulative research (9 Jan 2014) Why does so much research go unpublished? (12 Jan 2014) The University as big business: the case of King's College London (18 June 2014) Should vice-chancellors earn more than the prime minister? (12 July 2014) Replication and reputation: Whose career matters? (29 Aug 2014) Some thoughts on use of metrics in university research assessment (12 Oct 2014) Tuition fees must be high on the agenda before the next election (22 Oct 2014) Blaming universities for our nation's woes (24 Oct 2014) Staff satisfaction is as important as student satisfaction (13 Nov 2014) Metricophobia among academics (28 Nov 2014) Why evaluating scientists by grant income is stupid (8 Dec 2014) Dividing up the pie in relation to REF2014 (18 Dec 2014) Journals without editors: What is going on? (1 Feb 2015) Editors behaving badly? (24 Feb 2015)  

Celebrity scientists/quackery
Three ways to improve cognitive test scores without intervention (14 Aug 2010) What does it take to become a Fellow of the RSM? (24 Jul 2011) An open letter to Baroness Susan Greenfield (4 Aug 2011) Susan Greenfield and autistic spectrum disorder: was she misrepresented? (12 Aug 2011) How to become a celebrity scientific expert (12 Sep 2011) The kids are all right in daycare (14 Sep 2011)  The weird world of US ethics regulation (25 Nov 2011) Pioneering treatment or quackery? How to decide (4 Dec 2011) Psychoanalytic treatment for autism: Interviews with French analysts (23 Jan 2012) Neuroscientific interventions for dyslexia: red flags (24 Feb 2012) Why most scientists don't take Susan Greenfield seriously (26 Sept 2014)

Academic mobbing in cyberspace (30 May 2010) What works for women: some useful links (12 Jan 2011) The burqua ban: what's a liberal response (21 Apr 2011) C'mon sisters! Speak out! (28 Mar 2012) Psychology: where are all the men? (5 Nov 2012) Should Rennard be reinstated? (1 June 2014) How the media spun the Tim Hunt story (24 Jun 2015)

Politics and Religion
Lies, damned lies and spin (15 Oct 2011) A letter to Nick Clegg from an ex liberal democrat (11 Mar 2012) BBC's 'extensive coverage' of the NHS bill (9 Apr 2012) Schoolgirls' health put at risk by Catholic view on vaccination (30 Jun 2012) A letter to Boris Johnson (30 Nov 2013) How the government spins a crisis (floods) (1 Jan 2014)

Humour and miscellaneous Orwellian prize for scientific misrepresentation (1 Jun 2010) An exciting day in the life of a scientist (24 Jun 2010) Science journal editors: a taxonomy (28 Sep 2010) Parasites, pangolins and peer review (26 Nov 2010) A day working from home (23 Dec 2010) The one hour lecture (11 Mar 2011) The expansion of research regulators (20 Mar 2011) Scientific communication: the Comment option (25 May 2011) How to survive in psychological research (13 Jul 2011) Your Twitter Profile: The Importance of Not Being Earnest (19 Nov 2011) 2011 Orwellian Prize for Journalistic Misrepresentation (29 Jan 2012) The ultimate email auto-response (12 Apr 2012) Well, this should be easy…. (21 May 2012) The bewildering bathroom challenge (19 Jul 2012) Are Starbucks hiding their profits on the planet Vulcan? (15 Nov 2012) Forget the Tower of Hanoi (11 Apr 2013) How do you communicate with a communications company? ( 30 Mar 2014) Noah: A film review from 32,000 ft (28 July 2014)

Wednesday, 24 June 2015

How the media spun the Tim Hunt story

 I had vowed not to blog about the Tim Hunt affair. I thought everything that could have been said had been said, and I'd made my own position clear in a comment on Athene Donald's blog, and in a comment in the Independent.
But then I came across Stephen Ballentyne's petition to "Bring Back Tim Hunt", and I was transported back five years to my first ever blog post on "Academic Mobbing in Cyberspace," a strange tale about sex, fruitbats and internet twittermobs. I started blogging in 2010 because I wanted to highlight how the internet encourages people to jump in to support causes without really examining the facts of the matter. The Ballentyne petition points to an uncannily similar conclusion.
Let me start out by saying I am not arguing against people's right to take Tim Hunt's side. As many people have noted, he is a well-liked man who has done amazing science and there are many women as well as men who will speak up for him as a supporter of female scientists. Many of those who support him do so in full knowledge of the facts, out of a sense of fairness and, in the case of those who know him personally, loyalty.
My concern is about the number of signatories of Ballentyne's petition who have got themselves worked up into a state of indignation on the basis of wrong information. There are three themes that run through the comments that many people have posted:
a) They think that Tim Hunt has been sacked from his job
b) They think he is 'lost to science'
c) They think University College London (UCL) fired him in response to a 'Twitter mob'.
None of these things is true. (a) Hunt is a retired scientist who was asked to resign from an honorary position.  That's shaming and unpleasant, but an order of magnitude different from being sacked and losing your source of income. (b) Hunt continues to have an affiliation to the Crick Institute – a flagship research centre that recently opened in Central London. (c) UCL are explicit that their acceptance of his resignation from an honorary position had nothing to do with the reaction on social media.
So why do people think these things? Quite simply, this is the interpretation that has been put about in many of the mainstream media. The BBC has been particularly culpable. The Today programme on Radio 4 ran a piece which started by saying Hunt had 'lost his job'. This was a couple of days after the UCL resignation, when any self-respecting journalist would have known this to be false. Many newspapers fuelled the flames. An interview with Boris Johnson on the BBC website added the fictitious detail that Hunt had been sacked by the Royal Society. He is in fact still a Fellow – he has simply been asked to step down from a Royal Society committee. It is interesting to ask why the media are so keen to promote the notion of Hunt as victim, cruelly dismissed by a politically correct university.
It's fascinating analysing the comments on the petition.  After deleting duplicates, there were 630 comments. Of those commenters where gender could be judged, 71% were male. Rather surprisingly, only 52% of commenters were from the UK, and 12% from the US, with the remainder scattered all over the world.
There were 93 comments that explicitly indicated they thought that Hunt had been sacked from his job, and/or was now 'lost to science' – and many more that called for his 'reinstatement', where it was unclear whether they were aware this was an honorary position.  They seemed to think that Hunt was dependent on UCL for his laboratory work, and that he had a teaching position. For instance, "Don't let the world lose a great scientist and teacher over a stupid joke." I would agree with them that if he had been sacked from a regular job, then UCL's action would have been disproportionate. However, he wasn't.
Various commentators drew comparisons with repressive fascist or Marxist states, e.g. "It is reminiscent of the cultural revolution in China where 'revisionist' professors were driven out of their offices by their prospective students, to do farm labour." And there was an awful lot of blaming of women, Twitter and feminism in general, with comments such as "Too much of this feminist ranting going on. Men need to get their spines back and bat it away" and "A respected and competent scientist has been hounded out of his job because of an ignorant baying twitter mob who don't happen to like his views". And my favourite: "What he said was a joke. If lesbian feminist women can't take a joke, then they are the joke." Hmm.
It's unfortunate that the spread of misinformation about Hunt's circumstances have muddied the waters in this discussion.  A minority of those commenting on Ballentyne's petition are genuine Hunt supporters who are informed of the circumstances; the bulk seem to be people who are concerned because they have believed the misinformation about what happened to Hunt; a further set are opportunistic misogynists who do Hunt no favours by using his story as a vehicle to support their dislike of women. There is a much more informed debate in the comments section on Athene Donald's blog, which I would recommend to anyone who wants to understand both sides of the story.