I thought I would highlight a recent article, Dodgy dealings in UK Higher Education, posted on the Guardian Blog, Occam’s Corner by Jenny Rohn. Jenny Rohn also write a blog called Mind the Gap on the Occam’s Typewriter blog portal. What I particularly liked in the article was the quote by Einstein, “Strive not to be a success, but rather to be of value.”. If you’ve read any of my earlier posts (The Negative Impact of REF), you’ll know that I’m not a fan of what is proposed for REF2014. Einstein’s quote, to a large extent, highlights my main issue with the process. There are likely to be some measures of success and the goal is to appear as successful as possible. There is, however, no guarantee that what we measure as successful has any intrinsic value. A highly cited paper isn’t necessarily right, or even valuable. Furthermore, those who are perceived to not be successful are not necessarily doing things that don’t have value.
The system may, therefore, measure some people as being unsuccessful and hence doing work that doesn’t have value. This could have a hugely negative effect on their careers, but may be completely wrong. Ideally we want to publish our research and for it to be noticed. However, we can’t always be certain that it will be noticed. I’ve published a number of papers that I’ve really thought would be of great interest, that have generated very little interest. I’ve also published a number papers that take a while to generate interest. One in particular is starting to have quite a lot of impact now, but was published in 2006. The article by Jenny Rohn was highlighting – in particular – the use of Impact Factors. The Impact Factor of a Journal is the average number of citations per paper. What is well known is that a small fraction of the papers typically have most of the citations. Using this as a measure of how good an individual’s papers are is largely nonsensical. Some argue that if you publish a paper in a high impact journal then that implies the paper must be good, even if it doesn’t get many citations. What you’ve really done is convince an editor and a few (sometimes only one) reviewers that your paper is worth publishing in that journal. Not necessarily a statistically significant sample.