Kafkarna continues: REF gloves off at Lancaster University

I thought I would reblog this. Partly because I haven’t had a chance to write much recently and it gives me an opportunity to keep things ticking over, partly because it seems like something worth highlighting (although given my readership, this may not help much), and partly because I’ve written about the REF (and been quite critical) and this type of activity is one of the things that I expected to happen and illustrates the issues – in my opinion – with this type of assessment process. I particularly like, and agree with, this comment in the post Whatever this charade is, it is not a framework for research excellence and so I recommend giving this a good read.


Two days ago an open letter from Professor Paolo Palladino, former Head of the History Department at Lancaster University, appeared on the university’s internal History Staff list serve, with copies sent to senior administrators in the Faculty of Arts and Social Sciences (FASS) and Central Administration.  I responded the next day with a post supporting Professor Palladino’s position, sent to the same recipients. With Paolo’s permission, I am reproducing both of our letters here.  We both believe that the issues raised by Lancaster’s selective culling of research-active staff from submission in the 2014 REF–a practice in which it is not alone among British universities–deserve the widest possible public debate.  We would therefore urge anyone who shares our concerns to share this post through social media.

Professor Palladino’s letter:
Dear all,
Over the next few days, a number of colleagues across the university are to be informed that they will not…

View original post 1,060 more words

REF, QR funding and the science budget

I guess we should all be reasonably pleased that the science budget has remained ring-fenced. The reality, of course, is that this just means that it is ring-fenced in a flat-cash sense, not in an inflation adjusted sense. As scientists, maybe we should make sure they define the term more specifically in future. It does appear that the current definition would allow the government to claim that something has been ring-fenced, despite the spending power tending to zero. Logically, you mighty expect it to be defined in terms of spending power rather than pounds, but that would require deciding between RPI and CPI, and that is clearly for too difficult.

Anyway, enough cynicism. In truth, we should probably be grateful that the outcome hasn’t been worse. It certainly has been for some and I do feel that the current government has got its basic economic policies completely wrong. It seems like it’s time that someone explained to George Osborne that it’s not necessarily the size of the debt and deficit that matters. What matters is their size relative to the size of our economy. The way its going now it seems like he’s getting it wrong on both counts.

What is maybe more concerning, from a science funding perspective, is the possibility that the government may choose to axe the QR funding. This is the funding stream that comes from the Higher Education Funding Councils and how it is distributed is determined via the outcome of the Research Excellence Framework (REF) exercise. Now, it may well be part of the ring fence and it may well be safe, but I wouldn’t know whether to laugh or cry if it were cut. If you’ve read some of my earlier posts, you’ll know that I’ve been very critical of REF. This is both due to the manner in which it is implemented and due to the shenanigans taking place at UK universities; the potentially risking hiring, the morally/legally questionable redundancies, and the time and effort spent preparing for what is – in my opinion – a completely flawed exercise.

So, if it were to be cut, part of me would feel like saying “serves us right for taking something so silly so seriously, and for playing the kind of games that have not and will not benefit our fundamental role as teachers and researchers”. On the other hand, it is a lot of money (£1.5 billion I believe) and I certainly have no desire to see this money leave the Higher Education sector. As far as I can tell, some may struggle to survive as they are, even if there are no cuts to the QR funding. Well, I certainly hope that it isn’t cut but I also hope that in future, universities will find the backbone to tell the government that playing these kind of games is silly and that they should find a simpler and more effective mechanism for distributing this money (although I don’t think it should simply be given to the research councils, but that might be a topic for another post).

Declaration on Research Assessment

Just thought I would highlight the San Francisco Declaration on Research Assessment. It’s been a long day and I’m quite tired, so I don’t want to say too much. You can read most of this yourself, but basically a group of editors and publishers of scholarly journals met during the Annual Meeting of The American Society for Cell Biology (ASCB), and have come up with a set of recommendations about the use of journal-based metrics. The basic motivation was

  • the need to eliminate the use of journal-based metrics, such as Journal Impact Factors, in funding, appointment, and promotion considerations;

  • the need to assess research on its own merits rather than on the basis of the journal in which the research is published; and

  • the need to capitalize on the opportunities provided by online publication (such as relaxing unnecessary limits on the number of words, figures, and references in articles, and exploring new indicators of significance and impact).

  • A lot of it seemed to focus on the use of Journal Impact factors when assessing individual bits of research (which, as many have already pointed out, is horribly flawed) but there was indication that this was attempting to go further than just that. Citation metrics can be useful but can also be problematic. There can be a huge range of different practices even with the same basic area and so using them alone to assess an individual can disadvantage some potentially excellent researchers. It can also be advantageous to some who aren’t particularly good but who just happen to work in an area that is currently popular and in which papers are collecting citations easily.

    Anyway, this is an encouraging step and I hope it has some impact and that it’s taken seriously by funding agencies, interview panels and promotion boards. I suspect it’s too late to have much effect on the REF2014 panels, but maybe there’s hope for REF2021.

    The role of universities!

    I haven’t had much chance to write posts recently, although I don’t have much of a following so I suspect not many have actually noticed. I’ve been quite busy with some other things and am also trying to write a talk for a conference next week. My recent post about the integrity of universities did, however, generate a little bit of interest. If you’re interested in that topic can I recommend that you read some of the posts on Richard Hall’s Space. He has a lot of interesting posts about the role of universities and their place in society. The only real criticism I have of his posts is that they are written in a manner that requires quite a lot of concentration. This isn’t necessarily a bad thing, but does mean that I find them quite difficult to understand when I’m perusing them in the evenings after a long day at work.

    The integrity of universities

    There’s a very interesting (in my opinion) article by George Monbiot in the Guardian today. The article is called Oxford University won’t take funding from tobacco companies, but Shell’s OK. The basic premise is that universities should be acting for the common good, or as George Monbiot puts it

    the need for a disinterested class of intellectuals which acts as a counterweight to prevailing mores.

    I have to say that I agree completely with this. It has always surprised me how disinterested UK academics can be. I had always assumed that university academics had a role to play in defining what is acceptable in our societies. They are meant to be the intellectual members of our society; the people who think. If academics are reluctant to be involved in this, then who else is going to do it? This isn’t to say that everyone should bow to the views of academics, simply that academics should feel free to question what is accepted in our societies.

    There are probably many reasons why UK academics are reluctant to engage in discussions about our society. One may simply be that academics have become very focused. They see themselves as experts in quite specific areas and so don’t see it as appropriate to engage in areas outside their expertise. There is some merit to this, but it is a bit disappointing – in my opinion. Another may be that there is now quite a lot of pressure on academics. Universities have become very bureaucratic and there is quite a strong publish-or-perish attitude. Academics don’t have much spare time to contemplate the merits – or lack thereof – of our societal mores. Universities have also become much more like businesses. The goal is to maximise teaching and research income and, hence, academics are discouraged from doing anything that doesn’t enhance a university’s ability to generate income.

    Personally, I think the latter is the main reason why universities (in the UK) are no longer hotbeds of dissent. We are publicly funded and hence need to do what is expected of us. I’m often very critical, for example, of the Research Excellence Framework (REF2014) and even though most seem to agree, the typical response is “we just have to do this”. Well, yes, but do we have to do it happily. If we think it is damaging what we regard as strengths of the UK Higher Education system, shouldn’t we be making it clear that we’re doing it under duress. There’s also this view that we have to do what is best for UK PLC (i.e., what will best help economic growth in the UK). In a sense, I agree with this. What I disagree with is how we’re influenced to do this. University research has, for many decades, had a very positive impact on economic growth. However, this didn’t happen because politicians told universities to do this. It’s because people recognised the significance of some piece of research and used it to develop something that had economic value. It’s also largely unpredictable. It’s certainly my view that telling us to start predicting the economic benefit of our research will do more harm than good.

    The final thing I was going to say regards the main thrust of George Monbiot’s article. If there is increasing evidence that global warming is happening (as there is) and if there is evidence that such warming could lead to life-threatening climate change, shouldn’t the universities where this research is taking place act as though it’s important. Maybe universities shouldn’t be accepting funding from oil companies, if their own research indicates that we should significantly reduce our use of fossil fuels. I have heard some argue that we shouldn’t worry about the provenance of our funding as we’ll typically do good things with whatever money we can get. I think this is naive. The idea that one can take funding from oil companies without being influenced by the source of this funding seems highly unlikely.

    Niall Ferguson and John Maynard Keynes

    Niall Ferguson has been heavily, and quite rightly, criticised for suggesting that John Maynard Keynes’s economic ideas were influenced by his homosexuality and by his lack of children. The idea was that by not having children he was less concerned about the long-term impact of his economic ideas. Niall Ferguson’s claim is, in my opinion, highly offensive and entirely baseless. Niall Ferguson has now, however, produced an unqualified apology. As far as apologies go, this is pretty textbook. No excuses, no attempts to suggest that we needed to understand the context, simply an unreserved apology and an acknowledgement that he was wrong.

    Okay, but this doesn’t erase what he has said and even if one can accept the apology, I don’t see how one cannot still conclude that, ultimately, Niall Ferguson is homophobic and that one should be careful about how seriously to take any future comments that he makes.

    This reminded me, for some reason, of something I heard on the radio yesterday. The comment on the radio was that the Tory party would have to convince voters that they weren’t privileged and that they weren’t out-pf-touch with regards the realities of life in the UK. The first thing I thought when I heard this was that surely the only way they could do this was by lying. It’s fairly clear that a majority of leading Tories are extremely privileged and have lifestyles very different to the lifestyles lead by most in the UK.

    The Tories could try to convince voters that despite this they can still govern in a way that would be acceptable to most and that their policies will be aimed at making the country better for all, rather than simply for those who are privilege like themselves. This is very different to convincing voters that they aren’t privilege and out-of-touch, but at least it would seem to be honest.

    I guess, what struck me was that we now live in a world where people feel that they have to say what they think others want to hear. In a sense, I would rather that people were more honest. That’s not to say that I want to have high-profile people who are homophobic or government leaders who are openly dismissive of people who are not as privileged as they are. What I would like is to know what people really believe so that we can make informed decisions about who should be “allowed” to have a high-profile role in society. The media can choose not to give a platform to those who are openly homophobic (for example) and we can choose not to elect those who, deep down, don’t really wish to run the country in a manner that would be optimal for the majority. I know this is naive and simplistic, but I guess – to be honest – that’s probably what I am.

    REF and teaching

    There’s a recent article in the Guardian about the influence of the Research Excellence Framework (REF2014) on university teaching. The basic issue is that money will only be allocated on the basis of papers that score highly (3* and 4*) and that league table rankings will also be determined by these high-ranking papers. Therefore, there is an incentive for universities to only submit researchers who have enough (essentially 4) papers that will be judged to be 3* or 4*. The concern is therefore that those who do not qualify will be encouraged (forced) to focus primarily on teaching or (as in the case of Queen Mary, University of London) face redundancy.

    Many universities are making “clear pledges that not being entered to the REF in November will not damage an academic’s career“. There are others, however, where this is clearly already having an impact (Queen Mary, University of London, Kings College and Strathclyde are three that I’ve heard about). I personally think that it is potentially a real problem. There is a big difference between how research and teaching are evaluated at universities, with an individual’s contribution to the research ranking being much more obvious than an individual’s contribution to any teaching ranking. One concern is that it will create a hierarchy within universities with some able to focus more on research and others “encouraged” to focus primarily on teaching and administration. I don’t have an issue with different people contributing to an academic department in different ways. I just would rather it were dynamic and evolved in some “natural” way, rather than being forced upon us by an external assessment exercise.

    University leaders are trying, in general, to make it clear that research and teaching are both valued parts of an academic’s career. The problem is that they don’t get to decide if the staff regard them as being of similar value. It certainly seems that even students are concerned about the impact that REF might have on the motivation of staff who might be judged to be “unworthy” and hence encouraged into having a larger role in teaching. I certainly think that these concerns are justified, even if there isn’t any evidence that REF is having, in general, this kind of impact.

    There do seem to be two common views expressed by those who are more supportive of REF than maybe I am. One is that it is not unreasonable to expect academics to publish 4 good papers every 7 years. In general I agree with this, although there may be some exceptions. However, there is a difference between publishing 4 good papers and publishing 4 papers that will be judged (by a panel – many of whom may not be particular expert in your field) to be good. Maybe about one-quarter of my papers have done quite well (in terms of citations) but I don’t really have a good idea why they did well and why others didn’t. I can’t really look back and claim that I can now tell why some papers would be judged to be good, while others would not. I’m typically quite pleased with most of the papers I publish. Whether or not they do well (in metric terms) all seems a little random to me.

    The other claim that is often made is that REF has forced universities to take hiring more seriously and that hiring is now based on excellence. Firstly, this is presumably only “perceived excellence” in research. One of the perennial criticisms of university hiring has been that teaching ability hasn’t been taken seriously enough. I really can’t see that REF has helped here. My feeling is that it may have made the situation worse. The other issue I have with this claim is that it suggests that the typical academic today is somehow better (because of REF) than they were 20 or 30 years ago. Really? I thought universities in the UK have been world-class for decades. I’m sure many academics who were active in the 60s, 70s, 80s and 90s might be slightly insulted by this suggestion. I suspect there were issues with hiring practices in those days, but that was probably more to do with societal issues that have been remedied via equalities legislation, than via REF.

    It strikes me that there has been quite a lot of recent coverage about the negative aspects of REF, so maybe some of it will sink in. Not that hopeful though. Maybe I should be considering holding back some of my current work so as to publish papers that will qualify for REF2021.