Let the gaming begin!

Hoist by their own petard;

The UK is about to enter into one of the most important academic ranking exercises in its history. The Research Excellence Framework (or REF), starting in 2014, will determine how money is divided between departments and how the UK perceives the quality of its own universities and departments. As part of this process, university-based research-active academics throughout the UK will soon be submitting work to the REF. There will be one panel for each discipline, each made up of a number of highly esteemed academics who will review the submissions. This will be a difficult process for the panel to manage, with many thousands of submissions which are by their nature relatively new (published between 2008 and 2013).

Of course, economists are experts at decision-making under uncertainty, so we are uniquely well-placed to handle this. However, there is a roadblock that has been thrown up that makes that task a bit harder – the REF guidelines insist that the panel cannot make use of journal impact factors or any hierarchy of journals as part of the assessment process. It seems perplexing that any information should be ignored in this process, especially when it seems so pertinent. Here I will argue that journal quality is important and should be used, but only in combination with other relevant data. Since we teach our own students a particular method (courtesy of the Reverend Thomas Bayes) for making such decisions, why not practise what we preach?

A panel of experts, doling out the money. What could go wrong?