Open access roundup

Today I attended an interesting meeting, organised by the Repository Support Project, on implementing open access policies. I did some tweeting, so search for the hashtag, #RSPevent. But I also thought I would summarise some of the things I learned here as well.

There were some details for the RCUK policy review in 2014. The data collection for the review has been specified. I don't think this has been published yet, but the slides from the event will contain the details. Data collection will be up to July 2014, so presumably the review will take place in the second half of the year, and I expect it will be published right at the end of 2014.

In the margins of the meeting I heard about a fascinating project being run at the Open University. The Connecting Repositories (CORE) project is providing a single interface to a whole slew of open access repositories, and is providing semantic analysis services based on the full text content. Definitely worth a look if you haven't before.

There are also two JISC projects, RIOxx and V4OA which will be important parts of the evolving landscape for open access repositories.

I also learned some new (to me) information about the European scene. Most notable was the mandate of the European Research Council, which specifies a minimum six month embargo period for publications in all disciplines. The policy applies to monographs as well as journal articles and also includes research data. Finally, there is the OpenAire project, which is worth keeping an eye on.

The presentations from the meeting will be online in due course.

 

Posted in research policy | Tagged | Leave a comment

Supply and demand for innovation

To increase innovation-led growth, the UK needs increased policy focus on the demand-side. That is the conclusion of Richard Jones, PVC for Research and Innovation at the University of Sheffield:

Nor has [the last two decades'] policy framework been without positive results. Its successes include some substantial inward investment into the UK, which has led to a relatively high proportion of UK R&D being foreign-funded. The university science base is strong, well connected with industry, and there has been growing spin-out activity. But this has not been enough. Business-sponsored R&D continues its relative decline and university spin-outs seem unable to reach the scale required to make an impact on the wider economy.

It is time to admit that supply-side innovation policy is not enough, and that we need somehow to generate the demand for technological innovation that our current political economy doesn’t provide.

Read the rest here, or, for the more visually minded, there are four keys graphs.

Posted in research policy, Uncategorized | Tagged | Leave a comment

The end of journal impact factors?

Last week it was great to see the publication of the San Francisco Declaration on Research Assessment (DORA), which sets some important principles for the assessment of research. Advising on whether the Higher Education Funding Council for England should sign the declaration was one of my first tasks in my role there and I am really pleased that we are a founding signatory. Central to the declaration is the notion that journal impact factors should not be used as proxies for the assessment of the quality of research outputs. Quite right – this excellent post by Stephen Curry explains why.

In the major research assessment that we run, the Research Excellence Framework (REF), it is already clearly stated that journal impact factors will not be used (just search for ‘impact factor’ in the assessment panel working methods [pdf]). But clearly there is still a problem. The point is repeatedly made that universities are using journal impact factors in their internal decision-making and staff management processes relating to the REF.

Why is it that the use of impact factors persists in the face of clear policy guidance that they don’t form part of the assessment? Being able to answer this is central to tackling the problem. I can think of two possible explanations:

  • Despite the clarity of the panel working methods, there is a belief that panels will, in fact, use impact factors in their assessment of research outputs.
  • Within Universities, senior staff and managers feel that they lack the time or the expertise to come to a judgement on the quality of research outputs, so reach for the easy proxy of journal impact factor.

Maybe there are other explanations that I am missing. Either way I would be interested in hearing what people think. Why do journal impact factors continue to be used inappropriately? All thoughts in the comments thread would be most welcome.

Posted in research policy | Tagged , , | Leave a comment

Measured excellence in research

One of the comments on my piece last week on the Guardian science policy blog questioned my assertion that the UK research base is ‘in good shape’. The commenter used the relatively low UK spend on education as a justification for this. While I don’t think the education spend figure is especially relevant to research, it is the case that the UK public sector spend on research is low compared to some other nations, so why do I assert that the research base is in good shape?

I wouldn’t deny the importance of how much we spend on research; the whole thrust of my argument was that we should spend more. But I think in measuring performance of the research base it is output measures that matter, not input measures like how much is spent. In particular, I would highlight 3 output measures that demonstrate the strength of the UK research base:

  • The UK performance on article citation scores, that demonstrate that overall the UK is ranked second in the world after the US on this measure. The detail is contained in these reports (both pdf files).
  • The results of the last Research Assessment Exercise in 2008 demonstrated that nearly 90% of UK research is of an international standard.
  • The UK is the most successful nation in attracting prestigious awards from the European Research Council (Figure 8 in the ERC annual report demonstrates this).

Of course all of these are lagging indicators, but I think they do make a persuasive case for a healthy research base.

The real question is how much better research performance would be if the resources available were increased. This is a tricky question to answer, and as far as I am aware the evidence is simply lacking. One for the comments or a future post, perhaps…

Posted in research policy | Tagged , | Leave a comment

Spin out numbers

Sir Mark Walport, the new Government Chief Scientific Adviser, made his inaugural speech last week at the Centre for Science and Policy annual conference. One of the things he mentioned, subsequently reported by Times Higher Education, was a belief that rather than measuring the number of university spin-out companies, “The metric surely has to be the number of successful spin out companies”.

Superficially this seems self-evident, but in my view it is wrong. Spin out companies are all about risky ventures to exploit new knowledge. It is to be expected that many of them will fail. However, if universities are only judged on successful spin outs it is likely to make them risk averse, not pursuing the very spin outs that might make the biggest returns, to the ultimate detriment of the university and the nation.

Metrics are tricky things. It’s not just about what is measured, but also understanding the behavioural response the metric creates.

Posted in research policy | Tagged , | Leave a comment

On (not) writing

It’s been quiet here for a while. It could be that I have been busy. I am about to transition into a new work role, so the last few weeks I have been finishing things off, and now I will have lots of new things to learn.

But I don’t think the lack of content is related to that. I have started quite a few draft posts over the last few weeks that have not yet seen the light of day. No, I think the problem is that I worry too much about things being perfect, my version of what Steven Pressfield calls, in his book ‘The War of Art‘, the Resistance (with a deliberate capital ‘R’).

So, I am going to make some changes. I will aim for more short posts, and not worry whether everything is perfect and all my arguments watertight. And I will write on a wider variety of topics, not just research policy.

If you are interested to see how it goes, please subscribe, and I am always interesting in hearing your feedback.

Posted in writing | Leave a comment

The evidence divide

Earlier this month I attended a conference organised by the STEPS centre at the University of Sussex. The topic of the conference was the use of evidence, especially scientific evidence, to inform the development and implementation of policy. There was a great line-up of speakers, and the organisers should be congratulated for making enough space in the programme for discussion and debate.

A huge array of issues were explored at the conference, but one feature in particular stood out for me: the presence of a real divide between two markedly different ways of thinking about the relationship between scientific evidence and policy-making.

One the one hand there is the social science view. Based on empirical evidence and thoughtful consideration of current and past events, this view presents a nuanced consideration of how science should interact with policy. It values scientific evidence, but also emphasises the importance of other forms of knowledge and evidence, like the knowledge of practitioners working on the ground, who have real insight to bring to tackling complex problems. This view values considering a diversity of possible policy solutions, and considers evidence to be very much about presenting options within which rightly political decisions can be made. It values dialogue as a means to ensure that a wide range of views are included in the framing of problems and the evidence required to solve them, and cautions against the deficit model.

This view sits in contrast to the alternative view espoused by some members of the scientific community. While this community was not well represented at the conference, there was a strong sense that there is a very different world view in play. In this world, scientific evidence is regarded as having a special primacy, even to the extent that there is a call for more ‘scientific’ approaches, like randomized control trials, to be used right across the policy domain. For this community the problem is very much about a ‘deficit’ either in the public or policy-makers or both. The solution is either to fill the gap with more or better information, or to allow scientific evidence to dictate decision-making in the policy sphere.

There is ample scope for speculation about the reasons for these different world views. But there is also a central practical question raised: how do we deal with the differences? If the social science view is accepted, we are making errors in the policy-making process, not because of some inadequacy of the evidence (although there are surely areas where the evidence is inadequate), but because of the approach that is being taken, and the narrowness of the evidence base that is being used.

For me the answer to this dilemma lies with a third group of people, the advisers who sit within policy-making organisations, translators who bridge between the science and policy worlds. These people are of central importance because they can ensure that the plurality of evidence sources are included, that policy problems are framed in a broad way, through the appropriate use of public dialogue, and that research tackles the questions that policy-making needs, rather than those that the scientific community wants to address.

As the pressure builds towards smaller and smaller Government there is a risk that these crucial posts will become increasingly rare. Superficially, the intermediary role can seem wasteful and unnecessary. But removing translators means that scientific advice could excessively dominate, and within that advice the framing of questions might be derived from too limited a set of perspectives. We need to protect the translators, if we want policy making to be the best it can be.

Posted in research policy | Tagged | 6 Comments