The 2021 Research Excellence Framework (REF) has mostly likely overlooked the public impact of research and academics because there is limited recognition of the importance of social media.
That is the conclusion of a new research paper written by Dr Katy Jordan from the Faculty of Education at the University of Cambridge.
Dr Jordan says that reform of impact assessments and metrics is needed to ensure future REFs keep pace with the increasingly diffuse ways academics influence public understanding, debates and policy.
There is needed more “open-mindedness about what researchers achieve,” said Jordan. “In an increasingly complex, socially-networked culture, this would help to ensure that the broader effects of their work are not forgotten.”
The study analysed responses from more than 200 examples of how academics engage people outside their university in their scholarship.
REF 2021 was published on 12 May and will help inform how the UK’s four national funding bodies direct billions of pounds of quality-related research funding.
The official language presents impact as a top-down, outward flow from universities to a waiting public, but this is an outdated characterisation – if it was ever valid at all
– Dr Katy Jordan
For the assessment, universities submitted examples of their ‘impact’, including examples of social, cultural, economic and policy-based results.
Jordan supports the REF measures but argues it misses the true scope of research as social media blurs the distinction between ‘impact’ and public engagement’.
The REF measures “final outputs”, those realised on completion of research – but the survey suggests academics are engaged in ongoing “feedback loops” through the duration of a project, which can increase impact, awareness and the quality of research. The research paper also says the current measures are too vague and vary widely between subjects.
Jordan also highlights “somewhat confusing” REF official guidance on engagement: “Engaging the public with research does not count as impact. Impact is what happens when people interact with the research, take it up, react, or respond to it. Public engagement doesn’t just happen when the research is complete.”
Said Jordan: “The official language presents impact as a top-down, outward flow from universities to a waiting public, but this is an outdated characterisation – if it was ever valid at all. Ask researchers about their most impactful interactions on social media, and you’ll get a much wider range of examples than the REF covers.
“You could argue that this means too many researchers are misunderstanding what impact is, but it’s also potentially evidence that times have changed. There’s a huge amount to be said for asking universities to demonstrate their value to wider society, but it may be time to rethink how we measure this.”
Less than half of the examples analysed fitted current REF measurements of public impact. Academics typically use social platforms to share their views, stimulate discussion, and engage with people outside their networks. The paper includes anecdotal examples of academics finding engagement via social media with new communities, organisations and public bodies. They report social media helped generate invitations to panels, public lectures and advise organisations.
The survey respondents also used social media to test research ideas, report interim findings, crowdsource information and data, or advertise for research participants.
“One solution may be to amend the assessment so that it asks universities not just to provide evidence of research outcomes, but to explain the research process across a project’s lifetime,” Jordan said. “This isn’t a call for yet more ambiguity about what impact is, but for more open-mindedness about what researchers achieve. In an increasingly complex, socially-networked culture, this would help to ensure that the broader effects of their work are not forgotten.”
The research is published in Learning, media and technology.
A review of REF is underway. Sir Peter Gluckman, president-elect of the International Science Council and former chief science adviser to New Zealand’s prime minister, will advise the four UK higher education funding bodies on future research assessment.