The only man who behaves sensibly is my tailor; he takes my measurements anew every time he sees me, while all the rest go on with their old measurements â€• George Bernard Shaw
It could be argued that as long as there has been education, there have been metrics and a desire to apply scientific methods to analyse, track and assess student performance and academic attainment and improve teaching practice. But just as quote above identifies, understanding what you measure, how you fit methods to the means and the desired outcomes that you are seeking, is also fundamentally important. The unveiling of the government’s green paper in November 2015 heralded the emergence of what is now widely becoming known as the Teaching Excellence Framework (TEF). This presents a new challenge to the UK higher education sector every bit as intriguing as the insightful quotes above.
It is clear from the emerging debate around TEF that the measurement and evaluation of teaching is provoking a wide variety of opinions about the rapidly changing nature of academia and the marketization of HE. Some view TEF as a balancing force (against it’s opposite, the Research Excellence Framework, REF), whilst others see it solely as a mechanism to separate research from teaching-focussed academics (thereby setting up a two-tier university sector of those institutions that can afford to support either teaching or research – but not both, apart from only the very few top-tier universities). Although the TEF may signify an opportunity for reframing and improving the definition of quality higher education, many view it also as giving preference over support for established research-based career paths (which quite rightly, cannot be dismissed overnight).
But beyond the hopes and fears about what the TEF may or may not actually mean, in reality it is also widely acknowledged that some form of teaching-based measure will soon inform the future of the quality of UK university education and the resulting impact upon funding. The concept of measuring teaching quality is not a new idea. The TEF is a recent waypoint on the road that is dotted with numerous acronyms and frameworks, all of them sharing in whole or part aspects of what the green paper touches upon: a desire to improve quality as a basis for continuous improvement (like the European Foundation for Quality Management, EFQM model for HE); a method to explore elements of teaching quality “at source” (Teaching Quality Assessments, TQA); enhancement and innovation of teaching (Centres for Excellence in Teaching and Learning, CETL); and the identification of achieved student learning outcomes (such as via the OECD’s current vehicle, Assessment of Higher Education Learning Outcomes, AHELO).
Beyond the hopes and fears about what the TEF may or may not actually mean, in reality it is also widely acknowledged that some form of teaching-based measure will soon inform the future of the quality of UK university education and the resulting impact upon funding
All of these methods and mechanisms share a common objective of learning more about the drivers that influence the quality of teaching. Added to this is a universe of datasets that are available to university planning departments including those via HEFCE and HESA (including the Keep it Simple, KIS, Destination of Leavers from Higher Education, DLHE and UNISTATS datasets); student experience data (such as the NSS); as well as university-derived data on market and peer/competitor analysis, student admissions, enrolments, attrition/retention, employability, progression and graduation (proportion of good degrees attained).
All of these datasets could make up TEF input and output measures. However in order to assess excellence in teaching, additional metrics to capture (higher) intellectual skills and the added value of non-academic opportunities and experiences such as placements and engagement with employability activities may be needed. Further, if teaching excellence is to be a recognised metric which UK universities adhere to, the wider HE supply chain of stakeholders need to be convinced that such metrics truly reflect excellence in teaching (and not be conflating quality of teaching with the quality of students). Data on teaching and learning can be voluminous, sourced from different points in the student / teaching lifecycle and be open to interpretation depending on what perspective you wish to take.
Whatever the basket of metrics that might be chosen for the TEF, universities will need to become increasingly well prepared in terms of the capacity and capability to source, collect, store large inter-related, datasets. This is because teaching, learning and student experience involve a range of factors both pre- and post- study, neither of which have supremacy over the other but all of which are inter-related in some manner. UK universities should now be seeking to develop so-called “big data” competency, so that predictive analytical decisions about improving the quality of teaching and learning experience can be made. Universities should be well placed to do so – given that they are themselves, learning organisations (or should be). But as the parallel growth of big data, predictive analytics and data science shows universities will need a combination of reliable and data-rich source systems, and sufficient human skills and knowledge to make sense of inter-relating datasets. For example, university planning departments may need to remodel themselves to become HE data scientists; academics themselves may wish to seek a change in career path, providing their own analytical skills to further their institutions future success.
UK universities should now be seeking to develop so-called “big data” competency, so that predictive analytical decisions about improving the quality of teaching and learning experience can be made
The TEF – or something like it – will no doubt present itself in the coming years, of that there is little doubt. But by considering and implementing a big data strategy now, UK universities will be better prepared to link together diverse and large sources of information through analytical models which will allow them to make sense of teaching excellence data – and fit these data to the student experience and assessment of teaching quality before it is required. In terms of George Bernard Shaw’s quote, we must be ready and flexible to behave sensibly and take relevant and new measurements as needs dictate. Else the only clothes we will be left with, will be the Emperor’s new ones…
Professor Amir Sharif, Director of Teaching and Learning, Brunel Business School, Brunel University London, www.brunel.ac.uk
The views and opinions expressed in this article are solely those of the author and do not reflect or represent those of Brunel University London.