Conversations around the purpose, function and future of assessment have picked up recently – and, with regards to examinations, talk is being followed by action. Where candidates previously gathered in large halls, we are increasingly seeing assessment using computers.
Through this transition, the role of the invigilator (also known as the proctor) remains important, and I see great potential for technology to assist the process. Yet some of the systems that are being proposed appear to weaken and contradict the role of the invigilator.
These require careful scrutiny. I worry they could have a negative impact on both the purpose and effect of examinations.
The invigilator’s role
When considering ‘e-proctoring’, it’s important that we don’t start with the tech. Instead, it’s about understanding the role and asking how technology can help – even improve – invigilation.
Proctoring is, essentially, about helping candidates complete exams in as supportive an environment as possible. Invigilators do this through time-keeping (providing warnings when there is one hour or ten minutes to go), by investigating candidates’ questions (if the examination paper appears to have an error or is unclear), and by managing or recording unexpected situations.
In a physical exam hall, this might be a fire alarm or water spill. In an online context, perhaps it’s a loss of internet connectivity.
Then there’s the detection of improper conduct. Invigilators confirm that those sitting the exam are who they claim to be, ensure that candidates aren’t using unauthorised materials, and prevent or detect candidates’ prohibited attempts to communicate with others.
Most importantly, invigilators should avoid actions that students perceive as intrusive, disturbing, stressful, or otherwise harming to their performance within the exam rules.
Face-to-face versus online proctoring
During a traditional exam, invigilation consists of a combination of continuous observation from a distance and occasional close-focus inspection.
To reduce stress, the candidate is aware when the latter is taking place, as a candidate who feels under continuous close-focus human surveillance or recording is unlikely to perform their best.
For this reason, digital invigilation shouldn’t simply be a continuous video-conferencing link that effectively seats the invigilator on the candidate’s desk.
That approach fails everyone: the invigilator has to do at least as much work, the candidate is placed in a more stressful environment, and the technology is badly under-utilised.
Digital invigilation systems are most helpful for continuous ‘distance’ monitoring, raising an alert if they detect behaviour they do not understand or find suspicious, to be checked by a human
Using tech to reduce stress
Technology should contribute to the invigilation process, not merely allow it to be conducted remotely. It should become a part of the invigilator’s alerting and record-keeping process. It should reduce stress for both candidate and invigilator – and work more effectively than a physical human presence.
For example, an e-proctoring system might take a snapshot of the candidate’s work at the point of an alert, break, or interruption.
This allows the work to be checked for sudden bursts of ‘creativity’ or correction afterwards, that might suggest a candidate had been consulting unauthorised materials or another person.
An e-proctoring system might also detect unexpected sounds – such as changes in typing cadence (suggesting that the individual is no longer the intended candidate), the turning of pages or unusual patterns of system or network activity (that may suggest the candidate is engaged in unauthorised activity), or environmental changes (such as loss of network connectivity).
The system could record and alert the human invigilator to these.
Overall, digital invigilation systems are most helpful for continuous ‘distance’ monitoring, raising an alert if they detect behaviour they do not understand or find suspicious, to be checked by a human.
‘E-proctoring’ systems that do no more than reproduce, badly, the face-to-face invigilation process should be regarded with suspicion.
Proctoring must be part of an assessment system that is designed to be resistant to cheating – one that considers alternative ways of assessing or verifying performance, and designs assessment so that it is hard for candidates to benefit from collusion
The bigger picture
Invigilation is not the only thing that prevents candidates cheating. Therefore, proctoring must be part of an assessment system that is designed to be resistant to cheating – one that considers alternative ways of assessing or verifying performance, and designs assessment so that it is hard for candidates to benefit from collusion, such as ‘open book’ exams.
Moderation after assessment is crucial too, identifying candidates whose exam performance differs significantly from their expected grades, and reviewing invigilators’ records for anything that might affect performance (either negatively or positively).
Personally, as a lifelong student, I’ve gone through teaching, learning, assessment and exams both online and in traditional physical settings. I know these environments demand different ways of working.
As our education systems respond to the rapid move to remote working demanded by the Covid-19 pandemic, e-proctoring ought to be part of the conversation.
Approached with care, caution and vision, there’s much to be gained with technology – both for today’s candidates and for those of the future.
Andrew Cormack is chief regulatory adviser at Jisc