Three unconventional responses to the AI assessment challenge in higher education
Since late 2022, one thing has largely preoccupied higher education and dominated much of the discourse around it. This is the knotty problem that, if, through generative AI technologies, university students can either wholly or in part generate the assessment outputs required, what does this mean for universities and what can they do about it?
The logical option is to find a way to detect when it is used, or to validate when it has not been used, with this acting as a safeguard and deterrent. This is still considered by some to be a strong solution. Others clearly find the idea that you can use technology to combat technology both implausible and flawed and, regardless of where you stand on that, a genuine issue is the dynamism of this technology. We have yet to reach a culminating point in its development and, if framed as a battle, it is going to be an ongoing one for now.
Another option is a policy route, essentially changing policies to try to make clear when using generative AI is appropriate or not. The Russell Group principles on generative AI in education see this as a means to support students “in making informed decisions and to empower them to use these tools appropriately and acknowledge their use where necessary.” Many universities are using a RAG system to communicate to students when AI use is permissible, and the extent to which AI use is permissible.
Another option is a more structural type of change, through the redesign and more wholesale change of degree programmes. This is something covered in the “Enacting assessment reform in a time of artificial intelligence” document from Australia’s Tertiary Education Quality and Standards Agency (TEQSA). They call for the following:
“Rather than investing primarily in detection mechanisms, institutions need to emphasise the redesign of assessment to capture authentic demonstrations of student capability and comprehension.”
One key recommendation is programme-level assessment reform, which is interesting against a backdrop of microcredentials, the lifelong learning entitlement (LLE) and unbundled provision. Greater cohesion of assessment across a degree programme seems to be at odds with any moves to disaggregate this type of provision.
If you’ve been following these debates and developments closely, or in fact grappling with them within a university, you may have determined which of these options you consider to be most plausible, in terms of addressing challenges and in relation to which are most practically implementable within universities. Personally, no option that I’ve seen has especially grabbed me, but, like others, I find a technological detection solution to be unconvincing.
In fairness to many of those seeking a way through the challenges generative AI in particular presents to educating students, they are situated within the current constraints of higher education, whether that be driven by tradition, regulation, government or culture. To a certain extent, they are limited to more proximate solutions, which makes divergence more challenging to present and consider.
Given that, I thought I would present three more divergent ideas of possible changes in response to new realities presented by generative AI. These ideas are not perfect, they are not necessarily what I think should happen, and I’m certain some will find them utterly fanciful.
The main reason I wanted to present them is to attempt to provide some less conventional ways of looking at things and, to be candid, because I’m a bit bored by the sorts of ideas I’ve seen over the last few years. Here goes…
Two-tier degrees
The first idea is two-tiered degrees. Tier 2 is a route through a degree that enables free and unfettered use of AI for any contributions or outputs, either formative or summative. This route places no responsibility on a university to check or verify whether generative AI has been used, or to try to ascertain how it has been used.
Educational activities and assessments are not designed from the standpoint of defining when generative AI should and should not be used, either in terms of direct prohibitions or recommendations. If you are a student, you can freely use generative AI across the degree in a way that you determine, and no one will be considering whether or not you used it inappropriately.
However, those taking a Tier 2 route will have their degree certificate and award clearly labelled as an AI-track degree.
The alternative Tier 1 route would not be directly opposite to Tier 2, and would not completely prohibit generative AI. Instead, this route is one in which AI use is more prescribed, and the overall ethos of the route might be described as paying your dues and putting yourself through a process of learning to develop competence and mastery without being over-reliant on, or over-utilising, external aids.
This is the kind of “no pain, no gain” degree track where deep personal effort and engagement in the learning process is the expectation, framing and choice this option presents. Upon completion, you get a degree certificate and award that is clearly distinguishable from the Tier 2 AI route.
The reason I present this idea is the psychological dimension presented by this choice. My hunch is that what would happen in a higher education paradigm where this exists is that the AI route would be perceived as a lesser choice and a less prestigious degree. Not only by prospective students, but also by some prospective employers, who would be able to see this transparently on a degree certificate.
I think Tier 1 would carry more prestige and, given people would be making an upfront deliberate choice, it would be really interesting to observe what behaviour and attitudes that engenders in relation to the use of AI in particular, and more generally.
I have no doubt that lots of people will not like this idea. If you are in the camp that says we should embrace AI and make sure it permeates across higher education, you are really not going to like this. Fundamentally, this option is more sympathetic to those who see generative AI as being deleterious to learning and cognition, particularly for people who might be considered more on the novice end of the spectrum.
This option plays into human psychology around status and would therefore raise concerns that such an approach would reinforce existing social and economic hierarchies. Nevertheless, I think we do not consider psychology and persuasiveness enough when it comes to choices and behaviours we would like to encourage in students that we consider to be beneficial to them. This is ultimately the principle that lies behind this idea.
Experience-only degrees
The second idea is also pretty radical. This idea involves removing any formal assessment and grading from degrees, turning degrees into what you might call experience-only degrees. It is not your grades or your performance in assessments that are key signals, but the fact that you spent a dedicated period of time and were given access to a range of experiences over the course of a degree programme. There is no assessment that manifests itself in grades, but you will still be asked to undertake activities and produce outputs.
This idea presents a challenge to the role that grading plays in signalling people’s abilities relative to their performance throughout a degree. It essentially cuts short the days in which potential employers could filter out applicants on the basis of their degree classification. However, arguably, this would present no greater challenge to a recruiter than assessing work experience, where you are judged on experience and what you did during that period, rather than on a proxy metric. Students would need to articulate the things they did and experienced in their time at university rather than present their grades.
I am sure there are a range of potential issues with this idea. One would be in respect to motivation and the loss of the motivational factors that come with assessment and seeking to gain good grades. However, to a certain extent, this would take a lot of heat out of the debate around AI and assessment and grade integrity. When it comes to employment and the jobs market, people would have to be judged not through proxy metrics such as degree classifications and, although there might be some signalling relative to the university brand, a different type of assessment and judgement would need to be used by prospective employers.
This approach would potentially put a greater emphasis on the learning process and create conditions for educators to think about what would be good for students to do, experience and be exposed to, rather than have an education experience that is shaped by assessments. Whatever is included that is assessment-like would be able to be reframed, and the output would not come with the same pressure. Conditions could be more readily created for a work-in-progress mentality to prevail, and courses could be imbued with more of a metacognitive ethos.
From a learner perspective, you might be able to free ride through a degree, but the incentive to cheat and game it through, let’s say, injudicious use of generative AI for graded assessments would be lessened. Might this motivate learners to ensure they really can do and evidence the things the programmes aim to equip them in?
Collective visibility in assessment
One last idea, which is less all-encompassing and is one I have actually seen implemented in an online course outside of higher education, is for every student to view other students’ assignments. This is obviously a less radical suggestion and is not far removed from peer assessment or peer review.
For example, the idea would be that you submit your assessment, say an essay, and can read everyone else’s essays, comment on them, and like or upvote them.
The principle behind this is that the visibility of other students’ work intensifies comparison, which in turn may shape behaviour. It could do that in a number of ways. If, for example, generative AI is genuinely driving bland sameness, then this may encourage divergence to stand out, particularly if efforts at greater originality and distinctiveness are rewarded.
Equally, the upvoting element has the potential to reinforce this, as students may also identify what may be AI-driven formulaic work, and therefore this is not endorsed and gains a visibly lower status.
the opportunity to respond to it in some way. It potentially creates a kind of social regulation of more problematic or deleterious uses of AI, and peers, in some cases, may be able to better identify when someone clearly comes across as way smarter and more erudite than they actually are.
Again, this is an idea not without problems. False accusations or suspicions are clearly one potentially large problem. Similarly, AI may be used to generate divergent as well as convergent outputs just as well. Nevertheless, this approach may have a social norming effect that supports what is most valued in an output and the process undertaken to achieve it.
Thinking beyond the obvious
All I can say about these ideas is that they are unconventional ideas that I have not seen proposed anywhere else. They are offered in the spirit of divergent thinking. When challenges arrive, sometimes it is helpful to generate more radical solutions that are less fettered by day-to-day realities, on a journey towards coming up with plausible, realistic and desirable solutions and options.
Whether these options are desirable or possible in a highly regulated and constrained system is, of course, debatable. But, having read and seen a lot about AI and higher education assessment, they definitely include different ways of looking at things.
In the past few years, I’ve tended to see the same things come up, such as the need to move to viva-style or in-person assessment, or authentic assessment. This is currently pushing “AI literacy” close to being one of the most problematic terms in higher education teaching and learning at the moment, in my humble opinion.
My sense so far is that higher education has few genuine answers to the challenges that generative AI has presented. Unlike others, I don’t think universities should necessarily be berated for this. It is a knotty problem because we are talking about a higher education system response here, not simply changing how you teach and assess on a short course you have just slapped online.
I don’t think there are any easy answers, despite there being plenty of people out there who would like you to believe that there are. But hoping this challenge will go away, or pretending it does not exist, is not a winning strategy. These ideas are therefore attempts to move things on rather than present “answers”. Because if this challenge is going to be properly taken on, then I think it will require some creative experimentation.
