Is data narrowing how universities think about course portfolios?
For any university looking to develop or revitalise an online portfolio of courses, one of the key considerations is selecting courses to launch or update. For universities with broad subject coverage and an array of academic schools and departments, the options can be considerable.
In general, and particularly in these financially trying times, universities need to identify courses that will generate sufficient enrolments over time to be financially viable and sustainable. This need is becoming more acute, and it has resulted in many closing courses, and in some cases whole departments, as well as launching courses in different areas.
The shorthand for this is “portfolio rationalisation”, which in practice usually means closing courses that are not financially sustainable and launching courses in areas where there are higher volumes of students, such as degrees in artificial intelligence, for starters.
I would argue that, in general, online degrees have faced far more scrutiny around their financial viability from the outset and have therefore had to justify themselves far more robustly than on-campus courses ever have, but the financial climate might be gradually levelling things up somewhat.
Whether online or on campus, the need for financially sustainable courses is now even more important, which ultimately means more universities becoming more robust when it comes to selecting courses and requiring stronger justifications.
On the surface, this sounds good, particularly given that universities can have a bit of a reputation for being, shall we say, admirably eclectic in their course selections. Anyone who has sat on programme approval committees will have witnessed some proposals that are the course equivalent of the DriveSafe Glove, an infamous Dragon’s Den pitch for a single glove that you put on your right hand when you travel abroad to remind you to drive on the correct side of the road.
However, becoming more robust in how we identify and select courses is not a neutral act. Robustness is not a simple matter of having more evidence, more process or more scrutiny. It depends on what we choose to value, what kinds of evidence we accept, and what assumptions sit behind our decision-making. A course selection process can look rigorous while still being narrow or overly shaped by the measures that are easiest to quantify. Having more acceptable forms of justification for courses is not the same as making wiser decisions.
My worry, both as a whole and in relation to online education specifically, is that we have become too narrow in our thinking and in the methodologies we use to identify and justify courses, and we are seeing, and will continue to see, unintended and potentially harmful consequences of that.
The pull of portfolio convergence
The current doctrine of being robust in course identification and selection is through being “data-driven”. Sounds good, doesn’t it? And it’s almost certainly better than being “vibes-based”. But being data-driven can often mean something quite specific and narrow. Typically, it is a process that looks almost exclusively at quantitative data, or to put it more simply, numbers. This includes enrolment data tied to particular subject areas, labour market information and digital quantitative data points that we take as signals of interest or demand. None of these things are bad in and of themselves.
Is it useful to understand how many people have chosen to study courses in specific subject areas historically? Is it helpful to know how many people are searching for courses via specific digital channels? Is it useful to see how the job market related to specific course disciplines is changing? The answer to all these questions is yes. These things are useful.
The problem with this highly quantitative data approach is that it can easily become a narrow pursuit of the largest numbers. The related issue is that, as something of a herd species, when this so-called data-driven approach becomes orthodoxy and is cross-fertilised, everyone begins to do the same thing. The impact of this is a high level of convergence.
And let’s be honest, there is no shortage of number-jockey service companies out there that would happily support you in this type of endeavour. Supported by data scientists fusing together data points into pretty charts that, although no one can validate where any of the data came from, do at least use a colour palette a power Canva user would be proud of. Then there are the aggregator platforms, which invite you to believe that traffic from a few websites is not a very partial record of online browsing behaviour, but evidence of what students really want. Deep. Deep. Down. With filters and export functionality.
A high level of convergence is precisely what has happened in the online degree landscape in UK higher education. It would not be difficult for me to list the most common online degrees on the market, and that’s not due to some extraordinary feat of memory, but simply the recurrence of the same titles over many years. When a university enters the online education market, either directly or in partnership with an online programme management (OPM) company, I can usually make a pretty accurate guess about what the portfolio is going to include. Rarely are there any surprises.
This is not wholly wrong. Clearly, there is logic to selecting courses in areas where there are already lots of people choosing to study the same or similar courses. But when the way you approach identifying and justifying courses only ever leads you to the same place, and increasingly the exact same place as everyone else, you’re likely to have a problem, and one that will most probably compound.
Change is difficult because this approach is one of the easiest to justify, and in a financially constrained environment the requirement for sufficient justification grows. If I can show you big numbers across a range of data points that are either growing or have remained stable, then there is a good chance that quantification bias will kick in, and this will be easier to justify, easier to make the case for and easier to use to minimise blame if things don’t go to plan.
This approach can also be framed as being demand-driven, which, when combined with data-driven, creates the kind of proposal that is not so much to be discussed as rapidly approved, with the sort of evidential confidence that accompanies Hercule Poirot gathering everyone in the drawing room.
In a recent talk, I posed the question: what’s worse, developing a course in an area with weaker orthodox signals of potential that fails, or developing a course with all the right numbers behind it that leads you into a saturated market and is outperformed by the competition?
I think the answer is the latter, because it’s far easier to justify and there’s less chance of negative consequences for the proposer, even if the outcome is the same.
Don’t mishear me here. I’m not suggesting we shouldn’t obtain quantitative data to support the assessment and judgements we make around course selection. But we have to properly understand the limitations of this data, and how our human biases and nature can lead us to think too much of it, and to norm it as a singular robust method because it’s numerical and comes with some nice charts.
What the numbers leave out
Quantitative data, particularly when aggregated, has limitations and has a lot of nuance subtracted from it. When considered in relation to portfolio decisions, it tends to squeeze out consideration of other factors that lie behind things like enrolments. Enrolment success can easily become viewed purely as a consequence of the existence of a programme on the market and nothing else.
We fail to ask and investigate the reasons behind the enrolment performance of such programmes. How does pricing contribute to this? What role have marketing and recruitment played? What about course positioning? Or any number of other factors? We can easily stop being inquisitive. We are tempted to think that if others like us have done it, then we can replicate it by simply standing up a similar programme, and in some cases that works, until it doesn’t.
I have known of online degrees that have achieved significant enrolment success due to specific factors that are not more widely known and that can’t be evidenced on a spreadsheet. But if, in these instances, you were to approach things purely quantitatively, then you might be liable to be overconfident in numerical data as validation for creating a similar programme.
Digital, quantitative, aggregated data does not tell the whole story and can easily and comfortingly deceive. Its limitations are also related to how up-to-date it is and, in higher education, there is a significant time lag in, for example, the enrolment data from the Higher Education Statistics Agency (HESA).
There is also the challenge of how a narrow quantitative data approach can stifle any consideration of doing anything new or different. A case in point is provision under the banner of the Lifelong Learning Entitlement (LLE). If you read the UK higher education media, you will see, in almost every article about the LLE, doubt being cast on the demand for it.
What’s really meant here is that there is no quantifiable, comforting data that supports the case that lots of people want this. But why would there be? It’s completely new. That way of looking at things also fails to factor in how marketing, advertising, go-to-market strategies and other things can drive and influence demand.
Ultimately, resting too heavily on data-driven orthodoxy means you run the risk of constraining yourself from doing anything new or trying anything different. But I fear that, increasingly, this is the place the sector is inhabiting, and there is too little balance between quantitative and qualitative data, and limited appetite for discovery research, market testing and other thoughtful and creative ways of identifying and selecting courses.
In online education, narrow methodologies have brought about increased competition that has challenged universities and has at times been self-defeating. I fear it has also limited the breadth of areas where online education can be deployed to serve students. Not all of this is bad, not all of it should be curtailed, and not all of it is useless, but it is limited, leading and can be deceiving.
If we’re not conscious of that and don’t take care to adopt more well-rounded approaches to course identification, selection and justification, we’ll be headed in the direction of highly comforting and reassuring convergence, greater competition and ultimately sameness, while at the same time eradicating what only needs to be a small element of activity focused on pursuing opportunities, and new and different things.
