A personal history: 19. Some reflections on research into the costs and benefits of online learning

🔊 Listen to Post

I am writing an autobiography, mainly for my family, but it does cover some key moments in the development of open and online learning. I thought I would share these as there seems to be a growing interest in the history of educational technology.

Note that these posts are NOT meant to be deeply researched historical accounts, but how I saw and encountered developments in my personal life. If you were around at the time of these developments and would like to offer comments or a different view, please use the comment box at the end of each post. (There is already a conversation track on my LinkedIn site and on X). A full list of the posts to date will be found toward the end of this post.

This post is really in two parts. The first is about the research, mainly into the costs and benefits of online learning, conducted by the Distance Education and Technology unit at UBC. The second part is a reflection on what was learned about the costs and benefits as a result of this research.

Research in online and distance education at UBC, 1996-2003

When I was still at the Open Learning Agency, I was invited to participate in a proposal for the establishment of a Canadian Telelearning Network Centre of Excellence (TL-NCE) ‘to spearhead research in online learning applications, train students, transfer knowledge and technology, and inform practice and policy development in schools, post-secondary institutions, and workplaces.’ 

In particular, I submitted a proposal to develop a methodology to analyze the costs and benefits of ‘telelearning projects’ and was awarded a grant of $185,000 just before I left the OLA.

The total funding for the Network in 1995 was $12 million, spread over 240 public- and private-sector organizations including over 130 faculty from 30 Canadian universities. The network co-leaders were Linda Harasim and Tom Calvert and the CEO was Joanne Curry, all of Simon Fraser University. (For more on the NCE-Telelearning program, see the next post.)

I had been interested for some time in the relative costs of campus-based teaching and distance education. I had not specialised in this area as much as some of the other leaders in this field, particularly Greville Rumble at the U.K. Open University, Thomas Hülsmann at the University of Oldenburg, Germany, or Frank Jewett, at California State University, but I was aware of the literature, and as a manager of online learning programs, I wanted to understand how to accurately cost programs. I also believed that you had to look at benefits, not just costs.

My grant was in two stages, and the first stage was for $185,000 to apply a cost-benefit model to six case studies included in the NCE-Telelarning program. Part of the grant was used to hire a research fellow, Silvia Bartolic, who collected and analysed the data for most of the case studies. (Silvia went on to become Professor of Sociology at UBC). One of the UBC post-graduate certificate programs on technology-based distributed learning was one of the case studies.

Silvia and I used the standard costing methods of Rumble, Hülsmann and Jewett (they were very similar) to analyse costs, and parts of my ACTIONS model for media selection (Access, Teaching, Interaction, Novelty and Speed) to measure benefits.

In terms of costs, we looked at three variables:

• capital and recurrent costs
• production and delivery costs
• fixed and variable costs

We also looked at three types of benefits:

• performance-driven: for example, better learning outcomes
• value-driven: for example, increased student access to education.
• societal: for example, reduced traffic on campus; I provided input from the study to UBC’s 2005 Strategic Transportation Plan, which aimed to reduce car traffic to campus.

I followed up the NCE-Telelearning grant with a further grant of $220,000 from the Canadian Federal Government’s Office of Learning Technology (OLT) for a three-year study of the impact of new technologies on adult learners. I used this to hire another research fellow, Adnan Qayyum (who is now Professor of Education Innovation in the Distance Education program at Athabasca University.)

The NCE-Telearning grant was awarded when I was at OLA, but the OLT grant was awarded when I was at UBC. I had already hired Adnan when I got a memo from UBC’s Research Office. They wanted 20% of the grant as university overheads.

I was furious. The OLT program did not contribute to university overheads. Almost all the grant was needed to cover Adnan’s salary and the cost of collecting data from the sample projects. Continuing Studies covered the costs of DET’s overheads, and DET administered the grant in terms of accounting and travel costs. As far as I was concerned there were no overheads such as laboratories or equipment for UBC to cover.

I marched into UBC’s research office with Adnan. There was a heated exchange. I was adamant, and challenged the Research Officer:

‘The money has been paid to DET, and is in Continuing Studies’ accounts. I’m not handing over anything for UBC overheads.’

The Director of the Research Office said he would escalate the dispute to the next level of management, which presumably meant communication between the AVP Research and AVP Continuing Studies (Walter Uegama). I heard nothing back from either. I (and Walter) had protected my research budget but I had made another enemy.

By 2002, DET had built a substantial research program in learning technologies. In 2001-2002 alone we obtained $577,000 worth of research grants, mainly from the Office of Learning Technologies, but also from the private Donner Foundation. This enabled the establishment of the MAPLE Research Centre (Managing and Planning Learning Environments) which allowed me to hire researchers such as Tatiana Bourlova, Leah Macfadyen – who eventually went on to be Director of the MET program -and Josefina Rosado. Beth Hawkes and Mark Bullen played a major role in managing this research program.

Reflections on costing learning technologies

My experience both in researching the costs of online learning and in developing the business plan for the MET program resulted in my questioning the general principles of university budgeting, which is primarily expenditure-driven and historical. In other words in preparing annual budgets the base is the previous budget. Thus the budget is generally the same year-to-year for most departments, apart from across the board variations to take account of inflation or financial difficulties. If in one year cuts have to be made, they tend to be ‘shared’ across all the departments.

This really hampers innovation, which in general needs start-up costs and may require several years before sufficient income is generated to cover the innovation. Wise Provosts usually hold back relatively small sums in a general purpose fund to give themselves some flexibility. Innovation then tends to be driven by ‘Lone Rangers’, working on their own, or by relatively small innovation grants from the Provost’s Office. This limits scaling up or the wider adoption of innovative teaching. but the holding back of substantial funds for innovation, as the BC government did in 1995-1996, is very rare.

In British Columbia, the Ministry of Advanced Education withheld 2% of each institution’s budget for the 1995-1996 financial year, and 1% the following year. To get this money, the institutions had to put forward proposals on how they would spend this money on innovation in using technology for teaching and learning. In a number of institutions, individual faculty or departments had applied for funding, but in UBC’s case, they put in a single institutional bid and were successful. This resulted in an innovation grant of over $3 million for UBC. This had a huge impact (quite apart from helping me get my job at UBC). It helped eventually get WebCT off the ground and laid the basis for the current Centre for Teaching, Learning, and Technology, as well as a university-wide strategy for learning technologies in the late 1990s called ACCULT.

What we did to get the MET program off the ground was to use activity-based costing, tying the costs of the program to the actual revenues generated through tuition fees. The business plan calculated the break-even point for each course in terms of student numbers (in MET’s case, 42), using the same tuition fee as for on-campus students. Below this number the course would lose money but above this number it would not only cover costs but would result in a small profit per extra student beyond 42.

The key difference is that with activity-based budgeting, revenues do not go directly into general revenues and then are generally re-allocated across the university, but go specifically to running the program, with general overheads deducted. One advantage of this model is that general overheads are clearly identified and need to be justified and this puts pressure on administrative and support units to be more effective and keep down their costs. It also allows for new programming without raiding existing programs. However, it works best for new programs, and does not allow for cross-subsidies for programs that would not otherwise be self-financing.

Also, costs are not fixed like gold. Costs can be managed to a certain extent to take into account likely revenues. Less time can be allocated to course production, or adjuncts instead of full professors may be hired, for instance, but this will impact on quality, which is why benefits (or limitations) also need to be measured.

There is also a tendency to compare the costs of online learning with the costs of face-to-face classes. This is, to put it mildly, far too simplistic. In most cases they are not substitutes for one another, but complementary. They provide different benefits for different kinds of students. Thus any analysis of costs should also look at potential benefits (and drawbacks) at the same time.

This is rarely done though in the literature. In the 1990s, I used the ACTIONS model (later amended to SECTIONS) to identify benefits, but this is too narrow a framework. Environmental benefits (and costs) of online learning for instance are substantial, and we learned during Covid the value of online learning when campuses are closed for health reasons. We have also learned that excessive amounts of online learning can also be harmful for mental health, particularly for novice students.

Lastly, it is important to remember that variations within a condition are often more important than variations between conditions. Online learning can be done well, or it can be done badly; the same can be said of face-to-face teaching. Simple cost comparisons then between online learning and face-to-face teaching are not helpful. We need to look at the conditions in which each is used and that means taking account of the full context of teaching and learning. Costs are important, but tend to be badly tracked, and they must be viewed in context, and in particular costs need to be considered alongside the benefits (and limitations) of technology-based teaching. 

References

The NCE-Telelearning research on costs/benefits is summarized in the following published paper:

Bartolic, S. and Bates, A.W. (1999) ‘Investing in Online Learning: Potential Benefits and Limitations’ Canadian Journal of Communication Vol. 24, pp. 349-366.

Full details of the MET costing model can be found here:

Bates, A. and Sangrà, A. (2011) Managing Technology in Higher Education San Francisco: Jossey-Bass/John Wiley, pp. 162-179