Towards a theory or model of productivity for online learning: outcomes, scale and design

© BMW, 2013

In the HEQCO report and in his blog posts, Tom Carey raised the important issue of what part of online teaching can be scaled, and what cannot.

Outcomes and scale

I want to address this question in terms of what outcomes we are trying to achieve in education, because scaling must relate to what we are trying to do. For instance, it is quite possible to scale content delivery – this has been done since the early days of print and more recently radio and television. As Internet and mobile access becomes almost universal, the Internet can also deliver content at scale – this is what MOOCs do so well.

However, most educators at least would agree that education is more than just the delivery of information. There are lots of different ways to define learning outcomes, and at differing levels of detail, but I’ll start by quoting the following from Max Blouw, the President of Wilfred Laurier University, and Chair of the Council of Ontario Universities, in today’s Globe and Mail

Universities are primarily in the business of positive human development. They focus on enhancing the abilities of our graduates to communicate clearly and effectively, to analyze, to confront ambiguity with clear methods and confidence, to break down problems into manageable parts, to think critically and to question deeply.

Blouw’s main point is that a university’s job is not to provide employers with perfectly trained personnel who can ‘do the job’ on day 1. Employers have the responsibility to train graduates in the company’s specific needs once they start working. Because the economy, and hence jobs and work, change so rapidly, society needs people with flexibility, who can improve on existing enterprises, help solve or alleviate health or societal problems, create new kinds of enterprises, and who can continue to develop and grow throughout their life. Some of these more generic skills described by Blouw can be broken down a little into more specific ’21st century’ skills, such as good communication (using a variety of media); independent learning, adaptability, critical and/or original thinking, problem-solving, knowledge management (the ability to find, evaluate and apply information appropriately), and research skills.

I think the term ‘development’ is important here. It is not a question of filling heads with information, or even developing repeatable skills, but providing students with the ability and skills to continue to learn and develop at a high level of intellectual and social competence as they acquire experience in life outside the university.

For the sake of argument (and because I largely agree with Blouw), I will take these goals or outcomes for a university education as a rough starting point for the discussion. This leads to three questions:

• what teaching methods and/or learning environments will best achieve these outcomes? (And how do we measure/assess this?)
• can online learning provide an environment that enables such goals and teaching methods to be achieved?
• if so, can online learning do some or all these things at scale? If so, what can be scaled, and what can’t or shouldn’t be?

I would argue that we cannot answer the last question (which is the one Tom Carey so rightly posed) without first answering (and agreeing on the answers to) the first two questions.

What teaching methods and/or learning environments will best achieve the outcomes desired from a higher education?

Again, one could write a book on this (and many have). However, Christensen-Hughes and Mighty ‘s book, ‘Taking Stock‘, is a good place to start. This is a review of the research on teaching and learning in higher education. In Chapter 13, Chris Knapper (Queen’s University) states (p. 229):

‘There is an impressive body of evidence on how teaching methods and curriculum design affect deep, autonomous, and reflective learning. Yet, most faculty are largely ignorant of this scholarship, and instructional practices and curriculum planning are dominated by tradition rather than research evidence [nor – my comment – by educational theory]. As a result, teaching remains largely didactic, assessment of student work is trivial, and curricula are more likely to emphasise content coverage than acquisition of lifelong learning skills.

Note: Knapper here is referring to campus-based teaching. Here we see a clear gap between the stated goals, and the methods needed to achieve such goals. Thus, merely moving classroom methods of lectures and multiple-choice questions online will not achieve the desired learning outcomes, even if economies of scale can be achieved through online learning (as in MOOCs). Indeed, I have been shocked when discussing with instructors from elite universities designing MOOCs that there is an open acknowledgement that this is exactly what they are doing – moving ineffective classroom lecturing on to the Internet. As one put it to me: ‘It’s no good you criticizing the recording of lectures – lecturing is exactly what we do on campus, and often less well than on lecture capture, where we at least make an effort to be clear and comprehensible.’

Again, I could (and will) write a whole book on what methods are more likely to lead to the kind of outcomes Blouw claims for universities. Indeed, any discussion of scale and improvements in productivity will need to explore such methods in detail.

However, the reason I have quoted Knapper is that many in elite universities do not believe that online learning can produce high quality learners. For instance, at the MIT LINC conference in June, Sanjay Sarma, Director of MIT’s Office of Digital Learning, made the distinction between MOOCs as open courses available to anyone, reflecting the highest level of knowledge in particular subject areas, and the ‘magic’ of the on-campus experience, which is distinctly different from the online experience. He argued that it is difficult to define or pin down the magic that takes place on-campus, but referred to ‘in-the-corridor’ conversations between faculty and staff, hands-on engineering with other students outside of lectures and scheduled labs, and the informal learning that takes place between students in close proximity to one another.

Once again, though, we have this distinction between what happens in elite universities at graduate level, and what 80% of all students – undergraduates – get even or especially at elite universities. The challenge then for online learning advocates is to show that these high level learning outcomes are possible online, at least as cost-effectively as classroom teaching. I believe that not only can we show this – but many online programs have already done this, particularly by focusing on learner engagement, high-level online discussion, and strong instructor support. Thus any discussion of scale needs to focus on those online teaching methods that lead to desired outcomes – not comparing MOOCs with classroom lectures.

How can online learning achieve the desired learning outcomes?

The short answer to this question is: by using best curriculum design practices based on research into how students learn, adapted to the specific affordances of the Internet. (For more details on best online design practices, see my series: Nine steps to quality online learning and Designing online learning for the 21st century).

Course design by its nature can focus on the development of the higher level learning outcomes described earlier. For instance, a course could be designed around the goals of knowledge management and the development of independent learning skills. It should be noted that course design itself can be scaled up, in two ways. First, the basic principles of course design, such as designing for student engagement, can be scaled across many courses. Second, once a specific course is designed, this design will be available to all students who take the course or module. Thus ‘core’ design principles can by scaled both vertically (across all online courses) and horizontally (across all students), thus making course design an incredibly powerful productivity tool.

This provides a means to look at which of these best design practices (a) develop fully the affordances of the Internet and (b) offer the opportunity for scaling up activities.

Designing to the affordances of the Internet

As well as its ability to make the same content available to an unlimited number of students ( a broadcast affordance), the Internet has strong communicative aspects, allowing for full two-way communication not only between an instructor and individual students, but also between students and other students. More importantly though than the quantity of communication is the quality. What can instructors – or other students – bring to the conversation that results in deeper learning or different outcomes? To what extent can this added factor be scaled up?

Scaling up online activities

While not discussed in the HEQCO report, we know a lot about scaling activities in distance education from earlier work by Greville Rumble on Open University courses. Although the technologies were different (print, television and video-cassettes), many of the costing principles apply to Internet-based or online teaching (although others differ, as we will see).

One important concept is the difference between fixed costs and variable costs. In this context, fixed costs remain the same for a particular activity, irrespective of the number of students, whereas variable costs for an activity increase as the number of students increases. Thus content production and transmission is a fixed cost. MOOCs are good example. Once the lectures are developed and recorded, they can be accessed by an unlimited number of students.

However, essay-type marking/assessment is still today mainly a variable cost. The more students there are, the more time it takes to mark the essays, so marking costs are variable. It can therefore be seen that one possible productivity goal might be to look at ways to convert variable costs to fixed costs. For instance, a multiple-choice test can be automatically marked, so once the question wording and alternative answers are designed, it can be applied to multiple users. However, you have to be careful. Will changing the nature of the assessment change the likely learning outcomes? Or can the desired learning outcomes be assessed using the new assessment method?

Summary

This post is already too long, so in subsequent posts I will be exploring a range of online teaching and learning activities to examine more closely the effects of attempts at scaling, including some of the areas discussed in the HEQCO report and Tom Carey’s blog reflections. To summarise this post:

•  there are specific types of learning outcomes expected from higher education. These outcomes should drive teaching and technology decisions, not the other way round, although there is a symbiosis between outcomes and technology that needs to be considered
• these outcomes require certain kinds of teaching methods or approaches to learning, and online learning needs to be able to accommodate or support such approaches. This will mean exposing the ‘magic’ of on-campus learning and seeing if this can be defined and recreated in an online environment.
• I argue that much of this has already been achieved in online learning through good course design, which in itself is a key factor in improving productivity in higher education
• to get increases in productivity we need to examine which online learning activities can be re-designed to move from variable to fixed costs, while still maintaining or improving on the core learning outcomes expected of higher education. This will be explored more fully in later posts.

Other posts in this series

Technology, teaching and productivity: the need for theory

Is there a link between flexible access and ‘productivity’ in higher education? 

Book review by Sir John Daniel: Higher Education in the Digital Age

A review of the HEQCO report on productivity and quality in online learning in higher education

Tom Carey’s reflections on the HEQCO report on online learning and productivity: 1-Catching a teachable moment

Tom Carey’s reflections on the HEQCO report on online learning and productivity: 2 – What we left out – and why. 

References

Bates, A. (2005) Technology, e-Learning and Distance Education London: Routledge

Carey, T., & Trick, D. (2013). How Online Learning Affects Productivity, Cost and Quality in Higher Education: An Environmental Scan and Review of the Literature. Toronto: Higher Education Quality Council of Ontario

Christensen Hughes, J. and Mighty, J. (2010) Taking Stock: Research on Teaching and Learning in Higher Education Montreal anf  Kingston: McGill-Queen’s University Press