Productivity and online learning redux

Summarizing the previous posts

In previous posts (see end of this post), I tried to identify a range of areas where online learning might enable productivity gains. In this post I will bring them together and state what I believe are the areas that offer the potential for the greatest gains, given current knowledge. At this stage my conclusions are very subjective but I hope they will provide a framework for further studies and for better and more systematic data collection.

Access

There are fairly strong arguments (but little hard data) to suggest that online learning can help governments boost participation rates more effectively than by building more campuses and funding more campus-based education. Productivity is increased by eventually getting more graduates than would otherwise have been possible without online learning and the flexibility it allows.

The reasons to support this argument are as follows:

• Particularly in jurisdictions where there is already a high participation rate, increasing that rate further means reaching out to groups that have to date been largely excluded from post-secondary education. Since existing data indicates that online learning appeals particularly to lifelong learners, working adults and older learners, online learning is more likely to appeal to this target group. To date though there has been little attempt to measure the impact of reaching otherwise excluded groups through online learning. More hard data is needed to support this argument.
• there is some evidence to suggest that online learning has lower overhead costs than campus-based education. If this is correct, it may be more productive to expand online learning rather than build new campuses when attempting to increase participation rates. Again though there are few studies that provide hard data to show that overhead costs are indeed lower for online learning.
• it has been argued that hybrid and online learning provides existing students with more flexibility, allowing them to combine work with part-time study, thus allowing them either to complete studies they could not afford without part-time work, or to complete more quickly than they would without the flexibility that online learning provides. Of course the argument could work in reverse. By providing more flexibility, students may take longer to graduate. Once again, it should be possible to test either argument empirically, but again there are few if any studies that have looked at this.

Thus online learning offers the promise of increasing productivity through increasing participation and speed to graduation  at less cost, but there are few studies to date to either support or refute these claims. It should be noted though that in most cases, the data to test these arguments is within institutional databases; it has just not been extracted and analyzed for this purpose.

Free or massively scalable content

Nowhere in online learning is there such potential for increases in educational productivity as in content development and delivery. Once learning materials are created, they can be stored, accessed, delivered and used by an unlimited number of learners, thus potentially achieving large economies of scale and thereby reducing costs per learner (see graph at beginning of this post).

Another important factor contributing to economies of scale in online learning is the increasing availability of open educational resources. Particularly in foundational courses and many ‘standard’ undergraduate courses, ‘open’ material is already available and does not have to be re-created. The main cost is selecting and organizing existing open source materials, but this is likely to be less time-consuming for faculty than creating materials from scratch. Open online textbooks can have a direct and immediate impact on reducing student costs.

Nevertheless there are many impediments to achieving productivity gains through free or massively scalable content:

• faculty often see themselves as  creating unique and original material in their teaching; this is true occasionally and needs to be respected, especially where faculty are teaching about their own and related research. Often though faculty are merely repackaging prior knowledge. That prior knowledge is increasingly being made available and open for anyone to use.
• the shelf life of much academic material is increasingly short; thus content needs to be constantly maintained and updated
• there may not be a massive market for many specialist online courses thus preventing economies of scale from being achieved. However, there are many ways to increase market reach with online learning, including going global, collaborating and sharing materials, courses and programs with partner institutions in the same or other jurisdictions, repackaging content for different markets (e.g. for casual learners, certificates, or degrees). Such strategies though will also require reviewing and often changing admission policies, intellectual property agreements and other practices that restrict access to ‘institutional’ content.
• the quality of open educational resources developed by faculty working alone, without applying best course design practices, is often very low and such ‘open’ resources are often not considered suitable for re-use
• content development and delivery is a relatively small proportion of the cost of credit-based online learning (from 15-20%); the main costs are in learner support.

Despite these impediments, in certain circumstances (i.e. where there is a large market and best practices are applied to content design), online content development and delivery is already resulting in increased productivity in post-secondary education, although it has yet to be well measured.

Course design based on sound pedagogical principles

One important reason for the success of many for-credit online courses and programs has been the introduction of best practices in course design, drawing on cognitive science research, best teaching practices, and prior experience of teaching students at a distance. These practices include situated learning, drawing on learners’ own work and life experiences, student time-management support, collaborative learning, student activities resulting in greater time on task, and regular and constructive feedback to students through continuous assessment.

In particular a focus in online courses on ’21st century skills’ development, such as knowledge management and independent learning, would have two benefits. It would improve outputs (turning out graduates with the skills needed). Second, content development and delivery becomes subsidiary to helping students find, analyze, organize and apply content themselves. Thus less time would be spent by instructors on course development and delivery.

Such practices of course could be be used in classroom teaching, but good online course design templates are more easily scaled and reproduced, and the technology lends itself to such approaches to learning. Productivity is improved through application of such quality course design because more students achieve higher levels of learning and more students complete courses and programs. Thus although it is not the technology itself that results in better outcomes, the technology facilitates the change to more effective teaching methods.

Once again though, while teachers and students who have been engaged in such new designs often claim better outcomes, there is still a lack of convincing empirical research to support these beliefs. Nevertheless, a focus on better design replicated on a large scale through online learning should have a major impact on improving productivity.

Learner support

What little research that has been done on costs of credit-based online learning indicates that course delivery (which includes both learner support and student assessment) accounted for the largest overall cost of an online program (37%), almost three times more than course development, over the life of an online program (Bates and Sangra, 2011).

Instructional MOOCs (xMOOCs) have basically removed learner support, at least in terms of human (instructor) support, but this has resulted in a very low number of MOOC learners passing end-of-course assessments of learning. Indeed, prior research into credit-based learning has established that instructor online ‘presence’ is a critical factor in retaining students. So far, it has proved difficult to scale up learner support on a massive scale, except through the use of computer technology, such as automated feedback. However, Carey and Trick (2013) and indeed faculty at elite institutions who are offering xMOOCs (see Thrun and ‘the Magic of the Campus‘) have argued that such computer support does not support ‘the learning that matters most’.

Although computer-based feedback and adaptive learning facilitate comprehension and technical mastery outcomes, computer-based approaches to learner support to date has been inadequate for formal assessment of higher order learning skills such as original, critical or strategic thinking, evaluation of strategies or alternative explanations. To assess such forms of learning, deep expertise and qualitative assessment is required, and to date not only human instructors, but instructors with a deep subject understanding and high levels of expertise, are needed to both develop and assess such high level skills. Given the long history of trying to apply artificial intelligence to instruction, immediate and major breakthroughs seem unlikely, at least in the short term.

However, there are other ways in which the productivity of learner support might be improved. In cMOOCs that are more like communities of practice and thus contain many participants with already high levels of expertise, that expertise and judgement can be provided by the participants themselves. (The issue then is how do people get to such high levels of expertise in the first place – we need more research/experience with cMOOCs to know whether they are also good for learners with initially low levels of knowledge in a particular subject domain. Some combination of expert/instructors plus a community of practice approach might be necessary for such learners, but might still operate successfully with much higher instructor:learner ratios than in conventional, credit-based learning.)

Also, credit-based online learning has achieved some economies of scale and scope by re-organizing the learner support process, through the hire and training in online learner support of lower-paid contact adjuncts who still have high level academic qualifications, under the supervision of a senior faculty member. In other words, team teaching approaches, with the senior academic working more as a teaching consultant, setting curriculum, designing assessments and creating rubrics, and supervising the learner support provided by a team of adjuncts, can help not only reduce costs but also achieve modest economies of scale in learner support, especially when combined with best practices in course design.

Innovation vs standardization

In industry, innovation is often another way of saying ‘investment in technology’. However, there is more to innovation than just replacing a human activity with a computer-based activity. What the technology usually brings about is a change in process at the same time.

Thus there is a natural tension between ‘best practice’, based on experience of doing things in an ‘old’ way, and innovation, which means doing something differently. Indeed Christensen (2008) distinguished between ‘sustaining’ innovation, which builds and improves on best practices, and ‘disruptive’ innovation, where a new technology results in sweeping away old ways of doing something. Real, sustainable innovation occurs then when new technology is combined with new processes.

In education, perhaps the main ‘process’ that we need to examine is the instructional model, particularly that based around the lecture system. I am not arguing that lectures no longer have a purpose. However, the teaching model based on three lectures a week over 13 weeks used primarily to deliver information to students is now redundant, given that information is ubiquitous and if not free, increasingly available at low cost over the Internet. Thus knowledge management becomes more important than mere access to knowledge. If we look at xMOOCs though we have taken a new technology – video lecture capture and Internet transmission – and applied it to an outdated model of teaching. True innovation requires a change of process or method as well as a change of technology.

Earlier though I argued that we need to apply best practices in course design to the use of technology. By definition, best practices are based on tried and true methods. However, in post-secondary education, these ‘best practices’ are not the prevailing teaching method on most campuses (except perhaps the very elite, where they can be applied on a face-to-face basis to small classes.). As public post-secondary education has become massified, the lecture has become the default model, because in a classroom based system, it has proved the only way to ‘scale up.’ Online learning offers an opportunity to break out of this redundant and increasingly less productive lecture model of teaching, as it does not develop the skills needed in the 21st century.

There are then really two routes to innovation. The least risky is the sustainable development approach, finding ways to incorporate and more importantly adapt ‘best practices’ on a massive scale through online learning. This will mean increasing productivity in relatively small steps. The advantage of this approach is that it is more likely to preserve and protect the core values of ‘higher’ education. The other route is ‘disruptive’ innovation – jumping into an entirely different way of doing something based around a new or emerging technology. This is more likely to bring much greater productivity gains, but the risks are much higher. It could well result in throwing the baby out with the bathwater.

In reality, institutions, and individuals within those institutions, cannot control disruptive innovation – it comes from outside. However, institutions can control sustainable innovation. Indeed if they do not they are much more vulnerable to disruptive innovation. Thus it is important to find new best practices that are easily scalable, while meeting the needs of 21st century learners at high quality. This is probably the most sensible way to bring about radically better productivity. But it’s not going to be easy.

Conclusions

So here are my personal views on online learning and productivity, based on this analysis.

1. Government and institutional leaders need to set improved productivity as a key goal for investment in learning technologies. This means setting benchmarks and implementing means of measuring success or otherwise in improving productivity through learning technologies/online learning. Data analytics now make this measurement more feasible than in the past, but it also requires models or a theoretical framework for assessing what constitutes productivity in a post-secondary educational setting.

2. Understanding the basic cost structures of online learning, compared to the costs of classroom teaching, is an essential first step to increasing productivity in post-secondary education. It is risky to assume that online learning is always more cost-effective or productive; the circumstances need to be right.

3.Content is only one component of teaching (and an increasingly less important component); other components such as learner support and assessment are even more important. Care is needed then because changes in methods of online content development and delivery could have negative knock-on cost and productivity consequences in other areas of course delivery, such as learner support and assessment. In looking at productivity issues, all these factors need to be examined together.

4. Any attempts at scaling or increasing economies of scope in content development and delivery need to be balanced by ensuring quality does not suffer. However, online course development has the potential, through good course design, to improve quality rather than reduce it.

5. The ‘learning that matters most’ mainly addresses university teaching, but also increasingly technical, vocational and corporate training; the aim is to develop the knowledge and skills needed in a knowledge-based society. Online learning can handle the ‘learning that matters most’ as well, in most cases, as on-campus teaching, although there will always be some exceptions.

6. However, there are major difficulties in scaling up the learner support and assessment activities that are needed for the learning that matters most, both online or on campus. The danger in scaling up is the loss of quality in terms of learning outcomes.

7. Adaptive learning software that helps individualize learning, and learning analytics, may help to a small degree in enabling instructors to handle slightly more students without loss of quality, but cannot as yet replace a skilled instructor, and probably never will. Higher education requires expertise and qualitative assessment for the learning that matters most, and that will need human instructors.

8. New online course designs built around the use of new technologies have greater potential for increases in productivity – through producing better learning outcomes – for the learning that matters most, than through scaling up, i.e. by increasing teacher:student ratios.

9. We need more empirical research on the relationship between teaching methods, mode of delivery, costs, and the type of learning outcomes that constitute the ‘learning that matters most’ (not to mention better definitions).

Now it’s your turn

As I’ve said, this has been a struggle for me to work through the issues of online learning and productivity. The whole purpose of this arduous exercise is to promote debate and discussion about productivity and online learning. So I’d really welcome your comments. It would be great to hear from people with experience in productivity in other areas besides education, and to hear from those in education about the potential or dangers of applying the thinking around productivity to online learning.

So go for it!

Other posts in the series

• Technology, teaching and productivity: the need for theory
• Is there a link between flexible access and ‘productivity’ in higher education? 
• Book review by Sir John Daniel: Higher Education in the Digital Age
• A review of the HEQCO report on productivity and quality in online learning in higher education
• Tom Carey’s reflections on the HEQCO report on online learning and productivity: 1-Catching a teachable moment
• Tom Carey’s reflections on the HEQCO report on online learning and productivity: 2 – What we left out – and why.
• Towards a theory or model of productivity for online learning: outcomes, scale and design
• How online content development and delivery could improve the productivity of post-secondary education
• Alternative ways to improve productivity through online learner-content interaction
• Improving productivity in online learning: can we scale the ‘learning that matters most’?
• Can online learning lead to productivity gains through savings on campus facilities?
• Productivity and online learning: a summary of the main concepts