A balanced research report on the hopes and realities of MOOCs

Hollands, F. and Tirthali, D. (2014) MOOCs: Expectations and Reality New York: Columbia University Teachers’ College, Center for Benefit-Cost Studies of Education, 211 pp

We are now beginning to see a number of new research publications on MOOCs. The journal Distance Education will be publishing a series of research articles on MOOCs in June, but now Hollands and Tirthali have produced a comprehensive research analysis of MOOCs.

What the study is about

We have been watching for evidence that MOOCs are cost-effective in producing desirable educational outcomes compared to face-to-face experiences or other online interventions. While the MOOC phenomenon is not mature enough to afford conclusions on the question of long-term cost-effectiveness, this study serves as an exploration of the goals of institutions creating or adopting MOOCs and how these institutions define effectiveness of their MOOC initiatives. We assess the current evidence regarding whether and how these goals are being achieved and at what cost, and we review expectations regarding the role of MOOCs in education over the next five years. 

The authors used interviews with over 80 individuals covering 62 institutions ‘active in the MOOCspace’, cost analysis, and analysis of other research on MOOCs to support their findings. They identified six goals from the 29 institutions in the study that offered MOOCs, with following analysis of success or otherwise in accomplishing such goals:

1. Extending reach (65% 0f the 29 institutions)

Data from MOOC platforms indicate that MOOCs are providing educational opportunities to millions of individuals across the world. However, most MOOC participants are already well-educated and employed, and only a small fraction of them fully engages with the courses. Overall, the evidence suggests that MOOCs are currently falling far short of “democratizing” education and may, for now, be doing more to increase gaps in access to education than to diminish them. 

2. Building and maintaining brand (41%)

While many institutions have received significant media attention as a result of their MOOC activities, isolating and measuring impact of any new initiative on brand is a difficult exercise. Most institutions are only just beginning to think about how to capture and quantify branding-related benefits.

3. Reducing costs or increasing revenues (38%)

….revenue streams for MOOCs are slowly materializing but we do not expect the costs of MOOC production to fall significantly given the highly labor-intensive nature of the process. While these costs may be amortized across multiple uses and multiple years, they will still be additive costs to the institutions creating MOOCs. Free, non-credit bearing MOOCs are likely to remain available only from the wealthiest institutions that can subsidize the costs from other sources of funds. For most institutions, ongoing participation in the current MOOC experimentation will be unaffordable unless they can offer credentials of economic value to attract fee-paying participants, or can use MOOCs to replace traditional offerings more efficiently, most likely by reducing expensive personnel. 

4. Improving educational outcomes (38%)

for the most part, actual impact on educational outcomes has not been documented in any rigorous fashion. Consequently, in most cases, it is unclear whether the goal of improving educational outcomes has been achieved . However, there were two exceptions, providing evidence of improvement in student performance as a result of adopting MOOC strategies in on-campus courses

5. Innovation in teaching and learning (38%)

It is abundantly clear that MOOCs have prompted many institutions and faculty members to engage in new educational activities. The strategies employed online such as frequent assessments and short lectures interspersed with questions are being taken back on-campus. It is less clear what has been gained by these new initiatives because the value of innovation is hard to measure unless it can be tied to a further, more tangible objective. We …. conclude that most institutions are not yet making any rigorous attempt to assess whether MOOCs are more or less effective than other strategies to achieve these goals. 

6. Research on teaching and learning (28%)

A great deal of effort is being expended on trying to improve participant engagement and completion of MOOCs and less effort on determining whether participants actually gain skills or knowledge from the courses ….While the potential for MOOCs to contribute significantly to the development of personalized and adaptive learning is high, the reality is far from being achieved. 

Cost analysis

The report investigates the costs of developing MOOCs compared to those for credit-based online courses, but found wide variations and lack of reliable data.

Conclusions from the report

The authors came to the following conclusions:

1. there is no doubt that online and hybrid learning is here to stay and that MOOCs have catalyzed a shift in stance by some of the most strongly branded institutions in the United States and abroad.

2. MOOCs could potentially affect higher education in more revolutionary ways by:

• offering participants credentials of economic value

• catalyzing the development of true adaptive learning experiences

However, either of these developments face substantial barriers and will require major changes in the status quo.

My comments on the report

First this is an excellent, comprehensive and thoughtful analysis of the expectations and realities of MOOCs. It is balanced, but where necessary critical of the unjustified claims often made about MOOCs. This report should be required reading for anyone contemplating offering MOOCs.

Different people will take away different conclusions from this report, as one would expect from a balanced study. From my perspective, though, it has done little to change my views about MOOCs. MOOC providers to date have made little effort to identify the actual learning that takes place. It seems to be enough for many MOOC proponents to just offer a course, on the assumption that if people participate they will learn.

Nevertheless, MOOCs are evolving. Some of the best practices that have been used in credit-based online courses are now being gradually adopted as more MOOC players enter the market with experience of credit-based online learning. MOOCs will eventually occupy a small but important niche as an alternative form of non-formal, continuing and open education. They have proved valuable in making online learning more acceptable within traditional institutions that have resisted online learning previously. But no-one should fear them as a threat to credit-based education, either campus-based or online.