Tom Carey’s reflections on the HEQCO report on online learning and productivity: 2 – What we left out – and why.

Carey, T., & Trick, D. (2013). How Online Learning Affects Productivity, Cost and Quality in Higher Education: An Environmental Scan and Review of the Literature. Toronto: Higher Education Quality Council of Ontario.

Tom Carey is one of the authors of the above study, and as an example of the best of reflective practice, he has kindly provided his thoughts about the report, now that it is finished. His first thoughts were published yesterday. This is the second part.

Tom Carey: Part II of Reflections on Researching and Writing on Emerging Developments in Online Learning

In yesterday’s guest post, I provided some reflections on the process and product of a research project for the Higher Education Quality Council of Ontario (HEQCO): How Online Learning Affects Productivity, Cost and Quality in Higher Education: An Environmental Scan and Review of the Literature. That post described the Results that Surprised in the report, from my perspective as an author. Today’s post provides some insights about what we chose to not include in the report, following the old advice of expert film editors that the most interesting scenes in a movie may be those “left behind on the cutting room floor”.

Developments We Didn’t Include

There were some emerging developments on our original target list for which we could not find compelling examples at scale: Semantic Web, Mobile Learning, Ubiquitous Connective, etc. I am sure these are going to be important, but in the interests of preserving the Teachable Moment aspects we focused only on developments with convincing data for impacts on learning outcomes and productivity: convincing in the context of Ontario higher education institutions.  For example, the Ithaka study of the Open Learning Initiative software allowed us to highlight Adaptive Learning Systems at scale (and the recent follow-up book by William Bowen contains several other insights we could cite if we were starting now).

Similarly, the report only deals with Open Educational Resources as a sideline in the discussion of Open and Affordable Textbooks: the rationale was that the textbook developments were a hot topic in “peer” higher education systems ‒ British Columbia, California, New York, etc. ‒ represented low hanging fruit in terms of potential for building collaborations amongst students, faculty and academic leaders within institutions.  And we didn’t do any justice to the Canadian developments in connectivist MOOCs, mostly because we had our hands full trying to help our target audience make sense of the instructionist MOOCs that were hogging the headlines and couldn’t work out how to get beyond that without losing their attention. (I have already apologized to George for this: Stephen, Dave et al can consider themselves included in the apology.)   These choices about which innovations to highlight may have bypassed the disruptive in favour of the radical, but helping decision makers to make sense of – and act on – opportunities for radical change was more than enough for us to bite off.

Issues We Couldn’t Include

Some of the content we wrote but could not include in the report was just not clear enough or complete enough to be included in the public document. For a few topics, we were keenly aware that more work had to be done but that we had not made sense of what that work might be. We didn’t want to go on at length in the report about these points for several reasons: calling for further research sounds like too familiar an ending to a Research Report, including more than a quick mention for what is not yet clear seemed to detract from our Teachable Moment goal, and some of the further exploration needed would be an outcome of our proposed Call for Action through collaborative sense-making across institutions.  For those interested in ‘where to next’ in terms of understanding the impact of emerging developments, here is my personal list of high priority issues that need more clarity.

The different roles of various online learning interactions in various contexts: I would like to have referenced the work by Terry Anderson and others on how increases in one form of learning interaction can result in a decreased need for another type of interaction. This was implicit in our Call to Action around understanding and leveraging scalability: use more scalable interactions where appropriate in order to redirect resources – especially time – into other interactions which are less readily scaled.

Here is my current woefully incomplete attempt to reframe our analysis of emerging developments in online learning using different types of learning interactions – whether online or not:

• Learner-content interactions can be used effectively to advance Quality and Productivity for technical mastery outcomes, e.g., performance tasks with single solutions and predictable pathways to completion (allowing adaptive systems to provide task guidance)
• Learner-learner interactions can be used effectively to advance Quality and Productivity for (some) of the question-and-answer and formative feedback roles traditionally carried out with learner-instructor interactions, and seem to be essential (at the moment?) for outcomes involving complex challenges with diverse ways of knowing.
• Learner-instructor interactions appear to be essential for outcomes involving deep personal change related to learning itself:  grappling with threshold concepts, enhancing learning mindsets and strategies, and ‘getting better at getting better’ for knowledge-intensive work
• Learner-expert interactions are required for formation of learners’ identity and practice as members of knowledge-building communities, whether in professional/career contexts or in their roles as community members and global citizens.

Much more work to be done in this area, including ensuring that the outcomes listed above that are not readily scaled don’t get left out in the quest for greater productivity:  if we neglect such outcomes, where would the ‘higher’ be in higher education?

Institutional productivity gains may be possible @ scale in traditional institutions: you may have noticed that the list of interactions above has unbundled the role of “instructor” (who can apply expertise in pedagogical content knowledge) from the role of “practice expert” who can help learners transition into full engagement with knowledge-practice networks. Traditional institutions may struggle to unbundle such roles, or even to respect their differing contributions.

Traditional institutions – and in Ontario higher education, that is all we have at the moment ‒ may also struggle to reinvest the results of productivity gains from online learning beyond a course context. As long as we think of ‘workload’ in terms of ‘courses taught’, any savings in effort may disappear into other localized activities. How can we reframe the work, and workload, of teaching to optimize educator and learner time, without resorting to an alien ‘managerial’ language? (Mention “activity-based-costing” in a budget meeting and the challenges to reinvesting productivity gains become all too clear J.)

I tried adapting the idea of Constructive Alignment from instructional design, with an expansion into “Productive Alignment” where educators also include in their designs the goal of optimizing resource usage. If certain students can achieve certain learning outcomes with reduced learner-instructor interactions, e.g., with MOOC resources used in a hybrid course format, then effective instructional design requires that we achieve this Productive Alignment to optimize time and resources. I couldn’t explain this notion effectively enough to include it in the report, but I am convinced that some such changes in the ways we talk about educator roles and responsibilities are going to be needed if the full potential of online learning is to be achieved. And this is going to be both more necessary and more difficult with the ‘higher’ learning experiences and outcomes listed above, which develop slowly over the course of a program and are not readily described as discrete competencies to be tested in a short-term performance task.

Dealing with Quality and Productivity in tandem is a fiscal, political and pedagogical necessity: finally, I wish we had been able to make a better case for the pedagogical rationale for dealing with Scalability, Quality and Productivity issues in parallel. We did include some rationale for determined action now on systematic collaborations across institutions to understand and leverage the emerging potential of online learning.

However, that argument was framed mostly in terms of fiscal and political realities. The fiscal reality across higher education systems requires that we get more focused on deploying the least resources to achieve the highest level of outcomes, and the political reality requires that we in public higher education either ‘do or be done to’ in our dealings with Quality and Productivity.

But there is another rationale that is more closely linked to our purposes and ideals in higher education. Even if we did not have our current fiscal constraints and the expectations of stricter constraints in the future…even if public higher education had the full confidence of political leaders as to our ability to change and adapt to our changing circumstances…if our students see us clinging to traditional practices and structures rather than taking on our challenges with boldness and confidence, what model are we presenting to them about how to deal with challenges in their workplaces, their families and communities, in the earth’s environment and the global knowledge economy?  Will our plea for engagement with the knowledge and wisdom of the past, present and future fall on deaf ears if we don’t practice what we preach?

Making that case was beyond our reach in this project, but it remains on my personal to-do list. I keep thinking of Parker Palmer’s concise formulation in The Courage to Teach: “How we teach is a critical part of what we teach”. That is the pedagogical rationale for our taking charge of higher education’s fate by applying emerging knowledge and wisdom about online learning…with care, compassion and courage.