Post-Pandemic Lesson 10: We need more (and better) data

This is the last of 10 Lessons from a Post-Pandemic World. For the other nine, click here.

“Research studies were of necessity quick and dirty, often with small and probably unrepresentative samples of students, instructors, administrators and institutions.”

“More than 100,000 instructors moved their courses into emergency online learning in two weeks or less.”

“Blended learning, in particular, is now our future. Despite the widespread challenges of the pandemic, the impact on post-secondary education may well be beneficial in the long term.”

I have identified several very useful research studies on the impact of emergency online learning. Such studies were of necessity quick and dirty, often with small and probably unrepresentative samples of students, instructors, administrators and institutions.

The need for more systematic research

What we do not have is a systematic, evidence-based overview of the current state of teaching and learning as a result of the pandemic.

Fortunately, the Canadian Digital Learning Research Association (CDLRA) collected data on online learning from all Canadian post-secondary institutions in 2017, 2018 and 2019.

As a result, we know that before the pandemic, between 8 and 10 per cent of all course enrolments were in fully online courses, and that number grew at a rate of 10 per cent a year for the past few years.

Without the pandemic, we could have expected about 12 per cent of all course enrolments to be fully online by April 2021. If the CDLRA can repeat its survey in 2021, we will know how much fully online learning was impacted, at least in terms of enrolments.

Data-collection challenges

However, there are some challenges in data collection.

The first is that although the CDLRA identified about three-quarters of the Canadian institutions offering some form of blended learning in 2019, the actual number of enrolments, and the actual forms of blended learning adopted, were not tracked by institutions.

Tracking blended or hybrid learning is a particular challenge. There are many forms of blended learning, ranging from just additional online work for regular classes to a deliberate reduction in classroom time to allow for more and deeper online learning activities.

Some institutions are offering HyFlex courses, in which each student decides the amount of time spent on campus or online, depending on their circumstances. Some instructors are carefully designing courses around the respective benefits of classroom and online learning, while others are merely repeating online what they are doing in class.

The decision about whether or how to implement blended learning is typically made by the individual instructor, which means most institutions have no means to track blended learning. This would require surveys of individual instructors.

Unreliable, inconsistent data

A second challenge is that barely half the institutions surveyed by CDLRA in 2019 provided reliable and consistent data on fully online learning, even though such courses are often flagged in registration systems.

Institutions must put in place more consistent ways to record and track the type of courses being offered, and this means having consistent definitions across institutions. CDLRA is working with Canadian institutions to ensure more consistency and reliability in data collection in these areas so the 2021 survey will have reliable data for a larger number of institutions. 

Narrow scope

A third challenge is to widen the scope of the research from institutional quantitative data about enrolments, plans and technology used to more qualitative data about instructor and student responses to emergency online learning, blended learning and fully online learning. Ideally such research would be linked to learning outcomes.

It is clear that more research and, above all, more data are urgently needed for institutions and instructors to make evidence-based decisions about the future of online and blended learning.

There is a need for a funding base with contribution from Canada’s provinces.

The national research councils, especially Canada’s Social Sciences and Humanities Research Council, could also have a particularly important role to play in studying what is fast becoming a major source of strength in Canadian post-secondary teaching.

This concludes the 10 lessons. For a conclusion see here 

Reference

Wise, A. and Bergner, Y. (2020), College in the Time of Corona: Spring 2020 Student Survey Report, Learning Analytics Research Network (LEARN), New York University