Lies, Damned Lies and Statistics: WCET’s analysis of distance education enrolments in the USA

Russell Poulin and Terri Straut have done an invaluable analysis of recent data on distance education enrolments in the USA in the following three blog posts:

Straut, T. and Poulin, R. (2015) IPEDS Fall 2013: Higher Ed Sectors Vary Greatly in Distance Ed Enrollments Boulder CO: Western Co-operative for Educational Technologies

Straut, T. and Poulin, R. (2015) IPEDS Fall 2013: Distance Education Data Reveals More Than Overall Flat Growth Boulder CO: Western Co-operative for Educational Technologies

Straut, T. and Poulin, R. (2015) IPEDS Fall 2013: Less than Half of Fully Distant Students Come from Other States Boulder CO: Western Co-operative for Educational Technologies

These reports should be read in conjunction with these equally valuable posts:

Hill, P. and Poulin, R. (2014) Investigation of IPEDS Distance Education Data: System Not Ready for Modern Trends Boulder CO: Western Co-operative for Educational Technologies/e-Literate

Allen, I.E. and Seaman, J. (2013) Changing Course: Ten Years of Tracking Online Education in the United States  Wellesley MA: Babson College/Quahog Research Group

I am pulling this together in this one post for convenience, but I strongly recommend that you read carefully the original reports.

There are serious methodological issues in the USA data

Over the last ten years or so, the most consistent analyses of enrolments in online learning have been the annual Babson College surveys conducted by Elaine Allen and Jeff Seaman, with support from the Sloan Foundation. However, this was a voluntary survey, based on a carefully drawn sample of chief academic officers across the USA. The Babson Surveys showed consistent growth of online course enrolments in the order of 10-20 per cent per annum over a the last 10 years, compared with around 2-3 per cent growth in on-campus enrolments, with in 2013 approximately one third of all higher education students in the USA taking at least one fully online course.

However, since the Babson surveys were voluntary, sample-based and dependent on the good will of participating institutions, there was always a concern about the reliability of the data, and especially that the returns might be somewhat biased towards enrolments from institutions actively engaged in online learning, thus suggesting more online enrolments than in reality. Despite these possible limitations the Babson Surveys were invaluable because they provided a comparable set of national data across several years. So while the actual numbers may be a little shaky, the trends were consistent.

Then in 2012 the U.S. Federal Integrated Postsecondary Education Data System (IPEDS) survey, conducted by the National Center for Education Statistics, a division of the U.S. Federal Department of Education, for the first time included distance education in its compulsory annual survey of enrolments in higher education. (One might ask why it took until 2012 to ask for data on distance education, but hey, it’s a start.) Since this is a census rather than a survey, and since it is obligatory, one would expect that the IPEDS data would be more reliable than the Babson surveys.

However, it turns out that there are also major problems with the IPEDS survey. Phil Hill (of the blog e-Literate) and Russell Poulin have indicated the following limitations with IPEDS:

• problems of definition: Babson focused only on students enrolled in fully online courses; IPEDS asks for enrolments in distance education. Although many institutions have moved their print-based courses online, there are still many print-based distance education courses still out there. How many? We don’t know. Also the IPEDS definition rules out reporting on blended or hybrid courses, and is not precise enough to ensure that different institutions don’t interpret who to include and who to exclude on a consistent basis
• under-reporting: IPEDS collected data on the assumption that all students enrolled through continuing education departments were taking non-credit distance education courses, and therefore these enrolments were to be excluded. However, in many institutions, continuing education departments have continued to administer for-credit online courses, which institutions have seen as just another form of distance education. (In other institutions, distance education departments have been integrated with central learning technology units, and are thus included in enrolment counts.)
• the IPEDS survey does not work for innovative programs such as those with continuous enrolments, competency-based learning, or hybrid courses.

Hill and Poulin come to the following conclusions about the 2012 survey:

• we don’t know the numbers – there are too many flaws in the the data collection methods
• thus the 2012 numbers are not a credible baseline for future comparisons
• there are hundreds of thousands of students who have never been reported on any IPEDS survey that has ever been conducted.

It is against this background that we should now examine the recent analyses by Straut and Poulin on the IPEDS data for  2013. However, note their caveat:

Given the errors that we found in colleges reporting to IPEDS, the Fall 2012 distance education reported enrollments create a very unstable base for comparisons.

Main results for 2013

1. Most DE enrolments are in public universities

For those outside the USA, there are quite different types of HE institution, dependent on whether they are publicly funded or privately funded, and whether they operate for profit or not for profit. Distance education is often associated in the USA with diploma mills, or offered by for-profit private institutions, such as the University of Phoenix or Kaplan. As it turns out, this is a fundamental mis-conception. Nearly three-quarters of all DE enrolments are in publicly funded universities. Less than 10% of all DE enrolments are in for-profit private institutions.

2. Students studying exclusively at a distance

Students studying exclusively at a distance constitute about 13% of all enrolments. However, non-profits rely much more on distance students, who make up half their enrolments. Less than 10% of students in public universities are studying exclusively at a distance. The significance of this is that for most students in public universities, DE is a relatively small part of their studies, an option that they exercise occasionally and as needed, and is not seen as a replacement for campus-based studies. On the other hand, there is a substantial if small minority for whom DE is the only option, and for many of these, the for-profits are their the only option if their local public universities do not offer such programs in the discipline they want.

3. DE enrolments were down slightly in 2013

IPEDS shows an overall decrease in DE enrolments of 4% from 2012 to 2013. The biggest area was the for-profits, which declined by 17%. The drop in public universities for those taking fully online courses was a marginal 2%. However, this is a major difference from the trends identified by the Babson Surveys.

This is probably the most contentious of the conclusions, because the differences are relatively small and probably within the margin of error, given the unreliability of the data. The for-profit sector has been particularly badly hit by changes to federal financial aid for students.

However, I have been predicting that the rate of students taking fully online courses in the USA (and Canada) is likely to slow in the future for two reasons:

• there is a limit to the market for fully online studies and after 10 years of fairly large gains, it is not surprising that the rate now appears to be slowing down
• as more and more courses are offered in a hybrid mode, students have another option besides fully online for flexible study.

The counter trend is that public universities still have much more scope for increasing enrolments in fully online professional masters programs, as well as for certificates, diplomas and badges.

4. Students studying fully online are still more likely to opt for a local university

Just over half of all students enrolled exclusively in DE courses take their courses from within state. This figure jumps to between 75-90% for those enrolled in a public university. On the other hand, 70% of students enrolled in a DE course in a for-profit take their courses from out-of-state. This is not surprising, since although non-profits have to have their headquarters somewhere, they operate on a national basis.

The proportion of institutions reporting that they serve students who are outside the U.S. remains small, no more than 2% in any sector. This again may be a reporting anomaly, as 21% of institutions reported that they have students located outside the U.S. Probably of more concern is that many institutions did not report data on the location of their DE students. This may have something to do with the need for authorization for institutions to operate outside the home state, and this is a uniquely American can of worms that I don’t intend to open.

Not good, but it’s better than nothing

I have an uncomfortable feeling about the IPEDS data. It needs to be better, and it’s hard to draw any conclusions or make policy decisions on what we have seen so far.

However, it’s easy for someone outside the USA to criticise the IPEDS data, but at least it’s an attempt to measure what is an increasingly significant – and highly complex – area of higher education. We have nothing similar in Canada. At least the IPEDS data is likely to improve over time, as institutions press for clearer definitions, and are forced to collect better and more consistent data.

Also, I can’t praise too highly first of all Elaine Allen and Jeff Seaman for their pioneering efforts to collect data in this area, and Phil Hill, Russell Poulin and Terri Straut for guiding us through the minefield of IPEDS data.

For a nice infographic on this topic from WCET, click on the image below: