The importance of redesign in online learning: the NCAT experience

Twigg, C. (2012) Improving Learning and Reducing Costs: Initial Project Outcomes from Changing the Equation The Learning MarketSpace, July

The results of a major program aimed at re-designing math curricula in two year colleges are announced in the National Centre for Academic Transformation‘s July Newsletter .

The program

Changing the Equation is a major program to engage the community colleges in the USA in a successful redesign of their remedial/developmental math sequences. There were 38 participating two-year colleges.

Each participant in Changing the Equation redesigned its entire developmental math sequence–all sections of all developmental courses offered–using NCAT’s Emporium Model and commercially available instructional software. Each redesign modularized the curriculum, allowing students to progress through the developmental course sequence at a faster pace if possible or at a slower pace if necessary, spending the amount of time needed to master the course content. Institutions piloted their redesign plans in spring 2011 and fully implemented their plans in fall 2011.

Results

Preliminary results are in for 20 of the 38 colleges. The main results are as follows:

Improved learning outcomes

Twenty institutions redesigned a total of 54 developmental math courses. Student learning outcomes in the traditional and redesigned formats of the courses were measured by comparing performance on common final examinations, common exam items and/or gains on pre- and post-tests with the following results:

• 46 courses (85%): showed significant improvements
• 6 courses (13%): showed no significant difference
• 2 courses (2%): insufficient data was collected to make a comparison

Course Completion

NCAT asked each institution to compare course-by-course completion rates (grades of C or better or grades of P in a P/F system) between the traditional and redesigned formats. Twenty institutions redesigned a total of 54 developmental math courses with the following course-by-course completion results:

• 7 courses (13%): had higher completion rates, 3 of which were significantly higher
• 5 courses (9%): showed no significant difference in completion rates
• 28 courses (52%): had lower completion rates, 16 of which were significantly lower
• 12 courses (22%): completion could not be calculated due to the collapse of multiple courses into one
• 2 courses (4%): insufficient data were collected to make a comparison

This suggests that in terms of completion, the redesign was unsuccessful. However, the reasons put forward by NCAT to explain this are quite interesting:

• prior grade inflation: pass rates in the traditional format were inflated by prior inconsistencies in grading practices.
• mastery learning requirement in the redesign: students were required to master all of the content of all of the courses.
• the redesigned courses were more difficult: more content and more activities such as tests, quizzes which meant students had to work longer/harder; it should be remembered too that what matters is the final completion rate, how many succeed at graduation. NCAT argues that students who succeed in the redesigned courses do better at graduating, thus boosting the final graduation rates. (Evidence to support this argument is not yet available).

Cost reductions

With regard to cost savings, the CTE results were almost universally successful. All but one of the 20 CTE projects reduced their costs.

The actual reduction in the cost-per-student for the 20 institutions was about 20%.

• 4 institutions (20%) reduced the cost-per-student between 30% and 40%.
• 10 institutions (50%) reduced the cost-per-student between 15% and 30%.
• 5 institutions (25%) reduced the cost-per-student 15% or less.
• 1 institution (5%) did not produce cost savings.

There were two primary ways that cost reduction was achieved: 1) increasing section size or 2) increasing the number of sections that full-time and adjunct faculty counted toward their load (e.g., teaching two redesigned sections rather than one section for one workload credit.) Both of these strategies were implemented without increasing faculty workload because of the elimination of repetitive tasks such as hand-grading homework, quizzes and exams.

Comment

Hybrid learning is not new. Since 1999, Carol Twigg and the National Center for Academic Transformation have been plugging away at the redesign of core or foundational courses in higher education institutions in the USA. (Go the the NCAT site for more results and reports of other projects).

This program has had some remarkable successes, without any of the mass media hype and hysteria around MOOCs. It appears from these results  – and other NCAT programs in universities – that the NCAT redesign program results in higher quality learning at less cost (although the average cost per graduating student may be affected by the lower completion rates – it is impossible to tell from the report).

Course redesign is not easy. It requires hard work and discipline. Institutions sometimes find the requirements for course redesign too onerous, or simply ignore some of the conditions that NCAT demands. Also, NCAT’s results are not independently published or verified, and some institutions have complained in the past that they had to absorb many of the costs. There are also other possible redesign models than the NCAT model that would better fit other learning philosophies and subject requirements.

Nevertheless, what the NCAT projects demonstrate in spades is the importance of redesign. It can lead to both better student performance and reduced costs. Yes, the use of computers and online learning is a critical component, but it is the redesign and the more effective use of faculty time that makes the difference. Just adding technology to the classroom or replicating the classroom through lecture capture will not produce the same results.