This is the third post reviewing articles in the journal Distance Education, Vol. 40, No.3, a special edition on learning analytics in distance education. The first post was on analytics and learning design at the UKOU. The second post was on analytics and personality traits in a high school in China.
As always if you find these posts of interest, please read the original articles. Your conclusions will almost certainly be different from mine.
The aim of the study
The study used learning analytics to understand the effects of gamification on students’ online interactive patterns, posting quantity and peer feedback quality.
Method
The research was conducted on 96 undergraduate students in a university in Hong Kong taking a blended learning information management foundation course that combined face-to-face lectures with online discussion forums.
The study used a quasi-experimental design, comparing results from a version of the course that received gamification treatment with a version of the same course that did not receive the gamification treatment.
The design of the gamification version was based on the GAFCC design model (goal, access, feedback, challenge, collaboration). This is a theory-driven approach to gamification (Huang and Hew, 2018).
The study measured the impact of gamification strategies in comparison with the control group on the following:
- pre- and post-tests
- quantity of student online postings
- quality of students’ peer feedback in discussion forums (using a critical thinking evaluation scale – Greenlaw and Deloach, 2003)
- analysis of students’ online interactive patterns using social network analysis using the UCINET tool (Borgatti, Everett and Freeman 2002), in terms of centralisation and centrality
Results
Pre- and post-test
Students in the gamification group improved significantly more between pre- and post-tests, compared with the control group (there was no significant difference in prior knowledge).
Quantity of online postings
In six of the nine tutorial groups, students in the gamification group posted significantly more comments than in the control group.
Quality of online postings
Students in the gamification group scored more highly than students in the control group on the higher levels of critical thinking in the Greenlaw and Deloach critical thinking test.
Students’ online interactive relationships
The social network analysis indicated that students in the gamification group had slightly more interactions and the discussion was more equally distributed between participants than in the control group.
‘Overall, the learning analytics results provided evidence that students using the gamification-based online learning approach achieved at higher levels than the students using the conventional online learning approach.’
Comment
Although one can always quibble about minor points – the authors themselves pointed out some weaknesses in the design – this was an excellent paper on several grounds:
- It was able to show clear benefits for a particular approach to teaching (gamification).
- The design of the study was based on firm educational, theoretical bases (game theory, social network analysis, critical thinking).
- The study used a combination of direct qualitative and quantitative measurements of learning.
- The way the results were arrived at were clear, logical and understandable, especially the data analysis.
- The study indicated the value of using learning analytics to measure a particular educational intervention (gamification). Learning analytics in this study was a means to an end, not an end in itself.
These are pretty useful guidelines if you are thinking of using learning analytics.
Next up
Predicting how far a student is from mastery of knowledge.
Reference
Borgatti, S., Everett, M. and Freeman, L. (2002) UCINET for Windows: Software for social network analysis Harvard MA: Analytic Technologies
Greenlaw, S. and Deloach, S. (2003) Teaching critical thinking with electronic discussion The Journal of Economic Education, Vol. 34
Huang, B. and Hew, K. (2018) Implementing an theory-driven gamification model in higher education flipped courses: effect on out-of-class activity completion and quality of artifacts Computers and Education, Vol.125
Dr. Tony Bates is the author of eleven books in the field of online learning and distance education. He has provided consulting services specializing in training in the planning and management of online learning and distance education, working with over 40 organizations in 25 countries. Tony is a Research Associate with Contact North | Contact Nord, Ontario’s Distance Education & Training Network.
