Let students see their course data: a learning analytics project from the University of Michigan

Love, J., DeMonner, S., and Teasley, S. (2021) Show Students Their Data: Using Dashboards to Support Self-Regulated Learning Educause Review, July 20

Learning analytics (LA) have tended to be focused on the needs of administrators, primarily, then on instructors’ needs, but LA can also be very useful for students, especially but not only on online courses, where it may be difficult for students to compare how they are progressing compared with other students.

At the University of Michigan, the School of Education worked with the School of Information, and also in collaboration with developers at the University of British Columbia:

to develop a learning analytics dashboard [called MyLA] for students based on research-informed design and with the intention to learn more about the efficacy of such a dashboard in student learning. The design approach was based on principles from learning theory, human–computer interaction, and information visualization. In addition to incorporating student voices in the design phases, the team surveyed students and studied usage patterns when the dashboard was rolled out on campus.

I suggest you read the original article for details, but here’s a quick summary:

Key guidelines

With a focus on supporting self-regulated learning for students, the MyLA team identified and applied six design guidelines:

• Support self-regulated learning and mastery orientation
• Allow students to control what they see
• Support social comparison
• Use a simple, consistent visualization/interaction vocabulary
• Preserve students’ privacy
• Integrate Canvas and UDP for scalability and adoption

Where the data comes from

MyLA draws on learner activity data that is stored in the Unizin Data Platform (UDP), which collects data from the institutional LMS (Canvas), Kultura and other tools. MyLA was used in more than ninety courses at U-M, with over 7,200 students having access to it, some of them having access in three or more courses. It is also being piloted in seven other institutions.

What the dashboard shows

Resources accessed

Students can see which files, videos, and other resources in the course are accessed most often by classmates (see banner figure). The goal of the view is to help students plan their use of class resources by allowing them to see which resources other students are using that they might have overlooked. File names are active links, enabling students to immediately open resources they haven’t yet reviewed. Students can also customize their view in several ways to explore the resources of interest to them.

Assignment planning

Students can use the progress bar to see their current score and whether they are meeting learning goals

Students have the option to set an overall course goal—as well as individual assignment goals—as they are planning their work. They can then revise and update the goals throughout the term.

Grade distribution

The Grade Distribution view lets students monitor their performance in the class relative to the performance of their classmates. The team took care to protect the privacy of low-scoring students by adopting a k-anonymity approach to binning the lowest scores together and not displaying explicitly where they fall on the axis.

Student usage and feedback

Researchers have reviewed the voluntary use of MyLA in sixty-one U-M courses from fall 2018 through fall 2020. Overall student responses have been highly positive. Almost all student users (92%) said they would like the dashboard to be available in all of their courses.

Comment

I think this study is just the tip of an iceberg. The results are interesting, the design and analysis were well conducted, and I am delighted to see students having access to analyses of their own online behaviour.

However, LA could provide students with much more valuable data, such as comparisons between their different online activities and grade performance, tracking of their learning outcomes across a program rather than a course, and also tracking of their instructors’ interactions with themselves and other students. 

Of course, ethics and privacy are issues that will hound all applications of LA, so it is essential that students are, as in this study, fully involved in the design and analysis of LA on their activities.

I noticed MyLA was using open source programming, but what I would have liked is some idea of the costs and work involved in doing this, and the extent to which in can be easily replicated or extended. Overall, though, this is an interesting project.