Answering questions about teaching online: assessment and evaluation

Following on from my Contact North webinar on the first five chapters of my book, Teaching in a Digital Age, and my blog post on this yesterday, there were four follow-up questions from the seminar to which I posted written answers. Here they are:

Unanswered Questions

Q: ­If multiple choice is not great for applied learning assessment – could you please give us some tips for more effective assessment in the virtual environment?­

Big question! There are several ways to assess applied learning, and their appropriateness will depend on the subject area and the learning goals (Look particularly at Appendix A, Section 8). Here are some examples:

• via project work, where the outcome of the project is assessed. (This could be either an individual or group project). Marking a project that may take several weeks work on the part of students helps keep the marking workload down, although this may be offset to some extent by the help that may need to be given to learners during the project.
• through e-portfolios, where students are asked to apply what they are learning to practical real-life contexts. The e-portfolio is then used to assess what students have learned by the end of the course.
• use of online discussion forums, where students are assessed on their contributions, in particular on their ability to apply knowledge to specific real world situations (e.g. in contemporary international politics)
• using simulations where students have to input data to solve problems, and make decisions. The simulation collects the data and allows for qualitative assessment by the instructor. (This depends on there being suitable simulations available or the ability to create one.)

Q: I am finding in my post-graduate online courses the professor is interacting less and less in the online weekly forums, while I know there are competing theories as to how much they should interact with students, do you have an opinion on whether or not professors should or should not interact weekly? Personally, I enjoy their interaction I find it furthers my learning.

This is another big issue. In general, the research is pretty much consistent: in online learning, instructor ‘presence’ is critical to the success of many students. Look particularly at Chapter 4, Section 4 and Chapter 11, Section 10. However presence alone is not sufficient. The online discussion must be designed properly to lead to academic learning and the instructor’s intervention should be to raise the level of thinking in the discussion (see 4.4.2 in the book). Above all, the discussion topics must be relevant and from a student’s perspective clearly contribute to answering assessment questions better. The instructors should in my view be checking daily their online discussion forums and should respond or intervene at least weekly. Again though this is a design issue; the better the course design, the less they should need to log in daily.

Q: Can you give an example of how a MOOC can supplement a face-to-face or fully online course?

I think the best way is to consider a MOOC as an open educational resource (OER). There is a whole chapter (Chapter 10) in the book on OERs. Thus MOOCs (or more likely parts of MOOCs) might be used in a flipped classroom context, where students study the MOOC then come to class to do work around it. But be careful. Many MOOCs are not OER. They are protected by copyright and cannot be used without permission. They may be available only for a limited period. If it is your own MOOC, on the other hand, that’s different. My question is though: is the MOOC material the best OER material available or are there other sources that would fit the class requirement better, such as an open textbook? Or even better, should you look at designing the course completely differently, to increase student interaction, self-learning and the development of higher order thinking skills, by using one of the other teaching methods in the book?

Q: Would better learning analytics reports help teachers have a more relevant role in MOOCS?

Learning analytics can be helpful but usually they are not sufficient. Analytics provide only quantitative or measurable data, such as time on task, demographics about successful or unsuccessful students, analysis of choices in multiple-choice tests, etc. This is always useful information but will not necessarily tell you why students are struggling to understand or are not continuing. Compare this with a good online discussion forum where students can raise questions and the instructor can respond. Students’ comments, questions and discussion can provide a lot of valuable feedback about the design of the course, but require in most cases some form of qualitative analysis by the instructor. This is difficult in massive online courses and learning analytics alone will not resolve this, although they can help, for instance, in focusing down on those parts of the MOOC where students are having difficulties.

Any more questions?

I’m more than happy to post regular responses to any questions you may have about online teaching, either related to the book or quite independent of it. Just send them to me at tony.bates@ubc.ca