Over the holiday season, lots of people play games such as Scrabble, cards or crossword puzzles. I decided to play with ChatGPT by testing it in areas where I consider myself an expert. (For more about ChatGPT, go to Broom, 2022)

I will first of all show you the responses I got from ChatGPT, then I will discuss the results, comparing them to what I wrote about these topics in Teaching in a Digital Age.

Asynchronous vs synchronous learning

My question:

What is the difference between synchronous and asynchronous learning? Give references. 

ChatGPT:

Synchronous learning refers to educational activities that take place in real-time, with all participants present at the same time. This could include traditional in-person classes, as well as virtual classes that are conducted via video conferencing or live streaming. In synchronous learning, there is typically a set schedule for classes and a designated instructor who leads the discussion and provides feedback.

Asynchronous learning, on the other hand, refers to educational activities that do not take place in real-time. This type of learning is often self-paced and flexible, allowing students to complete coursework at their own pace and on their own schedule. Asynchronous learning can include activities such as online courses, pre-recorded lectures, and other self-guided learning materials.

There are benefits and drawbacks to both synchronous and asynchronous learning. Synchronous learning allows for immediate interaction and feedback, which can be beneficial for some students. Asynchronous learning, on the other hand, allows for greater flexibility and can be a good option for students who have busy schedules or prefer to learn at their own pace. Ultimately, the choice between synchronous and asynchronous learning will depend on the needs and preferences of the individual student and the goals of the educational program.

My response

Damned good! I didn’t even ask for advantages or disadvantages and it gave them. It was accurate, and more concise than what I wrote in Chapter 7, Section 4 of Teaching in a Digital Age. However, Teaching in a Digital Age gives more examples or instances and it was more detailed. I would have given a student who submitted ChatGPT (without my knowing) 8/10 for this answer.

ChatGPT provided references that I was unaware of, but they are impeccable sources. I was disappointed that my book wasn’t included in the references, but the discussion of synchronous-asynchronous is just a small part of the overall book, although the section can be specifically referenced. 

Teaching chemistry online

My question

What are the limitations of teaching chemistry online? Give references

ChatGPT

There are several limitations to teaching chemistry online that have been identified in the literature:

  1. Lack of hands-on experiences: One of the most significant limitations of teaching chemistry online is the inability to provide hands-on experiences to students (Barger, 2014). Many chemistry concepts and skills require hands-on practice, which can be difficult to replicate in an online setting (Celik et al., 2015).
  2. Limited access to lab equipment: Chemistry labs often require specialized equipment and materials that may not be readily available to students in an online setting (Mehta et al., 2016). This can make it challenging for students to fully engage with and understand certain concepts and techniques (Celik et al., 2015).
  3. Difficulty in building a sense of community: In-person classrooms often provide an opportunity for students to interact and build a sense of community (Barger, 2014). This can be more challenging in an online setting, where students may feel isolated and disconnected from their peers and instructors (Mehta et al., 2016).
  4. Technical difficulties: Online learning can be disrupted by technical issues such as internet connectivity problems or difficulties with online platforms and tools (Barger, 2014). These issues can be frustrating for both students and instructors and can impact the effectiveness of the learning experience (Mehta et al., 2016).
  5. Limited access to one-on-one support: In an online setting, students may have limited access to one-on-one support from instructors or teaching assistants, which can be beneficial for those who need additional help or clarification on certain concepts (Celik et al., 2015).

References:

  • Barger, M. D. (2014). The impact of online chemistry courses on student success. Journal of College Science Teaching, 43(4), 16-21.
  • Celik, S., Kavak, N., & Kavak, M. (2015). An investigation of the effectiveness of online chemistry laboratory courses. The Journal of Educational Research, 108(2), 91-99.
  • Mehta, R., Kaur, G., & Singh, J. (2016). A comparative study of online and face-to-face teaching of chemistry. Asian Journal of Education and Training, 2(4), 218-224.

My response

OK, but nothing I could not have ‘guessed’ myself, although the references were new to me. Some of the conclusions I would have challenged, such as the limited access to one-on-one support which is not a specific condition of online learning. I would also have liked some reference to ways in which online learning can replicate lab work such as the use of virtual reality. I would have given a student 5/10 for this response. However, I could have phrased the question better, perhaps. 

Video

My question

What are the affordances of video in teaching? Give references

ChatGPT

Video can be an effective teaching tool because it has several affordances that can enhance the learning experience. Some of the affordances of video in teaching include:

    Engagement: Video can be engaging and can capture the attention of learners, particularly if it is well-produced and includes interesting visuals and audio.

    Accessibility: Video can be accessed from anywhere with an internet connection and can be watched at any time, making it convenient for learners who may have busy schedules or who are learning remotely.

    Flexibility: Video can be used in a variety of teaching contexts, including in-person and online classes, and can be used to teach a wide range of subjects.

    Visual learning: Video can be particularly effective for visual learners, as it allows learners to see and hear information, rather than just reading or listening to it.

    Demonstration: Video can be used to demonstrate concepts and processes, which can be particularly useful in fields such as science and technology where hands-on experience may not always be possible.

    Review: Video can be used as a tool for review, allowing learners to revisit material and reinforce their understanding.

References:

    “The Role of Video in Teaching and Learning.” (n.d.). Retrieved from https://www.edutopia.org/article/role-video-teaching-learning

    “The Benefits of Using Video in Education.” (n.d.). Retrieved from https://www.teachthought.com/learning/the-benefits-of-using-video-in-education/

    “Using Video in Education: Benefits and Best Practices.” (n.d.). Retrieved from https://www.commons.ucalgary.ca/using-video-in-education/

I also asked the question a second time, but asking for examples instead of references. Here are the examples it gave:

Here are some examples of how video can be used in teaching:

  • Flipped classroom: Video lectures can be used as a way to introduce new material to students outside of class, freeing up class time for more interactive activities such as discussion or problem-solving.
  • Demonstrations: Video can be used to demonstrate concepts or skills, such as a science experiment or a physical therapy exercise.
  • Virtual field trips: Video can be used to take students on virtual field trips or to visit locations or events that may be difficult to access in person.
  • Group discussions: Video can be used to facilitate group discussions, such as through video conferencing or recorded video responses.
  • Feedback: Video can be used to provide feedback on student work, such as through recorded video comments or critiques.

My response

First, I was impressed ChatGPT recognised the specialist term ‘affordances’ and correctly interpreted it. It gave six affordances. I would not disagree with these but they were fairly general. There are also many more affordances of video, which is why I asked for examples. Again, the ChatGPT examples are good ones, but again there are many more. Once again I was disappointed there was no reference to Teaching in a Digital Age, especially as Chapter 8.4 is a much more thorough treatment of the affordances of video. I would have given a student 7/10 for this response.

Conclusions

  1. Impressive, and not just in the syntax, which was flawless in all examples. There were no noticeable errors. Even some results I would challenge were nevertheless reasonable to include.
  2. I was pleased that my approach to the chosen topics was independently confirmed by other sources, some of which I was not aware of.
  3. I can understand why ChatGPT is scaring the hell out of a lot of instructors. It will provide at least adequate responses to many questions that instructors might ask, without the students having to do any of the work.
  4. If I was setting a question as an assignment, I would try it out first on ChatGPT, then see if I could improve the question. 
  5. However, what ChatGPT does very well is to summarise briefly and accurately particular topics. A great exercise would be to give students ChatGPT’s answers to a question, then ask the students to improve on it, either by giving more or better examples, or by suggesting alternative responses. Of course, ChatGPT may get better as well….
  6. The danger of ChatGPT being so good in general is: what if it does get something really wrong, or provides responses based on biased or hateful sources? I guess that is now another responsibility for experts, to keep an eye on it and point out or correct obvious errors. In particular, its reputation will depend on who manages ChatGPT and how – Wikipedia springs to mind.
  7. Still disappointed I didn’t get a reference to topics in which I feel I am an expert!

 

RELATED ARTICLESMORE FROM AUTHOR

13 COMMENTS

  2. Please notice that ChatGPT is a generative Language AI, so there is no guarantee that what it tells you is correct. it is not a search engine, simply it can construct fake references that even don’t exist.
    for example, if you asked, tell me a bible sentence that talk about bla bla, and give something that is never happened in the bible, it will construct a bible like text for what you asked.
    I did it with Islamic Quran and it constructed a Quran text that does not exist.
    and I have more examples for that.

  3. Hi Tony
    ChatGPT will transform personal learning networking and be used creatively by learners. ChatGPT will improve with more users using/training the large language model with feedback. If you repeat your questions in a new chat, the response changes and refines your question. The training data is known and predictable so original ideas will be scarce.
    It will also change how learners are assessed. For example, the use of multi-stage assessments on a particular theme and context, group work, oral presentations with a Q&A, games-based assessment and traditional invigilated exams will make a strong comeback for finals.
    I’d like to see studies on ChatGPT’s biases. This a particularly difficult task without the help of the ChatGPT designers’ input. Being conscious of your own biases is a progressive and evolving experience that is aided by reflection, teachers, reviewers and collaborators. I wonder what ChatGPT uses.
    Interestingly, a system that mimics human intelligence encourages one to reflect on how one’s own intelligence functions.
    Robert

  4. I am afraid you have not completed the analysis; you also should have checked the references The 4 references offered in response to your second query (Teaching chemistry online) are fake. I have experienced the same situation (AI hallucination) in each of my tests (see https://funredes.org/lc/ChatGPT for a short example of that worrying situation). If I search for the titles given in the 4 references the only result is your page! The journal exists, the authors exist but the cited article do not: they are pure IA inventions.

    • Thanks, Daniel, and all the others who have responded to this article.

      Yes, you’re right, I should have checked the urls for the references – bad Tony!

      I was fooled because I know that UNC Chapel Hill has great resources on online learning. Indeed, the correct url is: https://keepteaching.unc.edu/modes-of-teaching-implementation-guide-modes-3-4-or-5-remote-only. Similarly, UC Sacramento has this web site: https://www.csus.edu/student-affairs/course-modality.html

      It’s puzzling why ChatGPT did not provide these urls as they are obviously the source. This seems more like a minor glitch in the programming that can be rectified than evidence that ChatGPT is generally unreliable.

      • As a matter of fact, I did exactly like you, testing ChatGPT in my field of expertise and was, like you, quite impressed (I even quoted the ability to balance subtly conflicting arguments as “artificial wisdom”). I asked for references also and was extremely surprised to discover competent researchers working in my field with such motivating papers that I tried immediately to read them with absolutely no suspicion! It seems that this (quite unethical) behavior of inventing credible references with existing authors and publication support is systematic and intrinsic part of its way of performing. I hope they will correct that in next release because it is a terrible drawback… especially in education context.

  5. Ha! Ive been playing around with it too but amusingly asking it questions about what I should do with my life; what goals I should achieve in the years before formal retirement; even getting it to create a storyline of my life as a book/novel outline to see how it could transpire. Force it to come with creative responses. Alas, it’s a bit dry with what it comes back to me. It’s missing a ‘je ne sais quoi’, a certain pizazz with the answering. To be honest, I’m disappointed. But not so much. Ha ha!!

    I’m blogging my findings over at my own blog at Activate Learning Solutions. Now what shall I ask it next?

    • I tried to challenge its false reference’s answer stating they were false. First it apologizes for a possible error but confirm they were correct. Second when I insisted asking “In what book or Journal did you found those references?” I got that funny answer : “I cannot tell since they do no exist.”… and in sequence it provides another set of false references. Finally I answered that all references provided where systematically invented and asked “Are you having hallucination?” 🙂 it answered: “Sorry for providing non existing references. It was not my intention to bring confusion or provide incorrect information”. That dialog was in Spanish and i am translating the exchanges.

  8. As a corollary to my response above, I asked ChatGPT a couple of questions about cheating on University assessments using a clever AI like ChatGPT. Read on…

    Topic: Prevent Cheating On Assessments

    roblespar@gmail.com asked…
    Imagine you are an educational developer and you need to solve a pedagogical problem of cheating on university assessments using ChatGPT. What assessments would you design to prevent cheating?

    ChatGPT answered…
    There are several ways to prevent cheating on university assessments using ChatGPT. One approach would be to design assessments that require students to apply their knowledge in a novel or creative way, rather than simply recalling information. For example, you could design assessments that require students to solve a problem, analyze a case study, or create a project. These types of assessments are less susceptible to cheating because they require students to demonstrate their understanding and skills in a way that is difficult to replicate.

    Another approach would be to use proctoring software or online invigilation during the assessment. This can help to prevent cheating by monitoring students’ behavior during the assessment and flagging any suspicious activity.

    Additionally, you could use randomized question sets or randomized answer options to prevent students from sharing answers.

    Finally, you could also use plagiarism detection software to check students’ work for signs of academic dishonesty, such as copied text from other sources.

    In summary, to prevent cheating on university assessments using ChatGPT, you can design assessments that require students to apply their knowledge in a novel or creative way, use proctoring software or online invigilation during the assessment, randomized question sets or randomized answer options and plagiarism detection software to check students’ work for signs of academic dishonesty.

    roblespar@gmail.com asked…
    This answer did not address specifically the use of ChatGPT to cheat on University assessments. Can you please have another response but this time looking at cheating using a clever AI like ChatGPT?

    ChatGPT answered…
    Certainly, using a language model like ChatGPT to cheat on university assessments is a potential concern. To prevent this type of cheating, one approach would be to design assessments that are not easily solvable by a language model. For example, you could use assessments that require students to demonstrate their understanding of complex concepts, apply their knowledge in a real-world context, or perform tasks that involve critical thinking and creativity. These types of assessments would be more difficult for a language model to replicate, as they require a deeper understanding of the material and the ability to apply it in a unique way.

    Another approach would be to use measures that specifically target language models. For example, you could use anti-cheating algorithms to detect and prevent the use of automated scripts or language models during assessments. Additionally, you could use text-based CAPTCHAs to confirm that a human is completing the assessment.

    You could also use a combination of multiple-choice questions and open-ended questions to make it harder for a language model to cheat.

    Another approach would be to use assessments that require students to apply their knowledge in a real-world context and to interact with other students. These types of assessments would be more difficult for a language model to replicate, as they require a deeper understanding of the material and the ability to apply it in a unique way.

    In summary, to prevent cheating on university assessments using a language model like ChatGPT, you can design assessments that are not easily solvable by a language model, use measures that specifically target language models, and use a combination of multiple-choice questions and open-ended questions and assessments that require students to apply their knowledge in a real-world context and to interact with other students.
    —
    I was not satisfied by the responses. The exisiting issue of cheating is widespread and will require assessment redesign.

    • Great piece of research, Robert.
      The problem is that cheating will always be with us – it goes back at least to the Chinese Imperial exams in the sixth century AD. It is a continuous cat and mouse game, both for in-person and online teaching.
      I don’t disagree with the recommendations to avoid cheating given by ChatGPT – I am wondering what would satisfy you?
      I would have added having discussions with students about cheating, and the consequences not only immediately (such as suspension if caught) but long-term, in not having the knowledge they will need in future.
      The other problem is that the concept of cheating is shifting – for instance, why not look up stuff if you need it in a hurry? Is that cheating? Or should we be encouraging the ability to search for information and verifying it, with perhaps references?

  9. Look up some of the articles that ChatGPT cited. The authors exist. The journals exist. The articles may exist. But not by those authors in those journals in those years. They look like citations, but aren’t.

LEAVE A REPLY

Please enter your comment!
Please enter your name here