Effective assessment is a nuanced art. As educators we endeavour to create assessment that adequately evaluates student understanding, provides the opportunity for deep learning and reflection and enables constructive feedback. We also aim to assess in a range of ways to account for different ways of learning and in a manner that is equitable and fair, all while remaining relevant and in-line with the latest pedagogies and technologies. This ever-evolving landscape of assessment best-practice has meant the days of handwritten essays are all but over, and there is growing popularity for exams to be delivered online.
Now, of course, educators recognise the opportunity for misconduct in the online space. Historically, to combat this we have equipped ourselves with a range of tools, such as Turnitin, which detect and flag plagiarism. We have also adopted novel approaches to exams such as randomised questions, assigning students individual problems to work on, and posing case studies and theoretical scenarios. These tools, although not completely infallible, have largely preserved the integrity of assessments.
But this, the push-pull relationship between assessment and misconduct, is an evolutionary arms race.
So what do we do when little Alice suddenly learns to run a whole lot faster?
Chat GPT
Over the last couple of months, you may have heard of ChatGPT. Released in November 2022 by Open AI, ChatGPT is a large language model chatbot. It interacts with the user in a conversational way, with the ability to remember conversations, provide follow-up answers, challenge incorrect premises, and correct itself. It is currently completely free and open to everyone.
Alice just got a jet-pack booster upgrade.
This following text is written entirely by AI.
ChatGPT has the potential to revolutionise the way university students approach writing essays and other forms of assessment.
ChatGPT makes is easier for students to access and use writing resources. With the ability to understand and respond to natural language, ChatGPT can be programmed to provide students with information on specific topics, including sources of research and examples of well-written essays.
ChatGPT can also provide students with instant feedback on their writing. With the ability to understand and analyze natural language, ChatGPT can quickly and accurately grade essays, providing students with detailed feedback on grammar, structure, and content.
ChatGPT can be used to help students study for exams in a few ways. The model can be used to provide students with information and resources on specific topics, including summaries of key concepts, examples, and explanations. ChatGPT can also be used to generate practice questions and answers, which can help students to test their understanding of the material and identify areas where they need further study.
Now, back to me. Pretty impressive right? Maybe a touch daunting. Shocking. Exciting. All of the above?
However you feel about this tool – and its output – it’s here to stay and there’s no point pretending our students won’t know about it or be using it to assist them in their studies. And why shouldn’t they? It’s a really great search engine – compiling lots and lots of data points in to information that’s accessible and easy to understand.
But even if we accept that everyone will be using these tools, as educators we still have an obligation to ensure that our graduates leave University with the necessary skills and tools required in their chosen careers, and not just with great skills in using AI and pressing copy paste.
So what can we do about it?
I asked ChatGPT, and according to AI, there are four ways we can better design assessment to assess the student, and not the AI chatbot:
- Use of open-ended or creative questions: One way to design university assessments so that students cannot use AI to help them is by using open-ended or creative questions that require original thought and personal interpretation. These types of questions cannot be easily answered by AI and require students to demonstrate their own understanding and knowledge of a subject.
- Use of proctored exams: Another way to prevent students from using AI to cheat on university assessments is by administering proctored exams. Proctored exams are typically administered in a controlled environment, such as a classroom or testing center, and are supervised by a teacher or proctor. This makes it much more difficult for students to use AI to cheat.
- Use of plagiarism detection software: A third way to prevent students from using AI to cheat on university assessments is by using plagiarism detection software. These software programs are designed to identify and flag instances of plagiarism in written work, making it much more difficult for students to use AI-generated content in their essays.
- Use of hands-on or practical assessments: Another way to design assessments so that students cannot use AI to help them is by using hands-on or practical assessments that require students to demonstrate a skill or apply their knowledge in a real-world setting. These types of assessments cannot be easily replicated by AI and require students to demonstrate their own abilities and understanding. For example, in engineering or programming classes, an assessment that requires students to build a robot or code a program, will be difficult to replicate by AI.
In addition to these four ways, it’s also important to have a robust academic integrity policy in place and communicate it clearly to students. It would also be a good idea to train the educators and proctors on the usage of such software and technology to detect cheating, so they are able to spot any suspicious activity or use of AI by the students.
It’s Nici again.
So using in-person exams and practical assessment is a way around the use of AI. ChatGPT suggests using anti-plagarism software, but I’ll note that I have run some AI generated content through Turnitin and it did not flag plagiarism issues. Under current practices, it’s unlikely that this tool alone would enable us to pick up the use of AI.
AI is constantly evolving, and it will become more advanced and intuitive in the future – that’s the whole point of it. As of January 2023, you might be wondering if it’s possible to spot the use of AI in your students work.
Chat GPT has some hints:
- Unusual patterns of language usage: AI-generated text often has patterns and characteristics that are different from those found in human-written text. For example, AI-generated text may have a higher frequency of certain words or phrases, or may have a more mechanical or formulaic structure. Educators can use software tools such as plagiarism checkers to detect these patterns and flag any suspicious text.
- Lack of critical thinking and originality: AI-generated text typically lacks the critical thinking and originality that is expected in student assessments. Educators can look for signs of this, such as a lack of personal perspective or original ideas, in the student’s work.
- Inconsistency in the work: AI-generated text may have inconsistencies in terms of tone, style, or level of detail. Educators can compare different parts of the student’s work to look for any inconsistencies that may indicate AI use.
- Inconsistency in the writing style: Educators can compare the student’s work over time to look for changes in writing style that could indicate the use of AI.
- Lack of understanding of the subject matter: Educators can also ask students to explain or elaborate on their work to see if they have a deep understanding of the subject matter, which is unlikely if the work is generated by AI.
It’s important to note that these methods are not foolproof, and the use of AI may not always be obvious. However, using a combination of these methods can help educators to detect and address any instances of AI use in student assessments.
As educators, we will need to say informed of the advancements in AI and adapt our assessment methods accordingly. If we want to assess our students understanding and provide constructive feedback that ensures our graduates excel in life outside university, we need to be creative and likely step outside our own comfort zones.
If you have been looking for a reason to re-design your assessment ahead of semester kicking off, this might just be it. You might even want to use ChatGPT for some inspiration.
Run Alice, run.
February 2023
Dr Nici Sweaney is a lecturer, Honours and Masters Coursework Convener in the Fenner School of Environment and Society, ANU College of Science.