ADELAIDE (Australia), July 8: Since ChatGPT appeared almost three years ago, the impact of artificial intelligence (AI) technologies on learning has been widely debated. Are they handy tools for personalised education, or gateways to academic dishonesty?
Most importantly, there has been concern that using AI will lead to a widespread “dumbing down”, or decline in the ability to think critically. If students use AI tools too early, the argument goes, they may not develop basic skills for critical thinking and problem-solving.
Is that really the case? According to a recent study by scientists from MIT, it appears so. Using ChatGPT to help write essays, the researchers say, can lead to “cognitive debt” and a “likely decrease in learning skills”.
So what did the study find?
The difference between using AI and the brain alone
Over the course of four months, the MIT team asked 54 adults to write a series of three essays using either AI (ChatGPT), a search engine, or their own brains (“brain-only” group). The team measured cognitive engagement by examining electrical activity in the brain and through linguistic analysis of the essays.
The cognitive engagement of those who used AI was significantly lower than the other two groups. This group also had a harder time recalling quotes from their essays and felt a lower sense of ownership over them.
Interestingly, participants switched roles for a final, fourth essay (the brain-only group used AI and vice versa). The AI-to-brain group performed worse and had engagement that was only slightly better than the other group’s during their first session, far below the engagement of the brain-only group in their third session.
The authors claim this demonstrates how prolonged use of AI led to participants accumulating “cognitive debt”. When they finally had the opportunity to use their brains, they were unable to replicate the engagement or perform as well as the other two groups.
Cautiously, the authors note that only 18 participants (six per condition) completed the fourth, final session. Therefore, the findings are preliminary and require further testing.
Does this really show AI makes us stupider?
These results do not necessarily mean that students who used AI accumulated “cognitive debt”. In our view, the findings are due to the particular design of the study.
The change in neural connectivity of the brain-only group over the first three sessions was likely the result of becoming more familiar with the study task, a phenomenon known as the familiarisation effect. As study participants repeat the task, they become more familiar and efficient, and their cognitive strategy adapts accordingly.
When the AI group finally got to “use their brains”, they were only doing the task once. As a result, they were unable to match the other group’s experience. They achieved only slightly better engagement than the brain-only group during the first session.
To fully justify the researchers’ claims, the AI-to-brain participants would also need to complete three writing sessions without AI.
Similarly, the fact the brain-to-AI group used ChatGPT more productively and strategically is likely due to the nature of the fourth writing task, which required writing an essay on one of the previous three topics.
As writing without AI required more substantial engagement, they had a far better recall of what they had written in the past. Hence, they primarily used AI to search for new information and refine what they had previously written.
What are the implications of AI in assessment?
To understand the current situation with AI, we can look back to what happened when calculators first became available.
Back in the 1970s, their impact was regulated by making exams much harder. Instead of doing calculations by hand, students were expected to use calculators and spend their cognitive efforts on more complex tasks.
Effectively, the bar was significantly raised, which made students work equally hard (if not harder) than before calculators were available.
The challenge with AI is that, for the most part, educators have not raised the bar in a way that makes AI a necessary part of the process. Educators still require students to complete the same tasks and expect the same standard of work as they did five years ago.
In such situations, AI can indeed be detrimental. Students can for the most part offload critical engagement with learning to AI, which results in “metacognitive laziness”.
However, just like calculators, AI can and should help us accomplish tasks that were previously impossible – and still require significant engagement. For example, we might ask teaching students to use AI to produce a detailed lesson plan, which will then be evaluated for quality and pedagogical soundness in an oral examination.
In the MIT study, participants who used AI were producing the “same old” essays. They adjusted their engagement to deliver the standard of work expected of them.
The same would happen if students were asked to perform complex calculations with or without a calculator. The group doing calculations by hand would sweat, while those with calculators would barely blink an eye.
Learning how to use AI
Current and future generations need to be able to think critically and creatively and solve problems. However, AI is changing what these things mean.
Producing essays with pen and paper is no longer a demonstration of critical thinking ability, just as doing long division is no longer a demonstration of numeracy.
Knowing when, where and how to use AI is the key to long-term success and skill development. Prioritising which tasks can be offloaded to an AI to reduce cognitive debt is just as important as understanding which tasks require genuine creativity and critical thinking. (The Conversation) (PTI)