A study of over 15 million biomedical papers on PubMed revealed a significant increase in the use of AI-associated words such as “delve” and “showcasing,” suggesting that AI-assisted writing tools, particularly OpenAI’s ChatGPT, are becoming more prevalent in scientific research. The study, conducted by researchers from Northwestern University and the Hertie Institute for AI in Brain Health at the University of Tübingen, identified a sharp rise in word patterns linked to AI-generated writing in 2024. These patterns included both uncommon terms like “delves,” “underscores,” and “showcasing,” as well as more familiar words such as “potential,” “findings,” and “crucial.”
To quantify this trend, the researchers compared word frequencies in 2024 with baseline data from 2021 and 2022. They identified 454 words that are frequently overused by AI models, including “encapsulates,” “noteworthy,” “underscore,” “scrutinizing,” and “seamless.” However, experts caution that word frequency alone is not sufficient evidence of AI use. Stuart Geiger, assistant professor of communication at UC San Diego, noted that language evolves over time, and the increased use of certain words could be due to broader societal influences, including the popularity of AI tools like ChatGPT.
Geiger emphasized that detecting AI in writing is not just a technical challenge but also an ethical one. He pointed out that the only reliable way to detect the use of large language models (LLMs) is to monitor the writing process, which raises significant logistical, moral, and technical concerns. Geiger warned against making assumptions based on surface-level clues without understanding the full context. He suggested that the increased use of certain words might reflect a shift in what is considered good writing, influenced by exposure to AI-generated text.
As AI-generated text becomes more common, educators and researchers are turning to detection tools to identify its use. However, the effectiveness of these tools varies widely. In October 2024, a test of leading AI detection tools, including Grammarly, Quillbot, GPTZero, and ZeroGPT, yielded inconsistent results. For example, ZeroGPT claimed that the U.S. Declaration of Independence was 97.93% AI-generated, while GPTZero gave it just 10%. Geiger described many of these detection tools as “snake oil,” highlighting the need for more reliable methods.
The debate over AI writing tools echoes past discussions about the impact of technology on writing, such as spell check, Wikipedia, and CliffsNotes. These debates reflect deeper questions about the purpose of writing, authorship, and trust. People are concerned that relying on AI to generate text may compromise the originality and quality of research. However, Kathleen Perley, a professor of business at Rice University, argued that AI writing tools can help researchers overcome challenges such as language barriers or learning disabilities, thereby democratizing participation in formal research.
Perley noted that while AI writing often shows patterns, such as repeated structures or overused words, the key is whether it enhances research without compromising quality. She believes that AI-assisted writing can be beneficial, especially for non-native English speakers or those facing other challenges. Perley also highlighted the dilemma of people changing their writing style out of fear of being accused of using AI, noting that she has become more conscious of certain words that might be flagged as potentially AI-generated. Despite criticisms of this style for lacking personality, Perley sees AI-assisted writing as a tool that can broaden participation in research, making it more inclusive and accessible.
