Is the genie out of the bottle? Can you trust ChatGPT in scientific writing?

Since its launch in November 2022, ChatGPT, a state-of-the-art language model developed by OpenAI, L.L.C. in San Francisco, CA, U.S., has garnered the attention of the scientific community. As debates about ChatGPT’s impact on every aspect of academia surge worldwide, a team from the University of Veterinary Medicine Hannover, Germany, lead a project with multiple international partner to explore the potential limitations and advantages associated with the application of ChatGPT in scientific writing.

This study, entitled “ChatGPT and Scientific Papers in Veterinary Neurology: Is the Genie Out of the Bottle?” published in Frontiers in Veterinary Science, Section Veterinary Neurology and Neurosurgery, investigates the capability of ChatGPT in generating scientific papers within the field of veterinary neurology.

Abstracts and introduction sections, including references, for three different original research papers and ChatGPT-generated papers in veterinary neurology were evaluated using AI output detectors and plagiarism detectors. Thirteen Board-Certified neurologists were also asked to determine the originality and trustworthiness of the content.

“We are thrilled that this work published at the beginning of the academic year. Like any other technology, an AI tool like ChatGPT can either pose a threat to scientific integrity and transparency or assist researchers, depending on how they are used. I strongly recommend the integration of education on both the proper use and potential misuse of AI-based tools in academia as a fundamental aspect of good scientific practice,” Samira Abani, the first author from the University of Veterinary Medicine Hannover, says.

Jasmin Neßler, the corresponding author from the University of Veterinary Medicine Hannover, comments, “We believe that the popularity of AI requires interdisciplinary scientific collaboration to establish clear guidelines for its responsible use, ensuring integrity and transparency in published literature. Banning AI tools may not always be the most effective approach to preventing misuse; instead, we should embrace this opportunity to harness AI’s potential for the benefit of society.”

Prof. Holger Volk, head of the Clinic for Small Animals at the University of Veterinary Medicine Hannover, adds, “We learned that, Large Language Models (LLMs) like ChatGPT are trained to mimic human writing styles and produce coherent, plausible-sounding texts, which can sometimes trick human experts. However, models like ChatGPT are prone to hallucination, and they may generate biased content.”

“This raises concerns regarding the integrity of utilizing such models in academic writing—we must remember that, despite recent advances, we should be skeptical about completely relying on ChatGPT or any AI tools, at least for now.”

Provided by
Stiftung Tierärztliche Hochschule Hannover