Monday, February 26, 2024
HomeAi TechIs the genie out of the bottle? Can you trust ChatGPT in...

Is the genie out of the bottle? Can you trust ChatGPT in scientific writing?


Is the genie out of the bottle? can you trust ChatGPT in scientific writing?
Presents the performance of specialist reviewers in identifying ChatGPT-generated scientific manuscripts vs. original published manuscripts in three tests, each containing two stages; the initial assessment, referred to as abstract detection, and the final assessment, which includes the introduction with references. The reviewers’ assessments are represented using colors, with red indicating “Incorrect” (misidentification), green indicating “Correct” (accurate identification), and yellow indicating “Uncertain” (indicating uncertainty). Reviewers demonstrated a better performance in identifying ChatGPT-generated manuscripts in Test 2 (the highly familiar subject of steroid-responsive meningitis-arteritis in dogs) vs. Test 1 (SARS-CoV-2 detection dogs) and Test 3 (Cannabinoid receptors in canine epilepsy), which were referred to as less familiar and moderately familiar subjects. In the final assessment, reviewers demonstrated improved performance in three tests, indicating that the reviewers were able to refine their assessments and make more accurate conclusions by incorporating specific information and references. Credit: Frontiers in Veterinary Science (2023). DOI: 10.3389/fvets.2023.1272755

Since its launch in November 2022, ChatGPT, a state-of-the-art language model developed by OpenAI, L.L.C. in San Francisco, CA, U.S., has garnered the attention of the scientific community. As debates about ChatGPT’s impact on every aspect of academia surge worldwide, a team from the University of Veterinary Medicine Hannover, Germany, lead a project with multiple international partner to explore the potential limitations and advantages associated with the application of ChatGPT in scientific writing.

This study, entitled “ChatGPT and Scientific Papers in Veterinary Neurology: Is the Genie Out of the Bottle?” published in Frontiers in Veterinary Science, Section Veterinary Neurology and Neurosurgery, investigates the capability of ChatGPT in generating scientific papers within the field of veterinary neurology.

Abstracts and introduction sections, including references, for three different original research papers and ChatGPT-generated papers in veterinary neurology were evaluated using AI output detectors and plagiarism detectors. Thirteen Board-Certified neurologists were also asked to determine the originality and trustworthiness of the content.

“We are thrilled that this work published at the beginning of the academic year. Like any other technology, an AI tool like ChatGPT can either pose a threat to scientific integrity and transparency or assist researchers, depending on how they are used. I strongly recommend the integration of education on both the proper use and potential misuse of AI-based tools in academia as a fundamental aspect of good scientific practice,” Samira Abani, the first author from the University of Veterinary Medicine Hannover, says.

Jasmin Neßler, the corresponding author from the University of Veterinary Medicine Hannover, comments, “We believe that the popularity of AI requires interdisciplinary scientific collaboration to establish clear guidelines for its responsible use, ensuring integrity and transparency in published literature. Banning AI tools may not always be the most effective approach to preventing misuse; instead, we should embrace this opportunity to harness AI’s potential for the benefit of society.”

Prof. Holger Volk, head of the Clinic for Small Animals at the University of Veterinary Medicine Hannover, adds, “We learned that, Large Language Models (LLMs) like ChatGPT are trained to mimic human writing styles and produce coherent, plausible-sounding texts, which can sometimes trick human experts. However, models like ChatGPT are prone to hallucination, and they may generate biased content.”

“This raises concerns regarding the integrity of utilizing such models in academic writing—we must remember that, despite recent advances, we should be skeptical about completely relying on ChatGPT or any AI tools, at least for now.”

More information:
Samira Abani et al, ChatGPT and scientific papers in veterinary neurology; is the genie out of the bottle?, Frontiers in Veterinary Science (2023). DOI: 10.3389/fvets.2023.1272755

Provided by
Stiftung Tierärztliche Hochschule Hannover

Citation:
Is the genie out of the bottle? Can you trust ChatGPT in scientific writing? (2023, October 19)
retrieved 20 October 2023
from https://techxplore.com/news/2023-10-genie-bottle-chatgpt-scientific.html

This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no
part may be reproduced without the written permission. The content is provided for information purposes only.



Leah Sirama
Leah Sirama
Leah Sirama, a lifelong enthusiast of Artificial Intelligence, has been exploring technology and the digital realm since childhood. Known for his creative thinking, he's dedicated to improving AI experiences for all, making him a respected figure in the field. His passion, curiosity, and creativity drive advancements in the AI world.
RELATED ARTICLES

LEAVE A REPLY

Please enter your comment!
Please enter your name here

- Advertisment -

Most Popular