Assessing the Applicability of Artificial Intelligence (AI) in Fabrication and Plagiarism of Scientific Literature in the Field of Orthodontics
Objectives: The study aims to evaluate the applicability of Chat Generative Pre-trained Transformer (ChatGPT-4) to fabricate an orthodontic research paper and assess the ability of human reviewers (clinicians and academic reviewers) and artificial intelligence (AI) detection tools to identify such fabricated content.
Methods: This study employed a descriptive exploratory research design. ChatGPT-4 was used to generate a research paper on the orthodontic topic. A search strategy was used to extract relevant research studies from three databases, then rewritten using ChatGPT-4. A panel of 10 orthodontic experts were given a mix of AI-rewritten and human-written abstracts to identify fake and real ones. Additionally, AI detection tools were tested for their efficiency in identifying AI-generated content. The results were analyzed using descriptive statistics and comparative percentages.
Results: Orthodontic experts and researchers correctly identified only 68% of AIgenerated abstracts as fake, while 86% of human-written abstracts were recognized as real. Online AI detection tools demonstrated weak performance, with overall sensitivity and specificity of just 89% and accuracy of 44.5%. GPTZero had 100% specificity and 100% predictive values, along with the highest sensitivity and accuracy compared to others. However, writers AI demonstrates the poorest performance, indicating 0 predictive values in identifying AI-fabricated research works.
Conclusions: Reviewers experienced difficulties in distinguishing between AI-generated abstracts and those written by humans due to the substantial resemblance to real papers. To safeguard scientific integrity, it is crucial to use measures such as improving detection tools, refining peer review, and creating a transparent culture around AI use.
1. V Allareddy, S Frazier-Bowers, JH Park, GH Gilbert. Relevance of practice-based research to orthodontics. Angle Orthod. 2021;91(6):856-7.
2. Pacha MM, Fleming PS, Johal A. A comparison of the efficacy of fixed versus removable functional appliances in children with class II malocclusion: a systematic review. Eur J Orthod. 2016;38(6):621-30.
3. Fiorillo L, Mehta V. Accelerating editorial processes in scientific journals: Leveraging AI for rapid manuscript review. Oral Oncology Reports. 2024;10:100511.
4. Bhargava DC, Jadav D, Meshram VP, Kanchan T. ChatGPT in medical research: challenging time ahead. Med Leg J. 2023;91(4):223-5.
5. Huang J, Tan M. The role of ChatGPT in scientific communication: writing better scientific review articles. Am J Cancer Res. 2023;13(4):1148–54.
6. Biswas SS. ChatGPT for research and publication: a stepby-step guide. J Pediatr Pharmacol Ther. 2023;28(6): 576-84.
7. Alfarraj YF, Wardat Y. Exploring the impact of ChatGPT on scientific research: assessing strengths, weaknesses, opportunities, and threats. EaC. 2024;28:1-27.
8. Ahn S. The transformative impact of large language models on medical writing and publishing: current applications, challenges and future directions. Korean J Physiol Pharmacol. 2024;28(5):393-401.
9. Miao J, Thongprayoon C, Suppadungsuk S, Garcia Valencia OA, Qureshi F, Cheungpasitporn W. Ethical dilemmas in using AI for academic writing and an example framework for peer review in nephrology academia: a narrative review. Clin Pract. 2023;14(1):89-105.
10. Temsah MH, Altamimi I, Jamal A, Alhasan K, Al-Eyadhy A. ChatGPT surpasses 1000 publications on pubmed: envisioning the road ahead. Cureus. 2023;15(9):e44769.
11. Elali FR, Rachid LN. AI-generated research paper fabrication and plagiarism in the scientific community. Patterns (N Y). 2023;4(3):100706.
12. Safrai M, Orwig KE. Utilizing artificial intelligence in academic writing: an in-depth evaluation of a scientific review on fertility preservation written by ChatGPT-4. J Assist Reprod Genet. 2024;41(7):1871-80.
13. Alkaissi H, McFarlane SI. Artificial hallucinations in ChatGPT: implications in scientific writing. Cureus. 2023;15(2):e35179.
14. Caprioglio A, Paglia L. Fake academic writing: ethics during chatbot era. Eur J Paediatr Dent. 2023;24(2):88-9.
15. Gu J, Wang X, Li C, Zhao J, Fu W, Liang G, et al. AIenabled image fraud in scientific publications. Patterns (N Y). 2022;3(7):100511.
16. Dash KS, Mehta V, Kharat P. We are entering a new era of problems: AI-generated images in research manuscripts. Oral Oncology Reports. 2024;10:100289.
17. Mehta V, Thomas V, Mathur A. AI-dependency in scientific writing. Oral Oncology Reports. 2024;10:100269.
18. Else H. Abstracts written by ChatGPT fool scientists. Nature. 2023;613(7944):423.
19. Nazarovets S. Dealing with Research Paper Mills, Tortured Phrases, and Data Fabrication and Falsification in Scientific Papers [Internet]. In: Joshi PB, Churi PP, Pandey M, editors. Scientific Publishing Ecosystem. Singapore: Springer Nature Singapore; 2024 [cited 2025 Mar 19]. p. 233-54. Available from: https://link.springer.com/10.1007/978-981-97-4060-...
20. Haider J, Söderström KR, Ekström B, Rödl M. GPTfabricated scientific papers on Google Scholar: Key features, spread, and implications for preempting evidence manipulation. HKS Misinfo Review. 2024;5(5). doi.org/10.37016/mr-2020-156.
21. Ray PP. ChatGPT: A comprehensive review on background, applications, key challenges, bias, ethics, limitations and future scope. Internet Things Cyber Phys Syst. 2023;3: 121-54.
22. Gao CA, Howard FM, Markov NS, Dyer EC, Ramesh S, Luo Y, et al. Comparing scientific abstracts generated by ChatGPT to real abstracts with detectors and blinded human reviewers. NPJ Digit Med. 2023;6(1):75.
23. Tan TF, Thirunavukarasu AJ, Campbell JP, Keane PA, Pasquale LR, Abramoff MD, et al. Generative artificial intelligence through ChatGPT and other large language models in ophthalmology: clinical applications and challenges. Ophthalmol Sci. 2023;3(4):100394.
24. Flanagin A, Bibbins-Domingo K, Berkwits M, Christiansen SL. Nonhuman “Authors” and implications for the integrity of scientific publication and medical knowledge. JAMA 2023;329(8):637.
25. Kim HJ, Yang JH, Chang DG, Lenke LG, Pizones J, Castelein R, et al. Assessing the Reproducibility of the structured abstracts generated by ChatGPT and bard compared to human-written abstracts in the field of spine surgery: comparative analysis. J Med Internet Res. 2024;26:e52001.
26. Ariyaratne S, Iyengar KP, Nischal N, Chitti Babu N, Botchu R. A comparison of ChatGPT-generated articles with human-written articles. Skeletal Radiol. 2023;52(9):1755-8.
27. Lingard L. Writing with ChatGPT: an Illustration of its capacity, limitations & implications for academic writers. Perspect Med Educ. 2023;12(1):261-70.
28. Jahani Yekta MM. The general intelligence of GPT–4, its knowledge diffusive and societal influences, and its governance. Meta Radiol. 2024;2(2):100078.
29. Kendall G, Teixeira Da Silva JA. Risks of abuse of large language models, like CHATGPT , in scientific publishing: authorship, predatory publishing, and paper mills. Learned Publ. 2024;37(1):55-62.
30. Hwang T, Aggarwal N, Khan PZ, Roberts T, Mahmood A, Griffiths MM, et al. Can ChatGPT assist authors with abstract writing in medical journals? evaluating the quality of scientific abstracts generated by ChatGPT and original
abstracts. PLoS One. 2024;19(2):e0297701.
31. Hatia A, Doldo T, Parrini S, Chisci E, Cipriani L, Montagna L, et al. Accuracy and completeness of ChatGPT-generated information on interceptive orthodontics: a multicenter collaborative study. J Clin Med. 2024;13(3):735.
32. Levin G, Pareja R, Viveros-Carreño D, Sanchez Diaz E, Yates EM, Zand B, et al. Association of reviewer experience with discriminating human-written versus ChatGPT-written abstracts. Int J Gynecol Cancer. 2024;34(5):669-74.
33. Bi AS. What’s important: the next academic-ChatGPT AI? J Bone Joint Surg Am. 2023;105(11):893-5.
34. Roumeliotis KI, Tselikas ND. ChatGPT and Open-AI models: a preliminary review. Future Internet. 2023;15(6):192.
35. Chakraborty S, Bedi AS, Zhu S, An B, Manocha D, Huang F. On the Possibilities of AI-Generated Text Detection [Internet]. 2023 [cited 2025 Mar 19];Available from: http://arxiv.org/abs/2304.04736
36. Ibrahim K. Using AI-based detectors to control AI-assisted plagiarism in ESL writing: “The Terminator Versus the Machines.” Lang Test Asia. 2023;13(1):46.
37. Kar SK, Bansal T, Modi S, Singh A. How sensitive are the free ai-detector tools in detecting AI-generated texts? a comparison of popular AI-detector tools. Indian J Psychol Med. 2025;47(3):275-278.
38. Odri GA, Ji Yun Yoon D. Detecting generative artificial intelligence in scientific articles: evasion techniques and implications for scientific integrity. Orthop Traumatol Surg Res. 2023;109(8):103706.
39. Charnley J. Surgery of the hip-joint: present and future developments. Br Med J. 1960;1(5176):821-6.
40. Ji Z, Lee N, Frieske R, Yu T, Su D, Xu Y, et al. Survey of hallucination in natural language generation. ACM Comput Surv. 2023;55(12):1-38.
41. Flitcroft MA, Sheriff SA, Wolfrath N, Maddula R, McConnell L, Xing Y, et al. Performance of artificial intelligence content detectors using human and artificial intelligence-generated scientific writing. Ann Surg Oncol.
2024;31(10):6387-93.
42. Weber-Wulff D, Anohina-Naumeca A, Bjelobaba S, Foltýnek T, Guerrero-Dib J, Popoola O, et al. Testing of detection tools for AI-generated text. Int J Educ Integr. 2023;19(1):26.
43. Liang W, Yuksekgonul M, Mao Y, Wu E, Zou J. GPT detectors are biased against non-native English writers [Internet]. 2023 [cited 2025 Jun 11]; Available from: http://arxiv.org/abs/2304.02819
44. Alawida M, Mejri S, Mehmood A, Chikhaoui B, Isaac Abiodun O. A comprehensive study of ChatGPT: advancements, limitations, and ethical considerations in natural language processing and cybersecurity. Information. 2023;14(8):462.