(28) Is Current Artificial Intelligence (AI) at the Same Level of an Oral and Maxillofacial Surgeon?

Abstract: Statement of the problem: The use of artificial intelligence (AI) has grown exponentially in the past years in the field of medicine. Currently, AI is being utilized in multiple medical fields for diverse tasks such as screening for diabetic retinopathy and skin cancer, estimating cardiovascular risks in specific populations, predicting Alzheimer progression, amongst others.1 In dentistry, AI software has been developed to aid in the diagnosis of caries, tooth wear, and radiograph evaluation.2 However, it is still being seldom used in the realm of oral and maxillofacial surgery. It is important to explore the potential of new technologies which could aid in the treatment planning of patients in our field. Materials and methods: The study design was a multi-center retrospective cohort study (10 years). The University of Puerto Rico research department granted an approval status after the institutional review protocol (#2401177157). The study population (n = 100) was composed of patients which underwent orthognathic surgery at the selected institutions. The purpose of this investigation was to determine if public access AI software ChatGPT®(4.0) has the ability to diagnose dentofacial deformities and to propose treatment that satisfies the current surgical treatment standards. The specific aims were to formulate questions whose objectives were to: 1. Determine if the software can interpret anatomic and orthodontic parameters to diagnose dentofacial deformities. 2. Determine if it can interpret these parameters and can propose an adequate surgical treatment plan. 3. Determine if it can create a specific surgical treatment plan providing angles and measurements (Figure 1). The authors hypothesized that ChatGPT®(4.0) has the potential to diagnose and propose treatment plans equivalent to the current treatment standards with minimal error rates (Figure 2).Methods of data analysis: Descriptive statistics were performed to analyze population and results. Kruskal-Wallis test was used to determine if the accuracy of the three groups of answers were statistically significant (P < .05).

Results: Among the three questions asked to ChatGPT®(4.0), the most accurate was Q1 which assessed its diagnostic capabilities (84.7%a), Q2 assessed if it can determine which type of surgery the patient required (90.5%), and Q3 assessed if it can provide an specific surgical plan including angulation and milimeters (63.6%). Kruskal-Willis test demonstrated the accuracy between the three groups of answers was statistically significant (0.001) (Figure 2). Outcomes data: Among the three questions, the software was more proficient in Q2 (treatment plan). It was the least accurate in Q3 (CASP). Q1 and Q3 requires the software to analyze raw data input, then through its own algorithms provide the diagnosis and CASP. For Q2, the software uses information available on the web, facilitating an adequate surgery type proposal (Le Fort, BSSO, etc.), since it is information that is readily available on the internet. For Q3 (CASP), it is more difficult to provide an accurate response since it must interpret by itself the provided parameters. Of note, results were not influenced by the previous question responses. In other words, ChatGPT provided solely a response based on the information provided in the question itself. Of note, the software experienced issues for analyzing the data of Class III patients over Class II, the reason is uknown (Figure 4).

Conclusions: Through this investigation we conclude that current public access AI technology has a fair reasoning level to diagnose and propose treatment for dentofacial deformities. References1. 2. Kaul, V., Enslin, S., & Gross, S. A. (2020). The history of artificial intelligence in medicine. Gastrointestinal Endoscopy. doi:10.1016/j.gie.2020.06.040 2. Ding H, Wu J, Zhao W, Matinlinna JP, Burrow MF and Tsoi JKH (2023) Artificial intelligence in dentistry—A review. Front. Dent. Med 4:1085251.

Figure 1. Investigation workflow explaining the material and methods of the investigation (created by ChatGPT® (4.0)).
Figure 3. Overall accuracy of ChatGPT® in the three prompts (4.0).
Figure 4. Error distribution according to different occlusion types.