ChatGPT vs Dermatology Residents on Exam Questions

TAMPA, FL, UNITED STATES, September 17, 2024 /EINPresswire.com/ -- ChatGPT is a generative artificial intelligence with numerous professional applications. Although applications in medical education are currently being explored, ChatGPT 4.0 performance on image-based dermatology boards-style practice questions has not been assessed.

In this new study in SKIN: The Journal of Cutaneous Medicine^®, Logan R Smith and his coauthors determine the accuracy with which ChatGPT can answer dermatology boards examination practice questions. They evaluated the results of 300 multiple-choice questions from DermQBank into ChatGPT, where 169 were text-only and 133 had associated images. Their results show ChatGPT answered 77.3% of dermatology board examination practice questions correctly, but performed worse than over half of the dermatologists-in-training. but performed significantly better with text-only questions than with questions that included images.

ChatGPT applications are still being investigated and the risk is that it may present fabricated information as fact; this concept has been termed “hallucination". This study found that ChatGPT chose an incorrect answer 22.7% of the time.

This is important for highlighting discretion in the accuracy of AI approaches like ChatGPT as an educational resource and should be utilized with caution.

SKIN: The Journal of Cutaneous Medicine^® is a peer-reviewed online medical journal that is the official journal of The National Society for Cutaneous Medicine. The mission of SKIN is to provide an enhanced and accelerated route to disseminate new dermatologic knowledge for all aspects of cutaneous disease.

For more details, please visit www.jofskin.org or contact jofskin@gmail.com.

Smith, L., Hanna, R., Hatch, L. and Hanna, K. 2024. Computer Vision Meets Large Language Models: Performance of ChatGPT 4.0 on Dermatology Boards-Style Practice Questions. SKIN The Journal of Cutaneous Medicine. 8, 5 (Sep. 2024), 1815–1821. DOI:https://doi.org/10.25251/skin.8.5.5.

Logan R Smith
University of South Florida Morsani College of Medicine
Lrsmith1@usf.edu