A recent study by researchers at the Massachusetts Institute of Technology (MIT) indicates that when diagnosing skin diseases based on images, doctors are less accurate when the patient has darker skin. The study involved over 1,000 dermatologists and general practitioners. The accuracy rate of dermatologists diagnosing correctly dropped from 38% on images of lighter skin to 34% on darker skin. Similar accuracy decrease was observed among general practitioners.
It was also revealed that the assistance of artificial intelligence (AI) algorithms could improve doctors’ diagnostic accuracy, although the improvement was more pronounced for patients with lighter skin tones. The study is the first to highlight physician diagnostic disparities across skin tones, possibly attributable to dermatology textbooks predominantly using images of lighter skin tones and doctors’ lack of experience treating patients with darker skin.
The study was led by Matt Groh, Ph.D., an assistant professor at the Northwestern University Kellogg School of Management, and Rosalind Picard, an MIT professor of media arts and sciences. The researchers compiled a total of 364 images representing 46 skin diseases across various skin tones, which were shown to the study participants. Despite specialist dermatologists having higher accuracy rates (38% compared to 19% for general practitioners), both groups saw about a four percentage point drop in accuracy when diagnosing skin conditions on darker skin.
The researchers also developed AI algorithms trained with 30,000 images, which achieved an accuracy rate of around 47%. A modified version of the algorithm artificially achieved an 84% success rate to assess the influence of model accuracy on doctors’ decision-making. Either AI algorithm used improved accuracy for both dermatologists (up to 60%) and general practitioners (up to 47%). Interestingly, doctors were more likely to accept AI suggestions after multiple correct answers and rarely incorporated incorrect AI suggestions.
While both AI-assisted dermatologists and general practitioners were more accurate, general practitioners showed more significant improvement on images of lighter skin than darker skin. Researchers concluded that the study should prompt medical schools and textbooks to incorporate more training on patients with darker skin while providing insights into deploying AI assistance programs in dermatology. The research was funded by the MIT Media Lab Consortium and the Harold Horowitz Student Research Fund.
