Evaluating skin tone scales for dermatologic dataset labeling: a prospective-comparative study.

Weir, Vanessa R; Li, Yingjoy; Gillis, Maura C; Kurtansky, Nicholas R; Salvador, Trina; Halpern, Allan C; Nelson, Kelly C; Lester, Jenna C et al. · NPJ Digit Med · 2025

prospective_cohort · Level II

Where this comes from

Abstract

Skin tone affects artificial intelligence (AI) performance in dermatology. While labeling datasets for skin tone could improve algorithm generalizability for detecting dermatologic malignancies, large-scale validation of skin tone assessments is lacking. This prospective observational study assessed reliability of subjective tools (Fitzpatrick Skin Type [FST], Monk Skin Tone [MST], Pantone SkinTone Guide) and an objective colorimeter for in-person and photography-based settings to evaluate utility for labeling dermoscopic datasets. Colorimetry (gold standard for color measurement) demonstrated high precision with in-person measurements. Of subjective scales, MST demonstrated slightly tighter clustering in the color space and high repeatability for in-person and photography-based assessments (latter varied by lighting). Dermoscopic image-extracted color values correlated poorly with colorimetry values. For subjective ratings, MST more effectively captured differences in AI melanoma classification scores than FST. Findings underscore that FST is not a proxy for skin tone; an important role remains for skin tone assessment to improve AI performance.