Skip to main navigation Skip to search Skip to main content

Assessing Open-world Foundation Models for Zero-shot Skin Segmentation in Clinical Dermatological Photographs

  • Yihao Liu*
  • , Andrew J. McNeil
  • , Bohan Jiang
  • , Michael E. Kim
  • , Inga Saknite
  • , Aaron T. Zhao
  • , Placide Mbala-Kingebeni
  • , Olivier Tshiani Mbaya
  • , Tyra Silaphet
  • , Rachel Weiss
  • , Lori E. Dodd
  • , Edward W. Cowen
  • , Steven Z. Pavletic
  • , Veronique Nussenblatt
  • , Benoit M. Dawant
  • , Bennett A. Landman
  • , Eric R. Tkaczyk*
  • *Corresponding author for this work
  • Vanderbilt University
  • Department of Veterans Affairs
  • National Institutes of Health
  • Institut National de Recherche Biomédicale
  • Leidos Inc

Research output: Chapter in Book/Report/Conference proceeding › Conference paper › Research › peer-review

Abstract

Skin segmentation from clinical photography is a crucial step in dermatological image analysis. However, the variability in skin tones, lighting conditions, anatomical regions, and the presence of additional objects introduces significant challenges. Due to these complexities, the segmentation process is often performed manually, as developing an algorithm capable of handling such diverse conditions is particularly difficult. Recently, open-world foundation models have emerged, offering the potential to generalize across diverse and unseen conditions. These models present a promising opportunity for dermatology. In this work, we adopt two such models-Grounding DINO and SAM 2-to construct a pipeline for zero-shot skin segmentation in dermatology. We evaluated our approach on two clinical skin photography datasets comprising 27,378 images. Based on a manual rating protocol, 77.1% of the segmentations were deemed acceptable, demonstrating robustness in handling real-world clinical photographs. Our results highlight the potential of open-world foundation models to address a challenging problem in dermatology with minimal human involvement.

Original languageEnglish
Title of host publicationMedical Imaging 2026
Subtitle of host publicationClinical and Biomedical Imaging
EditorsBarjor S. Gimi, Andrzej Krol
PublisherSPIE
ISBN (Electronic)9781510697959
DOIs
Publication statusPublished - 1 Apr 2026
EventMedical Imaging 2026: Clinical and Biomedical Imaging - Vancouver, Canada
Duration: 16 Feb 2026 → 20 Feb 2026

Publication series

NameProgress in Biomedical Optics and Imaging - Proceedings of SPIE
Volume13929
ISSN (Print)1605-7422
ISSN (Electronic)2410-9045

Conference

ConferenceMedical Imaging 2026: Clinical and Biomedical Imaging
Country/TerritoryCanada
CityVancouver
Period16/02/26 → 20/02/26

OECD Field of Science

  • 2.6 Medical Engineering

Keywords

  • Zero-shot segmentation
  • dermatology
  • foundation model
  • skin segmentation

Fingerprint

Dive into the research topics of 'Assessing Open-world Foundation Models for Zero-shot Skin Segmentation in Clinical Dermatological Photographs'. Together they form a unique fingerprint.

Cite this