Healthcare (Commonwealth Union) – A new artificial intelligence (AI) system could significantly simplify — and reduce the cost of — training medical imaging software, even when only a limited number of patient scans are available.
This AI innovation enhances a technique known as medical image segmentation, where each pixel in a scan is identified based on what it represents, such as healthy or cancerous tissue. Traditionally, this detailed task is carried out by highly skilled professionals, though deep learning technology has shown potential to automate the process.
However, one of the major hurdles is that deep learning models typically require vast amounts of precisely labeled pixel-by-pixel annotated image data to function effectively, according to Li Zhang, a Ph.D. candidate in the Department of Electrical and Computer Engineering at the University of California San Diego. Producing these datasets is time-consuming, expensive, and heavily reliant on expert input — a luxury that many medical conditions and clinical environments simply don’t have.
To address this issue, Zhang and a research team led by UC San Diego professor Pengtao Xie have created an AI solution capable of learning segmentation tasks using only a small set of annotated examples. This reduces the typical data requirement by up to 20 times and could pave the way for quicker, more cost-effective diagnostic tools, particularly in healthcare settings with constrained resources.
“This project was born from the need to break this bottleneck and make powerful segmentation tools more practical and accessible, especially for scenarios where data are scarce,” explained Zhang, who is the first author of the study.
The artificial intelligence tool was evaluated across a range of medical image segmentation challenges. It successfully learned to detect skin abnormalities in dermoscopy photos, identify breast tumors in ultrasound images, map placental vessels in fetoscopic visuals, locate polyps in colonoscopy footage, and recognize foot ulcers in standard photographic images — to name just a few examples. The approach was also adapted for use with 3D imaging, such as scans of the hippocampus or liver.
In scenarios where labeled data was scarce, the AI system improved performance by 10 to 20% over existing methods. It required only a fraction — between one-eighth and one-twentieth — of the training data typically needed by conventional techniques, all while often equaling or surpassing their results.
According to Zhang, the tool could be especially useful for dermatologists diagnosing skin cancer. Instead of needing to collect and annotate thousands of images, a specialist might only need to mark up around 40. The AI could then analyze dermoscopy images from patients in real time and flag potentially cancerous lesions. Zhang indicated that this could enable quicker, more precise diagnoses.
The system operates in multiple phases. Initially, it learns to produce synthetic images from segmentation masks—these are essentially color-coded maps that indicate which areas of an image correspond to healthy or diseased tissue. Applying this knowledge, the system then generates new, artificial image-mask combinations to supplement limits on sets of real-world samples. The segmentation model then learns on both the real and synthetic data. As the process goes further a feedback mechanism permits the system to fine-tune the images it generates based on how effectively they improve the model’s learning.
This feedback loop is a key factor in the system’s success, as indicated by Zhang. He further pointed out that instead of handling data generation and model training as separate tasks, their approach is the first to merge them into a single, unified process. The model’s segmentation findings directly influence the way the synthetic data are created. This means the generated images aren’t just visually convincing—they’re optimized to boost segmentation performance.
Looking to the future, the team aims to make the AI tool even more intelligent and adaptable. They also intend to integrate input from medical professionals into the training process, helping ensure the synthetic data are closely aligned with clinical needs.
