1. News
  2. AI models & agents
  3. AI · Skin health
AI · Skin health2 min read

A 10-year-old AI tool beat top chatbots at spotting skin cancer

A dermatologist tested two top AI chatbots on 200 biopsy-proven skin lesions. A small 2015-era image tool beat both, but none were reliable.

By the Dashline deskSource: Hacker News (100+ points)
A 10-year-old AI tool beat top chatbots at spotting skin cancer
Photo: Tdorante10, CC BY-SA 4.0

A dermatologist gave two of the most capable AI chatbots of October 2026 a skin cancer test, and a ten-year-old image tool did better. On 200 lesions confirmed by biopsy, the old tool named the right diagnosis 62% of the time. GPT-6 Astra and Claude Fable 5.1 each got 42%.

How the test worked

Dr Magnus Lynch drew 200 lesions at random from a public dataset called MILK10k. Each had two photos: a regular close-up and a dermoscopy image, taken through the lit magnifier skin doctors use. Every lesion was biopsied, even the harmless ones, so these were hard cases. Each was a spot that worried a clinician.

The chatbots saw only the two images. They had to pick one of six diagnoses, give a confidence level, and say whether the patient could go home with no treatment, biopsy or follow-up. Neither refused a case.

The old tool

The rival was Inception v3, a type of image-recognition software Google first published in 2015. Lynch retrained it on 68,011 skin images in about twenty minutes. The 200 test lesions, and any other photos of the same lesions or patients, were removed from its training data.

What the numbers showed

  • Melanoma, the most dangerous type: the old tool found 13 of 30, GPT-6 Astra 8, Claude Fable 5.1 11.
  • Moles: 37 of 50 for the old tool, 16 for GPT-6 Astra, 33 for Claude Fable 5.1.
  • Seborrhoeic keratosis, a harmless warty growth: 26 of 50, against 13 and 6.
  • Basal cell carcinoma: GPT-6 Astra got all 25, the old tool 24, Claude Fable 5.1 18.

The old tool was behind both chatbots on actinic keratosis, a rough sun-damage patch. When a second guess was allowed, it reached 173 of 200, against 134 and 138.

The warning sign

In one case, both chatbots called a lesion a harmless seborrhoeic keratosis and said the patient could be discharged. The old tool reached the right answer by using the dermoscopy image. In another, GPT-6 Astra called a thin melanoma a basal cell carcinoma with 95% confidence.

What it means for you

A general chatbot is not a safe way to judge a mole or skin spot. Confident answers can be wrong. Even the best tool here was wrong on 38% of lesions and missed 17 of 30 melanomas. A doctor's exam is still the right step for any spot that changes or worries you.

This is one doctor's test on one dataset. The old tool can be tried in a phone browser, and Lynch says nothing is uploaded. It is a research demo, not a medical device.

The takeaway

Do not use a general chatbot to judge a skin spot. Even the best tool in this test missed many cases, so see a doctor.

More articles

You should really read these.

Next step

Let’s talk