A 10-year-old CNN outperforms frontier LLMs for skin cancer diagnosis

Re:A 10-year-old CNN outperforms frontier LLMs for skin canc... by johnshahbazian on Sunday October 25, 2026 @06:48PM (#50236820)

People like me who are excited about Jev-style models aren't going to use them to diagnose cancer. We are going to use them to make fairly straightforward decisions very quickly (300 ms or less) and very cheaply ($0.042 per M input tokens - output tokens aren't billed). Even the lower end frontier models will take much longer and be much more expensive, so these decision models are perfect for certain things.

[ Parent | on HN ]

Re:A 10-year-old CNN outperforms frontier LLMs for skin canc... by antonvs on Sunday October 25, 2026 @08:12PM (#50237361)

But as the GP said, you’d likely be better off if you used an LLM to help you train a custom model to do what you’re trying to do.

If you really evaluate the best options, it’s very likely that a Jev-style decision model won’t give the best results.

[ Parent | on HN ]

Re:A 10-year-old CNN outperforms frontier LLMs for skin canc... by johnshahbazian on Sunday October 25, 2026 @10:19PM (#50238334)

If you know the exact domain you're working in, then a trained model could potentially do better, but its not a guarantee. For example, I fine-tuned Muse Glimmer on a specific domain and then tested it against Mercury Decide - which I've found does better than Jev - and they were even (and no fine-tuning was done on Mercury). But Mercury Decide was was faster and cheaper. The P95 for latency was about 5.6x lower. Decision models are very, very fast.

[ Parent | on HN ]