In 2018, the researcher Pranav Rajpurkar was working on an algorithm that could find blood clots in patients’ legs from ultrasound images. It spotted them very well, but when he went looking for what the algorithm had picked up on in the images to make its predictions, he saw it had been cheating: it was looking at the metadata in the top right corner of every ultrasound.
This got him thinking about how to evaluate whether AI models are actually pointing at the right spots on medical images. He designed what he calls a “pointing game” between radiologists and AI algorithms. “If you ask a person and an algorithm to point at a spot, are they near each other?” said Rajpurkar, an assistant professor of biomedical informatics at Harvard Medical School.
While many models perform just as well as doctors in discerning disease, they can’t fully explain their logic in ways that clinicians can understand. One set of explainability methods, called saliency methods, can offer a window into how these models make their predictions by highlighting the areas of a medical image that most affected an algorithm’s prediction. The result takes the form of a heatmap overlaid onto the image, as though the algorithm has drawn a border around a concentrated area of disease such as a lung nodule on a chest X-ray.
This article is exclusive to STAT+ subscribers
Unlock this article — and get additional analysis of the technologies disrupting health care — by subscribing to STAT+.
Already have an account? Log in
Already have an account? Log in
To submit a correction request, please visit our Contact Us page.
STAT encourages you to share your voice. We welcome your commentary, criticism, and expertise on our subscriber-only platform, STAT+ Connect