The false hope of current approaches to explainable artificial intelligence in health care

Lancet Digital Health
Nov 2021 Volume 3 Number 11 e684-e750
https://www.thelancet.com/journals/landig/issue/current

 

Viewpoint
The false hope of current approaches to explainable artificial intelligence in health care
Marzyeh Ghassemi, Luke Oakden-Rayner, Andrew L Beam
Summary
The black-box nature of current artificial intelligence (AI) has caused some to question whether AI must be explainable to be used in high-stakes scenarios such as medicine. It has been argued that explainable AI will engender trust with the health-care workforce, provide transparency into the AI decision making process, and potentially mitigate various kinds of bias. In this Viewpoint, we argue that this argument represents a false hope for explainable AI and that current explainability methods are unlikely to achieve these goals for patient-level decision support. We provide an overview of current explainability techniques and highlight how various failure cases can cause problems for decision making for individual patients. In the absence of suitable explainability methods, we advocate for rigorous internal and external validation of AI models as a more direct means of achieving the goals often associated with explainability, and we caution against having explainability be a requirement for clinically deployed models.