The advent of machine learning (ML) has led to the development of highly complex and opaque artificial intelligence (AI) systems. The opacity of these systems raises significant societal challenges concerning, e.g., fairness, trustworthiness, safety, responsibility, and autonomy. A widely proposed solution to these challenges is explainable AI (XAI). XAI is a research field aiming to explain the behavior of complex AI systems by mostly focusing on computational approaches. The explanations provided by XAI approaches are intended to induce understanding about opaque systems that allows various stakeholders to tackle the challenges described above.

While XAI is certainly helpful in at least some contexts, we believe that it is also crucially limited and thus not (yet) up to addressing many of the challenges it is intended for. In this paper, we highlight four reasons for XAI’s shortfall: (1) XAI fails to deliver faithful explanations, (2) even faithful explanations may fail to induce the right kind of understanding, (3) XAI may not provide the right kind of information, and (4) addressing some societal challenges requires more than explainability. We shall consider each of these points in turn before suggesting solutions.