With the broader and highly successful usage of machine learning (ML) in industry and the sciences, there has been a growing demand for explainable artificial intelligence (XAI). Interpretability and ...explanation methods for gaining a better understanding of the problem-solving abilities and strategies of nonlinear ML, in particular, deep neural networks, are, therefore, receiving increased attention. In this work, we aim to: 1) provide a timely overview of this active emerging field, with a focus on " post hoc " explanations, and explain its theoretical foundations; 2) put interpretability algorithms to a test both from a theory and comparative evaluation perspective using extensive simulations; 3) outline best practice aspects, i.e., how to best include interpretation methods into the standard usage of ML; and 4) demonstrate successful usage of XAI in a representative selection of application scenarios. Finally, we discuss challenges and possible future directions of this exciting foundational field of ML.
Current developments in Artificial Intelligence (AI) led to a resurgence of Explainable AI (XAI). New methods are being researched to obtain information from AI systems in order to generate ...explanations for their output. However, there is an overall lack of valid and reliable evaluations of the effects on users' experience of, and behavior in response to explanations. New XAI methods are often based on an intuitive notion what an effective explanation should be. Rule- and example-based contrastive explanations are two exemplary explanation styles. In this study we evaluate the effects of these two explanation styles on system understanding, persuasive power and task performance in the context of decision support in diabetes self-management. Furthermore, we provide three sets of recommendations based on our experience designing this evaluation to help improve future evaluations. Our results show that rule-based explanations have a small positive effect on system understanding, whereas both rule- and example-based explanations seem to persuade users in following the advice even when incorrect. Neither explanation improves task performance compared to no explanation. This can be explained by the fact that both explanation styles only provide details relevant for a single decision, not the underlying rational or causality. These results show the importance of user evaluations in assessing the current assumptions and intuitions on effective explanations.
Nowadays, Industry 4.0 can be considered a reality, a paradigm integrating modern technologies and innovations. Artificial intelligence (AI) can be considered the leading component of the industrial ...transformation enabling intelligent machines to execute tasks autonomously such as self-monitoring, interpretation, diagnosis, and analysis. AI-based methodologies (especially machine learning and deep learning support manufacturers and industries in predicting their maintenance needs and reducing downtime. Explainable artificial intelligence (XAI) studies and designs approaches, algorithms and tools producing human-understandable explanations of AI-based systems information and decisions. This article presents a comprehensive survey of AI and XAI-based methods adopted in the Industry 4.0 scenario. First, we briefly discuss different technologies enabling Industry 4.0. Then, we present an in-depth investigation of the main methods used in the literature: we also provide the details of what, how, why, and where these methods have been applied for Industry 4.0. Furthermore, we illustrate the opportunities and challenges that elicit future research directions toward responsible or human-centric AI and XAI systems, essential for adopting high-stakes industry applications.
•Provide insights into AI-Human communication.•Define levels of explanation with identified techniques that align with AI cognitive processes.•Discuss insights into Broad eXplainable Artificial ...Intelligence (Broad-XAI).•Align AI explanation to human communication.
Over the last few years there has been rapid research growth into eXplainable Artificial Intelligence (XAI) and the closely aligned Interpretable Machine Learning (IML). Drivers for this growth include recent legislative changes and increased investments by industry and governments, along with increased concern from the general public. People are affected by autonomous decisions every day and the public need to understand the decision-making process to accept the outcomes. However, the vast majority of the applications of XAI/IML are focused on providing low-level ‘narrow’ explanations of how an individual decision was reached based on a particular datum. While important, these explanations rarely provide insights into an agent's: beliefs and motivations; hypotheses of other (human, animal or AI) agents' intentions; interpretation of external cultural expectations; or, processes used to generate its own explanation. Yet all of these factors, we propose, are essential to providing the explanatory depth that people require to accept and trust the AI's decision-making. This paper aims to define levels of explanation and describe how they can be integrated to create a human-aligned conversational explanation system. In so doing, this paper will survey current approaches and discuss the integration of different technologies to achieve these levels with Broad eXplainable Artificial Intelligence (Broad-XAI), and thereby move towards high-level ‘strong’ explanations.
The evaluation of the fidelity of eXplainable Artificial Intelligence (XAI) methods to their underlying models is a challenging task, primarily due to the absence of a ground truth for explanations. ...However, assessing fidelity is a necessary step for ensuring a correct XAI methodology. In this study, we conduct a fair and objective comparison of the current state-of-the-art XAI methods by introducing three novel image datasets with reliable ground truth for explanations. The primary objective of this comparison is to identify methods with low fidelity and eliminate them from further research, thereby promoting the development of more trustworthy and effective XAI techniques. Our results demonstrate that XAI methods based on the direct gradient calculation and the backpropagation of output information to input yield higher accuracy and reliability compared to methods relying on perturbation based or Class Activation Maps (CAM). However, these methods tends to generate more noisy saliency maps. These findings have significant implications for the advancement of XAI methods, enabling the elimination of erroneous explanations and fostering the development of more robust and reliable XAI.
There has been a recent resurgence in the area of explainable artificial intelligence as researchers and practitioners seek to provide more transparency to their algorithms. Much of this research is ...focused on explicitly explaining decisions or actions to a human observer, and it should not be controversial to say that looking at how humans explain to each other can serve as a useful starting point for explanation in artificial intelligence. However, it is fair to say that most work in explainable artificial intelligence uses only the researchers' intuition of what constitutes a ‘good’ explanation. There exist vast and valuable bodies of research in philosophy, psychology, and cognitive science of how people define, generate, select, evaluate, and present explanations, which argues that people employ certain cognitive biases and social expectations to the explanation process. This paper argues that the field of explainable artificial intelligence can build on this existing research, and reviews relevant papers from philosophy, cognitive psychology/science, and social psychology, which study these topics. It draws out some important findings, and discusses ways that these can be infused with work on explainable artificial intelligence.
While artificial intelligence (AI), and by extension machine learning (ML), continues to be adopted in parallel engineering disciplines, the integration of AI/ML into the structural engineering ...domain remains minutus. This resistance towards AI and ML primarily stems from two folds: 1) the fact that coding/programming is not a frequent element in structural engineering curricula, and 2) these methods are displayed as blackboxes; the opposite of that often favored by structural engineering education and industry (i.e., testing, empirical analysis, numerical simulation, etc.). Naturally, structural engineers are reluctant to leverage AI/ML during their tenure as such technology is viewed as opaque. In the rare instances of engineers adopting AI/ML, a clear emphasis is displayed towards chasing goodness metrics to imply “viable” inference. However, and just like the notion of correlation does not infer causation, forced goodness is prone to indicate a false sense of inference. To overcome this challenge, this paper advocates for a modern form of AI, one that is humanly explainable; thereby eXplainable Artificial Intelligence (XAI) and interpretable machine learning (IML). Thus, this work dives into the inner workings of a typical analysis to demystify how AI/ML model predictions can be evaluated and interpreted through a collection of agnostic methods (e.g., feature importance, partial dependence plots, feature interactions, SHAP (SHapley Additive exPlanations), and surrogates) via a thorough examination of a case study carried out on a comprehensive database compiled on reinforced concrete (RC) beams strengthened with fiber-reinforced polymer (FRP) composite laminates. In this case study, three algorithms, namely: Extreme Gradient Boosted Trees (ExGBT), Light gradient boosted trees (LGBT), and Keras Deep Neural Networks (KDNN), are applied to predict the maximum moment capacity of FRP-strengthened beams and the propensity of the FRP system to fail under various mechanisms. Finally, a philosophical engineering perspective into future research directions pertaining to this domain is presented and articulated.
•Explainable AI and interpretable ML are discussed from a structural engineering perspective.•Three algorithms are examined.•Five explainability methods are explored in depth.•Potential implications of XAI and IML in engineering contexts are discussed.
Our research is a step toward ascertaining the need for personalization in XAI, and we do so in the context of investigating the value of explanations of AI-driven hints and feedback in Intelligent ...Tutoring Systems (ITS). We added an explanation functionality to the Adaptive CSP (ACSP) applet, an interactive simulation that helps students learn an algorithm for constraint satisfaction problems by providing AI-driven hints adapted to their predicted level of learning. We present the design of the explanation functionality and the results of a controlled study to evaluate its impact on students' learning and perception of the ACPS hints. The study includes an analysis of how these outcomes are modulated by several user characteristics such as personality traits and cognitive abilities, to asses if explanations should be personalized to these characteristics. Our results indicate that providing explanations increase students' trust in the ACPS hints, perceived usefulness of the hints, and intention to use them again. In addition, we show that students' access of the ACSP explanation and learning gains are modulated by three user characteristics, Need for Cognition, Contentiousness and Reading Proficiency, providing insights on how to personalize the ACSP explanations to these traits, as well as initial evidence on the potential value of personalized Explainable AI (XAI) for ITS.
Display omitted
•Comprehensive survey to provide guidance and formalize the field of explainable AI.•Assessment of quantitative evaluation metrics for explainability.•Step-by-step guidance to choose ...between classes of explainable AI methods.•Explainable AI can contribute to trustworthy AI, but other measures might be needed.
Artificial intelligence (AI) has huge potential to improve the health and well-being of people, but adoption in clinical practice is still limited. Lack of transparency is identified as one of the main barriers to implementation, as clinicians should be confident the AI system can be trusted. Explainable AI has the potential to overcome this issue and can be a step towards trustworthy AI. In this paper we review the recent literature to provide guidance to researchers and practitioners on the design of explainable AI systems for the health-care domain and contribute to formalization of the field of explainable AI. We argue the reason to demand explainability determines what should be explained as this determines the relative importance of the properties of explainability (i.e. interpretability and fidelity). Based on this, we propose a framework to guide the choice between classes of explainable AI methods (explainable modelling versus post-hoc explanation; model-based, attribution-based, or example-based explanations; global and local explanations). Furthermore, we find that quantitative evaluation metrics, which are important for objective standardized evaluation, are still lacking for some properties (e.g. clarity) and types of explanations (e.g. example-based methods). We conclude that explainable modelling can contribute to trustworthy AI, but the benefits of explainability still need to be proven in practice and complementary measures might be needed to create trustworthy AI in health care (e.g. reporting data quality, performing extensive (external) validation, and regulation).
Explainable Artificial Intelligence (XAI) has experienced a significant growth over the last few years. This is due to the widespread application of machine learning, particularly deep learning, that ...has led to the development of highly accurate models that lack explainability and interpretability. A plethora of methods to tackle this problem have been proposed, developed and tested, coupled with several studies attempting to define the concept of explainability and its evaluation. This systematic review contributes to the body of knowledge by clustering all the scientific studies via a hierarchical system that classifies theories and notions related to the concept of explainability and the evaluation approaches for XAI methods. The structure of this hierarchy builds on top of an exhaustive analysis of existing taxonomies and peer-reviewed scientific material. Findings suggest that scholars have identified numerous notions and requirements that an explanation should meet in order to be easily understandable by end-users and to provide actionable information that can inform decision making. They have also suggested various approaches to assess to what degree machine-generated explanations meet these demands. Overall, these approaches can be clustered into human-centred evaluations and evaluations with more objective metrics. However, despite the vast body of knowledge developed around the concept of explainability, there is not a general consensus among scholars on how an explanation should be defined, and how its validity and reliability assessed. Eventually, this review concludes by critically discussing these gaps and limitations, and it defines future research directions with explainability as the starting component of any artificial intelligent system.
Display omitted
•Proposal of a hierarchical system to organise the notions of explainability.•Proposal of a hierarchical system to organise the evaluation approaches for XAI methods.•Summary of approaches for defining an explanation and its effectiveness.•Summary of scattered evaluation approaches for assessing XAI methods.•Proposal of a framework with explainability as core element.
PDF
