Explaining Machine Learned Relational Concepts in Visual Domains : Effects of Perceived Accuracy on Joint Performance and Trust

Faculty/Professorship: Cognitive Systems  
Author(s): Thaler, Anna Magdalena ; Schmid, Ute  
Title of the Journal: Proceedings of the Annual Meeting of the Cognitive Science Society
ISSN: 1069-7977
Corporate Body: Cognitive Science Society
Conference: Annual Meeting of the Cognitive Science Society, Vienna, Austria
Year of publication: 2021
Volume: 43
Pages: 1705-1711
Language(s): English
URL: https://escholarship.org/uc/item/8wr7s491
Most machine learning based decision support systems are black box models that are not interpretable for humans. However, the demand for explainable models to create comprehensible and trustworthy systems is growing, particularly in complex domains involving risky decisions. In many domains, decision making is based on visual information. We argue that nevertheless, explanations need to be verbal to communicate the relevance of specific feature values and critical relations for a classification decision. To address that claim, we introduce a fictitious visual domain from archeology where aerial views of ancient grave sites must be classified. Trustworthiness among other factors relies on the perceived or assumed correctness of a system's decisions. Models learned by induction of data, in general, cannot have perfect predictive accuracy and one can assume that unexplained erroneous system decisions might reduce trust. In a 2×2 factorial online experiment with 190 participants, we investigated the effect of verbal explanations and information about system errors. Our results show that explanations increase comprehension of the factors on which classification of grave sites is based and that explanations increase the joint performance of human and system for new decision tasks. Furthermore, explanations result in more confidence in decision making and higher trust in the system.
GND Keywords: Kausale Erklärung; Prognosefehler; Vertrauen; Maschinelles Lernen
Keywords: Explainability, Verbal Explanations, Prediction Errors, Trust, Relational Learning
DDC Classification: 004 Computer science  
RVK Classification: ST 300   
Peer Reviewed: Ja
International Distribution: Ja
Type: Conferenceobject
URI: https://fis.uni-bamberg.de/handle/uniba/53481
Release Date: 4. April 2022