Situativität, Funktionalität und Vertrauen: Ergebnisse einer szenariobasierten Interviewstudie zur Erklärbarkeit von KI in der Medizin

Manuela Marquardt; Philipp Graf; Eva Jansen; Stefan Hillmann; Jan-Niklas Voigt-Antons

doi:10.14512/tatup.33.1.41

Authors

Manuela Marquardt Institute of Medical Sociology and Rehabilitation Science, Charité – University medecine Berlin, Berlin (DE) https://orcid.org/0000-0003-4355-1243 manuela.marquardt@charite.de
Philipp Graf Department of Applied Social Sciences, University of Applied Sciences Munich, Munich (DE) https://orcid.org/0000-0003-4556-956X philipp.graf@hm.edu
Eva Jansen Institute of Medical Sociology and Rehabilitation Science, Charité – University medecine Berlin, Berlin (DE) https://orcid.org/0000-0001-9890-6641 eva.jansen@charite.de
Stefan Hillmann Quality & Usability Lab, Technical University Berlin, Berlin (DE) https://orcid.org/0000-0002-0795-9834 stefan.hillmann@tu-berlin.de
Jan-Niklas Voigt-Antons Immersive Reality Lab, Hochschule Hamm-Lippstadt, Hamm-Lippstadt (DE) https://orcid.org/0000-0002-2786-9262 jan-niklas.voigt-antons@hshl.de

DOI:

https://doi.org/10.14512/tatup.33.1.41

Keywords:

explainability, XAI, AI in healthcare, embodied AI, voice dialog system

Abstract

A central requirement for the use of artificial intelligence (AI) in medicine is its explainability, i. e., the provision of addressee-oriented information about its functioning. This leads to the question of how socially adequate explainability can be designed. To identify evaluation factors, we interviewed healthcare stakeholders about two scenarios: diagnostics and documentation. The scenarios vary the influence that an AI system has on decision-making through the interaction design and the amount of data processed. We present key evaluation factors for explainability at the interactional and procedural levels. Explainability must not interfere situationally in the doctor-patient conversation and question the professional role. At the same time, explainability functionally legitimizes an AI system as a second opinion and is central to building trust. A virtual embodiment of the AI system is advantageous for language-based explanations

References

Aminololama-Shakeri, Shadi; López, Javier (2019): The doctor-patient relationship with artificial intelligence. In: American Journal of Roentgenology 212 (2), S. 308–310. https://doi.org/10.2214/AJR.18.20509 DOI: https://doi.org/10.2214/AJR.18.20509

Barredo-Arrieta, Alejandro et al. (2020): Explainable artificial intelligence (XAI). Concepts, taxonomies, opportunities and challenges toward responsible AI. In: Information Fusion 58, S. 82–115. https://doi.org/10.1016/j.inffus.2019.12.012 DOI: https://doi.org/10.1016/j.inffus.2019.12.012

Barter, Christine; Renold, Emma (1999): The use of vignettes in qualitative research. In: Social Research Update 25. Online verfügbar unter https://sru.soc.surrey.ac.uk/SRU25.html, zuletzt geprüft am 11. 01. 2024.

Becker, Aliza (2019): Artificial intelligence in medicine. What is it doing for us today? In: Health Policy and Technology 8 (2), S. 198–205. https://doi.org/10.1016/j.hlpt.2019.03.004 DOI: https://doi.org/10.1016/j.hlpt.2019.03.004

Bossen, Claus; Pine, Kathleen (2023): Batman and Robin in healthcare knowledge work. Human-AI collaboration by clinical documentation integrity specialists. In: ACM Transactions on Computer-Human Interaction 30 (2), S. 1–29. https://doi.org/10.1145/3569892 DOI: https://doi.org/10.1145/3569892

Čartolovni, Anto; Tomičić, Ana; Lazić Mosler, Elvira (2022): Ethical, legal, and social considerations of AI‑based medical decision-support tools. A scoping review. In: International Journal of Medical Informatics 161, S. 104738. https://doi.org/10.1016/j.ijmedinf.2022.104738 DOI: https://doi.org/10.1016/j.ijmedinf.2022.104738

Dickel, Sascha (2021): Wenn die Technik sprechen lernt. Künstliche Kommunikation als kulturelle Herausforderung mediatisierter Gesellschaften. In: TATuP – Zeitschrift für Technikfolgenabschätzung in Theorie und Praxis 30 (3), S. 23–29. https://doi.org/10.14512/tatup.30.3.23 DOI: https://doi.org/10.14512/tatup.30.3.23

Feldhus, Nils; Wang, Qianli; Anikina, Tatiana; Chopra, Sahil; Oguz, Cennet; Möller, Sebastian (2023): InterroLang. Exploring NLP models and datasets through dialogue-based explanations. In: Houda Bouamor, Juan Pino und Kalika Bali (Hg.): Findings of the Association for Computational Linguistics: EMNLP 2023. Singapur: Association for Computational Linguistics, S. 5399–5421. https://doi.org/10.18653/v1/2023.findings-emnlp.359 DOI: https://doi.org/10.18653/v1/2023.findings-emnlp.359

Gupta, Akshit; Basu, Debadeep; Ghantasala, Ramya; Qiu, Sihang; Gadiraju, Ujwal (2022): To trust or not to trust. How a conversational interface affects trust in a decision support system. In: Frédérique Laforest et al. (Hg.): Proceedings of the ACM Web Conference 2022. New York, NY: Association for Computing Machinery, S. 3531–3540. https://doi.org/10.1145/3485447.3512248 DOI: https://doi.org/10.1145/3485447.3512248

HEG-KI – Hochrangige Expertengruppe für KI (2019): Ethik-Leitlinien für eine vertrauenswürdige KI. Brüssel: Europäische Kommission. https://doi.org/10.2759/22710

Heil, Reinhard et al. (2021): Artificial intelligence in human genomics and biomedicine. Dynamics, potentials and challenges. In: TATuP – Zeitschrift für Technikfolgenabschätzung in Theorie und Praxis 30 (3), S. 30–36. https://doi.org/10.14512/tatup.30.3.30 DOI: https://doi.org/10.14512/tatup.30.3.30

Hillmann, Stefan; Möller, Sebastian; Michael, Thilo (2021): Towards speech-based interactive post hoc explanations in explainable AI. In: Astrid Carolus, Carolin Wienrich und Ingo Siegert (Hg.): Proceedings of the 1st AI‑Debate Workshop. Magdeburg: Universität Magdeburg, pp. 13–15. http://dx.doi.org/10.25673/38473

Hughes, Rhidian; Huby, Meg (2004): The construction and interpretation of vignettes in social research. In: Social Work and Social Sciences Review 11 (1), S. 36–51. https://doi.org/10.1921/swssr.v11i1.428 DOI: https://doi.org/10.1921/17466105.11.1.36

Jenkins, Nicholas; Bloor, Michael; Fischer, Jan; Berney, Lee; Neale, Joanne (2010): Putting it in context. The use of vignettes in qualitative interviewing. In: Qualitative Research 10 (2), S. 175–198. https://doi.org/10.1177/1468794109356737 DOI: https://doi.org/10.1177/1468794109356737

Kinar, Yaron et al. (2017): Performance analysis of a machine learning flagging system used to identify a group of individuals at a high risk for colorectal cancer. In: PLoS One 12 (2), S. e0171759. https://doi.org/10.1371/journal.pone.0171759 DOI: https://doi.org/10.1371/journal.pone.0171759

Kloker, Anika; Fleiß, Jürgen; Koeth, Christoph; Kloiber, Thomas; Ratheiser, Patrick; Thalmann, Stefan (2022): Caution or trust in AI? How to design XAI in sensitive use cases? In: AMCIS 2022 Proceedings 16.

Kosow, Hannah; Gaßner, Robert (2008): Methoden der Zukunfts- und Szenarioanalyse. Überblick, Bewertung und Auswahlkriterien. Berlin: Institut für Zukunftsstudien und Technologiebewertung. Online verfügbar unter https://www.researchgate.net/publication/262198781_Methoden_der_Zukunfts-und_Szenarioanalyse_Uberblick_Bewertung_und_Auswahlkriterien, zuletzt geprüft am 15. 01. 2024.

Mahmood, Amama; Huang, Chien-Ming (2022): Effects of rhetorical strategies and skin tones on agent persuasiveness in assisted decision-making. In: Carlos Martinho, João Dias, Joana Campos und Dirk Heylen (Hg.): Proceedings of the 22nd ACM International Conference on Intelligent Virtual Agents. New York, NY: Association for Computing Machinery, S. 1–8. https://doi.org/10.1145/3514197.3549628 DOI: https://doi.org/10.1145/3514197.3549628

Markus, Aniek; Kors, Jan; Rijnbeek, Peter (2021): The role of explainability in creating trustworthy artificial intelligence for health care. A comprehensive survey of the terminology, design choices, and evaluation strategies. In: Journal of Biomedical Informatics 113, S. 103655. https://doi.org/10.1016/j.jbi.2020.103655 DOI: https://doi.org/10.1016/j.jbi.2020.103655

Meuser, Michael; Sackmann, Reinhold (1992): Zur Einführung. Deutungsmusteransatz und empirische Wissenssoziologie. In: Michael Meuser und Reinhold Sackmann (Hg.): Analyse sozialer Deutungsmuster. Beiträge zur empirischen Wissenssoziologie. Pfaffenweiler: Centaurus-Verlagsgesellschaft, S. 9–37.

Miller, Tim (2019): Explanation in artificial intelligence. Insights from the social sciences. In: Artificial Intelligence 267, S. 1–38. https://doi.org/10.1016/j.artint.2018.07.007 DOI: https://doi.org/10.1016/j.artint.2018.07.007

Panch, Trishan; Mattie, Heather; Atun, Rifat (2019): Artificial intelligence and algorithmic bias. Implications for health systems. In: Journal of Global Health 9 (2), S. 010318. https://doi.org/10.7189/jogh.09.020318 DOI: https://doi.org/10.7189/jogh.09.020318

Pentzold, Christian; Bischof, Andreas; Heise, Nele (2018): Einleitung. Theoriegenerierendes empirisches Forschen in medienbezogenen Lebenswelten. In: Christian Pentzold, Andreas Bischof und Nele Heise (Hg.): Praxis grounded theory. Theoriegenerierendes empirisches Forschen in medienbezogenen Lebenswelten. Wiesbaden: Springer, S. 1–24. https://doi.org/10.1007/978-3-658-15999-3_1 DOI: https://doi.org/10.1007/978-3-658-15999-3_1

Powell, John (2019): Trust me, I’m a chatbot. How artificial intelligence in health care fails the Turing Test. In: Journal of Medical Internet Research 21 (10), S. e16222. https://doi.org/10.2196/16222 DOI: https://doi.org/10.2196/16222

Samhammer, David et al. (2023): Klinische Entscheidungsfindung mit Künstlicher Intelligenz. Ein interdisziplinärer Governance-Ansatz. Heidelberg: Springer. https://doi.org/10.1007/978-3-662-67008-8 DOI: https://doi.org/10.1007/978-3-662-67008-8

Scharowski, Nicolas; Perrig, Sebastian; Svab, Melanie; Opwis, Klaus; Brühlmann, Florian (2023): Exploring the effects of human-centered AI explanations on trust and reliance. In: Frontiers in Computer Science 5, S. 1–15. https://doi.org/10.3389/fcomp.2023.1151150 DOI: https://doi.org/10.3389/fcomp.2023.1151150