AI‑based decision support systems and society: An opening statement

Diana Schneider; Karsten Weber

doi:10.14512/tatup.33.1.9

Authors

Diana Schneider Fraunhofer Institute for Systems and Innovation Research ISI, Karlsruhe (DE) https://orcid.org/0000-0002-7131-5005 diana.schneider@isi.fraunhofer.de
Karsten Weber Institute for Social Research and Technology Assessment, OTH Regensburg, Regensburg (DE) https://orcid.org/0000-0001-8875-2386 karsten.weber@oth-regensburg.de

DOI:

https://doi.org/10.14512/tatup.33.1.9

Keywords:

artificial intelligence (AI), decision support, socio-technical systems, regulation, social impacts

Abstract

Although artificial intelligence (AI) and automated decision-making systems have been around for some time, they have only recently gained in importance as they are now actually being used and are no longer just the subject of research. AI to support decision-making is thus affecting ever larger parts of society, creating technical, but above all ethical, legal, and societal challenges, as decisions can now be made by machines that were previously the responsibility of humans. This introduction provides an overview of attempts to regulate AI and addresses key challenges that arise when integrating AI systems into human decision-making. The Special topic brings together research articles that present societal challenges, ethical issues, stakeholders, and possible futures of AI use for decision support in healthcare, the legal system, and border control.

References

Braun, Matthias; Hummel, Patrik; Beck, Susanne; Dabrock, Peter (2020): Primer on an ethics of AI‑based decision support systems in the clinic. In: Journal of Medical Ethics 47 (12), p. e3. https://doi.org/10.1136/medethics-2019-105860 DOI: https://doi.org/10.1136/medethics-2019-105860

Butcher, James; Beridze, Irakli (2019): What is the state of artificial intelligence governance globally? In: The RUSI Journal 164 (5–6), pp. 88–96. https://doi.org/10.1080/03071847.2019.1694260 DOI: https://doi.org/10.1080/03071847.2019.1694260

Čartolovni, Anto; Tomičić, Ana; Lazić Mosler, Elvira (2022): Ethical, legal, and social considerations of AI‑based medical decision-support tools. A scoping review. In: International Journal of Medical Informatics 161, p. 104738. https://doi.org/10.1016/j.ijmedinf.2022.104738 DOI: https://doi.org/10.1016/j.ijmedinf.2022.104738

Devlieghere, Jochen; Gillingham, Philip; Roose, Rudi (2022): Dataism versus relationshipism. A social work perspective. In: Nordic Social Work Research 12 (3), pp. 328–338. https://doi.org/10.1080/2156857X.2022.2052942 DOI: https://doi.org/10.1080/2156857X.2022.2052942

European Union (2023): Briefing – EU legislation in process. Artificial intelligence act. Available online at https://www.europarl.europa.eu/RegData/etudes/BRIE/2021/698792/EPRS_BRI(2021)698792_EN.pdf, last accessed on 18. 01. 2024.

European Commission (2021): Proposal for a regulation of the European Parliament and of the Council laying down harmonised rules on artificial intelligence (Artificial Intelligence Act) and amending certain union legislative acts. Brussels: European Commission. Available online at https://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:52021PC0206, last accessed on 18. 01. 2024.

Gillingham, Philip (2015): Electronic information systems in human service organisations. The what, who, why and how of information. In: British Journal of Social Work 45 (5), pp. 1598–1613. https://doi.org/10.1093/bjsw/bcu030 DOI: https://doi.org/10.1093/bjsw/bcu030

Gillingham, Philip (2020): The development of algorithmically based decision-making systems in children’s protective services. Is administrative data good enough? In: The British Journal of Social Work 50 (2), pp. 565–580. https://doi.org/10.1093/bjsw/bcz157 DOI: https://doi.org/10.1093/bjsw/bcz157

Heyen, Nils; Salloch, Sabine (2021): The ethics of machine learning-based clinical decision support. An analysis through the lens of professionalisation theory. In: BMC Medical Ethics 22 (112), 9 pp. https://doi.org/10.1186/s12910-021-00679-3 DOI: https://doi.org/10.1186/s12910-021-00679-3

Kempt, Hendrik; Nagel, Saskia (2022): Responsibility, second opinions and peer-disagreement. Ethical and epistemological challenges of using AI in clinical diagnostic contexts. In: Journal of Medical Ethics 48 (4), pp. 222–229. https://doi.org/10.1136/medethics-2021-107440 DOI: https://doi.org/10.1136/medethics-2021-107440

Pedersen, John (2019): The digital welfare state. Dataism versus relationshipism. In: John Pedersen and Adrian Wilkinson (eds.): Big Data. Promise, application and pitfalls. Cheltenham: Edward Elgar Publishing, pp. 301–324. https://doi.org/10.4337/9781788112352.00019 DOI: https://doi.org/10.4337/9781788112352.00019

Schiff, Daniel; Laas, Kelly; Biddle, Justin; Borenstein, Jason (2022): Global AI ethics documents. What they reveal about motivations, practices, and policies. In: Kelly Laas, Michael Davis and Elizsabeth Hildt (eds.): Codes of ethics and ethical guidelines. Cham: Springer International, pp. 121–143. https://doi.org/10.1007/978-3-030-86201-5_7 DOI: https://doi.org/10.1007/978-3-030-86201-5_7

Schmitt, Lewin (2022): Mapping global AI governance. A nascent regime in a fragmented landscape. In: AI and Ethics 2 (2), pp. 303–314. https://doi.org/10.1007/s43681-021-00083-y DOI: https://doi.org/10.1007/s43681-021-00083-y

Schneider, Diana (2022): Ensuring privacy and confidentiality in social work through intentional omissions of information in client information systems. A qualitative study of available and non-available data. In: Digital Society 1 (26), 21 pp. https://doi.org/10.1007/s44206-022-00029-9 DOI: https://doi.org/10.1007/s44206-022-00029-9

Schneider, Diana; Maier, Angelika; Cimiano, Philipp; Seelmeyer, Udo (2022a): Exploring opportunities and risks in decision support technologies for social workers. An empirical study in the field of disabled people’s services. In: Sozialer Fortschritt 71 (6–7), pp. 489–511. https://doi.org/10.3790/sfo.71.6-7.489 DOI: https://doi.org/10.3790/sfo.71.6-7.489

Schneider, Diana; Sonar, Arne; Weber, Karsten (2022b): Zwischen Automatisierung und ethischem Anspruch. Disruptive Effekte des KI‑Einsatzes in und auf Professionen der Gesundheitsversorgung. In: Mario Pfannstiel (ed.): Künstliche Intelligenz im Gesundheitswesen. Entwicklungen, Beispiele und Perspektiven. Wiesbaden: Springer, pp. 325–348. https://doi.org/10.1007/978-3-658-33597-7_14 DOI: https://doi.org/10.1007/978-3-658-33597-7_14

Sujan, Mark et al. (2019): Human factors challenges for the safe use of artificial intelligence in patient care. In: BMJ Health & Care Informatics 26 (1), p. e100081. https://doi.org/10.1136/bmjhci-2019-100081 DOI: https://doi.org/10.1136/bmjhci-2019-100081

Topol, Eric (2019): Deep medicine. How artificial intelligence can make healthcare human again. New York, NY: Basic Books.

Tucker, Catherine (2023): Algorithmic exclusion. The fragility of algorithms to sparse and missing data. Working Paper. In: Brookings Institution, 02. 02. 2023. Available online at https://www.brookings.edu/research/algorithmic-exclusion-the-fragility-of-algorithms-to-sparse-and-missing-data/, last accessed on 17. 01. 2024.

Ulnicane, Inga; Knight, William; Leach, Tonii; Stahl, Bernd; Wanjiku, Winter-Gladys (2021): Framing governance for a contested emerging technology. Insights from AI policy. In: Policy and Society 40 (2), pp. 158–177. https://doi.org/10.1080/14494035.2020.1855800 DOI: https://doi.org/10.1080/14494035.2020.1855800

Waag, Philipp; Rink, Konstantin (2023): Digitalisierung als Irritation. Von ideologischen zu reflexionstheoretischen Selbstbeschreibungen der Sozialen Arbeit im Zuge ihrer Auseinandersetzung mit digitalen Technologien. In: Neue Praxis 23 (4), pp. 292–306.

Webb, Stephen (2003): Technologies of care. In: Elizabeth Harlow and Stephen Webb (eds.): Information and communication technologies in the welfare services. London: Jessica Kingsley Publishers, pp. 223–238.